Job Sequence restartability

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
praburaj
Premium Member
Premium Member
Posts: 133
Joined: Thu Jun 30, 2005 1:26 am
Location: philippines

Job Sequence restartability

Post by praburaj »

Hi All,

I have one question. Suppose we are running our job with 10000 rows. While running the job, the job got aborted in middle. Now out of 10,000 records, 5000 records are loaded in target. For this kind of situation normally we will use sequencer. We will enable the check point options. Then sequencer will run from where job got aborted.

My Question is How sequencer know that the job should restart from 5001 record onwards? How it is Internally works?.

If anybody explain about this process really helpful for me.

I know this question very chilly question. But your Input is really valuable for me.

Any help is really appreciated.
prabakaran.v
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

It doesn't. If you want that, you have to code for it. Create a "recovery" job. Incidentally, it's a sequence, not a sequencer.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
BI-RMA
Premium Member
Premium Member
Posts: 463
Joined: Sun Nov 01, 2009 3:55 pm
Location: Hamburg

Re: Regarding Sequencer...

Post by BI-RMA »

Quite simply: it does not know. If the commit-level within the job is lower than 5000 these 5000 rows are in the table after the abort. A checkpointed restart of your sequence will just skip the jobs that finished successfully before the abort. If your job just loads data to a target-table without any validation you will end up either with duplicate natural-keys in your table or - if the target table has got a primary key that is not generated by your job - with a lot of duplicate key warnings in the job-logs.

You will have to cleanup before you restart or use some logic within your job to make sure data is not loaded twice (Lookup or Join against target-tables).
"It is not the lucky ones are grateful.
There are the grateful those are happy." Francis Bacon
Post Reply