Sequence forced restart

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
sspreethi
Participant
Posts: 25
Joined: Mon Dec 01, 2003 2:27 am

Sequence forced restart

Post by sspreethi »

I have the below job sequence :
Execution action - Reset if required, then run

JobA
|
rtn1
|(returnvalue=0)
JobB
|
rtn2(returnvalue=0) --- Terminator
|(returnvalue!=0)
JobC

When the sequence fails (after I forcefully abort), I would want it to re-start from JobB. I have checkpoints ON. But since JobB status is 'Success' & only sequence status is forcefully set to 'Abort', it doesnt start from JobB, rather skips.

I have attached below the log of 2 runs.
1st run scenario - returnvalue=0
2nd run scenario - returnvalue!=0 (Inspite aborts, since sequence doesnt run JobB)

Summary of sequence run
11:46:58: Sequence started (checkpointing on)
11:46:58: JobA (JOB A) started
11:47:17: JobA (JOB A) finished, status=1 [Finished OK]
11:47:18: rtn1 (ROUTINE ) started
11:47:18: rtn1 finished, reply=0
11:47:18: JobB (JOB B) started
11:47:32: JobB (JOB B) finished, status=1 [Finished OK]
11:47:32: rtn2 (ROUTINE DSU.rGetLinkInfo) started
11:47:32: rtn2 finished, reply=0
11:47:33: Terminator Executed
11:47:33: Sequence terminated


Summary of sequence run
11:47:51: Sequence restarted after failure
11:47:51: JobA (JOB A) skipped
11:47:51: rtn1 (ROUTINE ) skipped
11:47:51: JobB (JOB B) skipped
11:47:51: rtn2 (ROUTINE ) skipped
11:47:51: Terminator Executed
11:47:51: Sequence terminated

Please let me know if I am doing anything wrong & also how to acheive this.

-Preethi
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

A restarting Sequence starts at the Aborted step. If JobB aborted, it would have restarted from there. Aborting the Sequence after it completes successfully buys you nothing.
-craig

"You can never have too many knives" -- Logan Nine Fingers
sspreethi
Participant
Posts: 25
Joined: Mon Dec 01, 2003 2:27 am

Post by sspreethi »

I agree to that Chulett.
JobA & B are only pre-checks before I proceed with Job C. Based on the tgt rowcount of Job B(for which I use routine), I decide if I shud stop or proceed to Job C.
gateleys
Premium Member
Premium Member
Posts: 992
Joined: Mon Aug 08, 2005 5:08 pm
Location: USA

Re: Sequence forced restart

Post by gateleys »

Why not change your design so as to Write some garbage to a file 'AbortFlagFile' when rtn2.$ReturnValue = 0 and terminate. So, your design is now as follows.
1. getFileSize is a Routine to get size of file for 'AbortFlagFile'.
2. clearFile is a Routine/Command stage to clear content of flagfile.
3. writeFlagFile is a Routine/Command or a job to write some junk into flagfile.
4. Ensure that getFileSize and clearFile are checked for 'Do Not checkpoint run'.

Code: Select all

                                                        /writeFlagFile----Terminate
              __clearFile\                   (RetVal=0)/
    (size>0) /            Sequencer(Any)---JobB----rtn2
getFileSize/                 /              (RetVal != 0)\
           \                /                             \JobC
  (size = 0)\---JobA---rtn1/

Let me know what you think. And yes, that design took some time to post.
gateleys
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Ah... the joys of ASCII Art! :lol:
-craig

"You can never have too many knives" -- Logan Nine Fingers
gateleys
Premium Member
Premium Member
Posts: 992
Joined: Mon Aug 08, 2005 5:08 pm
Location: USA

Post by gateleys »

Alternatively, you can use a Nested Condition activity with 2 triggers, one set to start jobA-->rtn1, and another set to a Sequencer (Any)-->jobB-->rtn2 and so on. Also, output from rtn1 will go to input of Sequencer. Define list param StartPoint with possible values 1 and 2. Set the first trigger of Nested Condition to StartPoint = 1 (default)and the second trigger to StartPoint = 2. During regular runs, the the trigger 1 will fire. In case you want to restart after an abort of jobs other than jobA or rtn1, you can execute the job sequence with StartPoint = 2.
Last edited by gateleys on Mon Nov 19, 2007 2:06 pm, edited 2 times in total.
gateleys
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

That will have to come later, I only have time for a quick wise-crack or two right now, can't really spare any serious think time. :(
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

If you always want JobB to run, you can check its "do not checkpoint" check box (and recompile the job sequence).
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
gateleys
Premium Member
Premium Member
Posts: 992
Joined: Mon Aug 08, 2005 5:08 pm
Location: USA

Post by gateleys »

ray.wurlod wrote:If you always want JobB to run, you can check its "do not checkpoint" check box (and recompile the job sequence).
The problem is if he recompiles the job sequence, JobA and rtn1 will run again.
gateleys
sspreethi
Participant
Posts: 25
Joined: Mon Dec 01, 2003 2:27 am

Post by sspreethi »

Thanks gateleys. I will try this now. Are there any built in routines for getFileSize & clearfile.
gateleys
Premium Member
Premium Member
Posts: 992
Joined: Mon Aug 08, 2005 5:08 pm
Location: USA

Post by gateleys »

sspreethi wrote:Are there any built in routines for getFileSize & clearfile.
No, you will have to code it.

1. To clear the file, you can write a simple job with the file in question as the target. Use a constraint of @FALSE in the output link of the transformer to write 'nothing' to the file.

2. Use the STATUS statement to read the file size from the dynamic array. Alternatively, you may count lines in the file with wc -l. If it is 0, then size = 0, else size > 0.
gateleys
sspreethi
Participant
Posts: 25
Joined: Mon Dec 01, 2003 2:27 am

Post by sspreethi »

Thanks a lot gateleys. Both ur approaches worked just like I wanted.
One quick question. Can you let me know if you meant any specific stage by this - 'Sequencer (any)'; to merge my 2 flows into 1. For time being I am having 2 separate flows(no good). I couldnt find a stage which accepted more than 1 input.

Appreciate all your help.

Thanks again.
sspreethi
Participant
Posts: 25
Joined: Mon Dec 01, 2003 2:27 am

Post by sspreethi »

Got it... :D
sequencer activity with ANY mode...
Post Reply