Run set of jobs based on input file using loop

[opdas]

Hi,
I have a scenario where Job1, Job2 and Job3 run in a sequence one after another. Input for Job 1 is a sequential file and output for job3 is the same sequential file.
I want to put these 3 jobs in a sequence and run in a loop untill the input sequential file is "empty". Can this be achieved using Datastage sequence?
I have been searching here but couldnt find a similar query.
Really appreciate if anybody could guide me here.

Regards,
OP

[ray.wurlod]

What empties the file?

[opdas]

The Job3 overwrites the file.

[ray.wurlod]

To use a job sequence create a loop that executes a large number of times and test the size of the file within the loop exiting from the loop if the file is empty. Or write a job control routine that uses an uncounted loop that exits on the same condition.

[opdas]

Thanks for the response.
how do I exit from the loop? using terminator activity? but this would abort the sequence. Can I exit grace fully without aborting the sequence.

[Ragunathan Gunasekaran]

Why dont use the execute command activity and do some manipulation.Assign the value of the execute command activity to the TO Value of the loop counters .When the loop encounters a logical false condition it automatically exits out without issuing abort

[chulett]

how do I exit from the loop? using terminator activity? but this would abort the sequence. Can I exit grace fully without aborting the sequence.

Just do a conditional branch to something 'outside' the loop, past the End Loop stage, to get out early and gracefully.

[OddJob]

It is possible for a sequence to call itself, it's called recursion.

Create a sequence that...

a). is multi instance, and expects a numeric instance id
b). use a 'User Variable' stage to evaluate DSJobInvocationId+1, which will be used as the invocation id for the next call to the same sequence
c). sequence the jobs required to process the file
d). test to see if the output file is empty e.g. execute command stage using 'test -s <file name>'
e). on the trigger from the exec command stage, if the return value is zero (file exists and is greater than 0 bytes), call the same sequence again, this time using the job invocation derived in user variable stage i.e. input job invocation + 1.

Run the sequence with a starting invocation id of 0 (or 1 if you prefer).

You should see director automatically running as many invocations of your sequence as required to get the job done. You may want an upper limit on the number of invocations, otherwise this thing will run forever(!) if the file never gets empty, just add a 'nested condition' stage to check the job invocation id isn't higher than a parameterised value.

Hope this helps!