SEQUENCER running multiple jobs

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
rkumar28
Participant
Posts: 43
Joined: Tue Mar 30, 2004 9:39 am

SEQUENCER running multiple jobs

Post by rkumar28 »

Hi,

I have a Master job in a sequence(that does some inserts in a table). After succesfull completion it in turn starts the 5 other jobs simultaneuosly. There is one more job that has to be run ONLY after the sucessfull completion of all the five jobs. Is it possible to start the last job after the succesfull completion of the prior 5 jobs. Can we use SEQUENCER in this case.

Thanks
Raj
chucksmith
Premium Member
Premium Member
Posts: 385
Joined: Wed Jun 16, 2004 12:43 pm
Location: Virginia, USA
Contact:

Post by chucksmith »

Yes!

Using your sequencer:

job1 job activity stage links to a sequencer 1 with mode set to any or all (since this is only one job, both are the same)

Sequencer 1 has 5 output links to jobs 2-6 job activity stages.

Each job activity stage for jobs 2 - 6 output links to sequencer 2.

Set sequencer 2's mode to all.

The output link from sequencer 2 links to job 7 job activity stage.

All of the above mentioned job activity output links should have their Expression Type set to OK on the trigger tab.

Chuck
rkumar28
Participant
Posts: 43
Joined: Tue Mar 30, 2004 9:39 am

Post by rkumar28 »

Thanks Chuck for the quick response.

I should have asked it with my initial question. I have one related question to this. Please let me know if I should be posting this separately.

The Scenario is:
I have 5 flat files(f1,f2,f3,f4,f5). These files are FTPed daily to an assigned folders on unix. 5 jobs(j1,j2,j3,j4,j5)( that I mentioned in my initial question)points to 5 different folder on unix where I have separate predefined flat files that datastage jobs points to. I call a unix script on before job routine that loads the ftped files into the 5 predefined files where all my datastage jobs(j1,j2,j3,j4,j5) is pointing to. Since I am running all the jobs in one sequence...all the 5 jobs runs simultaneously and once finished calls the last job.

The problem is if one source doesn't sends a file on a particular day, the DataStage jobs aborts because of no data in the predefined file where my datastage is pointing to. I am trying to automate this process and am trying to see if I can avoid the abort if there is no data, so that I can successfully kicks my last job.But my jobs needs to abort if there is some problem with the data in the flat file.

Please advice how to go about on this.
chucksmith wrote:Yes!

Using your sequencer:

job1 job activity stage links to a sequencer 1 with mode set to any or all (since this is only one job, both are the same)

Sequencer 1 has 5 output links to jobs 2-6 job activity stages.

Each job activity stage for jobs 2 - 6 output links to sequencer 2.

Set sequencer 2's mode to all.

The output link from sequencer 2 links to job 7 job activity stage.

All of the above mentioned job activity output links should have their Expression Type set to OK on the trigger tab.

Chuck
vmcburney
Participant
Posts: 3593
Joined: Thu Jan 23, 2003 5:25 pm
Location: Australia, Melbourne
Contact:

Post by vmcburney »

Take your FTP script execution out of the before-job routine and put it into your sequence job, you will find this gives you much more control and flexibility.

Consider creating a job parameter as a flag field called something like FTPSOURCEFILES with values of true and false. The first time you run your sequence job it is set to true, if you are restarting after a failure and your files are already FTP'd and you can set it to false. When you are testing your jobs repeatedly you can set it to false to save time.

Now someone could correct me on this but I think if you try and execute five routine, execute or FTP stages in parallel in a sequence job it will bottleneck and run them in sequence as the control code generated will run sequencially and wait for a response from each stage before continuing. If this is the case then consider creating a multiple instance sequence job that calls an FTP routine and put this job on your main sequence job five times.

Set the trigger following your ftp stage to only continue if the FTP is successful or to go to an error notification stage if it is unsuccessful.
Post Reply