How to run 3 shell scripts in parallel within datastage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
sourabhverma
Participant
Posts: 74
Joined: Thu Jan 05, 2006 2:07 am

How to run 3 shell scripts in parallel within datastage

Post by sourabhverma »

Hi All,

I am having a requirement wherein i have to execute 3 shell scripts in parallel after one of the parallel job gets completed. So for that, I have created one sequencer where i have attached the parallel job & after that i have attached 3 Eecute_Command stage for executing 3 shell scripts in parallel.

The thing is that even though the execute command stages are attached after the parallel job with the same trigger condition ,they are getting executed sequentially.

Is there any configuration setting i need to take care or are there any other factors.

Any help is appreciated.
Thanks,
Sourabh Verma
srinivas.g
Participant
Posts: 251
Joined: Mon Jun 09, 2008 5:52 am

Post by srinivas.g »

Just use Sequencer activity between job and execute command actvites.
Srinu Gadipudi
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Explain exactly what you mean by "i have attached 3 Execute_Command stage for executing 3 shell scripts in parallel."
-craig

"You can never have too many knives" -- Logan Nine Fingers
sourabhverma
Participant
Posts: 74
Joined: Thu Jan 05, 2006 2:07 am

Post by sourabhverma »

Hi All, Thanks for the reply.

Yes i already tried using sequencer activity in between but even then, no parallel execution of scripts.

The design is like :

one DS job -> Sequencer -> 3 links each one having Execute_Command.

I hope the design is clear now.
Thanks,
Sourabh Verma
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

We've discusses this a few times lately. Since there's no background activity for commands, they are blocking operations. From what I recall, you may be able to build three other Sequence jobs that each have a single Execute Command stage in them and run those just after the Sequencer.

Or a single script to run the three scripts, one after the other in the background, but then you could have a control issue with knowing when all three are complete.
-craig

"You can never have too many knives" -- Logan Nine Fingers
sourabhverma
Participant
Posts: 74
Joined: Thu Jan 05, 2006 2:07 am

Post by sourabhverma »

Thanks chulett,

I basically created separate sequencer for each of the shell script & called according to my requirment & i got what i wanted to do.
Thanks,
Sourabh Verma
Post Reply