Max no of jobs that can run in parallel ?

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
hamzaqk
Participant
Posts: 249
Joined: Tue Apr 17, 2007 5:50 am
Location: islamabad

Max no of jobs that can run in parallel ?

Post by hamzaqk »

Hi,

just a quick one. is there any limitations on the number of jobs that can run in parallel in DS both in server and parallel edition and can you limit the jobs that can run in parallel at a given time ? if i want to limit DS to run no more than 5 jobs at the same time in parallel .i.e. 5 jobs start to run at the same time. 4 are still running and one finishes. DS selects one more job from the pool and runs it such that at any given time only 5 jobs can run at the same time . is this possible ?
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

The maximum number of concurrent jobs that can run is dependant upon so many different variable factors that it is impossible to give a number. It can range from very few to several hundred or more depending upon your configuration.

There is no builtin mechanism to effectively form a queue for jobs, but I (and I'm sure several others here) have written such programs in the past. This type of "throttling" is often done on larger systems where a resource contention can cause all jobs to fail (due to timeouts).

How you implement this type of a mechanism is dependant upon where you wish to control processes. If you start these jobs through a command line then you can use a simple "ps -ef | grep" type of command to see how much is going on, then just pause until that number goes below your set threshold. Or you can use the DS/BASIC published calls to DSGetJobInfo() to see how many jobs in the project are running and wait there.
Post Reply