Maximum number of jobs running at a given time

ankursaxena.2003 · Post by **ankursaxena.2003** » Fri Sep 28, 2012 9:10 am

Hi,

I was having chat with my manager yesterday and he was asking me that what is the maximum number of jobs that we can run in DataStage at a given time. Because some tools like Informatica has limit.

I have never ran more than 3 to 4 jobs at a given time.
So, I am curious to know what would be the maximum number of jobs anyone has run at a given time.

I know that the number of jobs will depend on environment like CPU's, hard disk space available for scratch disk and so on.

padmalavanya · Post by **padmalavanya** » Fri Sep 28, 2012 10:01 am

This is not a DataStage limit, but an operating system limit on the number

ArndW · Post by **ArndW** » Fri Sep 28, 2012 10:09 am

To expand on the previous answer: there are several hardcoded time limits within DataStage that, when they are passed, trigger jobs to fail.
The more things you run in parallel in a timesharing operatings system such as UNIX or Windows, the less time each process gets and therefore the slower processes execute. At some point the times get so slow that DataStage will abort. When this point is reached depends upon so many different factors that it is not really possible to specify an exact number or even to accurately predict when this happens.

ankursaxena.2003 · Post by **ankursaxena.2003** » Fri Sep 28, 2012 11:17 am

Thanks for the reply.

One of my colleague has concurrently ran 200 jobs at the same time.
So, what is the maximum number of jobs ran by you guys at a given time.

chulett · Post by **chulett** » Fri Sep 28, 2012 11:36 am

I'm sorry but does that really matter? There are SO many variables to this equation that any number posted here really is meaningless when it comes to what you can do on your system with your particular jobs.

ankursaxena.2003 · Post by **ankursaxena.2003** » Fri Sep 28, 2012 11:58 am

No.
It doesn't matter in my current project.

I was just curious.
I also thought that even if there was any such number than it would be too high.
And practically it would be impossible to reach that number.

ray.wurlod · Post by **ray.wurlod** » Fri Sep 28, 2012 4:31 pm

The main governing factor is the amount of resources (including CPU, memory, I/O bandwidth) that the server can supply. Bigger server; more jobs.

ankursaxena.2003 · Post by **ankursaxena.2003** » Mon Oct 01, 2012 8:01 am

Thanks guys for all your input.