Page 1 of 1

Parallel job startup time

Posted: Tue Aug 21, 2007 2:30 pm
by lgharis
I have read several topics and the Parallel Job Developer's Guide to understand there are several things that are done when a parallel job starts up. What I would like to know is what people have done to reduce startup time short of redesigning the job? The developer's are saying it is taking 15-20 minutes for a parallel job to start.

I do know that transformations and multiple nodes will slow down the start up. What else have you experienced that slows down the startup?

Posted: Tue Aug 21, 2007 3:45 pm
by ArndW
A startup time of 15-20 minutes is quite long. It probably includes the time required for the lookups used (be they lookup stages to databases or to PX files) to be loaded to memory.
The actual startup times for the processes to load (which increases as the number of nodes and stages goes up) will be in the range of several seconds to somewhere around a minute - you should check to see how many processes your job has at runtime by using APT_DUMP_SCORE.

Transformations do not slow down the startup time.