Information about DataStage job log

invincibleDeepak · Post by **invincibleDeepak** » Tue May 25, 2010 9:55 am

Occurred: 11:37:27 AM On date: 12/15/2005 Type: Info
Event: main_program: Startup Time 0:02 Production Run Time 0:00

Can somebody throw light on Startup Time and Production Run Time. What is the significance of this?

ray.wurlod · Post by **ray.wurlod** » Tue May 25, 2010 5:24 pm

Startup time is the amount of time it took to get all the nodes into a running state. Processing time is the amount of time consumed actually processing data. Each is rounded to the nearest whole second.

In your case, processing took less than 0.5 seconds.

Startup time tends to be fairly constant for a given number of nodes (depending on network traffic, perhaps). Processing time should increase with the volume of data processed in an almost linear manner.

Thus, for example, with millions of records your processing time might be tens of seconds, but the startup time would remain two seconds. The moral of the story is that parallel jobs are intended for "heavy lifting". Server jobs, which don't have the startup overhead, can be preferred for smaller volumes.