DSXchange

Hi,

In one of my job, i can see the following -
Message: main_program: Startup time, 1:39:42; production run time, 34:23

What may be the issue here? I am loading data in 3 tables (1.5 Million, 21 Million, 13 Million)

Thanks,
Vishal Gupta

Take a look at this technote from IBM:

http://www-01.ibm.com/support/docview.w ... wg21580712

I don't think that is applicable here. We're talking about a "start up" time of over an hour and a half, not a couple of minutes when it should normally be a handful of seconds. I'm guessing this job always has this discrepancy...

I'd start by looking into what all is encompassed in that start up time. For instance, are you creating huge lookup caches?

I've noticed that "start up" time includes the time it takes to do a GENERATE STATISTICS in the case of a Netezza Connector as one example of something not intuitive.

Scan your job log in detail for clues about what may be included in your "start up" time.

Mike

How to check what is included in 'startup time' ?

There's not going to be anything that says "included in start up time".

Look at the timestamps for each log entry. Focus on large time gaps between consecutive log entries.

You may or may not find any clues.

Mike

I think "start up" time really should be called "overhead" time or "time not accounted for anywhere else" time.

Mike

Another thought... since generating stats doesn't seem to be included in production run time, I would guess that rebuilding indexes might be handled the same... and that could certainly account for a long time. Are you rebuilding indexes?

Mike

I'm sure that Ray will be along eventually to school us but as Mike mentioned it seems to me it would be worthwhile to check the log timestamps to see what all of the activities were that took place during that initial 1:39:42 of run time. Unless we're in that "not accounted for elsewhere" zone and start up time also includes wind down time... or the times when rows are not flowing through the job?

You can get more information about the startup by setting the environment variable APT_STARTUP_STATUS.
Startup time includes all the work done by the conductor up till the time that player processes begin execution. This includes:

• verifying licensing
  composing the score from the generated OSH and configuration file
  sending requests to start section leader process on each node
  waiting for section leaders to respond that they've started
  verifying licensing (= existence of parallel engine) on each node
  distributing the score to the section leaders

The environment variable setting will log events for most of these.

And, yes, anything more than a few seconds should be cause for concern.

Hi Ray

Thanks for that. I do have the same issue in my project and let me track it.

thanks Vishal Gupta for the question you raised.