Lengthy Startup time and short run time

pavankvk · Post by **pavankvk** » Fri May 22, 2009 8:26 am

Hi

I have couple of jobs in my sequencer that are taking un usually high start up time( ~ 8 min) irrespective of number of records in the transaction.. the run time is very short say 20 secs etc..What could be the reason for this high start up time? The job is not complex..couple of look ups and a transformer with say 7 stage variables.

Please help!!

TIA

IBM Analytics Champion 2009 - 2020 · Post by **asorrell** » Fri May 22, 2009 9:06 am

Can't say - you didn't give enough info. First guess would be you are waiting on a response from the database. Even if the job only has to process 20 records, if the SQL statement is parsing a 10GB database to find those 20 records you are going to be waiting for a while.

If you want more than a guess, please post details including data sources and targets, SQL statements, and any commonality between the jobs that are affected.

chulett · Post by **chulett** » Fri May 22, 2009 9:15 am

And also define what you mean by "startup time". For the job to say that it is "started"? Or perhaps log the "Initiated" entry? For an initial query to start returing rows? What?

pavankvk · Post by **pavankvk** » Fri May 22, 2009 9:25 am

chulett wrote:And also define what you mean by "startup time". For the job to say that it is "started"? Or perhaps log the "Initiated" entry? For an initial query to start returing rows? What? ...

I am seeing the log and thats where i get the startup time and run time..the job doesnt do anything. Its not reading from any huge database..just 2 look up tables with few hundreds of records, followed by a transformer with 7 stage variables..

The job doesnt kick of..in the log, the job starts..and takes a long time to show up message that Config file is loaded and then finishes quickly..No other job in the same sequencer behaves like this.

pavankvk · Post by **pavankvk** » Fri May 22, 2009 9:30 am

pavankvk wrote:
chulett wrote:And also define what you mean by "startup time". For the job to say that it is "started"? Or perhaps log the "Initiated" entry? For an initial query to start returing rows? What? ...
I am seeing the log and thats where i get the startup time and run time..the job doesnt do anything. Its not reading from any huge database..just 2 look up tables with few hundreds of records, followed by a transformer with 7 stage variables..

The job doesnt kick of..in the log, the job starts..and takes a long time to show up message that Config file is loaded and then finishes quickly..No other job in the same sequencer behaves like this.

This is what i see in the log.

Occurred: 11:02:02 AM On date: 5/22/2009 Type: Info
Event: main_program: orchgeneral: loaded (...)
Occurred: 11:10:04 AM On date: 5/22/2009 Type: Info
Event: main_program: APT configuration file: /opt/IBM/InformationServer/Server/Configurations/config.apt (...)

check the time diff..its 8 min.

Event: main_program: Startup time, 8:12; production run time, 0:28.

chulett · Post by **chulett** » Fri May 22, 2009 9:52 am

Ah... didn't realize it actually (and officially) reported "Startup time" in the log, my bad. I wonder exactly what that all entails? It doesn't include anything 'before job', just checked and verified that.

I wonder if it would help at all to dump the score? Also, does this job use a different config file than the others, perhaps one stored in a different location?

ArndW · Post by **ArndW** » Mon May 25, 2009 12:47 am

Could you say something about the jobs that are taking a long time to start? How many nodes (and does startup time change if you add/remove nodes) and how many processes get started?

ray.wurlod · Post by **ray.wurlod** » Mon May 25, 2009 5:12 am

It might be useful to have the individual processes show their startup time, by setting APT_STARTUP_STATUS environment variable.