DSGetJobInfo take too much time

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
bart12872
Participant
Posts: 82
Joined: Fri Jan 19, 2007 5:38 pm

DSGetJobInfo take too much time

Post by bart12872 »

Hi,

I developped a routine to audit the execution of a sequencer.
So, this routine, get all jobs and determine jobs that ran.
For this jobs, I check the logs to display warning and fatal error in a file.
this is classical.

But when I developped and test this routine, It took 2 minutes. Today this routine take 1 hour.
But, the number of jobs is identical and the job logs are auto purge (3 execution of history).

What could cause this ?

thanks,
Martin.
priyadarshikunal
Premium Member
Premium Member
Posts: 1735
Joined: Thu Mar 01, 2007 5:44 am
Location: Troy, MI

Re: DSGetJobInfo take too much time

Post by priyadarshikunal »

bart12872 wrote:Hi,

I developped a routine to audit the execution of a sequencer.
So, this routine, get all jobs and determine jobs that ran.
For this jobs, I check the logs to display warning and fatal error in a file.
this is classical.

But when I developped and test this routine, It took 2 minutes. Today this routine take 1 hour.
But, the number of jobs is identical and the job logs are auto purge (3 execution of history).

What could cause this ?

thanks,
Martin.

How to debug that routine without looking at the code?

1. look for unnecessary looping in that code.
2. Avoid fetching un used details from the jobs.

for anything else paste the code

regards,
Priyadarshi Kunal

Genius may have its limitations, but stupidity is not thus handicapped. :wink:
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

That is why with EtlStats I run without waiting. So the next sequence kicks off while EtlStats us running the background. That is also why I do not like after job routines which gather this information. It adds a couple minutes to each job ran. Not good. Your overall run times will increase when you do it this way. These tasks are not CPU intensive so getting row counts while other jobs are running will not slow down any normal job.
Mamu Kim
Post Reply