elapsed time in "-report" for multi instance jobs

zulfi123786 · Post by **zulfi123786** » Mon Jul 07, 2014 9:27 am

Hi,

There is a multi instance job which is run with around 10 instances each taking around 5 minutes for each run but when I try to get the elapsed time using

Code: Select all

dsjob -report

Its giving a huge value

Code: Select all

etlld0001|zsyed10|/tmp> dsjob -report dev_fab DF_batch_Poc 2>/dev/null

**************************************************
STATUS REPORT FOR JOB: DF_batch_Poc
Generated: 2014-07-07 10:25:44
   Job start time=2013-12-13 01:02:54
   Job end time=2014-07-07 10:25:44
   Job elapsed time=4953:22:50
   Job status=99 (Not running)

This keeps me wondering how its calculating the elapsed time for multi instance jobs. Even if I count the individual elapsed times it would not account any closer to what it has reported.

Thanks

ArndW · Post by **ArndW** » Mon Jul 07, 2014 12:00 pm

Multi-instance jobs were added quite a while after DataStage was originally released and the way it was implemented was as good as could be done without changing the repository at that time, but there are still anomalies. This is particularly noticeable when you use the "auto-purge" on multi-instance jobs. I would guess that you have at least one instance that ran on December 13, 2013 still in the job log and DataStage is getting that wrong.

zulfi123786 · Post by **zulfi123786** » Mon Jul 07, 2014 12:14 pm

wouldn't that be serious misrepresentation of information ?

Anyways, I was designing a script to fetch elapsed times for all jobs including the multi instance ones. Is there a way to get through this issue or am I at a dead end ?

Thanks

PaulVL · Post by **PaulVL** » Mon Jul 07, 2014 12:35 pm

What number do you get when you add the invocation ID to your jobname?

dsjob -report dev_fab DF_batch_Poc.my_invocation_id 2>/dev/null

zulfi123786 · Post by **zulfi123786** » Mon Jul 07, 2014 1:59 pm

PaulVL wrote:What number do you get when you add the invocation ID to your jobname?

When the invocation id is used with the job name the result is all good but in our project (unfortunately) there are cases where a job was later changed to enable multi instance and I have to capture elapsed time of the parent job (one with no invocation id) and this is where I am getting stuck.

Also in Non-Production environment people just run the job with no invocation id and this too to be captured in the stats file.

ArndW · Post by **ArndW** » Mon Jul 07, 2014 2:06 pm

I you have auto-purge turned on (and you should have it turned on) then a job run without an instance won't show up correctly when you have instances.

That would be a bug, but I wouldn't hold my breath waiting for a quick patch from IBM for this.

ray.wurlod · Post by **ray.wurlod** » Mon Jul 07, 2014 3:37 pm

The elapsed time is the simple difference between start time and end time. Look at the timestamps on your job - they're six months apart!

Try not redirecting errors, to determine whether any errors are generated during execution of the dsjob command.

PaulVL · Post by **PaulVL** » Mon Jul 07, 2014 10:41 pm

What version of datastage are you running?

Is it a version that contains the DSODB Ops Console?

If so, you could query that database for the stats you need.

Have you try to list the invocations of a job via "dsjob -linvocations <proj> <job>" then loop for each entry and get your report that way? Should the start time be outside of your window (reporting on past 24 hours of jobs) then discard...

zulfi123786 · Post by **zulfi123786** » Tue Jul 08, 2014 5:52 am

It just got more interesting

Code: Select all

etlld0001|zsyed10|/u/zsyed10> dsjob -report dev_fab DF_batch_Poc

**************************************************
STATUS REPORT FOR JOB: DF_batch_Poc
Generated: 2014-07-08 06:48:47
   Job start time=2013-12-13 01:02:54
   Job end time=2014-07-08 06:48:47
   Job elapsed time=4973:45:53

Status code = 0
   Job status=99 (Not running)

Whenever I run the -report command the Job end time is the time when I run the command.

The log shows that the last time an instance of this job finished at 12/19/2013 7:16:44 AM but there seems to be a bug causing the end time to be current time .

Thanks

chulett · Post by **chulett** » Tue Jul 08, 2014 7:17 am

Report it. (no pun intended)

PaulVL · Post by **PaulVL** » Tue Jul 08, 2014 8:49 am

Could the job still be running? (an invocation of it I mean)