Page 1 of 1
elapsed time in "-report" for multi instance jobs
Posted: Mon Jul 07, 2014 9:27 am
by zulfi123786
Hi,
There is a multi instance job which is run with around 10 instances each taking around 5 minutes for each run but when I try to get the elapsed time using
Its giving a huge value
Code: Select all
etlld0001|zsyed10|/tmp> dsjob -report dev_fab DF_batch_Poc 2>/dev/null
**************************************************
STATUS REPORT FOR JOB: DF_batch_Poc
Generated: 2014-07-07 10:25:44
Job start time=2013-12-13 01:02:54
Job end time=2014-07-07 10:25:44
Job elapsed time=4953:22:50
Job status=99 (Not running)
This keeps me wondering how its calculating the elapsed time for multi instance jobs. Even if I count the individual elapsed times it would not account any closer to what it has reported.
Thanks
Posted: Mon Jul 07, 2014 12:00 pm
by ArndW
Multi-instance jobs were added quite a while after DataStage was originally released and the way it was implemented was as good as could be done without changing the repository at that time, but there are still anomalies. This is particularly noticeable when you use the "auto-purge" on multi-instance jobs. I would guess that you have at least one instance that ran on December 13, 2013 still in the job log and DataStage is getting that wrong.
Posted: Mon Jul 07, 2014 12:14 pm
by zulfi123786
wouldn't that be serious misrepresentation of information ?
Anyways, I was designing a script to fetch elapsed times for all jobs including the multi instance ones. Is there a way to get through this issue or am I at a dead end ?
Thanks
Posted: Mon Jul 07, 2014 12:35 pm
by PaulVL
What number do you get when you add the invocation ID to your jobname?
dsjob -report dev_fab DF_batch_Poc.my_invocation_id 2>/dev/null
Posted: Mon Jul 07, 2014 1:59 pm
by zulfi123786
PaulVL wrote:What number do you get when you add the invocation ID to your jobname?
When the invocation id is used with the job name the result is all good but in our project (unfortunately) there are cases where a job was later changed to enable multi instance and I have to capture elapsed time of the parent job (one with no invocation id) and this is where I am getting stuck.
Also in Non-Production environment people just run the job with no invocation id and this too to be captured in the stats file.
Posted: Mon Jul 07, 2014 2:06 pm
by ArndW
I you have auto-purge turned on (and you should have it turned on) then a job run without an instance won't show up correctly when you have instances.
That would be a bug, but I wouldn't hold my breath waiting for a quick patch from IBM for this.
Posted: Mon Jul 07, 2014 3:37 pm
by ray.wurlod
The elapsed time is the simple difference between start time and end time. Look at the timestamps on your job - they're six months apart!
Try not redirecting errors, to determine whether any errors are generated during execution of the dsjob command.
Posted: Mon Jul 07, 2014 10:41 pm
by PaulVL
What version of datastage are you running?
Is it a version that contains the DSODB Ops Console?
If so, you could query that database for the stats you need.
Have you try to list the invocations of a job via "dsjob -linvocations <proj> <job>" then loop for each entry and get your report that way? Should the start time be outside of your window (reporting on past 24 hours of jobs) then discard...
Posted: Tue Jul 08, 2014 5:52 am
by zulfi123786
It just got more interesting
Code: Select all
etlld0001|zsyed10|/u/zsyed10> dsjob -report dev_fab DF_batch_Poc
**************************************************
STATUS REPORT FOR JOB: DF_batch_Poc
Generated: 2014-07-08 06:48:47
Job start time=2013-12-13 01:02:54
Job end time=2014-07-08 06:48:47
Job elapsed time=4973:45:53
Status code = 0
Job status=99 (Not running)
Whenever I run the -report command the Job end time is the time when I run the command.
The log shows that the last time an instance of this job finished at 12/19/2013 7:16:44 AM but there seems to be a bug causing the end time to be current time .
Thanks
Posted: Tue Jul 08, 2014 7:17 am
by chulett
Report it. (no pun intended)
Posted: Tue Jul 08, 2014 8:49 am
by PaulVL
Could the job still be running? (an invocation of it I mean)