To capture job statistics

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
parsi_cnu
Charter Member
Charter Member
Posts: 43
Joined: Thu Dec 04, 2003 4:26 pm

To capture job statistics

Post by parsi_cnu »

how to read the job log and to capture the statistics

like datastage job name,start time,end time ,project, duration,sucess etc
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

Do search for EtlStats. It is a bunch of jobs I give away on my tips page.
Mamu Kim
ghila
Premium Member
Premium Member
Posts: 41
Joined: Mon Mar 15, 2004 2:37 pm
Location: France

Re: To capture job statistics

Post by ghila »

parsi_cnu wrote:how to read the job log and to capture the statistics

like datastage job name,start time,end time ,project, duration,sucess etc
You can also use the "dsjob -report" command ( check the Advanced Developer's Guide for a list of all available options ).
I do prefer to use the XML option because it can easily be formated in HTML and that option lets you have the most detailed informations ( like PID of processes, calculated CPU time... ).
Regards,

Daniel
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

The main job in EtlStats uses dsjob -report XML to load these stats into tables. The DDL for ETL_JOB_HIST, ETL_ROW_HIST and ETL_PARAM_HIST is included in the zip file. The bugs in XML forced me to process the XML file 3 times. This main job is called DSJobReportDb.

If you call job DSJobReportDbDriver at the end of your sequences then it will get your row counts for all jobs in this sequence. It will call EtlChkSeqJobsNotRun and email you the log in an html attachment of all jobs which failed.

You get the source code to every job and sequence to you can modify to meet your needs.
Mamu Kim
Post Reply