Performance statistics in a log file

lakshman_gk · Post by **lakshman_gk** » Tue Nov 16, 2004 6:07 pm

Hi,

Is there a way to get the performance statistics in the log file like I would like to have the information for each link with link name and number of records processed for that link in the log file. Is there a way to accomplish this. Please do reply as we are facing this requirement in our project right now.

kduke · Post by **kduke** » Tue Nov 16, 2004 6:24 pm

Go to either my tips page or ADN and download EtlStats.zip. This is a few jobs to get row counts and store them in tables. Do a search. I have explained this in detail on both web sites. Very easy to run and setup. We store row counts in our data warehouse so we can look back over time and see if a job is slowing down.

I have several more jobs which I need to post to link ETL_ROW_HIST to ETL_JOB to get the job category. I now also have ETL_TABLE which will allow you to report by table name instead of just job and link name. I may have included the jobs to update ETL_JOB already. I have to look. I have been busy. We went live with phase 1 this week. I should post it soon.

This history of run times and row counts are very important. I would like to get mb/sec added soon like Ray suggested this week. Rows/sec are already there.

If anyone has used these then please let me know. Also if you have made improvements then let me know. You will not hurt my feelings if you improved any of these jobs. Post your solutions where all can see them either here or ADN.

ray.wurlod · Post by **ray.wurlod** » Tue Nov 16, 2004 8:39 pm

More detailed statistics about performance can be had by choosing the Tracing tab in the Job Run Options dialog (when you're about to run a job in Director), selecting an active stage from the list, and checking the Performance Statistics check box before clicking Run.

trokosz · Post by **trokosz** » Wed Nov 17, 2004 12:34 pm

In Administrator you can go to environment variables for the Project and set APT_RECORD_COUNTS in Parallel to true.

ray.wurlod · Post by **ray.wurlod** » Wed Nov 17, 2004 3:22 pm

APT_RECORD_COUNTS and the other APT... environment variables apply only to parallel jobs. They will have no effect at all on server jobs.

trokosz · Post by **trokosz** » Wed Nov 17, 2004 4:33 pm

Correct...sorry I didn't see it was a Server Job....