improving performance
Posted: Fri May 13, 2011 1:46 am
Hi,
I have several datastage jobs which run to update multiple dim and fact tables in a data warehousing application.
Most of the jobs take around 3 to 4 minutes to complete and 2 or 3 jobs take around 40 minutes to complete. Altogether it takes around 4 to 5 hours to complete. We have an SLA for these jobs to complete within 6 hours. Although most of the days it completes on time, 2 or 3 days in a month, we miss the SLA.
Also, sometimes the job aborts due to more CPU utilization, where the same unix server is used for multiple applications
On days when the job takes more than 6 hours, i am not able to find out what the reason is. The number of records seem to be almost same, and the log doesnt show anything indifferent.
I suspect two things
1. More CPU utilization--I am not sure whether this will affect the speed of the job, will it?
2. More DB usage. I am checking with the DBA, he hasnt responded yet
What are the other parameters we need to look for?
I have several datastage jobs which run to update multiple dim and fact tables in a data warehousing application.
Most of the jobs take around 3 to 4 minutes to complete and 2 or 3 jobs take around 40 minutes to complete. Altogether it takes around 4 to 5 hours to complete. We have an SLA for these jobs to complete within 6 hours. Although most of the days it completes on time, 2 or 3 days in a month, we miss the SLA.
Also, sometimes the job aborts due to more CPU utilization, where the same unix server is used for multiple applications
On days when the job takes more than 6 hours, i am not able to find out what the reason is. The number of records seem to be almost same, and the log doesnt show anything indifferent.
I suspect two things
1. More CPU utilization--I am not sure whether this will affect the speed of the job, will it?
2. More DB usage. I am checking with the DBA, he hasnt responded yet
What are the other parameters we need to look for?