Hi
We have a DataStage server (7.5x2) with 2 projects; SysTest and UAT
I have an issue where job performance is not consistent across these projects.
Although job design, job sequencing, file sizes, directory structure - everything appears to be the same.
I've confirmed the projects are setup the same.
A job in one project ran in 1 hour and now runs 5+ hours in the other project.
And just to be interesting, a job ran in 2 hours in one project and then runs only a 1/2 hour in the other.
If this issue happened on the same job(s) or the same spot during the process, I would be able to review and make appropriate job design changes.
But this is not the case.
Our System support people have been monitoring the server but we haven't seen
anything that would cause a job to be long-running (ie. high cpu usage,high disk i/o).
The latest example is a job that ran in SysTest in 1.5 hours and has been running for over 8 hours in UAT.
I'm not sure what else to check.
Any ideas\suggestions are appreciated.
Thanks Debbie
Inconsistent job performance
Moderators: chulett, rschirm, roy
The jobs so far that have had performance issues use only flat files and hash files.
We haven't had issues extracting from DB2 on z\os. Likely due to our job design (one job per table\query extracted to flat file).
We've been monitoring the read\write I/O on the various files and the processes aren't anywhere near capacity for the server.
Thanks for the suggestion
-d
We haven't had issues extracting from DB2 on z\os. Likely due to our job design (one job per table\query extracted to flat file).
We've been monitoring the read\write I/O on the various files and the processes aren't anywhere near capacity for the server.
Thanks for the suggestion
-d
-
- Participant
- Posts: 597
- Joined: Fri Apr 29, 2005 6:19 am
- Location: Singapore
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Or perhaps to differing volumes of data. Are you using read or write cache with the hashed files? Try disabling cache and see whether the differences disappear (you can re-enable later).
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Just to post the final findings on this issue.
After many tests, performance monitoring, and explaining how DataStage works.....
A registry change (by the Tech folks) to 'fix' a previous FTP issue we were having has been found to cause problems with file caching, clearing memory caches and some other interesting os-level problems -- ticket opened with Microsoft.
Thanks for the suggestions
After many tests, performance monitoring, and explaining how DataStage works.....
A registry change (by the Tech folks) to 'fix' a previous FTP issue we were having has been found to cause problems with file caching, clearing memory caches and some other interesting os-level problems -- ticket opened with Microsoft.
Thanks for the suggestions
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: