Inconsistent job performance

dstoddard · Post by **dstoddard** » Sat Dec 20, 2008 1:35 am

Hi

We have a DataStage server (7.5x2) with 2 projects; SysTest and UAT

I have an issue where job performance is not consistent across these projects.
Although job design, job sequencing, file sizes, directory structure - everything appears to be the same.
I've confirmed the projects are setup the same.

A job in one project ran in 1 hour and now runs 5+ hours in the other project.
And just to be interesting, a job ran in 2 hours in one project and then runs only a 1/2 hour in the other.

If this issue happened on the same job(s) or the same spot during the process, I would be able to review and make appropriate job design changes.
But this is not the case.

Our System support people have been monitoring the server but we haven't seen
anything that would cause a job to be long-running (ie. high cpu usage,high disk i/o).

The latest example is a job that ran in SysTest in 1.5 hours and has been running for over 8 hours in UAT.

I'm not sure what else to check.

Any ideas\suggestions are appreciated.
Thanks Debbie

laknar · Post by **laknar** » Sat Dec 20, 2008 1:39 am

what about database is this the same DB or different.

If so please check the database level performance regarding reindex and indexes in tables which you are using and compare.

dstoddard · Post by **dstoddard** » Sat Dec 20, 2008 9:34 am

The jobs so far that have had performance issues use only flat files and hash files.

We haven't had issues extracting from DB2 on z\os. Likely due to our job design (one job per table\query extracted to flat file).

We've been monitoring the read\write I/O on the various files and the processes aren't anywhere near capacity for the server.

Thanks for the suggestion
-d

kandyshandy · Post by **kandyshandy** » Sat Dec 20, 2008 11:33 am

i guess this is totally related to the resources available for the jobs while running!!

ray.wurlod · Post by **ray.wurlod** » Sat Dec 20, 2008 2:39 pm

Or perhaps to differing volumes of data. Are you using read or write cache with the hashed files? Try disabling cache and see whether the differences disappear (you can re-enable later).

dstoddard · Post by **dstoddard** » Mon Jan 19, 2009 5:40 pm

Just to post the final findings on this issue.

After many tests, performance monitoring, and explaining how DataStage works.....

A registry change (by the Tech folks) to 'fix' a previous FTP issue we were having has been found to cause problems with file caching, clearing memory caches and some other interesting os-level problems -- ticket opened with Microsoft.

Thanks for the suggestions

ray.wurlod · Post by **ray.wurlod** » Mon Jan 19, 2009 6:27 pm

Gee, and I didn't even get to do my "what's changed?" schtick.

chulett · Post by **chulett** » Mon Jan 19, 2009 7:13 pm

Clearly "nothing" is not the correct answer.