performance tuning on dastage jobs

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
ravipuli
Participant
Posts: 22
Joined: Thu Jul 12, 2012 8:12 am

performance tuning on dastage jobs

Post by ravipuli »

we have same jobs on two diffrent prodution environment.All jobs are same there is no diffrence on jobs.
in one production with 50K data job has finishedsuccessfully..aprox..15 mints
In other production with 50K data job has taking nearly 1 hr.evry thing is same on both env jobs.

Please help me on this....where we can tune
BI-RMA
Premium Member
Premium Member
Posts: 463
Joined: Sun Nov 01, 2009 3:55 pm
Location: Hamburg

Post by BI-RMA »

Hi ravipuli,

it is almost impossible to give an answer to your question. You say that both jobs and both environments are (at least as far as you can see) identical in terms of hardware. Hopefully you've also got approximately the same workload on both servers while running the job in question (there is no traffic-jam on the slow server).

Are you using identical data-sources? If I interpret your text correctly, the job only moves 50.000 records and takes 15 (!) minutes for that, at best. Which would seem to me to be an awful lot, unless the job contains overwhelmingly complex SQL-statements...
"It is not the lucky ones are grateful.
There are the grateful those are happy." Francis Bacon
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

I would bet you need to analyze the target table.
Mamu Kim
PaulVL
Premium Member
Premium Member
Posts: 1315
Joined: Fri Dec 17, 2010 4:36 pm

Post by PaulVL »

What does the job do? Read from a database or write to a database?

Understand that the typical slow part in a job is IO. That could be writing to your disk or writing to a database.

to test out some stuff, make a copy of the job and replace the target database with a copy stage.

recompile and run.

Now you have eliminated the target DB from the equation. Does your job still under perform on your prod environment? If it is still very slow... then it's not your DB interaction. (but honestly... it's most likely a DB interaction slowness when you are talking mins vs hours). Point is... start breaking down your job into it's logical sections and narrow down the possibilities.
ravipuli
Participant
Posts: 22
Joined: Thu Jul 12, 2012 8:12 am

thank you..very much to all

Post by ravipuli »

Got some resolutions from DBA.We need to work on SQL queerys.Need to filter the data more and need to create an index some coulmns.....thanks to all
ravikumar
Post Reply