Performance improvement for server job

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
somu_june
Premium Member
Premium Member
Posts: 439
Joined: Wed Sep 14, 2005 9:28 am
Location: 36p,reading road

Performance improvement for server job

Post by somu_june »

Hi

I have a server job with two active stages transformers and the first transformer stage had a complex logic like calling routines and 80 stage variables where in second transfromer is having less logic it had 4 stage variables and writing to a sequential file. The first transformer has 5 outputs and writing to a 4 sequential files and second transformer stage. The job is running with 750,000 records and ending in 06:50 minutes and the %CPU for first transformer it is showing 87% and for second transformer it is showing only 4% . I tried changing the inter process row buffer from 128 to 1024 KB and the job runtime decreased by only couple of seconds. The first transformer while processing it is processing only couple of 100 rows/second in couple of links. Is there any parameters that I need to fine tune the job to increase the performance of server job. Please let me know.


Thanks,
Soma
somaraju
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

Increasing the buffering isn't going to help. Your main transformer is taking a long time to process and is slowing down your "pipe" of data. Making the pipe bigger isn't going to help.

Speed up the first transformer by reducing the number of stage variables and routines called if possible.
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
arunkumarmm
Participant
Posts: 246
Joined: Mon Jun 30, 2008 3:22 am
Location: New York
Contact:

Post by arunkumarmm »

What is your source? Where does your transformer gets the data from? Are you using look-up?
Arun
Post Reply