Datastage performance
Posted: Mon Feb 06, 2012 8:57 am
Can someone please let me know how running multiple instances of the same job vary in utilizing the resources such as data stage server/memory/input output throughtput perspective.
I have a job that runs on 2x1 configuration file and completes in under 5 minutes consistently. The job is processing around 3 million records. The job has only 3 stages, reading from a fileset, data processing happenning in transformer stage and writing out the output in a text using sequential file stage.
When the same job runs in 4 instances, each instance takes 13 to 20 minutes to completes. The output file record count could vary depending on the busines rules in the transformer stage. the output file record count can vary from 28k to 3 million.
Jobs are running on AIX server using datasstage version 8.1
Thanks
Pavan
I have a job that runs on 2x1 configuration file and completes in under 5 minutes consistently. The job is processing around 3 million records. The job has only 3 stages, reading from a fileset, data processing happenning in transformer stage and writing out the output in a text using sequential file stage.
When the same job runs in 4 instances, each instance takes 13 to 20 minutes to completes. The output file record count could vary depending on the busines rules in the transformer stage. the output file record count can vary from 28k to 3 million.
Jobs are running on AIX server using datasstage version 8.1
Thanks
Pavan