Datastage performance

[pavan_test]

Can someone please let me know how running multiple instances of the same job vary in utilizing the resources such as data stage server/memory/input output throughtput perspective.

I have a job that runs on 2x1 configuration file and completes in under 5 minutes consistently. The job is processing around 3 million records. The job has only 3 stages, reading from a fileset, data processing happenning in transformer stage and writing out the output in a text using sequential file stage.

When the same job runs in 4 instances, each instance takes 13 to 20 minutes to completes. The output file record count could vary depending on the busines rules in the transformer stage. the output file record count can vary from 28k to 3 million.

Jobs are running on AIX server using datasstage version 8.1

Thanks
Pavan

[suse_dk]

Are you running the 4 instances simultane and writing to a single (the same) sequential file?

[pavan_test]

I am running 4 instances of the same job and each instance writing its own output. (different data, different o/p name of the file)

They are not writing to the same sequential file.

[qt_ky]

Just a guess from your description that your disk I/O is a bottleneck... Try some AIX monitoring commands like vmstat, iostat, mpstat, topas. What are the timings if your run each of the 4 instances one after another?