Performance Issue

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
chandra.shekhar@tcs.com
Premium Member
Premium Member
Posts: 353
Joined: Mon Jan 17, 2011 5:03 am
Location: Mumbai, India

Performance Issue

Post by chandra.shekhar@tcs.com »

Performance issue :

Parallel job as follows:

1>Sequential file ---> transformer------->db2connector (upsert method)
2>Job running on datastage 8 node.Array size =2000 ,transaction size =4000,
readers per node=8.

Problem description: While running same job individually it takes 1 min36 sec to process 146000 records. while same job we run from sequencer it takes 4 mints to complete.

Why is it so ? we want to achieve same job should be run in 1 min36 sec using sequencer.

Kindly suggest.

Kishor
zulfi123786
Premium Member
Premium Member
Posts: 730
Joined: Tue Nov 04, 2008 10:14 am
Location: Bangalore

Post by zulfi123786 »

When all parameters set to same in both cases the difference should only be due to multiple jobs running from sequencer which might be using the same database where the bandwhidth is shared.
When this job is running from sequencer observe what others jobs are running simultaniously, it should help understand the reason.

Also did you try running the individual job and from sequencer multiple times or the statistics were just from single run?
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

Is the sequencer changing any paramters? Specifically check to see if it is changing the APT Config file.
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
chandra.shekhar@tcs.com
Premium Member
Premium Member
Posts: 353
Joined: Mon Jan 17, 2011 5:03 am
Location: Mumbai, India

Post by chandra.shekhar@tcs.com »

zulfi123786 wrote:When all parameters set to same in both cases the difference should only be due to multiple jobs running from sequencer which might be using the same database where the bandwhidth is shared.
When this job is running from sequencer observe what others jobs are running simultaniously, it should help understand the reason.

Also did you try running the individual job and from sequencer multiple times or the statistics were just from single run?

Hi,

I have run the same job individually without sequencer as well as sequencer also for (single same file)

without individually it is taking less time then sequencer .

like 2min.46sec individually and 4min.36sec while using sequencer.
Thanx and Regards,
ETL User
chandra.shekhar@tcs.com
Premium Member
Premium Member
Posts: 353
Joined: Mon Jan 17, 2011 5:03 am
Location: Mumbai, India

Post by chandra.shekhar@tcs.com »

asorrell wrote:Is the sequencer changing any paramters? Specifically check to see if it is changing the APT Config file.

for individual file we are not changing any parameter. All values are hard coded.
Thanx and Regards,
ETL User
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

How long does the job take to start under each circumstance?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply