Increase in number of instances of the job increases time.

[chetan.c]

Hi ,
I have a multiple instance job which im calling through a sequence job which is also multiple instance.

There is another sequence where i call instances of the sequence.

Code: Select all

Master_Sequence--->Sequence(multiple instance)--->Parallel job(multipleinstance)

.

In the master sequence if the number of instance is 2 time consumption is 2 mins per instance.

If the instance is increased to 5 time consumption is 4:10 mins.
Almost 50% increase in execution time.
Kindly let me know why this could be happening?

Thanks,
Chetan

[chulett]

Resource contention. More != faster/better. Sometimes also known as the Law of Diminishing Returns.

[chetan.c]

Oh but i have to process that volume.One instance processes around 100 mb data and i have 5 instances to touch 500 mb.
Is there a workaround for this?


Thanks,
Chetan.C

[ray.wurlod]

Certainly. One workaround is to run on more powerful hardware. Simply adding more processing nodes (on other machines) may suffice.

[qt_ky]

It can depend on if the bottleneck is memory, CPU, disk, network. Tune the job design first. Also check any OS settings that can impose limits.

[chetan.c]

Thanks Ray, I have very little knowledge on that. Reading up on that and searching through the forum for that.

It can depend on if the bottleneck is memory, CPU, disk, network. Tune the job design first. Also check any OS settings that can impose limits.

Thanks eric.
Can you give me some pointers on how I can check CPU and disk.

Thanks,
Chetan.C

[qt_ky]

The topic link below highlights many options. If you go the unix command route, then I suggest you work locally with your unix admin for help on those...

viewtopic.php?t=144892

[chetan.c]

Thanks Eric.

Will work on it and post if i make any improvements.

Thanks,
Chetan.C