Sort Stage Vs CPU utilization

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
dhanashreepanse
Participant
Posts: 25
Joined: Fri Jan 11, 2008 12:49 am
Location: Pune, India

Sort Stage Vs CPU utilization

Post by dhanashreepanse »

I read that the sort stage in PX is a lot faster and it is recommended that by using this before the join stage would improve the job performance.
But at the same time, the CPU utilisation is also high.

How should one decide how many sort stages should be used in one job so as to keep the CPU utilisation within limits and also considering the fact that along with this job, there are 10-20 other jobs also running in parallel and they should also not be starved of resources.
Please share your thoughts on this.

Thanks in advance.

Regards,
Dhanashree
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

How long is a piece of string?

Seriously yours is not a question that can be answered without knowing the results of resource monitoring on your machine.

If you don't use a Sort stage or a link sort but a sort is required, then a tsort operator will be inserted when the score is composed. So you end up doing the sorting anyway.

How much total load the machine(s) can bear is a function of the total demand for resources of everything that is running simultaneously. Simple supply-and-demand economics. Supply is constant (until you buy more), demand can be monitored/measured.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply