PX Lookup performance

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
crouse
Charter Member
Charter Member
Posts: 204
Joined: Sun Oct 05, 2003 12:59 pm
Contact:
Contact crouse

PX Lookup performance

Post by crouse »

Anyone know if the performance of a PX job is hindered by having 7 separate lookup stages vs. 1 lookup stage doing 7 lookups?

Reason I have them busted out into 7 separate lookups is so that I can identify which lookup fails.

I have a scheme cooking to identify them via a single lookup, but wondering if there is a definite answer on "N lookup stages vs. 1 lookup stage doing N lookups".

Before you go there, I know I can code it up and try it. :)

Just wondering if it's a truism.

Thanks,
Craig
Craig Rouse
Griffin Resouces, Inc
www.griffinresources.com
Top
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:
Contact ray.wurlod

Post by ray.wurlod »

My feeling would be that one is better than many, because of the amount of work avoided buffering data between processes. Of course, diagnosis becomes ugly - but if it's only failed lookup you're interested in, then you have a Merge stage.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Top
Post Reply

Return to “IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)”