Lookup stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
just4u_sharath
Premium Member
Premium Member
Posts: 236
Joined: Sun Apr 01, 2007 7:41 am
Location: Michigan

Lookup stage

Post by just4u_sharath »

Small Question
Why we have to entire partition the reference data on Lookup stage?
because we are loading the whole reference data into memory and what is the need to use entire partitioning. When we are getting whole data into memory, do we still need partitioning on reference link. When we entire partition the reference link on a 4 node system, single copy of data is brought to memory and 4 nodes creates their own pointers to memory. In look up stage where did this nodes come from. (i understand look stage is Physical ram). please help
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

This is really the same question as the one in this post which was also posted by you today.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
just4u_sharath
Premium Member
Premium Member
Posts: 236
Joined: Sun Apr 01, 2007 7:41 am
Location: Michigan

Post by just4u_sharath »

ray.wurlod wrote:This is really the same question as the one in this post which was also posted by you today.
Sorry abt that double copy.
Post Reply