Multiple lookups in one Lookup Stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Amuktamalyada
Participant
Posts: 19
Joined: Mon May 01, 2006 11:27 am

Multiple lookups in one Lookup Stage

Post by Amuktamalyada »

Good afternoon DSXchangers,

Is it a red flag if I use 1 lookup stage to do multiple different lookups in terms of performance? I am essentially using one lookup stage instead of many lookup stages to do lookups against different tables.

The algorithm is simplified as below.

Stream ----Only LookUp Stage--Lookup Dates Dim Table -- Date Key
--Lookup One_Dim Table -- One Key
--Lookup Two_Dim Table --Two Key

I test with decent data volume and does not cause an issue data or performance wise so far.

Any input would be greatly appreciated.

Thank you!!
nagarjuna
Premium Member
Premium Member
Posts: 533
Joined: Fri Jun 27, 2008 9:11 pm
Location: Chicago

Post by nagarjuna »

It depends on many factors ...How big is your reference data , How big is the memory ...How many processes are running at that time etc
Nag
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Re: Multiple lookups in one Lookup Stage

Post by chulett »

Amuktamalyada wrote:I am essentially using one lookup stage instead of many lookup stages
That makes absolutely no difference.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Amuktamalyada
Participant
Posts: 19
Joined: Mon May 01, 2006 11:27 am

Re: Multiple lookups in one Lookup Stage

Post by Amuktamalyada »

Thanks Craig!!
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

If you find a need to lookup multiple rows in more than one reference link then those would need separate lookup stages.
Choose a job you love, and you will never have to work a day in your life. - Confucius
Amuktamalyada
Participant
Posts: 19
Joined: Mon May 01, 2006 11:27 am

Post by Amuktamalyada »

Thanks Eric.
Post Reply