Question about Merge and Lookup stages

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Dmitriy
Participant
Posts: 24
Joined: Wed Apr 14, 2010 9:30 pm

Question about Merge and Lookup stages

Post by Dmitriy »

Hello!

I have a few questions about two stages, Merge and Lookup.

1. Merge stage -
Is it possible to use Merge stage to translate several links with different set of columns into one output link?
It asks for a key field, so as it may seem that it's possible to inject DUMMY column with equal value (for ex. 1 ) in each link and then inside Merge, all the stream data from those several links will be gathered into one output.
Is it so?
If it's not so, what stage could fit this task assuming that several links doesn't have equal set of columns in them?

2. Lookup -
Is it possible some how to lookup a value in a table, inside the erivation (expression editor) in a Transformer stage where you can assign lookuped value to a stage variable?
Some analog to Informatica stand-alone lookup, where you can use it as a function inside an expression editor.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

1. The Merge stage really implements a "master and updates" model, but can handle differently structured Update links. I imagine it is possible to use different lookup keys on each Update link. However there's no means within the Merge stage for generating a common key - perhaps you could use a Column Generator stage on each input link to do this. (DataStage philosophy is "one task, one stage type").

2. This is not possible out of the box but, if you wanted to create your own routines, you could call these from the Transformer stage. However, why do you want to do it this way? If you use an upstream Lookup stage then the looked-up value will be available on the input link to the Transformer stage.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

2. An Informatica "disconnected" lookup is akin to a "conditional" lookup. There's not much sense in doing one for every record in the input source like a regular connected one would do, you use it in a derivation only "when needed". For example, I'm using them to emulate the "Initial Value" propery that DataStage has for stage variables that Informatica lacks, so rather than setting it once there and then leveraging it in the job, I have to disconnect the lookup, keep a counter for the records read and only call it for the first record through the mapping. After that the variable port reuses the looked up value rather than making the call again.

FYI.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply