DataStage parallel jobs

srilatha · Post by **srilatha** » Sun Nov 11, 2007 4:00 am

hi all,
i want to know the exact difference between join,merge,look up stages in
data stage parallel jobs.

roy · Post by **roy** » Sun Nov 11, 2007 6:51 am

Hi,
What insight have you gained by reading the manual of those stages?
After you do so and explain what you don't understand or what is not working according t wat you understand then we might be able to help further.

srilatha · Post by **srilatha** » Mon Nov 12, 2007 1:46 am

hi sir,
what i mean is in
join stage we can combine two or more data sets with a common key column.
merge stage also does the same thing with the sorted input data sets.
where as the lookup also checks the primary data with the reference data sets and combines the data sets.suppose i want to combine more than two data sets,which stage i have to go for?
Is there any limitation for these stages?

ray.wurlod · Post by **ray.wurlod** » Mon Nov 12, 2007 4:21 am

All three stage types support more than two inputs. Which stage type you use depends on precisely how you need to combine them.

vijay.rajendran · Post by **vijay.rajendran** » Mon Nov 12, 2007 6:32 pm

srilatha wrote:hi sir,
what i mean is in
join stage we can combine two or more data sets with a common key column.
merge stage also does the same thing with the sorted input data sets.
where as the lookup also checks the primary data with the reference data sets and combines the data sets.suppose i want to combine more than two data sets,which stage i have to go for?
Is there any limitation for these stages?

Hi Srilatha,

One difference with lookup stage is how the data set is looked up on. The lookup dataset will be loaded into the memory before the lookup is done. This is not recommended to use lookup stage if your lookup dataset is big, unless you have unlimited or huge memory.