Stage Memory usage

[shrey3a]

Hi Gurus,

I'm in middle of designing the process and confused which stage to use i.e.
Join vs Merge , i can achieve the result from both the stages but we will be processing loads of data and trying to join/ merge 7-8 links with same key value.

I wanted to know which of the stage will use less memory and will be faster. Should I use Join stage or merge stage

Regards,

[wesd]

Hi Gurus,

I'm in middle of designing the process and confused which stage to use i.e.
Join vs Merge , i can achieve the result from both the stages but we will be processing loads of data and trying to join/ merge 7-8 links with same key value.

I wanted to know which of the stage will use less memory and will be faster. Should I use Join stage or merge stage

Regards,

Depends on whether you want to have reject links or not. Join has great performance but a Merge will allow you to use rejects for unmatched columns and error tracking.

[ray.wurlod]

Behaviour is different if there are duplicates on the inputs. In a Merge stage rows are consumed from the Update inputs.

[vmcburney]

Why not do performance testing on both designs? If you have the core of the job built it should be easy providing a version with each stage in it. So much depends on your scratch space, I/O, RAM utilisation across the rest of the job, RAM to CPU ration etc etc etc. The only firm answer I can give you is to test both to find out.