Hi Gurus,
I'm in middle of designing the process and confused which stage to use i.e.
Join vs Merge , i can achieve the result from both the stages but we will be processing loads of data and trying to join/ merge 7-8 links with same key value.
I wanted to know which of the stage will use less memory and will be faster. Should I use Join stage or merge stage
Regards,
Stage Memory usage
Moderators: chulett, rschirm, roy
Re: Stage Memory usage
Depends on whether you want to have reject links or not. Join has great performance but a Merge will allow you to use rejects for unmatched columns and error tracking.shrey3a wrote:Hi Gurus,
I'm in middle of designing the process and confused which stage to use i.e.
Join vs Merge , i can achieve the result from both the stages but we will be processing loads of data and trying to join/ merge 7-8 links with same key value.
I wanted to know which of the stage will use less memory and will be faster. Should I use Join stage or merge stage
Regards,
Wes Dumey
Senior Consultant
Data Warehouse Projects
Senior Consultant
Data Warehouse Projects
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
Why not do performance testing on both designs? If you have the core of the job built it should be easy providing a version with each stage in it. So much depends on your scratch space, I/O, RAM utilisation across the rest of the job, RAM to CPU ration etc etc etc. The only firm answer I can give you is to test both to find out.
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn