Multiple Join stages with Same Join Key
Posted: Wed Dec 11, 2013 9:36 am
Hi All,
I need to join more than 5 tables from different databases on same key. My current job design is - using three join stages to join all these tables in same job.
Is it suggestible to have one join stage in design to join all tables at once or joining two tables with join stage and remaining two tables with other join stage by keeping the same partition?
If I use three join stages to join all these tables, I think I wont overload scratch space (As it is parallel, it will free up the scratch space as data is moving to next stage). In this design, off course it will create more processes but I feel it give more performance.
Please share your ideas/suggestions on this design.
Thanks..
I need to join more than 5 tables from different databases on same key. My current job design is - using three join stages to join all these tables in same job.
Is it suggestible to have one join stage in design to join all tables at once or joining two tables with join stage and remaining two tables with other join stage by keeping the same partition?
If I use three join stages to join all these tables, I think I wont overload scratch space (As it is parallel, it will free up the scratch space as data is moving to next stage). In this design, off course it will create more processes but I feel it give more performance.
Please share your ideas/suggestions on this design.
Thanks..