Hello Mentors,
Hope you doing well.
i have somthing to discuss relating join stage and its performance.
Its lot been dicussed but i would like to know how would optimize more in JOIN stage of DS...
lets take a scenerio
Join two tables each having 10 millions records....inner join
what i did is sorted them by qyerying to db and then again applied hash partition on 2-node system....i have flexibilty to select 4-node, 8-node
as well...
let me know if i have acheived great level of optimization or i can improve it further.
Thanks,
Devesh
Performance tuning in DS for Join..
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 148
- Joined: Thu Apr 10, 2008 12:47 am
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
DataStage is unnecessarily re-sorting your data. Add a Sort stage to each input link to the Join stage. In each Sort stage specify "don't sort (already sorted)" for the sort key columns.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.