Explicit sort vs. Sort in the join stage?

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
mctny
Charter Member
Charter Member
Posts: 166
Joined: Thu Feb 02, 2006 6:55 am

Explicit sort vs. Sort in the join stage?

Post by mctny »

Helloo Guys/Girls,

I have a simple question, when you are joining two tables by using join stage, do you have to sort and partition the tables by using a sort stage or is it enough to click on the auto box inside the join stage. in the reference book it says it is enough to just click on the auto check box and it will automatically sort by using the key fields and partition the file.

if it is enough which one is better? a person told me it is not a good idea to use the auto click box inside the join stage, it can give wrong results, poor performance etc, but I don't completely agree with him. just wanted to make sure here.

p.s. I am new to DS PX and since I don't have a parallel extender I cannot test by myself. just trying to learn by reading only , for now.

thanks in advance
Thanks,
Chad
__________________________________________________________________
"There are three kinds of people in this world; Ones who know how to count and the others who don't know how to count !"
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

Ok I assume the 'click box auto' that you mean is the Auto partiton.
Yes by enabling auto partiton, orchestrate it self will introduce a sort and partiton the data based on the key specifed in the join stage.
But based on the previous experience (of all who use this this forum) it is all better and effecient to use a manual sort stage and partion the data based on the key.
If you use auto partion, it not gaurented the type of partiton used. For scalability and for better troubleshooting, follow a the harder method.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Its always better to sort in a seperate stage, this will help in your performance as mentioned by kumar.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
Post Reply