Page 1 of 1

Sort VS. AGGR

Posted: Mon Jul 18, 2005 8:33 am
by ds_is_fun
Hi,
I currently have a design where I need to Sort and Aggr.
Do you recomment a Sort in AGGR stage (or) a SORT stage followed by a AGGR stage. If so, why?
Thanks!

Posted: Mon Jul 18, 2005 8:39 am
by ArndW
I would always sort before the aggregator. The sort method to use really does depend upon your incoming data (i.e. coming from a table you can use the Database's mechanism, a flat file might be better sorted outside of DS, etc.).

Posted: Mon Jul 18, 2005 9:31 am
by Sainath.Srinivasan
I agree with ArndW. The agg stage is best doing its job alone.

Posted: Mon Jul 18, 2005 10:19 am
by ds_is_fun
Wouldnt it be better to use the parallelism mechanism on the SORT in DS instead of a flat file sort outside of DS.
My understanding is a sort of DS is not utilizing the PX mechanism. Im assuming partitioning and sorting in the SORT stage would be faster.
Thanks! Pl. reply!

Posted: Mon Jul 18, 2005 2:52 pm
by ray.wurlod
Do you need to sort the entire DataSet? Or just the data on each partition? In the latter case, the PX sort may well be faster.