Sort VS. AGGR

ds_is_fun · Post by **ds_is_fun** » Mon Jul 18, 2005 8:33 am

Hi,
I currently have a design where I need to Sort and Aggr.
Do you recomment a Sort in AGGR stage (or) a SORT stage followed by a AGGR stage. If so, why?
Thanks!

ArndW · Post by **ArndW** » Mon Jul 18, 2005 8:39 am

I would always sort before the aggregator. The sort method to use really does depend upon your incoming data (i.e. coming from a table you can use the Database's mechanism, a flat file might be better sorted outside of DS, etc.).

Sainath.Srinivasan · Post by **Sainath.Srinivasan** » Mon Jul 18, 2005 9:31 am

I agree with ArndW. The agg stage is best doing its job alone.

ds_is_fun · Post by **ds_is_fun** » Mon Jul 18, 2005 10:19 am

Wouldnt it be better to use the parallelism mechanism on the SORT in DS instead of a flat file sort outside of DS.
My understanding is a sort of DS is not utilizing the PX mechanism. Im assuming partitioning and sorting in the SORT stage would be faster.
Thanks! Pl. reply!

ray.wurlod · Post by **ray.wurlod** » Mon Jul 18, 2005 2:52 pm

Do you need to sort the entire DataSet? Or just the data on each partition? In the latter case, the PX sort may well be faster.