Aggregrator stage behaving indifferent

supraja10 · Post by **supraja10** » Fri Mar 11, 2011 7:36 am

Hi,

In my job, I have a input stage and pass it to aggregator stage.
My aggregator sometimes gives correct value and sometimes not. There is no change in data

I have values in below manner :
GroupKey COL_1 COL_2
a 8.127035 2.608613
a 3.499208 -3.6E-5

Sometimes it aggregrates and displays the result as below
GroupKey COL_1 COL_2
a 11.626243 2.608577

At sometimes, It displays the result in below manner
GroupKey COL_1 COL_2
a 8.127035 2.608613

There is no aggregation taking place here, It displays only one record and other record is not found in output.
Please suggest.

chulett · Post by **chulett** » Fri Mar 11, 2011 7:50 am

How many nodes are you running on and how are you partitioning the data?

supraja10 · Post by **supraja10** » Sun Mar 13, 2011 11:15 pm

We are using two nodes and applying Sort method

ray.wurlod · Post by **ray.wurlod** » Sun Mar 13, 2011 11:53 pm

How are you partitioning the data?

"Sort method" is how you are asking the Aggregator stage to work (and requires that the incoming data are sorted).

soumya5891 · Post by **soumya5891** » Thu Mar 17, 2011 10:46 pm

First use a sort stage ,sort on group keys.Make the partition to hash,with the same order as the sorting order.

Now use a aggregator stage with partition as same.Hope it will works.