DSXchange

Hi
I am using a Aggregator stage in a PX job , Doing hash partitioning on it and sorting the data as well
I am getting these warnings

Agg,1: Hash table has grown to 16384 entries.
Agg,0: Hash table has grown to 16384 entries.
.....

Can anyone please suggest how to get rid of these warnings

Thanks

In the options property of the aggregator stage select Method=Sort not Hash. This should take care of the warning.

Hi
I was getting this warning when i did that

Agg: When checking operator: User inserted sort "Agg.ToAgg_Sort" does not fulfill the sort requirements of the downstream operator "APT_SortedGroup2Operator in Agg"

But when i unchecked the Perform Sort in the Aggregator it worked

Thanks

Make sure Data is sorted and key partitioned before passing it on to aggregator stage to get the accurate results.

You can use a SORT stage before the AGGR stage to sort the data and the partitioning must be on the same fields as the grouping keys in the AGGR stage.

Hash table aggregation method is recommended for 1000 or fewer distinct grouping values per MB. Looks like you've exceeded this limit.

Do you initially have any partiton and sort on the key on which the aggregation is done?

ray.wurlod wrote:Hash table aggregation method is recommended for 1000 or fewer distinct grouping values per MB. Looks like you've exceeded this limit.

How can we know how much MB our aggregator stage is using. If we know that we can count of record length and chosse between hash or sort.
please reply

ray.wurlod wrote:Hash table aggregation method is recommended for 1000 or fewer distinct grouping values per MB. Looks like you've exceeded this limit.

How can we know how much MB our aggregator stage is using. If we know that we can count of record length and chosse between hash or sort.
please reply

APT_PM_PLAYER_MEMORY