partitioning data on key1 and aggregating on key2

[satish_valavala]

In a PX job, If data is partitioned on key 1 and then aggregated on key 2, what issues could arise?

Thx
VS

[ray.wurlod]

In a PX job, If data is partitioned on key 1 and then aggregated on key 2, what issues could arise?
Thx
VS

Are you sorting on key 2?

If not, the Aggregator stage will use a lot more memory than otherwise.

Partitioning on other than key2 may mean that some key2 values are on node 1 and some key2 values are on node 2 and so on. That is, you may not get all the key2 values in one group in the final result. This is almost certainly not desirable.

[satish_valavala]

Thank you all

[kumar_s]

Hi,
Just an update...
Once the partion is done on Key1, as long as you sort on both the key (Key1 and Key2), you can aggregate on both the key.

-Kumar