In a PX job, If data is partitioned on key 1 and then aggregated on key 2, what issues could arise?
Thx
VS
partitioning data on key1 and aggregating on key2
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 123
- Joined: Wed May 18, 2005 7:41 am
- Location: USA
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Re: partitioning data on key1 and aggregating on key2
Are you sorting on key 2?satish_valavala wrote:In a PX job, If data is partitioned on key 1 and then aggregated on key 2, what issues could arise?
Thx
VS
If not, the Aggregator stage will use a lot more memory than otherwise.
Partitioning on other than key2 may mean that some key2 values are on node 1 and some key2 values are on node 2 and so on. That is, you may not get all the key2 values in one group in the final result. This is almost certainly not desirable.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 123
- Joined: Wed May 18, 2005 7:41 am
- Location: USA