Aggregator_49: When checking operator: User inserted sort "Sort_300" does not fulfill the sort requirements of the downstream operator "APT_SortedGroup2Operator in Aggregator_49"
I encounter this warning..
in the job, i use a sort before Aggregator.
in the sort, i partition with 13 columns and do sort for these 13 columns.
in Aggregator, i use same as partition, and group the 13 columns to sum output.
i am not sure why shows this warning. does the Aggregator must have partition setting in the stage ?
User inserted sort does not fulfill the sort requirements
Moderators: chulett, rschirm, roy
Re: Aggregator_49: When checking operator: User inserted sor
my colleague told me i should not use Hash to partition in the Sort stage. i am going to try another partition type
wuruimao
Re: Aggregator_49: When checking operator: User inserted sor
i found this in the forum
in the sort stage i need to sort 9 million data. and i set 1000 M in the sort stage.
So is that possible because of too big volumn of data?manoj wrote:Before join stage use sort stage and try this..
I think it may hapen because of memory issue..
Inline sort handle less volume of data.
in the sort stage i need to sort 9 million data. and i set 1000 M in the sort stage.
wuruimao
-
- Premium Member
- Posts: 376
- Joined: Sat Jan 07, 2012 12:25 pm
- Location: Piscataway
Did you check if the order of the keys in which you're sorting is the same as the order of your grouping keys in the aggregator?
Jerome
Data Integration Consultant at AWS
Connect With Me On LinkedIn
Life is really simple, but we insist on making it complicated.
Data Integration Consultant at AWS
Connect With Me On LinkedIn
Life is really simple, but we insist on making it complicated.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
This error is typical when your aggregation (or join) after the sort uses different keys than the sort. If you aggregate on 1 field then you need to sort on that 1 field (and not 2 or 3 keys).
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>