Hi,
I'm reading a sequential file and passing it to transformer to do hash partition.
data from trnsformer is passed to Sort stage where I'm doing Sorting on natural key in ascending and timestamp in descending to get the latest record at the top then it is passed to remove duplicate stage to retain the first record for that natural key.
I'm getting the below warning :
RDUP_NATKEY_TS: When checking operator: User inserted sort "SORT_NATKEY_TS_Desc" does not fulfill the sort requirements of the downstream operator "RDUP_NATKEY_TS"
can any please let me know how to fix this warning.
Thanks
Suresh
Warning in when using Remove duplicate and sort stage
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
I think your descending sort may be the problem. Most sorts by default are ascending. Your sort stage is doing an ascending, descending sort and your remove duplicates stage is probably configured for an ascending, ascending row order. You need to keep them synchronised to avoid the warning.
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
-
- Charter Member
- Posts: 822
- Joined: Sat Sep 17, 2005 5:25 pm
- Location: USA
Read this post.
viewtopic.php?t=95001&highlight=When+ch ... erted+sort
viewtopic.php?t=95001&highlight=When+ch ... erted+sort
I haven't failed, I've found 10,000 ways that don't work.
Thomas Alva Edison(1847-1931)
Thomas Alva Edison(1847-1931)
-
- Premium Member
- Posts: 397
- Joined: Wed Apr 12, 2006 2:28 pm
- Location: Tennesse