SORT stage does not fulfill the sort requirements for remove

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
nishantrk
Premium Member
Premium Member
Posts: 23
Joined: Fri May 27, 2011 11:43 am

SORT stage does not fulfill the sort requirements for remove

Post by nishantrk »

First of all ..yes..have read the previous posts on this...

My question here is what should I do if I need to sort it on 3 key ,to get themy required records as the first record and then remove duplicate only on 2 keys and keep the first record...
and don't get these warning..

Input

xxxx yyyy 1
pppp qqqq 2
xxxx yyyy 2
rrrr ffff 2
rrrr ffff 3
xxxx yyyy 3

required output

xxxx yyyy 1
pppp qqqq 2
rrrr ffff 2

is there any other way to achieve this without using the remove duplicate stage so that I don't get any warning message???
Mike
Premium Member
Premium Member
Posts: 1021
Joined: Sun Mar 03, 2002 6:01 pm
Location: Tampa, FL

Post by Mike »

In your sort stage, partition by the 2 keys defined in your remove duplicates stage, then sort by those 2 keys + the additional third key (i.e. partition by first 2 keys and sort by all 3 keys).

Mike
nishantrk
Premium Member
Premium Member
Posts: 23
Joined: Fri May 27, 2011 11:43 am

Post by nishantrk »

Mike wrote:In your sort stage, partition by the 2 keys defined in your remove duplicates stage, then sort by those 2 keys + the additional third key (i.e. partition by first 2 keys and sort by all 3 keys).

Mike
Thanks Mike...
Post Reply