Handling Duplicates

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
ppp
Participant
Posts: 21
Joined: Mon Aug 31, 2009 11:53 am

Handling Duplicates

Post by ppp »

My input schema is
Grp_ID
Act_ID
agtxid
agtin

And if I have a duplicate row then I use a sort sage to sort on all the 4 fields and set the key change column to true.
So, in my transformer I want to put the word "MULTIPLE" if I see a duplicate record in a new output column. How can I achieve this?
dsedi
Participant
Posts: 220
Joined: Wed Jun 02, 2004 12:38 am

Post by dsedi »

Use the BASIC transformer and the RowProcCompareWithPreviousValue Transform funtion will accomplish the requirement.
Accept that some days you're the pigeon and some days you're the statue.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Do it based on the value of the key change column. Once you run some data through and see what it looks like, the 'how' will be obvious.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply