how fine duplicate records using transformer stage in server edition
[/b]
datastage
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 30
- Joined: Tue Jan 30, 2007 5:21 am
- Location: bangalore
datastage
prasad
-
- Participant
- Posts: 54
- Joined: Wed Oct 25, 2006 11:07 pm
- Location: Hyderabad
Re: datastage
Hi, If u click stage property button in Transformer u'll be presented with a dialogue box. Select the input tab. Select the partitioning u wanna use (Most probably Hash). Select 2 check boxes Sort, unique. Select the key column on which u wanna perform the Sorting....That's it. It will eliminate any duplicate records based on the Key columns u have selected
Re: datastage
The above option is not available in server edition.sudeepmantri wrote:Hi, If u click stage property button in Transformer u'll be presented with a dialogue box. Select the input tab. Select the partitioning u wanna use (Most probably Hash). Select 2 check boxes Sort, unique. Select the key column on which u wanna perform the Sorting....That's it. It will eliminate any duplicate records based on the Key columns u have selected
-
- Premium Member
- Posts: 210
- Joined: Wed Feb 16, 2005 7:17 am
HI,
In server you can achieve this by using some stage vars. For ex.
Sort the source data.
Orig=key1
Check= if Orig=Dup Then 1 Else 0
Dup=Orig
Assume that you have two o/p links, one for normal data / the other for dupliacte data. Then on the link for correct data, set the constrain to
Ckeck=0 And @INROWNUM>1
On the duplicate link, set the constarint to
Check=1
Then at the end, you will be required to pass the last row of your source file to the correct data file.
Hope the psuedo code works for you.
Thanks
Aakash
In server you can achieve this by using some stage vars. For ex.
Sort the source data.
Orig=key1
Check= if Orig=Dup Then 1 Else 0
Dup=Orig
Assume that you have two o/p links, one for normal data / the other for dupliacte data. Then on the link for correct data, set the constrain to
Ckeck=0 And @INROWNUM>1
On the duplicate link, set the constarint to
Check=1
Then at the end, you will be required to pass the last row of your source file to the correct data file.
Hope the psuedo code works for you.
Thanks
Aakash