I am having a dataset which contain 372 records. If i put remove duplicates stage , only 371 records are getting passed.It shws only one duplicate is there.
I want to peek that 1 duplicate recrd alone.
Whats the simplest way?
a)I have tried to load the data to sequential file and planned to use uniq -d.
But i dont have permission to create a sequential file.
b)Even if i peek all the records, it's very difficult to find the duplicate recrd.
C)Another way i can think is having 372 records(Orginal dataset), and having 371 recrds in another dataset(Created after removing duplicates).Then using Change capture, we can capture .
Is there any simplest way to find that duplicate record in datastage?
Thanks
Peeking the Duplicate record alone
Moderators: chulett, rschirm, roy
Peeking the Duplicate record alone
pandeeswaran
-
- Premium Member
- Posts: 353
- Joined: Mon Jan 17, 2011 5:03 am
- Location: Mumbai, India
Re: Peeking the Duplicate record alone
Use sort stage and choose the option Create Key Change Column = True and take it in the following TFM.
DS User
DS User
Yes, to use this method you do need to use stage variables, which will hold the key values from the previous record and set a "flag" to use in your constraint
Alternatively, you can generate a Key Change column in your sort stage and simply check the value of that column for 0 in your transformer, which will indicate the duplicate record (based on the sort key values).
Regards,
Alternatively, you can generate a Key Change column in your sort stage and simply check the value of that column for 0 in your transformer, which will indicate the duplicate record (based on the sort key values).
Regards,
- james wiles
All generalizations are false, including this one - Mark Twain.
All generalizations are false, including this one - Mark Twain.