how to remove duplicates in transformer stage..?
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 783
- Joined: Mon Jan 16, 2006 10:17 pm
- Location: Sydney, Australia
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Use pairs of stage variables (the technique has been discussed here often) to detect change in the sorted keys - here "keys" has the same meaning as in the Remove Duplicates stage - and set a flag that can be used to constrain the output.
Get yourself a premium membership so you can read all of the above.
Get yourself a premium membership so you can read all of the above.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Re: how ro remove duplicates in transformer stage..?
Partition the Data in Transformer stage and use unique record option. You can remove duplicate record.
Nirdesh Kumar