how to remove duplicates in transformer stage..?
Posted: Sun Jul 25, 2010 10:43 pm
Please provide solution for above query?
DSXchange
A short text to describe your forum
http://dsxchange.com./
how to remove duplicates in transformer stage..?
Page 1 of 1
Posted: Sun Jul 25, 2010 11:12 pm
search for "remove Duplicate"
Posted: Mon Jul 26, 2010 12:10 am
Use pairs of stage variables (the technique has been discussed here often) to detect change in the sorted keys - here "keys" has the same meaning as in the Remove Duplicates stage - and set a flag that can be used to constrain the output.
Get yourself a premium membership so you can read all of the above.
Get yourself a premium membership so you can read all of the above.
Re: how ro remove duplicates in transformer stage..?
Posted: Tue Jul 27, 2010 12:34 pm
Partition the Data in Transformer stage and use unique record option. You can remove duplicate record.
Posted: Thu Jul 29, 2010 5:13 am
Partition the Data in Transformer stage using hash partition and use unique record option.
Posted: Thu Jul 29, 2010 7:22 am
You could remove duplicates using transformer stage 1) with the help of stage variables 2) by doing unique/stable sort.
Posted: Thu Jul 29, 2010 7:30 am
No need for stable...