REMOVE DUPLICATES IN TRANSFORMER STAGE

[hemanth12]

Our requirement is

DB-->TR-->Seq or DB(rest of the records in target)
|
Rejected file(seq)(in this i need duplicates in rejected file rest of the records in target)

plz give me the briefly explanation.if it is posible with screen shots to send me..
my mail-id:hemanthrao.k@gmail.com

Thanks in advance...

[ray.wurlod]

Set up a unique sort on the input link to the Transformer stage.

But it's a silly requirement to want to do this in the Transformer stage.

[hemanth12]

thanks for reply....

but my requirement is
Don't use any of the part ions technic and sort ...unique

I need duplication records in one file and remaining records in another file .using only for transformer stage..don't use any remove duplicate .in transformer don't use any of the partion technic ....


Regards
Hemanth

Set up a unique sort on the input link to the Transformer stage.

But it's a silly requirement to want to do this in the Transformer stage. ...

[ray.wurlod]

Resist stupid requirements. You can't do it without sorted data. And that implies properly partitioned data.

[Vidyut]

Ray is right without sorted data u cant remove duplicates using transformer.
What you can do is sort the data on key and then use two stage variables to remove the duplicates.
one having the last value and one current and by comparing both u can remove the duplicates