How to remove Duplicate Record ?

[ray.wurlod]

Are you using a server job, a mainframe job or a parallel job?

In server and mainframe jobs, the easiest way is to have a sorted input stream and use a stage variable (in a Transformer stage) to detect duplicates; that stage variable is then used in the output constraint expression. It can also be done with a hashed file stage (server jobs only) or an aggregator stage.

In parallel jobs there is an explicit "remove duplicates" stage.



Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518