DSXchange

Hi

I have the following set of records:

Empno name
1 Ash
1 Ush
2 Reeta
3 x
4 Y

I have retain first record and capture rejected records...I cant use remove duplicates since it does not have reject link.

Please help

you can use stage variables in transformer stage to get this result. make sure that records are hash partitioned and sorted on the required keys before stage variable logic.

Sort the data on your key field, then use a transform stage that stores the last record key value in a stage variable and use the stage constraints to output records accordingly

[quote="Ush"]Hi

I have the following set of records:

Empno name
1 Ash
1 Ush
2 Reeta
3 x
4 Y

I have retain first record and capture rejected records...I cant use remove duplicates since it does not have reject link.

Please help[/quote]


Also,

Help me in fetching only the Unique records out of the remove duplicate stage.

For eg,

If I have the following set of records:

Empno name
1 Ash
1 Ush
2 Reeta
3 x
4 Y

The result set out of remove duplicates should contain the following set of records:

Empno
1
2
3
4

Please help.

Nirmala84 - the same method of using stage variables applies.

Remove duplicate stage can retain first or last duplicate record.

here is a suggestion:
Use a sort stage..sort on EmpNo and make the clusterKeyChange value "True". Then put a constraint on your transformer where all records having clusterKeyChangeValue 1 should go in one dataset and the other to the other dataset. You will get 2 datasets..one with all unique employee nos and the other having all the duplicate records.