how remove duplicate in transformer

[sureshbabu]

hi

can any body sugg me how do romoveduplecate in transformar stage useing stage varible


regards
sure

[ArndW]

2 Stage Variables:

Code: Select all

svDuplicate          IF In.Key=svLastKey THEN 1 ELSE 0
svLastKey          In.Key

constraint "svDuplicate=0"

[chulett]

Or, exactly the same but my preference would be:

Code: Select all

svDuplicate        In.Key=svLastKey
svLastKey          In.Key

Constraint "Not(svDuplicate)"

[ArndW]

Craig, I agree with you - I would program it that way but expanded the logic to make the statement more understandable.

[mk_ds09]

You can use the following logic to remove duplicates using the Transformer Stage, however for that you need to have data sorted.

Before using the transformer Stage, use Sort Stage to sort the data.

In Transformer Stage, take the

FirstRecordVar=SecondRecordVar
IncomingLnk.Value=FirstRecordVar

Then in constraint you can put constraint like

FirstRecordVar<>SecondRecordVar

This will ensure that duplicates records are not passed to the next stage.

Hope this helps !

[chulett]

Craig, I agree with you - I would program it that way but expanded the logic to make the statement more understandable.

Of course and I figured as much... just couldn't resist the perfect setup.

[agpt]

You can use the following logic to remove duplicates using the Transformer Stage, however for that you need to have data sorted.

Before using the transformer Stage, use Sort Stage to sort the data.

In Transformer Stage, take the

FirstRecordVar=SecondRecordVar
IncomingLnk.Value=FirstRecordVar

Then in constraint you can put constraint like

FirstRecordVar<>SecondRecordVar

This will ensure that duplicates records are not passed to the next stage.

Hope this helps !

Hi Sorry to interrupt but how do you achieve :

FirstRecordVar=SecondRecordVar
?

What exact syntax is used for the same?

[galmeida]

hi

can any body sugg me how do romoveduplecate in transformar stage useing stage varible


regards
sure

Why not just use the de-duplicate stage?