DSXchange

Hi thanks for ur quick response...
but still i have one doubt that ..in that scenario if u have duplicate records it will work..as u explained in the example what happens to first record itself..the condition will not satisfy..right....for the second record it is ok...

please clarify me ....

thanks

anil kumar wrote:Hi thanks for ur quick response...
but still i have one doubt that ..in that scenario if u have duplicate records it will work..as u explained in the example what happens to first record itself..the condition will not satisfy..right....for the second record it is ok...

please clarify me ....

thanks

You can do it by using Hash stage or.. by loading into any database stage and then select distince.

Is it make sence...Let me know, if you have any questions

If u r sorting the records using the Sort stage,use the cluster key change feature to remove duplicates.
This works on the same logic as the StgVariables.

Hello Kumar

Where can difine the Sort command?


thanks in advance
RK

Anywhere you like, but typically in the Input Values field when using ExecSH as a before-job or before-stage subroutine.

Anil,

When you define the Stage variables assign them an initial value which wont appear in the input data

Yamini

thank you very much...

it is working fine.

I agree with this logic that Stage variable can be used to remove the duplicate files but stage variable stores the previous field value. so its better to have sort the data and club the duplicate recs together and then use the stage variable concept.