Page 1 of 1

capture duplicate

Posted: Thu Sep 24, 2009 12:09 am
by rajee1234
Hi,

I need to capture the duplicate record to the seperate table......only by using transformer stage

Thanks in advance

Using sortstage. capture the duplecate records.

Posted: Thu Sep 24, 2009 12:41 am
by jammala.ds
Using sortstage. capture the duplecate records.

Thanks
jk

Posted: Thu Sep 24, 2009 1:07 am
by rajee1234
My requirement is need to use only transformer stage..... for Capturing duplicate records........

can u help me with some logic

Thanks in advance

Posted: Thu Sep 24, 2009 2:37 am
by Sainath.Srinivasan
You cannot capture duplicates using transformer alone.

Did you check with your interviewer and ask them for a solution ? :roll:

Posted: Thu Sep 24, 2009 3:22 am
by rajee1234
we need to apply some logic in transformer stage using stage variable... i am not sure with the logic used.....

source file target 1 target 2
102 100 102
106 102 106
108 106
102 108
106
100

can any one tell me some logic for this....

Posted: Thu Sep 24, 2009 4:43 am
by nbelwadk
we can do this by using key change column function in sort stage followed by a lookup stage.

Posted: Thu Sep 24, 2009 5:26 am
by chulett
nbelwadk wrote:y dont u use remove duplicate stage nd dn a compare stage.
aneways transformer degrades the performance
:!: Two points, last one first:

Transformer performance has not been an issue for quite some time, that was addressed years ago, so your advice while good intentioned is out of date.

Please use a professional level of English here, this isn't your cell phone so take an extra second to spell out your words and check for typos. Not everyone's language of choice here is English, please keep that in mind.

Posted: Thu Sep 24, 2009 5:29 am
by chulett
I can't tell from your example what the "duplicates" would be. General advice - sort by the grouping key(s) then check for duplicates using stage variables. First time you see a 'group' is ok, if you see the same one again, not so ok and you should be able to discard or capture them fairly easily via a constraint.

Posted: Thu Sep 24, 2009 5:36 pm
by ray.wurlod
You have marked the job type as TX. Please verify whether this is correct. From your terminology I doubt that it is.

Posted: Thu Sep 24, 2009 5:46 pm
by chulett
I too assumed that was not really the case.