no it doesnt hold good for more than 2 same duplicate records...ray.wurlod wrote:The job design I posted will give you what you want. ...
to get duplicate records
Moderators: chulett, rschirm, roy
Abhay, your question itself is not very clear.
Do you want to capture all the records that occur more than once, and also the count of occurences? If yes, then try the following logic:
Use a copy stage to split the incoming records into two streams. One stream goes to an Aggregator stage that groups the records by key field(s) and counts the number of records in each group and outputs the results to the COUNT field. The output from Aggregator stage is then joined to the other stream using a Join stage on key field(s) and the results are then passed to Transformer stage. In Transformer, you could put a constraint like COUNT>2.
Do you want to capture all the records that occur more than once, and also the count of occurences? If yes, then try the following logic:
Use a copy stage to split the incoming records into two streams. One stream goes to an Aggregator stage that groups the records by key field(s) and counts the number of records in each group and outputs the results to the COUNT field. The output from Aggregator stage is then joined to the other stream using a Join stage on key field(s) and the results are then passed to Transformer stage. In Transformer, you could put a constraint like COUNT>2.
Nitin Jain | India
If everything seems to be going well, you have obviously overlooked something.
If everything seems to be going well, you have obviously overlooked something.
-
ray.wurlod
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Yes it does. Because every row from source appears on the left input of the Join stage.abhay10 wrote:no it doesnt hold good for more than 2 same duplicate records...ray.wurlod wrote:The job design I posted will give you what you want. ...
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.