how to eliminate duplicate records from source

ramsdanda · Post by **ramsdanda** » Tue Oct 17, 2006 5:04 am

Hi ,

How to eliminate duplicate records from the source without using remove dulicate stage and aggregator stage.any body help me the same.

Thanks and regards
ramsdanda

DSguru2B · Post by **DSguru2B** » Tue Oct 17, 2006 5:40 am

This smells like an interview question. If not, then give me more details, your source, is it a flat file or a database, why do you want to avoid using those stages? Any particular reason.

balajisr · Post by **balajisr** » Tue Oct 17, 2006 5:40 am

ramsdanda wrote:Hi ,

How to eliminate duplicate records from the source without using remove dulicate stage and aggregator stage.any body help me the same.

Thanks and regards
ramsdanda

What is your source? Is it sequential file or database?

ragunathan13 · Post by **ragunathan13** » Tue Oct 17, 2006 5:42 am

what is u'r source....

it's a sequential file or any other

kumar_s · Post by **kumar_s** » Tue Oct 17, 2006 6:30 am

Isn't too many same question at same time?

Ramsdanda - The reason been asked is, if you have Database as source stage, you can eliminate the duplicated while selecting by doing a group by.
You can also eliminate by using Sort stage, by assiging a key and Allow Duplicates option to false.

ArndW · Post by **ArndW** » Tue Oct 17, 2006 6:40 am

DSguru2B wrote:This smells like an interview question.

I didn't notice that until I looked at all of his posts. Certainly taken together nothing but fishing for interview answers.

DSguru2B · Post by **DSguru2B** » Tue Oct 17, 2006 6:57 am

I didnt either. I answered his first query. Then when i came to this one i sensed it. Then when i saw a third one i didnt bother commenting.

DSXchange

how to eliminate duplicate records from source

how to eliminate duplicate records from source

Re: how to eliminate duplicate records from source