Page 1 of 1

how to eliminate duplicate records from source

Posted: Tue Oct 17, 2006 5:04 am
by ramsdanda
Hi ,

How to eliminate duplicate records from the source without using remove dulicate stage and aggregator stage.any body help me the same.

Thanks and regards
ramsdanda

Posted: Tue Oct 17, 2006 5:40 am
by DSguru2B
This smells like an interview question. If not, then give me more details, your source, is it a flat file or a database, why do you want to avoid using those stages? Any particular reason.

Re: how to eliminate duplicate records from source

Posted: Tue Oct 17, 2006 5:40 am
by balajisr
ramsdanda wrote:Hi ,

How to eliminate duplicate records from the source without using remove dulicate stage and aggregator stage.any body help me the same.

Thanks and regards
ramsdanda
What is your source? Is it sequential file or database?

Posted: Tue Oct 17, 2006 5:42 am
by ragunathan13
what is u'r source....

it's a sequential file or any other

Posted: Tue Oct 17, 2006 6:30 am
by kumar_s
Isn't too many same question at same time? :wink:
Ramsdanda - The reason been asked is, if you have Database as source stage, you can eliminate the duplicated while selecting by doing a group by.
You can also eliminate by using Sort stage, by assiging a key and Allow Duplicates option to false.

Posted: Tue Oct 17, 2006 6:40 am
by ArndW
DSguru2B wrote:This smells like an interview question.
I didn't notice that until I looked at all of his posts. Certainly taken together nothing but fishing for interview answers.

Posted: Tue Oct 17, 2006 6:57 am
by DSguru2B
I didnt either. I answered his first query. Then when i came to this one i sensed it. Then when i saw a third one i didnt bother commenting.