Page 1 of 1
how to seperate duplicates and unique records from seq file
Posted: Tue May 04, 2010 9:09 am
by pvijaybe
Hi All
I have a sequential file consists of records like (1,2,2,3,4,4,5,6,7)
I require ttwo outputs one should contain(2,2,4,4) and other should contain(1,3,5,6,7)
How can i do this by using which stage?
Can u pls guide me?
Thanks and regards
Vijay
Posted: Tue May 04, 2010 10:02 am
by relati
Use transformer stage, route duplicates into one stream and others into another stream. You can do this by checking if the current value is the same as previous value or not with the help of stage variable(s).
Posted: Tue May 04, 2010 4:41 pm
by ray.wurlod
The second person personal pronoun in English is spelled "you" - not "u". U is the name of one of our posters.
Please strive for a professional standard of written English on DSXchange - it makes life easier for those whose first language is not English (like U, for example, who is Burmese nationality working in Singapore).
Posted: Tue May 04, 2010 4:44 pm
by ray.wurlod
I would use a fork-join design with an Aggregator stage performing a count. Downstream of the Join stage a Filter, Switch or Transformer stage can direct rows with a count of 1 to one stream and other rows to the other stream.