hi guys,
i am really facing a problem , removing duplicates with sequential files .
my job is like this
Seqential_file1------------>Transformer-------------->Sequential_file2
(source)
plz any one give solutions..................
how can i remove duplicates from sequential file
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 160
- Joined: Tue May 02, 2006 7:49 am
As Kris Mentioned. Use the hashed file and from there again transfer the data into a Sequential file.
Also, recognize the duplicates carefully.
For eg:
If you want remove redundancy of the entire record, define all the coulmns as keys.
If you want to just remove duplicates based on a certain column, define that column as a key.
Regards,
Also, recognize the duplicates carefully.
For eg:
If you want remove redundancy of the entire record, define all the coulmns as keys.
If you want to just remove duplicates based on a certain column, define that column as a key.
Regards,
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
-
- Premium Member
- Posts: 1255
- Joined: Wed Feb 02, 2005 11:54 am
- Location: United States of America
Hi,
You can also achieve this in UNIX by using the "uniq" command in the before-stage-routine of the transformer. At this point, I'm not able to recall if there is any similar way in Windows.
Thanks,
Whale.
You can also achieve this in UNIX by using the "uniq" command in the before-stage-routine of the transformer. At this point, I'm not able to recall if there is any similar way in Windows.
Thanks,
Whale.
Anything that won't sell, I don't want to invent. Its sale is proof of utility, and utility is success.
Author: Thomas A. Edison 1847-1931, American Inventor, Entrepreneur, Founder of GE
Author: Thomas A. Edison 1847-1931, American Inventor, Entrepreneur, Founder of GE