Filter Condition in sequential file
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 86
- Joined: Mon Nov 27, 2006 3:42 am
- Location: Wilmington
Filter Condition in sequential file
Hi,
My Source file is an Sequential file, Where i have duplicate values in my Sequential file.
I have select only unique records from my source.
I dont wanted to use any other stage's to filter the duplicates, Because we have Filter option in Sequential file.
By using the filter option can i do it ???
Please revert back with an clear unix commend to be used.
Regards
Magesh
My Source file is an Sequential file, Where i have duplicate values in my Sequential file.
I have select only unique records from my source.
I dont wanted to use any other stage's to filter the duplicates, Because we have Filter option in Sequential file.
By using the filter option can i do it ???
Please revert back with an clear unix commend to be used.
Regards
Magesh
Re: Filter Condition in sequential file
What, is that an order? and by the way, this is not a unix forum.Magesh_bala wrote:
Please revert back with an clear unix commend to be used.
Try : sort -u
It took me fifteen years to discover I had no talent for ETL, but I couldn't give it up because by that time I was too famous.
-
- Participant
- Posts: 86
- Joined: Mon Nov 27, 2006 3:42 am
- Location: Wilmington
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
It IS the right one.
The -u option for the sort command means "unique" - it returns unique values of the sort keys.
Therefore, if it is not giving you the answer you require, then you are not specifying the sort command appropriately.
The -u option for the sort command means "unique" - it returns unique values of the sort keys.
Therefore, if it is not giving you the answer you require, then you are not specifying the sort command appropriately.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 86
- Joined: Mon Nov 27, 2006 3:42 am
- Location: Wilmington
-
- Participant
- Posts: 86
- Joined: Mon Nov 27, 2006 3:42 am
- Location: Wilmington
Magesh_bala
before arguing please check the option.
Sort -u sorts and returns the unique records. Only thing is duplicate removal will be on the record basis not of the field baseis.
For example :
original file
abc xyz
abc xyz
xyz abc
lmn abc
lmn fgh
sort -u file will return
abc xyz
lmn abc
lmn fgh
xyz abc
If your requirement is different state it clearly.
before arguing please check the option.
Sort -u sorts and returns the unique records. Only thing is duplicate removal will be on the record basis not of the field baseis.
For example :
original file
abc xyz
abc xyz
xyz abc
lmn abc
lmn fgh
sort -u file will return
abc xyz
lmn abc
lmn fgh
xyz abc
If your requirement is different state it clearly.
Birendra
-
- Participant
- Posts: 86
- Joined: Mon Nov 27, 2006 3:42 am
- Location: Wilmington
-
- Participant
- Posts: 86
- Joined: Mon Nov 27, 2006 3:42 am
- Location: Wilmington
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
"It's not working" is singularly uninformative.
If you want us to help you need to specify the actual command(s) that you used, what you expected to get, what you actually got, and why you believe that they are different.
If you want us to help you need to specify the actual command(s) that you used, what you expected to get, what you actually got, and why you believe that they are different.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Premium Member
- Posts: 783
- Joined: Mon Jan 16, 2006 10:17 pm
- Location: Sydney, Australia
-
- Participant
- Posts: 86
- Joined: Mon Nov 27, 2006 3:42 am
- Location: Wilmington
Hi My Source File
11|Anil |99|MIG |POLIC|Motor
12|Guna |66|MI |OBJEC|Home
11|baba |23|MIG |POLIC|Motor
13|Ji |77|MM |cover|Tata
11|Anil |99|MIG |POLIC|Motor
14|qqq |77|QQ |Objec|MIG
22|aaa |99|aa |Gate |zz
I have used sort -u file_name.txt to eliminate duplicate records
But I am not getting the expected result
My output
11|Anil |99|MIG |POLIC|Motor
11|Anil |99|MIG |POLIC|Motor
11|baba |23|MIG |POLIC|Motor
12|Guna |66|MI |OBJEC|Home
13|Ji |77|MM |cover|Tata
14|qqq |77|QQ |Objec|MIG
22|aaa |99|aa |Gate |zz
Expected result
11|Anil |99|MIG |POLIC|Motor
11|baba |23|MIG |POLIC|Motor
12|Guna |66|MI |OBJEC|Home
13|Ji |77|MM |cover|Tata
14|qqq |77|QQ |Objec|MIG
22|aaa |99|aa |Gate |zz
help me!!! to resolve these problem.
11|Anil |99|MIG |POLIC|Motor
12|Guna |66|MI |OBJEC|Home
11|baba |23|MIG |POLIC|Motor
13|Ji |77|MM |cover|Tata
11|Anil |99|MIG |POLIC|Motor
14|qqq |77|QQ |Objec|MIG
22|aaa |99|aa |Gate |zz
I have used sort -u file_name.txt to eliminate duplicate records
But I am not getting the expected result
My output
11|Anil |99|MIG |POLIC|Motor
11|Anil |99|MIG |POLIC|Motor
11|baba |23|MIG |POLIC|Motor
12|Guna |66|MI |OBJEC|Home
13|Ji |77|MM |cover|Tata
14|qqq |77|QQ |Objec|MIG
22|aaa |99|aa |Gate |zz
Expected result
11|Anil |99|MIG |POLIC|Motor
11|baba |23|MIG |POLIC|Motor
12|Guna |66|MI |OBJEC|Home
13|Ji |77|MM |cover|Tata
14|qqq |77|QQ |Objec|MIG
22|aaa |99|aa |Gate |zz
help me!!! to resolve these problem.