Remove Duplicates
Moderators: chulett, rschirm, roy
Remove Duplicates
Hi,
I want to remove the duplicate data on the DATE column data.
The date format is DDMMYYYY.
Using Datastage Server Edition (7.5) , source is Flat file.
Any one have the solution for this?
Regards,
Arshi
I want to remove the duplicate data on the DATE column data.
The date format is DDMMYYYY.
Using Datastage Server Edition (7.5) , source is Flat file.
Any one have the solution for this?
Regards,
Arshi
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
hi ,arshi wrote:Hi,
First sort the data by using the Sort stage and use the stage variables to remove the duplicates.
By using the sort stage , It will work fine for a particular month data but not on entire data.
Any one have the solution for this.
Regards,
Arshi.
do one thing in the sort stage options tab just specify the options
ALLOW DUPLICATES IS TRUE ,no need to use aditionally remove duplicates stage.make sure that select the execution mode is sequential.it shoud works fine.
D.N .MURTHY
Hi Murthy,
I didnot found any options tab in the sort stage . I am using the server edition (7.5) . Can you explain where it is exactly?
Hi Sainath,
As per my requirement I have to sort column1 and column2. Here, column2 having the date data (DDMMYYYY).
If i use the sort stage its not giving the correct result.I think you understand my requirement.
Regards,
Arshi
I didnot found any options tab in the sort stage . I am using the server edition (7.5) . Can you explain where it is exactly?
Hi Sainath,
As per my requirement I have to sort column1 and column2. Here, column2 having the date data (DDMMYYYY).
If i use the sort stage its not giving the correct result.I think you understand my requirement.
Regards,
Arshi
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
-
- Premium Member
- Posts: 425
- Joined: Sat Nov 19, 2005 9:26 am
- Location: New York City
- Contact:
Hi Archi,
Before sorting the data add an extra column where you want to have the date formatted as YYYYMMDD and use it as your sort variable, use the stage variables to remove the duplicates.
It will work fine for on entire data.
Before sorting the data add an extra column where you want to have the date formatted as YYYYMMDD and use it as your sort variable, use the stage variables to remove the duplicates.
It will work fine for on entire data.
Julio Rodriguez
ETL Developer by choice
"Sure we have lots of reasons for being rude - But no excuses
ETL Developer by choice
"Sure we have lots of reasons for being rude - But no excuses
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: