Hi,
I have a scenario of loading records from a Sequential file into DB2-UDB Stage in server edition of DS.
I would like to know removing duplicate records completely along with the original one from a flat file while loading the DB2 UDB Table without using any temporary staging tables.
Also I would like to insert only rows that are not present in the target table of DB2-UDB Stage.I could not find any update action like "Insert New rows only" in the DB2-UDB Stage.
Appreciate your help.
Thanks
Krish.
Removing Duplicate records from a flat file
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 100
- Joined: Wed Feb 02, 2005 4:02 am
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
Sairamkrish,
you have two basic ways of doing this - either within the DataStage job or by using the DB2 database to filter out duplicates.
If you have a unique key and do an INSERT you will get warnings (which you can catch by using a reject link) when trying to update existing records, so this might work for you.
If you use DataStage you have several options; my favorite if the file is sorted (or you can do a sort) then a simple transform stage will let you remove duplicates.
you have two basic ways of doing this - either within the DataStage job or by using the DB2 database to filter out duplicates.
If you have a unique key and do an INSERT you will get warnings (which you can catch by using a reject link) when trying to update existing records, so this might work for you.
If you use DataStage you have several options; my favorite if the file is sorted (or you can do a sort) then a simple transform stage will let you remove duplicates.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 100
- Joined: Wed Feb 02, 2005 4:02 am
Hi all,
I dont need to update existing rows in the target Db2 table,I need to check the particular key column in the target table against the column in the source data file.
If it exists then dont update anything in the target table.
If it does not exist then insert the row into the target table.
Hope i have made things clear.
thanks
Krish
I dont need to update existing rows in the target Db2 table,I need to check the particular key column in the target table against the column in the source data file.
If it exists then dont update anything in the target table.
If it does not exist then insert the row into the target table.
Hope i have made things clear.
thanks
Krish
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom