how to handle duplicate records in sequential file

ds_dwh · Post by **ds_dwh** » Mon Jun 14, 2010 5:51 am

hi,

how to handle duplicate records in sequential file

either deleting or sending to rej link

chulett · Post by **chulett** » Mon Jun 14, 2010 5:55 am

There are multiple ways to identify duplicate records regardless of source. What have you tried?

nagarjuna · Post by **nagarjuna** » Mon Jun 14, 2010 8:10 am

duplicates stage , sort stage , transformer stage ......

ds_dwh · Post by **ds_dwh** » Tue Jun 15, 2010 3:57 am

nagarjuna wrote:duplicates stage , sort stage , transformer stage ......

with using sequential file only

not using sort,remove duplicate,T/F

Sainath.Srinivasan · Post by **Sainath.Srinivasan** » Tue Jun 15, 2010 4:01 am

Is this an interview question ?

Maybe you can edit the file and manually remove them !!

kondeti · Post by **kondeti** » Tue Jun 15, 2010 4:19 am

There is an option available in Sequential File Stage called Filter. This option directly deals with Operating System where your DataStage server is Installed either Unix Flavour or Windows. Write respective script(either dos/Unix based on your Operating system) in the Filter option and ristrict duplicate records. Thank you.

chulett · Post by **chulett** » Tue Jun 15, 2010 6:27 am

ds_dwh wrote:
nagarjuna wrote:duplicates stage , sort stage , transformer stage ......
with using sequential file only

Why?

ray.wurlod · Post by **ray.wurlod** » Tue Jun 15, 2010 4:28 pm

Use Filter command in Sequential File stage. The command sort -u #filepath# might be a good choice.

DSXchange

how to handle duplicate records in sequential file

how to handle duplicate records in sequential file

with using sequential file only

Re: with using sequential file only