Remove bad rows before loading

sam334 · Post by **sam334** » Thu Mar 05, 2015 2:47 pm

All need a help in removing bad rows while loading.

We have a job which reads a pipe delimited file and load in DB2 database. Issue is that, sometimes in one or two records before pipe we are getting # sign. Datastage assuming it is a delimiter and aborting the job.

Can we do something to reject the # included rows whenever it comes, by which the job will not abort.

This is a source system issue but unfortunately can not do anything with it.

Thanks..

chulett · Post by **chulett** » Thu Mar 05, 2015 4:30 pm

Can you post the 'abort' you are getting? Don't understand why it would think a # is a delimiter so wondering if the actual abort text might help clear that up.

ray.wurlod · Post by **ray.wurlod** » Thu Mar 05, 2015 7:00 pm

Assuming you're using a Sequential File stage you could use a Filter command to drop or edit the # signs out of the data.

For example to bypass those records, use the grep command, while to alter them use the sed or awk command.