DSXchange

I have a datastage job that loads a csv file into an Oracle Database. The csv file has some disclosure text at the bottom which prevents the file from being loaded. Is there a way I can filter out this unwanted text while loading the data.

Thank you

Define unwanted data? What is the error message that you are getting?

The error message is required column is missing. The reason I am getting this is because I have mapped the sequential file columns with the columns in the oracle table.

In the CSV file I have **DISCLOSURE** text in one of the columns at the end of file and nothing in the other columns. I want to filter these rows.

Thank you

DSguru2B wrote:Define unwanted data? What is the error message that you are getting?

In the sequential file stage, activate filter command and put the following in the filter command box This will delete the last line before the stage even starts reading it.

Thank you. Another question I have is how would I delete all the lines below the Disclosure text.

DSguru2B wrote:In the sequential file stage, activate filter command and put the following in the filter command box

Code: Select all

sed '$d'

This will delete the last line before the stage even starts reading it.

With a more appropriate sed or awk script, of course. Might be time to invest some effort in learning these tools.

I believe the following should work. Test it out before you stick to it.

Thank you. I will try it.

DSguru2B wrote:I believe the following should work. Test it out before you stick to it.

Code: Select all

sed '/**DISCLOSURE**/;/^*$/d'