data issues

[samsuf2002]

Hi All , i am running a parallel job using source as a sequential file with pipe | as a delimiter but the data coming is bad like there are some carriage returns in some columns and some columns has data which contain pipe therefore DS is taking it as a new column results in rejection of records. i just want to know can we get rid of these issues using DS jobs if yes then how?

any help will be appreciated
thanks

[ray.wurlod]

Are the text strings that contain delimiter and terminator characters quoted? If so, DataStage can manage them, if not it can not, so you will need "them" to create a file in a legal format, or pre-process it yourself.

[samsuf2002]

Thanks ray, ok i can create that but how the data stage will manage it can i please explain me

[ray.wurlod]

If you specify the quote character as a field level property, it will find the opening quote and scan characters into the field until the closing quote is found. Then it will resume its search for delimiter characters.

[rparimi]

I had some issues similar to this. This could be a LOW VALUE with ASCII value less than 32. you can write a condition to see if the input ASCII value is less than CHAR(32) but that is bad data. By the way are you getting this from the source table?

[samsuf2002]

my source is flat file delimited with pipe

[ray.wurlod]

You need to know what every byte in the stream is. The advice from rparimi is sound, and you need to be able to handle what ever is in the file. If that means forcing "them" to supply validly quoted character strings, then that's what it will take.