Read files with end-of-line character in its fields

[kumarjit]

Hello all.

I might be asking the same question as it has already been discussed a multiple times, but I did not find a proper answer.

I'm using a Datastage parallel job to load for a siebel identity S_COMMENTS, but one of its field , COMMENT is a free flowing text field that can have end of line character within itself, which is legitimate.

A sample structure of the data from source for this identity is given below.

Code: Select all

USER_ID|COMMENT|FROM_DATE
1-10001|Test Comment line1|2015/02/21
1-10002|Test Comment
line2|2015/02/26
1-10003|Test Comment line3|2015/03/17

As shown, for the second data line, the data flows over to the next line as the COMMENT field has an end of line character in it, but this is not an error as this field is a free flowing text field both at source and target systems

Please advise me on how ton handle this scenario in a parallel job.(I'm not conversant with server jobs)

My idea: create a pre job sub routine, and parse the file using a sed/awk command to replace the in-between EOL characters with a fixed character of my choice, and revert back in the downstream stages.


Any better ideas will be highly appreciated.

Thanks.

Warm Regards,
Kumarjit.

[ray.wurlod]

Use a server Sequential Fille stage in a server shared container. This stage type can handled line terminators in text fields.

[vinothkumar]

Check whether this post helps you.

viewtopic.php?t=143239