Read files with end-of-line character in its fields
Posted: Thu Mar 19, 2015 8:28 pm
Hello all.
I might be asking the same question as it has already been discussed a multiple times, but I did not find a proper answer.
I'm using a Datastage parallel job to load for a siebel identity S_COMMENTS, but one of its field , COMMENT is a free flowing text field that can have end of line character within itself, which is legitimate.
A sample structure of the data from source for this identity is given below.
As shown, for the second data line, the data flows over to the next line as the COMMENT field has an end of line character in it, but this is not an error as this field is a free flowing text field both at source and target systems
Please advise me on how ton handle this scenario in a parallel job.(I'm not conversant with server jobs)
My idea: create a pre job sub routine, and parse the file using a sed/awk command to replace the in-between EOL characters with a fixed character of my choice, and revert back in the downstream stages.
Any better ideas will be highly appreciated.
Thanks.
Warm Regards,
Kumarjit.
I might be asking the same question as it has already been discussed a multiple times, but I did not find a proper answer.
I'm using a Datastage parallel job to load for a siebel identity S_COMMENTS, but one of its field , COMMENT is a free flowing text field that can have end of line character within itself, which is legitimate.
A sample structure of the data from source for this identity is given below.
Code: Select all
USER_ID|COMMENT|FROM_DATE
1-10001|Test Comment line1|2015/02/21
1-10002|Test Comment
line2|2015/02/26
1-10003|Test Comment line3|2015/03/17
Please advise me on how ton handle this scenario in a parallel job.(I'm not conversant with server jobs)
My idea: create a pre job sub routine, and parse the file using a sed/awk command to replace the in-between EOL characters with a fixed character of my choice, and revert back in the downstream stages.
Any better ideas will be highly appreciated.
Thanks.
Warm Regards,
Kumarjit.