DSXchange

Posted: **Fri Feb 21, 2014 2:40 pm**

Hi Experts,

We are reading .csv file which has multiple new-line character in multiple columns. I tried different options to read these records, but still the records are getting rejected.
Below are the option I tired.. but no luck

In the Format tab

In the Record level
1. changed the final delimiter to '\n', end, none,null
2. Add the record delimiter string = Dos format

In the Field defaults

1. Add the delimiter string chr(10), char(13)
2. Removed double quotes add it back

Our data file is :

Code: Select all

"ID" , "Comments", "Modified_date","Modified by"
"1", " THIS iS 
BIG COMMENT
AND BIG VALUE","2013-09-20T05:00:00Z" ," TESTER"                            --------> record 1
"2",

"<div><font face="Calibri" size="2" color="black">#nllsd	some big value
and 
more value

","2015-09-20T05:00:00Z" ," TESTER1"                       		    --------> record 2

Plese let me know which option shall I set the read the newline character from this file.

Appreciate all your help!!

Thanks in advance

Posted: **Fri Feb 21, 2014 4:23 pm**

The sequential file stage in parallel jobs has no tolerance for embedding the record delimiter in the data. In a parallel job, you could read the individual "short" records and piece them together until you arrive at a "whole" record. This of course means defining the input as a single unbounded varchar field.

The sequential file stage in a server job is more tolerant. You could consider a server shared container for reading your input file.

Mike

Posted: **Sat Feb 22, 2014 8:52 pm**

You may also consider a Unix based solution to fix the file, before processing it in DataStage.

See attached links :
http://www.unix.com/shell-programming-s ... -file.html
http://stackoverflow.com/questions/1483 ... ix-solaris
http://www.unix.com/hp-ux/147184-how-re ... -file.html

Posted: **Mon Feb 24, 2014 9:57 am**

Thanks for the help.

I am using the server job to laod this file, but it is giving me error for the some of the records which are having huge xml format. below is the error message

Code: Select all

Sequential_File_0.DSLink2: nls_map_buffer_in() - NLS mapping error, row 96338 (approx), row ="<div><font face="Calibri" size="2" color="black"> THIS is huge
comment </font></div>"

Values for the above records are inserted as null

DSXchange

Removing newline character while reading from sequential fil

Removing newline character while reading from sequential fil