DSXchange

Hi,

We have a job reading the data from .CSV file(Contains 30 million records) through sequential file stage. When we are running this job we are getting the below warning for some records and those records are dropping.

seqSVCRegCustomers,0: Field "AFFIL_EMAIL_CNTCT_IND" with 'delim=end' did not consume entire input, at offset: 162
seqSVCRegCustomers,0: Import warning at record 732966.
seqSVCRegCustomers,0: Import unsuccessful at record 732966.

I want to eliminate these warnings and droppings.

Please let me know how we can do this.

Thanks

Your data and metadata need to match. You have rows which contain more data then your metadata allows for.

Thanks ArndW for your response.

I am taking the exact matching number of columns in metadata and file data.

Please let me know, is it because of wrong data in file hence I am getting the below warning.

Thanks ArndW for your response.

I am taking the exact matching number of columns in metadata and file data.

Please let me know, is it because of wrong data in file hence I am getting the below warning.

Your data is different in row 732966, I suggest you try

"head -732967 {SourceFile} | tail -3" to see one line before and one line after the incorrect one. The line length is short enough so that you should be able to detect the anomaly.

Hi Arndw,

I do not see any difference. below is the data please see.
"{CCF998D0-F60C-4211-8E4A-07C420BD64C1}",1,2006-11-09 13:45:31,"erinerinmc@hotmail.com","Erin McElroy","329 Spruce Street 1B","","Philadelphia","PA","19106","US"," "
,"1","1"," "
"{C300763E-EF7C-46F4-9A08-20989958AB67}",1,2005-01-05 06:44:10,"chiaradonna19@hotmail.com","Lauren Manze","631 Maryland Ave.","#5","Pittsburgh","PA","15232","US"," "
,"1"," "," "
"{98F48041-F9FA-4057-9763-E4390727FF13}",2,2005-01-11 09:24:39.780000000,"","Diane McKenzie","""Little Wood"" RR#1","","Hillsburgh","ON","N0B1Z0","CA"," "," ","1","

a) could you randomize that data a bit by editing your post and making the names and email addresses illegible?
b) did you cut off the last double quote by mistake?
c) which column is "AFFIL_EMAIL_CNTCT_IND"?

Search the forum for discussion about parallel jobs not being able to handle the conventional "" meaning a single double-quote character within a double-quote quoted string. The string "Little Wood" in your data fits into this category.