Getting the warning and droping those records

Sridhar Sivakoti · Post by **Sridhar Sivakoti** » Fri Mar 21, 2008 3:43 am

Hi,

We have a job reading the data from .CSV file(Contains 30 million records) through sequential file stage. When we are running this job we are getting the below warning for some records and those records are dropping.

seqSVCRegCustomers,0: Field "AFFIL_EMAIL_CNTCT_IND" with 'delim=end' did not consume entire input, at offset: 162
seqSVCRegCustomers,0: Import warning at record 732966.
seqSVCRegCustomers,0: Import unsuccessful at record 732966.

I want to eliminate these warnings and droppings.

Please let me know how we can do this.

Thanks

ArndW · Post by **ArndW** » Fri Mar 21, 2008 3:45 am

Your data and metadata need to match. You have rows which contain more data then your metadata allows for.

Sridhar Sivakoti · Post by **Sridhar Sivakoti** » Fri Mar 21, 2008 3:56 am

Thanks ArndW for your response.

I am taking the exact matching number of columns in metadata and file data.

Please let me know, is it because of wrong data in file hence I am getting the below warning.

Sridhar Sivakoti · Post by **Sridhar Sivakoti** » Fri Mar 21, 2008 3:57 am

Thanks ArndW for your response.

I am taking the exact matching number of columns in metadata and file data.

Please let me know, is it because of wrong data in file hence I am getting the below warning.

ArndW · Post by **ArndW** » Fri Mar 21, 2008 3:57 am

Your data is different in row 732966, I suggest you try

"head -732967 {SourceFile} | tail -3" to see one line before and one line after the incorrect one. The line length is short enough so that you should be able to detect the anomaly.

Sridhar Sivakoti · Post by **Sridhar Sivakoti** » Fri Mar 21, 2008 4:12 am

Hi Arndw,

I do not see any difference. below is the data please see.
"{CCF998D0-F60C-4211-8E4A-07C420BD64C1}",1,2006-11-09 13:45:31,"erinerinmc@hotmail.com","Erin McElroy","329 Spruce Street 1B","","Philadelphia","PA","19106","US"," "
,"1","1"," "
"{C300763E-EF7C-46F4-9A08-20989958AB67}",1,2005-01-05 06:44:10,"chiaradonna19@hotmail.com","Lauren Manze","631 Maryland Ave.","#5","Pittsburgh","PA","15232","US"," "
,"1"," "," "
"{98F48041-F9FA-4057-9763-E4390727FF13}",2,2005-01-11 09:24:39.780000000,"","Diane McKenzie","""Little Wood"" RR#1","","Hillsburgh","ON","N0B1Z0","CA"," "," ","1","

ArndW · Post by **ArndW** » Fri Mar 21, 2008 4:18 am

a) could you randomize that data a bit by editing your post and making the names and email addresses illegible?
b) did you cut off the last double quote by mistake?
c) which column is "AFFIL_EMAIL_CNTCT_IND"?

ray.wurlod · Post by **ray.wurlod** » Fri Mar 21, 2008 5:13 am

Search the forum for discussion about parallel jobs not being able to handle the conventional "" meaning a single double-quote character within a double-quote quoted string. The string "Little Wood" in your data fits into this category.