Hi,
We have a job reading the data from .CSV file(Contains 30 million records) through sequential file stage. When we are running this job we are getting the below warning for some records and those records are dropping.
seqSVCRegCustomers,0: Field "AFFIL_EMAIL_CNTCT_IND" with 'delim=end' did not consume entire input, at offset: 162
seqSVCRegCustomers,0: Import warning at record 732966.
seqSVCRegCustomers,0: Import unsuccessful at record 732966.
I want to eliminate these warnings and droppings.
Please let me know how we can do this.
Thanks
Getting the warning and droping those records
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 35
- Joined: Tue Feb 13, 2007 5:30 am
Your data and metadata need to match. You have rows which contain more data then your metadata allows for.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 35
- Joined: Tue Feb 13, 2007 5:30 am
-
- Participant
- Posts: 35
- Joined: Tue Feb 13, 2007 5:30 am
Your data is different in row 732966, I suggest you try
"head -732967 {SourceFile} | tail -3" to see one line before and one line after the incorrect one. The line length is short enough so that you should be able to detect the anomaly.
"head -732967 {SourceFile} | tail -3" to see one line before and one line after the incorrect one. The line length is short enough so that you should be able to detect the anomaly.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 35
- Joined: Tue Feb 13, 2007 5:30 am
Hi Arndw,
I do not see any difference. below is the data please see.
"{CCF998D0-F60C-4211-8E4A-07C420BD64C1}",1,2006-11-09 13:45:31,"erinerinmc@hotmail.com","Erin McElroy","329 Spruce Street 1B","","Philadelphia","PA","19106","US"," "
,"1","1"," "
"{C300763E-EF7C-46F4-9A08-20989958AB67}",1,2005-01-05 06:44:10,"chiaradonna19@hotmail.com","Lauren Manze","631 Maryland Ave.","#5","Pittsburgh","PA","15232","US"," "
,"1"," "," "
"{98F48041-F9FA-4057-9763-E4390727FF13}",2,2005-01-11 09:24:39.780000000,"","Diane McKenzie","""Little Wood"" RR#1","","Hillsburgh","ON","N0B1Z0","CA"," "," ","1","
I do not see any difference. below is the data please see.
"{CCF998D0-F60C-4211-8E4A-07C420BD64C1}",1,2006-11-09 13:45:31,"erinerinmc@hotmail.com","Erin McElroy","329 Spruce Street 1B","","Philadelphia","PA","19106","US"," "
,"1","1"," "
"{C300763E-EF7C-46F4-9A08-20989958AB67}",1,2005-01-05 06:44:10,"chiaradonna19@hotmail.com","Lauren Manze","631 Maryland Ave.","#5","Pittsburgh","PA","15232","US"," "
,"1"," "," "
"{98F48041-F9FA-4057-9763-E4390727FF13}",2,2005-01-11 09:24:39.780000000,"","Diane McKenzie","""Little Wood"" RR#1","","Hillsburgh","ON","N0B1Z0","CA"," "," ","1","
a) could you randomize that data a bit by editing your post and making the names and email addresses illegible?
b) did you cut off the last double quote by mistake?
c) which column is "AFFIL_EMAIL_CNTCT_IND"?
b) did you cut off the last double quote by mistake?
c) which column is "AFFIL_EMAIL_CNTCT_IND"?
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Search the forum for discussion about parallel jobs not being able to handle the conventional "" meaning a single double-quote character within a double-quote quoted string. The string "Little Wood" in your data fits into this category.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.