Unable to view all records from sequential file

ray.wurlod · Post by **ray.wurlod** » Wed Dec 07, 2011 12:16 am

Do the terminal $ characters have any function, such as "continuation character"? Does your metadata take these into account?

mac4rfree85 · Post by **mac4rfree85** » Wed Dec 07, 2011 12:36 am

actually $ character came only when i did cat -vte <filename>. When i am opening the file with vi or cat o head, am unable to see it. Also, one more thing is the last column data is truncated... even if i give varchar(100) even though the column is having only 30 characters max, i am unable to see the data.

mac4rfree85 · Post by **mac4rfree85** » Wed Dec 07, 2011 12:46 am

Ray, i made some progress. I copied the contents (only) from windows and created the file in Unix itself. I also changed Record delimiter as UNIX Newline and Field delimiter as none. Now i am able to process 988 records and rest are going as rejected. I have captured them and this is how it appears.

Code: Select all

{20 2 0 1 1 - 0 5 - 0 3 H H R 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 R E T A I L 20 U S 20 20 20 20 20 20 20 20 20 7 C H E V R O L E T 20 T R U C K S 20 20 20 20 20 20 20 20 20 20 V A N / M P V}

zulfi123786 · Post by **zulfi123786** » Wed Dec 07, 2011 3:45 am

use the unix "od" command to perform complete diagnosis of the file, i guess it will give you a better insight of where the file records are getting messed up

ray.wurlod · Post by **ray.wurlod** » Wed Dec 07, 2011 2:58 pm

That looks like reasonable data, though maybe the final field is truncated as you say. Can you please provide the record schema that describes the record structure, and one more example of a failed row?

qt_ky · Post by **qt_ky** » Fri Dec 09, 2011 9:32 am

Just to clear things up a bit, there should not be any actual $ sign at the end of each line in your file. The -vte options on the cat command are to force the display of some nonprinting characters as visible characters, like a $ for the end of each line and ^I for tabs, etc. It's similar in purpose to opening the file in vi or using the od -x commands on the file.

How is your metadata defined, as several have asked. What is in your Columns tab? Is it a single column of type VarChar(1000) or what?

What are all the options you have set now on the Format tab?

Are you getting any error now or still getting some records rejected?