Reading error in sequentail stage

sagar deshmukh · Post by **sagar deshmukh** » Tue Jan 20, 2009 4:35 am

I have a records like this....
'99999','BB1','01/13/2009','08:38:16','000064529100110122500003','-38.50','-1'

I am using sequential stage to read it....

but sequentila stage is reading it as '?99999' why this is comming...?

I suppose to get only 99999 no quotes....

ray.wurlod · Post by **ray.wurlod** » Tue Jan 20, 2009 4:45 am

Make very sure that you have what you think you have, by viewing the file with something that also shows non-printing characters, such as od -c

Also make sure that your metadata are correct, in particular that the quote character is the single quote (by default it is the double quote).

sagar deshmukh · Post by **sagar deshmukh** » Tue Jan 20, 2009 4:48 am

Yes my meta data is correct only...

ray.wurlod · Post by **ray.wurlod** » Tue Jan 20, 2009 4:51 am

Make very sure that you have what you think you have, by viewing the file with something that also shows non-printing characters, such as the UNIX command od -c

sagar deshmukh · Post by **sagar deshmukh** » Tue Jan 20, 2009 4:58 am

Hi after checking od -c i got this ----->

Code: Select all

0000000 357 273 277   '   4   8   3   1   1   '   ,   '   B   B   2   '
0000020   ,   '   0   1   /   1   6   /   2   0   0   9   '   ,   '   1
0000040   6   :   3   3   :   1   5   '   ,   '   0   0   0   0   7   0
0000060   0   0   6   7   0   0   1   1   0   2   7   2   5   0   0   0
0000100   0   0   '   ,   '   1   .   0   0   '   ,   '   1   '  \n 


i should not get this...(357 273 277)

what to do to eliminate it?

Katie09 · Post by **Katie09** » Tue Jan 20, 2009 7:36 am

U also can try with unix command `cat -v` to display the special characters form your input file, if there are any, you need to manually remove those characters to make it in proper format..

hope this helps..

sagar deshmukh · Post by **sagar deshmukh** » Tue Jan 20, 2009 8:21 am

well is there any way to eliminate it without doing manual work....

Katie09 · Post by **Katie09** » Tue Jan 20, 2009 8:30 am

at least i am not aware of that if there is any such utiity..
as special charaters are different and it will be quite dificult to have such utility...

if you are having small number of records you can do it manually..

or is there are many records and then you have to check your input data source.. it should be clean i.e. without any special characters !

hope this helps !

chulett · Post by **chulett** » Tue Jan 20, 2009 8:54 am

You need to determine how they got there... created at the source? Problems during a transfer? Then ensure they are not in the file before you have to process it.

sagar deshmukh · Post by **sagar deshmukh** » Tue Jan 20, 2009 9:32 am

Actually i am importing file from remote location then....i am concatinating it with a available file.....
so the specila characters are comming....
not sure whether it is happening coz of transfer or coz of input sourse....

bt one thing which i observed....
when i am copy pasting the data into a new file, this character is not comming.....so it is related with file properties i guess....

chulett · Post by **chulett** » Tue Jan 20, 2009 9:51 am

As noted, you need to figure this out, we can't. Are the characters in this other 'available' file? Or in the source file after the transfer? Or only there after you cat the two together? How exactly are you "importing the file from remote location"? How exactly are you doing the concatenation? Details, man, details.

sagar deshmukh · Post by **sagar deshmukh** » Tue Jan 20, 2009 10:04 am

well through shell script i am importing the files.....
it never happened with any other file but only with this file it is happeing....
so i want to know whether it is the sourse file problem...
by using CAT i am concatinating the files,......in shell script...

also....is there any way by which we can avoid this...??

chulett · Post by **chulett** » Tue Jan 20, 2009 10:12 am

First you need to figure out what "this" is before you'll have a clue how to fix / avoid it. Use some basic detective skills. Check the file on the remote location before the transfer, are the characters there? Check after the transfer, are they there now? Etc. Pinpoint at what stage in the process they appear.

sagar deshmukh · Post by **sagar deshmukh** » Tue Jan 20, 2009 10:26 am

yes those special characters are available in remote file...
so is there no way to avoid it....
as that remote file will be comming from third party company

chulett · Post by **chulett** » Tue Jan 20, 2009 11:15 am

Then you need to have a chat with this third party company, see if they can provide you with a working file before you try to come up with ways to work around their... crap.