Page 1 of 1

Box Characters while reading CSV files

Posted: Fri Mar 26, 2010 3:15 am
by rohithmuthyala
Hi All,

Iam moving a CSV file from Windows to UNIX, then reading the CSV file in one of my job and when trying to view data in dataset i'm able to see a box character appended to the value of a column which has a datatype of VARCHAR(8).

The box character is not appended for the values if the string is of length 8.

Eg: For the value "abcdefgh" there is no box character appended, whereas if the value is "abcdef" there is a box character appended.

Is there a way to handle these box characters in Datastage.

Posted: Fri Mar 26, 2010 3:28 am
by Sainath.Srinivasan
Is that varchar column last one in your file ?

Check the record delimiter property in your input.

Posted: Fri Mar 26, 2010 3:36 am
by rohithmuthyala
Yes..... this column is the last one in my file.
Record level final delimiter is end.

Posted: Fri Mar 26, 2010 6:18 am
by Schava
rohithmuthyala wrote:Yes..... this column is the last one in my file.
Record level final delimiter is end.
when you transfer file from Windows to Unix or in other way , "Ctrl M" Character will be appended to end of each line. That has to be removed fom the file, otherwise Datastage will not able to read the file.

thanks,
Sri

Posted: Fri Mar 26, 2010 6:22 am
by surajkumar
How ur ftp file from windows to unix? ru using any tools to do this?

ur getting box like character because of ur ftp tool by default selecting
Binary mode

While doing ftp u should select transfer mode is text

Posted: Fri Mar 26, 2010 6:33 am
by rohithmuthyala
I'm using Ipswitch S/W to transfer the file,by changing the mode of transfer the box character won't appear. But if you observe I've asked if there is any way these characters can be handled in Datastage?

Posted: Fri Mar 26, 2010 6:43 am
by chulett
Convert(CHAR(13),"",YourField)

Posted: Fri Mar 26, 2010 8:18 am
by Sainath.Srinivasan
Set your record delimiter as "DOS type".

Posted: Fri Mar 26, 2010 8:24 am
by srinivas.g
run below command before using the CSv file:

Dos2unix filename filename

Posted: Fri Mar 26, 2010 8:58 am
by asorrell
3rd solution (heh) - Transfer the file using FTP and select text mode instead of binary. Most ftp solutions automatically flip the end-of-line character if you select text mode.

Any of the above three solutions (sequential file - DOS delimiter, dos2unix or ftp text mode) will work.

Posted: Mon Mar 29, 2010 12:53 am
by rohithmuthyala
Thank you Craig ... It worked...!! :D