Box Characters while reading CSV files

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
rohithmuthyala
Participant
Posts: 57
Joined: Wed Oct 21, 2009 4:46 am
Location: India

Box Characters while reading CSV files

Post by rohithmuthyala »

Hi All,

Iam moving a CSV file from Windows to UNIX, then reading the CSV file in one of my job and when trying to view data in dataset i'm able to see a box character appended to the value of a column which has a datatype of VARCHAR(8).

The box character is not appended for the values if the string is of length 8.

Eg: For the value "abcdefgh" there is no box character appended, whereas if the value is "abcdef" there is a box character appended.

Is there a way to handle these box characters in Datastage.
Rohith
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

Is that varchar column last one in your file ?

Check the record delimiter property in your input.
rohithmuthyala
Participant
Posts: 57
Joined: Wed Oct 21, 2009 4:46 am
Location: India

Post by rohithmuthyala »

Yes..... this column is the last one in my file.
Record level final delimiter is end.
Rohith
Schava
Participant
Posts: 1
Joined: Fri May 15, 2009 3:42 am

Post by Schava »

rohithmuthyala wrote:Yes..... this column is the last one in my file.
Record level final delimiter is end.
when you transfer file from Windows to Unix or in other way , "Ctrl M" Character will be appended to end of each line. That has to be removed fom the file, otherwise Datastage will not able to read the file.

thanks,
Sri
Thanks,
Sri C
surajkumar
Participant
Posts: 17
Joined: Wed Feb 06, 2008 5:09 am

Post by surajkumar »

How ur ftp file from windows to unix? ru using any tools to do this?

ur getting box like character because of ur ftp tool by default selecting
Binary mode

While doing ftp u should select transfer mode is text
SURAJKUMAR M
rohithmuthyala
Participant
Posts: 57
Joined: Wed Oct 21, 2009 4:46 am
Location: India

Post by rohithmuthyala »

I'm using Ipswitch S/W to transfer the file,by changing the mode of transfer the box character won't appear. But if you observe I've asked if there is any way these characters can be handled in Datastage?
Rohith
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Convert(CHAR(13),"",YourField)
-craig

"You can never have too many knives" -- Logan Nine Fingers
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

Set your record delimiter as "DOS type".
srinivas.g
Participant
Posts: 251
Joined: Mon Jun 09, 2008 5:52 am

Post by srinivas.g »

run below command before using the CSv file:

Dos2unix filename filename
Srinu Gadipudi
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

3rd solution (heh) - Transfer the file using FTP and select text mode instead of binary. Most ftp solutions automatically flip the end-of-line character if you select text mode.

Any of the above three solutions (sequential file - DOS delimiter, dos2unix or ftp text mode) will work.
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
rohithmuthyala
Participant
Posts: 57
Joined: Wed Oct 21, 2009 4:46 am
Location: India

Post by rohithmuthyala »

Thank you Craig ... It worked...!! :D
Rohith
Post Reply