Page 1 of 1

special characters handling

Posted: Mon Dec 28, 2009 8:10 pm
by dodda
Hello

I have an issue while processing data from a file. Iam using seq file stage as source and i need to produce fixed width file so i am using seq file stage as target too.The file is fixed width (310 byte size) and i have file layout which specifies each and every field length. While i was trying to process the file using the seq file stage i encountered some East european data which has special character é in one record which altered the record length to 311 bytes beacuse of which my job failed as i am doing some string to time stamp conversion on one of the field. In datastage admin the parallel locale has been set as UTF-8 and stage NLS-Map is Project Default (UTF-8). I tried with ISO-8859-1 also. We can expect such type of characters in future. I even tried by specifying unicode under the extended properties in the columns tab of the sequential file which i am using to read the input file.

Am i missing some thing. Please assit me in how to handle these type of special characters.

Posted: Tue Dec 29, 2009 12:46 pm
by dodda
can any one help me on this

Regards

Posted: Tue Dec 29, 2009 3:43 pm
by ray.wurlod
If they can they will post. Be patient. Many folks are on vacation at present.

Re: special characters handling

Posted: Tue Dec 29, 2009 5:51 pm
by nikhilanshuman
If these special characters do not contain any useful data,you can trim/convert them.

Try using convert("é","",inputlinkname.columnname)

Second option :

use left and ALNUM function.

Step 1: Use function Alnum(inputlinkname.columnname).This function returns 1 if the value contains characters which are not alphnumeric(i.e. if any special character is there).

Step2: If output of step1 is "1"the do following :

left(inputlinkname.columnname,len(inputlinkname.columnname)-2)

Here the above function will take all the characters from left except the last two characters.

Hence,the complete derivation will be :

if Alnum(inputlinkname.columnname)=1 then left(inputlinkname.columnname,len(inputlinkname.columnname)-2)
else inputlinkname.columnname

Posted: Wed Dec 30, 2009 1:40 am
by nani0907
hi,

give a try by taking datatype as Nvarchar without defining length.Hope it will work