Prblems with polish character

[Mike3000]

This is from DS manual:
"DataStage uses special maps for client server communication, with names ending in "-CS" (for "client server")".

One of those "-CS" mapping should be used for client/server
communication. Mapping should be changed on a server,
NOT a client

[soumbeet]

thanks for your input mike but the server has got other projects also catering to different geographies.

i would appreciate if you can be a bit descriptive on this like what exactly you want me to change/do with perspective to polish text.

[Mike3000]

Go to DS Administrator -> Projects ("select your project1") -> NLS
and change a mapping only for "your project1".
It won't affect the others.

[soumbeet]

I changed the NLS map for the project to UTF-8 in Administrator but the issue still persists.
Please update me if any other idea crosses your mind regarding the resolution of this issue.

[Mike3000]

I believe you are doing the right thing in a right place,
but try to play with different Code Pages.
Anything else, sorry i don't know.

[nisaumande]

You should also check your NLS_LANG setting for the Oracle client you're using with Datastage.

[soumbeet]

The NLS_LANG setting is not UTF-8 because it caters to data from other geographies.

Isn't there some way in Datastage by which we can load data into a target table in the desired character format irespective of the existing map.

Basically I am interested to know if there is anything which is possible within datastage.

[ray.wurlod]

DataStage with NLS enabled only uses its own, idiosyncratic encoding of Unicode internally, and therefore every interface to the "outside world" must have an appropriate character map. Typically the UNIX maps and the Windows map (for connection to the clients) will have different names for the same thing, for example ISO8859-1 on UNIX and MS1252 on Windows.

[chulett]

The problem that I am facing is, when I view data in datastage for the sequential file , the output is weird (it has special characters like ?).

This is specific to your NLS_LANG setting on your PC not DataStage. The DataStage setting will only be used when the job runs.

I'm not familiar with an NLS enabled DataStage install, but this is what I would do if it wasn't. Not sure if it will help or not. You need to tell Oracle what the character set is you are loading and - if it is different from the database character set - it will attempt to convert things during the operation. Note that not all characters from X will fit in Y so your mileage may vary.

Do this by setting a User Defined Environment Variable for NLS_LANG in the Administrator for the project. Set its value to $ENV which will ensure it stays at the currently set value. Then, add the parameter in your loading job and override the default value there to UTF-8, which would be AMERICAN_AMERICA.UTF8 I believe. Perhaps something else for you, check with your DBA. I've also found you need to do the same for LC_CTYPE and override it in the job to C.utf8 as well.

This will set these values for just this job and tell Oracle what it needs to do conversion-wise. As I said, I have no idea if this is proper for an NSL enabled installation, others will have to chime in on that. All I can say is it works for me - not specifically for Polish characters but others. Perhaps it will give you some ideas or help someone else.

You do need to verify that what you've got is actually UTF-8 encoded - I've received XML files saying they are UTF-8 which would not load properly, turned out they had characters outside of the UTF-8 character set in them. Specifically because they futzed with the file on a Windows PC before sending it to me.