Polish Charecter Nls Support

yalamanchili · Post by **yalamanchili** » Fri May 26, 2006 6:21 am

Hi Friends,

I am facing a real problem While loading polish data using Data Stage . i am using Server jobs and teh server is unix OS. The oracle data base is using AL32UTF8 settings. Coming to Data stage i am using Server locale as pl_polish but there was no server maps available to support the polsih language.

i am getting flat files from the client in utf8 format. when i am loading data directly from the file to oracle data base using data stage, i am not getting special charecters. But when i am fetching data from one instance of data base and loading it to another instance, i am getting special charecters like ? and | in data base.

while i am viewing data from a sequential file in data stage it is showing special charecters.

Please any one help me to sort out this problem.

i am waiting for your replies.

ashwin141 · Post by **ashwin141** » Fri May 26, 2006 6:49 am

Hi

We faced a similar kind of problem while processing data for latin american countries. The solution which we implemented was using a job before we start the processing for converting UTF8 to ASCII and a job at the end before loading the data which converts this ASCII to UTF8 again. This helped us in avoiding the changes in all the intermediate jobs.

I hope it helps.

Regards
Ashwin

yalamanchili · Post by **yalamanchili** » Sat May 27, 2006 4:24 am

Hi ashwin,

i have tried with what you said,but it is not working in my case.

Hi anyone please help me find a way out of this problem.

Waiting for your responses.

chulett · Post by **chulett** » Sat May 27, 2006 6:08 am

How did you try what they said? How is it not working?

Details, man! If people don't know what exactly you've tried and in what way it didn't work for you, how can you expect them to help? Besides, it can save us all from alot of pointless posts...

"Did you try X?

"Yes."

"And did it work for you?"

"No".

"Ok, did you try Y"?

"Yes, it did not work."

"I hope that Z will help you."

"It does not appear to be working either."

...days pass...

"Anyone?"

Ashwin, it would probably help if you could provide details of your solution as well. It may help clarify things for Yalamanchili or give them some other ideas on approaches to try.

ray.wurlod · Post by **ray.wurlod** » Sat May 27, 2006 3:47 pm

Maps do not support languages, they support character sets. Polish should be able to be achieved entirely within ISO8859; check the NLS database (how to do this is in the NLS manual) to find out which subset of this you need to include the accented characters. Polish characters ought, if my memory serves, to be able to be handled in the "Extended European" character set - ISO8859-x where x is the appropriate number.

I don't have access to NLS-enabled DataStage at the moment, so will have to ask you to experiment. Also search the forum for the correct settings from the Oracle environment variables (such as NLS_LANG).

yalamanchili · Post by **yalamanchili** » Mon May 29, 2006 12:52 am

Hi Guys,

Here i am giving a little bit clarification to what i have tried.

The input file: Unicode format(utf8)
while opening the file r u able to see the special charecters

"Yes"

How did you convert the file from utf8 to ascii

"By Data Stage one to one mapping "

when you converted the file form utf8 to ascii using data stage r u able to see polish charecters.

"No", when i open the file i am able to see ? symbols.

while loading from file to database using ODBC stage r u able to see the data without ?.

"Yes", the data looks something like english charecters without any ? and vertical tabs.

So where do you face the Problem

When i fetch the same data from loaded table and load it to another table.

what problem r u facing

I am able to see Vertical tabs and ? symbols in the data.
I have explained my problem to what i can do.if i miss any important points plese let me know. since i am new to this tool i hope you will help me sort out this problem.

chulett wrote:How did you try what they said? How is it not working?

Details, man! If people don't know what exactly you've tried and in what way it didn't work for you, how can you expect them to help? Besides, it can save us all from alot of pointless posts...

"Did you try X?

"Yes."

"And did it work for you?"

"No".

"Ok, did you try Y"?

"Yes, it did not work."

"I hope that Z will help you."

"It does not appear to be working either."

...days pass...

"Anyone?"

Ashwin, it would probably help if you could provide details of your solution as well. It may help clarify things for Yalamanchili or give them some other ideas on approaches to try.

raoraghunandan · Post by **raoraghunandan** » Tue May 30, 2006 4:18 am

Hi,

Can you copy and paste here..a polish character that you are not able to see properly ?

Thanks,
Raghu

yalamanchili wrote:Hi Guys,

Here i am giving a little bit clarification to what i have tried.

The input file: Unicode format(utf8)
while opening the file r u able to see the special charecters

"Yes"

How did you convert the file from utf8 to ascii

"By Data Stage one to one mapping "

when you converted the file form utf8 to ascii using data stage r u able to see polish charecters.

"No", when i open the file i am able to see ? symbols.

while loading from file to database using ODBC stage r u able to see the data without ?.

"Yes", the data looks something like english charecters without any ? and vertical tabs.

So where do you face the Problem

When i fetch the same data from loaded table and load it to another table.

what problem r u facing

I am able to see Vertical tabs and ? symbols in the data.
I have explained my problem to what i can do.if i miss any important points plese let me know. since i am new to this tool i hope you will help me sort out this problem.

chulett wrote:How did you try what they said? How is it not working?

Details, man! If people don't know what exactly you've tried and in what way it didn't work for you, how can you expect them to help? Besides, it can save us all from alot of pointless posts...

"Did you try X?

"Yes."

"And did it work for you?"

"No".

"Ok, did you try Y"?

"Yes, it did not work."

"I hope that Z will help you."

"It does not appear to be working either."

...days pass...

"Anyone?"

Ashwin, it would probably help if you could provide details of your solution as well. It may help clarify things for Yalamanchili or give them some other ideas on approaches to try.

yalamanchili · Post by **yalamanchili** » Tue May 30, 2006 6:13 am

Hi All,

Here i am giving a Some Sample Polish Words

Please try this at your end let me know if it is working correctly.

The Words are
"Pielęgniarka",
"Proszę ",
"Jak się masz",
"Tęsknię za Tobą ",
"Nie mogę bez Ciebie żyć ",
"Wesołych Świąt Wielkanocnych! "

How to Load

Create a txt file with this words in utf8 format.

1st job in data stage

directly file to odbc load

2nd job in data stage

from one odbc to 2nd odbc

what to verify

Please see whether you are getting correct data in oracle without vertical tabs and ?

Please try this at your end, and if you come up with a solution please share with me.

Waiting for your responses.

If anything required,Please let me know.

raoraghunandan wrote:Hi,

Can you copy and paste here..a polish character that you are not able to see properly ?

Thanks,
Raghu

yalamanchili wrote:Hi Guys,

Here i am giving a little bit clarification to what i have tried.

The input file: Unicode format(utf8)
while opening the file r u able to see the special charecters

"Yes"

How did you convert the file from utf8 to ascii

"By Data Stage one to one mapping "

when you converted the file form utf8 to ascii using data stage r u able to see polish charecters.

"No", when i open the file i am able to see ? symbols.

while loading from file to database using ODBC stage r u able to see the data without ?.

"Yes", the data looks something like english charecters without any ? and vertical tabs.

So where do you face the Problem

When i fetch the same data from loaded table and load it to another table.

what problem r u facing

I am able to see Vertical tabs and ? symbols in the data.
I have explained my problem to what i can do.if i miss any important points plese let me know. since i am new to this tool i hope you will help me sort out this problem.

chulett wrote:How did you try what they said? How is it not working?

Details, man! If people don't know what exactly you've tried and in what way it didn't work for you, how can you expect them to help? Besides, it can save us all from alot of pointless posts...

"Did you try X?

"Yes."

"And did it work for you?"

"No".

"Ok, did you try Y"?

"Yes, it did not work."

"I hope that Z will help you."

"It does not appear to be working either."

...days pass...

"Anyone?"

Ashwin, it would probably help if you could provide details of your solution as well. It may help clarify things for Yalamanchili or give them some other ideas on approaches to try.

chulett · Post by **chulett** » Tue May 30, 2006 6:50 am

Why are we quoting entire messages? Are people not aware there is a simple Reply button at the bottom of the post? If you really need to quote someone, at least only quote the specific parts you are replying to, trim out all the other noise.

This isn't usenet or someplace where we may not have access to previous messages - they are all there. Thanks.

yalamanchili wrote:Hi All,

Here i am giving a Some Sample Polish Words

Please try this at your end let me know if it is working correctly.

The Words are
"Pielęgniarka",
"Proszę ",
"Jak się masz",
"Tęsknię za Tobą ",
"Nie mogę bez Ciebie żyć ",
"Wesołych Świąt Wielkanocnych! "

How to Load

Create a txt file with this words in utf8 format.

1st job in data stage

directly file to odbc load

2nd job in data stage

from one odbc to 2nd odbc

what to verify

Please see whether you are getting correct data in oracle without vertical tabs and ?

Please try this at your end, and if you come up with a solution please share with me.

Waiting for your responses.

If anything required,Please let me know.

raoraghunandan wrote:Hi,

Can you copy and paste here..a polish character that you are not able to see properly ?

Thanks,
Raghu

yalamanchili wrote:Hi Guys,

Here i am giving a little bit clarification to what i have tried.

The input file: Unicode format(utf8)
while opening the file r u able to see the special charecters

"Yes"

How did you convert the file from utf8 to ascii

"By Data Stage one to one mapping "

when you converted the file form utf8 to ascii using data stage r u able to see polish charecters.

"No", when i open the file i am able to see ? symbols.

while loading from file to database using ODBC stage r u able to see the data without ?.

"Yes", the data looks something like english charecters without any ? and vertical tabs.

So where do you face the Problem

When i fetch the same data from loaded table and load it to another table.

what problem r u facing

I am able to see Vertical tabs and ? symbols in the data.
I have explained my problem to what i can do.if i miss any important points plese let me know. since i am new to this tool i hope you will help me sort out this problem.

ray.wurlod · Post by **ray.wurlod** » Tue May 30, 2006 7:04 am

Can you create a job that uses Sequential File stages for both source and target? If that works successfully then it is clear that DataStage is handling the characters, and that the problem must be somewhere in the ODBC connection.

We await your response on that, rather than suggest strategies that may turn out to be inappropriate.

yalamanchili · Post by **yalamanchili** » Tue May 30, 2006 9:27 am

Hi ray.wurlod

Thank You for your advice. i ahve tried what you said.

what i have tried

writing from sequential file to sequential file

The data is coming neatly.

writing from odbc to sequential file

the data is written with some special symbols in the output file.

i hope the problem is coming when i am trying to read data from odbc.

Any advices regarding this.

yalamanchili · Post by **yalamanchili** » Wed Jun 07, 2006 2:34 am

hi Friends,

we have solved the polish character problem in data stage to a maximum extent. now we are facing problem only whlie reading data from either odbc driver or oci driver.

is there any nls setings i have to set at server end so that the data will be read by drivers properly. we are using Solaris server.

Please anyone help me in finding a proper solution.

waiting for your reply.

ray.wurlod · Post by **ray.wurlod** » Wed Jun 07, 2006 3:34 pm

The first, and most likely candidates, are NLS_LANG (for Oracle) and LANG_C (for Solaris). Check with your Oracle DBA for the appropriate value for NLS_LANG.

yalamanchili · Post by **yalamanchili** » Wed Jun 07, 2006 10:42 pm

Hi ray.wurlod,

i have changed oracle nls_lang setting on the client side, now i want to change the solaris settings. could you please tell me what settings i have to do, since i don't know unix much.

hai please assist me in this . waiting for your reply. i am using oci driver in data stage for connecting to data base

ray.wurlod · Post by **ray.wurlod** » Wed Jun 07, 2006 11:30 pm

NLS_LANG and LANG_C

Change them in dsenv as well as in your .profile script.

DSXchange