National characters converted data from Baltic/ Poland

devesh_ssingh · Post by **devesh_ssingh** » Sat Mar 08, 2014 4:58 am

Hi,

any input are much appreciated!!!

Br,
Devesh

Atangell · Post by **Atangell** » Sat Mar 08, 2014 5:52 am

Hi Devesh,

Are you reading from a file or a database?
It is important to know the nls-settings from the source team, used to produce the source data so that it can be read correctly.

If your are reading from a database and using an odbc stage - the dsn in odbc.ini needs to be modified to have the correct codepage - for eg/- if source data is written in ms1252 - add the codepage - 2252 to read it.

devesh_ssingh · Post by **devesh_ssingh** » Mon Mar 24, 2014 6:59 am

Hi,

File is on mainframe server. I am ftping it to UNIX server by simple get command.
The job loads a simple Teradata stage table.
The NLS setting on source system level is set to NLS IBM1112 and on ETL ( Datastage and TD) server is ISO-8859-1 i.e UTF 8.
When i tried to change NLS ( from ISO-8859-1 to IBM1112 ) on job level it aborts by saying import has failed.

This is issue is still open. any more input is highly appreciated.

Thanks and regards,
Devesh singh

priyadarshikunal · Post by **priyadarshikunal** » Tue Mar 25, 2014 4:20 am

which codepage you tried for this? are you getting the same value when you write it to a file or just in database?

devesh_ssingh · Post by **devesh_ssingh** » Tue Mar 25, 2014 10:23 am

I have tried with IBM1112 Estonian char set.
is this somthing wrong with DS NLS setting. if we changed code page it should read it properly.

IBM Analytics Champion 2009 - 2020 · Post by **asorrell** » Tue Mar 25, 2014 12:39 pm

When you get it via FTP, is the transfer mode set to binary or text mode? It should be binary mode to prevent any changes.

chulett · Post by **chulett** » Fri May 23, 2014 9:58 am

It's been three months now... have you not engaged your official support provider to help with this?

devesh_ssingh · Post by **devesh_ssingh** » Sat May 24, 2014 5:05 am

yes and solotion reached till unix, DS loading is still issue.

quick input is much appreciated!!!

ArndW · Post by **ArndW** » Sat May 24, 2014 5:49 am

You've confirmed that the data is stored in UNIX in 8859-4. So you need to tell DataStage to use this codepage when reading the source file. After that, any character set conversions you do will work, since the data has been correctly read in.

devesh_ssingh · Post by **devesh_ssingh** » Sat May 24, 2014 6:36 pm

Hi Arndw,

i applied the NLS setting with same code page but same is not appearing while loading in table.

Is this something DS is not set to support that code page.
i have checked all the installed code page by iconv command and can see same 8859-4 in the list.
not sure how to proceed further.

Any more input?

Br,
Devesh

devesh_ssingh · Post by **devesh_ssingh** » Sat May 24, 2014 6:49 pm

one more observation.

TD connector is being used to load the data into target table.

the settting for character setting is done as below.

Client-character set -- UTF8

server character set --Unicode.

do we have option to include NLS setting to 8859-4 in connector property?

is it converting character while loading the file?

ArndW · Post by **ArndW** » Sun May 25, 2014 8:01 am

There are several possibilities where these characters are being interpreted incorrectly.

1. FTP from Mainframe to server.
2. sequential read into DataStage
3. Output from DataStage to Teradata

Doing a "view data", especially from within DataStage, is not going to help, as it introduces yet another source of errors.

You need to identify the row and column with the error, then make a copy of your job which uses the SEQ() function to give you the numeric value of the single character in question. First from UNX on the source file after FTP. If that one character is correct there, then run the DataStage job and do a peek of that value, if it is correct in the DataStage job then the error source is in the DataStage-Teradata interface.

devesh_ssingh · Post by **devesh_ssingh** » Mon May 26, 2014 8:52 am

content is hidden for paid user.

looks like you point to setting on TD.

devesh_ssingh · Post by **devesh_ssingh** » Mon May 26, 2014 9:29 am

I have removed the TD connector with seq file stage and run the job and can see the desried output. so it issue with TD connector setting.

next i have tried to set NLS setting ISO_8859_4:1988 in TD but non of character are getting inserted in Table.

devesh_ssingh · Post by **devesh_ssingh** » Tue May 27, 2014 5:05 am

I have checked the code support page on terdata client and found below are installed

Teradata character set Language IBM DB2 code set
Latin1_0A" English (Latin 1) ISO8859-1
Latin1252_0A" English (Win Latin) ISO8859-1/15
LATIN9_0A not sure
LATIN1252_3A0 not sure
LATIN1252_0A

there is no ISO8859-4 support on TD client ? can this be issue?
i am just guessing..