National characters converted data from Baltic/ Poland
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 148
- Joined: Thu Apr 10, 2008 12:47 am
Hi Devesh,
Are you reading from a file or a database?
It is important to know the nls-settings from the source team, used to produce the source data so that it can be read correctly.
If your are reading from a database and using an odbc stage - the dsn in odbc.ini needs to be modified to have the correct codepage - for eg/- if source data is written in ms1252 - add the codepage - 2252 to read it.
Are you reading from a file or a database?
It is important to know the nls-settings from the source team, used to produce the source data so that it can be read correctly.
If your are reading from a database and using an odbc stage - the dsn in odbc.ini needs to be modified to have the correct codepage - for eg/- if source data is written in ms1252 - add the codepage - 2252 to read it.
Cheers
Ashwin!
Ashwin!
-
- Participant
- Posts: 148
- Joined: Thu Apr 10, 2008 12:47 am
Hi,
File is on mainframe server. I am ftping it to UNIX server by simple get command.
The job loads a simple Teradata stage table.
The NLS setting on source system level is set to NLS IBM1112 and on ETL ( Datastage and TD) server is ISO-8859-1 i.e UTF 8.
When i tried to change NLS ( from ISO-8859-1 to IBM1112 ) on job level it aborts by saying import has failed.
This is issue is still open. any more input is highly appreciated.
Thanks and regards,
Devesh singh
File is on mainframe server. I am ftping it to UNIX server by simple get command.
The job loads a simple Teradata stage table.
The NLS setting on source system level is set to NLS IBM1112 and on ETL ( Datastage and TD) server is ISO-8859-1 i.e UTF 8.
When i tried to change NLS ( from ISO-8859-1 to IBM1112 ) on job level it aborts by saying import has failed.
This is issue is still open. any more input is highly appreciated.
Thanks and regards,
Devesh singh
-
- Premium Member
- Posts: 1735
- Joined: Thu Mar 01, 2007 5:44 am
- Location: Troy, MI
-
- Participant
- Posts: 148
- Joined: Thu Apr 10, 2008 12:47 am
-
- Participant
- Posts: 148
- Joined: Thu Apr 10, 2008 12:47 am
-
- Participant
- Posts: 148
- Joined: Thu Apr 10, 2008 12:47 am
Hi Arndw,
i applied the NLS setting with same code page but same is not appearing while loading in table.
Is this something DS is not set to support that code page.
i have checked all the installed code page by iconv command and can see same 8859-4 in the list.
not sure how to proceed further.
Any more input?
Br,
Devesh
i applied the NLS setting with same code page but same is not appearing while loading in table.
Is this something DS is not set to support that code page.
i have checked all the installed code page by iconv command and can see same 8859-4 in the list.
not sure how to proceed further.
Any more input?
Br,
Devesh
-
- Participant
- Posts: 148
- Joined: Thu Apr 10, 2008 12:47 am
one more observation.
TD connector is being used to load the data into target table.
the settting for character setting is done as below.
Client-character set -- UTF8
server character set --Unicode.
do we have option to include NLS setting to 8859-4 in connector property?
is it converting character while loading the file?
TD connector is being used to load the data into target table.
the settting for character setting is done as below.
Client-character set -- UTF8
server character set --Unicode.
do we have option to include NLS setting to 8859-4 in connector property?
is it converting character while loading the file?
There are several possibilities where these characters are being interpreted incorrectly.
1. FTP from Mainframe to server.
2. sequential read into DataStage
3. Output from DataStage to Teradata
Doing a "view data", especially from within DataStage, is not going to help, as it introduces yet another source of errors.
You need to identify the row and column with the error, then make a copy of your job which uses the SEQ() function to give you the numeric value of the single character in question. First from UNX on the source file after FTP. If that one character is correct there, then run the DataStage job and do a peek of that value, if it is correct in the DataStage job then the error source is in the DataStage-Teradata interface.
1. FTP from Mainframe to server.
2. sequential read into DataStage
3. Output from DataStage to Teradata
Doing a "view data", especially from within DataStage, is not going to help, as it introduces yet another source of errors.
You need to identify the row and column with the error, then make a copy of your job which uses the SEQ() function to give you the numeric value of the single character in question. First from UNX on the source file after FTP. If that one character is correct there, then run the DataStage job and do a peek of that value, if it is correct in the DataStage job then the error source is in the DataStage-Teradata interface.
-
- Participant
- Posts: 148
- Joined: Thu Apr 10, 2008 12:47 am
-
- Participant
- Posts: 148
- Joined: Thu Apr 10, 2008 12:47 am
-
- Participant
- Posts: 148
- Joined: Thu Apr 10, 2008 12:47 am
I have checked the code support page on terdata client and found below are installed
Teradata character set Language IBM DB2 code set
Latin1_0A" English (Latin 1) ISO8859-1
Latin1252_0A" English (Win Latin) ISO8859-1/15
LATIN9_0A not sure
LATIN1252_3A0 not sure
LATIN1252_0A
there is no ISO8859-4 support on TD client ? can this be issue?
i am just guessing..
Teradata character set Language IBM DB2 code set
Latin1_0A" English (Latin 1) ISO8859-1
Latin1252_0A" English (Win Latin) ISO8859-1/15
LATIN9_0A not sure
LATIN1252_3A0 not sure
LATIN1252_0A
there is no ISO8859-4 support on TD client ? can this be issue?
i am just guessing..