Convert data from LATIN1 to UTF8
Moderators: chulett, rschirm, roy
Convert data from LATIN1 to UTF8
Hi,
Our requirement is to ocnvert data from LATIN1 to UTF8 format? How can it be implemented throgh DataSage?
Please let me know if we have any other way to approach this problem?
Thanks,
Nasimul
Our requirement is to ocnvert data from LATIN1 to UTF8 format? How can it be implemented throgh DataSage?
Please let me know if we have any other way to approach this problem?
Thanks,
Nasimul
Re: Convert data from LATIN1 to UTF8
Hi,
Is there any other way to convert LATIN1 format to UTF8?
Actually in datastage NLS is not enabled.
Please let me know any other solution.
Thanks,
Nasimul
Is there any other way to convert LATIN1 format to UTF8?
Actually in datastage NLS is not enabled.
Please let me know any other solution.
Thanks,
Nasimul
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
How do you expect to manipulate character sets if the tool for doing so is disabled?
The answer is no.
You can try using NVarChar as a data type but no guarantees are made. It may or may not work, when NLS is disabled.
The answer is no.
You can try using NVarChar as a data type but no guarantees are made. It may or may not work, when NLS is disabled.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
If you aren't using NLS for whatever reason, the safest way would be to extract the data from LATIN-1 source and dump the data to a sequential file on a UTF-8 box. I think this will create the file in the codepage format you want and then you could bulk load the file to DB2. It is a one-off fix, but I think it will work.
Bestest!
John Miceli
System Specialist, MCP, MCDBA
Berkley Technology Services
"Good Morning. This is God. I will be handling all your problems today. I will not need your help. So have a great day!"
John Miceli
System Specialist, MCP, MCDBA
Berkley Technology Services
"Good Morning. This is God. I will be handling all your problems today. I will not need your help. So have a great day!"
Re: Convert data from LATIN1 to UTF8
in the unix platform,there is some command you can invoke directly to transfer a latin1 file to uat-8 file
nasimul wrote:Hi,
Our requirement is to ocnvert data from LATIN1 to UTF8 format? How can it be implemented throgh DataSage?
Please let me know if we have any other way to approach this problem?
Thanks,
Nasimul
The only conversion command on UNIX that I know of is "dd", which can perform ASCII to EBCDIC and vice-versa.
If declare your source file as 8859 and output to UTF-8 and a "diff" command shows no difference then you have done something wrong at your settings since Datastage wil have done some conversion. I suspect your source stage settings aren't what you think.
If declare your source file as 8859 and output to UTF-8 and a "diff" command shows no difference then you have done something wrong at your settings since Datastage wil have done some conversion. I suspect your source stage settings aren't what you think.
Actually there is a command, I'd not heard of it until someone else mentioned it here some time ago. And I tend to forget about it because of the name - iconv.
![Wink :wink:](./images/smilies/icon_wink.gif)
man iconv wrote:iconv -f fromcode -t tocode [file ...]
iconv converts the encoding of characters in the input file from the fromcode code set to the tocode code set, and writes the results to standard output.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
Hi All,
Thanks for these unix tips !
But in our project we have NLS installed, so would like to do this conversion (Latin 1 to UTF-8) through Data Stage V 8.0.
We have set the job level NLS map to ISO-8859-1 and able to read Latin 1 chars properly.
Job looks like:
Source Seq file - > Transformer - > Target Seq file
Other properties are,
Default collation locale for stages : Project(OFF) at job level
NLS Locale at transformer : Project(OFF)
Target Seq file NLS : UTF-8.
But finally job is not converting the Latin 1 chars to UTF-8 at output file. All latin chars are passed as it's. Can you please provide suggestions to solve this ?
Thanks,
Ankita
Thanks for these unix tips !
But in our project we have NLS installed, so would like to do this conversion (Latin 1 to UTF-8) through Data Stage V 8.0.
We have set the job level NLS map to ISO-8859-1 and able to read Latin 1 chars properly.
Job looks like:
Source Seq file - > Transformer - > Target Seq file
Other properties are,
Default collation locale for stages : Project(OFF) at job level
NLS Locale at transformer : Project(OFF)
Target Seq file NLS : UTF-8.
But finally job is not converting the Latin 1 chars to UTF-8 at output file. All latin chars are passed as it's. Can you please provide suggestions to solve this ?
Thanks,
Ankita