why DS export to utf-8 became iso-8859-1

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

Post Reply
kurics40
Premium Member
Premium Member
Posts: 61
Joined: Wed Nov 18, 2009 10:01 am

why DS export to utf-8 became iso-8859-1

Post by kurics40 »

Hello All,

I need to export files to UTF-8. I set the NLS to UTF8 but in the end my export file code page became ISO-8859-1.

I could convert it from Linux with the

Code: Select all

iconv -f iso-8859-1 -t utf8 file >newfile
What could I make wrong that the UTF-8 code settings became overdefined?

Br,
Janos
kurics40
Premium Member
Premium Member
Posts: 61
Joined: Wed Nov 18, 2009 10:01 am

Re: why DS export to utf-8 became iso-8859-1

Post by kurics40 »

Hi All,

I figured out what is the reason to get ISO-8859-1 instead of UTF-8.

When I let the NLS in Project default(UTF8) then the linux command

Code: Select all

file --mime <filename>
said it is in unknown format.

When I set the Sequential File stage NLS to UTF8 I got ISO-8859-1.

When I changed the fields property one by one to UNICODE then I got finally UTF8.

So it is resolved.
kurics40
Premium Member
Premium Member
Posts: 61
Joined: Wed Nov 18, 2009 10:01 am

Re: why DS export to utf-8 became iso-8859-1

Post by kurics40 »

Almost true....

http://www-01.ibm.com/support/docview.w ... wg21588645
Message: sourcefile,0: Invalid character(s) ([xF1]) found converting string (code point(s): user Jorge Ya[xF1]ez) from codepage UTF-8 to Unicode, substituting. le, the job (or project) was setup to default to UTF-8 character set, but the file contents were ISO-8859-1 characters.

The characters x'ED' and x'F1' are valid characters (accented i and tilde characters) in the ISO-8859-1 character set but are undefined for UTF-8 character set which does not have single-byte characters in range x'80'-x'FF'.


Change Project NLS to ISO-8859-1 end the export part to UTF-8 and extended: Unicode.
Dont let the Porject default in UTF8..
Post Reply