Datastage, Oracle and UTF-8

[Paul M]

For a client that has Siebel with Oracle 10g installed wih UTF-8 support, we have to extract data from Siebel into flat XML files. The proces abort after a while, we found out this is probably due to varchar string export that is UTF-8 encoded. The Datastage server software was installed with UTF-8 support, so I can't see what's happening. The datastage log file does not give any information, the job just aborts. I found messages in this forum about the same subject, I read something about the Oracle NLS_LANG setting, but I guess this relates to Windows client oriented software, not server jobs generating files on AIX.

Thanks in advance for any reply, gr. Paul Mulder, The Netherlands

[nisaumande]

The NLS_LANG setting is used by any Oracle client, Windows or Unix.

Do you have any error from Datastage ?
Are the NLS activated in Datastage ?

[ArndW]

...we found out this is probably due to varchar string export that is UTF-8 encoded... The datastage log file does not give any information, the job just aborts...

What leads you to believe that the problem is being caused by NLS and UTF-8 settings?

[ray.wurlod]

UTF-8 is not a single standard. To say that UTF-8 is a standard is like saying that UNIX is a standard. There are many eight bit Unicode Transformation Format schemes. Even the one used inside DataStage is idiosyncratic, as it preserves dynamic array delimiter markers as single-byte characters.

It's usually a matter of experimenting with the mechanisms and maps available with both sides of the problem (in your case Oracle and DataStage) to find a combination that works. For example, for Oracle you probably need both NLS_LANG and LANG_C environment variables to be set. Do you have NLS enabled for DataStage?