UTF-8 to UTF-16 conversion in Datastage
Posted: Wed Dec 06, 2017 5:50 pm
Hi, I have searched on DSXchange for an answer but couldn't find one that worked. I did find a thread with the same issue. The suggested resolution to set the job level to the destination format did not work for me. So I am posting here.
The source is UTF-8 from Salesforce and the destination sequential file stage is UTF-16 or is intended to be encoded to UTF-16. Our data in Salesforce, in our database and our Datastage project is all set to UTF-8. So my goal is to convert the data to UTF-16 in the output file to the vendor.
UStringToString and StringToUstring are not working for the unicode to unicode conversion. I have since successfully tried the UNIX command ICONV and it works but I would like to learn a Datastage only solution, if one exists.
So just to clear, Salesforce.com UTF-8 -> Xfm (StringToUstring UTF-16) -> Sequential File (UTF-16). This is producing garbage data. If I change all that back to UTF-8, the data comes through good. So I assume the Datastage conversion functions are not working. I mean, I know they work for non-unicode to unicode because I have seen that work in the past. But unicode to unicode is not working properly. Any ideas?
Also, I should mention the field datatypes were set to NVarChar and also I tried VarChar Unicode.
Thanks, Matt
The source is UTF-8 from Salesforce and the destination sequential file stage is UTF-16 or is intended to be encoded to UTF-16. Our data in Salesforce, in our database and our Datastage project is all set to UTF-8. So my goal is to convert the data to UTF-16 in the output file to the vendor.
UStringToString and StringToUstring are not working for the unicode to unicode conversion. I have since successfully tried the UNIX command ICONV and it works but I would like to learn a Datastage only solution, if one exists.
So just to clear, Salesforce.com UTF-8 -> Xfm (StringToUstring UTF-16) -> Sequential File (UTF-16). This is producing garbage data. If I change all that back to UTF-8, the data comes through good. So I assume the Datastage conversion functions are not working. I mean, I know they work for non-unicode to unicode because I have seen that work in the past. But unicode to unicode is not working properly. Any ideas?
Also, I should mention the field datatypes were set to NVarChar and also I tried VarChar Unicode.
Thanks, Matt