Japanese .xlsx data handling in DatStage 8.7

naveed.zuber · Post by **naveed.zuber** » Wed Jul 16, 2014 4:36 am

Facing issue in reading Japanese data in .xlsx format.
Tried to convert .xlsx to .csv,csv file converting Japanese charaters to ?????

What are the options available read/process the .xlsx(Japanese) data in DatStage 8.7(DB2 and Unix)

Please suggest.

ray.wurlod · Post by **ray.wurlod** » Wed Jul 16, 2014 5:03 am

Welcome aboard.

There are no options for reading XLSX files on UNIX in DataStage 8.7.
You need first to export them to CSV or some other text format. And you will need to do this on a Windows machine set to use similarly coded Japanese characters.

In version 9.1.2 and later you might be able to use the Unstructured Data stage to read XLSX files directly.

naveed.zuber · Post by **naveed.zuber** » Thu Jul 17, 2014 1:14 am

Thanks Ray!

Converted XLSX to CSV and the CSV enabled to support Japanese charaters.
Now after moving CSV file from Windows to Unix,same CSV file showing Junk characters inplace of Japanese.
Tried with both Binay and ASCII mode of transfer from SSH Tectia File transfer.
Kindly suggest.

chulett · Post by **chulett** » Thu Jul 17, 2014 7:02 am

naveed.zuber wrote:Now after moving CSV file from Windows to Unix,same CSV file showing Junk characters inplace of Japanese.

Showing how / where / using what tool?

ray.wurlod · Post by **ray.wurlod** » Thu Jul 17, 2014 4:16 pm

How did you move the file to UNIX? If you used FTP, did you use BINARY mode?

chulett · Post by **chulett** » Thu Jul 17, 2014 5:41 pm

Not familiar with the tool used but they did say this:

naveed.zuber wrote:Tried with both Binay and ASCII mode of transfer from SSH Tectia File transfer.

chulett · Post by **chulett** » Fri Jul 18, 2014 7:59 am

You answered Ray's question but not mine which is about the viewing of the transferred data rather than the transfer itself.

ray.wurlod · Post by **ray.wurlod** » Sat Jul 19, 2014 3:36 pm

What is the language setting on the Windows machine where you're saving the CSV file, for this seems to be at the heart of the problem. Can you (or have someone else) save the CSV file on a Japanese version of Windows?

naveed.zuber · Post by **naveed.zuber** » Sun Jul 20, 2014 4:02 am

The CSV file created by someone else.
I have been updated that the CSV enabled for Japanese characters created under unicode.
Your previous post seems to be premium content which it couldn't see completely.
What are the options available now? Please suggest.

ray.wurlod · Post by **ray.wurlod** » Mon Jul 21, 2014 2:19 am

The apparent problem at source appears to have been resolved. You now need accurate knowledge about how the Japanese characters are encoded in the CSV file, and you must effect a binary transfer to guarantee that nothing in the file is changed during its transition to UNIX file system. This should also mean that the file has Windows line terminators, so you may need to allow for that in the Sequential File stage used to read the file.

ray.wurlod · Post by **ray.wurlod** » Thu Jul 24, 2014 11:15 pm

What are you using to view the contents of the DB2 table? Do not rely upon DataStage's data browser (View Data) - it is notoriously bad when the language is anything but Amurrican. Try some other tool - even the db2 command itself.