There is no need to do this and it might cause some issues, particularly as I don't trust MS Notepad to do this correctly. Just read the file into into DataStage and specify that it is UTF-8 encoded.mayankrakesh wrote:..3. The txt file is being opened in Notepad and being saved as UTF-8 encoded file....
Using Latin collation on non-Latin data will result in very odd sort order. Latin collation is simple in that it takes the bytes of a string from the left and does a numeric greater-or-less-than comparison. This will only work correctly for Latin text, or texts or multibyte characters will not collate correctly when doing this.mayankrakesh wrote:...The collate being used for the database is Latin...
Try to output to a Unicode text file after doing your logic and see if the data is represented correctly. I would prefer using wordpad.exe over notepad.exe from experience (this is when looking at Japanese and Chinese text). If this works then output to your MS SQL and see if things are being represented correctly.
</a>
