Page 1 of 1

how to remove "shift in/out characters"

Posted: Fri Oct 29, 2010 2:53 am
by yimwai
1st,I use PxCFF stage to read an EBCDIC file "F" and load data to table "T", then i read the same file again and use difference stage to compare the data from "T" and "F". ( only compare on column with Chinese characters.)
I find that result is "all different" which is supposed to be "all the same".
I guess that it is caused by shift in/out characters "0E 0F",right?
If right, why DS just discard them when reading the EBCDIC file?then when DS discard the characters?only when writing into the database?can I do the remove action and make the result "all the same"?

Posted: Fri Oct 29, 2010 3:04 am
by ArndW
The Shift-In and Shift-Out characters will be the same only if you ensure that you define the two stages (CFF and database) with the same NLS settings, otherwise you won't be able to compare the two streams.

Posted: Fri Oct 29, 2010 11:39 am
by ray.wurlod
Shift-In and Shift-Out are sometimes used to indicate a change of mapping (particularly in Japanese character sets). Beware that these might be important characters before arbitrarily discarding them. Once they're inside a job, they should be able to be discarded with a simple Convert() function.