Translation of '!' to '|'

[vrishabhsagar]

Hi,

I have this simple job where I pull data from TeraData source and load into a complex flat file. Now the job works flawlessly except when a '!" character is encountered in the source columns,

When incoming varchar source column contains '!' character; it seems that it automatically gets translated to '|' character. I viewed the output file in Mainframe system and see that the hex value has indeed '4F' instead of '5A'.

What is more interesting is that when I read the same file back into DataStage jobs and try to peek the value the '!' symbols are displayed correctly!!

Does anyone have any idea of why would this happen?

[bcarlson]

I'm sorry I don't have more info, but I think it is a difference in the EBCDIC to ASCII mapping between the mapping that DS uses to convert to EBCIDC and what the mainframe is expecting. See if you can determine the codepage that DS uses for EBCDIC and compare it to the codepage used on your mainframe.

I have a document somewhere in my 'very organized' files that talks about this, but my files are too organize for me to find it right now. If I can find it, I'll see if I can post it.

Brad.

[bcarlson]

Found the reference, in fact I posted in back in April:

EBCDIC ! and ] not translated properly

The IBM site is posted there towards the bottom. It shows that code pages 037 and 500 map 4F to '|' and '!', respectively, and to 5A to '!' and ']', respectively.

Hope this helps.

Brad.

[vrishabhsagar]

Found the reference, in fact I posted in back in April:

EBCDIC ! and ] not translated properly

The IBM site is posted there towards the bottom. It shows that code pages 037 and 500 map 4F to '|' and '!', respectively, and to 5A to '!' and ']', respectively.

Hope this helps.

Brad.

Brad, Thanks a lot for this reference. It really helped me understand the actual problem. Is there a live scenario workaround / solution available?

I am yet to confirm the codepage differences between my DS server and the Mainframe server (as I am @ home now), but can someone help me on how to change my DS codepage (translation code page??) if there is a difference between the two?

Also I believe (from what I saw on wikipedia EBCDIC page) that there exists more than one codepage for EBCDIC formats, so what happens when my target file moves from one type of mainframe to other?

Thanks a lot Guys!

[bcarlson]

I think we ended up manually changing the data on output in a buildop (a transform would work, too).

I believe there is a system parameter (a APT_?? variable?) that can be used to set the DataStage code page, but while we have talked about it, we have not done this so I have no specifics.

Brad.

[vrishabhsagar]

I think we ended up manually changing the data on output in a buildop (a transform would work, too).

I believe there is a system parameter (a APT_?? variable?) that can be used to set the DataStage code page, but while we have talked about it, we have not done this so I have no specifics.

Brad.

Hmm, did you scan each incoming col for these particular characters and then replaced each occurence of '!'??

I must be making a mistake in understanding this..... How to handle this in a Xf-TFM?

[vrishabhsagar]

I think we ended up manually changing the data on output in a buildop (a transform would work, too).

I believe there is a system parameter (a APT_?? variable?) that can be used to set the DataStage code page, but while we have talked about it, we have not done this so I have no specifics.

Brad.

Hey Brad / All,

This was finally resolved today. We set $APT_EBCDIC_VERSION environment variable to IBM037. This caused ds to use correct code page.

Another 'workaround' solution could be to convert all '!' in the source file to '|'.... this way when the target file is read in 037 codepage... it gets converted back into '!'...

Thanks everyone.... Brad specially...