Hi,
I have this simple job where I pull data from TeraData source and load into a complex flat file. Now the job works flawlessly except when a '!" character is encountered in the source columns,
When incoming varchar source column contains '!' character; it seems that it automatically gets translated to '|' character. I viewed the output file in Mainframe system and see that the hex value has indeed '4F' instead of '5A'.
What is more interesting is that when I read the same file back into DataStage jobs and try to peek the value the '!' symbols are displayed correctly!!
Does anyone have any idea of why would this happen?
Translation of '!' to '|'
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 33
- Joined: Mon Nov 12, 2007 1:02 am
- Location: Bangalore
Translation of '!' to '|'
Rishabh Sagar V
Bangalore
Bangalore
I'm sorry I don't have more info, but I think it is a difference in the EBCDIC to ASCII mapping between the mapping that DS uses to convert to EBCIDC and what the mainframe is expecting. See if you can determine the codepage that DS uses for EBCDIC and compare it to the codepage used on your mainframe.
I have a document somewhere in my 'very organized' files that talks about this, but my files are too organize for me to find it right now. If I can find it, I'll see if I can post it.
Brad.
I have a document somewhere in my 'very organized' files that talks about this, but my files are too organize for me to find it right now. If I can find it, I'll see if I can post it.
Brad.
It is not that I am addicted to coffee, it's just that I need it to survive.
Found the reference, in fact I posted in back in April:
EBCDIC ! and ] not translated properly
The IBM site is posted there towards the bottom. It shows that code pages 037 and 500 map 4F to '|' and '!', respectively, and to 5A to '!' and ']', respectively.
Hope this helps.
Brad.
EBCDIC ! and ] not translated properly
The IBM site is posted there towards the bottom. It shows that code pages 037 and 500 map 4F to '|' and '!', respectively, and to 5A to '!' and ']', respectively.
Hope this helps.
Brad.
It is not that I am addicted to coffee, it's just that I need it to survive.
-
- Participant
- Posts: 33
- Joined: Mon Nov 12, 2007 1:02 am
- Location: Bangalore
Brad, Thanks a lot for this reference. It really helped me understand the actual problem. Is there a live scenario workaround / solution available?bcarlson wrote:Found the reference, in fact I posted in back in April:
EBCDIC ! and ] not translated properly
The IBM site is posted there towards the bottom. It shows that code pages 037 and 500 map 4F to '|' and '!', respectively, and to 5A to '!' and ']', respectively.
Hope this helps.
Brad.
I am yet to confirm the codepage differences between my DS server and the Mainframe server (as I am @ home now), but can someone help me on how to change my DS codepage (translation code page??) if there is a difference between the two?
Also I believe (from what I saw on wikipedia EBCDIC page) that there exists more than one codepage for EBCDIC formats, so what happens when my target file moves from one type of mainframe to other?
Thanks a lot Guys!
Rishabh Sagar V
Bangalore
Bangalore
I think we ended up manually changing the data on output in a buildop (a transform would work, too).
I believe there is a system parameter (a APT_?? variable?) that can be used to set the DataStage code page, but while we have talked about it, we have not done this so I have no specifics.
Brad.
I believe there is a system parameter (a APT_?? variable?) that can be used to set the DataStage code page, but while we have talked about it, we have not done this so I have no specifics.
Brad.
It is not that I am addicted to coffee, it's just that I need it to survive.
-
- Participant
- Posts: 33
- Joined: Mon Nov 12, 2007 1:02 am
- Location: Bangalore
Hmm, did you scan each incoming col for these particular characters and then replaced each occurence of '!'??bcarlson wrote:I think we ended up manually changing the data on output in a buildop (a transform would work, too).
I believe there is a system parameter (a APT_?? variable?) that can be used to set the DataStage code page, but while we have talked about it, we have not done this so I have no specifics.
Brad.
I must be making a mistake in understanding this..... How to handle this in a Xf-TFM?
Rishabh Sagar V
Bangalore
Bangalore
-
- Participant
- Posts: 33
- Joined: Mon Nov 12, 2007 1:02 am
- Location: Bangalore
Hey Brad / All,bcarlson wrote:I think we ended up manually changing the data on output in a buildop (a transform would work, too).
I believe there is a system parameter (a APT_?? variable?) that can be used to set the DataStage code page, but while we have talked about it, we have not done this so I have no specifics.
Brad.
This was finally resolved today. We set $APT_EBCDIC_VERSION environment variable to IBM037. This caused ds to use correct code page.
Another 'workaround' solution could be to convert all '!' in the source file to '|'.... this way when the target file is read in 037 codepage... it gets converted back into '!'...
Thanks everyone.... Brad specially...
Rishabh Sagar V
Bangalore
Bangalore