Binary datatype issue
Moderators: chulett, rschirm, roy
Binary datatype issue
Hi All,
we are migrating PX jobs from Ds 7.5 windows to DS 8.5 linux os.we have the datatype binary for one of the column. the incoming values are same for both the DS 7.5 code and DS 8.5 code .the output data mismatches in bothe versions.below is example and datatype is binary (20).please suggest.
Incoming value -ds8.5
00510353050000000000
output result
{0 00 0 00 5 00 1 00 0 00 3 00 5 00 3 00 0 00 5 00}
Incoming value -ds7.5
00510353050000000000
output result
{0 0 5 1 0 3 5 3 0 5 0 0 0 0 0 0 0 0 0 0}
we are migrating PX jobs from Ds 7.5 windows to DS 8.5 linux os.we have the datatype binary for one of the column. the incoming values are same for both the DS 7.5 code and DS 8.5 code .the output data mismatches in bothe versions.below is example and datatype is binary (20).please suggest.
Incoming value -ds8.5
00510353050000000000
output result
{0 00 0 00 5 00 1 00 0 00 3 00 5 00 3 00 0 00 5 00}
Incoming value -ds7.5
00510353050000000000
output result
{0 0 5 1 0 3 5 3 0 5 0 0 0 0 0 0 0 0 0 0}
Can you still test on the old 7.5? If yes, make a simple job that outputs to a text file and uses a row generator stage for the 1 sample row and see if you still have the error - that can then be submitted as a bug to IBM. If the error isn't reproduceable then you'll have to experiment with the database, as the cause might lie not in Datastage but with the DB.
But it sounds like a bug that needs to be addressed.
But it sounds like a bug that needs to be addressed.
This looks to me to be an issue with unicode vs non-unicode string data. The example from IS 8.5 appears to be a unicode string (2+ bytes-per-character), while the DS 7.5 example appears to be a non-unicode string (1-byte-per-character).
As you're moving from Windows to Linux: how was the source file created for the DS 8.5 job? If you saved it in some text editor as a unicode file, you probably need to either a) recreate it as a non-unicode file or b) read the field as a unicode string rather than a binary field.
You can use the od -xc command in linux to look at a dump of the file.
Regards,
As you're moving from Windows to Linux: how was the source file created for the DS 8.5 job? If you saved it in some text editor as a unicode file, you probably need to either a) recreate it as a non-unicode file or b) read the field as a unicode string rather than a binary field.
You can use the od -xc command in linux to look at a dump of the file.
Regards,
- james wiles
All generalizations are false, including this one - Mark Twain.
All generalizations are false, including this one - Mark Twain.
Hi Andrew,
yes we, have Ds 7.5 old server too. i had created a simple job by taking row generator stage with column datatype Binary and writing to text file. so the output data in the Datastage 7.5 is different as compared to datastage 8.5 . so this means the datastage 8.5 has bug int it ? or fix need to applied..please suggest
yes we, have Ds 7.5 old server too. i had created a simple job by taking row generator stage with column datatype Binary and writing to text file. so the output data in the Datastage 7.5 is different as compared to datastage 8.5 . so this means the datastage 8.5 has bug int it ? or fix need to applied..please suggest
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
When displaying data type Raw (= binary) each byte is shown as a single character if that character is a printable character, or as a two digit hex number if the character is non-printable. The 00 characters in your data are ASCII NUL characters (Char(0)).
Your 8.5 raw string shows Little-Endian representations of two-byte character representations of the characters you have in your 7.5 output.
You would need to convert your string to an integer or decimal number to be able successfully to store it as an integer.
Your 8.5 raw string shows Little-Endian representations of two-byte character representations of the characters you have in your 7.5 output.
You would need to convert your string to an integer or decimal number to be able successfully to store it as an integer.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
What is the definition of the data on the mainframe file? Do you have a table definition for it (Cobol copybook, I'm assuming)? Also, how the data is transferred (binary vs. text) has no relevance to how the data is parsed by DataStage. For example, I use FTP Enterprise to read EBCDIC directly into the job, with a transfer type of binary and the Cobol copybook defining every column as it is read.synsog wrote:but the derived columns which have char data type comes from the mainframe file with EDCBIC character set and dataformat in Binary.pls suggest
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
I'm not sure I follow you, so allow me to ask some basic questions.
Is your mainframe file (EBCDIC) written using the same copybook?
What is your DataStage method (stage, sequence) for reading the mainframe file? Is it a native read (input stage to next stage) or is there a separate download to a sequential file?
Can you give an example of a "binary" column on the mainframe record which you are trying to derive to another type (like integer)?
The important point here is that DataStage does not "care" how the data is stored on disk. If you read it the exact same way it was written, you should not have the problems you are describing.
Is your mainframe file (EBCDIC) written using the same copybook?
What is your DataStage method (stage, sequence) for reading the mainframe file? Is it a native read (input stage to next stage) or is there a separate download to a sequential file?
Can you give an example of a "binary" column on the mainframe record which you are trying to derive to another type (like integer)?
The important point here is that DataStage does not "care" how the data is stored on disk. If you read it the exact same way it was written, you should not have the problems you are describing.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
Then there is a difference in 8.5 that you haven't found yet. Without a better understanding (answers to my previous questions), I can only guess.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872