Conveting Binary feed to ASCII feed in UNIX
Moderators: chulett, rschirm, roy
Conveting Binary feed to ASCII feed in UNIX
Hi,
Can we convert a Binary feed to ASCII feed in UNIX? Can you pls provide a sample code, if available?
We have a Datastage job to 'split' the Binary feed containing Header, detail recods and Trailer record. Thought is to replace the Datastage 'split' job to a UNIX script which can parse the feed and separate the Header, Detail and Trailer record.
Thanks.
Can we convert a Binary feed to ASCII feed in UNIX? Can you pls provide a sample code, if available?
We have a Datastage job to 'split' the Binary feed containing Header, detail recods and Trailer record. Thought is to replace the Datastage 'split' job to a UNIX script which can parse the feed and separate the Header, Detail and Trailer record.
Thanks.
Ria
The only way of finding the limits of the possible is by going beyond them into the impossible
The only way of finding the limits of the possible is by going beyond them into the impossible
-
- Premium Member
- Posts: 376
- Joined: Sat Jan 07, 2012 12:25 pm
- Location: Piscataway
Re: Conveting Binary feed to ASCII feed in UNIX
Not too sure what you are looking for but I think "iconv" can do the job for you.
Syntax: iconv -f encoding -t encoding
Syntax: iconv -f encoding -t encoding
Jerome
Data Integration Consultant at AWS
Connect With Me On LinkedIn
Life is really simple, but we insist on making it complicated.
Data Integration Consultant at AWS
Connect With Me On LinkedIn
Life is really simple, but we insist on making it complicated.
-
- Premium Member
- Posts: 376
- Joined: Sat Jan 07, 2012 12:25 pm
- Location: Piscataway
My bad! Will learn to read the content before jumping to a solution next time
Jerome
Data Integration Consultant at AWS
Connect With Me On LinkedIn
Life is really simple, but we insist on making it complicated.
Data Integration Consultant at AWS
Connect With Me On LinkedIn
Life is really simple, but we insist on making it complicated.
Ria,
My first question would be why is it necessary to accomplish the split in Unix? I don't see any performance gain to be had between (for example) dropping the header and trailer in Unix and dropping it after the input stage.
Nearly every job I have starts with an FTP stage from a mainframe dataset. When the dataset includes more than one record type, I parse the records in a transformer stage using substring derivations and the record type as a constraint. Complex Flat File does it on the input.
If your source is mainframe and EBCDIC, the FAQ on this may be helpful.
My first question would be why is it necessary to accomplish the split in Unix? I don't see any performance gain to be had between (for example) dropping the header and trailer in Unix and dropping it after the input stage.
Nearly every job I have starts with an FTP stage from a mainframe dataset. When the dataset includes more than one record type, I parse the records in a transformer stage using substring derivations and the record type as a constraint. Complex Flat File does it on the input.
If your source is mainframe and EBCDIC, the FAQ on this may be helpful.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
Thanks for your quick responses!
Yes, its EBCDIC file with packed data format containing heade, Detail and Trailer record. We've an existing DataStage Server job for splitting the file into different formats.
iconv has been used in UNIX. But But, its showing a message ""Can not open converter". Command:
iconv -f UTF-8 -t ASCII TPS_DDS_20120411.DAT
(UTF-8 was also checked with UTF-16, BIN also, but the error msg was the same). Can you pls share if above command needs to be corrected?
od command was also. But, it was not useful.
od -c TPS_DDS_20120411.DAT
od -x TPS_DDS_20120411.DAT
Thanks in advance for your immediate attention.
Yes, its EBCDIC file with packed data format containing heade, Detail and Trailer record. We've an existing DataStage Server job for splitting the file into different formats.
iconv has been used in UNIX. But But, its showing a message ""Can not open converter". Command:
iconv -f UTF-8 -t ASCII TPS_DDS_20120411.DAT
(UTF-8 was also checked with UTF-16, BIN also, but the error msg was the same). Can you pls share if above command needs to be corrected?
od command was also. But, it was not useful.
od -c TPS_DDS_20120411.DAT
od -x TPS_DDS_20120411.DAT
Thanks in advance for your immediate attention.
Ria
The only way of finding the limits of the possible is by going beyond them into the impossible
The only way of finding the limits of the possible is by going beyond them into the impossible
Stick with the DataStage job. Any straight 'conversion' of the entire file will corrupt the packed fields and it's more trouble than it is worth to try to parse out the different record types, converting the individual string fields and unpacking the others using a script. Of course, you could fall back on Perl or something of that ilk but if you've got something that works why bother?
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
I can't help with your command line questions. I can offer a successful method I use, which I summarized in the FAQ viewtopic.php?t=143596&sid=96688420e3bd ... f6045c5a9b
Read the beginning section, then look for the question "Q: My mainframe file has two or more record types. If I'm not using Complex Flat File, how do I read the file?"
Post any further questions you may have here, and I'll try to help.
Read the beginning section, then look for the question "Q: My mainframe file has two or more record types. If I'm not using Complex Flat File, how do I read the file?"
Post any further questions you may have here, and I'll try to help.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
-
- Premium Member
- Posts: 145
- Joined: Fri Sep 15, 2006 12:23 pm
the quickest way would be to use datastage to do this task. However, there is option like dd to convert the ebcidic file to ascii, this does not convert compressed or comp fields. so, the best way is to write a c or cobol program that converts byte by byte.
alternatively, if you have perl and know how to use it then you could check this link which has got some code which does this taskhttp://search.cpan.org/~cxl/Convert-EBC ... /EBCDIC.pm
Hope this helps!!
alternatively, if you have perl and know how to use it then you could check this link which has got some code which does this taskhttp://search.cpan.org/~cxl/Convert-EBC ... /EBCDIC.pm
Hope this helps!!