OCONV-MCP replacement for PX

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
gsherry1
Charter Member
Charter Member
Posts: 173
Joined: Fri Jun 17, 2005 8:31 am
Location: Canada

OCONV-MCP replacement for PX

Post by gsherry1 »

Hello Forum,

Is there a general string cleansing function in parallel job like OCONV-MCP to remove all non-printable characters? The only way I can figure to do this is to explicitly list all the characters you wish to translate in a convert function call.

Any other suggestions?

Thanks,

Greg
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

No builtin PX equivalent to the "MCP" code of Iconv/Oconv. :(
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

BASIC Transformer stage? This will allow you to use Oconv() admittedly with a throughput penalty.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
mhester
Participant
Posts: 622
Joined: Tue Mar 04, 2003 5:26 am
Location: Phoenix, AZ
Contact:

Post by mhester »

Greg,

Don't know if you found a decent solution to your issue, but you could use the following unix command -

Code: Select all

tr -cd '\11\12\40-\176' < $INPUT_FILE > $OUTPUT_FILE
This will strip all non-printable characters from a string and I would think it would be much more efficient than throttling down to a sequential flow using the BASIC transformer.

I created a wrapper which invoked this from a stream and it works as advertised.

Try not to develop in terms of mixing server and parallel processes (unless absolutely necessary), but to separate these processes.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

:idea: That approach would probably also work in an External Filter stage.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
mhester
Participant
Posts: 622
Joined: Tue Mar 04, 2003 5:26 am
Location: Phoenix, AZ
Contact:

Post by mhester »

Ray,
That approach would probably also work in an External Filter stage.
You are correct, but it's a bit tougher.
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Yea, but isnt this dude sitting on windows? :roll:
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Duzn't matta!

Parallel jobs on Windows actually run on MKS Toolkit (a UNIX emulator). So you have all the UNIX shell functionality you need. The tr command is definitely available.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply