Reusable Multiple-Stage Process

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
newerakb
Participant
Posts: 6
Joined: Fri Jun 18, 2010 2:46 pm

Reusable Multiple-Stage Process

Post by newerakb »

I need to do some name cleansing, and I'm using the USNAME Standardize stage, but there's some additional prep and cleanup I need to do on the names using Transformer stages.

However, I have about 20 very different input files that all need to run through this identical name cleanse process, which consists of about 3-5 stages. Is there a way I can reuse this group of stages within all my jobs, rather than create the entire sequence of stages each time?

Basically, I want to create my own stage, but instead of using BASIC or C++, I want to create it like any other parallel job in the DS Designer. Does this functionality exist?

I know that I could make the name cleanse process its own job, but then I'd need to split each of my file-processing jobs into two parts: one before the name job, and one after. I'd like to just drop this process in the middle of existing jobs as a stage.

If this isn't possible, it should be, and it seems very obvious and useful :)
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Check out "Shared Containers". You don't need to create your own stage (though that's possible too, though you would need to code it in C++), you just need to create a shared container. And you can do that by selecting the existing five stages and creating the container from them.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
newerakb
Participant
Posts: 6
Joined: Fri Jun 18, 2010 2:46 pm

Post by newerakb »

exactly what I was looking for, thanks ray!
Post Reply