Never use basic transformers!!!

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

manuel.gomez
Premium Member
Premium Member
Posts: 291
Joined: Wed Sep 26, 2007 11:23 am
Location: Madrid, Spain

Post by manuel.gomez »

pneumalin wrote:You may need to do the following to add Basic Transformer to your palette:
1. Create a PX job in designer.
2. Right Click on Palette window and select "Customization->Customize"
3. On the Top-Left window, click on StageType->Parallel->Process to select "Basic_Transformer"
4. Click on right arrow button to add to your PX job palette.

Let me know if you run into any problem using it in PX job.
Worked!!!!!! Thanks very much!
pneumalin
Premium Member
Premium Member
Posts: 125
Joined: Sat May 07, 2005 6:32 am

Post by pneumalin »

Did you compile the job and run through some data?
Please tell me what is your OS and V8.0.1 or V8.0.0.
manuel.gomez
Premium Member
Premium Member
Posts: 291
Joined: Wed Sep 26, 2007 11:23 am
Location: Madrid, Spain

Post by manuel.gomez »

pneumalin wrote:I'd like to defend on the method using Basic Transformer in PX job via following observations:
1. Place Basic Transformer in a PX job won't break Partitioned Parallelism inscribed by PX engine, since it can be run in Parallel to map with Partitioned method declared in its Predecessor stage and its Successor stage. I once monitor this type of job running in Unix, and it spawns multiple physical processes in the system to fit in the parallelism requested by DataStage job.
2. I agree the performance penalty for using Basic Transformer cannot be avoided due to the high resource usage overhead(Mutilple Physcial Processes), unlike the generic PX transformer smart enough to combine the operator in runtime. However, if the data volume is not dramatically huge, and you need to use PX job to access certain useful stages(such as Join, Difference) that are not available in Server job, and you need to use some great Server functions such as IConv/OConv that are not available in PX job, then why not using this method. You won't see too much of the performance penalty and it's not a sin :)
3. As I was told by Ascential Advanced Engineer, the Basic Transformer is designed this way with the purpose listed in 1. and 2. I have designed some jobs fit in the condition as 2. for my client, and they works just fine with customer's satisfaction. As a matter of fact, my client turned down my offer to build them the C routines(hard to support in the future) that can be called from PX Transformer, and they prefer to have Basic Routines that can be called in Basic Transformer for the purpose of easier support than C code.

That's my personal opinion, please feel free to comment on this.
if you include a basic transformer on a PX job.....does it make the whole job to run in sequential mode?

Thanks very much
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Not necessarily. But it does incur overheads in translating to and from the typeless environment of BASIC.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
telenet_bi
Premium Member
Premium Member
Posts: 33
Joined: Wed Jul 23, 2008 7:33 am
Location: Mechelen, Belgium
Contact:

Post by telenet_bi »

returning to the first topic of the post: Never use Basic transformers.

An extra reason not to use them: If you log something with IBM support where you have a basic transform in the job they will not look for the issue, they will point to the basic transform.

(expect conversations like:I get an error in my join stage. Response: it's the basic transformer used in the previous job running 1H before this one)
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

So what if they say that. BASIC Transformers are part of the product and need to be supported.
pneumalin
Premium Member
Premium Member
Posts: 125
Joined: Sat May 07, 2005 6:32 am

Post by pneumalin »

You should probably learn how to escalate the issue to Critic 1 PMR, and they usually respond with proper answers. I just got a Basic Transfromer bug fixpack recently from IBM, and it just took them 2 weeks to figure it out. Good luck...
nasimul
Participant
Posts: 37
Joined: Wed Jan 25, 2006 1:38 am

Post by nasimul »

I did the following to add Basic Transformer to your palette:
1. Create a PX job in designer.
2. Right Click on Palette window and select "Customization->Customize"
3. On the Top-Left window, click on StageType->Parallel->Process to But here I don't see "Basic_Transformer"

Can you please tell me how to get it?
Post Reply