DSXchange

Following on my documentation study (for certification), it insists over and over again in not using basic transformer

Can anyone tell me what is a basic transformer???????

A Server job Transformer stage. The concept of partitioned parallelism is defeated when you place serialized operations into it. A Server job BASIC Transformer stage will be invoked within an instance of a job passing data in and out thru pipes. It's horrifically slow, but lets you exploit existing BASIC logic and functions that you don't have access to in PX.

Sometimes you need it...and in those sometimes, a "25 mph speed limit" may be perfectly acceptable. As Ken says, you may have some great logic to exploit, making it worth doing. To say "never use it" out of hand throws away the value of whatever effective solution you may have built previously, or built more easily using uv/basic. Just use it wisely.

Ernie

I'd like to defend on the method using Basic Transformer in PX job via following observations:
1. Place Basic Transformer in a PX job won't break Partitioned Parallelism inscribed by PX engine, since it can be run in Parallel to map with Partitioned method declared in its Predecessor stage and its Successor stage. I once monitor this type of job running in Unix, and it spawns multiple physical processes in the system to fit in the parallelism requested by DataStage job.
2. I agree the performance penalty for using Basic Transformer cannot be avoided due to the high resource usage overhead(Mutilple Physcial Processes), unlike the generic PX transformer smart enough to combine the operator in runtime. However, if the data volume is not dramatically huge, and you need to use PX job to access certain useful stages(such as Join, Difference) that are not available in Server job, and you need to use some great Server functions such as IConv/OConv that are not available in PX job, then why not using this method. You won't see too much of the performance penalty and it's not a sin
3. As I was told by Ascential Advanced Engineer, the Basic Transformer is designed this way with the purpose listed in 1. and 2. I have designed some jobs fit in the condition as 2. for my client, and they works just fine with customer's satisfaction. As a matter of fact, my client turned down my offer to build them the C routines(hard to support in the future) that can be called from PX Transformer, and they prefer to have Basic Routines that can be called in Basic Transformer for the purpose of easier support than C code.

That's my personal opinion, please feel free to comment on this.

Partitioned Parallelism and either a mixed MPP and SMP environment or clustered environment won't work with DS BASIC Transformers. Then DS BASIC Transformer is only available on the one SMP server running the DS Engine.

Unless, of course, you install a full DataStage Engine on every machine. Talk to your account rep - they'd LOVE to hear from you ($$$$$$).

ray.wurlod wrote:Unless, of course, you install a full DataStage Engine on every machine. Talk to your account rep - they'd LOVE to hear from you ($$$$$$).

Only on each SMP machine in the cluster - the MPP are out of consideration. I doubt anyone will pay to license a DS Engine on every phyiscal machine in the SMP cluster just to satisfy having a BASIC Transformer stage operational in full parallelism.

MPP is for sure not in the case I described in my previous post. My clients has already paid the premium price upfront upon purchasing the product, so the license issues never come across to their attention. It's good to know the limitation for helping out the new clients in the future, Thanks for the info.
I still believe there is a valid reason for Basic Transformer usage to exist in the product, or we sould advise IBM to take away this feature in future release.
Appreciated if someone can find out this feature still existing in 8.0.1 or not?

pneumalin wrote: Appreciated if someone can find out this feature still existing in 8.0.1 or not?

Great having started such an interesting debate

As far as I am trying, in version 8 is not possible to use basic transformers any more.

Not avaliable in tool palete, and I tried Ctrl+Cing from a server job and then pasting, but pasted nothing.

Could anybody use it?

It's never been available on the palette by default, you have to specifically customize the pallet to add it.

The the BASIC transform stage still exists at V8

ArndW wrote:The the BASIC transform stage still exists at V8 ...

May you please tell me how shall I incorporate it to a parallel job?
Thanks a lot

Go to the list of stages -> Parallel -> Processing and then drag-and-drop the BASIC transform stage to your canvas.

I am sure we are talking about the same, because this cannot be that difficult!!!

going to parallel-->Processing, I do have 'Transformers' (I guess this is the parallel one, the one that is C-coded at compilation time) but none basic transformer
Is it the same?

You may need to do the following to add Basic Transformer to your palette:
1. Create a PX job in designer.
2. Right Click on Palette window and select "Customization->Customize"
3. On the Top-Left window, click on StageType->Parallel->Process to select "Basic_Transformer"
4. Click on right arrow button to add to your PX job palette.

Let me know if you run into any problem using it in PX job.