parallel stage transformer

[just4u_sharath]

i have a question. many people said notto use a parallel transformer becuase of performance issues. is this true. i am talking about the parallel xformer not basic xformer.
Another question instead of using a xformer, if i build a wrapper (custom stage) and use, will there be any performance improvement.
Replies will be appreciated.
why the performance degrades when we use a parallel xformer.(is this true)

[ArndW]

The only real performance issue with a PX transform stage, in my opinion, is that it seems to take an eternity to compile; runtime performance is quite good as it is executing c++ code.
Could you cite some specific examples of cases where people state that the PX transform is slow?

PX has many stages to perform specific functions. I think it makes sense to use those stages where possible. Some things (i.e. changing a column data type) could be done either in a purpose-built stage (in this case, a MODIFY stage) or in a transform stage. I think that both will perform at similar speeds and you can test this if you wish; but it does make sense to use a MODIFY stage rather than a transform stage. If you not only need to convert a datatype but also check it's value you would need 2 modify stages - even though these might be combined at runtime it makes sense to perform this type of transformation in a transform stage, since it would only take one derivation expression to do both actions.

A specific wrapper stage to perform an action might be faster if well-written - but there really is no way to give a correct answer to this question as there are far too many variables.

Once somebody can show a performance degradation with the parallel transformer stage (degradation compared to what?) then it might be explained.

[ray.wurlod]

It used to be true. In the late 7.1 versions and subsequently the Transformer stage was completely re-vamped and is quite efficient - it now generates a transform operator (in osh terminology).

Yes, it takes longer to compile than other stages, but that's because more things have to happen - generation of C++ source code, compiling and linking of the result - but that means that the run-time stage is efficient. Also, if you make a change to the job but do not change the Transformer stage, recompiling the job does not re-compile the Transformer (unless you select Force Compile from the menu).

[ameyvaidya]

Also if you build a custom stage, the onus is on you to ensure that it is properly coded, tested and does not have any memory leaks.. etc.

Also the person who will support it afterwards better understand C++ and what exactly the stage does.

Too much work/risk IMO. Especially when there is a standard alternative available.

[sri.cbv]

i have a question. many people said notto use a parallel transformer becuase of performance issues. is this true. i am talking about the parallel xformer not basic xformer.
Another question instead of using a xformer, if i build a wrapper (custom stage) and use, will there be any performance improvement.
Replies will be appreciated.
why the performance degrades when we use a parallel xformer.(is this true)

no one will tell u that not to use transformer stage because of performance.
i agree with you....transformer has its own interface , so it decreases the perfomance

[ray.wurlod]

Your knowledge is out of date. In version 7.5 compiling the Transformer produces an operator, which uses exactly the same "interface" as all other operators.