How bad is a transformer?

ashwin2912 · Post by **ashwin2912** » Mon Jul 04, 2005 3:22 am

I know its best to avoid a transformer; but how bad is a transformer?

Does a transformer simply by being in the job affect performance or does records passing through it hit the performance?

ray.wurlod · Post by **ray.wurlod** » Mon Jul 04, 2005 3:14 pm

I don't know where this is coming from? Certainly including a Transformer stage in a parallel job will require more time to compile, because source code has to be generated and compiled. And there will be a small overhead in initial invocation of any libraries needed at run time. But would not the same functionality in Modify, Switch, Copy and other stage types require the same amount of work (assuming that the expressions in the Transformer stage were efficiently constructed)?

vmcburney · Post by **vmcburney** » Mon Jul 04, 2005 4:48 pm

In the Ascential World 2004 tech sessions on Enterprise Edition they advised people to steer clear of the transformer stage where the same thing could be done in a copy, modify or filter stage. I have found that if you can replace a transformer with a single alternative stage the job is about 25% faster (with a lot of factors to take in consideration).

We still make heavy use of Transformers. For starters they are usually doing a wide range of tasks and would need to be replaced by two or three parallel stages. Secondly the modify stage is an absolute pain in the you know what. There is no right mouse click menu for building code, there is no reject link for failed records, the error messages can be difficult to debug and there are some limitations on what the functions can do to your fields.

We decided maintainability and flexibility were going to be more important then speed.

We still use native parallel stages where possible, especially the filter stage, but most of our jobs have a transformer.

ashwin2912 · Post by **ashwin2912** » Mon Jul 04, 2005 9:43 pm

Thanks, Vmcburney for the statistics, I was looking for something like that. So when you have to use a transformer (not replaceable with a single stage); you have no choice but to use a transformer.

Hope in the next DS version, we dont have to hear this:

vmcburney wrote:In the Ascential World 2004 tech sessions on Enterprise Edition they advised people to steer clear of the transformer stage where the same thing could be done in a copy, modify or filter stage.

Lets hope they either make the transformer more efficient or add more transformer like features to the copy, modify and filter stages.