I know its best to avoid a transformer; but how bad is a transformer?
Does a transformer simply by being in the job affect performance or does records passing through it hit the performance?
How bad is a transformer?
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 57
- Joined: Wed Jan 26, 2005 10:22 pm
- Location: India
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
I don't know where this is coming from? Certainly including a Transformer stage in a parallel job will require more time to compile, because source code has to be generated and compiled. And there will be a small overhead in initial invocation of any libraries needed at run time. But would not the same functionality in Modify, Switch, Copy and other stage types require the same amount of work (assuming that the expressions in the Transformer stage were efficiently constructed)?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
In the Ascential World 2004 tech sessions on Enterprise Edition they advised people to steer clear of the transformer stage where the same thing could be done in a copy, modify or filter stage. I have found that if you can replace a transformer with a single alternative stage the job is about 25% faster (with a lot of factors to take in consideration).
We still make heavy use of Transformers. For starters they are usually doing a wide range of tasks and would need to be replaced by two or three parallel stages. Secondly the modify stage is an absolute pain in the you know what. There is no right mouse click menu for building code, there is no reject link for failed records, the error messages can be difficult to debug and there are some limitations on what the functions can do to your fields.
We decided maintainability and flexibility were going to be more important then speed.
We still use native parallel stages where possible, especially the filter stage, but most of our jobs have a transformer.
We still make heavy use of Transformers. For starters they are usually doing a wide range of tasks and would need to be replaced by two or three parallel stages. Secondly the modify stage is an absolute pain in the you know what. There is no right mouse click menu for building code, there is no reject link for failed records, the error messages can be difficult to debug and there are some limitations on what the functions can do to your fields.
We decided maintainability and flexibility were going to be more important then speed.
We still use native parallel stages where possible, especially the filter stage, but most of our jobs have a transformer.
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
-
- Participant
- Posts: 57
- Joined: Wed Jan 26, 2005 10:22 pm
- Location: India
Thanks, Vmcburney for the statistics, I was looking for something like that. So when you have to use a transformer (not replaceable with a single stage); you have no choice but to use a transformer.
Hope in the next DS version, we dont have to hear this:
![Smile :)](./images/smilies/icon_smile.gif)
Hope in the next DS version, we dont have to hear this:
Lets hope they either make the transformer more efficient or add more transformer like features to the copy, modify and filter stages.vmcburney wrote:In the Ascential World 2004 tech sessions on Enterprise Edition they advised people to steer clear of the transformer stage where the same thing could be done in a copy, modify or filter stage.
![Smile :)](./images/smilies/icon_smile.gif)