Hi all,
I have this generic query related to design approach.
How do I make a call for using either a Buildop or a parallel Transformer for simple transformations? For eg, lets say if I want to simply trim the join keys, some data types conversions and rename them before performing a join, should I go for a custom c++ code (buildop) or use already existing functions(transformer)?
My understanding is, since we already have a stage which can perform all these tasks easily and in a single place, why should i spend considerable effort and time in writing my own code, especially if the voumes are not high enough to impact the performance (ya I agree buildops are efficient compared to transformers).
I understand this will be a trade-off between effort and performance gains. But in case volumes are not high and transformations simple, transformer is prefered option.
Can I get some expert opinions on the subject?
Thanks in advance.
Buildop vs Transformer
Moderators: chulett, rschirm, roy
Welcome Aboard :D !!!!
On general approach, Buildops ar used for the functionality which may not be acheivable or difficult to acheive by using existing stages.
Say eg, looping or some complex transfomation logic which involves some iteration of the string etc.,
"I agree buildops are efficient compared to transformers"
And this not true for all the cases.
On general approach, Buildops ar used for the functionality which may not be acheivable or difficult to acheive by using existing stages.
Say eg, looping or some complex transfomation logic which involves some iteration of the string etc.,
"I agree buildops are efficient compared to transformers"
And this not true for all the cases.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
If you can get your work done with the tools you have, dont need to invent a new one. So yes, for simple tasks as trim and renaming you can use transformer and modify stage. You dont need to build an extra stage for that. As Kumar noted, utilize buildops for functionlity not readily available from the present stages.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
I agree..Why re-invent the wheel? Get your stuff done with available options, and in your case, all that can be achieved via a transformer. In cases wherein you have to use a Pivot etc, you might want to consider Build-op for performance. Hope this helpsDSguru2B wrote:If you can get your work done with the tools you have, dont need to invent a new one. So yes, for simple tasks as trim and renaming you can use transformer and modify stage. You dont need to build an extra stage for that. As Kumar noted, utilize buildops for functionlity not readily available from the present stages.