Is there a maximum number of stage variables?

[gowrishankar_h]

Hi,

I just wondered how many stage variage can we have in a single transformer.I have almost 75 stage variable need to use?

Thanks,
Gowri Shankar H

[arunkumarmm]

Not sure about the maximum but 75 should not be a problem... I have used in 100s...

[ray.wurlod]

There is no documented limit.

[gowrishankar_h]

Thanks...

Will it hit performance if i use more stage variable?

[ArndW]

Since transform stages are converted into c++ programs in parallel jobs. A quick look at the standards didn't show a limit, but often the symbol table will overflow when too many objects are used - but this is probably an unsigned long integer so you've got a lot of headroom.

Most likely the DataStage designer front-end will have a limit on the number of entries you can have in the stage variable box, but as others have stated that they've used 100s you shouldn't reach that limit. If you do, it might be worth thinking about either the approach or in using a custom build-op instead.

Since the stage variables are generally simple assignments and not complex or lengthy loops, the performance impact is going to be small - but that deepends upon how you are assigning and using the values. Doing a "SVtemp = In.ColumnA" is going to be much faster than a long expression with type conversions and the like.

[Mayurgupta7744]

Hi,

In the above post you informed us that the trasformer is converted into C++ code in parallel job.
I really very anxious to know about following:

# As per my knowledge Data Stage on the back end converts the job components in to Orchestrate & C++ code for processing.
So is here, logic in trasformer will get converted in to C++ code.

# Are not all components get converted in to C++ & Orchestrate lang?

# If not which components only get converted into C++?

# Are the components which gets converted in to C++ expensive, in terms of performance?

Flashing light in to these points will be really helpful!!!

[ray.wurlod]

The entire Transformer stage is rewritten in C++ at compile time, and this code is compiled and linked into a library/object (according to current settings).

The performance hit is taken at compile time, not at run time. You may notice that a job with a Transformer stage will take longer to compile than a job with none. If a job is changed, but its Transformer stages are unchanged, then recompiling does not recompile the Transformer stages, in the interests of efficiency (unless Force Compile is specified).

Generated C++ source code for job number nnn can be found in the project subdirectory RT_SCnnn along with osh scripts for executing the compiled version.