Parallel Transformer execution order

[Developer9]

I know this question is asked many many times , i see 2 answers 1. SV, Constraints, Derivations 2. Constraints , SV, Derivations. I tried for this in IBM documentation and couldn't find it anywhere.

I personally feel its SV, Const and Derivations for the reason why process all the SV's for all input records if the constraint drops a lot of records.

Please throw some light on this.

[ArndW]

Considering that one can use stage variables in constraints, the order is first to evaluate the stage variables, then evaluate constraints and finally to evaluate derivations.

[Developer9]

ArndW, Thanks for the reply , This answer looks simple but is also why i debate with myself.

Considering the Stage variables can be used inside constraints , shouldn't the execution order be dynamic? first Check if a SV is used inside a constraint if yes then process SV before Constraints. And if not first process Constraints then SV. isn't it the better solution ?

[ArndW]

Going that route might save some processing cycles in jobs where a large majority of the rows are filtered by a constraint; but it would add complexity to the process of compiling transform stages plus it introduces potential confusion.
The Server transform stage would also need to be changed...

Thus I think that while an enhancement request might have merit, it won't be acted on by engineering because the potential gain [speed in certain cases] is outweighed by the risks [of breaking something].

[rkashyap]

As previous poster has said, order of evaluation is SV, Constraints, Derivations

Generated code from transformer stage in folder RT_SCn of project directory (where n is JobNo). You can see that order of evaluation over there.

See snippet below:

Code: Select all

mainloop {
        // declare our intermediate variables for this section
        dfloat InterVar0_0;
        dfloat InterVar0_1;
        // evaluate the stage variables first
        StageVar0_StageVar = in_dat.PART_NUM;
        // initialise the rejected row variable
        RowRejected0 = 1;
        // evaluate constraint and columns for link: out_dat
        if ((in_dat.COST_PRICE > in_dat.SALE_PRICE))
        {
                if ((in_dat.SALE_PRICE >= in_dat.COST_PRICE)) {
                        InterVar0_0 = 0.0;
                } else {
                        InterVar0_1 = in_dat.SALE_PRICE;
                        InterVar0_0 = InterVar0_1;
                }
                if ((InterVar0_0 > 0)) {
                        out_dat.CLEARENCE_IND_Corrected = InterVar0_2;
                } else {
                        out_dat.CLEARENCE_IND_Corrected = InterVar0_3;
                }
                //;
                writerecord 0;
                RowRejected0 = 0;
        }
}

[qt_ky]

1. Stage Variables top to bottom
2. Loop Variables top to bottom
3. Output links & Constraints top to bottom
4. Derivations top to bottom

[chulett]

Also, you can see from the posted code that when a constraint evaluates to FALSE, the derivations in the link are not... derived.

Wanted to add that I'm not sure I'd want to see anything about the order it evaluates things be made 'dynamic'. For whatever that is worth.

[Developer9]

Very useful information , Thanks to everyone.

[qt_ky]

It is well documented, but a bit buried. I did not find it by searching "execution order" but by searching "stage variable" or "evaluation sequence."

Here is a link.

[ray.wurlod]

Hmm, nobody mentioned initialization of stage variables, initialization of loop variables or execution of loop variables.

[chulett]

Hmmm... true about initializations but execution of loop variables was mentioned.

[qt_ky]

The IBM doc link given above details it quite well.

[ray.wurlod]

There's also logic to detect whether any otherwise/log and reject output links need to be used.