Defining array size

[CelesteT]

Within a transformer, I had to define a long IF ELSE IF statement in the derivation section. I've encountered the following error while compiling the job:

Compiling: Source = 'RT_BP760/JOB.2125494777.DT.1344463330.TRANS5', Object = 'RT_BP760.O/JOB.2125494777.DT.1344463330.TRANS5'
****************************************************************************************************************************************************************************************LEX Error: input string too long to fit in yywtext[].
Increase the array size by defining YYLMAX to be a higher value
(test.trapnull)

How and where do I define YYLMAX to be a higher value? What is the maximum value I can set to? Will there be any consequences (e.g. slow query performance) that can result from setting a higher value?

Is there another way to handle huge IF ELSE statements?

[ray.wurlod]

There is an upper limit to the complexity of nested If..Then..Else statements. The limit is not documented, but IS hard-wired. You can't change it.

There is an also an upper limit to the overall length of an expression. This used to be documented, but I can't find it now. From memory it is about 8KB.

The array yylmax[] is in the C code for the expression parser, way down in the bowels of the DataStage engine. Unless you have access to DataStage Engine source code, you can't change it. As you can guess from the error message, the size of this array can be changed (but only by Ascential Engineering), and there is still this hard-wired upper limit.

About all you can do is to move your expression into a Routine (where it's probably easier to maintain). The limits are also considerably larger; the limit on a Routine is that its compiled code must not exceed 64KB. This means a huge amount of source code.

Beware, though, that the more complex the expression/Routine, the more processing that needs to be done, and therefore the longer that will take, reducing your overall throughput. Try to determine whether there is any way to simplify the code while still achieving the same result.

Also, put the most-likely-to-occur conditions at the head of CASE or ON GOSUB constructs wherever possible, again to avoid unnecessary processing.