User Defined Env Variable in Derivation of Transformer

[trobinson]

I have this variable defined at the project level and is in the Job properties.
$COB_DT.

Here is the compile error when using this parameter in a derivation of a Transformer stage in a PX job.

##E TFCP 000001 08:05:09(006) <transform> Error when checking composite operator: Unexpected tokens: [line 11,character 7]
##E TFCP 000012 08:05:09(007) <transform> Error when checking composite operator: Invalid local variable declaration: [line 11,character 7]
##E TFSR 000019 08:05:09(012) <main_program> Could not check all operators because of previous error(s)
##E TCOS 000029 08:05:09(013) <main_program> Creation of step finished with status = FAILED (ASODS2DataSet_1.XDerivations)

*** Internal Generated Transformer Code follows:
0001: //
0002: // Generated file to implement the C9C11V0S75_repos_XDerivations transform operator.
0003: //
0004:
0005: // define our input/output link names
0006: inputname 0 LCodes;
0007: outputname 0 LDersOut;
0008:
0009: global {
0010: // Job parameter declaration
0011: string $COB_DT;
0012: }


When I define the parameter locally to the job as plain old COB_DT, the job compiles fine.

Could something this simple be a bug?

[kumar_s]

Assign the parameter name as COB_DT and Default value as $COB_DT so that the value is retrived dymaically. and Use the parameter as COB_DT in transformer.

[trobinson]

Actually here is the workaround as detailed in the README.txt for this version of DataStage:

Job Parameters which are Environment Variables Fail to Compile in Transformer
-----------------------------------------------------------------------------
(Parallel Jobs only)

You can select and insert job parameters which are environment variables
into transformer expressions on the Parallel and Server Job transformer GUI.
However, on the Parallel canvas only, the jobs fail to compile.

Workaround for Parallel jobs:

The environment variable can be retrieved directly from the environment
at the time of transformer initialisation.

Define a new stage variable, e.g. EnvValue. In the Stage Variable
Properties dialog, accessed by right-clicking on the Stage Variables
area of the transformer GUI, set the "Initial Value" for this stage
variable to GetEnvironment("ENVVAR"), where ENVVAR is the environment
variable you have defined as a job parameter (dollar sign should not be
required). You can then use the stage variable in your derivations.

You can use the same code as above directly in a derivation if you wish,
but your job will run slower due to the need to make the function call
each time it is used in a row instead of once at the start of the stage.
Remember that you are picking up the environment variable value
dynamically; if for some reason it has changed since the start of the
job it will not be the same as the job parameter value you initially
defined.

[kumar_s]

Iam wondering with the comment given in README.txt

You can use the same code as above directly in a derivation if you wish,
but your job will run slower due to the need to make the function call
each time it is used in a row instead of once at the start of the stage....
.....if for some reason it has changed since the start of the
job it will not be the same as the job parameter value you initially
defined.

The stage varialbe will also be updated for each new row, if not how can it can reflect the changed value dynamically.

[trobinson]

If by dynamically you mean changeable while the job is running then it cannot, nor should it. The parameter is set once when the job starts and remains a given value for the entire time the job is executing in both your example and the README example. A stage variable with a default value that does not change value based on a derivation will remain the default value for the life of the job execution.

[kumar_s]

If by dynamically you mean changeable while the job is running then it cannot, nor should it. The parameter is set once when the job starts and remains a given value for the entire time the job is executing in both your example and the README example. A stage variable with a default value that does not change value based on a derivation will remain the default value for the life of the job execution.

But there should be some reason why they give this....

Code: Select all

.....if for some reason it has changed since the start of the 
job it will not be the same as the job parameter value you initially 
defined. 

[bandish]

Hi,

As per my knowledge, this compile time error occurs when you use an environment variable in the derivation directly as $COB_DT.
Instead the derivation should be

GetEnvironment('COB_DT')

, which will fetch the value of the Environment variable.

If you have a Job parameter, you can directly mention the name of the job parameter in the transformer stage like : COB_DT and it works.

[ray.wurlod]

You can remove the uncertainty by using the Expression Editor to build your expressions

[newtier]

You can remove the uncertainty by using the Expression Editor to build your expressions ...

Unfortunately that is not correct. Using the expression editor will not yield correct results. Using the Expression Editor with the GetEnvironment(%envvar%) function, then letting it substitute an JobParameter in for %envvar% with put the $value in, not the double-quoted value without the $.

If this were a "wizard" it may be that smart.

The help is not explicit enough (it only says string, and there is nothing in the EE Developer Guide or Advanced EE Developer Guide.)

The BEST documentation I have found so far to figure this out has been the earlier post of this chain, that bother to explain that you use GetEnvironment("EnvName") and not GetEnvironment($EnvName).

Thanks trobinson!