Error Compiling Parallel Job with Transformer Stage

[ravivarma20]

Hi,

Parallel job throwing compilation error at transformer stage.

Error Log(Captured by hitting More button in complie window):

Code: Select all

Output from transformer compilation follows:

SH: SH:: command not found (MKG_FDW_MGR_01_MovementData.TRN_Store_Conv)

*** Internal Generated Transformer Code follows:
0001: //
0002: // Generated file to implement the V0S3_MKG_FDW_MGR_01_MovementData_TRN_Store_Conv transform operator.
0003: //
0004: 
0005: // define our input/output link names
0006: inputname 0 LNK_ItemID_LookupFail;
0007: outputname 0 LNK_Map;
0008: outputname 1 LNK_ItemRej;
0009: 
0010: initialize {
0011: 	// define our row rejected variable
0012: 	int8 RowRejected0;
0013: 
0014: 	// define our null set variable
0015: 	int8 NullSetVar0;
0016: 
0017: }
0018: 
0019: mainloop {
0020: 	// initialise our row rejected variable
0021: 	RowRejected0 = 1;
0022: 
0023: 	// evaluate columns (no constraints) for link: LNK_Map
0024: 	LNK_Map.GEN = LNK_ItemID_LookupFail.UPC_GENERATION;
0025: 	LNK_Map.STORE_NUMBER1 = LNK_ItemID_LookupFail.STORE_NUMBER1;
0026: 	writerecord 0;
0027: 	RowRejected0 = 0;
0028: 	// evaluate columns (no constraints) for link: LNK_ItemRej
0029: 	writerecord 1;
0030: 	RowRejected0 = 0;
0031: }
0032: 
0033: finish {
0034: }
0035: 
*** End of Internal Generated Transformer Code

Thanks,
Ravi.

[richdhan]

Hi Ravi,

I have come across these errors when the

1. Configuration file is not defined properly

2. Stages variables have some logical problems

3. Environment Variables defined as Job parameters are used without proper conversion

You are using 2 output links but the second output link does not have any output columns. Is the problem because of that?

--Rich

[ray.wurlod]

You are using 2 output links but the second output link does not have any output columns. Is the problem because of that?

When the second output link is a reject link from a Sequential File stage you can not dictate the metadata; you get one column, the raw input record.

Something similar is true in the Other/Log link in a Transformer stage (or, indeed, any output link). Passthrough columns do not show up in the derivations. Instead, they are set up as pointers to the same variable. (This is also the technique used in server/BASIC Transformer stages.)

[ravivarma20]

Hey guys,

Thanks for your replies. I strongly feel that there was not a problem with meta data definition, stage variables or environment variables. Definitely there was a problem with the configuration, because all the jobs in our project throwing the similar error at transformer stage.

Thanks,
Ravi.