DSXchange

Hi,
I am getting the compilation error. The job reads the data from the sequential flat file, does data validation like - data type (Date, Decimal,Integer) and Null/Empty check for the incoming fields in the source file. There are approximately 150+ fields where the data validation is being done.
Once the validation is done the data is being loaded into the Oracle table.

I think due to the number of fields being validated the job is not getting compiled. I tried to compile without doing any validation or any fields then the job got compiled. The data validation is being done in stage variables.

Below is the error that i am getting while compiling. Please help me out on this as i am not able to progress anywhere.


Compilation Error Message -

##W IIS-DSEE-TBLD-00000 22:14:29(009) <main_program> Error when checking composite operator: Output from subprocess: mpiler(iropt) warning: Out of memory during IPA analysis, recovering...
compiler(iropt) warning: skip interprocedureal analysis due to out of memory
/opt/ibm/sunstudio12.1/prod/bin/iropt:out_of_memory_handler+0x138
/opt/ibm/sunstudio12.1/prod/bin/iropt:heap_tmpalloc+0xc0
/opt/ibm/sunstudio12.1/prod/bin/iropt:new_heap_set+0x3c
/opt/ibm/sunstudio12.1/prod/bin/iropt:find_dominators+0x1f8
/opt/ibm/sunstudio12.1/prod/bin/iropt:find_loops+0xd0
/opt/ibm/sunstudio12.1/prod/bin/iropt:0x2a11fc
/opt/ibm/sunstudio12.1/
##W IIS-DSEE-TBLD-00000 22:14:29(010) <main_program> Error when checking composite operator: Output from subprocess: prod/bin/iropt:0x2a01bc
/opt/ibm/sunstudio12.1/prod/bin/iropt:main+0x730
/opt/ibm/sunstudio12.1/prod/bin/iropt:_start+0x108
compiler(iropt) warning: Out of memory during optimizing void APT_TransformOperatorImplV0S255_VAL_AMB_VW_CONTRACT_to_BIH_STG_JB_Trfm_Validate_Source_Null::processInputRecordChild(int,int*,int*) in /opt/ibm/projects/BIH_GAF_D1/RT_BP90.O/V0S255_VAL_AMB_VW_CONTRACT_to_BIH_STG_JB_Trfm_Validate_Source_Null.C, recovering...
-- 458152 bytes in 18 permanent Pools
compiler(iro
##W IIS-DSEE-TBLD-00000 22:14:29(011) <main_program> Error when checking composite operator: Output from subprocess: pt) warning: skip optimization on void APT_TransformOperatorImplV0S255_VAL_AMB_VW_CONTRACT_to_BIH_STG_JB_Trfm_Validate_Source_Null::processInputRecordChild(int,int*,int*) in /opt/ibm/projects/BIH_GAF_D1/RT_BP90.O/V0S255_VAL_AMB_VW_CONTRACT_to_BIH_STG_JB_Trfm_Validate_Source_Null.C due to out of memory
/opt/ibm/sunstudio12.1/prod/bin/iropt:out_of_memory_handler+0x1a0
/opt/ibm/sunstudio12.1/prod/bin/iropt:heap_tmpalloc+0xc0
/opt/ibm/sunstudio12.1/prod/bin/iropt:new_heap_set+0x3c
/opt/ibm/sunstudio12.1/prod/bi
##W IIS-DSEE-TBLD-00000 22:14:29(012) <main_program> Error when checking composite operator: Output from subprocess: n/iropt:find_dominators+0x1f8
/opt/ibm/sunstudio12.1/prod/bin/iropt:find_dda_dump_loops+0x114
/opt/ibm/sunstudio12.1/prod/bin/iropt:0x29e84c
/opt/ibm/sunstudio12.1/prod/bin/iropt:write_irfile+0x14
/opt/ibm/sunstudio12.1/prod/bin/iropt:main+0x8c8
/opt/ibm/sunstudio12.1/prod/bin/iropt:_start+0x108
compiler(iropt) warning: Out of memory during optimizing void APT_TransformOperatorImplV0S255_VAL_AMB_VW_CONTRACT_to_BIH_STG_JB_Trfm_Validate_Source_Null::processInputRecordChild(int,int*,int*) in /opt/ibm/projects/
##I IIS-DSEE-TBLD-00079 22:14:29(013) <transform> Error when checking composite operator: /opt/ibm/sunstudio12.1/bin/CC -KPIC -O -I/opt/ibm/InformationServer/Server/PXEngine/include -dalign -O -PIC -library=iostream -c -xarch=v9 /opt/ibm/projects/BIH_GAF_D1/RT_BP90.O/V0S255_VAL_AMB_VW_CONTRACT_to_BIH_STG_JB_Trfm_Validate_Source_Null.C -o /opt/ibm/projects/BIH_GAF_D1/RT_BP90.O/V0S255_VAL_AMB_VW_CONTRACT_to_BIH_STG_JB_Trfm_Validate_Source_Null.tmp.o.
##W IIS-DSEE-TBLD-00000 22:14:29(014) <main_program> Error when checking composite operator: Output from subprocess: BIH_GAF_D1/RT_BP90.O/V0S255_VAL_AMB_VW_CONTRACT_to_BIH_STG_JB_Trfm_Validate_Source_Null.C, cannot recover, terminating

##E IIS-DSEE-TCOS-00029 22:14:29(015) <main_program> Creation of a step finished with status = FAILED. (VAL_AMB_VW_CONTRACT_to_BIH_STG_JB.Trfm_Validate_Source_Null)

Running out of memory during compilation. Is your system under-configured?

Hi Ray,

The system is configured properly. Thisis the first time that we ar egetting this issue as we have jobs where the data validations are being done for more than 100 fields.

Thanks

Out of memory for the stage variable, what is the heapsize value?

Does "properly configured" mean that the system meets the minimum documented requirements for the installed tiers and nothing more? If so, you may have hit a situation where the actual usage of the system demands more than the minimum requirements for running the software.

Look into the various options the compiler and linker provide to reduce their memory usage (optimization level, for instance). Also, as Sia suggests, perhaps the heap size for the compiler can be increased, assuming that system resources will allow it.

Regards,

We had the same problem sometime back. Instead of using Stage variable if you switch all your conditions to the derivation part of transformer then it would work fine. I am assuming that you dont have any dependency between the stage variables.

Thanks for showering light on this issue. I will try out the options provided by all of you and let you know the results.

The stage variable logi is implemented and the jobs were split into 3 to make the jobs compiled. After doing this change, the jobs got compiled.
Thank you very much to all for helping.