COmpilation Error in DataStage
Moderators: chulett, rschirm, roy
Either add the options to $APT_COMPILEOPT or $APT_LINKOPT, as required per the compiler documentation, in DataStage Administrator (at the project level), add those variables to the job properties and add the options there, or set the compiler options at the stage level on the build tab of the transformer.
The environment variables are documented within the Information Server documentation. The compiler options are documented in the compiler's man pages or online documentation.
Regards,
The environment variables are documented within the Information Server documentation. The compiler options are documented in the compiler's man pages or online documentation.
Regards,
- james wiles
All generalizations are false, including this one - Mark Twain.
All generalizations are false, including this one - Mark Twain.
This Job was working fine till last week and I have made few metadata changes and that I am getting the above compilation error.
My Job is reading 47 different record types and I guess it might be a memory issue because of the large volume and operators.
Total number of columns read from the CFF stage are 2573 (47 different records and 47 Record IDS). I have recently updated the metadata for one of the record that has 50 columns to 250 columns (filler column expanded to multiple column). If I make the columns back to 50 then the job is compiling properly.
My Job is reading 47 different record types and I guess it might be a memory issue because of the large volume and operators.
Total number of columns read from the CFF stage are 2573 (47 different records and 47 Record IDS). I have recently updated the metadata for one of the record that has 50 columns to 250 columns (filler column expanded to multiple column). If I make the columns back to 50 then the job is compiling properly.
The documentation for VisualAge C++ demonstrates how to specify the maxmem and spill options. -qmaxmem=value and -qspill=value
You may also consider disabling compiler optimization as well if modifying the maxmem and spill settings don't resolve the issue. I think it may be -qnooptimize, but verify for your version of VisualAge.
With this many columns, links (and not knowing what your derivations look like), you've likely simply exceeded the default capacity of some of the internal tables the compiler builds while compiling the transformer's C++ code. The same happened to me back in v7.0 on AIX with an input stream with a very high number of columns (somewhere in the 1700+ range). Disabling compiler optimization resolved that particular issue.
Regards,
You may also consider disabling compiler optimization as well if modifying the maxmem and spill settings don't resolve the issue. I think it may be -qnooptimize, but verify for your version of VisualAge.
With this many columns, links (and not knowing what your derivations look like), you've likely simply exceeded the default capacity of some of the internal tables the compiler builds while compiling the transformer's C++ code. The same happened to me back in v7.0 on AIX with an input stream with a very high number of columns (somewhere in the 1700+ range). Disabling compiler optimization resolved that particular issue.
Regards,
- james wiles
All generalizations are false, including this one - Mark Twain.
All generalizations are false, including this one - Mark Twain.
The following setting are done at Project level.
APT_COMPILEOPT=-O -q64 -c
APT_LINKOPT = -G -q64
I have added the $APT_COMPILEOPT environment variable to the specific Job Parameters and assigned the value as follows.
APT_COMPILEOPT=-O0 -q64 -c
The Job has compiled successfully and as per my knowledge this should not have any performance implication at run time. Any advice on how to use these parameters. Would it be good at Job level or at project level. I have kept this at Job level so that these changes are applicable for only this job.
ANother option I am thinking is adding these values in "Build" tab of the Transformer Properties.
APT_COMPILEOPT=-O -q64 -c
APT_LINKOPT = -G -q64
I have added the $APT_COMPILEOPT environment variable to the specific Job Parameters and assigned the value as follows.
APT_COMPILEOPT=-O0 -q64 -c
The Job has compiled successfully and as per my knowledge this should not have any performance implication at run time. Any advice on how to use these parameters. Would it be good at Job level or at project level. I have kept this at Job level so that these changes are applicable for only this job.
ANother option I am thinking is adding these values in "Build" tab of the Transformer Properties.