Parallel Jobs Complilation problem
Moderators: chulett, rschirm, roy
Parallel Jobs Complilation problem
Hi all,
I have nearly 6 parallel jobs which are recently moved from development environment to Test Environment. all the jobs are working fine development environment, but in test i am unable to compile the jobs.when i try to compile the jobs i encountered with the following errors:
0211: if (null(LNK_Read_Input.ORIG_UPD_CD)) {
0212: NullSetVar0 = 1;
0213: } else {
0214: LNK_Write_Output.ORIG_UPD_CD = LNK_Read_Input.ORIG_UPD_CD;
0215: }
0216: //;
0217: LNK_Write_Output.WTDAVG = LNK_Read_Input.WTDAVG;
0218: LNK_Write_Output.Q1MIN = LNK_Read_Input.Q1MIN;
0219: LNK_Write_Output.Q1MAX = LNK_Read_Input.Q1MAX;
0220: LNK_Write_Output.Q2MAX = LNK_Read_Input.Q2MAX;
0221: LNK_Write_Output.Q3MAX = LNK_Read_Input.Q3MAX;
0222: LNK_Write_Output.Q4MAX = LNK_Read_Input.Q4MAX;
0223: writerecord 0;
0224: RowRejected0 = 0;
0225: }
0226: // evaluate constraint and columns for link: LNK_Inv_Rej
0227: InterVar0_7 = RowRejected0;
0228: if (InterVar0_7)
0229: {
0230: LNK_Inv_Rej.sqlcode = StageVar0_StgVar10;
0231: writerecord 1;
0232: RowRejected0 = 0;
0233: RowCount0_1 = RowCount0_1 + 1;
0234: }
0235: }
0236:
0237: finish {
0238: // Log warnings for any reject links
0239: string LogMsg0;
0240: string LogLink0;
0241: if (RowCount0_1 > 0) {
0242: LogMsg0 = RowCount0_1;
0243: LogLink0 = " rows written to reject link: ";
0244: LogMsg0 = LogMsg0 + LogLink0;
0245: LogLink0 = "LNK_Inv_Rej";
0246: LogMsg0 = LogMsg0 + LogLink0;
0247: print_message(LogMsg0);
0248: }
0249:
0250: }
0251:
*** End of Internal Generated Transformer Code
Previously these jobs were moved to this environment and were working superb.These were deleted few days back from thios environment and once again exported newly,and after this comes the problem with lot power.... and this environment is insalled with C++ compiler.its not an issue with C++ compiler.
plaese advise what would be the issue.
Thanks,
aquilis.
I have nearly 6 parallel jobs which are recently moved from development environment to Test Environment. all the jobs are working fine development environment, but in test i am unable to compile the jobs.when i try to compile the jobs i encountered with the following errors:
0211: if (null(LNK_Read_Input.ORIG_UPD_CD)) {
0212: NullSetVar0 = 1;
0213: } else {
0214: LNK_Write_Output.ORIG_UPD_CD = LNK_Read_Input.ORIG_UPD_CD;
0215: }
0216: //;
0217: LNK_Write_Output.WTDAVG = LNK_Read_Input.WTDAVG;
0218: LNK_Write_Output.Q1MIN = LNK_Read_Input.Q1MIN;
0219: LNK_Write_Output.Q1MAX = LNK_Read_Input.Q1MAX;
0220: LNK_Write_Output.Q2MAX = LNK_Read_Input.Q2MAX;
0221: LNK_Write_Output.Q3MAX = LNK_Read_Input.Q3MAX;
0222: LNK_Write_Output.Q4MAX = LNK_Read_Input.Q4MAX;
0223: writerecord 0;
0224: RowRejected0 = 0;
0225: }
0226: // evaluate constraint and columns for link: LNK_Inv_Rej
0227: InterVar0_7 = RowRejected0;
0228: if (InterVar0_7)
0229: {
0230: LNK_Inv_Rej.sqlcode = StageVar0_StgVar10;
0231: writerecord 1;
0232: RowRejected0 = 0;
0233: RowCount0_1 = RowCount0_1 + 1;
0234: }
0235: }
0236:
0237: finish {
0238: // Log warnings for any reject links
0239: string LogMsg0;
0240: string LogLink0;
0241: if (RowCount0_1 > 0) {
0242: LogMsg0 = RowCount0_1;
0243: LogLink0 = " rows written to reject link: ";
0244: LogMsg0 = LogMsg0 + LogLink0;
0245: LogLink0 = "LNK_Inv_Rej";
0246: LogMsg0 = LogMsg0 + LogLink0;
0247: print_message(LogMsg0);
0248: }
0249:
0250: }
0251:
*** End of Internal Generated Transformer Code
Previously these jobs were moved to this environment and were working superb.These were deleted few days back from thios environment and once again exported newly,and after this comes the problem with lot power.... and this environment is insalled with C++ compiler.its not an issue with C++ compiler.
plaese advise what would be the issue.
Thanks,
aquilis.
Aquilis
Aquilis - you have posted your generated code, but haven't told us what error you are getting at compilation.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
You have also not revealed what's different between the two environments (development and this one). In particular, are the compiler and linker options (environment variables) set the same way?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Oh... sorry about that.
Actual error is;
##I TUTL 000031 01:48:57(001) <main_program> The open files limit is 100; raising to 65536.
##I TOSH 000002 01:48:57(002) <main_program> orchgeneral: loaded
##I TOSH 000002 01:48:58(000) <main_program> orchsort: loaded
##I TOSH 000002 01:48:58(001) <main_program> orchstats: loaded
##I TFSC 000001 01:48:58(004) <main_program> APT configuration file:/ascential/Ascential/DataStage/Configurations/default.apt
##W TFPA 000017 01:48:58(005) <main_program> Warning: the value of the PWD environment variable (....XYZ.....) does not appear to be a synonym for the current working directory (.........abc.......). The current working directory will be used, but if your ORCHESTRATE job does not start up correctly, you should set your PWD environment variable to a value that will work on all nodes of your system.
##I TLIC 000002 01:48:59(000) <APT_LicenseCheckOp in APT_LicenseOperator,0> Node 'bos-stage-7' has 12 physical processors.
##W TLIC 000003 01:48:59(001) <APT_LicenseCheckOp in APT_LicenseOperator,0> DataStage has found more CPU's on your system than your current license allows.
##W TLIC 000004 01:48:59(002) <APT_LicenseCheckOp in APT_LicenseOperator,0> Please contact Ascential Client Support or your Ascential Account Manager.
##W TCOS 000049 01:48:59(000) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
##E TBLD 000000 01:48:59(002) <main_program> Error when checking composite operator: Subprocess command failed with exit status 256.
##E TFSR 000019 01:48:59(003) <main_program> Could not check all operators because of previous error(s)
##W TFCP 000000 01:48:59(004) <transform> Error when checking composite operator: The number of reject datasets "0" is less than the number of input datasets "1".
##W TFCP 000025 01:48:59(005) <transform> Error when checking composite operator: Converting number to string.
##W TFCP 000000 01:48:59(006) <transform> Error when checking composite operator: Implicit conversion from source type "Int32" to result type "String".
##W TFCP 000025 01:48:59(007) <transform> Error when checking composite operator: Converting string to number.
##W TFCP 000000 01:48:59(008) <transform> Error when checking composite operator: Implicit conversion from source type "String" to result type "Int32".
##W TFCP 000025 01:48:59(009) <transform> Error when checking composite operator: Converting number to string.
##W TFCP 000000 01:48:59(010) <transform> Error when checking composite operator: Implicit conversion from source type "UInt64" to result type "String".
##W TBLD 000000 01:48:59(011) <main_program> Error when checking composite operator: Output from subprocess: sh: /opt/SUNWspro/bin/CC: not found
##W TBLD 000000 01:48:59(012) <main_program> Error when checking composite operator: Output from subprocess:
Ray:
all environment variables (compiler & Linkers) are also same for both the environments..
Actual error is;
##I TUTL 000031 01:48:57(001) <main_program> The open files limit is 100; raising to 65536.
##I TOSH 000002 01:48:57(002) <main_program> orchgeneral: loaded
##I TOSH 000002 01:48:58(000) <main_program> orchsort: loaded
##I TOSH 000002 01:48:58(001) <main_program> orchstats: loaded
##I TFSC 000001 01:48:58(004) <main_program> APT configuration file:/ascential/Ascential/DataStage/Configurations/default.apt
##W TFPA 000017 01:48:58(005) <main_program> Warning: the value of the PWD environment variable (....XYZ.....) does not appear to be a synonym for the current working directory (.........abc.......). The current working directory will be used, but if your ORCHESTRATE job does not start up correctly, you should set your PWD environment variable to a value that will work on all nodes of your system.
##I TLIC 000002 01:48:59(000) <APT_LicenseCheckOp in APT_LicenseOperator,0> Node 'bos-stage-7' has 12 physical processors.
##W TLIC 000003 01:48:59(001) <APT_LicenseCheckOp in APT_LicenseOperator,0> DataStage has found more CPU's on your system than your current license allows.
##W TLIC 000004 01:48:59(002) <APT_LicenseCheckOp in APT_LicenseOperator,0> Please contact Ascential Client Support or your Ascential Account Manager.
##W TCOS 000049 01:48:59(000) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
##E TBLD 000000 01:48:59(002) <main_program> Error when checking composite operator: Subprocess command failed with exit status 256.
##E TFSR 000019 01:48:59(003) <main_program> Could not check all operators because of previous error(s)
##W TFCP 000000 01:48:59(004) <transform> Error when checking composite operator: The number of reject datasets "0" is less than the number of input datasets "1".
##W TFCP 000025 01:48:59(005) <transform> Error when checking composite operator: Converting number to string.
##W TFCP 000000 01:48:59(006) <transform> Error when checking composite operator: Implicit conversion from source type "Int32" to result type "String".
##W TFCP 000025 01:48:59(007) <transform> Error when checking composite operator: Converting string to number.
##W TFCP 000000 01:48:59(008) <transform> Error when checking composite operator: Implicit conversion from source type "String" to result type "Int32".
##W TFCP 000025 01:48:59(009) <transform> Error when checking composite operator: Converting number to string.
##W TFCP 000000 01:48:59(010) <transform> Error when checking composite operator: Implicit conversion from source type "UInt64" to result type "String".
##W TBLD 000000 01:48:59(011) <main_program> Error when checking composite operator: Output from subprocess: sh: /opt/SUNWspro/bin/CC: not found
##W TBLD 000000 01:48:59(012) <main_program> Error when checking composite operator: Output from subprocess:
Ray:
all environment variables (compiler & Linkers) are also same for both the environments..
Aquilis
-
- Charter Member
- Posts: 822
- Joined: Sat Sep 17, 2005 5:25 pm
- Location: USA
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Every time I've seen it it's been a licensing issue on the compiler. Some C++ compilers are licensed for only one user at a time, and they behave differently if a second compile is attempted while one is occurring.
You might also like to disable operator combination to find out exactly the source of the error.
You might also like to try adding a reject link to the Transformer stage.
You might also like to disable operator combination to find out exactly the source of the error.
You might also like to try adding a reject link to the Transformer stage.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Just to verify the New environment ,i built a simple job without transformer stage in it and executed and it was executed sucessfully.
Again i built one more job using transformer in it,but this time i was unable to compile my job over here.i end up with the same above mentioned error i.e.
##I TUTL 000031 07:48:42(001) <main_program> The open files limit is 100; raising to 65536.
##I TOSH 000002 07:48:42(002) <main_program> orchgeneral: loaded
##I TOSH 000002 07:48:42(003) <main_program> orchsort: loaded
##I TOSH 000002 07:48:42(004) <main_program> orchstats: loaded
##I TFSC 000001 07:48:42(007) <main_program> APT configuration file: /tech/ascential/Ascential/DataStage/Configurations/default.apt
##W TFPA 000017 07:48:42(008) <main_program> Warning: the value of the PWD environment variable (/tech/ascential/Ascential/DataStage/DSEngine/bin) does not appear to be a synonym for the current working directory (/apps/evdfds/DataStage/evdf). The current working directory will be used, but if your ORCHESTRATE job does not start up correctly, you should set your PWD environment variable to a value that will work on all nodes of your system.
##I TLIC 000002 07:48:43(000) <APT_LicenseCheckOp in APT_LicenseOperator,0> Node 'bos-stage-7' has 12 physical processors.
##W TLIC 000003 07:48:43(001) <APT_LicenseCheckOp in APT_LicenseOperator,0> DataStage has found more CPU's on your system than your current license allows.
##W TLIC 000004 07:48:43(002) <APT_LicenseCheckOp in APT_LicenseOperator,0> Please contact Ascential Client Support or your Ascential Account Manager.
##W TCOS 000049 07:48:43(000) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
##E TBLD 000000 07:48:44(000) <main_program> Error when checking composite operator: Subprocess command failed with exit status 256.
##E TFSR 000019 07:48:44(001) <main_program> Could not check all operators because of previous error(s)
##W TFCP 000000 07:48:44(002) <transform> Error when checking composite operator: The number of reject datasets "0" is less than the number of input datasets "1".
##W TBLD 000000 07:48:44(003) <main_program> Error when checking composite operator: Output from subprocess: sh: /opt/SUNWspro/bin/CC: not found
##I TFCP 000000 07:48:44(004) <transform> Error when checking composite operator: /opt/SUNWspro/bin/CC -KPIC -O -I ascential/Ascential/DataStage/PXEngine/include -dalign -O -PIC -library=iostream -c /apps/DataStage/project1/RT_BP91.O/V0S2_z_test_compilation_Transformer_2.C -o /apps/DataStage/project1/RT_BP91.O/V0S2_z_test_compilation_Transformer_2.tmp.o.
##E TCOS 000029 07:48:44(005) <main_program> Creation of a step finished with status = FAILED. (z_test_compilation.Transformer_2
so my assumption is that ,its a problem with c++ compilers again.
But still i am not able to understand what is the actual problem because as i said above these all the jobs moved to this environment were executed sucessfully over here once before(in this same environment).once these same were deleted and reimported again from the backups,i am not able to compile these things and also now i come to know not only these,but newly developed jobs are also not compiling.
so shall i assume that environment setup has got changed in regard with these compilers.
Again i built one more job using transformer in it,but this time i was unable to compile my job over here.i end up with the same above mentioned error i.e.
##I TUTL 000031 07:48:42(001) <main_program> The open files limit is 100; raising to 65536.
##I TOSH 000002 07:48:42(002) <main_program> orchgeneral: loaded
##I TOSH 000002 07:48:42(003) <main_program> orchsort: loaded
##I TOSH 000002 07:48:42(004) <main_program> orchstats: loaded
##I TFSC 000001 07:48:42(007) <main_program> APT configuration file: /tech/ascential/Ascential/DataStage/Configurations/default.apt
##W TFPA 000017 07:48:42(008) <main_program> Warning: the value of the PWD environment variable (/tech/ascential/Ascential/DataStage/DSEngine/bin) does not appear to be a synonym for the current working directory (/apps/evdfds/DataStage/evdf). The current working directory will be used, but if your ORCHESTRATE job does not start up correctly, you should set your PWD environment variable to a value that will work on all nodes of your system.
##I TLIC 000002 07:48:43(000) <APT_LicenseCheckOp in APT_LicenseOperator,0> Node 'bos-stage-7' has 12 physical processors.
##W TLIC 000003 07:48:43(001) <APT_LicenseCheckOp in APT_LicenseOperator,0> DataStage has found more CPU's on your system than your current license allows.
##W TLIC 000004 07:48:43(002) <APT_LicenseCheckOp in APT_LicenseOperator,0> Please contact Ascential Client Support or your Ascential Account Manager.
##W TCOS 000049 07:48:43(000) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
##E TBLD 000000 07:48:44(000) <main_program> Error when checking composite operator: Subprocess command failed with exit status 256.
##E TFSR 000019 07:48:44(001) <main_program> Could not check all operators because of previous error(s)
##W TFCP 000000 07:48:44(002) <transform> Error when checking composite operator: The number of reject datasets "0" is less than the number of input datasets "1".
##W TBLD 000000 07:48:44(003) <main_program> Error when checking composite operator: Output from subprocess: sh: /opt/SUNWspro/bin/CC: not found
##I TFCP 000000 07:48:44(004) <transform> Error when checking composite operator: /opt/SUNWspro/bin/CC -KPIC -O -I ascential/Ascential/DataStage/PXEngine/include -dalign -O -PIC -library=iostream -c /apps/DataStage/project1/RT_BP91.O/V0S2_z_test_compilation_Transformer_2.C -o /apps/DataStage/project1/RT_BP91.O/V0S2_z_test_compilation_Transformer_2.tmp.o.
##E TCOS 000029 07:48:44(005) <main_program> Creation of a step finished with status = FAILED. (z_test_compilation.Transformer_2
so my assumption is that ,its a problem with c++ compilers again.
But still i am not able to understand what is the actual problem because as i said above these all the jobs moved to this environment were executed sucessfully over here once before(in this same environment).once these same were deleted and reimported again from the backups,i am not able to compile these things and also now i come to know not only these,but newly developed jobs are also not compiling.
so shall i assume that environment setup has got changed in regard with these compilers.
Aquilis
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
It's a reasonable assumption. Only Transformer and Build stages require the C++ compiler.
The fact that /opt/SUNWspro/bin/CC was not found suggests that the compiler is not actually at that location. Find out where it really is, and set APT_COMPILER and APT_LINKER environment variables accordingly.
The fact that /opt/SUNWspro/bin/CC was not found suggests that the compiler is not actually at that location. Find out where it really is, and set APT_COMPILER and APT_LINKER environment variables accordingly.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.