Parallel Jobs Complilation problem

Aquilis · Post by **Aquilis** » Tue Nov 20, 2007 6:39 am

Hi all,
I have nearly 6 parallel jobs which are recently moved from development environment to Test Environment. all the jobs are working fine development environment, but in test i am unable to compile the jobs.when i try to compile the jobs i encountered with the following errors:

0211: if (null(LNK_Read_Input.ORIG_UPD_CD)) {
0212: NullSetVar0 = 1;
0213: } else {
0214: LNK_Write_Output.ORIG_UPD_CD = LNK_Read_Input.ORIG_UPD_CD;
0215: }
0216: //;
0217: LNK_Write_Output.WTDAVG = LNK_Read_Input.WTDAVG;
0218: LNK_Write_Output.Q1MIN = LNK_Read_Input.Q1MIN;
0219: LNK_Write_Output.Q1MAX = LNK_Read_Input.Q1MAX;
0220: LNK_Write_Output.Q2MAX = LNK_Read_Input.Q2MAX;
0221: LNK_Write_Output.Q3MAX = LNK_Read_Input.Q3MAX;
0222: LNK_Write_Output.Q4MAX = LNK_Read_Input.Q4MAX;
0223: writerecord 0;
0224: RowRejected0 = 0;
0225: }
0226: // evaluate constraint and columns for link: LNK_Inv_Rej
0227: InterVar0_7 = RowRejected0;
0228: if (InterVar0_7)
0229: {
0230: LNK_Inv_Rej.sqlcode = StageVar0_StgVar10;
0231: writerecord 1;
0232: RowRejected0 = 0;
0233: RowCount0_1 = RowCount0_1 + 1;
0234: }
0235: }
0236:
0237: finish {
0238: // Log warnings for any reject links
0239: string LogMsg0;
0240: string LogLink0;
0241: if (RowCount0_1 > 0) {
0242: LogMsg0 = RowCount0_1;
0243: LogLink0 = " rows written to reject link: ";
0244: LogMsg0 = LogMsg0 + LogLink0;
0245: LogLink0 = "LNK_Inv_Rej";
0246: LogMsg0 = LogMsg0 + LogLink0;
0247: print_message(LogMsg0);
0248: }
0249:
0250: }
0251:
*** End of Internal Generated Transformer Code

Previously these jobs were moved to this environment and were working superb.These were deleted few days back from thios environment and once again exported newly,and after this comes the problem with lot power.... and this environment is insalled with C++ compiler.its not an issue with C++ compiler.
plaese advise what would be the issue.

Thanks,
aquilis.

ArndW · Post by **ArndW** » Tue Nov 20, 2007 7:19 am

Aquilis - you have posted your generated code, but haven't told us what error you are getting at compilation.

ray.wurlod · Post by **ray.wurlod** » Tue Nov 20, 2007 3:05 pm

You have also not revealed what's different between the two environments (development and this one). In particular, are the compiler and linker options (environment variables) set the same way?

Aquilis · Post by **Aquilis** » Wed Nov 21, 2007 1:00 am

Oh... sorry about that.
Actual error is;

##I TUTL 000031 01:48:57(001) <main_program> The open files limit is 100; raising to 65536.
##I TOSH 000002 01:48:57(002) <main_program> orchgeneral: loaded
##I TOSH 000002 01:48:58(000) <main_program> orchsort: loaded
##I TOSH 000002 01:48:58(001) <main_program> orchstats: loaded
##I TFSC 000001 01:48:58(004) <main_program> APT configuration file:/ascential/Ascential/DataStage/Configurations/default.apt
##W TFPA 000017 01:48:58(005) <main_program> Warning: the value of the PWD environment variable (....XYZ.....) does not appear to be a synonym for the current working directory (.........abc.......). The current working directory will be used, but if your ORCHESTRATE job does not start up correctly, you should set your PWD environment variable to a value that will work on all nodes of your system.
##I TLIC 000002 01:48:59(000) <APT_LicenseCheckOp in APT_LicenseOperator,0> Node 'bos-stage-7' has 12 physical processors.
##W TLIC 000003 01:48:59(001) <APT_LicenseCheckOp in APT_LicenseOperator,0> DataStage has found more CPU's on your system than your current license allows.
##W TLIC 000004 01:48:59(002) <APT_LicenseCheckOp in APT_LicenseOperator,0> Please contact Ascential Client Support or your Ascential Account Manager.
##W TCOS 000049 01:48:59(000) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
##E TBLD 000000 01:48:59(002) <main_program> Error when checking composite operator: Subprocess command failed with exit status 256.
##E TFSR 000019 01:48:59(003) <main_program> Could not check all operators because of previous error(s)
##W TFCP 000000 01:48:59(004) <transform> Error when checking composite operator: The number of reject datasets "0" is less than the number of input datasets "1".
##W TFCP 000025 01:48:59(005) <transform> Error when checking composite operator: Converting number to string.
##W TFCP 000000 01:48:59(006) <transform> Error when checking composite operator: Implicit conversion from source type "Int32" to result type "String".
##W TFCP 000025 01:48:59(007) <transform> Error when checking composite operator: Converting string to number.
##W TFCP 000000 01:48:59(008) <transform> Error when checking composite operator: Implicit conversion from source type "String" to result type "Int32".
##W TFCP 000025 01:48:59(009) <transform> Error when checking composite operator: Converting number to string.
##W TFCP 000000 01:48:59(010) <transform> Error when checking composite operator: Implicit conversion from source type "UInt64" to result type "String".
##W TBLD 000000 01:48:59(011) <main_program> Error when checking composite operator: Output from subprocess: sh: /opt/SUNWspro/bin/CC: not found
##W TBLD 000000 01:48:59(012) <main_program> Error when checking composite operator: Output from subprocess:

Ray:
all environment variables (compiler & Linkers) are also same for both the environments..

us1aslam1us · Post by **us1aslam1us** » Wed Nov 21, 2007 1:29 am

First check whether you have user defined environment variable PWD defined or not? It should have complete project path value in it. Also ooking at that message, It seems you have some DS licensing problem too.

Aquilis · Post by **Aquilis** » Wed Nov 21, 2007 2:21 am

Its not all about licencing & PWD settings in it.everything is assigned and its a licenced one.Its just simply throwing the error like that,thats it.

Aquilis · Post by **Aquilis** » Mon Nov 26, 2007 7:27 am

Hi all,
Anybody has faced this kind of problem before, If so please share the solution.

Thank you........

ray.wurlod · Post by **ray.wurlod** » Mon Nov 26, 2007 2:18 pm

Every time I've seen it it's been a licensing issue on the compiler. Some C++ compilers are licensed for only one user at a time, and they behave differently if a second compile is attempted while one is occurring.

You might also like to disable operator combination to find out exactly the source of the error.

You might also like to try adding a reject link to the Transformer stage.

sjordery · Post by **sjordery** » Tue Nov 27, 2007 5:31 am

Check the Parallel job compiler options in the project's Environment Variables (using Administrator) to ensure that they are identical for both the old and new environments. Are both servers using the same OS?

Regards,
S

Aquilis · Post by **Aquilis** » Wed Nov 28, 2007 3:43 am

project's Environment Variables are identical for both the old and new environments. And both servers using the same OS (ie.Sunsolaris) also.

Aquilis · Post by **Aquilis** » Wed Dec 19, 2007 7:35 am

Just to verify the New environment ,i built a simple job without transformer stage in it and executed and it was executed sucessfully.
Again i built one more job using transformer in it,but this time i was unable to compile my job over here.i end up with the same above mentioned error i.e.

##I TUTL 000031 07:48:42(001) <main_program> The open files limit is 100; raising to 65536.
##I TOSH 000002 07:48:42(002) <main_program> orchgeneral: loaded
##I TOSH 000002 07:48:42(003) <main_program> orchsort: loaded
##I TOSH 000002 07:48:42(004) <main_program> orchstats: loaded
##I TFSC 000001 07:48:42(007) <main_program> APT configuration file: /tech/ascential/Ascential/DataStage/Configurations/default.apt
##W TFPA 000017 07:48:42(008) <main_program> Warning: the value of the PWD environment variable (/tech/ascential/Ascential/DataStage/DSEngine/bin) does not appear to be a synonym for the current working directory (/apps/evdfds/DataStage/evdf). The current working directory will be used, but if your ORCHESTRATE job does not start up correctly, you should set your PWD environment variable to a value that will work on all nodes of your system.
##I TLIC 000002 07:48:43(000) <APT_LicenseCheckOp in APT_LicenseOperator,0> Node 'bos-stage-7' has 12 physical processors.
##W TLIC 000003 07:48:43(001) <APT_LicenseCheckOp in APT_LicenseOperator,0> DataStage has found more CPU's on your system than your current license allows.
##W TLIC 000004 07:48:43(002) <APT_LicenseCheckOp in APT_LicenseOperator,0> Please contact Ascential Client Support or your Ascential Account Manager.
##W TCOS 000049 07:48:43(000) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
##E TBLD 000000 07:48:44(000) <main_program> Error when checking composite operator: Subprocess command failed with exit status 256.
##E TFSR 000019 07:48:44(001) <main_program> Could not check all operators because of previous error(s)
##W TFCP 000000 07:48:44(002) <transform> Error when checking composite operator: The number of reject datasets "0" is less than the number of input datasets "1".
##W TBLD 000000 07:48:44(003) <main_program> Error when checking composite operator: Output from subprocess: sh: /opt/SUNWspro/bin/CC: not found

##I TFCP 000000 07:48:44(004) <transform> Error when checking composite operator: /opt/SUNWspro/bin/CC -KPIC -O -I ascential/Ascential/DataStage/PXEngine/include -dalign -O -PIC -library=iostream -c /apps/DataStage/project1/RT_BP91.O/V0S2_z_test_compilation_Transformer_2.C -o /apps/DataStage/project1/RT_BP91.O/V0S2_z_test_compilation_Transformer_2.tmp.o.
##E TCOS 000029 07:48:44(005) <main_program> Creation of a step finished with status = FAILED. (z_test_compilation.Transformer_2

so my assumption is that ,its a problem with c++ compilers again.
But still i am not able to understand what is the actual problem because as i said above these all the jobs moved to this environment were executed sucessfully over here once before(in this same environment).once these same were deleted and reimported again from the backups,i am not able to compile these things and also now i come to know not only these,but newly developed jobs are also not compiling.
so shall i assume that environment setup has got changed in regard with these compilers.

ray.wurlod · Post by **ray.wurlod** » Wed Dec 19, 2007 3:58 pm

It's a reasonable assumption. Only Transformer and Build stages require the C++ compiler.

The fact that /opt/SUNWspro/bin/CC was not found suggests that the compiler is not actually at that location. Find out where it really is, and set APT_COMPILER and APT_LINKER environment variables accordingly.