Page 1 of 1

Parallel job compilation error with a transformer in the job

Posted: Mon Jul 20, 2009 2:50 am
by Alethesnake
Hi All,
I read a lot of things about this problem in the forum but I was not able to solve the problem by myself.

This is the error I get (from the more button after an unsuccessful compilation):

Code: Select all

Output from transformer compilation follows:

##I IIS-DSEE-TFCN-00001 10:25:16(000) <main_program> 
IBM WebSphere DataStage Enterprise Edition 8.1.0.5040 
Copyright (c) 2001, 2005-2008 IBM Corporation. All rights reserved
 


##I IIS-DSEE-TFCN-00006 10:25:16(001) <main_program> conductor uname: -s=AIX; -r=3; -v=5; -n=svil; -m=00C87F7F4C00
##I IIS-DSEE-TOSH-00002 10:25:16(002) <main_program> orchgeneral: loaded
##I IIS-DSEE-TOSH-00002 10:25:16(003) <main_program> orchsort: loaded
##I IIS-DSEE-TOSH-00002 10:25:16(004) <main_program> orchstats: loaded
##W IIS-DSEE-TOSH-00049 10:25:17(000) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
##E IIS-DSEE-TBLD-00076 10:25:20(000) <main_program> Error when checking composite operator: Subprocess command failed with exit status 256.
##E IIS-DSEE-TFSR-00019 10:25:20(001) <main_program> Could not check all operators because of previous error(s)
##W IIS-DSEE-TFTM-00012 10:25:20(002) <transform> Error when checking composite operator: The number of reject datasets "0" is less than the number of input datasets "1".
##W IIS-DSEE-TBLD-00000 10:25:20(003) <main_program> Error when checking composite operator: Output from subprocess: munch: The input file /sw/IBM/InformationServer/Server/PXEngine/lib/liborchbuildopaix64.so is not valid in the current object mode.

##I IIS-DSEE-TBLD-00079 10:25:20(004) <transform> Error when checking composite operator: /usr/vacpp/bin/xlC_r    -L/sw/IBM/InformationServer/Server/Projects/DATAWAREHOUSE_CEDACRI/RT_BP255.O/ -L/sw/IBM/InformationServer/Server/PXEngine/lib -L/sw/IBM/InformationServer/Server/PXEngine/user_lib -G -lorchaix64 -lorchcoreaix64 -lorchbuildopaix64 /sw/IBM/InformationServer/Server/Projects/DATAWAREHOUSE_CEDACRI/RT_BP255.O/V0S0_PAR_001_EnvVariableInTransformer_Transformer_0.tmp.o -o /sw/IBM/InformationServer/Server/Projects/DATAWAREHOUSE_CEDACRI/RT_BP255.O/V0S0_PAR_001_EnvVariableInTransformer_Transformer_0.o.
##E IIS-DSEE-TCOS-00029 10:25:20(005) <main_program> Creation of a step finished with status = FAILED. (PAR_001_EnvVariableInTransformer.Transformer_0)

*** Internal Generated Transformer Code follows:
0001: //
0002: // Generated file to implement the V0S0_PAR_001_EnvVariableInTransformer_Transformer_0 transform operator.
0003: //
0004: 
0005: // define our input/output link names
0006: inputname 0 DSLink3;
0007: outputname 0 DSLink4;
0008: 
0009: initialize {
0010: 	// define our row rejected variable
0011: 	int8 RowRejected0;
0012: 
0013: 	// define our null set variable
0014: 	int8 NullSetVar0;
0015: 
0016: }
0017: 
0018: mainloop {
0019: 	// initialise our row rejected variable
0020: 	RowRejected0 = 1;
0021: 
0022: 	// evaluate columns (no constraints) for link: DSLink4
0023: 	writerecord 0;
0024: 	RowRejected0 = 0;
0025: }
0026: 
0027: finish {
0028: }
0029: 
*** End of Internal Generated Transformer Code
DS Information Server 8.1 is installed on an AIX 5.3 server.

lslpp -L | grep -i xlc gives:

Code: Select all

  xlC.adt.include            8.0.0.0    C     F    C Set ++ Application
  xlC.aix50.rte             10.1.0.0    C     F    XL C/C++ Runtime for AIX 5.3
  xlC.cpp                    9.0.0.0    C     F    C for AIX Preprocessor
  xlC.msg.en_US.cpp          9.0.0.0    C     F    C for AIX Preprocessor
  xlC.msg.en_US.rte         10.1.0.0    C     F    XL C/C++ Runtime
  xlC.rte                   10.1.0.0    C     F    XL C/C++ Runtime
and from the ibm web site the compiler versions availables are:

Code: Select all

- May 2008 XL C/C++ Enterprise Edition V8.0 for AIX PTF, Reference #4019338
- August 2008 XL C/C++ Enterprise Edition V9.0 for AIX, Reference #4020144 
- October 2008 XL C/C++ Enterprise Edition V10.1 for AIX, Reference #4021392
APT_LINKER and APT_COMPILER are set to /usr/vacpp/bin/xlC_r

If I execute /usr/vacpp/bin/xlC_r from the command line the compiler answers with its information page.
Compiler options are also set correctly.

Any suggestion on how can I solve the problem?

Thanks,

Ale.

Posted: Mon Jul 20, 2009 3:19 am
by ArndW
Your APT_COMPILEOPT should be "-O -c" and APT_LINKOPT should be "-G". Can you execute the compile from the command line? I also see that the error refers to "munch" instead of directly to the compiler.

Posted: Mon Jul 20, 2009 3:36 am
by Alethesnake
Hi ArndW,
Compiler options are those you mentioned.

If I execute this from command line:

Code: Select all

/usr/vacpp/bin/xlC_r -L/sw/IBM/InformationServer/Server/Projects/DATAWAREHOUSE_CEDACRI/RT_BP255.O/ -L/sw/IBM/InformationServer/Server/PXEngine/lib -L/sw/IBM/InformationServer/Server/PXEngine/user_lib -G -lorchaix64 -lorchcoreaix64 -lorchbuildopaix64 /sw/IBM/InformationServer/Server/Projects/DATAWAREHOUSE_CEDACRI/RT_BP255.O/V0S0_PAR_001_EnvVariableInTransformer_Transformer_0.tmp.o -o /sw/IBM/InformationServer/Server/Projects/DATAWAREHOUSE_CEDACRI/RT_BP255.O/V0S0_PAR_001_EnvVariableInTransformer_Transformer_0.o
I get an error:
/usr/vacpp/bin/xlC_r: 1501-228 input file /sw/IBM/InformationServer/Server/Projects/DATAWAREHOUSE_CEDACRI/RT_BP255.O/V0S0_PAR_001_EnvVariableInTransformer_Transformer_0.tmp.o not found
executing:

Code: Select all

dump -ohv /sw/IBM/InformationServer/Server/PXEngine/lib/liborchbuildopaix64.so
I get the same error mentioned in the report above:
dump: /sw/IBM/InformationServer/Server/PXEngine/lib/liborchbuildopaix64.so: 0654-108 file is not valid in the current object file mode.
Use the -X option to specify the desired object mode.

Posted: Mon Jul 20, 2009 4:50 am
by ArndW
Hmmm... that object should be a 32 bit one, in my opinion. I am working on 8.0.1 so can't answer this problem, which looks like a 8.1 specific one. On my system I only have "liborchbuildopaix3.so" in that subdir, no "liborchbuildopaix64.so"

Posted: Mon Jul 20, 2009 9:07 am
by Alethesnake
The problem was effettively linked to the 64bit objects linked to ds 8.1.
On last friday (friday the 17th :) ) we migrated from version 8.0.1 to 8.1 and we restored into the new installation all the old environment variables set before the migration.

The problem was that compiler options are slightly for 8.1 version:

Code: Select all

APT_COMPILEOPT=-O -q64 -c
APT_LINKOPT=-G -q64
Everything is ok now, thanks for support.

Bye,

Ale.

Posted: Wed Dec 02, 2009 6:51 pm
by trokosz
So are you saying the following options worked or didn't?

APT_COMPILEOPT=-O -q64 -c
APT_LINKOPT=-G -q64

I have the same ones and get the same error.....So is the solution to remove the -q64 as someone indicated? or what did you actually do to fix?

I migrated nothing yet.....Thanks

Posted: Thu Dec 03, 2009 11:03 am
by Alethesnake
Hi trokosz,

The -q64 parameter indicates the compiler to run in 64bit mode. Was your system booted in 64bit mode? (bootinfo -K)

Regards,

Alessandro.

Posted: Thu Dec 03, 2009 7:46 pm
by trokosz
OK, I resolved....The OS was AIX 6.1 (64.bit) and IS 8.1 and XLC 10.1 so the 10.1 XLC plus adding the -q64 in this environemnt solved the issue. Thanks

Posted: Thu Dec 03, 2009 7:47 pm
by trokosz
OK, I resolved....The OS was AIX 6.1 (64.bit) and IS 8.1 and XLC 10.1 so the 10.1 XLC plus adding the -q64 in this environemnt solved the issue. Thanks

Posted: Tue May 25, 2010 3:12 pm
by hsahay
trokosz wrote:OK, I resolved....The OS was AIX 6.1 (64.bit) and IS 8.1 and XLC 10.1 so the 10.1 XLC plus adding the -q64 in this environemnt solved the issue. Thanks
Could you please give me the details of the environmental variables.

We were facing the same issue in the same environment as you have resolved on
OS was AIX 6.1 (64.bit) and IS 8.1 and XLC 10.1

but still facing the compilation errors while using the values that you have mentioned in the above post:
APT_COMPILEOPT=-O -q64 -c
APT_COMPILER=/usr/vacpp/bin/xlC-r
APT_LINKER=/usr/vacpp/bin/xlC-r
APT_LINKOPT=-G -q64


Could you please check the above values as you have resolved!

Thanks-

Posted: Tue May 25, 2010 3:55 pm
by vinnz
hsahay wrote: APT_COMPILER=/usr/vacpp/bin/xlC-r
APT_LINKER=/usr/vacpp/bin/xlC-r
Is it a typo in your post or do you have a hyphen(-) in place of an underscore(_) ? I believe it needs to be /usr/vacpp/bin/xlC_r