Getting error compiling transformer in PX Job
Moderators: chulett, rschirm, roy
Getting error compiling transformer in PX Job
Hi to all,
I'm having problems compiling transformer in a PX Job.
The output link is very wide (562 columns!!!) ... does it depend from this or something else?
PLEASE HELP!
Output from transformer compilation follows:
##I TFCN 000001 10:38:14(000) <main_program>
Ascential DataStage(tm) Enterprise Edition 7.5.2
Copyright (c) 2004, 1997-2004 Ascential Software Corporation.
All Rights Reserved
##I TUTL 000031 10:38:14(001) <main_program> The open files limit is 2048; raising to 4096.
##I TOSH 000002 10:38:14(002) <main_program> orchgeneral: loaded
##I TOSH 000002 10:38:14(003) <main_program> orchsort: loaded
##I TOSH 000002 10:38:15(000) <main_program> orchstats: loaded
##I TFSC 000001 10:38:15(003) <main_program> APT configuration file: /software/dsadm/Ascential/DataStage/Configurations/default.apt
##I TUTL 000031 10:33:22(000) <node_node2> The open files limit is 2048; raising to 4096.
##W TCOS 000049 10:38:17(000) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
##E TBLD 000000 10:39:20(000) <main_program> Error when checking composite operator: Subprocess command failed with exit status 768.
##E TFSR 000019 10:39:20(001) <main_program> Could not check all operators because of previous error(s)
##W TFCP 000000 10:39:20(002) <transform> Error when checking composite operator: The number of reject datasets "0" is less than the number of input datasets "1".
##W TFCP 000025 10:39:20(003) <transform> Error when checking composite operator: Converting string to number.
##W TFCP 000000 10:39:20(004) <transform> Error when checking composite operator: Implicit conversion from source type "String" to result type "Int32".
##W TFCP 000025 10:39:20(005) <transform> Error when checking composite operator: Converting string to decimal.
##W TFCP 000000 10:39:20(006) <transform> Error when checking composite operator: Implicit conversion from source type "String" to result type "Decimal".
##W TFCP 000025 10:39:20(007) <transform> Error when checking composite operator: Possible range limitation.
##W TFCP 000000 10:39:20(008) <transform> Error when checking composite operator: Implicit conversion from source type "Int32" to result type "Decimal".
##W TFCP 000025 10:39:20(009) <transform> Error when checking composite operator: Converting string to number.
##W TFCP 000000 10:39:20(010) <transform> Error when checking composite operator: Implicit conversion from source type "String" to result type "Int32".
##W TFCP 000025 10:39:20(011) <transform> Error when checking composite operator: Converting string to number.
##W TFCP 000000 10:39:20(012) <transform> Error when checking composite operator: Implicit conversion from source type "String" to result type "Int64".
##W TFCP 000025 10:39:20(013) <transform> Error when checking composite operator: Converting string to number.
##W TFCP 000000 10:39:20(014) <transform> Error when checking composite operator: Implicit conversion from source type "String" to result type "Int64".
##W TBLD 000000 10:39:20(015) <main_program> Error when checking composite operator: Output from subprocess: aCC: Not enough memory is available to finish the compilation.
##I TFCP 000000 10:39:20(016) <transform> Error when checking composite operator: /opt/aCC/bin/aCC +DD64 +W890 +Z -O -I/software/dsadm/Ascential/DataStage/PXEngine/include +DD64 -O -c -ext -z +Z /data/Progetti/BASE_NELLO/RT_BP1467.O/V18S1_CopyOfTcpOdsConDhbBss020BssMonthlyDownload_TR_BSS.C -o /data/Progetti/BASE_NELLO/RT_BP1467.O/V18S1_CopyOfTcpOdsConDhbBss020BssMonthlyDownload_TR_BSS.tmp.o.
##I TBLD 000000 10:39:20(017) <main_program> Error when checking composite operator: Output from subprocess: MSTAT af
##I TBLD 000000 10:39:20(018) <main_program> Error when checking composite operator: Output from subprocess: ter OutOfMemory error Bytes total=1052760064, in use=1049078160, free=3681904
Pages after OutOfMemory error: Text=8560 Data=256046 Stack
##I TBLD 000000 10:39:20(019) <main_program> Error when checking composite operator: Output from subprocess: =244
##E TCOS 000029 10:39:20(020) <main_program> Creation of a step finished with status = FAILED. (CopyOfTcpOdsConDhbBss020BssMonthlyDownload.TR_BSS)
I'm having problems compiling transformer in a PX Job.
The output link is very wide (562 columns!!!) ... does it depend from this or something else?
PLEASE HELP!
Output from transformer compilation follows:
##I TFCN 000001 10:38:14(000) <main_program>
Ascential DataStage(tm) Enterprise Edition 7.5.2
Copyright (c) 2004, 1997-2004 Ascential Software Corporation.
All Rights Reserved
##I TUTL 000031 10:38:14(001) <main_program> The open files limit is 2048; raising to 4096.
##I TOSH 000002 10:38:14(002) <main_program> orchgeneral: loaded
##I TOSH 000002 10:38:14(003) <main_program> orchsort: loaded
##I TOSH 000002 10:38:15(000) <main_program> orchstats: loaded
##I TFSC 000001 10:38:15(003) <main_program> APT configuration file: /software/dsadm/Ascential/DataStage/Configurations/default.apt
##I TUTL 000031 10:33:22(000) <node_node2> The open files limit is 2048; raising to 4096.
##W TCOS 000049 10:38:17(000) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
##E TBLD 000000 10:39:20(000) <main_program> Error when checking composite operator: Subprocess command failed with exit status 768.
##E TFSR 000019 10:39:20(001) <main_program> Could not check all operators because of previous error(s)
##W TFCP 000000 10:39:20(002) <transform> Error when checking composite operator: The number of reject datasets "0" is less than the number of input datasets "1".
##W TFCP 000025 10:39:20(003) <transform> Error when checking composite operator: Converting string to number.
##W TFCP 000000 10:39:20(004) <transform> Error when checking composite operator: Implicit conversion from source type "String" to result type "Int32".
##W TFCP 000025 10:39:20(005) <transform> Error when checking composite operator: Converting string to decimal.
##W TFCP 000000 10:39:20(006) <transform> Error when checking composite operator: Implicit conversion from source type "String" to result type "Decimal".
##W TFCP 000025 10:39:20(007) <transform> Error when checking composite operator: Possible range limitation.
##W TFCP 000000 10:39:20(008) <transform> Error when checking composite operator: Implicit conversion from source type "Int32" to result type "Decimal".
##W TFCP 000025 10:39:20(009) <transform> Error when checking composite operator: Converting string to number.
##W TFCP 000000 10:39:20(010) <transform> Error when checking composite operator: Implicit conversion from source type "String" to result type "Int32".
##W TFCP 000025 10:39:20(011) <transform> Error when checking composite operator: Converting string to number.
##W TFCP 000000 10:39:20(012) <transform> Error when checking composite operator: Implicit conversion from source type "String" to result type "Int64".
##W TFCP 000025 10:39:20(013) <transform> Error when checking composite operator: Converting string to number.
##W TFCP 000000 10:39:20(014) <transform> Error when checking composite operator: Implicit conversion from source type "String" to result type "Int64".
##W TBLD 000000 10:39:20(015) <main_program> Error when checking composite operator: Output from subprocess: aCC: Not enough memory is available to finish the compilation.
##I TFCP 000000 10:39:20(016) <transform> Error when checking composite operator: /opt/aCC/bin/aCC +DD64 +W890 +Z -O -I/software/dsadm/Ascential/DataStage/PXEngine/include +DD64 -O -c -ext -z +Z /data/Progetti/BASE_NELLO/RT_BP1467.O/V18S1_CopyOfTcpOdsConDhbBss020BssMonthlyDownload_TR_BSS.C -o /data/Progetti/BASE_NELLO/RT_BP1467.O/V18S1_CopyOfTcpOdsConDhbBss020BssMonthlyDownload_TR_BSS.tmp.o.
##I TBLD 000000 10:39:20(017) <main_program> Error when checking composite operator: Output from subprocess: MSTAT af
##I TBLD 000000 10:39:20(018) <main_program> Error when checking composite operator: Output from subprocess: ter OutOfMemory error Bytes total=1052760064, in use=1049078160, free=3681904
Pages after OutOfMemory error: Text=8560 Data=256046 Stack
##I TBLD 000000 10:39:20(019) <main_program> Error when checking composite operator: Output from subprocess: =244
##E TCOS 000029 10:39:20(020) <main_program> Creation of a step finished with status = FAILED. (CopyOfTcpOdsConDhbBss020BssMonthlyDownload.TR_BSS)
Last edited by sunRoy68 on Wed Jul 30, 2008 3:45 am, edited 2 times in total.
You can edit your post to remove all the details, they are not required in this case.
Most likely the code is too big for your installation's optimizer. I would recommend turning off optimization either via your ADMIN tool for all jobs, or to change the compiler options for this particular transform stage. Each UNIX has different options, but I think that "-o0" usually does the trick.
Most likely the code is too big for your installation's optimizer. I would recommend turning off optimization either via your ADMIN tool for all jobs, or to change the compiler options for this particular transform stage. Each UNIX has different options, but I think that "-o0" usually does the trick.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
First few warnings are related to String to decimal, String to number conversions. i think these will have to be handled explicitly. Not sure of about this last warning though.[/code]##W TBLD 000000 10:39:20(015) <main_program> Error when checking composite operator: Output from subprocess: aCC: Not enough memory is available to finish the compilation.
"given enough eyeballs, all bugs are shallow" - Eric S. Raymond
Ignore the other errors, the abort is due to memory. I have had to turn off optimization in large transform stages before.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
The c++ compiler optimization chews up large amounts of resources. For typical transform stage functionality the compiler's optimization won't bring that much performance improvement, so turning off or lowering the optimization level is acceptable.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
Too much for me ... I can't make changes on the compiler.ArndW wrote:The c++ compiler optimization chews up large amounts of resources. For typical transform stage functionality the compiler's optimization won't bring that much performance improvement, so turning off o ...
I solved treating the columns in 4 transformers dividing the flow in "blocks". I know, I'm the first to say that it's absolutely ugly putting 4 transformers in row, but in this case I'm forced because I didn't find other solution.
I hope it's the first and last time that I have to treat a table of 562 columns!!!
Have a nice day.
sunRoy68 ^__^
Sunroy68 - you aren't changing the compiler, just the "compiler options". Depending upon your platform, the standard options will contain something reading "-O" or "-o" and that defaults to some level of optimization. Changing that to "-O0" or "-o0" turns off optimization.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: