v8.5 - Error Compiling Parallel Transformer

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Suhel_2112
Premium Member
Premium Member
Posts: 10
Joined: Sun Jul 15, 2012 11:34 pm
Location: Bangalore

v8.5 - Error Compiling Parallel Transformer

Post by Suhel_2112 »

Hi All,

I would appreciate if i can get some help in solving this issue for which i have been looking for information for over 2 days through various forums. :cry: :cry:

I have installed Infosphere Datastage v8.5 on windows environment for testing and have run a couple of jobs, but when i run a job containing a transformer i get compilation error.

Job: Row generator -> Tfx -> Sequential File

Based on my search through various forums i find that Similar issue has been faced by others as well.But i am unable to find a concrete answer regarding the following:
1. A Compiler that i need to install and the related SDK that needs to be installed along with it
2. or A combined pack that has both compiler+SDK

The Installation Guide talks about Microsoft Visual Studio .NET 2008 Express Edition C++: but does not mention which is the associated SDK that needs to be installed ?
or is there a newer version of C++ compiler+SDK that will work on my environment

My Environment Details:
===============

OS - Windows Server 2008 R2(Standard) SP1 - 64 bit

IIS - Infosphere Information Server v8.5 64bit Multilingual

--================
.Net framework
--================

Microsoft .Net framework 1.1
Microsoft .Net framework 4 Client Profile
Microsoft .Net framework 4 Extended

--=====
SDK
--=====
Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 SP1

This SDK created 2 directories path , which are as follows.
C:\Program Files\Microsoft SDKs
C:\Program Files (x86)\Microsoft Visual Studio 9.0


Windows Environment Variables:
=====================

INCLUDE - C:\IBM\SQLLIB\INCLUDE;C:\IBM\SQLLIB\LIB;C:\Program Files\Microsoft SDKs\Windows\v7.0\Include

LIB - C:\Program Files\Microsoft SDKs\Windows\v7.0\Lib;C:\IBM\SQLLIB\LIB

CLASSPATH - .;C:\IBM\SQLLIB\java\db2java.zip;C:\IBM\SQLLIB\java\db2jcc.jar;C:\IBM\SQLLIB\java\sqlj.zip;
C:\IBM\SQLLIB\java\db2jcc_license_cu.jar;C:\IBM\SQLLIB\bin;C:\IBM\SQLLIB\java\common.jar

Path - C:\IBM\InformationServer\Server\DSComponents\bin;C:\Program Files (x86)\MKS Toolkit\mksnt;C:\PROGRA~2\MKSTOO~1\bin64;C:\PROGRA~2\MKSTOO~1\bin;
C:\PROGRA~2\MKSTOO~1\bin\X11;C:\PROGRA~2\MKSTOO~1\mksnt;C:\IBM\InformationServer\ASBNode\apps\jre\bin\classic;C:\IBM\InformationServer\ASBNode\lib\cpp;
C:\IBM\InformationServer\ASBNode\apps\proxy\cpp\vc60\MT_dll\bin;%systemroot%\system32;%systemroot%;
%systemroot%\system32\wbem;%systemroot%\system32\windowspowershell\v1.0\;c:\program files (x86)\microsoft visual studio 9.0\vc\bin;
c:\program files (x86)\microsoft visual studio 9.0\common7\ide;c:\program files\ibm\gsk8\lib64;C:\IBM\SQLLIB\BIN;C:\IBM\SQLLIB\FUNCTION;
C:\IBM\SQLLIB\SAMPLES\REPL;C:\Windows/SysWOW64


Compiler information from windows command prompt:
=================================

c:\Users\skh>cl.exe

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01 for 80x86
Copyright (C) Microsoft Corporation. All rights reserved.

usage: cl [ option... ] filename... [ /link linkoption... ]


Datastage Environment Variables:
=====================

APT_COMPILEOPT (old): -W/TP -W/EHa -DAPT_USE_ANSI_IOSTREAMS -c -W/Zc:wchar_t-
APT_COMPILEOPT (new): -W/TP -W/EHa -DAPT_USE_ANSI_IOSTREAMS -c


APT_LINKOPT(old): -s -W/dll -W/base:0x50000000 -W/Zc:wchar_t-
APT_LINKOPT(new): -s -W/dll -W/base:0x50000000


The (old) represent the default values that DS Environment variables had. I ran my job which had a transformer but i got the compilation error.
I changed the Environment variable values to what is stated in (new), still i get the compilation error mentioned below


******************************************************************************************

Output from transformer compilation follows:

##I IIS-DSEE-TFCN-00001 12:55:36(000) <main_program>
IBM WebSphere DataStage Enterprise Edition 8.5.0.5746
Copyright (c) 2001, 2005-2008 IBM Corporation. All rights reserved



##I IIS-DSEE-TFCN-00006 12:55:36(001) <main_program> conductor uname: -s=Windows_NT; -r=1; -v=6; -n=IN-MUM-IBMQLT; -m=Pentium
##I IIS-DSEE-TOSH-00002 12:55:36(002) <main_program> orchgeneral: loaded
##I IIS-DSEE-TOSH-00002 12:55:36(003) <main_program> orchsort: loaded
##I IIS-DSEE-TOSH-00002 12:55:36(004) <main_program> orchstats: loaded
##W IIS-DSEE-TOSH-00049 12:55:36(007) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
##E IIS-DSEE-TBLD-00076 12:55:39(000) <main_program> Error when checking composite operator: Subprocess command failed with exit status 256.
##E IIS-DSEE-TFSR-00019 12:55:39(001) <main_program> Could not check all operators because of previous error(s)
##W IIS-DSEE-TFTM-00012 12:55:39(002) <transform> Error when checking composite operator: The number of reject datasets "0" is less than the number of input datasets "1".
##I IIS-DSEE-TBLD-00000 12:55:39(003) <main_program> Error when checking composite operator: Output from subprocess: cl : Command line warning D9035 : option 'GX' has been deprecated and will be removed in a future release
cl : Command line warning D9036 : use 'EHsc' instead of 'GX'
cl : Command line warning D9025 : overriding '/GX' with '/EHa'

##I IIS-DSEE-TBLD-00000 12:55:39(004) <main_program> Error when checking composite operator: Output from subprocess: C:\IBM\InformationServer\Server\PXEngine\include\apt_util/ints.h(83) : fatal error C1083: Cannot open include file: 'strstream': No such file or directory

##W IIS-DSEE-TFEV-00025 12:55:39(005) <transform> Error when checking composite operator: Converting string to number.
##W IIS-DSEE-TFEV-00023 12:55:39(006) <transform> Error when checking composite operator: Implicit conversion from source type "String" to result type "DFloat".
##W IIS-DSEE-TFEV-00025 12:55:39(007) <transform> Error when checking composite operator: Converting number to string.
##W IIS-DSEE-TFEV-00023 12:55:39(008) <transform> Error when checking composite operator: Implicit conversion from source type "DFloat" to result type "String".
##W IIS-DSEE-TBLD-00000 12:55:39(009) <main_program> Error when checking composite operator: Output from subprocess: ld: script execution error in file 'C:/PROGRA~2/MKSTOO~1/etc/nutccg/ld.ccg' on line 1119
>>> execv: could not run 'link32': The system cannot find the file specified.

##I IIS-DSEE-TBLD-00079 12:55:39(010) <transform> Error when checking composite operator: cxx -LC:/IBM/InformationServer/Server/Projects/PRJ_QSPOC/RT_BP2.O/ -LC:/IBM/InformationServer/Server/PXEngine/lib -LC:/IBM/InformationServer/Server/PXEngine/user_lib -s -W/dll -W/base:0x50000000 -lliborchnt -lliborchcorent -lliborchbuildopnt C:/IBM/InformationServer/Server/Projects/PRJ_QSPOC/RT_BP2.O/V0S3_TestJob_Tfx_Transformer.tmp.o -o C:/IBM/InformationServer/Server/Projects/PRJ_QSPOC/RT_BP2.O/V0S3_TestJob_Tfx_Transformer.dll.
##E IIS-DSEE-TCOS-00029 12:55:39(011) <main_program> Creation of a step finished with status = FAILED. (TestJob_Tfx.Transformer)

*** Internal Generated Transformer Code follows:
0001: //
0002: // Generated file to implement the V0S3_TestJob_Tfx_Transformer transform operator.
0003: //
0004:
0005: // define our input/output link names
0006: inputname 0 DSLink2;
0007: outputname 0 DSLink5;
0008:
0009: initialize {
0010: // define our control variables
0011: int8 RowRejected0;
0012: int8 NullSetVar0;
0013:
0014: }
0015:
0016: mainloop {
0017:
0018: // declare our intermediate variables for this section
0019: dfloat InterVar0_0;
0020:
0021: // initialise the rejected row variable
0022: RowRejected0 = 1;
0023:
0024: // evaluate columns (no constraints) for link: DSLink5
0025: InterVar0_0 = DSLink2.Dummy1;
0026: DSLink5.Dummy1 = (InterVar0_0 + 5);
0027: writerecord 0;
0028: RowRejected0 = 0;
0029: }
0030:
0031: finish {
0032: }
0033:
*** End of Internal Generated Transformer Code
Thanks & Regards
Suhel Khan

Env Info:
DataStage Version : 8.5
DSEngine Platform : Windows Server 2008 R2
DWH Platform : Oracle 10g
Suhel_2112
Premium Member
Premium Member
Posts: 10
Joined: Sun Jul 15, 2012 11:34 pm
Location: Bangalore

Post by Suhel_2112 »

Hi stuart,

Thanks for responding.

I have installed the following SDK, which I assume is the problem. :(
Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 SP1

I did check the Planning and Installation Guide but I am a bit confused about the SDK to use for Windows Server 2008 R2(SP1) 64 bit, after some amount of searching I got the following link

http://blogs.msdn.com/b/windowssdk/arch ... rsion.aspx

This mentions the following.
Step 1: Install Microsoft Visual C++ 2008 Express.
Step 2: Install the Windows SDK for Windows Server 2008 and .NET Framework 3.5.

If I need to install these , do I need to uninstall the whole IIS suite and Windows 7 SDK or is there a workaround round to just uninstall the Windows 7 SDK and install the above mentioned Express+SDK for Windows Server 2008 and set the INCLUDE and LIB variables accordingly ! :roll:

Also , since my operating system in 64 bit , I am not sure if Microsoft Visual C++ 2008 Express will install a 64 bit compiler , or it would not matter if the compiler is 32 bit or 64 bit :?
Thanks & Regards
Suhel Khan

Env Info:
DataStage Version : 8.5
DSEngine Platform : Windows Server 2008 R2
DWH Platform : Oracle 10g
Suhel_2112
Premium Member
Premium Member
Posts: 10
Joined: Sun Jul 15, 2012 11:34 pm
Location: Bangalore

Post by Suhel_2112 »

This is how i went about fixing it. I hope this helps people facing similar problems and saves some time and effort.

so , here it goes :

I installed Visual Studio C++ 2008 Express Edition.

I then worked around vcvarsall.bat :idea: file .if you are wondering why , please check

(
http://msdn.microsoft.com/en-us/library ... s.80).aspx )

and

http://stackoverflow.com/questions/1359 ... python-2-7

)

I executed the following from the cmd window, to set the 64 bit compiler:
> vcvarsall.bat amd64

NOTE: my environment has a X64 AMD chipset windows server;I used echo %PROCESSOR_ARCHITECTURE% , from cmd window to determine this.

To cross check which compiler is set, execute the following and the output would indicate.
>cl.exe

Next , set the following windows system variables:
INCLUDE - C:\IBM\SQLLIB\INCLUDE;C:\IBM\SQLLIB\LIB;C:\Program Files\Microsoft SDKs\Windows\v6.0A\Include
LIB - C:\Program Files\Microsoft SDKs\Windows\v6.0A\Lib;C:\IBM\SQLLIB\LIB

Note:-
The folder path C:\Program Files\Microsoft SDKs\Windows\v6.0A is created by the Visual Studio C++ 2008 Express Edition. This should be the source for folder paths that are set for INCLUDE & LIB

There is a similar path that the SDK creates C:\Program Files\Microsoft SDKs\Windows\v7.0\ , but Include and Lib folder from this path should not be used to set the above mentioned Windows Environment variables , I made this mistake during my set up :cry:

for reference please check the Release Note of the SDK that is to be installed.

Set the following Datastage ENV variable,from the Administrator, to values below.
APT_COMPILEOPT : -W/TP -W/EHa -DAPT_USE_ANSI_IOSTREAMS -c -W/Zc:wchar_t-
APT_LINKOPT : -s -W/dll -W/base:0x50000000 -W/Zc:wchar_t-


I then restarted the Datastage services.

.. and the transformers started to work :D

PS : I did not uninstall Information Server nor the SDK (which i initially presumed was the problem)
Thanks & Regards
Suhel Khan

Env Info:
DataStage Version : 8.5
DSEngine Platform : Windows Server 2008 R2
DWH Platform : Oracle 10g
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Excellent... thanks for posting that.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply