Compilation error help using Transformer/Peek

VCInDSX · Post by **VCInDSX** » Thu Mar 13, 2008 11:14 am

I am trying to do a simple "One-Record-Generator" using a transformer that sends the output to a peek stage. I did a few searches and found Craig's suggestion of Dummy transformer to an output stage

Background:-
The original requirement is to have one record sent to a stored procedure. As that was not compiling successfully, i decided to keep it simple to see if i can generate a record from a transformer and push it to a peek stage.
Also another exercise with a different approach provide futile viewtopic.php?t=117212

Current job design

Code: Select all

Transformer ==> Peek.

Transformer settings
Constraint - @INROWNUM=1
2 stage variables - TestIntSV and TestStringSV (Dummy for now)

I have 3 job parameters that will should be sent to the PEEK via transformer. If this works, i will replace the PEEK with the Stored Procedure Stage. I have 3 output columns (Types Int, Varchar and Date) defined in the transformer and they are linked to the appropriate job parameters.

When I compile this job i get the following error listing (Pardon the detailed text as i did not want to omit anything that might help someone trying to help me)
Output from transformer compilation follows:

Code: Select all

##I XXXXXB 000001 <osh_conductor> Checking Authorization...
##I XXXXXE 000001 <osh_conductor> Using dynamic configuration script: /nfsgrid/nfsbin/IBM/InformationServer/Server/grid_enabled.3.5.1/Dynamic_grid.sh
##I XXXXXF 000001 <osh_conductor> Authorized to proceed.
##I IIS-DSEE-TFCN-00001 12:21:26(000) <main_program> 
IBM WebSphere DataStage Enterprise Edition 8.0.1.4668 
Copyright (c) 2001, 2005-2007 IBM Corporation. All rights reserved
 


##I IIS-DSEE-TOSH-00002 12:21:26(001) <main_program> orchgeneral: loaded
##I IIS-DSEE-TOSH-00002 12:21:26(002) <main_program> orchsort: loaded
##I IIS-DSEE-TOSH-00002 12:21:26(003) <main_program> orchstats: loaded
##W IIS-DSEE-TOSH-00049 12:21:26(006) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
##W IIS-DSEE-TOSH-00049 12:21:26(007) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
##I IIS-DSEE-TCOS-00022 12:21:26(009) <main_program> Explanation:
Step has 1 operator.
???, invoked with args: -outputschema record ( IntDataVal: int32;  StrDataVal: string[max=50];  DateDataVal: date; ) -expressionfile RT_SC96/V0S0_OneRecordGenerator_XFM_PEEK_Job_XFM_Pass_Params_To_Peek.trx -flag compile -compiler g++ -compileopt -O -fPIC -Wno-deprecated -c -linker g++ -linkopt -shared -Wl,-Bsymbolic,--allow-shlib-undefined -name V0S0_OneRecordGenerator_XFM_PEEK_Job_XFM_Pass_Params_To_Peek -dir /nfsgrid/nfsbin/IBM/InformationServer/Server/Projects/GEM_dev/RT_BP96.O 

Step has 0 data entities.
.
##I IIS-DSEE-TCOS-00023 12:21:26(010) <main_program> Dump:
{ 
  text="transform -outputschema record ( IntDataVal: int32;  StrDataVal: string[max=50];  DateDataVal: date; )\n-expressionfile RT_SC96/V0S0_OneRecordGenerator_XFM_PEEK_Job_XFM_Pass_Params_To_Peek.trx\n-flag compile\n-compiler 'g++'\n-compileopt '-O -fPIC -Wno-deprecated -c'\n-linker 'g++'\n-linkopt '-shared -Wl,-Bsymbolic,--allow-shlib-undefined'\n-name V0S0_OneRecordGenerator_XFM_PEEK_Job_XFM_Pass_Params_To_Peek\n-dir /nfsgrid/nfsbin/IBM/InformationServer/Server/Projects/GEM_dev/RT_BP96.O", 
  line=1, column=1, name="", qualname="", 
  op={ 
       text="transform -outputschema record ( IntDataVal: int32;  StrDataVal: string[max=50];  DateDataVal: date; )\n-expressionfile RT_SC96/V0S0_OneRecordGenerator_XFM_PEEK_Job_XFM_Pass_Params_To_Peek.trx\n-flag compile\n-compiler 'g++'\n-compileopt '-O -fPIC -Wno-deprecated -c'\n-linker 'g++'\n-linkopt '-shared -Wl,-Bsymbolic,--allow-shlib-undefined'\n-name V0S0_OneRecordGenerator_XFM_PEEK_Job_XFM_Pass_Params_To_Peek\n-dir /nfsgrid/nfsbin/IBM/InformationServer/Server/Projects/GEM_dev/RT_BP96.O", 
       line=1, column=1, name=transform, qualname=transform, 
       wrapout={},
       wrapperfile=transform, kind=non_wrapper_cdi_op, exec_mode=none, 
       args="'record ( IntDataVal: int32;  StrDataVal: string[max=50];  DateDataVal: date; )'-expressionfile'RT_SC96/V0S0_OneRecordGenerator_XFM_PEEK_Job_XFM_Pass_Params_To_Peek.trx'-flag'compile'-compiler'g++'-compileopt'-O -fPIC -Wno-deprecated -c'-linker'g++'-linkopt'-shared -Wl,-Bsymbolic,--allow-shlib-undefined'-name'V0S0_OneRecordGenerator_XFM_PEEK_Job_XFM_Pass_Params_To_Peek'-dir'/nfsgrid/nfsbin/IBM/InformationServer/Server/Projects/GEM_dev/RT_BP96.O'"
     }
}
.
##E IIS-DSEE-TFTM-00006 12:21:26(011) <transform> Error when checking composite operator:  The -flag=compile option was used, but no input schema was specified.
##E IIS-DSEE-TFSR-00019 12:21:26(012) <main_program> Could not check all operators because of previous error(s)
##E IIS-DSEE-TCOS-00029 12:21:26(013) <main_program> Creation of a step finished with status = FAILED. (OneRecordGenerator_XFM_PEEK_Job.XFM_Pass_Params_To_Peek)

*** Internal Generated Transformer Code follows:
0001: //
0002: // Generated file to implement the V0S0_OneRecordGenerator_XFM_PEEK_Job_XFM_Pass_Params_To_Peek transform operator.
0003: //
0004: 
0005: // define our input/output link names
0006: outputname 0 XFM_To_Peek_Lnk;
0007: 
0008: global {
0009: 	// Job parameter declaration
0010: 	ustring TestIntData;
0011: 	ustring TestStringData;
0012: 	ustring TestDateData;
0013: }
0014: 
0015: initialize {
0016: 	// define and initialise our input row count variable
0017: 	uint64 InRowNumber0;
0018: 	InRowNumber0 = 0;
0019: 
0020: 	// define our row rejected variable
0021: 	int8 RowRejected0;
0022: 
0023: 	// define our null set variable
0024: 	int8 NullSetVar0;
0025: 
0026: 	// Stage variable declaration and initialisation
0027: 	int32 StageVar0_TestIntSV;
0028: 	StageVar0_TestIntSV = TestIntData;
0029: 	string StageVar0_TestStringSV;
0030: 	StageVar0_TestStringSV = TestStringData;
0031: }
0032: 
0033: mainloop {
0034: 	// initialise our row rejected variable
0035: 	RowRejected0 = 1;
0036: 
0037: 	// increment our input row count
0038: 	InRowNumber0 = InRowNumber0 + 1;
0039: 
0040: 	// evaluate the stage variables first
0041: 
0042: 	// evaluate constraint and columns for link: XFM_To_Peek_Lnk
0043: 	if ((InRowNumber0 == 1))
0044: 	{
0045: 		XFM_To_Peek_Lnk.IntDataVal = TestIntData;
0046: 		XFM_To_Peek_Lnk.StrDataVal = TestStringData;
0047: 		XFM_To_Peek_Lnk.DateDataVal = date_from_ustring(TestDateData);
0048: 		writerecord 0;
0049: 		RowRejected0 = 0;
0050: 	}
0051: }
0052: 
0053: finish {
0054: }
0055: 
*** End of Internal Generated Transformer Code

I see 2 major pointers from this message.
1. ##W IIS-DSEE-TOSH-00049 12:21:26(006) <main_program> Parameter specified but not used in flow: DSPXWorkingDir

I did a search on this and found that most people had this due to an incorrect CC compiler path/settings on the server.

I have a few other jobs on this server that contain transfomer stages and they work fine. Could it be that in this particular combination (DummyTransformer ==> Peek) that the compiler is getting jittery.
Current Compiler Settings:-
APT_COMPILE_OPT -O -fPIC -Wno-deprecated -c
APT_COMPILER g++
APT_LINKER g++
APT_LINKPOT -shared -Wl,-Bsymbolic,--allow-shlib-undefined

Item# 2. ##E IIS-DSEE-TFTM-00006 12:21:26(011) <transform> Error when checking composite operator: The -flag=compile option was used, but no input schema was specified.
I am not sure what the second error points to.

I would greatly appreciate your help with these 2 errors.

Let me know if you need any additional details on this one.

Thanks in advance for your time,

kumar_s · Post by **kumar_s** » Thu Mar 13, 2008 3:10 pm

##E IIS-DSEE-TFTM-00006 12:21:26(011) <transform> Error when checking composite operator: The -flag=compile option was used, but no input schema was specified.

It works only Server Jobs no in PX.

ArndW · Post by **ArndW** » Fri Mar 14, 2008 3:43 am

In a PX job just use a "row generator" as the transform stage as a source will not work (unless you put in a BASIC transform stage).

ray.wurlod · Post by **ray.wurlod** » Fri Mar 14, 2008 3:47 am

I try to avoid using a parallel job to process one row. I prefer a server job. If the client insists on all parallel jobs I try to talk them out of it. If they still insist, I make sure it uses a single-node configuration file - any more is wasted.

VCInDSX · Post by **VCInDSX** » Fri Mar 14, 2008 8:27 am

Hi Kumar, ArndW & Ray,
Thanks for taking the time to review and help. Appreciate it.
That was a good lesson I learned from your feedback.

ArndW,
I tried the RowGenerator approach and got hit by a bug in the Row Generator stage when dealing with Dates viewtopic.php?t=117212
I did not go through the String to Date conversion in that design as i thought i will try the Transformer to Stored procedure and then go there. But then again, as Ray pointed out, if i were to do in PX, i would have to limit myself to a single node config.

Right now, i tried the Server job design with the same 3 parameters (String, Int and Date). I ran into an error message
STP_HW3_Test3_Job..Stored_Procedure_1: At row 1, link "DSLink2", while processing column "DateValue"
Value treated as NULL
Attempt to convert String value "2008-03-14" to Date type unsuccessful

I am sure I set the job parameter's type to Date and the transformer column to Date and the target STP column to Date. Not sure how this is treated as string in between.

I recall from the PX error message that the underlying schema is all ustring for the job parameters.

Code: Select all

0008: global { 
0009:    // Job parameter declaration 
0010:    ustring TestIntData; 
0011:    ustring TestStringData; 
0012:    ustring TestDateData; 
0013: }

Is this a common way in Datastage when dealing with parameters? Should i resort to IConv & OConv Pair to resolve this?

Thanks again,

ArndW · Post by **ArndW** » Fri Mar 14, 2008 8:39 am

I am not sure why you didn't use a row-generator stage to feed a transform stage which would produce your date values in the PX job.

Server jobs don't have the foggiest what a "date" type is -> the parameter is treated as a string and you need to do an appropriate ICONV() on the string to get the internal date format.

VCInDSX · Post by **VCInDSX** » Fri Mar 14, 2008 9:40 am

ArndW,
Apologies for the confusion caused. The ROW Generator has a bug (confirmed by IBM as Case # 96505) when it is used to generate custom dates ref. viewtopic.php?t=117212

I am trying to handle this via a server job for now. I have requested our DS Admin to provide a singe node config to try out an alternate approach suggested by Ray (sometime next week, i hope)

I will let the group know the outcome of IConv.

Thanks

ArndW · Post by **ArndW** » Fri Mar 14, 2008 9:46 am

VC - I realize that bug; but what I am saying is use the row generator to generate just one row with a dummy column which you can then throw away, but output that through a transform stage where you do generate your correct date column.