Problem using Convert in Transformer

betterthanever · Post by **betterthanever** » Tue Feb 03, 2009 2:12 pm

if the volume of data is not high ...i would suggest to go for a server job
and use ereplace funtion for this string matching OR
use the c++ program mentioned in the earlier post for the parallel version of ereplace function

krishna81 · Post by **krishna81** » Tue Feb 03, 2009 3:15 pm

Guru's i have used above programme but i am facing the below error while compile time.

##I IIS-DSEE-TFCN-00001 15:02:40(000) <main_program>
IBM WebSphere DataStage Enterprise Edition 8.0.1.5075
Copyright (c) 2001, 2005-2007 IBM Corporation. All rights reserved

##I IIS-DSEE-TUTL-00031 15:02:40(001) <main_program> The open files limit is 100; raising to 4096.
##I IIS-DSEE-TOSH-00002 15:02:40(002) <main_program> orchgeneral: loaded
##I IIS-DSEE-TOSH-00002 15:02:40(003) <main_program> orchsort: loaded
##I IIS-DSEE-TOSH-00002 15:02:40(004) <main_program> orchstats: loaded
##W IIS-DSEE-TOSH-00049 15:02:40(007) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
Local variables
record
( )
Input schema 0
record
( col: string[32700];
)
Output schema 0
record
( col: string[max=32700];
)
Output schema 1
record
( innum: int32;
)
##W IIS-DSEE-TFTM-00012 15:02:41(000) <transform> Error when checking composite operator: The number of reject datasets "0" is less than the number of input datasets "1".
##E IIS-DSEE-TFPS-00003 15:02:41(001) <transform> Error when checking composite operator: Unexpected tokens: String; [line 6, character 31].
##E IIS-DSEE-TFPS-00038 15:02:41(002) <transform> Error when checking composite operator: Expected semi-colon; [line 6, character 37].
##E IIS-DSEE-TFPS-00012 15:02:41(003) <transform> Error when checking composite operator: Invalid local variable declaration: string; [line 6, character 52].
##E IIS-DSEE-TFPS-00038 15:02:41(004) <transform> Error when checking composite operator: Expected semi-colon; [line 6, character 66].
##E IIS-DSEE-TFPS-00012 15:02:41(005) <transform> Error when checking composite operator: Invalid local variable declaration: int32; [line 6, character 77].
##E IIS-DSEE-TFPS-00038 15:02:41(006) <transform> Error when checking composite operator: Expected semi-colon; [line 6, character 86].
##E IIS-DSEE-TFPS-00038 15:02:41(007) <transform> Error when checking composite operator: Expected semi-colon; [line 9, character -1].
##E IIS-DSEE-TFPS-00038 15:02:41(008) <transform> Error when checking composite operator: Expected semi-colon; [line 9, character 11].
##E IIS-DSEE-TFPS-00038 15:02:41(009) <transform> Error when checking composite operator: Expected semi-colon; [line 10, character 0].
##E IIS-DSEE-TFPS-00038 15:02:41(010) <transform> Error when checking composite operator: Expected semi-colon; [line 10, character 13].
##E IIS-DSEE-TFPS-00038 15:02:41(011) <transform> Error when checking composite operator: Expected semi-colon; [line 11, character 0].
##E IIS-DSEE-TFPS-00038 15:02:41(012) <transform> Error when checking composite operator: Expected semi-colon; [line 11, character 13].
##E IIS-DSEE-TFSR-00019 15:02:41(013) <main_program> Could not check all operators because of previous error(s)
##E IIS-DSEE-TCOS-00029 15:02:41(014) <main_program> Creation of a step finished with status = FAILED.
[]: ROOT <> (line 0, char 0, len 0)
[.]: EXPR <> (line 6, char 37, len 0)
[.]: NONE <string> (line 6, char 38, len 6)
[.]: EXPR <> (line 6, char 59, len 0)
[r]: NAME <Replace> (line 6, char 59, len 7)
[.]: NONE <int32> (line 6, char 67, len 5)
[.]: EXPR <> (line 6, char 83, len 0)
[r]: NAME <Beg> (line 6, char 83, len 3)
[.]: SEMI <;> (line 6, char 87, len 1)
[.]: EXPR <> (line 9, char -1, len 0)
[.]: EXPR <> (line 9, char 9, len 0)
[r]: CONSTANT <0> (line 9, char 9, len 1)
[.]: SEMI <;> (line 9, char 18, len 1)
[.]: EXPR <> (line 10, char 0, len 0)
[.]: EXPR <> (line 10, char 11, len 0)
[r]: CONSTANT <0> (line 10, char 11, len 1)
[.]: SEMI <;> (line 10, char 20, len 1)
[.]: EXPR <> (line 11, char 0, len 0)
[.]: EXPR <> (line 11, char 11, len 0)
[r]: CONSTANT <1> (line 11, char 11, len 1)
[.]: SEMI <;> (line 11, char 21, len 1)
[.]: INIT <initialize> (line 13, char 0, len 10)
[r]: NONE <int8> (line 15, char 0, len 4)
[.]: NONE <int8> (line 18, char 0, len 4)
[.]: NONE <uint64> (line 21, char 0, len 6)
[.]: EXPR <> (line 22, char 1, len 0)
[r]: ASSIGN <=> (line 22, char 13, len 1)
[l]: NAME <RowCount0_1> (line 22, char 1, len 11)
[r]: CONSTANT <0> (line 22, char 15, len 1)
[.]: NONE <string> (line 25, char 0, len 6)
[.]: NONE <string> (line 26, char 1, len 6)
[.]: NONE <string> (line 27, char 1, len 6)
[.]: EXPR <> (line 30, char 0, len 0)
[r]: ASSIGN <=> (line 30, char 12, len 1)
[l]: NAME <InterVar0_0> (line 30, char 0, len 11)
[r]: CONSTANT <"inputstring inutstring inputstring"> (line 30, char 14, len 36)
[.]: EXPR <> (line 31, char 1, len 0)
[r]: ASSIGN <=> (line 31, char 13, len 1)
[l]: NAME <InterVar0_1> (line 31, char 1, len 11)
[r]: CONSTANT <"input"> (line 31, char 15, len 7)
[.]: EXPR <> (line 32, char 1, len 0)
[r]: ASSIGN <=> (line 32, char 13, len 1)
[l]: NAME <InterVar0_2> (line 32, char 1, len 11)
[r]: CONSTANT <"sub"> (line 32, char 15, len 5)
[.]: MAIN <mainloop> (line 35, char 0, len
[r]: EXPR <> (line 37, char 0, len 0)
[r]: ASSIGN <=> (line 37, char 13, len 1)
[l]: NAME <RowRejected0> (line 37, char 0, len 12)
[r]: CONSTANT <1> (line 37, char 15, len 1)
[.]: NONE <string> (line 40, char 0, len 6)
[.]: NONE <int32> (line 41, char 1, len 5)
[.]: EXPR <> (line 44, char 0, len 0)
[r]: ASSIGN <=> (line 44, char 12, len 1)
[l]: NAME <InterVar0_3> (line 44, char 0, len 11)
[r]: CALL <pxEreplace> (line 44, char 14, len 10)
[r]: EXPR <> (line 44, char 25, len 0)
[r]: NAME <InterVar0_0> (line 44, char 25, len 11)
[.]: EXPR <> (line 44, char 39, len 0)
[r]: NAME <InterVar0_1> (line 44, char 39, len 11)
[.]: EXPR <> (line 44, char 53, len 0)
[r]: NAME <InterVar0_2> (line 44, char 53, len 11)
[.]: EXPR <> (line 44, char 67, len 0)
[r]: CONSTANT <0> (line 44, char 67, len 1)
[.]: EXPR <> (line 44, char 71, len 0)
[r]: CONSTANT <1> (line 44, char 71, len 1)
[.]: EXPR <> (line 45, char 1, len 0)
[r]: ASSIGN <=> (line 45, char 13, len 1)
[l]: NAME <DSLink9> (line 45, char 1, len 7)
[.]: NAME <col> (line 45, char 9, len 3)
[r]: NAME <InterVar0_3> (line 45, char 15, len 11)
[.]: EXPR <> (line 46, char 1, len 0)
[r]: WRITERECORD <writerecord> (line 46, char 1, len 11)
[r]: CONSTANT <0> (line 46, char 13, len 1)
[.]: EXPR <> (line 47, char 1, len 0)
[r]: ASSIGN <=> (line 47, char 14, len 1)
[l]: NAME <RowRejected0> (line 47, char 1, len 12)
[r]: CONSTANT <0> (line 47, char 16, len 1)
[.]: EXPR <> (line 49, char 0, len 0)
[r]: ASSIGN <=> (line 49, char 12, len 1)
[l]: NAME <InterVar0_4> (line 49, char 0, len 11)
[r]: EXPR <> (line 49, char 15, len 0)
[r]: PLUS <+> (line 49, char 27, len 1)
[l]: NAME <RowCount0_1> (line 49, char 15, len 11)
[r]: CONSTANT <1> (line 49, char 29, len 1)
[.]: EXPR <> (line 50, char 1, len 0)
[r]: ASSIGN <=> (line 50, char 16, len 1)
[l]: NAME <DSLink11> (line 50, char 1, len
[.]: NAME <innum> (line 50, char 10, len 5)
[r]: NAME <InterVar0_4> (line 50, char 18, len 11)
[.]: EXPR <> (line 51, char 1, len 0)
[r]: WRITERECORD <writerecord> (line 51, char 1, len 11)
[r]: CONSTANT <1> (line 51, char 13, len 1)
[.]: EXPR <> (line 52, char 1, len 0)
[r]: ASSIGN <=> (line 52, char 14, len 1)
[l]: NAME <RowRejected0> (line 52, char 1, len 12)
[r]: CONSTANT <0> (line 52, char 16, len 1)
[.]: EXPR <> (line 53, char 1, len 0)
[r]: ASSIGN <=> (line 53, char 13, len 1)
[l]: NAME <RowCount0_1> (line 53, char 1, len 11)
[r]: PLUS <+> (line 53, char 27, len 1)
[l]: NAME <RowCount0_1> (line 53, char 15, len 11)
[r]: CONSTANT <1> (line 53, char 29, len 1)
[.]: FINISH <finish> (line 56, char 0, len 6)
[.]: EOF <> (line 59, char 0, len 0)
[]: NONE <> (line 0, char 0, len 0)
Symbol Table (CFF_TST1.Transformer_8)

*** Internal Generated Transformer Code follows:
0001: //
0002: // Generated file to implement the V1S1_CFF_TST1_Transformer_8 transform operator.
0003: //
0004:
0005: // define external functions used
0006: extern string pxEreplace(string String,string Substr,string Replace,int32 Num,int32 Beg);
0007:
0008: // define our input/output link names
0009: inputname 0 DSLink3;
0010: outputname 0 DSLink9;
0011: outputname 1 DSLink11;
0012:
0013: initialize {
0014: // define our row rejected variable
0015: int8 RowRejected0;
0016:
0017: // define our null set variable
0018: int8 NullSetVar0;
0019:
0020: // define and initialise each link row count variable required
0021: uint64 RowCount0_1;
0022: RowCount0_1 = 0;
0023:
0024: // declare our intermediate variables for this section
0025: string InterVar0_0;
0026: string InterVar0_1;
0027: string InterVar0_2;
0028:
0029: // initialise constant values which require conversion
0030: InterVar0_0 = "inputstring inutstring inputstring";
0031: InterVar0_1 = "input";
0032: InterVar0_2 = "sub";
0033: }
0034:
0035: mainloop {
0036: // initialise our row rejected variable
0037: RowRejected0 = 1;
0038:
0039: // declare our intermediate variables for this section
0040: string InterVar0_3;
0041: int32 InterVar0_4;
0042:
0043: // evaluate columns (no constraints) for link: DSLink9
0044: InterVar0_3 = pxEreplace(InterVar0_0 , InterVar0_1 , InterVar0_2 , 0 , 1);
0045: DSLink9.col = InterVar0_3;
0046: writerecord 0;
0047: RowRejected0 = 0;
0048: // evaluate columns (no constraints) for link: DSLink11
0049: InterVar0_4 = (RowCount0_1 + 1);
0050: DSLink11.innum = InterVar0_4;
0051: writerecord 1;
0052: RowRejected0 = 0;
0053: RowCount0_1 = RowCount0_1 + 1;
0054: }
0055:
0056: finish {
0057: }
0058:
*** End of Internal Generated Transformer Code

betterthanever · Post by **betterthanever** » Tue Feb 03, 2009 3:18 pm

did you fill in all the options correctly when creating a routine in the designer??

chulett · Post by **chulett** » Tue Feb 03, 2009 3:25 pm

krishna81 - please go back and delete your duplicate post from the FAQ Discussions forum before someone replies to it.

krishna81 · Post by **krishna81** » Tue Feb 03, 2009 3:29 pm

yes betterthanever ,
i have kept every thing in correct options.

krishna81 · Post by **krishna81** » Tue Feb 03, 2009 3:31 pm

The code is working fine when executed.But not working in datastage.

betterthanever · Post by **betterthanever** » Tue Feb 03, 2009 3:35 pm

post the values you filled in when creating the parallel routine in designer...

1. general tab
2. in the arguments tab.

i tested it ..and it works for me

krishna81 · Post by **krishna81** » Tue Feb 03, 2009 3:43 pm

I just deleted and recreated the routine.Right now it is not giving error in compile time but giving in runtime.

Transformer_8: Failed to load the library "V1S1_CFF_TST1_Transformer_8.so"; either the directory containing the library file
is not on the library search path, or the library was compiled on a system
that is incompatible with this system: Could not load "V1S1_CFF_TST1_Transformer_8": ld.so.1: osh: fatal: relocation error: file /opt/etl/Projects/ETL/RT_BP2996.O/V1S1_CFF_TST1_Transformer_8.so: symbol __1cKpxEreplace6Fpc00ii_0_: referenced symbol not found.

Here are the arguments i have given
Arguments I/Otype Native type
Str I char*
subStr I char*
rep I char*
num I int
beg I int

betterthanever · Post by **betterthanever** » Tue Feb 03, 2009 3:47 pm

post what you filled in the general tab...

kittu.raja · Post by **kittu.raja** » Tue Feb 03, 2009 4:06 pm

The volume is very huge. 8 million records. I cant go with server

krishna81 · Post by **krishna81** » Tue Feb 03, 2009 4:08 pm

Here are my enteries in general tab.

Routine name =pxEreplace
Type = External Function
Object Type = Object
External subroutine name = pxEreplace
Return Type =char*
Library Path =/code/ETL/dsroutine/ pxEreplace.o

betterthanever · Post by **betterthanever** » Tue Feb 03, 2009 4:09 pm

check the parallel ereplace mentioned in the other post

betterthanever · Post by **betterthanever** » Tue Feb 03, 2009 4:30 pm

did you move the .o file from the directory where you compiled it???

chulett · Post by **chulett** » Tue Feb 03, 2009 4:32 pm

kittu.raja wrote:The volume is very huge. 8 million records. I cant go with server

Baloney. I routinely do multi-million record volumes in Server... and you're just going flat to flat? Should fly.

krishna81 · Post by **krishna81** » Tue Feb 03, 2009 4:40 pm

No, I did not moved .o file to any where.but my make file copying file to prior directory.

DSXchange

Problem using Convert in Transformer

PXEreplace

Re: PXEreplace