Page 1 of 1
SCD Stage Operators
Posted: Tue Mar 19, 2013 1:42 pm
by rameshrr3
Does anybody have an idea of what operators are combined for the SCD stage ? Something tells me it uses a combination of lookup and transform operators , but it could also be change capture( with a possible interface to a Generator operator perhaps) . Does anybody have an in-depth understanding of this stage ?
What would be the effect if I disabled combination for this stage?
Thanks
Ramesh
Posted: Tue Mar 19, 2013 2:54 pm
by ray.wurlod
The easiest way to find out is to look at the score for that job.
Then, maybe, you can report your findings back to us.
Posted: Tue Mar 19, 2013 4:17 pm
by rameshrr3
Correct . But Im afraid my job wouldnt compile to get me that far - if it used a transform operator. We have a problem since our xlc compiler ( AIX) is a trial version and expired.
Posted: Tue Mar 19, 2013 4:31 pm
by rameshrr3
An Update : Yes i found much to my dismay it does use a transform operator . I tried compiling a job with SCD stage
and here's the error I got
Code: Select all
##W IIS-DSEE-TBLD-00000 17:29:56(012) <main_program> Error when checking composite operator: Output from subprocess: sh: /usr/vacpp/bin/xlC_r: not found.
##I IIS-DSEE-TBLD-00079 17:29:56(013) <transform> Error when checking composite operator: /usr/vacpp/bin/xlC_r -O -I/datastage/IBM/InformationServer/Server/PXEngine/include -O -q64 -c /datastage/IBM/InformationServer/Server/Projects/CVM_Dev/RT_BP671.O/V0S1_Test_SCD2_Slowly_Changing_Dimension_1.C -o /datastage/IBM/InformationServer/Server/Projects/CVM_Dev/RT_BP671.O/V0S1_Test_SCD2_Slowly_Changing_Dimension_1.tmp.o.
Posted: Tue Mar 19, 2013 6:27 pm
by ray.wurlod
Range lookups also compile to transform operators.
I'd suggest that you need to get a compiler licence.
Posted: Wed Mar 20, 2013 11:38 am
by rameshrr3
Yes, I heard you on the range lookup issue also. Another place where a compiler is a must would be the custom compile flags in the lookup stage.
The compiler license is being checked with IBM. (point to ponder : Looks like the compiler licensing is also based on number of individual cores within a single processor )
That raises an interesting question ( some may consider this off topic) - Does Orchestrate parallelism recognize cores for partitioning data and does it internally execute instructions in parallel on a single processor with multiple cores ?
Posted: Wed Mar 20, 2013 11:39 am
by chulett
It's got nothing to do with cores. It's a logical concept not a physical one. so it's not tied to the processor type or number.
Posted: Wed Mar 20, 2013 1:59 pm
by ray.wurlod
Cores relate only to licensing ("processor value units").
Nodes, and therefore partitioning, are driven purely from what's in the configuration file. As Craig notes, these are a logical concept.