SCD Stage Operators

rameshrr3 · Post by **rameshrr3** » Tue Mar 19, 2013 1:42 pm

Does anybody have an idea of what operators are combined for the SCD stage ? Something tells me it uses a combination of lookup and transform operators , but it could also be change capture( with a possible interface to a Generator operator perhaps) . Does anybody have an in-depth understanding of this stage ?

What would be the effect if I disabled combination for this stage?

Thanks
Ramesh

ray.wurlod · Post by **ray.wurlod** » Tue Mar 19, 2013 2:54 pm

The easiest way to find out is to look at the score for that job.

Then, maybe, you can report your findings back to us.

rameshrr3 · Post by **rameshrr3** » Tue Mar 19, 2013 4:17 pm

Correct . But Im afraid my job wouldnt compile to get me that far - if it used a transform operator. We have a problem since our xlc compiler ( AIX) is a trial version and expired.

rameshrr3 · Post by **rameshrr3** » Tue Mar 19, 2013 4:31 pm

An Update : Yes i found much to my dismay it does use a transform operator . I tried compiling a job with SCD stage
and here's the error I got

Code: Select all

##W IIS-DSEE-TBLD-00000 17:29:56(012) <main_program> Error when checking composite operator: Output from subprocess: sh: /usr/vacpp/bin/xlC_r:  not found.

##I IIS-DSEE-TBLD-00079 17:29:56(013) <transform> Error when checking composite operator: /usr/vacpp/bin/xlC_r   -O   -I/datastage/IBM/InformationServer/Server/PXEngine/include -O -q64 -c /datastage/IBM/InformationServer/Server/Projects/CVM_Dev/RT_BP671.O/V0S1_Test_SCD2_Slowly_Changing_Dimension_1.C -o /datastage/IBM/InformationServer/Server/Projects/CVM_Dev/RT_BP671.O/V0S1_Test_SCD2_Slowly_Changing_Dimension_1.tmp.o.

ray.wurlod · Post by **ray.wurlod** » Tue Mar 19, 2013 6:27 pm

Range lookups also compile to transform operators.

I'd suggest that you need to get a compiler licence.

rameshrr3 · Post by **rameshrr3** » Wed Mar 20, 2013 11:38 am

Yes, I heard you on the range lookup issue also. Another place where a compiler is a must would be the custom compile flags in the lookup stage.

The compiler license is being checked with IBM. (point to ponder : Looks like the compiler licensing is also based on number of individual cores within a single processor )

That raises an interesting question ( some may consider this off topic) - Does Orchestrate parallelism recognize cores for partitioning data and does it internally execute instructions in parallel on a single processor with multiple cores ?

chulett · Post by **chulett** » Wed Mar 20, 2013 11:39 am

It's got nothing to do with cores. It's a logical concept not a physical one. so it's not tied to the processor type or number.

ray.wurlod · Post by **ray.wurlod** » Wed Mar 20, 2013 1:59 pm

Cores relate only to licensing ("processor value units").

Nodes, and therefore partitioning, are driven purely from what's in the configuration file. As Craig notes, these are a logical concept.