Page 1 of 1

Combinability mode. What is it used for?

Posted: Thu Aug 03, 2006 11:20 pm
by splayer
Can anyone explain? The help says that it saves significant amount of data copying and preparation in passing data between operators. Which operators are we talking about? Stages? How can we save data copying and preparation time?

Posted: Fri Aug 04, 2006 1:17 am
by kumar_s
Each stage can be of single operator or of many. So if you have n stages in a job, with m nodes, you will end up with n*m process running simulteneously.
There are some operators which can be combined during runtime. By allowing the combinability to true you make them to combine and hence, the required memory and resource for each individual process is combined to single.
I wont say if 2 process got combined, the requried cpu and memory will be 1/2, but will be less that the previous.
But you cannot expect very good log out of this, during any fault analysis.
So for any debuggin purpose it is advisable to turn if off.
I recall, there is a discussion on this by roy. Search.
In some cases it acts the other ways too.

Posted: Fri Aug 04, 2006 10:37 am
by richdhan
Hi,

By default the stage operators can be combined. If you want to override for a specfic stage you can set the combinability mode for the stage to Dont Combine.

For Fault analysis you have to use the environment variable APT_DISABLE_COMBINATION to True and run the job. The job log will show exactly which stage is causing the problem.

HTH
--Rich

Posted: Wed Aug 09, 2006 2:30 pm
by splayer
kumar_s, can you explain the statement "Each stage can be of single operator or of many"? What is the difference between an operator and a stage?

Posted: Wed Aug 09, 2006 11:06 pm
by kumar_s
Stages is GUI term used in front end, operator term used in the underlying design of datastage.
For example, sort operator, funnel operator, sortfunnel operator... Can be explicitly available as a stage, or will be supplied by datastage along with other stages like, funnel, join, sequential file Stage, when called implictly.
You can refer Orchestrate Operators Reference for more detials.