DSXchange

I am trying to understand the partitioning mechanism with DS and cannot seem to get it working properly.

In the Transformer stage I created a Stage Variable containing the init. value as

@PARTITIONNUM - @NUMPARTITIONS + 1 =====> Stored in svCounter

In my stage derivation I put

MAXEVNTID + svCounter + @NUMPARTITIONS

Where MAXEVNTID is the MAX record from the last write from the prior run.

When I look at the key, it is incrementally increase by huge numbers. I wanter to increase by 1 spread out by partition.

Could someone please help me understand what the @PART.. @NUM..
determine and how do I get the patitions correctly increased.

Thanks

@PARTITIONNUM is the ordinal number of the partition on which the process is executing. Possible value is in the range 0 through (@NUMPARTITIONS - 1).

@NUMPARTITIONS is constant, established by the number of nodes mentioned in the configuration file or in the node pool in which the stage is executing.

These numbers do not control partitioning. That is managed by the selected partitioning algorithm - by default DB2 for the DB2/UDB Enterprise stage.

Are you trying to generate a sequence, or just the partition number? If the latter, then @PARTITIONNUM alone will do it for you. Otherwise you may need @INROWNUM or some other counter in your expression.

Or use the Surrogate Key Generator stage, which generates a unique sequence across all partitions.

Or search the forum for other techniques.

Hi Raftsman,

I generally use -

@PARTITIONNUM + (@NUMPARTITIONS * (@INROWNUM - 1)) + 1 + SeedVal (where SeedVal si the MAX record from the last write from the prior run)

to generate counters/surrogate keys.

The Bird.

Thank you very much...I can't tell you how much this has frustrated me. Thank god for this forum..

Raftsman wrote:Thank you very much...I can't tell you how much this has frustrated me. Thank god for this forum..

Hi,

Its nice to hear that your problem was resolved. But, mark it this post as RESOLVED and it will be nice.