I am trying to understand the partitioning mechanism with DS and cannot seem to get it working properly.
In the Transformer stage I created a Stage Variable containing the init. value as
@PARTITIONNUM - @NUMPARTITIONS + 1 =====> Stored in svCounter
In my stage derivation I put
MAXEVNTID + svCounter + @NUMPARTITIONS
Where MAXEVNTID is the MAX record from the last write from the prior run.
When I look at the key, it is incrementally increase by huge numbers. I wanter to increase by 1 spread out by partition.
Could someone please help me understand what the @PART.. @NUM..
determine and how do I get the patitions correctly increased.
Thanks
Partitioning - Writing to DB2
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
@PARTITIONNUM is the ordinal number of the partition on which the process is executing. Possible value is in the range 0 through (@NUMPARTITIONS - 1).
@NUMPARTITIONS is constant, established by the number of nodes mentioned in the configuration file or in the node pool in which the stage is executing.
These numbers do not control partitioning. That is managed by the selected partitioning algorithm - by default DB2 for the DB2/UDB Enterprise stage.
Are you trying to generate a sequence, or just the partition number? If the latter, then @PARTITIONNUM alone will do it for you. Otherwise you may need @INROWNUM or some other counter in your expression.
Or use the Surrogate Key Generator stage, which generates a unique sequence across all partitions.
Or search the forum for other techniques.
@NUMPARTITIONS is constant, established by the number of nodes mentioned in the configuration file or in the node pool in which the stage is executing.
These numbers do not control partitioning. That is managed by the selected partitioning algorithm - by default DB2 for the DB2/UDB Enterprise stage.
Are you trying to generate a sequence, or just the partition number? If the latter, then @PARTITIONNUM alone will do it for you. Otherwise you may need @INROWNUM or some other counter in your expression.
Or use the Surrogate Key Generator stage, which generates a unique sequence across all partitions.
Or search the forum for other techniques.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 222
- Joined: Tue Aug 30, 2005 2:07 am
- Location: pune
- Contact:
Hi,Raftsman wrote:Thank you very much...I can't tell you how much this has frustrated me. Thank god for this forum..
Its nice to hear that your problem was resolved. But, mark it this post as RESOLVED and it will be nice.
NageshSunkoji
If you know anything SHARE it.............
If you Don't know anything LEARN it...............
If you know anything SHARE it.............
If you Don't know anything LEARN it...............