Duplicate Surrogate Keys
Moderators: chulett, rschirm, roy
Duplicate Surrogate Keys
While running a job using 2 nodes, I encountered an issue with duplicate surrogate keys. A previous collegue created all job stages using sequential processing. In order to take advantage of parallel processing, I switched all stages back to Default (parallel). I used the transformer stage (Next Surrogate()) to create the keys. For some reason, a record on each node was assigned the same surrogate. Is this a known bug or do I need to structure the function differently.
I was thinks about thowing out the transformer stage and replacing it with the surrogate key generator stage. Would this eliminate the problem.
Thanks in advance
I was thinks about thowing out the transformer stage and replacing it with the surrogate key generator stage. Would this eliminate the problem.
Thanks in advance
Jim Stewart
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
The Surrogate Key Generator stage will, like the Transformer stage, do exactly what you tell it to do, though its defaults are more likely to be well behaved.
In a Transformer stage construct your expression using @PARTITIONNUM (plus any initial constant) as the initial value, and increment by @NUMPARTITIONS. This will necessarily yield a unique sequence of numbers.
In a Transformer stage construct your expression using @PARTITIONNUM (plus any initial constant) as the initial value, and increment by @NUMPARTITIONS. This will necessarily yield a unique sequence of numbers.
Last edited by ray.wurlod on Tue Jan 06, 2009 3:30 pm, edited 1 time in total.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 5
- Joined: Sun Jun 25, 2006 12:31 am
- Location: Melbourne, Australia
What is the purpose of the NextSurrogateKey function if it can't control multiple nodes. Since this is a version 8.0 function, I would of assumed it to work correctly. If I understand the internal mechanism, should it not assign unique number even though multiple node are being used. I know it will work correctly if I use one node or sequential processing. Can anyone elaborate on why this function doesn't work correctly. Is there a patch.
Thanks
Thanks
Jim Stewart
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Read again that the job was first run on a single node. I'm guessing, therefore, that something "sequential" has happened, maybe in the mechanism that initializes the state file or something within the function itself.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.