Getting irregular values after using Surrogate Key Generator

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
sateeshbabu
Participant
Posts: 48
Joined: Sat Oct 22, 2005 9:56 am

Getting irregular values after using Surrogate Key Generator

Post by sateeshbabu »

Hi,
Our team is designing a job in which we are using surrogate key generator [In parallel mode] .It is being used to generate value for the key column "User_ID".

Problem ::

After using this stage , the User_ID does not appear in sequence.


We want it to be in a sequence.

PLEASE SUGGEST SOME SOLUTION SO THAT THE USER_ID COULD BE OBTAINED IN A SEQUENCE USING PARALLEL MODE.

** The mode of execution can be made sequential,but it will slow down the performance.Performance is a big issue FOR US because we have to deal with a large number of records.
Last edited by sateeshbabu on Sun Mar 16, 2008 10:39 pm, edited 1 time in total.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

You can never get them in sequence in parallel mode. Think about it. With two nodes, one side is generating the odd numbers, the other node is generating the even numbers. Assuming both nodes are operating at the same speed, the output would be in sequence: 0,1,2,3,4,...

But with variations in speed of execution, the order will be garbled.

If you want them to be sorted, sort them. But there's a throughput cost to doing so.

It does not matter that they're out of order. The definition of a surrogate key is that it provides uniqueness, nothing more. Nor can you control the order of storage in a database table - it will put rows wherever it decides to.

For the same reason gaps in sequences of surrogate keys, caused by different numbers of rows processed on different nodes (probably because of the partitioning algorithm selected) do not matter either.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply