issue with surrogate key stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
snt_ds
Premium Member
Premium Member
Posts: 280
Joined: Wed Oct 18, 2006 11:53 am
Location: Pune, India

issue with surrogate key stage

Post by snt_ds »

Hi All,

I have a scenario to generate the sequence numbers with surrogate key stage for a sequential file. In version 7.5 i am able to generate the incremental sequence numbers. But in 8.0.1 it is not generating the incremental order.

My job design is File------>Surrogate Key Stage------>File

I have two records in file.It is generating the numbers like 1, 3001. Why it is like that? I think it should be 1 and 2.

It would be great if any one suggest the options or approch how can I achieve the sequence number in incremental order.

Required numbers are 1,2,3,4................................ How can I achieve this?



Thanks in Advance
Vijay
snt_ds
Mike
Premium Member
Premium Member
Posts: 1021
Joined: Sun Mar 03, 2002 6:01 pm
Location: Tampa, FL

Post by Mike »

By making sure the state file is created properly... view the state file before you use it to make sure it looks correct.... you'll have to create a simple utility job and view the value in the job log. How did you create and update your state file? I found using "update" as the action caused issues like you're seeing. I think it's probably a bug. I always use the "create and update" action to build my state files. I have a utility job that deletes a state file and selects the max value from my table that gets passed on an input link into the Surrogate Key Generator stage which recreates the state file before every run.

Don't worry about gaps in surrogate keys. They're unintelligent keys. In parallel processing, you'll always get some gaps... data partitions are rarely skew-free... Even round-robin partitioning will leave gaps when the number of records is not an even multiple of the number of partitions.

Mike
Post Reply