Generating Surrogate keys under PX
Posted: Tue Apr 22, 2003 7:44 am
Hi,
The project I am working on will integrate data coming from various sources not necessarily compatible with each other. For that reason, we need to associate every record we handle with a unique ID number. Since we are using PX, we can't access sdk functions like keymgt inside a job transformer. Therefore, I send my data through a column generator which adds a number to every row starting from 1 to x. I then pass the data inside a transformer adding to the ID column the current ID number (CurID) defined inside am evironment variable. The resulting ID's are something like 1+CurID to x+CurID. Finally, I add x to the CurID and move on the next incoming data file. I am encountering troubles using some functions provided with PX so I was wondering if any of you ever had to design something similar and if yes what was your design, I am afraid mine may be a little complicated. I need to use PX because I am handling millions of rows and can't wait ten times more for a server job.
Thanks for your thought.
MAT
The project I am working on will integrate data coming from various sources not necessarily compatible with each other. For that reason, we need to associate every record we handle with a unique ID number. Since we are using PX, we can't access sdk functions like keymgt inside a job transformer. Therefore, I send my data through a column generator which adds a number to every row starting from 1 to x. I then pass the data inside a transformer adding to the ID column the current ID number (CurID) defined inside am evironment variable. The resulting ID's are something like 1+CurID to x+CurID. Finally, I add x to the CurID and move on the next incoming data file. I am encountering troubles using some functions provided with PX so I was wondering if any of you ever had to design something similar and if yes what was your design, I am afraid mine may be a little complicated. I need to use PX because I am handling millions of rows and can't wait ten times more for a server job.
Thanks for your thought.
MAT