oracle sequence resued for processing.

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

efxuser
Premium Member
Premium Member
Posts: 50
Joined: Tue Jun 24, 2008 9:00 am

oracle sequence resued for processing.

Post by efxuser »

I searched the forum for oracle sequences for surrogate key generator.

My requirement is that I use oracle sequence .nextval to insert into oracle table.Also use these sequence numbers for later use.

I came across so many posts about sparse lookup and getting max sequence number from oracle ,keymanagement functions that now i am totally lost.

can some body kindly detail me steps i should follow for these? I am trying to take a gues shere...

1. Try to find maximum sequence number.
2. use .nextval to insert into oracle
3.Lost here...how do I get these rows along with sequence numbers to be used for further rpocessing?

Even you point me to other posts which i should foillow will also be more helpful.

Also, my admin seems convinced that surrogate key generator stage in ds is not efficient .ANy truth to his claim?

Thanks
[/img]
EFX
betterthanever
Participant
Posts: 152
Joined: Tue Jan 13, 2009 8:59 am

Re: oracle sequence resued for processing.

Post by betterthanever »

you want to use that DB sequence to create the unique number every time you run one particular job or you want to use that DB sequence in other jobs as well???
betterthanever
Participant
Posts: 152
Joined: Tue Jan 13, 2009 8:59 am

Re: oracle sequence resued for processing.

Post by betterthanever »

Code: Select all

                  source
                    |
                    |
                    |
                    |
Ora(Reference)-----Lkp-------Target 
( Lkp type: sparse)
(Read Method : User-Defined sql)
(SQL Query : select sample_seq.nextval as seq_num from dual)
on the Lkp stage map the field seq_num to the ouput field which is unique seq number on the target side
Mike
Premium Member
Premium Member
Posts: 1021
Joined: Sun Mar 03, 2002 6:01 pm
Location: Tampa, FL

Post by Mike »

If given a choice, I never use a database sequence (or identity column). They are too slow and and promote inefficient flow design.

Database sequences are fine for OLTP processing, but add far too much overhead for bulk data processing.

The surrogate key generator stage with a state file is the way to go. The NextSurrogetKey() function is available for use in a transformer stage or SCD stage when and where it is needed for efficient flow design.

The more parallel processing you have, the worse the database sequence bottleneck will become.

Mike
betterthanever
Participant
Posts: 152
Joined: Tue Jan 13, 2009 8:59 am

Post by betterthanever »

i agree....
betterthanever
Participant
Posts: 152
Joined: Tue Jan 13, 2009 8:59 am

Post by betterthanever »

i agree....
efxuser
Premium Member
Premium Member
Posts: 50
Joined: Tue Jun 24, 2008 9:00 am

Post by efxuser »

thanks for your quick response.can you please elaborate about the first stage you have (oracle reference) what is that to do? also in the lookup stage I dont find any option for sparse lookup?
EFX
efxuser
Premium Member
Premium Member
Posts: 50
Joined: Tue Jun 24, 2008 9:00 am

Post by efxuser »

thanks for your quick response.can you please elaborate about the first stage you have (oracle reference) what is that to do? also in the lookup stage I dont find any option for sparse lookup?
EFX
efxuser
Premium Member
Premium Member
Posts: 50
Joined: Tue Jun 24, 2008 9:00 am

Post by efxuser »

thanks for your quick response.can you please elaborate about the first stage you have (oracle reference) what is that to do? also in the lookup stage I dont find any option for sparse lookup?
EFX
betterthanever
Participant
Posts: 152
Joined: Tue Jan 13, 2009 8:59 am

Post by betterthanever »

efxuser wrote:thanks for your quick response.can you please elaborate about the first stage you have (oracle reference) what is that to do? also in the lookup stage I dont find any option for sparse lookup?

can you please elaborate about the first stage you have (oracle reference) what is that to do?----just to fetch the number from DB sequence.

also in the lookup stage I dont find any option for sparse lookup?---you would set this in oracle stage not in lookup stage
betterthanever
Participant
Posts: 152
Joined: Tue Jan 13, 2009 8:59 am

Post by betterthanever »

duplicate post
Last edited by betterthanever on Tue Mar 03, 2009 10:41 am, edited 1 time in total.
betterthanever
Participant
Posts: 152
Joined: Tue Jan 13, 2009 8:59 am

Post by betterthanever »

duplicate post
Last edited by betterthanever on Tue Mar 03, 2009 10:41 am, edited 1 time in total.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Sheesh... posts so good they need to be repeated - three times. Seriously, please start checking after you post and delete any duplicates, especially if it takes 'some time' to complete - that's pretty much a sure sign you'll have duplicate posts.
-craig

"You can never have too many knives" -- Logan Nine Fingers
betterthanever
Participant
Posts: 152
Joined: Tue Jan 13, 2009 8:59 am

Post by betterthanever »

[quote="chulett"]Sheesh... posts so good they need to be repeated - [i]three times.[/i] Seriously, please start checking after you post and delete any duplicates, especially if it takes 'some time' to complete - that's pretty much a sure sign you'll have duplicate posts.[/quote]
can you help me delete the duplicate ones???
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Once they've been replied to, they cannot be deleted, which is why it's important to check right away when you suspect there might have been a problem. Best you could do now would be to edit two of them and just say 'duplicate post' or some such in the body.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply