Using OCI stage for lookup

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
Suparv
Participant
Posts: 17
Joined: Wed Aug 25, 2004 9:33 am
Location: Pennsylvania

Using OCI stage for lookup

Post by Suparv »

Hi

I am trying to use a Oracle stage for a lookup, instead of creating a hashfile. Could anyone tell me if this would create any performance issues, on the Database side as well as on the Job?

Thanx
ogmios
Participant
Posts: 659
Joined: Tue Mar 11, 2003 3:40 pm

Re: Using OCI stage for lookup

Post by ogmios »

It depends... If your hash file needs a lot of records (100.000's) and needs a full refresh every run while you only need a couple of 100 rows an OCI lookup on the original data will be much faster than using a hash file.

If you only have a couple of 100 rows in your hash file, it would beat an OCI lookup any time.

When trying to avoid hash files and if the data you want to look up is in the same Oracle database as your primary input, put the lookup in your input sql using "left outer joins". If the data is not in the same database you could e.g. consider a job to first move data to the database where your primary input resides.

Ogmios
In theory there's no difference between theory and practice. In practice there is.
Suparv
Participant
Posts: 17
Joined: Wed Aug 25, 2004 9:33 am
Location: Pennsylvania

Re: Using OCI stage for lookup

Post by Suparv »

Thank you for the suggestion
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

If the Oracle instance is not on the same machine as the DataStage server, the hashed file will win every time. Lookups across networks are very inefficient.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
elavenil
Premium Member
Premium Member
Posts: 467
Joined: Thu Jan 31, 2002 10:20 pm
Location: Singapore

Post by elavenil »

It is not recommended to use DB lookup. Hash file lookup is much faster than DB lookup. As Ray's mentioned, network traffic is one of the reason.

Regards
Saravanan
Post Reply