Page 1 of 1

Query regarding Lookup Stage

Posted: Thu Nov 29, 2007 12:35 am
by sid19
I have one query regarding Lookup Stage.

In the previous post of this forum some body had mentioned the lookup as in-memory lookups .

So I want to know, how many types of lookup present in Lookup Stage, and how do they differe in implemention, in result as well as in performance too.

Thanks & regards,
Sid

Re: Query regarding Lookup Stage

Posted: Thu Nov 29, 2007 1:45 am
by Havoc
sid19 wrote:I have one query regarding Lookup Stage.

In the previous post of this forum some body had mentioned the lookup as in-memory lookups .

So I want to know, how many types of lookup present in Lookup Stage, and how do they differe in implemention, in result as well as in performance too.

Thanks & regards,
Sid
There is only one 'type' of Lookup in a Lookup Stage, but its the implementation of conditional lookups, or handling of data when lookup conditions are not met that will result in variation of the output. I am sure, the
manual that comes along with the installation will be sufficient to answer all those questions.

Performance varies with the hardware, row volume,etc.

Posted: Thu Nov 29, 2007 2:06 am
by ray.wurlod
Most lookups performed by a Lookup stage are "in-memory" lookups, in that the reference table is pre-loaded into a virtual Data Set.

This is manifested as a composite operator consisting of LUT_CreateOp and LUT_ProcessOp operators; the first loads the table, the second uses it to perform lookups. You can see these in the job score.

The only exception is a so-called "sparse lookup", in which the lookup is performed directly against a database table. Sparse lookup is only supported via DB2 and Oracle Enterprise stages.