Page 1 of 1
Query regarding Lookup Stage
Posted: Thu Nov 29, 2007 12:35 am
by sid19
I have one query regarding Lookup Stage.
In the previous post of this forum some body had mentioned the lookup as in-memory lookups .
So I want to know, how many types of lookup present in Lookup Stage, and how do they differe in implemention, in result as well as in performance too.
Thanks & regards,
Sid
Re: Query regarding Lookup Stage
Posted: Thu Nov 29, 2007 1:45 am
by Havoc
sid19 wrote:I have one query regarding Lookup Stage.
In the previous post of this forum some body had mentioned the lookup as in-memory lookups .
So I want to know, how many types of lookup present in Lookup Stage, and how do they differe in implemention, in result as well as in performance too.
Thanks & regards,
Sid
There is only one 'type' of Lookup in a Lookup Stage, but its the implementation of conditional lookups, or handling of data when lookup conditions are not met that will result in variation of the output. I am sure, the
manual that comes along with the installation will be sufficient to answer all those questions.
Performance varies with the hardware, row volume,etc.
Posted: Thu Nov 29, 2007 2:06 am
by ray.wurlod
Most lookups performed by a Lookup stage are "in-memory" lookups, in that the reference table is pre-loaded into a virtual Data Set.
This is manifested as a composite operator consisting of LUT_CreateOp and LUT_ProcessOp operators; the first loads the table, the second uses it to perform lookups. You can see these in the job score.
The only exception is a so-called "sparse lookup", in which the lookup is performed directly against a database table. Sparse lookup is only supported via DB2 and Oracle Enterprise stages.