DSXchange

Hi

Is the look up table created by look up stage and the look up table created by look up fileset the same ?

Yes.

The difference is when.

The lookup table (LUT) for a Lookup File Set is created when the Lookup File Set is written to. If any other stage is the producer for a reference input to a Lookup stage, then that LUT and its index is created at run time by the LUT_CreateOp operator.

ray.wurlod wrote:Yes

Thanks Ray, but there appears to be a subtle difference in the way both are treated.

The lookup stage dedups records based on key columns and retains the first record (multiple rows from reference link disabled) and generates a warning for the same but the look up fileset has an option to store duplicates.
when I wrote duplicate records to the look up file set (allow duplicates=False) and viewed the file in hex editor I was able to find both records :s and when the same lookup up file set is used in next job as reference no warning generated.

Also it was my understanding that the key and location mapping table is stored elsewhere than data file but looks like the mapping appears just before data is stored.

The Lookup stage also offers an option that permits return of duplicates on one reference input link. But it is disabled by default.