Warnings in the lookup stage

kollurianu · Post by **kollurianu** » Tue Sep 14, 2010 2:56 pm

Hi All ,

I am getting following warnings

lkp_isin,0: Ignoring duplicate entry at table record 6; no further warnings will be issued for this table

lkp_SS_Asset,0: Ignoring duplicate entry at table record 8; no further warnings will be issued for this table

does that mean I am finding duplicate entries in key fields in reference files ie; multiple rows are returned on match criteria in the lookup file.. can some one confirm.

Thank you , appreciate your help.

anbu · Post by **anbu** » Tue Sep 14, 2010 3:07 pm

Yes you have duplicates in the reference files

ggarze · Post by **ggarze** » Fri Jan 28, 2011 1:49 pm

I received the same error when doing a lookup in Parallel. I wrote a small parallel job the read in my lookup file and process it through a Remove Duplicates stage and got the same record count going into the De-Dupe stage as came out. Therefore, duplicates is not the issue. This Warning only started to appear when we made a change to the job that creates the Look-Up file. The change we made was creating the Look-Up file using Entire Partitioning. The job used to be Auto Partitioning. Hope this helps.
T.

XRAY · Post by **XRAY** » Mon Jan 31, 2011 9:18 am

Are the keys in your lookup stage the same as the deduplicate stage ?
Have you verify the uniqueness manually ?

sajal.jain · Post by **sajal.jain** » Mon Jan 31, 2011 12:08 pm

By default the lookup gets the results from the first match found and for subsequent ones gives the warning.

Go to constraint tab on lookup stage and then select the reference link from 'Multiple rows returned from link' drop box - this way it will get the results with all matches with no warnings.

iHijazi · Post by **iHijazi** » Tue Feb 01, 2011 11:58 pm

Or yet better, use join stage. It's much faster and efficient. Trust me, lookup might put in situations where you'll say "Huh?"

Just a thought, and a little bit of experience

ray.wurlod · Post by **ray.wurlod** » Wed Feb 02, 2011 12:00 am

iHijazi wrote:Or yet better, use join stage. It's much faster and efficient.

With even more experience you'll appreciate that there are times when the Lookup stage is more efficient, and that there will be times when you need to use a Lookup stage because there's no other possibility (for example when performing a Range lookup).