DSXchange

Hi All ,

I am getting following warnings

lkp_isin,0: Ignoring duplicate entry at table record 6; no further warnings will be issued for this table

lkp_SS_Asset,0: Ignoring duplicate entry at table record 8; no further warnings will be issued for this table

does that mean I am finding duplicate entries in key fields in reference files ie; multiple rows are returned on match criteria in the lookup file.. can some one confirm.

Thank you , appreciate your help.

Yes you have duplicates in the reference files

I received the same error when doing a lookup in Parallel. I wrote a small parallel job the read in my lookup file and process it through a Remove Duplicates stage and got the same record count going into the De-Dupe stage as came out. Therefore, duplicates is not the issue. This Warning only started to appear when we made a change to the job that creates the Look-Up file. The change we made was creating the Look-Up file using Entire Partitioning. The job used to be Auto Partitioning. Hope this helps.
T.

Are the keys in your lookup stage the same as the deduplicate stage ?
Have you verify the uniqueness manually ?

By default the lookup gets the results from the first match found and for subsequent ones gives the warning.

Go to constraint tab on lookup stage and then select the reference link from 'Multiple rows returned from link' drop box - this way it will get the results with all matches with no warnings.

Or yet better, use join stage. It's much faster and efficient. Trust me, lookup might put in situations where you'll say "Huh?"

Just a thought, and a little bit of experience

iHijazi wrote:Or yet better, use join stage. It's much faster and efficient.

With even more experience you'll appreciate that there are times when the Lookup stage is more efficient, and that there will be times when you need to use a Lookup stage because there's no other possibility (for example when performing a Range lookup).