Warning on Datastage parallel job on duplicate entry

mayura · Post by **mayura** » Thu Dec 13, 2012 3:45 am

Recieved warning on ignoring duplicate entry at table ...no further warnings will be issued for this table.

Solution:
Made the one of the referance link in lookup stage for selecting multiple rows from link and warning has gone.

Query: Is there any impact of data flowed from link after selecting multiple rows from link or it is just removed warning and no impact on data which has been loaded from the link to target.

ray.wurlod · Post by **ray.wurlod** » Thu Dec 13, 2012 4:38 am

Yes, you'll get one row on the output for every row returned from the reference input.

mayura · Post by **mayura** » Thu Dec 13, 2012 5:29 am

So did this change is good sign to be implement or there is any other way to remove above warning.

As I dont want to return multiple rows for single line. or suggest

chulett · Post by **chulett** » Thu Dec 13, 2012 8:09 am

If you want one row being turned into multiple rows after the lookup, then the setting you used allows that to happen. If you do not want that, then either craft your lookup so it only returns a single row so no warning is thrown, or live with the warning as it throws away all of the 'extra' rows from the lookup.

mukejee · Post by **mukejee** » Thu Dec 13, 2012 8:32 am

go to look up stage > stage prperties > Input >giv partition and then click on uniq. Then u will not get the warning...

ray.wurlod · Post by **ray.wurlod** » Thu Dec 13, 2012 1:44 pm

Remove Duplicates on the reference input link.

mayura · Post by **mayura** » Tue Dec 18, 2012 1:36 am

I did checked table data there were no duplicates hence using remove duplicate stage warning remains the same.

Any other resolultiion please...

jerome_rajan · Post by **jerome_rajan** » Tue Dec 18, 2012 1:43 am

On what basis did you check if there are duplicates? Is it not possible that there are multiple records for the same value of your lookup keys?

mayura · Post by **mayura** » Fri Dec 21, 2012 6:24 am

I did used remove duplicate stage for the records on which warning coming out.

I used the same key column based on what data has been extracted or lookedup in lookup stage.

Is there any possiblity of line break or any junk character into file.

May be i will check data set data one more time..
But please suggestion is open.

chulett · Post by **chulett** » Fri Dec 21, 2012 8:33 am

Shouldn't be all that hard to confirm. As noted, the error tells you that based on your lookup key that it found more than one match in the lookup data. So please, check your 'data set' again.