Page 1 of 1

Warning on Datastage parallel job on duplicate entry

Posted: Thu Dec 13, 2012 3:45 am
by mayura
Recieved warning on ignoring duplicate entry at table ...no further warnings will be issued for this table.

Solution:
Made the one of the referance link in lookup stage for selecting multiple rows from link and warning has gone.

Query: Is there any impact of data flowed from link after selecting multiple rows from link or it is just removed warning and no impact on data which has been loaded from the link to target. :roll:

Posted: Thu Dec 13, 2012 4:38 am
by ray.wurlod
Yes, you'll get one row on the output for every row returned from the reference input.

Posted: Thu Dec 13, 2012 5:29 am
by mayura
So did this change is good sign to be implement or there is any other way to remove above warning.

As I dont want to return multiple rows for single line. or suggest

Posted: Thu Dec 13, 2012 8:09 am
by chulett
If you want one row being turned into multiple rows after the lookup, then the setting you used allows that to happen. If you do not want that, then either craft your lookup so it only returns a single row so no warning is thrown, or live with the warning as it throws away all of the 'extra' rows from the lookup.

Posted: Thu Dec 13, 2012 8:32 am
by mukejee
go to look up stage > stage prperties > Input >giv partition and then click on uniq. Then u will not get the warning...

Posted: Thu Dec 13, 2012 1:44 pm
by ray.wurlod
Remove Duplicates on the reference input link.

Posted: Tue Dec 18, 2012 1:36 am
by mayura
I did checked table data there were no duplicates hence using remove duplicate stage warning remains the same.

Any other resolultiion please...

Posted: Tue Dec 18, 2012 1:43 am
by jerome_rajan
On what basis did you check if there are duplicates? Is it not possible that there are multiple records for the same value of your lookup keys?

Posted: Fri Dec 21, 2012 6:24 am
by mayura
I did used remove duplicate stage for the records on which warning coming out.

I used the same key column based on what data has been extracted or lookedup in lookup stage.

Is there any possiblity of line break or any junk character into file.

May be i will check data set data one more time..
But please suggestion is open.

Posted: Fri Dec 21, 2012 8:33 am
by chulett
Shouldn't be all that hard to confirm. As noted, the error tells you that based on your lookup key that it found more than one match in the lookup data. So please, check your 'data set' again.