Hi,
When Iam running a job which has lookup stage that is looking to db2 table. Iam getting a warning . Please help me in getting out of this warning.
warning message:
Lookup,0: Ignoring duplicate entry at table record 5,216; no further warnings will be issued for this table
Thanks,
Somaraju
Warning message about duplicate entry
Moderators: chulett, rschirm, roy
Hi Kumar,
I selected Enter custom sql in DB2 stage and I provided sql as select Distinct xxx, uuu,yyy,zzzz fom ttttttttt where xxx= '230' . But Iam getting the same warning as I mentioned above. duplicate entry warning. You siad allow duplicate in look up stage .How to do that please guide me. Iam giving key as
xxx and zzzz in lookup
Thanks,
Somaraju
I selected Enter custom sql in DB2 stage and I provided sql as select Distinct xxx, uuu,yyy,zzzz fom ttttttttt where xxx= '230' . But Iam getting the same warning as I mentioned above. duplicate entry warning. You siad allow duplicate in look up stage .How to do that please guide me. Iam giving key as
xxx and zzzz in lookup
Thanks,
Somaraju
somaraju
Hi,
You query extracts the distinct value among the combination of xxx, uuu,yyy,zzzz. Where as your key is only xxx and zzz. Where still there is the possibilites of duplicate of the key.
To ensue the non duplicates try with select count(*) from ttttttt group by xxx, zzz and ensure you are getting all 1s.
And more over i dont recall, that i told regarding the option of allow duplictes in Lookup.
You query extracts the distinct value among the combination of xxx, uuu,yyy,zzzz. Where as your key is only xxx and zzz. Where still there is the possibilites of duplicate of the key.
To ensue the non duplicates try with select count(*) from ttttttt group by xxx, zzz and ensure you are getting all 1s.
And more over i dont recall, that i told regarding the option of allow duplictes in Lookup.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
Somu, You might as well consider using a sort stage between the db2 lookup stage and look you stage. The advantage of using sort stage is two fold.
1) you can remove duplicates
2) Having sorted data will help improve performence.
1) you can remove duplicates
2) Having sorted data will help improve performence.
M.T.Anwer
The day the child realizes that all adults are imperfect he becomes an adolescent;
the day he forgives them, he becomes an adult; the day he forgives himself, he becomes wise.
-Aiden Nowlan
The day the child realizes that all adults are imperfect he becomes an adolescent;
the day he forgives them, he becomes an adult; the day he forgives himself, he becomes wise.
-Aiden Nowlan