Hi I am joining two tables , If I use lookup stage, due to volume of data it getting aborted, because of data limit is exceeding 4 gb.
that is why I am replacing Joiner over there. If I use joiner I am getting warning and job getting fail.
error
Join_67: When checking operator: On input data set 1: When binding input interface field "BSNS_SGMNT_ID" to field "BSNS_SGMNT_ID": Converting a nullable source to a non-nullable result;
a fatal runtime error could occur;
use a modify operator to specify the value to which the null should be converted.
for the BSNS_SGMNT_ID column left link is not null , right link is null (yes)
we are doing left outer join.
If I use lookup some times job getting success.
Replacing lookup with Joiner
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 457
- Joined: Tue Sep 25, 2007 4:05 pm
Re: Replacing lookup with Joiner
Think about it - if you are doing a left outer join and a column in the left link is NOT NULLABLE, then why would you want that to be NULL in the output of the join stage? Have it as NOT NULL too.deva wrote:Hi I am joining two tables , If I use lookup stage, due to volume of data it getting aborted, because of data limit is exceeding 4 gb.
that is why I am replacing Joiner over there. If I use joiner I am getting warning and job getting fail.
error
Join_67: When checking operator: On input data set 1: When binding input interface field "BSNS_SGMNT_ID" to field "BSNS_SGMNT_ID": Converting a nullable source to a non-nullable result;
a fatal runtime error could occur;
use a modify operator to specify the value to which the null should be converted.
for the BSNS_SGMNT_ID column left link is not null , right link is null (yes)
we are doing left outer join.
If I use lookup some times job getting success.
Vivek Gadwal
Experience is what you get when you didn't get what you wanted
Experience is what you get when you didn't get what you wanted
Re: Replacing lookup with Joiner
both source and lookup both are not null columns but no records are inserting into database. if I run the job using lookup records are inserting
Re: Replacing lookup with Joiner
both source and lookup both are not null columns but no records are inserting into database. if I run the job using lookup records are inserting.
when I replacing lookup with join do i need to modify anything?
when I replacing lookup with join do i need to modify anything?
-
- Premium Member
- Posts: 457
- Joined: Tue Sep 25, 2007 4:05 pm
Right...Keeping the partitioning scheme to Auto for a Join stage should do Key partition automatically. However, it is also a good practice if you explicitly mention the key partition.Kryt0n wrote:Any output columns coming from the right side of your join need to be nullable.
On the input side, the columns on which you are joining should be non-nullable.
Also make sure your partitioning is the same for both input streams
Vivek Gadwal
Experience is what you get when you didn't get what you wanted
Experience is what you get when you didn't get what you wanted