i have a look up which is joining a table that contain 40 million records with the input sequential file that contain same no. of records but when it reaches to this look up it is getting failed error i am gettin is "unable to operate on large objects"
Hope you are using Lookup stage in Parallel job. Lookup stage process is RAM based and you may use it only for smaller lookup tables. For larger lookup tables, consider using DISK based stages such as JOIN. One problem with Join stage is that you can't have more than one lookup table. Good Luck!
Not true. Join stage supports more than two inputs. They are called Left, Intermediate and Right. I believe that it still executes pairwise joins into intermediate result sets, however, the same as most database engines do.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
As Raja said, the Join stage is better than Lookup stage when volumes are high. You can still lookup on multiple tables by using a DB join in the database stage.