Page 1 of 1

Lookup size 3373708168 bytes

Posted: Tue Aug 31, 2010 2:13 pm
by sreddy
Hi

I design a job to find out missing data. But i have find out Reject data that is why i am using Lookupstage.

i got below message .

Lkp_HD_HS,0: Could not map table file "/DataStage/data/DEV/Datasets/lookuptable.20100831.qq5u5ka (size 3373708168 bytes)": Invalid argument
Error finalizing / saving table /DataStage/scratch/dynLUT24408db862d68


Can any one given me idea, what can i do next.


Thanks
SReddy

Re: Lookup size 3373708168 bytes

Posted: Tue Aug 31, 2010 3:13 pm
by arun_im4u
sreddy wrote:Hi

I design a job to find out missing data. But i have find out Reject data that is why i am using Lookupstage.
If you are using the Lookup stage just because you have to capture the reject rows, then you can do a left join in a Join stage and filter out the reject records in a transformer or Merge stage.

Posted: Tue Aug 31, 2010 10:00 pm
by chulett
32bit operating system? Your lookup data was too large to be cached in memory.