Lookup size 3373708168 bytes

[sreddy]

Hi

I design a job to find out missing data. But i have find out Reject data that is why i am using Lookupstage.

i got below message .

Lkp_HD_HS,0: Could not map table file "/DataStage/data/DEV/Datasets/lookuptable.20100831.qq5u5ka (size 3373708168 bytes)": Invalid argument
Error finalizing / saving table /DataStage/scratch/dynLUT24408db862d68


Can any one given me idea, what can i do next.


Thanks
SReddy

[arun_im4u]

Hi

I design a job to find out missing data. But i have find out Reject data that is why i am using Lookupstage.

If you are using the Lookup stage just because you have to capture the reject rows, then you can do a left join in a Join stage and filter out the reject records in a transformer or Merge stage.

[chulett]

32bit operating system? Your lookup data was too large to be cached in memory.