DSXchange

Hi,

Is there any way to estimate the max possible size of data that can be used as secondary link for Lookup Stage.

Thanks

Yes. The size of each data type is documented (from memory in Chapter 2 of the Parallel Job Developer's Guide). Add these together for all the columns in the row that you're processing, then multiply by the number of rows in the reference data set.

ray.wurlod wrote:Yes. The size of each data type is documented (from memory in Chapter 2 of the Parallel Job Developer's Guide). Add these together for all the columns in the row that you're processing, then ...

Hi Ray,

Sorry I didn't post my question properly.

After calculating the size, how can we determine whether Lookup might work without memory issues.

Thanks

How much memory do you have? How much scratchdisk in the nodes on which the Lookup stage operates? How many other stages are there in the job? How long is a piece of string? (Try it and see.)