How to refresh a lookup table after every iteration

[msuri]

Hello all,

In a Datastage Enterprise Edition job, is it possible to refresh a lookup to reflect inserts done to the lookup table within the same job?
My job writes records to a table X, It is capable of insert new records, insert new versions and update old versions of existing records. The lookup to identify existing records and new versions is done to the same table X. The problem arises because the input data may have several versions of the same record inside.

Thank you in advance

[ArndW]

No, the only way to do this is to use sparse lookups and no buffering of output.

[Vardhini]

No, the only way to do this is to use sparse lookups and no buffering of output. ...

Will the sparce lookup auto commit records:?:
and also using a sparce lookups will decrease the performance of the job.

[ray.wurlod]

Will the sparce lookup auto commit records:?:

That's entirely a matter of how you design, and (I believe) what Arnd meant by "no buffering of output". YOU control the number of rows per transaction - for auto-commit set it to 1.

and also using a sparce lookups will decrease the performance of the job.

That depends. Define "performance" in an ETL context.

[yserrano]

Main post has some similiraty with this one.