Lookup table

[shivan]

hi,
I am trying to use look up table. What i want to do is:
I have a job which takes 1.7 million records. I run the job first time, so it processes all 1.7 million records. Now i have a look up table. So what when i run the job again, and there are only .08 records added or updated. I only update those records or insert records rather than going through 1.7 million records.

Now the way it is working is like this:
there are only 1000 new records. So when it goes from transformer to aggregator it is is processing the whole number of records which is 1.7 million rather than just sending 1000 new records to the aggregator.

I really appreciate if someone explain me what i am doing wrong.

thanks
shivan

[Sainath.Srinivasan]

Do you refresh the lookup table?

Else use the target as lookup.