Delta Load logic

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
invadermak
Participant
Posts: 2
Joined: Tue Aug 12, 2008 4:45 am

Delta Load logic

Post by invadermak »

Hi All,

My requirement is that i need to run my jobs with the Delta data only. I am familiar with the CDC logic, but the twist here is that i have 1 source table and multiple lookup tables.

Though it may sound easy that i just need to do a CDC on all the tables and get the delta data, then i can go ahead with the job.

But lets say a record R1 is changed in my source table and a record R2 is changed in my lookup table. In this scenario, both the records should come as a part of Delta data. But neither will the source record get a match in Lookup nor will there be any record for the Lookup record in the source. Hence neither of them is a part of delta data.

I hope the issue is understood. :(

Thanks,
MaK
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

Not sure I would do this without staging tables for all the sources. You might be able to do it with last change date on all the source tables.
Mamu Kim
Post Reply