We have a framework that controls the operational metadata, scheduling and other housekeeping activities. The framework records the last successful completion time for every job into an audit table.
We have a job(say jb_test) that reads data from a voluminous staging table on a weekly basis. Due to the huge volume, we are looking at an incremental extract using a query that looks something like
Code: Select all
select * from xyz where tran_date>last run date