Have patience for my lengthy question/discussion
![Twisted Evil :twisted:](./images/smilies/icon_twisted.gif)
SCENARIO :-
Lets say I have got three sources SorA, SorB, SorC.
SorA -> has a field in its schema, to indicate that the data is valid prior to the date specified in this field (which means we can pull data only prior to specified date).
SorB -> The availability of data is known after a manual process of verification.
SorC-> Extracts should be on a particular date of Every Month.
I need to have extract dates for each source to be incorporated in target DW.
As far as i know till now is to have a table called "Audit Table" to store the extract dates (later on these values are populated to DW).
Question 1:
What are the fields that I can/should have for Audit Table?
My Comments
-> is it Sur_KEY, SorA_ExtDate, SorB_ExtDate, SorC_ExtDate,ProcessDate (anything else??)
Question 2 :
How/When should they be populated?
My Comments
-> Should I write the system date to audit table when ever the respective sequences are triggered, like eg. if SorA was triggered on 1st of Sept 2006, shall i take the date generated by datastage or the field which was specified in the field (as mentioned above). My guess from the source
-> In other two cases, my guess System Date
Question 3:
When it comes to next month, how will the extraction start
My Comments
-> should i first take extract date from audit table, add +1 to it and read data from there on???? to the current extract date (which is from Source for SorA and system date for SorB and SorC).
-> How can we be sure that the latest record ? we took from Audit Table is the last extract date????
Question 4 :
-> To trigger these three different sources at three different times, three Unix Scripts are to be writtien seperately, isnt it??