Data initialization
Posted: Wed Jun 12, 2013 11:28 am
Hi all,
I have used dataset for staging area and first time I will do the historical load and the data will be in the staging dataset (option: overwrite) . Once after after the dimensions and facts are loaded ,I am taking max (updated_ts) in the dataset and load into a sequential file and via routine am passing it for the next run.
My question is,
1) Whats the case if one incremental load doesnt have data and zero records will come from the source and during that time what will be value in the dataset, will it be initialized if zero records goes into it.
2) even if the dataset is initialized, there wont be any record in it and its not possible to maintain the same updated timestamp in the target sequential file, because this job will run in a schedule.
When there is no records in the dataset for incremental load, how to maintain the previous maximum updated ts value
Thanks,
dubuku
I have used dataset for staging area and first time I will do the historical load and the data will be in the staging dataset (option: overwrite) . Once after after the dimensions and facts are loaded ,I am taking max (updated_ts) in the dataset and load into a sequential file and via routine am passing it for the next run.
My question is,
1) Whats the case if one incremental load doesnt have data and zero records will come from the source and during that time what will be value in the dataset, will it be initialized if zero records goes into it.
2) even if the dataset is initialized, there wont be any record in it and its not possible to maintain the same updated timestamp in the target sequential file, because this job will run in a schedule.
When there is no records in the dataset for incremental load, how to maintain the previous maximum updated ts value
Thanks,
dubuku