Automizing parameter values with a concurrent architecture
Posted: Wed Sep 26, 2007 11:10 am
We currently have an architecture where we will be running batch loads based on a RUNDATE on a daily basis.
There are three stages that the data batch needs to pass through.
Stage 1. Source - Standardization
Stage 2. Standardization - Cleansing
Stage 3. Cleansing - Target
In the first stage we extract based on a parameter RUNDATE.
Going forward from Stage 1-2, Stage 2-3 Im currently placing the RUNDATE, Batch_LoadID in a hash file for all the transformation required.
If another job is started at the sametime then it is possible that it would incorrectly refer to that hash file.
What is a good method to prevent this non-concurrent method in a DS architecture design? where no third party tool is used for scheduling and only Job sequence, DS designer is relied upon.
Thanks
There are three stages that the data batch needs to pass through.
Stage 1. Source - Standardization
Stage 2. Standardization - Cleansing
Stage 3. Cleansing - Target
In the first stage we extract based on a parameter RUNDATE.
Going forward from Stage 1-2, Stage 2-3 Im currently placing the RUNDATE, Batch_LoadID in a hash file for all the transformation required.
If another job is started at the sametime then it is possible that it would incorrectly refer to that hash file.
What is a good method to prevent this non-concurrent method in a DS architecture design? where no third party tool is used for scheduling and only Job sequence, DS designer is relied upon.
Thanks