Running job in parallel to load dataset

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
tsktsk123
Participant
Posts: 32
Joined: Thu Dec 25, 2003 11:59 am

Running job in parallel to load dataset

Post by tsktsk123 »

Hi,

We have a requirement to do a loading from multiple source files i.e. the source system will send data based on extract date, if the extract date is less than the sysdate then we need to load the file as the reload process. Say if the sysdate is 05-Dec-2008 and we receive 3 different files from same source system ABC with three different extract date 02-Dec-2008,. 03-Dec-2008 and 04-Dec-2008. We are having 3 staging areas and using dataset as target for the staging and finally loading the value to the target tables.

I like to understand if datastage can write to the same dataset when i run multiple jobs to load to the same dataset i.e. let's assume i am running 3 jobs parallely to load data from 02-Dec-2008,. 03-Dec-2008 and 04-Dec-2008 files to target, in the staging area we will be loading data to dataset.

Appreciate any help on this.
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

Writing to a file parallely at the same time even in append mode would give you error at any time.
You can try to load in to the table directly instead if you have partitions enables.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
Post Reply