Controlling Data Commit While Loading Multiple Tables

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
jerome_rajan
Premium Member
Premium Member
Posts: 376
Joined: Sat Jan 07, 2012 12:25 pm
Location: Piscataway

Controlling Data Commit While Loading Multiple Tables

Post by jerome_rajan »

Hi,

This was a question that came in a brain storming session and I was wondering if any of you have encountered this before.

Assume, we load 10 different tables in 10 parallel jobs all within the same sequencer. Let us also assume that all these laods happen sequentially because of integrity requirements.

Now, if my last table load fails / aborts, I want to ensure that all the previous loads are undone / rolled back.

Is there any way to ensure that the commit is held back until the last job?
To the best of my knowledge, the connector seems to commit a transaction individually.

Any assistance in this matter would be very helpful. Thanks!
Jerome
Data Integration Consultant at AWS
Connect With Me On LinkedIn

Life is really simple, but we insist on making it complicated.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Across jobs? Can't be done, at least nothing automatic. That's not to say you could build something to go back and "undo" a partial load but you would need to code the deletes / updates needed to reverse the load to that point.
-craig

"You can never have too many knives" -- Logan Nine Fingers
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

There used to be a Distributed Transaction stage you could use for some purpose like that within one job to insert/update/delete into multiple tables then commit.
Choose a job you love, and you will never have to work a day in your life. - Confucius
Post Reply