Restart a job
Posted: Tue May 19, 2009 12:07 am
Hi
I was seeking some suggestions for designing a job restartability scenario.
Scenario :
I have a DS job consisting of StageA,StageB and StageC.Suppose for incoming 100 rows StageA and StageB have done their processing of 100 records but job fails at StageC .
Is it possible to reuse the already processed data from StageA or StageB which could be fed directly to StageC for processing.
I was wondering if processed link data from StageB could be reused and sent to StageC.
I do not want to use intermediate Datasets as they increase the I/Os.
Any suggestions are highly appreciated.
Thanks
I was seeking some suggestions for designing a job restartability scenario.
Scenario :
I have a DS job consisting of StageA,StageB and StageC.Suppose for incoming 100 rows StageA and StageB have done their processing of 100 records but job fails at StageC .
Is it possible to reuse the already processed data from StageA or StageB which could be fed directly to StageC for processing.
I was wondering if processed link data from StageB could be reused and sent to StageC.
I do not want to use intermediate Datasets as they increase the I/Os.
Any suggestions are highly appreciated.
Thanks