Checkpoint Restart Capability
Moderators: chulett, rschirm, roy
Checkpoint Restart Capability
Guys
Many of our jobs were originally written in DataStage 5.2. Now we have moved to DataStage 7.1r2.
Whenever a Job sequencer fails, we have to go through the tedious process of running the remaining jobs manually or have written a script which reads from a file as to what remaining jobs to run.
In 7.2, we want to get away from that by using the check point restart. Can you please advise
1. How to implement it
Please include details more than just the check box tick. I mean whether DS will automatically handle if we simply rerun the Job etc.
Regards
Rangs
Many of our jobs were originally written in DataStage 5.2. Now we have moved to DataStage 7.1r2.
Whenever a Job sequencer fails, we have to go through the tedious process of running the remaining jobs manually or have written a script which reads from a file as to what remaining jobs to run.
In 7.2, we want to get away from that by using the check point restart. Can you please advise
1. How to implement it
Please include details more than just the check box tick. I mean whether DS will automatically handle if we simply rerun the Job etc.
Regards
Rangs
There's not a whole lot more to it than checking the box. It's explained in the Designer's Guide pdf manual installed with your client.
In the chapter on 'Job Sequences', checkpoint restarts are documented in the General Page portion of the Job Sequence Properties section as well as in the Restarting Job Sequences section.
In the chapter on 'Job Sequences', checkpoint restarts are documented in the General Page portion of the Job Sequence Properties section as well as in the Restarting Job Sequences section.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
Re: Checkpoint Restart Capability
Rangs wrote:Guys
Many of our jobs were originally written in DataStage 5.2. Now we have moved to DataStage 7.1r2.
Whenever a Job sequencer fails, we have to go through the tedious process of running the remaining jobs manually or have written a script which reads from a file as to what remaining jobs to run.
In 7.2, we want to get away from that by using the check point restart. Can you please advise
1. How to implement it
Please include details more than just the check box tick. I mean whether DS will automatically handle if we simply rerun the Job etc.
Regards
Rangs
Was it that tedious in release 5.2
![Mad :x](./images/smilies/icon_mad.gif)
![Laughing :lol:](./images/smilies/icon_lol.gif)
regards
kumar
-
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
It is as simple as checking the check box!
The main considerations - you need to make sure you restart with the same job parameters as the failed sequence. You need to consider whether you should should be restarting jobs that update aggregate database tables or perform database inserts. Your support staff need to be aware that a Stopped/Restartable sequence job can be restarted from the checkpoint by running it or restarted from the beginning by resetting and running it.
Other then that it works and it works well.
The main considerations - you need to make sure you restart with the same job parameters as the failed sequence. You need to consider whether you should should be restarting jobs that update aggregate database tables or perform database inserts. Your support staff need to be aware that a Stopped/Restartable sequence job can be restarted from the checkpoint by running it or restarted from the beginning by resetting and running it.
Other then that it works and it works well.
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
-
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
Any reason why you are upgrading to 7.1 and not the newer 7.5.1?
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
You may need to be a little careful...one thing I have noted is that the Checkpoint Restart information must be held within the jobs information in Universe, and if for some reason you need to re-build the job you can lose that restart capability (depending on were the error was).
Another solution may be to create your own home grown checkpointing outside datastage, by creating flag files as jobs start and complet successfully. There is a little bit of pain in doing this, but if your confronted with an error in one of you primary sequence jobs that was missing something, and had to be changed (promoted), you do not lose your restartability....
IMHO
![Wink :wink:](./images/smilies/icon_wink.gif)
![Shocked :shock:](./images/smilies/icon_eek.gif)
Another solution may be to create your own home grown checkpointing outside datastage, by creating flag files as jobs start and complet successfully. There is a little bit of pain in doing this, but if your confronted with an error in one of you primary sequence jobs that was missing something, and had to be changed (promoted), you do not lose your restartability....
IMHO
![Wink :wink:](./images/smilies/icon_wink.gif)
Regards
Chris Fuller
"Reality is what refuses to go away when you stop believing in it"
Chris Fuller
"Reality is what refuses to go away when you stop believing in it"
-
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
I think that's an argument for doing testing and training rather then for writing your own restart checkpointing.
Checkpoint can very easily be cleared out, if someone resets the sequence job or recompiles the sequence job. Perhaps the drawback of sequence job restart is that you cannot manually set a check point to a particular part of the sequence like you could with an external check point system.
![Smile :)](./images/smilies/icon_smile.gif)
Checkpoint can very easily be cleared out, if someone resets the sequence job or recompiles the sequence job. Perhaps the drawback of sequence job restart is that you cannot manually set a check point to a particular part of the sequence like you could with an external check point system.
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
I actually wrote the code, with a lot of help from someone else, in version 6, and have implemented it at 2 different sites I've worked at very successfully. Checkpoint restart was not available in V6, and when you are running a job stream containing over 1500 jobs, you don't want to have to run the last 800 manually at 2am in the morning if you have a problem...if you know what I mean.
![Laughing :lol:](./images/smilies/icon_lol.gif)
![Laughing :lol:](./images/smilies/icon_lol.gif)
Regards
Chris Fuller
"Reality is what refuses to go away when you stop believing in it"
Chris Fuller
"Reality is what refuses to go away when you stop believing in it"
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: