Stoping and Restarting a Job in a sequencer

[tunde]

Hi all,
I have a sequencer with 4 jobs. The first two completed Ok but the third did not. However my sequencer ended OK. I use dsjob to call the job and with the sequencer completing successfully, my dsjob command was fine. The problem is how can i find out that a job failed and if i want to restart from where it failed how can i go about it.

Thanks

[kcbland]

A job sequencer does not track its execution. Therefore, restarting a sequencer job is the same as running the sequencer job. If you wish the ability to jump over completed stages within the sequencer, you must build into the sequencer the necessary logic to hop to points within the design based on some input value (job parameter, initialization logic in a beginning stage, etc). This is a worthy task, because I don't believe a scalable, managable design is possible. This is why I do not use the sequencer job design for job control.

Just so you know what the sequencer job is missing, you can read this post to see what I did with the job control API library to get the functionality that most of my customers have required:

viewtopic.php?t=85210

[Amos.Rosmarin]

Working with job sequencer is like writing your own code in a manner that the error handling is your responsibility.
After each job you get a trigger and it's up to you what you want to do with it. So, for example , you sequencer can look like this :
Reset if required, then run job1
If you ended OK then : Reset if required, then run job2
Else if job1 ended with warning then write warning msg to the log
Else write abort msg to the log and send email .....

You must explicitly catch the errors and define what you want to do in each case.


HTH,
Amos

[aaronej]

Tunde,

Ahhh... Restartability, not something easily achieved, but definately something you can do if you put some thought into your overall sequence structure, dependancies, what makes a job fail or what makes an entire sequence fail.

First, I will answer the question as to why the sequence showed a status of 'Finished' when a job inside the sequence failed. You see, the sequence doesn't care what status the jobs that it runs finish with, it could be that these jobs are SUPPOSED to fail and that their failure actually consitiutes success. You need to somehow force the jobs into telling the sequencer that it failed and that this is bad and what to do (Send email, quit, etc.). You could use the JobStatus to provide this information (lots of information on using this in this forum), also on each job in the sequence you could set an email notification on failure to let you know that a job failed. I use a combination of email notification and actually aborting sequences if any job inside the sequence fails, I then have a routine which checks the status of all these sequences and determins if any have failed. If any sequences have failed the master sequence stops right there, if not it goes on. I use this between my dimension load sequence and my fact load sequence, it allows me to make sure that all the dimensions have successfully loaded prior to loading any fact tables. Anyhow, thats how I do it.

Back to restartability... What I have done is to use a switch (read Nested Condition Stage) that allows me to selectively choose which jobs in the sequence I want to run and which I do not by setting a flag on the stage and passing a 1 or a 0 through as a job paramenter. Basically, it allows me to 'turn on' or 'turn off' different parts of my sequence. So for example, if everything ran successfully except for 1 dimension load, I can set that one job to run and nothing else. I'm sure others will have more elegant solutions, but this works for us right now.

Overall, I think you need to develop your jobs and sequences with the big picture in mind. For example, we sat down before we even started developing jobs and drew out a rough design of everything, jobs, sequences and how it all fit together. We were then able to use best practices along with our needs to determine the best procdure for us. So far, so good. The important thing to keep in mind is that this is not simple, it is fairly complex and will only get more and more complex as you move forward, so finding a solid design now and sticking with it will help remove some of the complexity.

Good luck!

Aaron

[roy]

Hi,
I might break your not easily achieved to not to hard to achieve.

if you point each logic step with a number you save somewhere (file probably or table) you could use your nested condition stage to simply determine if you need to run this job activity at the current run or not.
then on every run you get the last successfull numeric point that was successfull and determine if this run is a whole run or partial run from the last successfull point.
lets say you have 10 steps sequence, the start of it gets the last run's latest successfull point and if the last step was 10 or the current point is bigger then the point you got you should run this job activity.

this is a general possibility with all GUI, just watch you pit-falls
As someone said: "If you can't think of at least 2 ways to do it in DS you havn't thought enough"

IHTH

[kcbland]

Ahhh, if life only was so simple. Numbering each activity means that a serialized process is the only option. If you have multiple concurrent streams of logic, then you have to be a lot more fancy. Passing in the starting milestone is only good if you have synchronization milestones within a complex stream.

+-- head hurts --+

The sequencer is not looking so pretty if you've got to put all that logic in and have 800 jobs to arrange.

[roy]

well,
multi instance means you can write the job names, later load them to an array and check it.
similar to parameters concept