Sequencer issue

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

ag_ram
Premium Member
Premium Member
Posts: 524
Joined: Wed Feb 28, 2007 3:51 am

Post by ag_ram »

Well Chulett,
Of course. However, like most of the time, there's more than one way to skin this cat. All I was attempting to point out was that (in my opinion) the focus on just warnings was too narrow, but that rather you should handle all outcomes outside of the 'OK' box.
Of course NOT. There are two main parameters we need to give profound focus, which are nothing but detecting Job Warning in the Job and, if warnings, aborting the Job Sequence with restartablility.
1. It's all there, not sure what makes you think anything is missing. Why not build what I suggested and verify for yourself that it does indeed restart when appropriate.

2. Of course, by definition it covers all situations not otherwise covered by the existent triggers. Your 'abort with no warnings' scenario is... odd. The goal here is a Sequence which is restartable when there is any kind of problem. As Ray noted, there is no need for any other solutions as solutions have been given. You just parroted what Ray said and drew some lovely ascii art. After retracting the first "that isn't true" misstep on my part, I posted an alternative. Both satisfy the requirement.
What I got when I experimented with your new solution:

When the selected Job1 gives OK status and Job2 produces warning(s) at any point of time, flow validates Trigger [Otherwise] and makes Job Sequence aborted in restartable condition with the help of Sequencer Activity and Terminator Activity at the end. Here the notable point is that Job2 is not aborted and so the checkpoint is not recorded at this point. Later then, when I restart my Job Sequence, the execution of Job1 and Job2 is skipped(checkpoint not recorded here) and the Job Sequence is always aborted in restartable condition with the help of Sequencer Activity and Terminator Activity.
Your 'abort with no warnings' scenario is... odd. The goal here is a Sequence which is restartable when there is any kind of problem.
pat_raman2000 wrote: My requirement is when it encounters warning it should abort and be there at Job1
For your information:

In the Job Sequence, the checkpoints are NOT added when an unhandled abort encountered in Job.

In other words,
1. When the Job is aborted and the abort condition is NOT handled in the Trigger section, the checkpoint is NOT added.
2. When the Job is aborted and the abort condition is handled in the Trigger section, the check point is added.
Last edited by ag_ram on Mon Jun 02, 2008 1:34 am, edited 1 time in total.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

That is so totally wrong, ag_ram.

A checkpoint is only added if the activity completes successfully, and then only if "do not checkpoint run" is cleared for that activity.

It is the presence of a checkpoint that allows an activity to be skipped on re-start.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
ag_ram
Premium Member
Premium Member
Posts: 524
Joined: Wed Feb 28, 2007 3:51 am

Post by ag_ram »

You are correct ray.wurlod.

I edited my post now.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

ag_ram - Please don't completely delete or edit incorrect posts. If you make mistakes, update the post with the correct information, but leave the original there. That is the only professional and correct way to behave in forums such as this one.
ag_ram
Premium Member
Premium Member
Posts: 524
Joined: Wed Feb 28, 2007 3:51 am

Post by ag_ram »

Thanks ArndW.

I follow it from next time.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Good. Your point #2 remains incorrect.
ag_ram wrote:2. When the Job is aborted and the abort condition is handled in the Trigger section, the check point is added.
No it isn't. Only non-failure completion will add a checkpoint.

That is why, if all activities complete successfully, you get an event logged indicating that all checkpoints are being cleared, so that the entire job sequence will run the next time it is started.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Oh Bejeebus, sucked back in again. :roll:
ag_ram wrote:Well Chulett,
Of course. However, like most of the time, there's more than one way to skin this cat. All I was attempting to point out was that (in my opinion) the focus on just warnings was too narrow, but that rather you should handle all outcomes outside of the 'OK' box.
Of course NOT. There are two main parameters we need to give profound focus, which are nothing but detecting Job Warning in the Job and, if warnings, aborting the Job Sequence with restartablility.
Dude, don't "of course NOT" me. That reply was for Ray, not you. And since I'm late to the party and missed your original unstabilities and profound focus pre-edit, I'm not sure what to comment on. Just this, I guess.
ag_ram also wrote:What I got when I experimented with your new solution:
Good. As least you are 'experimenting' as I hoped you would. And what have we learned from all this? The meaning of the word 'appropriate'? Jobs that do not abort cannot be restartable checkpoints. Period. Of course you could always roll your own job control to do whatever was in your power to create, but out of the box via the GUI in its current incarnation what the OP seems to want and you keep going 'round about ain't gonna happen.

Bottom line, I stand behind my original design. I use it in many (many) hands of production jobs. The Sequence must abort to be restartable. A job must abort to for it to restart at that point.
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Show us, from the job log (unedited) which activities had checkpoints added. Even though they were cleared at the end.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
rivajtp
Participant
Posts: 27
Joined: Tue Jul 10, 2007 8:49 pm
Location: Bangalore

Post by rivajtp »

What chutte wrote will work out

chulett wrote:
What I would setup would still involve that trailing Sequencer so that each job has a full set up triggers associated with them, including the last one. In addition to the 'OK' trigger that passes execution on to the job step, I would use an 'Otherwise' trigger from each job that all go to a single Sequencer set to 'Any'. That sequencer would then lead to a Terminator which aborts the Sequence job and optionally sends stop requests to any currently running jobs as well. To me that would be a 'cleaner' design. YMMV.

but instead of terminator stage what i will suggest is let the otherwise link from all job activity go to sequencer and for 'Any' option. after that you put a routine activity stage and then use the routine UtilityAborttoLog routine in routine--->SDK---->Utility..


This is better i think rather make it too complex. What say chulett?

Regards
Rivaj T P
ag_ram
Premium Member
Premium Member
Posts: 524
Joined: Wed Feb 28, 2007 3:51 am

Post by ag_ram »

ray.wurlod wrote:Show us, from the job log (unedited) which activities had checkpoints added. Even though they were cleared at the end.
In my case(with adding 'Automatic handling' for a neat conclusion), If all checkpoints are removed at the end, isn't meaning that the controller adds checkpoints(as flow successfully validated) for all activties, unless Job3 is aborted?
ag_ram
Premium Member
Premium Member
Posts: 524
Joined: Wed Feb 28, 2007 3:51 am

Post by ag_ram »

chulett wrote:Good. As least you are 'experimenting' as I hoped you would. And what have we learned from all this? The meaning of the word 'appropriate'? Jobs that do not abort cannot be restartable checkpoints. Period. Of course you could always roll your own job control to do whatever was in your power to create, but out of the box via the GUI in its current incarnation what the OP seems to want and you keep going 'round about ain't gonna happen.

Bottom line, I stand behind my original design. I use it in many (many) hands of production jobs. The Sequence must abort to be restartable. A job must abort to for it to restart at that point.
I have been late to look back your statements here.

In the experimentation i had with your design, i missed to achieve the restartability and just showed you how it does not content that term. I still have to have a grand look at your design which you use for many years. There is no intention to doubt it because there should be a stronger reason(base)behind always.

But i request you to share your views on "Why it does not make sense in the experimention i had with your design"
ag_ram
Premium Member
Premium Member
Posts: 524
Joined: Wed Feb 28, 2007 3:51 am

Post by ag_ram »

but instead of terminator stage what i will suggest is let the otherwise link from all job activity go to sequencer and for 'Any' option. after that you put a routine activity stage and then use the routine UtilityAborttoLog routine in routine--->SDK---->Utility..


This is better i think rather make it too complex. What say chulett?
You didnt ask me but my comment is:

Nothing is complex here. I dont find any difference in this modified design in the context of 'Restartability'. Shall i recommend you to see the Experimental result i just had?

I still have a question with chulett to give the best reason for his design to be on the side of Restartability as well.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

ag_ram wrote:If all checkpoints are removed at the end, isn't meaning that the controller adds checkpoints(as flow successfully validated) for all activties, unless Job3 is aborted?
No. Checkpoints are only added for all activities that do not fail, unless the activity is marked "do not checkpoint run".

Therefore, if Job2 aborts, no checkpoint is added for Job2, whether or not the job sequence then goes on to execute Job3.

Look in the job sequence for "Added checkpoint..." events to prove this.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
ag_ram
Premium Member
Premium Member
Posts: 524
Joined: Wed Feb 28, 2007 3:51 am

Post by ag_ram »

ray.wurlod wrote: Therefore, if Job2 aborts, no checkpoint is added for Job2, whether or not the job sequence then goes on to execute Job3.
I dont get Job2 rerun but only Job3 when i restarted my Job Sequence, if the Job Sequence is already failed because of Job3 abort. Please give reason for this statement.
ag_ram
Premium Member
Premium Member
Posts: 524
Joined: Wed Feb 28, 2007 3:51 am

Post by ag_ram »

ray.wurlod wrote: Therefore, if Job2 aborts, no checkpoint is added for Job2, whether or not the job sequence then goes on to execute Job3.
I dont get Job2 rerun but only Job3 when i restarted my Job Sequence, if the Job Sequence is already failed because of Job3 abort. Please give reason for this statement.
Post Reply