Staggered start in a sequence
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 1044
- Joined: Wed Sep 29, 2004 3:30 am
- Location: Nottingham, UK
- Contact:
Staggered start in a sequence
I want three jobs to run independently of each other, but I want them to start in a staggered sequence - the first one immediately, the second one after 1 minute, and the third one after 2 minutes. How can I do this? A while ago I did find a stage - it might have been an Execute Command - that I could use to implement a time delay, but I found that this time delay kicked in first and prevented anything else from happening until it had completed, and then the first and second jobs both started simultaneously.
Phil Hibbs | Capgemini
Technical Consultant
Technical Consultant
Re: Staggered start in a sequence
yes. Execute command will do the trick. But what do you mean by it prevented other things. are you running other jobs in parallel etc.
if you introduce a delay between two jobs, it will be only for the two.
if you introduce a delay between two jobs, it will be only for the two.
-
- Premium Member
- Posts: 1044
- Joined: Wed Sep 29, 2004 3:30 am
- Location: Nottingham, UK
- Contact:
Re: Staggered start in a sequence
What I expected was that it would start J1 and also the Execute Command stage both in parallel, and then start J2 when the Execute Command finished. What actually happens is that it runs the Execute Command, waits for it to finish, and then starts J2 and J1. It doesn't "wait for" in the same sense that it loops round monitoring a job status waiting for it to finish - it shells out to the command specified and only resumes doing anything else when the shell returns.mouthou wrote:yes. Execute command will do the trick. But what do you mean by it prevented other things. are you running other jobs in parallel etc.
Code: Select all
+----+ +-----+
| J1 | | Cmd |
+----+ +-----+
|
|
+-----+
| J2 |
+-----+
Phil Hibbs | Capgemini
Technical Consultant
Technical Consultant
You won't be able to do this with stages in a Sequence, you'll need hand code. Use DSRunJob() to start the first job but then don't immediately call DSWaitForJob() like the generated code does but sleep then start your second job. After both have started you can go back to the 'wait for job' call for both.
Or, if there are downstream activities dependant on these two and you want it to move along as soon as possible, build a loop to check the status of both and exit when both checks show completion.
Or, if there are downstream activities dependant on these two and you want it to move along as soon as possible, build a loop to check the status of both and exit when both checks show completion.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Premium Member
- Posts: 1044
- Joined: Wed Sep 29, 2004 3:30 am
- Location: Nottingham, UK
- Contact:
I don't like that idea. I think a DataStage job that runs a sleep command in a BeforeJobRoutine would be better. That way, it's clear what is happening when someone opens the job design, and the sequence can be amended and added to just like any other sequence.chulett wrote:You won't be able to do this with stages in a Sequence, you'll need hand code.
Phil Hibbs | Capgemini
Technical Consultant
Technical Consultant
If that 'delay' works for you. You still won't be able to manage a true 'staggered start' in the Sequence job, however. You could, however, arrange for both to start at the same time and then delay one as you noted. Only 'downside' I can see is the fact that it will always have that built in delay, even when run stand-alone.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Premium Member
- Posts: 1044
- Joined: Wed Sep 29, 2004 3:30 am
- Location: Nottingham, UK
- Contact:
I meant a separate Server Job whose purpose is only to do the sleep command.chulett wrote:You could, however, arrange for both to start at the same time and then delay one as you noted. Only 'downside' I can see is the fact that it will always have that built in delay, even when run stand-alone.
Phil Hibbs | Capgemini
Technical Consultant
Technical Consultant
Ah... no need for that, the answer is so simply I'm ashamed it took so long for it to pop into my brain. Back to two jobs and a routine...
Staggered start.
Code: Select all
+---RA----JA
Seqr ----|
+---JA
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Premium Member
- Posts: 1044
- Joined: Wed Sep 29, 2004 3:30 am
- Location: Nottingham, UK
- Contact:
So just drop a Sequencer stage in before J1 and the Execute Command stage? That fails to compile, "This activity must be triggered by another activity".
*Update*: Oh, RA is Routine Activity?
*Update2* Routine Activity behaves the same as Execute Command, they suspend execution of the Sequence Job until the timer expires, so if I have J1 immediate, J2 after a 10 second delay, and J3 after a 20 second delay, all on three independent activity chains, J3 occurs after 30 seconds not 20.
*Update*: Oh, RA is Routine Activity?
*Update2* Routine Activity behaves the same as Execute Command, they suspend execution of the Sequence Job until the timer expires, so if I have J1 immediate, J2 after a 10 second delay, and J3 after a 20 second delay, all on three independent activity chains, J3 occurs after 30 seconds not 20.
Last edited by PhilHibbs on Tue Oct 27, 2009 7:30 am, edited 1 time in total.
Phil Hibbs | Capgemini
Technical Consultant
Technical Consultant
-
- Premium Member
- Posts: 1044
- Joined: Wed Sep 29, 2004 3:30 am
- Location: Nottingham, UK
- Contact:
Code: Select all
+----------------------------+
| Nested Condition |
+----------------------------+
| | |
| | |
+----+ +-----+ +-----+
| J1 | | RA1 | | RA2 |
+----+ +-----+ +-----+
| |
| |
+----+ +----+
| J2 | | J3 |
+----+ +----+
Incidentally, how do I get a Server Job to compile if it has no active components? I've dropped a Transformer on sending a single row to /dev/null to get it to compile.
Phil Hibbs | Capgemini
Technical Consultant
Technical Consultant
You skipped the Sequencer, something you need to get this to work. That's what "Seqr" meant, btw, a "Sequencer" stage, not that this was the start point for a "Sequence" job. Hence the "r".
The transformer needs a stage variable to compile. You don't need to use it, it just needs to be there. I use 'svDummy' for that. And if you don't really need that single row, a constraint set to @FALSE works just fine. Really depends on exactly what you are doing there. If it's just so you can have an 'empty' canvas so a Before-job routine can be the only thing run, try putting a comment or a call to DSLogInfo() in the Job Control tab instead. Of course, an annotation on the canvas stating all that is going on would be good, too.
The transformer needs a stage variable to compile. You don't need to use it, it just needs to be there. I use 'svDummy' for that. And if you don't really need that single row, a constraint set to @FALSE works just fine. Really depends on exactly what you are doing there. If it's just so you can have an 'empty' canvas so a Before-job routine can be the only thing run, try putting a comment or a call to DSLogInfo() in the Job Control tab instead. Of course, an annotation on the canvas stating all that is going on would be good, too.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
Why not put an execute command stage for "sleep 60" and "sleep 120" into the stream for each job that should be staggered?
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Premium Member
- Posts: 1044
- Joined: Wed Sep 29, 2004 3:30 am
- Location: Nottingham, UK
- Contact:
Like this?chulett wrote:You skipped the Sequencer, something you need to get this to work. That's what "Seqr" meant, btw, a "Sequencer" stage, not that this was the start point for a "Sequence" job. Hence the "r".
Code: Select all
+-----------------------------+
| Nested Condition |
+-----------------------------+
| | |
+-----+ +-----+ +-----+
| Sqr | | Sqr | | Sqr |
+-----+ +-----+ +-----+
| | |
+----+ +-----+ +-----+
| J1 | | RA1 | | RA2 |
+----+ +-----+ +-----+
| |
+----+ +----+
| J2 | | J3 |
+----+ +----+
Ah, a single * in the Job Control does the trick. Thanks.If it's just so you can have an 'empty' canvas so a Before-job routine can be the only thing run, try putting a comment or a call to DSLogInfo() in the Job Control tab instead.
*Update:*
This doesn't work either:
Code: Select all
+-----------------------------+
| Nested Condition |
+-----------------------------+
|
+-----------------------------+
| Sequencer |
+-----------------------------+
| | |
+----+ +-----+ +-----+
| J1 | | RA1 | | RA2 |
+----+ +-----+ +-----+
| |
+----+ +----+
| J2 | | J3 |
+----+ +----+
Phil Hibbs | Capgemini
Technical Consultant
Technical Consultant
-
- Premium Member
- Posts: 1044
- Joined: Wed Sep 29, 2004 3:30 am
- Location: Nottingham, UK
- Contact:
Indeed, detaching the links to RA1 and J1 and re-attaching them in reverse order fixes my problem. I get J1 and a 10 second delay, then J2 and a 20 second delay, then J3. So RA2 only needs to be 10 seconds, as it has to wait for RA1 to finish before it starts. This is still not ideal, though, as any disruption to the link ordering will break it.PhilHibbs wrote:all three activities J1, RA2, and RA2 happen in an arbitrary order which presumably depends on the order in which I linked up the stages...
*Update:* Marking as resolved, as I can use a Server Job that calls the sleep command. It appears that there is no way to do this with Execute Command or Routine Activity stages, as they both suspend execution of the Sequence job until they have elapsed. Also the Sequencer and Nested Condition stages are not needed:
Code: Select all
+----+ +----+ +----+
| J1 | | S1 | | S2 |
+----+ +----+ +----+
| |
+----+ +----+
| J2 | | J3 |
+----+ +----+
Last edited by PhilHibbs on Tue Oct 27, 2009 8:37 am, edited 1 time in total.
Phil Hibbs | Capgemini
Technical Consultant
Technical Consultant