Implementing job dependencies in a sequence job

sbass1 · Post by **sbass1** » Wed Mar 11, 2009 10:02 pm

Hi,

Our job dependencies are something like:

Group1:
JobA
JobB
JobC
JobD
JobE

These jobs are all independent of each other, and I would prefer they run multi-threaded. But, they should all successfully complete before...

Group2:
JobF --> JobG --> JobH --> JobI

These jobs are all dependent on the success of the parent job (creation of surrogate keys). However, different jobs are dependent on jobs in Group1, for example:

JobF depends on A & C
JobG depends on B
JobH depends on all jobs
JobI depends on Job D & E

Once these jobs finish...

Group3:
JobJ
JobK
JobL

These jobs again are all independent of each other, but dependent on the successful run of all jobs in Group2. Again, I would like these jobs to run multi-threaded if possible.

What is the best practice approach for implementing these dependencies in a Job Sequence job? It would be nice if there was a "Container" in which I could drop each group of jobs, with a dependency of the container. But, I don't see this capability in the sequence jobs.

Thanks,
Scott

ray.wurlod · Post by **ray.wurlod** » Wed Mar 11, 2009 10:36 pm

Sequencers.

It's not clear whether G depends on F. Does it?

sbass1 · Post by **sbass1** » Wed Mar 11, 2009 11:12 pm

Yes. F creates SK's based on some NK. G then saves the F's SK as an FK based on the NK.

So F needs to successfully complete before G starts.

Likewise for the other dependent jobs.

ray.wurlod · Post by **ray.wurlod** » Thu Mar 12, 2009 12:42 am

Code: Select all

         B --+
             |
A --+        +--> G --+
    |        |        |      --> I
    +--> F --+    D --+--> H --> J  
    |                 |      --> K
C --+             E --+

+--> represents an "All" Sequencer

sbass1 · Post by **sbass1** » Thu Mar 12, 2009 1:45 am

So I have to code all dependencies "manually". Bummer.

Was really hoping there was some way to:

* Run A-E concurrently
* Once A-E have finished successfully, run F->G->H->I sequentially
* Once I has finished successfully, run J-L concurrently

Thus the concept of "containers" to hold groups of jobs.

Oh well

ray.wurlod · Post by **ray.wurlod** » Thu Mar 12, 2009 2:55 am

Yes but that would take longer and introduce dependencies that don't actually exist, e.g. H depending on A.

Code: Select all

A --+
    |
B --+
    |                        --> J
C --+--> F --> G --> H --> I --> K
    |                        --> L
D --+
    |
E --+

sima79 · Post by **sima79** » Thu Mar 12, 2009 2:56 am

Your concept of "containers" can be implemented by using multiple job sequences.
Job Sequence 1: Run A-E concurrently
Job Sequence 2: F->G->H->I
Job Sequence 3: J-L concurrently

You then need a master job sequence to control the 3 job sequences above. This will have the three job sequences above sequentially linked together.

sbass1 · Post by **sbass1** » Thu Mar 12, 2009 3:41 am

Doh! Of course. The "container" is the sequence job itself.

Thanks!