Sequence Jobs. Design issue or not ?

[nmacolin]

Hi all,

Looking for some advise here.

I've searched the site and looked up countless documentation but there doesn't seem to be a lot on using Sequence Jobs. (Or the finer points of using sequence jobs )

I have multiple loops and several nested conditions. I'm having a problem where the sequence job will perform several intereations and then when a nested condition returns true causing the inner loop not to be executed I get log information as follows and the job stops ( Without error )

jCHEOPS_Main_SeqJob_WithProjTypes..JobControl (@Seq_ProjCompleteOrContinue): Note: Sequencer 'Seq_ProjCompleteOrContinue' has been entered from more than one loop context, which may lead to unexpected results
jCHEOPS_Main_SeqJob_WithProjTypes..JobControl (@Coordinator): Note: Loop 'LoopCompanyCodes' was started, but did not complete all its iterations
jCHEOPS_Main_SeqJob_WithProjTypes..JobControl (@Coordinator): Note: Loop 'LoopProjectFiles' was started, but did not complete all its iterations
jCHEOPS_Main_SeqJob_WithProjTypes..JobControl (@Coordinator): Note: Sequencer 'Seq_ProjCompleteOrContinue' was entered, but never exited

Are there limitations on how the Sequencer can be used ?

Here is a overview of the Sequence job.

Loop Companies
(General Job Processing)
Loop Project Files for Companies
(General Processing)
Check if File is Empty (Nested Condition )
if not empty then process otherwise go to sequencer
Loop Projects
Extract Project data
Load Project Data
End projects loop
[b]Sequencer ( Set to Any )[/b]
End Loop Project Files for Companies
End Loop Companies

The nested condition is working correctly and the correct link is being followed, but why does the job packup when the sequencer is entered from this path and not when it is entered post completing the "Loop Projects"

Another way of showing it would be

Condition Nested ------ (No Data Found Link) ------>>> Sequencer (Any)
| ^
| |
| |
+-Data Found --> Loop Project <............................. End Loop
| |
| |
+---> Extract Data --> Load Data ------------------+
(Those bars should line up under the sequencer and end loop )

I have tried another version of the the entire sequence job but removing the "LOOP Project" loop above and just having a couple of exec commands to write to the file system as a debug and then linking to the Sequencer. This seems to work fine.

So is there an issue with having an "End Loop" stage and another job connected to a Sequencer ? Even though I've set the mode to "Any", why would it be concerned with possible incorrect results ?
I would like the "Loop Project" to be skipped over when there are no project files for the Project. I want the flow to go back to the end loop and perform another iteration for the next Project which may or maynot have project files.

Any assistance would be appreciated.

Regards
Nick

[chulett]

"Code" tags preserve whitespace.

Here is a overview of the Sequence job.

Code: Select all

Loop Companies
    (General Job Processing)
    Loop Project Files for Companies
       (General Processing)
       Check if File is Empty (Nested Condition )
       if not empty then process otherwise go to sequencer
          Loop Projects
             Extract Project data
             Load Project Data
          End projects loop
       [b]Sequencer ( Set to Any )[/b]
    End Loop Project Files for Companies
End Loop Companies

Visually:

Code: Select all

Condition Nested   ------ (No Data Found Link) ------>>> Sequencer (Any)
                                                                     ^
    |                                                                |
    |                                                                |    
    +-Data Found --> Loop Project <............................. End  Loop  
                            |                                        |
                            |                                        |
                            +---> Extract Data --> Load Data --------+

Sorry, don't have time to absorb this and reply properly, hopefully will have some time later. In the meantime, wanted to help with the "ASCII art".

[ray.wurlod]

I see only one loop in your design. Yet your words suggest that two loops are required. Make sure that every loop reaches its End Loop activity - don't "jump out".

[chulett]

I haven't had any issues "jumping out" that I recall, i.e. exiting a loop early. Just don't try to jump back in.

[ray.wurlod]

Jumping out can cause the "but did not complete all its iterations" or "was not exited" messages. While they're harmless, the original questions were about how to eliminate these messages.

[chulett]

True, in looking back I typically only exit the loop When Something Goes Horribly Wrong so don't mind the "did not complete" message.

However, the Sequencer message confuseth me a little, I thought that would only being output for Sequencer objects set to "All" when some but not all conditions were met before the job ended. The key being the "was entered" part of the "was entered but never exited" warning. Seems that only an "All" sequencer can be entered but not exited, an "Any" is basically a pass-through - anything entering does an immediate unconditional exit.

I'd be curious to see the "big picture", the complete design. This seems like a portion of the whole.

[ray.wurlod]

There's no output link from the Sequencer. That will cause the "did not exit" message. Probably the compiler should detect this; some folks use a Sequencer as a placeholder when iteratively designing job sequences, so they've left it (a Sequencer with no output) as a "valid" construct.

[chulett]

No, that's not the cause. I routinely use a Sequencer as the end point in a job flow so there's a trigger to check on the last Job Activity ("automatically handle") and there's never a "did not exit" message in the log since there's no exiting link.

I think the OP only gave us the "meat" of the design, there must be something before and after this snippet. As you noted, two loops were mentioned but only one shown.

[nmacolin]

Hi all,

Thanks for the Code tag hint. That should make my diagrams easy to understand.
I'll try and show the entire sequence job here, but note, I don't jump out of a loop. My nested conditions decides whether I perform the loop or not. So if there's no data, skip the loop and get the next iteration for the Project files loop. Hopefully using the code tag should make the flow easier to understand

Code: Select all

dertmine                                                                        
list of     ---> Loop Companies <......................................End Loop  
Companies         |                                                       |     
                  |                                                       |     
                Get Project Files                                         |     
                for Company                                               |     
                  |                                                       |     
                  |                                                       |     
                Loop Project Files <...................................End Loop 
                  |                                                       |     
                  |                                                       |     
                Is Project File Empty -------> YES -------------------> Sequence
                  |                                                     (Any)   
                  |                                                       |     
                 No                                                       |     
                  |                                                       |     
                Loop Project Data  <.................................End Loop   
                  |                                                       |     
                  |                                                       |     
                  +---> Extract Data -----> Load Data ------------------->+     

So from the Nested Condition "Is Project FIle Empty" determines if I proceed with the Loop Project Data or go to the sequencer and continue to itterate for the next item in the list.

As mentioned before If I remove this inner loop and use a couple of exec command calls then the sequencer behaves as expected and all the loops itterate as expected... for all companies and all project files within the companies.

So having the sequencer with an End Loop and Nested Conditions as input links causing issues ?

I'm not jumping out of loops . So all loops that are started are completed.
The Sequencer then output links to the next end loop for the previous iteration so there is an exit. So the flow is not terminating at the sequence
The Sequencer is set to Any.

Should I try and make the Inner loop another sequence job and then call it.

Any help is appreciated.

Regards
Nick

[nmacolin]

Hi all,

As I said in my last post I would separate out the Project data Loop.

:D This works fine.

So is anyone aware of issues with the End Loop stage connecting directly to a sequencer stage ?

The only difference now is that the JobActivityStage returns "TRUE" to the sequencer, rather than the END LOOP ( ??? not sure what this would return).

Any ideas ? I wont mark this resolved as there is a problem here, that needs to be discussed.

Regards
Nick