Job Control General Question

kumar_s · Post by **kumar_s** » Sun Feb 26, 2006 8:17 am

Though you have an option to split the Job sequence for group of jobs and trigger indiviually for good maintanability and clarity...
there is nothing wrong in calling all the job in a single sequence

chulett · Post by **chulett** » Sun Feb 26, 2006 8:34 am

Finally decided to throw my two cents in the ring on this. I work with about a dozen other developers right now and I'd say that I am the only one out of them that is comfortable writing my own 'Job Control' code.

Experience definitely colors that picture. I haven't been using the product since Version 0 like some folks here, but did start back with 3.1 when there was no such thing as a Sequence job. You wanted something to 'sequence' your jobs you wrote it yourself. Uphill. Both ways. In the snow. Barefoot.

The others, who in spite of the fact they may have done this pushing three years, have always been on a flavor of 7.x and so have always had Sequence jobs in one form or another. For them, I've found that hand coded job control is never a 'Best Way' as they worry about having to maintain it, tweak it, etc. Sequencer jobs, however, they can crank out with their eyes closed - and sometimes I swear that's exactly what they do. [/joke]

The issue with Sequence jobs not being 'clean' is as much the skill of the designer as anything else, in my opinion. Especially with the latest 7.5.x enhancements. Build your Sequence jobs just like you would your other jobs - modularly. Restartable, maintainable modules. Sequence jobs are perfectly happy called 'sub-sequence' jobs down as many levels as you need. Let them handle their own errors now - no need for those giant 6.x spiderwebs to the Exception Handler any longer. Let it do it's own checkpoint restarting. Loop activity stages. Use Variables. Commands and Routines. Automatic generation of HTML documentation. Yada yada, etcetera etcetera. You'll find that 99.999% of the time, everything you 'need' is there and can be maintained by the other developers on staff and especially by those who follow.

That being said, we still use a ton of Job Control to handle our many hundreds of nightly jobs, KBA code we adapted and modified extensively. Besides a 'better' form of restartability, the biggest thing it brings to the table that I like is 'load balancing', something you still can't do in the GUI. I can tell it to run these 90 jobs, with all their dependancies, and try and keep 12 of them running at any given time. One plate stops spinning and it puts another up to take its place.

However - I'm the only one touching it, enhancing it, supporting it. Not that others couldn't, but they do prefer that I do it for them. For now.

My .02...

ray.wurlod · Post by **ray.wurlod** » Sun Feb 26, 2006 2:36 pm

manishsk wrote:I have worked on modules which involved around 150 - 200 jobs running one shot. At such place I can't go for sequencers.

Wrong. There is actually no limit.
To keep things easy to read/understand, build a hierarchy of control, using Job Activities to call other job sequences.
I'm confident you can find some criteria on which to base the grouping of your jobs.
So - and I've taken it as a given that your hardware is up to 150-200 simultaneous jobs - there is no reason to avoid job sequences just on the basis of number.
Did you know that, if you right click on a Job Activity in a job sequence design, you can "drill down" to the next level down (that is, to the job or job sequence that's invoked)? Just choose Open Job from the pop-up menu.

Oh, and it's job sequences, not sequencers. Or were you really referring to Sequencers? If there's a limit on the number of links, you can create a hierarchy of Sequencers for decision making in a job sequence.

manishsk · Post by **manishsk** » Mon Feb 27, 2006 3:16 am

ray.wurlod wrote:
manishsk wrote:I have worked on modules which involved around 150 - 200 jobs running one shot. At such place I can't go for sequencers.
Wrong. There is actually no limit.
To keep things easy to read/understand, build a hierarchy of control, using Job Activities to call other job sequences.
I'm confident you can find some criteria on which to base the grouping of your jobs.
So - and I've taken it as a given that your hardware is up to 150-200 simultaneous jobs - there is no reason to avoid job sequences just on the basis of number.
Did you know that, if you right click on a Job Activity in a job sequence design, you can "drill down" to the next level down (that is, to the job or job sequence that's invoked)? Just choose Open Job from the pop-up menu.

Oh, and it's job sequences, not sequencers. Or were you really referring to Sequencers? If there's a limit on the number of links, you can create a hierarchy of Sequencers for decision making in a job sequence.

Yes ray. You are correct, that we can create hierarchy and some ways to simplify the things. But when huge number of jobs comes I feel its better to cocentrate on jobs rather than putting more development time in developing job sequences. By BASIC code its easy to track down things, its what I feel.
(Ray I was refering to Job sequeces not sequencers. sry for that.)

Thibal · Post by **Thibal** » Mon Feb 27, 2006 4:05 am

Very interesting topic

I will follow Chuck advice and use Job Sequence in my current project. Using it, is the best way to progress and to make my own opinion.

If Mr Wurlod uses Job Sequences, it can't be bad

About my "cleanliness" issues, I'll find tips when developing (It's a pity it doesn't exist a kind of Sequence Container).

Thanks a lot everybody.

chulett · Post by **chulett** » Mon Feb 27, 2006 8:09 am

Thibal wrote:About my "cleanliness" issues, I'll find tips when developing (It's a pity it doesn't exist a kind of Sequence Container)

Actually, there is! It's called another Sequence job.

kumar_s · Post by **kumar_s** » Mon Feb 27, 2006 9:11 am

[quote="Thibal"]
If Mr Wurlod uses Job Sequences, it can't be bad

quote]

BTW, nothing like 'BAD' in DataStage, where IBM/Ascential might have purposefully introduced in their own produce. Its all about how effeciently we use the available option to acheive the task in an easiest way.
Obviously, Pros and Cons are available in both method. Job Sequence were not versatile in earliar versions. Now many built in features as been introduced, just drag-drop kind.
Still there are many situations where Job control wins the Job Sequence, as Craig mentioned. Nonlinear Itrerations, Custom error logs, Partial restratablility.....
Again its all depends on the effeciency of coding/coder.
Its all depends on the person and his profeciency in his approach.

Thibal · Post by **Thibal** » Mon Feb 27, 2006 9:15 am

chulett wrote:
Thibal wrote:About my "cleanliness" issues, I'll find tips when developing (It's a pity it doesn't exist a kind of Sequence Container)
Actually, there is! It's called another Sequence job.

Sorry, I didn't explain what I'm thinking about.

Indeed Sequence Job could call another Job Sequence made up of a job activity and two routine activities but it can't transmit in parameter the name of the job to launch.
If I'm wrong, I wouldn't say anything more before the end my development

A solution could be using Job Sequence to call a Routine Activity with my current "Job Control" Code

gbusson · Post by **gbusson** » Mon Feb 27, 2006 9:20 am

[quote="Thibal"][quote="chulett"][quote="Thibal"]About my "cleanliness" issues, I'll find tips when developing (It's a pity it doesn't exist a kind of Sequence Container)[/quote]
Actually, there is! It's called another Sequence job.

[/quote]
Sorry, I didn't explain what I'm thinking about.

Indeed Sequence Job could call another Job Sequence made up of a job activity and two routine activities but it can't transmit in parameter the name of the job to launch.
If I'm wrong, I wouldn't say anything more before the end my development

A solution could be using Job Sequence to call a Routine Activity with my current "Job Control" Code [img]http://www.reimsvdt.com/forums/html/emo ... dinina.gif[/img][/quote]

This is exactly what i was saying. Job Sequence is good indeed, but not as flexible as Job Control.
Instead of Job Activity, i use a Routine Activity to run jobs.

manishsk · Post by **manishsk** » Mon Feb 27, 2006 9:26 am

gbusson wrote:
Thibal wrote:
chulett wrote: Actually, there is! It's called another Sequence job.
Sorry, I didn't explain what I'm thinking about.

Indeed Sequence Job could call another Job Sequence made up of a job activity and two routine activities but it can't transmit in parameter the name of the job to launch.
If I'm wrong, I wouldn't say anything more before the end my development

A solution could be using Job Sequence to call a Routine Activity with my current "Job Control" Code

This is exactly what i was saying. Job Sequence is good indeed, but not as flexible as Job Control.
Instead of Job Activity, i use a Routine Activity to run jobs.

Enough. Enough. I love job control

Thibal · Post by **Thibal** » Mon Feb 27, 2006 9:31 am

kumar_s wrote:
Thibal wrote: If Mr Wurlod uses Job Sequences, it can't be bad
quote]
BTW, nothing like 'BAD' in DataStage, where IBM/Ascential might have purposefully introduced in their own produce. Its all about how effeciently we use the available option to acheive the task in an easiest way.
Obviously, Pros and Cons are available in both method. Job Sequence were not versatile in earliar versions. Now many built in features as been introduced, just drag-drop kind.
Still there are many situations where Job control wins the Job Sequence, as Craig mentioned. Nonlinear Itrerations, Custom error logs, Partial restratablility.....
Again its all depends on the effeciency of coding/coder.
Its all depends on the person and his profeciency in his approach.

If you prefer, if Mr Wurlod uses Job Sequences, it can be good (even if Job Control can be better in some cases , no one is perfect

)

manishsk · Post by **manishsk** » Mon Feb 27, 2006 9:35 am

Thibal wrote:
kumar_s wrote:
Thibal wrote: If Mr Wurlod uses Job Sequences, it can't be bad
quote]
BTW, nothing like 'BAD' in DataStage, where IBM/Ascential might have purposefully introduced in their own produce. Its all about how effeciently we use the available option to acheive the task in an easiest way.
Obviously, Pros and Cons are available in both method. Job Sequence were not versatile in earliar versions. Now many built in features as been introduced, just drag-drop kind.
Still there are many situations where Job control wins the Job Sequence, as Craig mentioned. Nonlinear Itrerations, Custom error logs, Partial restratablility.....
Again its all depends on the effeciency of coding/coder.
Its all depends on the person and his profeciency in his approach.
If you prefer, if Mr Wurlod uses Job Sequences, it can be good (even if Job Control can be better in some cases , no one is perfect )

Mr. Wurlod please use Job Control from Next Time !!
Please Please :D

I_Server_Whale · Post by **I_Server_Whale** » Mon Feb 27, 2006 9:51 am

I think by now it is pretty clear to everybody about the differences and similarities between a job sequence and a job control.

ByeBye Job Sequence.

byebye Job Control.

Thanks all

Naveen.

Thibal · Post by **Thibal** » Mon Feb 27, 2006 9:58 am

I deny all responsibilities for what happens in this topic.

kumar_s · Post by **kumar_s** » Mon Feb 27, 2006 10:33 am

The whole post is getting colorfull... Not only because of the involvement of various posters and their thoughts but also due to the whole bunch of Smilies used