Job Control General Question
Moderators: chulett, rschirm, roy
Finally decided to throw my two cents in the ring on this. I work with about a dozen other developers right now and I'd say that I am the only one out of them that is comfortable writing my own 'Job Control' code.
Experience definitely colors that picture. I haven't been using the product since Version 0 like some folks here, but did start back with 3.1 when there was no such thing as a Sequence job. You wanted something to 'sequence' your jobs you wrote it yourself. Uphill. Both ways. In the snow. Barefoot.
The others, who in spite of the fact they may have done this pushing three years, have always been on a flavor of 7.x and so have always had Sequence jobs in one form or another. For them, I've found that hand coded job control is never a 'Best Way' as they worry about having to maintain it, tweak it, etc. Sequencer jobs, however, they can crank out with their eyes closed - and sometimes I swear that's exactly what they do. [/joke]
The issue with Sequence jobs not being 'clean' is as much the skill of the designer as anything else, in my opinion. Especially with the latest 7.5.x enhancements. Build your Sequence jobs just like you would your other jobs - modularly. Restartable, maintainable modules. Sequence jobs are perfectly happy called 'sub-sequence' jobs down as many levels as you need. Let them handle their own errors now - no need for those giant 6.x spiderwebs to the Exception Handler any longer. Let it do it's own checkpoint restarting. Loop activity stages. Use Variables. Commands and Routines. Automatic generation of HTML documentation. Yada yada, etcetera etcetera. You'll find that 99.999% of the time, everything you 'need' is there and can be maintained by the other developers on staff and especially by those who follow.
That being said, we still use a ton of Job Control to handle our many hundreds of nightly jobs, KBA code we adapted and modified extensively. Besides a 'better' form of restartability, the biggest thing it brings to the table that I like is 'load balancing', something you still can't do in the GUI. I can tell it to run these 90 jobs, with all their dependancies, and try and keep 12 of them running at any given time. One plate stops spinning and it puts another up to take its place.
However - I'm the only one touching it, enhancing it, supporting it. Not that others couldn't, but they do prefer that I do it for them. For now.
My .02...
Experience definitely colors that picture. I haven't been using the product since Version 0 like some folks here, but did start back with 3.1 when there was no such thing as a Sequence job. You wanted something to 'sequence' your jobs you wrote it yourself. Uphill. Both ways. In the snow. Barefoot.
The others, who in spite of the fact they may have done this pushing three years, have always been on a flavor of 7.x and so have always had Sequence jobs in one form or another. For them, I've found that hand coded job control is never a 'Best Way' as they worry about having to maintain it, tweak it, etc. Sequencer jobs, however, they can crank out with their eyes closed - and sometimes I swear that's exactly what they do. [/joke]
The issue with Sequence jobs not being 'clean' is as much the skill of the designer as anything else, in my opinion. Especially with the latest 7.5.x enhancements. Build your Sequence jobs just like you would your other jobs - modularly. Restartable, maintainable modules. Sequence jobs are perfectly happy called 'sub-sequence' jobs down as many levels as you need. Let them handle their own errors now - no need for those giant 6.x spiderwebs to the Exception Handler any longer. Let it do it's own checkpoint restarting. Loop activity stages. Use Variables. Commands and Routines. Automatic generation of HTML documentation. Yada yada, etcetera etcetera. You'll find that 99.999% of the time, everything you 'need' is there and can be maintained by the other developers on staff and especially by those who follow.
That being said, we still use a ton of Job Control to handle our many hundreds of nightly jobs, KBA code we adapted and modified extensively. Besides a 'better' form of restartability, the biggest thing it brings to the table that I like is 'load balancing', something you still can't do in the GUI. I can tell it to run these 90 jobs, with all their dependancies, and try and keep 12 of them running at any given time. One plate stops spinning and it puts another up to take its place.
However - I'm the only one touching it, enhancing it, supporting it. Not that others couldn't, but they do prefer that I do it for them. For now.
My .02...
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Wrong. There is actually no limit.manishsk wrote:I have worked on modules which involved around 150 - 200 jobs running one shot. At such place I can't go for sequencers.
To keep things easy to read/understand, build a hierarchy of control, using Job Activities to call other job sequences.
I'm confident you can find some criteria on which to base the grouping of your jobs.
So - and I've taken it as a given that your hardware is up to 150-200 simultaneous jobs - there is no reason to avoid job sequences just on the basis of number.
Did you know that, if you right click on a Job Activity in a job sequence design, you can "drill down" to the next level down (that is, to the job or job sequence that's invoked)? Just choose Open Job from the pop-up menu.
Oh, and it's job sequences, not sequencers. Or were you really referring to Sequencers? If there's a limit on the number of links, you can create a hierarchy of Sequencers for decision making in a job sequence.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Yes ray. You are correct, that we can create hierarchy and some ways to simplify the things. But when huge number of jobs comes I feel its better to cocentrate on jobs rather than putting more development time in developing job sequences. By BASIC code its easy to track down things, its what I feel.ray.wurlod wrote:Wrong. There is actually no limit.manishsk wrote:I have worked on modules which involved around 150 - 200 jobs running one shot. At such place I can't go for sequencers.
To keep things easy to read/understand, build a hierarchy of control, using Job Activities to call other job sequences.
I'm confident you can find some criteria on which to base the grouping of your jobs.
So - and I've taken it as a given that your hardware is up to 150-200 simultaneous jobs - there is no reason to avoid job sequences just on the basis of number.
Did you know that, if you right click on a Job Activity in a job sequence design, you can "drill down" to the next level down (that is, to the job or job sequence that's invoked)? Just choose Open Job from the pop-up menu.
Oh, and it's job sequences, not sequencers. Or were you really referring to Sequencers? If there's a limit on the number of links, you can create a hierarchy of Sequencers for decision making in a job sequence.
(Ray I was refering to Job sequeces not sequencers. sry for that.)
Very interesting topic
I will follow Chuck advice and use Job Sequence in my current project. Using it, is the best way to progress and to make my own opinion.
If Mr Wurlod uses Job Sequences, it can't be bad
About my "cleanliness" issues, I'll find tips when developing (It's a pity it doesn't exist a kind of Sequence Container).
Thanks a lot everybody.
I will follow Chuck advice and use Job Sequence in my current project. Using it, is the best way to progress and to make my own opinion.
If Mr Wurlod uses Job Sequences, it can't be bad
About my "cleanliness" issues, I'll find tips when developing (It's a pity it doesn't exist a kind of Sequence Container).
Thanks a lot everybody.
Thibault Hamel
Nanterre - France
Nanterre - France
[quote="Thibal"]
If Mr Wurlod uses Job Sequences, it can't be bad
quote]
BTW, nothing like 'BAD' in DataStage, where IBM/Ascential might have purposefully introduced in their own produce. Its all about how effeciently we use the available option to acheive the task in an easiest way.
Obviously, Pros and Cons are available in both method. Job Sequence were not versatile in earliar versions. Now many built in features as been introduced, just drag-drop kind.
Still there are many situations where Job control wins the Job Sequence, as Craig mentioned. Nonlinear Itrerations, Custom error logs, Partial restratablility.....
Again its all depends on the effeciency of coding/coder.
Its all depends on the person and his profeciency in his approach.
If Mr Wurlod uses Job Sequences, it can't be bad
quote]
BTW, nothing like 'BAD' in DataStage, where IBM/Ascential might have purposefully introduced in their own produce. Its all about how effeciently we use the available option to acheive the task in an easiest way.
Obviously, Pros and Cons are available in both method. Job Sequence were not versatile in earliar versions. Now many built in features as been introduced, just drag-drop kind.
Still there are many situations where Job control wins the Job Sequence, as Craig mentioned. Nonlinear Itrerations, Custom error logs, Partial restratablility.....
Again its all depends on the effeciency of coding/coder.
Its all depends on the person and his profeciency in his approach.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
Sorry, I didn't explain what I'm thinking about.chulett wrote:Actually, there is! It's called another Sequence job.Thibal wrote:About my "cleanliness" issues, I'll find tips when developing (It's a pity it doesn't exist a kind of Sequence Container)
Indeed Sequence Job could call another Job Sequence made up of a job activity and two routine activities but it can't transmit in parameter the name of the job to launch.
If I'm wrong, I wouldn't say anything more before the end my development
A solution could be using Job Sequence to call a Routine Activity with my current "Job Control" Code
Thibault Hamel
Nanterre - France
Nanterre - France
[quote="Thibal"][quote="chulett"][quote="Thibal"]About my "cleanliness" issues, I'll find tips when developing (It's a pity it doesn't exist a kind of Sequence Container)[/quote]
Actually, there is! It's called another Sequence job. [/quote]
Sorry, I didn't explain what I'm thinking about.
Indeed Sequence Job could call another Job Sequence made up of a job activity and two routine activities but it can't transmit in parameter the name of the job to launch.
If I'm wrong, I wouldn't say anything more before the end my development
A solution could be using Job Sequence to call a Routine Activity with my current "Job Control" Code [img]http://www.reimsvdt.com/forums/html/emo ... dinina.gif[/img][/quote]
This is exactly what i was saying. Job Sequence is good indeed, but not as flexible as Job Control.
Instead of Job Activity, i use a Routine Activity to run jobs.
Actually, there is! It's called another Sequence job. [/quote]
Sorry, I didn't explain what I'm thinking about.
Indeed Sequence Job could call another Job Sequence made up of a job activity and two routine activities but it can't transmit in parameter the name of the job to launch.
If I'm wrong, I wouldn't say anything more before the end my development
A solution could be using Job Sequence to call a Routine Activity with my current "Job Control" Code [img]http://www.reimsvdt.com/forums/html/emo ... dinina.gif[/img][/quote]
This is exactly what i was saying. Job Sequence is good indeed, but not as flexible as Job Control.
Instead of Job Activity, i use a Routine Activity to run jobs.
gbusson wrote:Thibal wrote:Sorry, I didn't explain what I'm thinking about.chulett wrote: Actually, there is! It's called another Sequence job.
Indeed Sequence Job could call another Job Sequence made up of a job activity and two routine activities but it can't transmit in parameter the name of the job to launch.
If I'm wrong, I wouldn't say anything more before the end my development
A solution could be using Job Sequence to call a Routine Activity with my current "Job Control" Code
This is exactly what i was saying. Job Sequence is good indeed, but not as flexible as Job Control.
Instead of Job Activity, i use a Routine Activity to run jobs.
Enough. Enough. I love job control
If you prefer, if Mr Wurlod uses Job Sequences, it can be good (even if Job Control can be better in some cases , no one is perfect )kumar_s wrote:Thibal wrote: If Mr Wurlod uses Job Sequences, it can't be bad
quote]
BTW, nothing like 'BAD' in DataStage, where IBM/Ascential might have purposefully introduced in their own produce. Its all about how effeciently we use the available option to acheive the task in an easiest way.
Obviously, Pros and Cons are available in both method. Job Sequence were not versatile in earliar versions. Now many built in features as been introduced, just drag-drop kind.
Still there are many situations where Job control wins the Job Sequence, as Craig mentioned. Nonlinear Itrerations, Custom error logs, Partial restratablility.....
Again its all depends on the effeciency of coding/coder.
Its all depends on the person and his profeciency in his approach.
Thibault Hamel
Nanterre - France
Nanterre - France
Mr. Wurlod please use Job Control from Next Time !!Thibal wrote:kumar_s wrote:If you prefer, if Mr Wurlod uses Job Sequences, it can be good (even if Job Control can be better in some cases , no one is perfect )Thibal wrote: If Mr Wurlod uses Job Sequences, it can't be bad
quote]
BTW, nothing like 'BAD' in DataStage, where IBM/Ascential might have purposefully introduced in their own produce. Its all about how effeciently we use the available option to acheive the task in an easiest way.
Obviously, Pros and Cons are available in both method. Job Sequence were not versatile in earliar versions. Now many built in features as been introduced, just drag-drop kind.
Still there are many situations where Job control wins the Job Sequence, as Craig mentioned. Nonlinear Itrerations, Custom error logs, Partial restratablility.....
Again its all depends on the effeciency of coding/coder.
Its all depends on the person and his profeciency in his approach.
Please Please :D
-
- Premium Member
- Posts: 1255
- Joined: Wed Feb 02, 2005 11:54 am
- Location: United States of America
I think by now it is pretty clear to everybody about the differences and similarities between a job sequence and a job control.
ByeBye Job Sequence.
byebye Job Control.
Thanks all
Naveen.
ByeBye Job Sequence.
byebye Job Control.
Thanks all
Naveen.
Anything that won't sell, I don't want to invent. Its sale is proof of utility, and utility is success.
Author: Thomas A. Edison 1847-1931, American Inventor, Entrepreneur, Founder of GE
Author: Thomas A. Edison 1847-1931, American Inventor, Entrepreneur, Founder of GE