> ----------
> From: Scott Brooks[SMTP:SBROOKS@psu.edu]
> Sent: Thursday, 31 August 2000 03:42
> To: ray.wurlod@informix.com
> Subject: Re: RE: Number of Items in Job
>
> Hi Ray,
>
> I saw your note to the DataStage Listserv list. Would you have some
> sample code that I could leverage for the "Multiple controlled jobs"
> solution?
>
> Thanks,
>
> Scott
>
> Scott Brooks
> Information Technology
> Hershey Medical Center
> (717)531-8374
> sbrooks@psu.edu
>
What follows is best viewed using a non-proportional font...
>
>
* DataStage BASIC does not have a ForEach construct, so we fake it,
* for example with a dynamic array of job names. This could also be
* handled with a List type job parameter, reading them from a file, etc.
JobList = "JobA"
JobList = "JobB"
JobList = "JobC"
* Obtain parameter values to load into these jobs. They can be gotten
* by any means (e.g. by reading a file). Here we will interrogate the
* parameter values of this, the controlling, job. We load these values
* into meaningfully named variables. For example:
DBPassword = DSGetParamInfo(DSJ.ME, "PWD", DSJ.PARAMVALUE)
* This construct runs the jobs consecutively. Adapt the code if there
* are jobs to run simultaneously.
Loop
Remove Name From JobList Setting MoreJobs
GoSub RunThisJob
While MoreJobs
Repeat
GoTo MainExit
RunThisJob: ; * Here we run a single job.
* Attach the controlled job to this, the controlling job.
* Its like talking to children - the first sentence is "Come here.".
hJob = DSAttachJob(Name, DSJ.ERRWARN)
* Not shown here, but you really do need to check that the job is runnable,
* by checking its current status. If it aborted you may need to do a reset run.
* If its already running, you dont want to try running it again, and so on.
* Set the controlled jobs parameter values. Any parameter values not provided
* will not be prompted for; rather the default values will be used.
ErrCode = DSSetParam(hJob, "PWD", DBPassword)
* Submit a request to the server to run the controlled job.
ErrCode = DSRunJob(hJob, DSJ.RUNNORMAL)
* Wait for it to finish
ErrCode = DSWaitForJob(hJob)
* Check its exit status, and any other information that you need the job
* handle to obtain.
ExitStatus = DSGetJobInfo(hJob, DSJ.JOBSTATUS)
RejectsEntry = DSGetNewestLogID(hJob, DSJ.REJECT)
If RejectsEntry 0
Then
LogEntry = DSGetLogEntry(RejectsEntry)
End
If ExitStatus DSJS.RUNOK And ExitStatus DSJS.RUNWARN
Then
* processing for failed job
End
* Detach the job to prevent future "attach-locked" problems
* (Note that the automatically generated code misses this bit.)
ErrCode = DSDetachJob(hJob)
Return ; * end of RunThisJob routine
MainExit:
Multiple Controlled Jobs
The next level of complexity is to be able to control more than one job at the same time, that is, run jobs in parallel.
Firstly, you do not wait for a job to finish - you have other jobs to manage as well.
We keep an array of all the jobs to be run and what the current status of each job is. This also includes information about which jobs have to wait for which other jobs.
Once every x seconds, we look through the array...
* to see if any currently running jobs have finished (by checking the
status)
* to see if any jobs can be started as all their predecessors have
finished successfully
Our control job does a whole lot more than this as well, but this is enough for the next level of complexity.
Just one last thought. All this code is written as a function which gets called from job control rather than putting the code directly in job control. This way we can easily write and maintain as many controlling jobs as we like. At times, our controlling jobs can be nested 3 or 4 levels deep before a real job that actually does some work is called.
Ray has a good point about keeping you job definitions in a file or similar. We keep our job details in the database. The function I referred to above takes a parameter which identifies the name of the suite it is to run.
-----Original Message-----
From: Ray Wurlod [SMTP:ray.wurlod@informix.com]
Sent: Friday, September 01, 2000 10:41 AM
To: Scott Brooks; informix-datastage@oliver.com
Subject: Multiple Controlled Jobs
> ----------
> From: Scott Brooks[SMTP:SBROOKS@psu.edu]
> Sent: Thursday, 31 August 2000 03:42
> To: ray.wurlod@informix.com
> Subject: Re: RE: Number of Items in Job
>
> Hi Ray,
>
> I saw your note to the DataStage Listserv list. Would you have some
> sample code that I could leverage for the "Multiple controlled jobs"
> solution?
>
> Thanks,
>
> Scott
>
> Scott Brooks
> Information Technology
> Hershey Medical Center
> (717)531-8374
> sbrooks@psu.edu
>
What follows is best viewed using a non-proportional font...
>
>
* DataStage BASIC does not have a ForEach construct, so we fake it,
* for example with a dynamic array of job names. This could also be
* handled with a List type job parameter, reading them from a file,
etc.
JobList = "JobA"
JobList = "JobB"
JobList = "JobC"
* Obtain parameter values to load into these jobs. They can be gotten
* by any means (e.g. by reading a file). Here we will interrogate the
* parameter values of this, the controlling, job. We load these
values
* into meaningfully named variables. For example:
DBPassword = DSGetParamInfo(DSJ.ME, "PWD", DSJ.PARAMVALUE)
* This construct runs the jobs consecutively. Adapt the code if there
* are jobs to run simultaneously.
Loop
Remove Name From JobList Setting MoreJobs
GoSub RunThisJob
While MoreJobs
Repeat
GoTo MainExit
RunThisJob: ; * Here we run a single job.
* Attach the controlled job to this, the controlling job.
* Its like talking to children - the first sentence is "Come here.".
hJob = DSAttachJob(Name, DSJ.ERRWARN)
* Not shown here, but you really do need to check that the job is
runnable,
* by checking its current status. If it aborted you may need to do a
reset run.
* If its already running, you dont want to try running it again, and
so on.
* Set the controlled jobs parameter values. Any parameter values not
provided
* will not be prompted for; rather the default values will be used.
ErrCode = DSSetParam(hJob, "PWD", DBPassword)
* Submit a request to the server to run the controlled job.
ErrCode = DSRunJob(hJob, DSJ.RUNNORMAL)
* Wait for it to finish
ErrCode = DSWaitForJob(hJob)
* Check its exit status, and any other information that you need the
job
* handle to obtain.
ExitStatus = DSGetJobInfo(hJob, DSJ.JOBSTATUS)
RejectsEntry = DSGetNewestLogID(hJob, DSJ.REJECT)
If RejectsEntry 0
Then
LogEntry = DSGetLogEntry(RejectsEntry)
End
If ExitStatus DSJS.RUNOK And ExitStatus DSJS.RUNWARN
Then
* processing for failed job
End
* Detach the job to prevent future "attach-locked" problems * (Note
that the automatically generated code misses this bit.)
ErrCode = DSDetachJob(hJob)
Return ; * end of RunThisJob routine
MainExit:
*************************************************************************
This e-mail and any files transmitted with it may be confidential and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in
error, please notify the sender by return e-mail, and delete this e-mail from your in-box. Do not copy it to anybody else
*************************************************************************
Firstly, you do not wait for a job to finish - you have other jobs to manage as well.
We keep an array of all the jobs to be run and what the current status of each job is. This also includes information about which jobs have to wait for which other jobs.
Once every x seconds, we look through the array...
* to see if any currently running jobs have finished (by checking the
status)
* to see if any jobs can be started as all their predecessors have
finished successfully
Our control job does a whole lot more than this as well, but this is enough for the next level of complexity.
Just one last thought. All this code is written as a function which gets called from job control rather than putting the code directly in job control. This way we can easily write and maintain as many controlling jobs as we like. At times, our controlling jobs can be nested 3 or 4 levels deep before a real job that actually does some work is called.
Ray has a good point about keeping you job definitions in a file or similar. We keep our job details in the database. The function I referred to above takes a parameter which identifies the name of the suite it is to run.
-----Original Message-----
From: Ray Wurlod [SMTP:ray.wurlod@informix.com]
Sent: Friday, September 01, 2000 10:41 AM
To: Scott Brooks; informix-datastage@oliver.com
Subject: Multiple Controlled Jobs
> ----------
> From: Scott Brooks[SMTP:SBROOKS@psu.edu]
> Sent: Thursday, 31 August 2000 03:42
> To: ray.wurlod@informix.com
> Subject: Re: RE: Number of Items in Job
>
> Hi Ray,
>
> I saw your note to the DataStage Listserv list. Would you have some
> sample code that I could leverage for the "Multiple controlled jobs"
> solution?
>
> Thanks,
>
> Scott
>
> Scott Brooks
> Information Technology
> Hershey Medical Center
> (717)531-8374
> sbrooks@psu.edu
>
What follows is best viewed using a non-proportional font...
>
>
* DataStage BASIC does not have a ForEach construct, so we fake it,
* for example with a dynamic array of job names. This could also be
* handled with a List type job parameter, reading them from a file,
etc.
JobList = "JobA"
JobList = "JobB"
JobList = "JobC"
* Obtain parameter values to load into these jobs. They can be gotten
* by any means (e.g. by reading a file). Here we will interrogate the
* parameter values of this, the controlling, job. We load these
values
* into meaningfully named variables. For example:
DBPassword = DSGetParamInfo(DSJ.ME, "PWD", DSJ.PARAMVALUE)
* This construct runs the jobs consecutively. Adapt the code if there
* are jobs to run simultaneously.
Loop
Remove Name From JobList Setting MoreJobs
GoSub RunThisJob
While MoreJobs
Repeat
GoTo MainExit
RunThisJob: ; * Here we run a single job.
* Attach the controlled job to this, the controlling job.
* Its like talking to children - the first sentence is "Come here.".
hJob = DSAttachJob(Name, DSJ.ERRWARN)
* Not shown here, but you really do need to check that the job is
runnable,
* by checking its current status. If it aborted you may need to do a
reset run.
* If its already running, you dont want to try running it again, and
so on.
* Set the controlled jobs parameter values. Any parameter values not
provided
* will not be prompted for; rather the default values will be used.
ErrCode = DSSetParam(hJob, "PWD", DBPassword)
* Submit a request to the server to run the controlled job.
ErrCode = DSRunJob(hJob, DSJ.RUNNORMAL)
* Wait for it to finish
ErrCode = DSWaitForJob(hJob)
* Check its exit status, and any other information that you need the
job
* handle to obtain.
ExitStatus = DSGetJobInfo(hJob, DSJ.JOBSTATUS)
RejectsEntry = DSGetNewestLogID(hJob, DSJ.REJECT)
If RejectsEntry 0
Then
LogEntry = DSGetLogEntry(RejectsEntry)
End
If ExitStatus DSJS.RUNOK And ExitStatus DSJS.RUNWARN
Then
* processing for failed job
End
* Detach the job to prevent future "attach-locked" problems * (Note
that the automatically generated code misses this bit.)
ErrCode = DSDetachJob(hJob)
Return ; * end of RunThisJob routine
MainExit:
*************************************************************************
This e-mail and any files transmitted with it may be confidential and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in
error, please notify the sender by return e-mail, and delete this e-mail from your in-box. Do not copy it to anybody else
*************************************************************************
>>> Ray Wurlod writes:
What follows is best viewed using a non-proportional font...
* DataStage BASIC does not have a ForEach construct, so we fake it,
* for example with a dynamic array of job names. * David Barham writes
The next level of complexity is to be able to control more than one job at the same time, that is, run jobs in parallel*.
What follows is best viewed using a non-proportional font...
* DataStage BASIC does not have a ForEach construct, so we fake it,
* for example with a dynamic array of job names. * David Barham writes
The next level of complexity is to be able to control more than one job at the same time, that is, run jobs in parallel*.