Hi,
We are using a LoadBalancing routine to run "N" jobs in parallel from a given list of jobs. Given below is the code snippet for the same.
The code below achieves the following functionality :- It starts N jobs in parallel but only after all the N jobs finish execution it takes the next set of "N" jobs to run.
We want to modify the code such that as soon as one among the "N" job finished execution we want to start the next job in sequence.
Please let us know how can this be achieved by modifying the existing code given below.
Thanks,
* Array holding the jobs to run.
DIM SubFileRunListArray(30)
MAT SubFileRunListArray=0
DIM JobHandleArray(ParallelJobsToRun)
MAT JobHandleArray=0
* Setup to run "N" parallel jobs
Loop Until SubFileRunListArray(i) = 0
For j=1 To ParallelJobsToRun Step 1
If JobHandleArray(j)=0
Then
vJobName=SubFileRunListArray(i)
JobHandleArray(j) = DSAttachJob(vJobName, DSJ.ERRNONE)
JobHandleArray(j) = DSPrepareJob(JobHandleArray(j))
If NOT(JobHandleArray(j)) Then
Call DSLogWarn("Job Attach Failed: " : vJobName, "LoadBalancing")
End
ErrCode = DSRunJob(JobHandleArray(j), DSJ.RUNNORMAL)
i=i+1
end
NEXT j
* Check if any of the jobs started above is finished. In that case make that jobs handle as null. Then start the next jobs in seq.
For k=1 To ParallelJobsToRun Step 1
JobCurrStatus=DSGetJobInfo(JobHandleArray(k),DSJ.JOBSTATUS)
JobName=DSGetJobInfo(JobHandleArray(k),DSJ.JOBNAME)
If JobCurrStatus=DSJS.RUNNING
Then
Call DSLogInfo("Running ":JobName, "LoadBalancing")
ErrCode=DSWaitForJob(JobHandleArray(k))
JobHandleArray(k)=0
End
If JobCurrStatus=DSJS.RUNFAILED Or JobCurrStatus=DSJS.CRASHED
Then
Call DSLogWarn("Aborted ":JobName, "LoadBalancing")
JobHandleArray(k)=0
End
If JobCurrStatus=DSJS.RUNOK Or JobCurrStatus=DSJS.RUNWARN
Then
Call DSLogInfo ("Finished ":JobName, "LoadBalancing")
JobHandleArray(k)=0
End
Next k
LeftJobs = ctrJobsToRun - i
If LeftJobs < ParallelJobsToRun
Then
ParallelJobsToRun = LeftJobs +1
End
Repeat
Load Balancing
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 232
- Joined: Fri Aug 04, 2006 1:20 am
- Location: Bangalore
Ken Bland has a nice utility in his website, for doing this.
if you want to change your code, the first thing you have to remove is DSWaitForJob(JobHandleArray(k)), it only make the control to wait until the job finishes.
And other thing is, i dont see DSDetach anywhere in your code. Have you posted the entire code or a part of your code? How the variable vJobName is assigned? I have commented the DSWaitForJob , and added JobHandleArray(j) = 0 when the job is not attached. I didnt even test this in the DS environment, i have just modified your code , take this as a starting point and proceed
if you want to change your code, the first thing you have to remove is DSWaitForJob(JobHandleArray(k)), it only make the control to wait until the job finishes.
And other thing is, i dont see DSDetach anywhere in your code. Have you posted the entire code or a part of your code? How the variable vJobName is assigned? I have commented the DSWaitForJob , and added JobHandleArray(j) = 0 when the job is not attached. I didnt even test this in the DS environment, i have just modified your code , take this as a starting point and proceed
Code: Select all
* Array holding the jobs to run.
DIM SubFileRunListArray(30)
MAT SubFileRunListArray=0
DIM JobHandleArray(ParallelJobsToRun)
MAT JobHandleArray=0
* Setup to run "N" parallel jobs
Loop Until SubFileRunListArray(i) = 0
For j=1 To ParallelJobsToRun Step 1
If JobHandleArray(j)=0
Then
vJobName=SubFileRunListArray(i)
JobHandleArray(j) = DSAttachJob(vJobName, DSJ.ERRNONE)
JobHandleArray(j) = DSPrepareJob(JobHandleArray(j))
If NOT(JobHandleArray(j)) Then
Call DSLogWarn("Job Attach Failed: " : vJobName, "LoadBalancing")
JobHandleArray(j) = 0
End
ErrCode = DSRunJob(JobHandleArray(j), DSJ.RUNNORMAL)
i=i+1
end
NEXT j
* Check if any of the jobs started above is finished. In that case make that jobs handle as null. Then start the next jobs in seq.
For k=1 To ParallelJobsToRun Step 1
JobCurrStatus=DSGetJobInfo(JobHandleArray(k),DSJ.JOBSTATUS)
JobName=DSGetJobInfo(JobHandleArray(k),DSJ.JOBNAME)
If JobCurrStatus=DSJS.RUNNING
Then
*Call DSLogInfo("Running ":JobName, "LoadBalancing")
*ErrCode=DSWaitForJob(JobHandleArray(k))
*JobHandleArray(k)=0
End
If JobCurrStatus=DSJS.RUNFAILED Or JobCurrStatus=DSJS.CRASHED
Then
Call DSLogWarn("Aborted ":JobName, "LoadBalancing")
JobHandleArray(k)=0
LeftJobs = ctrJobsToRun - i
End
If JobCurrStatus=DSJS.RUNOK Or JobCurrStatus=DSJS.RUNWARN
Then
Call DSLogInfo ("Finished ":JobName, "LoadBalancing")
JobHandleArray(k)=0
LeftJobs = ctrJobsToRun - i
End
Next k
*LeftJobs = ctrJobsToRun - i
If LeftJobs < ParallelJobsToRun
Then
ParallelJobsToRun = LeftJobs +1
End
Repeat
Hi,
Thanks for the response.
If we dont use DSWait function, how can we be sure that the job which has been started has completed. Because there are chances that the controlling routine will finish before all the jobs have been completed.
Alternative is in the end i can check if all JobHandleArray has the value '0' to make sure that all jobs have been completed.
I have used "JobHandleArray(k)=0" in place of Detach job after the job completes(success or failure). Please confirm if this is the right approach or do i need to use DSDetach explicitly.
vJobName takes the values from SubFileRunListArray which will contain the jobs to run.
Thanks for the response.
If we dont use DSWait function, how can we be sure that the job which has been started has completed. Because there are chances that the controlling routine will finish before all the jobs have been completed.
Alternative is in the end i can check if all JobHandleArray has the value '0' to make sure that all jobs have been completed.
I have used "JobHandleArray(k)=0" in place of Detach job after the job completes(success or failure). Please confirm if this is the right approach or do i need to use DSDetach explicitly.
vJobName takes the values from SubFileRunListArray which will contain the jobs to run.
-
- Premium Member
- Posts: 232
- Joined: Fri Aug 04, 2006 1:20 am
- Location: Bangalore
Yes, dont let your routine to complete until all the jobhandles are 0.
The current loop will execute until all the jobs are assigned, so start a loop after this, which will execute untill all jobhandles are 0. I would request to stop the process for a small amount of time by using SLEEP, instead of checking continiously.
The current loop will execute until all the jobs are assigned, so start a loop after this, which will execute untill all jobhandles are 0. I would request to stop the process for a small amount of time by using SLEEP, instead of checking continiously.
You cannot 'WaitForJob' to get the result you want. You need to loop through your list of jobs, starting whatever you can (without a wait) until you hit the limit. Then keep looping though the list, checking each running job to see if it has completed and starting new jobs when the running count drops below your limit. Exit the loop when all jobs have been run.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers