Processing Logic to Filter Job List

ccotton · Post by **ccotton** » Fri Apr 17, 2009 9:31 am

Hey Everyone,

I have a process that works but takes a while to run, and I'm looking to reduce the run times. I plan to do it through some sort of filtering where my problem lies. I have a process that goes through an entire project (about 3,000 jobs) and builds job reports for all jobs and process the reports and loads them into tables for reporting purposes. The problem lies in the about 3,000. I can process most jobs in a matter of 1-3 seconds, but I'm processing annual jobs that haven't run since the first of the year, first of the month, or even first of the week. Conceptually, i would like to be able filter and only process those jobs that have run since yesterday morning. I do know a way to do this but this is where the problem comes in. I get the project list and loop through the jobs checking the laststarttime from DSGetJobInfo output after a handle=DSAttachJob(jobname,DSJ.ERRNONE) of course. Once I have verified that the job ran since the time frame specified, it processes the records and things are great? Well the answer is that yes things are great to an extent however there are jobs that fail the attachment which are disregarded. It appears that my process gets to a point and just hangs. The server isn't overloaded or even running with high CPU or RAM utilization. Any ideas of other methods to go about performing this task that don't involve all of this extra overhead and the hanging problem? A side not when I process all the job reports I pull the start time and do a comparison there, but the process runs about 2 hours. I would like it to run in 30 minutes or less if possible. I thought I could filter the job list with the DSAttachJob logic because that's extremely fast usually. The hanging up is the only real problem with that process. If you need further details please let me know.

chulett · Post by **chulett** » Fri Apr 17, 2009 9:45 am

Wow... paragraphs would be nice. I've attached a couple of extra carriage returns for use in your next post.

Is this 'process' a job? A typical 'hang' in situations like this can occur if the job tries to attach to itself, any chance that is happening?

ccotton · Post by **ccotton** » Fri Apr 17, 2009 10:13 am

This was a rookie mistake. It would hang when trying to attach to itself and when attaching to the calling process.

Thanks for all your help!!! I have done this for many different tasks and I must have just forgot to correct that part lol. I copied the code from another process that I wrote about 1.5 years ago, and forgot the "ignore this job" logic!!! Works like a champ runs in under 15 minutes now!!!