Restarting a job with the same parameters

rleishman · Post by **rleishman** » Sun Feb 12, 2006 7:57 pm

We're using the standard checkpoint functionality to make our job sequences restartable. The only problem is that when you restart the top-level sequence, it asks for the parameters to be re-entered.

I want an easy foolproof way to restart any restartable job seq using the same parameters as when it failed.

My guess is that it is technically possible. If you know the DS database structure of the logs, you could write a routine that accepted a job name as input, found the latest job log for the job, sucked the parameters last used out of the log, and executed the job.

Problem is, this would take me a week and it's outside the scope of my job. ie. It's a nice-to-have, and we all know what happens to them.

What I'm hoping is that someone more talented than me has already done it, and is just dying to share their code. I am confident enough with BASIC that I reckon I could tweak anything similar to my requirements.

Thanks in advance...

kduke · Post by **kduke** » Sun Feb 12, 2006 8:08 pm

EtlStats gets parameter history as well as row counts. It stores the results in both tables and hash files. You can easily select th hash files and get what you want.

ray.wurlod · Post by **ray.wurlod** » Sun Feb 12, 2006 10:06 pm

The parameters last used are in the log "job started" entry for the previous run. Before the job starts they're also in the RT_STATUSxxx table for the job. Possibly easier to retrieve from there.

kduke · Post by **kduke** » Sun Feb 12, 2006 10:34 pm

No way. Much easier in RT_LOGnnn.

rleishman · Post by **rleishman** » Mon Feb 13, 2006 1:54 am

Thanks guys.

Kim, I found the EtlGetRowCountsFromLog job in EtlStats. I can see how I would go about grabbing params from there.

Ray, I looked at RT_STATUSnnnn and didn't think it would help that much, but then I opened the file for a restartable job, and lo-and-behold we have a CHECKPOINT row. Looks like just the thing.

So, I choose one of these methods to grab my param names and values, then its just a matter of DSAttachJob to create a job handle, DSSetParam to set my params, then DSRunJob to kick it off.

Is that about it? Any pitfalls to watch out for?

ArndW · Post by **ArndW** » Mon Feb 13, 2006 2:25 am

No pitfalls unless you have encrypted values in there; I'm not sure that they will re-parse.

rleishman · Post by **rleishman** » Thu Mar 02, 2006 9:03 pm

Followup: For anyone interested, I wrote the following routine called "RestartJob" that is no less nasty than it is cheap. It takes a single argument: pJobName - the name of the Job/JobSeq to restart.

It does the following:
- Finds the job
- Finds the previous settings of parameters
- Sets parameters to their previous settings, ignoring encrypted params and those with default of $PROJDEF.
- Calls UtilityRunJob to launch the job.

It does NOT:
- Handle encrypted parameters with any value other than $PROJDEF
- Handle non-$PROJDEF values for params with default $PROJDEF.
- Handle errors of any kind. eg. If your job is not in status Finished or Aborted/Restartable, then brace yourself.

Anyway, it works for my purposes.

Code: Select all

$INCLUDE DSINCLUDE JOBCONTROL.H

      Deffun UtilityRunJob(A1, A2, A3, A4) Calling "DSX.UTILITYRUNJOB"

      Equate RoutineName To 'RestartJob'
      lJobHandle = DSAttachJob(pJobName, DSJ.ERRFATAL)


      lParamList = DSGetJobInfo (lJobHandle, DSJ.PARAMLIST)
      lParamCount = Dcount(lParamList,',')

      lParamSet = ""

      For lParamNum = 1 to lParamCount
         lParamName = Field(lParamList, ',', lParamNum)

         If DSGetParamInfo(lJobHandle, lParamName, DSJ.PARAMTYPE) <> 1 Then

            If DSGetParamInfo(lJobHandle, lParamName, DSJ.PARAMDEFAULT) <> "$PROJDEF" Then
               lParamValue = DSGetParamInfo(lJobHandle, lParamName, DSJ.PARAMVALUE)
               lParamSet := "|" : lParamName : "=" : lParamValue
            End
         End
      Next lParamNum

      lParamSet = lParamSet[2,1000]

      lErrCode = DSDetachJob(lJobHandle)

      Ans = UtilityRunJob(pJobName, lParamSet, 0, 0)

kduke · Post by **kduke** » Thu Mar 02, 2006 9:14 pm

Nice.

chulett · Post by **chulett** » Tue May 09, 2006 4:55 pm

I suddenly find myself coveting a utility like this. Any clue what it would take to handle encrypted parameters?

kduke · Post by **kduke** » Tue May 09, 2006 6:07 pm

I want that answer too.

rleishman · Post by **rleishman** » Wed May 10, 2006 1:03 am

I've had no reason to handle the encrypted params - the only ones I use are always $PROJDEF - no exceptions - ever.

In testing the script I noticed that the value returned by DSGetParamInfo() for encrypted params was not the value you see in the DSParams file. It was just 2 or 3 chars of rubbish (rather than 20 or so chars of rubbish)

If DSGetParamInfo() can't be made to return the correct encrypted value, then I think you're stuffed, because I don't believe the correct encrypted value is in the logs either.

ray.wurlod · Post by **ray.wurlod** » Wed May 10, 2006 2:24 am

Have you actually tested with an encrypted parameter?
That aside, if DSGetParamInfo(hJob, paramname, DSJ.PARAMVALUE) and DSSetParam(hJob, paramname, paramvalue) can handle encrypted parameters, then it ought to be possible to create another routine - somewhat a clone of UtilityRunJob - to run the job collecting the encrypted value by other means.

chulett · Post by **chulett** » Wed May 10, 2006 6:51 am

rleishman wrote:I've had no reason to handle the encrypted params - the only ones I use are always $PROJDEF - no exceptions - ever.

We know you didn't need it, but others would. Tons of peoples, like my current client, started on a version that didn't support $PROJDEF so didn't have the luxury of your 'no exceptions' rule.

When I get some time, I figured I would just try it with encrypted passwords and see if it works as is. You never know. As noted, DSGetParamInfo can handle them and if that rubbish is the right rubbish... all will be golden. Stay tuned.

chulett · Post by **chulett** » Wed May 10, 2006 4:37 pm

FYI:

Code: Select all

JobControl (fatal error from DSGetParamInfo): Job control fatal error (-16)
(DSGetParamInfo) Cannot get the Default or Designed Default of an encrypted parameter for any job except the current job.

ray.wurlod · Post by **ray.wurlod** » Thu May 11, 2006 12:42 am

Well, it's not strongly encrypted, you could decrypt it.