Job Control question with multi-instance jobs

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Job Control question with multi-instance jobs

Post by ArndW »

I am trying to use the Job Control section to find out who called up a job. Using the DSGetJobInfo(DSJ.ME,DSJ.JOBCONTROLLER) works just find if the calling job is a single-instance job; but if it is a multi-instance job then I also need to get the controlling process' invocation id used to spawn my own job. Unfortunately, the value returned using the DSJ.JOBINVOCATIONID is the wrong invocation (that of the current process) and I cannot find another way to do this. Has anyone solved this problem before? Any suggestions?

Example: "Master.MInstance1" calls "Child.CInstance1"; from the Child process the DSGetJobInfo(DSJ.ME,DSJ.JOBCONTROLLER) call returns "Master" and DSGetJobInfo(DSJ.ME,DSJJOBINVOCATIONID) returns "CInstance1"; I need to find out that the child was called from "Master.MInstance1"
roy
Participant
Posts: 2598
Joined: Wed Jul 30, 2003 2:05 am
Location: Israel

Post by roy »

Hi,
The most simple way is to realize that invocationid is to single out one running instance of a job from another.
having that in mind, why not run the child multi instance job with the invocationid of the parent process, or one that combines the prent's invocationid in it; in a predifined syntax that you can reverse to gain the parent's invocationid.

IHTH,
Roy R.
Time is money but when you don't have money time is all you can afford.

Search before posting:)

Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Image
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Roy,

I wish I could do something like that, but it doesn't fit into the design here. The calling process is multi-instance and I need the child to be able to traverse up the calling stack to the originating process and get values from there. I've boiled down the problem description to make it understandable but there is more to the problem than I've described.

This really should be possible... if I could only find a way.
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

The parent job is stored in the log. I think it is the first or second record in the current grouping. Each run has a group of log records associated with it. The invocation id is field 7 or something like that in the log. I could look it up if you cannot find it.
Mamu Kim
roy
Participant
Posts: 2598
Joined: Wed Jul 30, 2003 2:05 am
Location: Israel

Post by roy »

sigh :(,
Is there anything you can do to make this more simple?
like use common and store an array/s of parent/child?
or add a prameter to the multiple instance jobs holding the controller.

or must you get the list of parent invocationids find running ones and fish log entries to verify this is the one you need?
Roy R.
Time is money but when you don't have money time is all you can afford.

Search before posting:)

Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Image
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Roy - I have used a COMMON block with some success, but to implement it correctly would mean that calls and/or parameters would need to be added to the ~300 jobs here. All of the other inheritance portions are working and I'm stuck on this finding an instance of a parent process. I am trying to get this done a bit differently now, using the DSJ.INVOCATIONS list of all parent processes and examing the logs of all instances found for a reference to the calling process. I'm still coding this potential workaround... I was hoping that the value would be stored in a COMMON element somewhere. I wish I had the code to DSGetJobInfo to see where the DSJ.JOBCONTROLLER is coming from.
roy
Participant
Posts: 2598
Joined: Wed Jul 30, 2003 2:05 am
Location: Israel

Post by roy »

Did you try to put a trace on and use a test of DSGetJobInfo from designer or manager to try if you can get some idea where it comes from?

I guess if you could consider changing the invocationids for ~300 jobs you might find some logic of manipulation on a dsx file to get it, but it might not be as do-able in some cases,though going over that dsx manually should be a 2-3 weeks job or so.

again if you can get Ascential to supply this info in an orderly mannor its the best way :).

I guess I never got to this position since I always use the called job as the invocationid of the parrent or the other way around if needed so no worries for me :wink:
Roy R.
Time is money but when you don't have money time is all you can afford.

Search before posting:)

Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Image
roy
Participant
Posts: 2598
Joined: Wed Jul 30, 2003 2:05 am
Location: Israel

Post by roy »

ArndW,
you do realize regarding the fix from Ascential, if it gives the stated behaviour it won't contain the invocationids, unless Ascential said they will give you that info as well.
as far as I read it says it should give a controling job chain seperated by dots.
Roy R.
Time is money but when you don't have money time is all you can afford.

Search before posting:)

Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Image
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Roy et al.

I put together a short Uv/Basic function that returns the parent process with invocation id, if present. I didn't even have to cheat, but used officially published calls.

Essentially I get the parent process's name (DSJ.JOBCONTROLLER), attach to the parent and get it's list of invocations using DSJ.JOBINVOCATIONS, loop through the invocations until I find an instance that matches 2 criteria - it's status must be running and the last log entry will contain the child's fully qualified job name [it will either be running that job or waiting for it, depending upon timings].

The job wasn't that complicated once I realized I could loop through the invocations. I can paste the code here if anyone is interested.
battaliou
Participant
Posts: 155
Joined: Mon Feb 24, 2003 7:28 am
Location: London
Contact:

Post by battaliou »

Cool, post the code. I would have passed the invocation id down to the sub job as a seperate parameter...
3NF: Every non-key attribute must provide a fact about the key, the whole key, and nothing but the key. So help me Codd.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

The function is RtrJobGetParent(JobNameToCheck)

Code: Select all

*******************************************************************************
** UV/BASIC Function to return the fully qualified name of the parent job.   **
** Since the DSGetJobInfo function with DSJ.JOBCONTROLLER doesn't return the **
** instance extension (if any) of a job, this routine goes through all of the**
** invocations of a job's parent and checks to see that two conditions are   **
** met to get the correct invocation:                                        **
** (a) if the invocation has a status of RUNNING                             **
** (b) if the invocations's last log entry contains the job name (and the    **
**     instance) of the child process.                                       **
**                                                                           **
** When both conditions are true then we have found the correct invocating   **
** job instance and return that.                                             **
**                                                                           **
** Parameters                                                                **
** ==========                                                                **
** JobNameToCheck - Job name or empty for current job.                       **
**                                                                           **
** Return Values                                                             **
** =============                                                             **
** Fully qualified jobname                                                   **
**                                                                           **
** Ver.  Date       Author   Description                                     **
** ===== ========== ======== ================================================**
** 2.0.0 24/03/2005 AWussing Initial coding, Testing & Implementation        **
**                                                                           **
*******************************************************************************
   ** Program Includes, COMMON, and Constants **
   *********************************************
   $INCLUDE DSS_JOB_INCLUDE DSSJobControl.H
   $INCLUDE DSINCLUDE JOBCONTROL.H
   EQUATE ProgramName TO 'RtrJobGetParent'

   *******************************************
   ** Attach to the job and get information **
   *******************************************
   Ans = '' ;** assume we have no parent or something went wrong.
   IF (NOT(JobNameToCheck) OR DSGetJobInfo(DSJ.ME,DSJ.JOBNAME)=JobNameToCheck) THEN MyJobHandle = DSJ.ME
                                                                               ELSE MyJobHandle = DSAttachJob(JobNameToCheck,DSJ.ERRFATAL)
   IF (MyJobHandle)
   THEN
      ****************************************
      ** Build the fully qualified job name **
      ****************************************
      MyParentName    = TRIM(DSGetJobInfo(MyJobHandle,DSJ.JOBCONTROLLER))
      MyJobName       = DSGetJobInfo(MyJobHandle,DSJ.JOBNAME)
      MyJobInvocation = DSGetJobInfo(MyJobHandle,DSJ.JOBINVOCATIONID)
      IF (MyJobInvocation) THEN MyJobName := '.':MyJobInvocation
      IF (MyParentName)
      THEN
         ************************************
         ** Get the Parent job information **
         ************************************ 
         ParentJobHandle       = DSAttachJob(MyParentName,DSJ.ERRFATAL)
         ParentJobInvocations  = DSGetJobInfo(ParentJobHandle,DSJ.JOBINVOCATIONS)
         ParentInvocationCount = DCOUNT(ParentJobInvocations,Comma)
         FOR i = 1 TO ParentInvocationCount
            InvocationJobName   = FIELD(ParentJobInvocations,Comma,i)
            InvocationJobHandle = DSAttachJob(InvocationJobName,DSJ.ERRFATAL)
            ParentStatus        = DSGetJobInfo(InvocationJobHandle,DSJ.JOBSTATUS)
            IF (ParentStatus = DSJS.RUNNING)
            THEN
               ***************************************************************************
               ** If the job is still running we might have a winner, see what the last **
               ** job entry contains. If the name of this job is in the text string is  **
               ** in the string we really have hit the jackpot.                         **
               ***************************************************************************
               LastLogId = DSGetNewestLogId(InvocationJobHandle,DSJ.LOGANY)
               LogString = DSGetLogEntry(InvocationJobHandle,LastLogId)
               IF ( INDEX(LogString,MyJobName,1) )
               THEN 
                  Ans = InvocationJobName ;** success!
                  EXIT                    ;** inelegant, but we save some loop iterations.
               END ;** of if-then we have a match
            END ;** of if-then the job is running
            Dummy = DSDetachJob(ParentJobHandle)
         NEXT i
      END ; ** of if-then we have a parent
      IF ( MyJobHandle#DSJ.ME ) THEN Dummy = DSDetachJob(MyJobHandle)
   END
   ELSE
      CALL DSLogFatal('Unable to attach to job "':JobNameToCheck:'".',ProgramName)
   END ;** of if-then-else we couldn't open the job requested 
Post Reply