Can multiple Sequencers call same job at same time?

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
horserider
Participant
Posts: 71
Joined: Mon Jul 09, 2007 1:12 pm

Can multiple Sequencers call same job at same time?

Post by horserider »

Below is the complete flow.

(1) Job 1 reads a text source and loads a temporary Teradata Table.
(2) Job 2 reads the Temporary Table and saves row count to a text file.
(3) If Text file in Job 2 has more than 0 rows, it runs a BTEQ script to
process rows from temporary table otherwise it terminates.

For Job 2 above, I have created a COMMON PARALLEL JOB that will accept a table name and also the target text file as parameter. It reads the table and write the rowcount to text file. Since the Table Name and Target File Names are passed as parameters, in one sense it can be run multiple times for different source and different target.

Question:-

I have 5 sequencers that runs in parallel that should call the Job 2. Is it OK to execute JOB 2 five times the same time through Sequencers?

If NO, what is the alternate method to get rowcount from a table for multiple parallel sequencers running at the same time?
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Only if the job is a Multi-Instance job. We do something like this and pass the table name as the Invocation ID.
-craig

"You can never have too many knives" -- Logan Nine Fingers
horserider
Participant
Posts: 71
Joined: Mon Jul 09, 2007 1:12 pm

Post by horserider »

So you mean

(1) Open Job and "CHECK" Allow Multiple Instances under Paramter Tab.
(2) In Sequencer from where the Job is called Pass the "Table Name" to
Invocation ID under Job Activity stage

Will this address my requirement?
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

At a high level, yes. Be aware sometimes you need a job parameter depending on where you are using it. And there is a macro to get the current Invocation Id in job.
-craig

"You can never have too many knives" -- Logan Nine Fingers
horserider
Participant
Posts: 71
Joined: Mon Jul 09, 2007 1:12 pm

Post by horserider »

I made the two changes and it seems that it is working but there is a sub routine that checks the actual row_count that was moved from table source to text target.

After calling the COMMON JOB, I am calling a Routine but is now returning me 0 rows, even when the job does show more than 0 rows were sent from table to text source. Even the text file shows more than 0 rows.

Will I have to change something in the Routine that gives me the ROW_COUNT for STAGE_NAME: LINK?

ROUTINE is passed 3 parameters as below

JOBNAME =
STAGELINKS=
JOBRUNID =

___________________________________________________________

$INCLUDE DSINCLUDE JOBCONTROL.H

err = ''
RtnSuccess = 0
RtnFailure = 1
RtnError = -1
RtnStatus = RtnError
Call DSLogInfo(" JOBNAME ":JOBNAME, ProgramName)
Call DSLogInfo("STAGELINKS ":STAGELINKS, ProgramName)
Call DSLogInfo(" JOBRUNID":JOBRUNID, ProgramName)
ProgramName = "MEDBMSCXXR999CheckLinkCntGTzero"
*myJOB = Trim(JOBNAME):".":Trim(JOBRUNID)
myJOB = Trim(JOBNAME)

Call DSLogInfo(" JobName: ":myJOB, ProgramName)
counter = DCount(STAGELINKS,",")
For i = 1 to counter

LinkToMonitor = Trim(Field(Field(STAGELINKS,",",i),":",2))
StageToMonitor= Trim(Field(Field(STAGELINKS,",",i),":",1))
JobHandle = DSAttachJob (myJOB, DSJ.ERRWARN )
result=DSGetLinkInfo( JobHandle,StageToMonitor,LinkToMonitor,DSJ.LINKROWCOUNT)
print "ForLoop, i : ":i

print "LinkName ":LinkToMonitor
print "StageName ":StageToMonitor
print " Link Info for ":LinkToMonitor:"is " :result

Call DSLogInfo("LinkName ":LinkToMonitor, ProgramName)
Call DSLogInfo("StageName ":StageToMonitor , ProgramName)
Call DSLogInfo(" Link Info for ":LinkToMonitor:"is " :result , ProgramName)
*Check result for error or value

Begin Case
Case result = DSJE.BADHANDLE
err = ' *JobHandle was invalid. Supplied job name: ':JOBNAME
RtnStatus = RtnError
Case result =DSJE.BADTYPE
err = ' * InfoType was unrecognized.'
RtnStatus = RtnError
Case result =DSJE.BADSTAGE
err = ' *StageName (':StageToMonitor:') does not refer to a known stage in the job (':JOBNAME:').'
RtnStatus = RtnError
Case result =DSJE.NOTINSTAGE
err = ' *StageName was DSJ.ME and the caller is not running within a stage.'
RtnStatus = RtnError
Case result =DSJE.BADLINK
err = ' *LinkName (':LinkToMonitor:') does not refer to a known link for the stage in question (':StageToMonitor:') for the job in question(':JOBNAME:').'
RtnStatus = RtnError
Case result = 0
RtnStatus = RtnFailure
Case result > 0
*for debugging only!*
PRINT "Link count for ":JOBNAME:":":StageToMonitor:":":LinkToMonitor:"=":result
RtnStatus = RtnSuccess
Case @TRUE
err = ' *Unexpected result from DSGetLinkInfo! Result=':result
RtnStatus = RtnError
End Case
print "EndCase, RtnStatus :":RtnStatus
print " RtnSuccess:":RtnSuccess
If RtnStatus <> RtnSuccess Then
Goto ExitRoutine
End

Next i


ExitRoutine:

If RtnStatus = RtnError Then
*for debugging only!*
PRINT err
Call DSLogWarn(err,ProgramName )
Call DSLogInfo("Routine returned error, aborting", ProgramName)
*Abort
End

Ans = RtnStatus
if Ans = 0 then
* Msg_Success = "The Sequencer ran Successfully"
*print "The Routine Status Success: ": Ans
Call DSLogInfo("The Routine Status : ": Ans , ProgramName)

end

if Ans <> 0 then
*Msg_Failure = "The Sequencer has the ":"Stage:** ":StageToMonitor: " ** and the Link :** ": LinkToMonitor :" ** has the RowCount : " : result :"(Zero)"
* print "The Routine Status Failed ": Msg_Failure
Call DSLogWarn("The Routine Status : ": Ans , ProgramName)


end
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

The 'name' you pass will need to include a dot and then the Invocation ID: Name=xxxxx.yyyy rather than Name=xxxxx as you are doing now.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply