How to capture the total no. of rows passed through a link
Moderators: chulett, rschirm, roy
How to capture the total no. of rows passed through a link
I need to capture the total number of rows that pass through any link in a job . For eg. 1. The number of rows from Input to transformer 2. The number of rows from Transformer to ouput1. 3. The number of rows from Transformer to ouput2. and so on.
I have tried with DSGetLinkInfo.
Thanks
sun
I have tried with DSGetLinkInfo.
Thanks
sun
I'm curious why this "should be done in the transformer"? Do you really want to collect these stats every time a row comes through your job?
This is typically done after job. If you download Kim's code, you'll see how it can be accomplished. Also, you've said you tried using DSGetLinkInfo... how exactly did you try it? What errors did you get?
This is typically done after job. If you download Kim's code, you'll see how it can be accomplished. Also, you've said you tried using DSGetLinkInfo... how exactly did you try it? What errors did you get?
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
-
- Participant
- Posts: 37
- Joined: Mon Jan 24, 2005 10:12 am
Re: How to capture the total no. of rows passed through a li
Can you do it on the Unix level? Just use wc -l filename to count the total number of lines on the input file.
sun786 wrote:I need to capture the total number of rows that pass through any link in a job . For eg. 1. The number of rows from Input to transformer 2. The number of rows from Transformer to ouput1. 3. The number of rows from Transformer to ouput2. and so on.
I have tried with DSGetLinkInfo.
Thanks
sun
This is how you call DSGetStageInfo. This code is a part of DsWebMon.
Code: Select all
subroutine DwnGetMonitor(JobName, MonitorGrid)
* -----------------------------------------------------------------------
* Program Name : DwnGetMonitor
* Program Dir : DWNBP
* Created by : Kim Greggory Duke
* Copyright (C) : Duke Consulting
* Programmer : kgd
* Date Created : 12-01-2002
* -----------------------------------------------------------------------
* Last Modified : 07-16-2003
* -----------------------------------------------------------------------
* Language : UniVerse BASIC
* -----------------------------------------------------------------------
* Usage : DwnGetMonitor(JobName, MonitorGrid)
* -----------------------------------------------------------------------
* Notes :
* -----------------------------------------------------------------------
* initialize standard variables
* ----------------------------------------------------------------------
ProgId = 'DwnGetMonitor'
$INCLUDE DSINCLUDE JOBCONTROL.H
$include DwnVersion
* ----------------------------------------------------------------------
MonitorGrid = ''
* -----------------------------------------------------------------------
open 'DS_JOBS' to DsJobs else goto TheEnd
* -----------------------------------------------------------------------
JobStatus = ''
* -----------------------------------------------------------------------
OldJobName = JobName
if index(JobName, '.', 1) > 0 then
MultipleInstace = @true
InvocationId = field(JobName, '.', 2, 9999)
JobName = field(JobName, '.', 1)
end else
MultipleInstace = @false
InvocationId = ''
end
read JobRec from DsJobs, JobName then
JobNo = JobRec<5>
* print 'DSAttachJob(':JobName:',3)'
RunHandle = DSAttachJob(OldJobName, DSJ.ERRNONE)
JStat = DSGetJobInfo(RunHandle, DSJ.JOBSTATUS)
ContinueFlag = @true
CanRunFlag = 1
CanStopFlag = 0
CanResetFlag = 1
* -----------------------------------------------------------------------
begin case
case JStat = 0
JobStatus = 'Running'
CanRunFlag = 0
CanResetFlag = 0
CanStopFlag = 1
case JStat = 1
JobStatus = 'Finished'
case JStat = 2 ; * Finished with warnings
JobStatus = 'Finished'
case JStat = 3
ContinueFlag = @false
JobStatus = 'Aborted'
CanRunFlag = 0
case JStat = 11
JobStatus = 'Finished'
case JStat = 12 ; * Validated with warnings
JobStatus = 'Finished'
case JStat = 13
* ContinueFlag = @false
JobStatus = 'Finished'
case JStat = 21
JobStatus = 'Finished'
case JStat = 96
ContinueFlag = @false
JobStatus = 'Aborted'
CanRunFlag = 0
case JStat = 97
* ContinueFlag = @false
JobStatus = 'Stopped'
CanRunFlag = 0
case JStat = 98
JobStatus = 'Finished'
case JStat = 99
JobStatus = 'Finished'
case @true
ContinueFlag = @false
JobStatus = 'Not Compiled'
CanRunFlag = 0
CanResetFlag = 0
end case
* -----------------------------------------------------------------------
if ContinueFlag then
StageName = ''
StageStatus = ''
LinkName = ''
LinkType =''
RowCount = ''
StartedTime = ''
ElapsedTime = ''
RowsPerSec = ''
PercentCP = ''
* -----------------------------------------------------------------------
StartedDateTime = DSGetJobInfo(RunHandle, DSJ.JOBSTARTTIMESTAMP)
iStartedDate = iconv(StartedDateTime[1,10], 'D4-YMD')
iStartedTime = iconv(StartedDateTime[12,8], 'MT')
if JobStatus = 'Running' then
FinishedDateTime = oconv(date(), 'D4-YMD'):' ':oconv(time(), 'MTS')
end else
FinishedDateTime = DSGetJobInfo(RunHandle, DSJ.JOBLASTTIMESTAMP)
end
iFinishedDate = iconv(FinishedDateTime[1,10], 'D4-YMD')
iFinishedTime = iconv(FinishedDateTime[12,8], 'MT')
* print iFinishedTime:' - ':iStartedTime
if FinishedDateTime<>'' and StartedDateTime<>'' and iFinishedTime<>'' and iStartedTime<>'' then
ElapsedTime = ((iFinishedDate-iStartedDate)*60*60)+iFinishedTime-iStartedTime
end
if ElapsedTime<0 then
ElapsedTime = ''
end
* -----------------------------------------------------------------------
* Get stage names
* -----------------------------------------------------------------------
Cnt = 0
StageNames = DSGetJobInfo(RunHandle, DSJ.STAGELIST)
NoStages = dcount(StageNames, ',')
for i=1 to NoStages
StageName = field(StageNames, ',', i)
* print i:'. ':StageName
* -----------------------------------------------------------------------
* Get link names per stage
* -----------------------------------------------------------------------
LinkType = DSGetStageInfo(RunHandle, StageName, DSJ.STAGETYPE)
LinkNames = DSGetStageInfo(RunHandle, StageName, DSJ.LINKLIST)
NoLinks = dcount(LinkNames, ',')
for j=1 to NoLinks
LinkName = field(LinkNames, ',', j)
* print j:'. ':LinkName
* -----------------------------------------------------------------------
RowCount = DSGetLinkInfo(RunHandle, StageName, LinkName, DSJ.LINKROWCOUNT)
if ElapsedTime>0 and ElapsedTime<>'' then
RowsPerSec = (RowCount / ElapsedTime)
RowsPerSec = oconv(RowsPerSec, 'MD0')
end
* -----------------------------------------------------------------------
Cnt += 1
if j=1 then
MonitorGrid < 1, Cnt> = StageName
MonitorGrid < 2, Cnt> = JobStatus
end else
MonitorGrid < 1, Cnt> = ' '
MonitorGrid < 2, Cnt> = ' '
end
MonitorGrid < 3, Cnt> = LinkName
MonitorGrid < 4, Cnt> = LinkType
MonitorGrid < 5, Cnt> = RowCount
MonitorGrid < 6, Cnt> = StartedDateTime
MonitorGrid < 7, Cnt> = ElapsedTime
MonitorGrid < 8, Cnt> = RowsPerSec
MonitorGrid < 9, Cnt> = PercentCP
MonitorGrid <10, Cnt> = CanRunFlag
MonitorGrid <11, Cnt> = CanStopFlag
MonitorGrid <12, Cnt> = CanResetFlag
MonitorGrid <13, Cnt> = FinishedDateTime
next j
next i
end
JStat = DSDetachJob(RunHandle)
end
* -----------------------------------------------------------------------
TheEnd:
return
Mamu Kim
kduke wrote:This is how you call DSGetStageInfo. This code is a part of DsWebMon.
Hi Kim,
could you please guide me to write the MonitorGrid to a Flat file by Stage name, LinkName and other details
Thanks in advance
Code: Select all
subroutine DwnGetMonitor(JobName, MonitorGrid) * ----------------------------------------------------------------------- * Program Name : DwnGetMonitor * Program Dir : DWNBP * Created by : Kim Greggory Duke * Copyright (C) : Duke Consulting * Programmer : kgd * Date Created : 12-01-2002 * ----------------------------------------------------------------------- * Last Modified : 07-16-2003 * ----------------------------------------------------------------------- * Language : UniVerse BASIC * ----------------------------------------------------------------------- * Usage : DwnGetMonitor(JobName, MonitorGrid) * ----------------------------------------------------------------------- * Notes : * ----------------------------------------------------------------------- * initialize standard variables * ---------------------------------------------------------------------- ProgId = 'DwnGetMonitor' $INCLUDE DSINCLUDE JOBCONTROL.H $include DwnVersion * ---------------------------------------------------------------------- MonitorGrid = '' * ----------------------------------------------------------------------- open 'DS_JOBS' to DsJobs else goto TheEnd * ----------------------------------------------------------------------- JobStatus = '' * ----------------------------------------------------------------------- OldJobName = JobName if index(JobName, '.', 1) > 0 then MultipleInstace = @true InvocationId = field(JobName, '.', 2, 9999) JobName = field(JobName, '.', 1) end else MultipleInstace = @false InvocationId = '' end read JobRec from DsJobs, JobName then JobNo = JobRec<5> * print 'DSAttachJob(':JobName:',3)' RunHandle = DSAttachJob(OldJobName, DSJ.ERRNONE) JStat = DSGetJobInfo(RunHandle, DSJ.JOBSTATUS) ContinueFlag = @true CanRunFlag = 1 CanStopFlag = 0 CanResetFlag = 1 * ----------------------------------------------------------------------- begin case case JStat = 0 JobStatus = 'Running' CanRunFlag = 0 CanResetFlag = 0 CanStopFlag = 1 case JStat = 1 JobStatus = 'Finished' case JStat = 2 ; * Finished with warnings JobStatus = 'Finished' case JStat = 3 ContinueFlag = @false JobStatus = 'Aborted' CanRunFlag = 0 case JStat = 11 JobStatus = 'Finished' case JStat = 12 ; * Validated with warnings JobStatus = 'Finished' case JStat = 13 * ContinueFlag = @false JobStatus = 'Finished' case JStat = 21 JobStatus = 'Finished' case JStat = 96 ContinueFlag = @false JobStatus = 'Aborted' CanRunFlag = 0 case JStat = 97 * ContinueFlag = @false JobStatus = 'Stopped' CanRunFlag = 0 case JStat = 98 JobStatus = 'Finished' case JStat = 99 JobStatus = 'Finished' case @true ContinueFlag = @false JobStatus = 'Not Compiled' CanRunFlag = 0 CanResetFlag = 0 end case * ----------------------------------------------------------------------- if ContinueFlag then StageName = '' StageStatus = '' LinkName = '' LinkType ='' RowCount = '' StartedTime = '' ElapsedTime = '' RowsPerSec = '' PercentCP = '' * ----------------------------------------------------------------------- StartedDateTime = DSGetJobInfo(RunHandle, DSJ.JOBSTARTTIMESTAMP) iStartedDate = iconv(StartedDateTime[1,10], 'D4-YMD') iStartedTime = iconv(StartedDateTime[12,8], 'MT') if JobStatus = 'Running' then FinishedDateTime = oconv(date(), 'D4-YMD'):' ':oconv(time(), 'MTS') end else FinishedDateTime = DSGetJobInfo(RunHandle, DSJ.JOBLASTTIMESTAMP) end iFinishedDate = iconv(FinishedDateTime[1,10], 'D4-YMD') iFinishedTime = iconv(FinishedDateTime[12,8], 'MT') * print iFinishedTime:' - ':iStartedTime if FinishedDateTime<>'' and StartedDateTime<>'' and iFinishedTime<>'' and iStartedTime<>'' then ElapsedTime = ((iFinishedDate-iStartedDate)*60*60)+iFinishedTime-iStartedTime end if ElapsedTime<0 then ElapsedTime = '' end * ----------------------------------------------------------------------- * Get stage names * ----------------------------------------------------------------------- Cnt = 0 StageNames = DSGetJobInfo(RunHandle, DSJ.STAGELIST) NoStages = dcount(StageNames, ',') for i=1 to NoStages StageName = field(StageNames, ',', i) * print i:'. ':StageName * ----------------------------------------------------------------------- * Get link names per stage * ----------------------------------------------------------------------- LinkType = DSGetStageInfo(RunHandle, StageName, DSJ.STAGETYPE) LinkNames = DSGetStageInfo(RunHandle, StageName, DSJ.LINKLIST) NoLinks = dcount(LinkNames, ',') for j=1 to NoLinks LinkName = field(LinkNames, ',', j) * print j:'. ':LinkName * ----------------------------------------------------------------------- RowCount = DSGetLinkInfo(RunHandle, StageName, LinkName, DSJ.LINKROWCOUNT) if ElapsedTime>0 and ElapsedTime<>'' then RowsPerSec = (RowCount / ElapsedTime) RowsPerSec = oconv(RowsPerSec, 'MD0') end * ----------------------------------------------------------------------- Cnt += 1 if j=1 then MonitorGrid < 1, Cnt> = StageName MonitorGrid < 2, Cnt> = JobStatus end else MonitorGrid < 1, Cnt> = ' ' MonitorGrid < 2, Cnt> = ' ' end MonitorGrid < 3, Cnt> = LinkName MonitorGrid < 4, Cnt> = LinkType MonitorGrid < 5, Cnt> = RowCount MonitorGrid < 6, Cnt> = StartedDateTime MonitorGrid < 7, Cnt> = ElapsedTime MonitorGrid < 8, Cnt> = RowsPerSec MonitorGrid < 9, Cnt> = PercentCP MonitorGrid <10, Cnt> = CanRunFlag MonitorGrid <11, Cnt> = CanStopFlag MonitorGrid <12, Cnt> = CanResetFlag MonitorGrid <13, Cnt> = FinishedDateTime next j next i end JStat = DSDetachJob(RunHandle) end * ----------------------------------------------------------------------- TheEnd: return
Actually, they did. Never understood why people felt the need to quote entire messages for no good reason and then compound the problem by burying their response inside the quote.
Titto wrote:Hi Kim,
could you please guide me to write the MonitorGrid to a Flat file by Stage name, LinkName and other details
Thanks in advance
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers