Issue retrieving number of rows processed
Moderators: chulett, rschirm, roy
Issue retrieving number of rows processed
I have two stages in my job, one ODBC stage named say "ODBC1" and another sequential file stage named say "FILE1".
ODBC1---------------->FILE1
The ODBC stage executes a stored procedure and the output rows are stored in the sequential file.
In the after job subroutine of this job i call a routine with some input parameters. This routines writes to a log file number of rows retrieved by ODBC stage to sequential file stage. For getting number of rows processed i was passing the sequential file stage name ie; "FILE1" as my first input parameter to my subroutine. In the routine i have code as below for getting total number of rows processed.
IStage = Field(InputArg,';',1)
JobHandle = DSJ.ME
ReportText ="***** Total rows processed: " : DSGetStageInfo(JobHandle, IStage , DSJ.STAGEINROWNUM) : Char(13) : Char(10)
But this was giving total number of rows processed as -7 always. So i passed the stage name as "FILE1.IDENT1" as input parameter and it did give proper output.
1) MY FIRST QUESTION IS WHY DO I HAVE TO PASS THE stage name as FILE1.IDENT1 instead of FILE1
2) Second issue i have is my same job with out any after job routine is getting aborted by throwing the following warning.
"Abnormal termination of stage FILE1.IDENT1 detected."
when i run this job individually there are rare chances of errors. but when i call this job using a job activity in a sequence it is randomly giving me erros. could you please suggest me a reason to avoid this ramdom error.
ODBC1---------------->FILE1
The ODBC stage executes a stored procedure and the output rows are stored in the sequential file.
In the after job subroutine of this job i call a routine with some input parameters. This routines writes to a log file number of rows retrieved by ODBC stage to sequential file stage. For getting number of rows processed i was passing the sequential file stage name ie; "FILE1" as my first input parameter to my subroutine. In the routine i have code as below for getting total number of rows processed.
IStage = Field(InputArg,';',1)
JobHandle = DSJ.ME
ReportText ="***** Total rows processed: " : DSGetStageInfo(JobHandle, IStage , DSJ.STAGEINROWNUM) : Char(13) : Char(10)
But this was giving total number of rows processed as -7 always. So i passed the stage name as "FILE1.IDENT1" as input parameter and it did give proper output.
1) MY FIRST QUESTION IS WHY DO I HAVE TO PASS THE stage name as FILE1.IDENT1 instead of FILE1
2) Second issue i have is my same job with out any after job routine is getting aborted by throwing the following warning.
"Abnormal termination of stage FILE1.IDENT1 detected."
when i run this job individually there are rare chances of errors. but when i call this job using a job activity in a sequence it is randomly giving me erros. could you please suggest me a reason to avoid this ramdom error.
The link name you mentioned is it the name of the link between transformer and sequential file stage?
What should i give for Infotype?
please answer my second question too. My same job is getting aborted randomly in the sequence eventhough i dont have a after job subroutine. Is it some other problem, may be network connection.
What should i give for Infotype?
please answer my second question too. My same job is getting aborted randomly in the sequence eventhough i dont have a after job subroutine. Is it some other problem, may be network connection.
kumar_s wrote:Hi,
Try to include a Transformer in between the two passive stages and find the stage info for that. I rather use DSGetLinkInfo(JobHandle, StageName, LinkName, InfoType). ...
DSJ.LINKROWCOUNT is the info type.
The IDENT1 is an invisible Transformer stage that the job compiler has inserted because your design includes a link between two passive stages. Hope both the issue should get solved by including a transformer between the stages.
The IDENT1 is an invisible Transformer stage that the job compiler has inserted because your design includes a link between two passive stages. Hope both the issue should get solved by including a transformer between the stages.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
I inserted a transformer stage and still it is giving that random error. My second scenario is still occuring. its getting aborted randomly saying
"Abnormal termination of stage sp_crepp_mi_ohq_extr_jan15..Transformer_8"
sp_crepp_mi_ohq_extr_jan15=jobname
transformer name=Transformer_8
Is there any possibility that this is hapenning due to network issue?
"Abnormal termination of stage sp_crepp_mi_ohq_extr_jan15..Transformer_8"
sp_crepp_mi_ohq_extr_jan15=jobname
transformer name=Transformer_8
Is there any possibility that this is hapenning due to network issue?
kumar_s wrote:DSJ.LINKROWCOUNT is the info type.
The IDENT1 is an invisible Transformer stage that the job compiler has inserted because your design includes a link between two passive stages. Hope both the issue should get solved by including a transformer between the stages.
Abnormal terminations are usually data related, such as using a NULL in a math calculation or using an uncompiled Function in a derivation/constraint.
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Its a good point to consider. For testing purpose, change the routing to ExecSH and use dsjob --linkinfo with the respective parameters.chulett wrote: I don't see how that would help. The OP stated that it failed occassionally, not constantly. Lack of that JOBCONTROL header file would cause something that needed it to not even run.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
I am still getting abnormal termination while running the job and that too randomly. I am using the same scenario and data is also same in all the cases and it is aborting randomly giving a warning abnormal termination. I have only two stages as below
ODBC ----------> Sequential file stage
And in all test cases I have same scenario. But it gets aborted randomly.
What can be the posible reason for this. Previous we had a patch update on our database. But since we were getting these issues they reverted back the patch update. Still we are having the abnormal termination issue.
ODBC ----------> Sequential file stage
And in all test cases I have same scenario. But it gets aborted randomly.
What can be the posible reason for this. Previous we had a patch update on our database. But since we were getting these issues they reverted back the patch update. Still we are having the abnormal termination issue.
kumar_s wrote:Its a good point to consider. For testing purpose, change the routing to ExecSH and use dsjob --linkinfo with the respective parameters.chulett wrote: I don't see how that would help. The OP stated that it failed occassionally, not constantly. Lack of that JOBCONTROL header file would cause something that needed it to not even run.
Add a transformer between the two stages, then use that name when you go to retrieve the rows processed. As a general rule, I never link two passive stages directly together. DataStage will add an 'invisible' one for you, why not make it visible and have full control? Besides, I've seen way too many flakey, 'it only happens occassionally' problems from jobs designed like that that went away when a 'real' transformer was introduced.
Give that a shot.
Give that a shot.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers