DSJ.LINKROWCOUNT not working properly for PX job
Moderators: chulett, rschirm, roy
DSJ.LINKROWCOUNT not working properly for PX job
Hi All,
We are using DSGetLinkInfo (with 4th option as DSJ.LINKROWCOUNT) to get row count of the link of PX job once job is finished successfully.
But we encountered a situation in which it is returning wrong value (for ex. it is giving row count = 0 instead of 1).
I did search on the different forums (Ref this ) and found that we can also use DSJ.INSTROWCOUNT as forth option in the DSGetLinkInfo. If we go for this option then we need to add all the row counts of the all the instance (node) of the PX job.
Can you please tell me
1.what can be problem with DSGetLinkInfo to fetch PX job link count?
2.what is correct way to get link count.
Thanks for your time.
We are using DSGetLinkInfo (with 4th option as DSJ.LINKROWCOUNT) to get row count of the link of PX job once job is finished successfully.
But we encountered a situation in which it is returning wrong value (for ex. it is giving row count = 0 instead of 1).
I did search on the different forums (Ref this ) and found that we can also use DSJ.INSTROWCOUNT as forth option in the DSGetLinkInfo. If we go for this option then we need to add all the row counts of the all the instance (node) of the PX job.
Can you please tell me
1.what can be problem with DSGetLinkInfo to fetch PX job link count?
2.what is correct way to get link count.
Thanks for your time.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Hi Ray,
I have an example below which explains the issue.
Check_If_Already_Run_Job - Multi instance job which checks if the process is completed for today.
Abc_Seq - Sequence invoking above job.
Based on a requirement we run this sequence for a period ex: 3:00pm - 7:00pm every 15 mins.
Multi instance job returns 1 record if file processed for today. Below example shows that file has been already processed (pls refer line maked with ==>). Sequence uses routine to decide the next step. Routine returned 0 & hence went into wrong path.(pls refer line maked with ==>)
This process inspite of completion continues to run every 15 mins till 7:00pm. It should start & stop. On this day we noticed this issue only at 4:00pm run. In all ther other runs routine returned 1.
Check_If_Already_Run_Job.Abc_Seq
dsjob -logsum -type INFO -max 400 ITA Check_If_Already_Run_Job.Abc_Seq | more
63785 INFO Thu Jul 17 16:01:27 2008
Environment variable settings: (...)
63786 INFO Thu Jul 17 16:01:28 2008
Parallel job initiated
63788 INFO Thu Jul 17 16:01:29 2008
main_program: Ascential DataStage(tm) Enterprise Edition 7.5.1A (...)
63791 INFO Thu Jul 17 16:01:29 2008
main_program: The open files limit is 2048; raising to 10000.
63792 INFO Thu Jul 17 16:01:30 2008
main_program: orchgeneral: loaded (...)
63796 INFO Thu Jul 17 16:01:33 2008
main_program: APT configuration file: /usr/local/etl/default.apt (...)
63813 INFO Thu Jul 17 16:01:38 2008
Src_donefile,0: Import complete; 1 records imported successfully, 0 rejected.
63814 INFO Thu Jul 17 16:01:38 2008
==> Tgt_file,0: Export complete; 1 records exported successfully, 0 rejected.
63815 INFO Thu Jul 17 16:01:38 2008
main_program: Step execution finished with status = OK.
63816 INFO Thu Jul 17 16:01:38 2008
main_program: Startup time, 0:09; production run time, 0:00.
63823 INFO Thu Jul 17 16:01:43 2008
Parallel job reports successful completion
63848 INFO Thu Jul 17 16:15:44 2008
Abc_Seq
Event Id: 15623
Time : Thu Jul 17 16:02:39 2008
Type : INFO
User : dsadm
Message :
Abc_Seq..JobControl (@Coordinator): Summary of sequence run
16:01:12: Sequence started (checkpointing on)
16:01:12: xxxFEE_Check_If_Already_Run_Job (JOB Check_If_Already_Run_Job.Abc_Seq) started
16:01:44: xxxFEE_Check_If_Already_Run_Job (JOB Check_If_Already_Run_Job.Abc_Seq) finished, status=1 [Finished OK]
16:01:44: RA_Check_If_Already_Run_Job (ROUTINE DSU.rGetLinkInfo) started
==> 16:01:44: RA_Check_If_Already_Run_Job finished, reply=0
16:01:44: JA_xxx_Check_FileCreationDate (JOB xxx_Check_FileCreationDate_Job.Abc_Seq) started
16:02:13: JA_xxx_Check_FileCreationDate (JOB xxx_Check_FileCreationDate_Job.Abc_Seq) finished, status=1 [Finished OK]
16:02:14: RA_xxx_Check_FileCreationDate (ROUTINE DSU.rGetLinkInfo) started
16:02:14: RA_xxx_Check_FileCreationDate finished, reply=0
16:02:14: xxx_yyy_BlankFile (JOB xxx_yyy_BlankFile_Job.Abc_Seq) started
16:02:38: xxx_yyy_BlankFile (JOB xxx_yyy_BlankFile_Job.Abc_Seq) finished, status=1 [Finished OK]
16:02:38: RA_Curr_Date (ROUTINE DSU.rGetCurrDate) started
16:02:38: RA_Curr_Date finished, reply=080717
16:02:38: RA_Rename_to_zzz (COMMAND sh) started
16:02:38: RA_Rename_to_zzz finished, reply=0
16:02:38: RA_Archive_BlankFile (ROUTINE DSU.rAdapterFileMove) started
16:02:38: RA_Archive_BlankFile finished, reply=0
16:02:38: Update_xxxFEE_Done (COMMAND sh) started
16:02:38: Update_xxxFEE_Done finished, reply=0
16:02:38: xxx_Blank_Msg (ROUTINE DSSendMail) started
16:02:39: xxx_Blank_Msg finished, reply=0
16:02:39: Sequence finished OK
Not sure if we should run the Transformer (or stage) being used by rGetLinkInfo in Sequential mode. Let me know if you need more details.
I have an example below which explains the issue.
Check_If_Already_Run_Job - Multi instance job which checks if the process is completed for today.
Abc_Seq - Sequence invoking above job.
Based on a requirement we run this sequence for a period ex: 3:00pm - 7:00pm every 15 mins.
Multi instance job returns 1 record if file processed for today. Below example shows that file has been already processed (pls refer line maked with ==>). Sequence uses routine to decide the next step. Routine returned 0 & hence went into wrong path.(pls refer line maked with ==>)
This process inspite of completion continues to run every 15 mins till 7:00pm. It should start & stop. On this day we noticed this issue only at 4:00pm run. In all ther other runs routine returned 1.
Check_If_Already_Run_Job.Abc_Seq
dsjob -logsum -type INFO -max 400 ITA Check_If_Already_Run_Job.Abc_Seq | more
63785 INFO Thu Jul 17 16:01:27 2008
Environment variable settings: (...)
63786 INFO Thu Jul 17 16:01:28 2008
Parallel job initiated
63788 INFO Thu Jul 17 16:01:29 2008
main_program: Ascential DataStage(tm) Enterprise Edition 7.5.1A (...)
63791 INFO Thu Jul 17 16:01:29 2008
main_program: The open files limit is 2048; raising to 10000.
63792 INFO Thu Jul 17 16:01:30 2008
main_program: orchgeneral: loaded (...)
63796 INFO Thu Jul 17 16:01:33 2008
main_program: APT configuration file: /usr/local/etl/default.apt (...)
63813 INFO Thu Jul 17 16:01:38 2008
Src_donefile,0: Import complete; 1 records imported successfully, 0 rejected.
63814 INFO Thu Jul 17 16:01:38 2008
==> Tgt_file,0: Export complete; 1 records exported successfully, 0 rejected.
63815 INFO Thu Jul 17 16:01:38 2008
main_program: Step execution finished with status = OK.
63816 INFO Thu Jul 17 16:01:38 2008
main_program: Startup time, 0:09; production run time, 0:00.
63823 INFO Thu Jul 17 16:01:43 2008
Parallel job reports successful completion
63848 INFO Thu Jul 17 16:15:44 2008
Abc_Seq
Event Id: 15623
Time : Thu Jul 17 16:02:39 2008
Type : INFO
User : dsadm
Message :
Abc_Seq..JobControl (@Coordinator): Summary of sequence run
16:01:12: Sequence started (checkpointing on)
16:01:12: xxxFEE_Check_If_Already_Run_Job (JOB Check_If_Already_Run_Job.Abc_Seq) started
16:01:44: xxxFEE_Check_If_Already_Run_Job (JOB Check_If_Already_Run_Job.Abc_Seq) finished, status=1 [Finished OK]
16:01:44: RA_Check_If_Already_Run_Job (ROUTINE DSU.rGetLinkInfo) started
==> 16:01:44: RA_Check_If_Already_Run_Job finished, reply=0
16:01:44: JA_xxx_Check_FileCreationDate (JOB xxx_Check_FileCreationDate_Job.Abc_Seq) started
16:02:13: JA_xxx_Check_FileCreationDate (JOB xxx_Check_FileCreationDate_Job.Abc_Seq) finished, status=1 [Finished OK]
16:02:14: RA_xxx_Check_FileCreationDate (ROUTINE DSU.rGetLinkInfo) started
16:02:14: RA_xxx_Check_FileCreationDate finished, reply=0
16:02:14: xxx_yyy_BlankFile (JOB xxx_yyy_BlankFile_Job.Abc_Seq) started
16:02:38: xxx_yyy_BlankFile (JOB xxx_yyy_BlankFile_Job.Abc_Seq) finished, status=1 [Finished OK]
16:02:38: RA_Curr_Date (ROUTINE DSU.rGetCurrDate) started
16:02:38: RA_Curr_Date finished, reply=080717
16:02:38: RA_Rename_to_zzz (COMMAND sh) started
16:02:38: RA_Rename_to_zzz finished, reply=0
16:02:38: RA_Archive_BlankFile (ROUTINE DSU.rAdapterFileMove) started
16:02:38: RA_Archive_BlankFile finished, reply=0
16:02:38: Update_xxxFEE_Done (COMMAND sh) started
16:02:38: Update_xxxFEE_Done finished, reply=0
16:02:38: xxx_Blank_Msg (ROUTINE DSSendMail) started
16:02:39: xxx_Blank_Msg finished, reply=0
16:02:39: Sequence finished OK
Not sure if we should run the Transformer (or stage) being used by rGetLinkInfo in Sequential mode. Let me know if you need more details.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Stage names do not carry invocation IDs. Only job names do.
Code: Select all
jobhandle = DSAttachJob(jobname.invocationid, DSJ.ERRNONE)
DSGetLinkInfo(jobhandle,stagename,linkname,DSJ.LINKROWCOUNT)
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
That's Correct.We are doing same way.chulett wrote:As noted, the job name requires the invocation id but the stage and link name do not. Correct your call.
Job Design :-
Code: Select all
...Job_Activity ---> Routine_Activity ....
Code: Select all
$INCLUDE DSINCLUDE JOBCONTROL.H
Ans = 0
handleJob = DSAttachJob(extractJobName, DSJ.ERRNONE)
ExtractRowCount = DSGetLinkInfo (handleJob, StageName, LinkName, DSJ.LINKROWCOUNT)
DetachResult = DSDetachJob(handleJob)
Ans= ExtractRowCount
Code: Select all
extractJobName = Job_Activity.$JobName(From Activity Variable, ex. JA_Company_Loaded_Staging_Job.$JobName)
StageName = Stage Name from PX job
LinkName = Link Name from PX job