DSJ.LINKROWCOUNT not working properly for PX job

snt_ds · Post by **snt_ds** » Sat Aug 23, 2008 2:51 pm

Hi All,

We are using DSGetLinkInfo (with 4th option as DSJ.LINKROWCOUNT) to get row count of the link of PX job once job is finished successfully.

But we encountered a situation in which it is returning wrong value (for ex. it is giving row count = 0 instead of 1).

I did search on the different forums (Ref this ) and found that we can also use DSJ.INSTROWCOUNT as forth option in the DSGetLinkInfo. If we go for this option then we need to add all the row counts of the all the instance (node) of the PX job.

Can you please tell me
1.what can be problem with DSGetLinkInfo to fetch PX job link count?
2.what is correct way to get link count.

Thanks for your time.

ray.wurlod · Post by **ray.wurlod** » Sat Aug 23, 2008 4:10 pm

I have never had a problem with DSJ.LINKROWCOUNT in either server or parallel jobs. How do you know it's the wrong count?

snt_ds · Post by **snt_ds** » Mon Aug 25, 2008 9:31 am

Hi Ray,

I have an example below which explains the issue.

Check_If_Already_Run_Job - Multi instance job which checks if the process is completed for today.
Abc_Seq - Sequence invoking above job.

Based on a requirement we run this sequence for a period ex: 3:00pm - 7:00pm every 15 mins.
Multi instance job returns 1 record if file processed for today. Below example shows that file has been already processed (pls refer line maked with ==>). Sequence uses routine to decide the next step. Routine returned 0 & hence went into wrong path.(pls refer line maked with ==>)

This process inspite of completion continues to run every 15 mins till 7:00pm. It should start & stop. On this day we noticed this issue only at 4:00pm run. In all ther other runs routine returned 1.

Check_If_Already_Run_Job.Abc_Seq
dsjob -logsum -type INFO -max 400 ITA Check_If_Already_Run_Job.Abc_Seq | more
63785 INFO Thu Jul 17 16:01:27 2008
Environment variable settings: (...)
63786 INFO Thu Jul 17 16:01:28 2008
Parallel job initiated
63788 INFO Thu Jul 17 16:01:29 2008
main_program: Ascential DataStage(tm) Enterprise Edition 7.5.1A (...)
63791 INFO Thu Jul 17 16:01:29 2008
main_program: The open files limit is 2048; raising to 10000.
63792 INFO Thu Jul 17 16:01:30 2008
main_program: orchgeneral: loaded (...)
63796 INFO Thu Jul 17 16:01:33 2008
main_program: APT configuration file: /usr/local/etl/default.apt (...)
63813 INFO Thu Jul 17 16:01:38 2008
Src_donefile,0: Import complete; 1 records imported successfully, 0 rejected.
63814 INFO Thu Jul 17 16:01:38 2008
==> Tgt_file,0: Export complete; 1 records exported successfully, 0 rejected.
63815 INFO Thu Jul 17 16:01:38 2008
main_program: Step execution finished with status = OK.
63816 INFO Thu Jul 17 16:01:38 2008
main_program: Startup time, 0:09; production run time, 0:00.
63823 INFO Thu Jul 17 16:01:43 2008
Parallel job reports successful completion
63848 INFO Thu Jul 17 16:15:44 2008

Abc_Seq

Event Id: 15623
Time : Thu Jul 17 16:02:39 2008
Type : INFO
User : dsadm
Message :
Abc_Seq..JobControl (@Coordinator): Summary of sequence run
16:01:12: Sequence started (checkpointing on)
16:01:12: xxxFEE_Check_If_Already_Run_Job (JOB Check_If_Already_Run_Job.Abc_Seq) started
16:01:44: xxxFEE_Check_If_Already_Run_Job (JOB Check_If_Already_Run_Job.Abc_Seq) finished, status=1 [Finished OK]
16:01:44: RA_Check_If_Already_Run_Job (ROUTINE DSU.rGetLinkInfo) started
==> 16:01:44: RA_Check_If_Already_Run_Job finished, reply=0
16:01:44: JA_xxx_Check_FileCreationDate (JOB xxx_Check_FileCreationDate_Job.Abc_Seq) started
16:02:13: JA_xxx_Check_FileCreationDate (JOB xxx_Check_FileCreationDate_Job.Abc_Seq) finished, status=1 [Finished OK]
16:02:14: RA_xxx_Check_FileCreationDate (ROUTINE DSU.rGetLinkInfo) started
16:02:14: RA_xxx_Check_FileCreationDate finished, reply=0
16:02:14: xxx_yyy_BlankFile (JOB xxx_yyy_BlankFile_Job.Abc_Seq) started
16:02:38: xxx_yyy_BlankFile (JOB xxx_yyy_BlankFile_Job.Abc_Seq) finished, status=1 [Finished OK]
16:02:38: RA_Curr_Date (ROUTINE DSU.rGetCurrDate) started
16:02:38: RA_Curr_Date finished, reply=080717
16:02:38: RA_Rename_to_zzz (COMMAND sh) started
16:02:38: RA_Rename_to_zzz finished, reply=0
16:02:38: RA_Archive_BlankFile (ROUTINE DSU.rAdapterFileMove) started
16:02:38: RA_Archive_BlankFile finished, reply=0
16:02:38: Update_xxxFEE_Done (COMMAND sh) started
16:02:38: Update_xxxFEE_Done finished, reply=0
16:02:38: xxx_Blank_Msg (ROUTINE DSSendMail) started
16:02:39: xxx_Blank_Msg finished, reply=0
16:02:39: Sequence finished OK

Not sure if we should run the Transformer (or stage) being used by rGetLinkInfo in Sequential mode. Let me know if you need more details.

ray.wurlod · Post by **ray.wurlod** » Mon Aug 25, 2008 3:46 pm

Are you attaching the correct instance when invoking DSGetLinkInfo() ?

snt_ds · Post by **snt_ds** » Mon Aug 25, 2008 5:46 pm

ray.wurlod wrote:Are you attaching the correct instance when invoking DSGetLinkInfo() ?

We are passing StageName.$JobName (Stage Name of Job Activity which calling Multi Instance PX Job)

ray.wurlod · Post by **ray.wurlod** » Mon Aug 25, 2008 8:11 pm

Stage names do not carry invocation IDs. Only job names do.

Code: Select all

jobhandle = DSAttachJob(jobname.invocationid, DSJ.ERRNONE)
DSGetLinkInfo(jobhandle,stagename,linkname,DSJ.LINKROWCOUNT)

snt_ds · Post by **snt_ds** » Tue Aug 26, 2008 6:37 am

I tried printing the stagename in Email subject & it gives me Jobname.InvocationId.

#JA_Company_Loaded_Staging_Job.$JobName#
returned - Company_Loaded_Staging_Job.TestInvocation_Seq

chulett · Post by **chulett** » Tue Aug 26, 2008 8:02 am

Because you appended it.

As noted, the job name requires the invocation id but the stage and link name do not. Correct your call.

snt_ds · Post by **snt_ds** » Tue Aug 26, 2008 9:37 am

chulett wrote:As noted, the job name requires the invocation id but the stage and link name do not. Correct your call.

That's Correct.We are doing same way.

Job Design :-

Code: Select all

  ...Job_Activity ---> Routine_Activity ....

Routine Code:-

Code: Select all

$INCLUDE DSINCLUDE JOBCONTROL.H 
Ans = 0 
handleJob = DSAttachJob(extractJobName, DSJ.ERRNONE) 
ExtractRowCount = DSGetLinkInfo (handleJob, StageName, LinkName, DSJ.LINKROWCOUNT) 
DetachResult = DSDetachJob(handleJob) 
Ans= ExtractRowCount

Routine Parameter:-

Code: Select all

 
extractJobName = Job_Activity.$JobName(From Activity Variable, ex. JA_Company_Loaded_Staging_Job.$JobName) 
StageName = Stage Name from PX job
LinkName = Link Name from PX job

Note - Job_Activity "stage" calling multi instance job.In Example JA_Company_Loaded_Staging_Job "job activity stage" calling JOB Company_Loaded_Staging_Job

chulett · Post by **chulett** » Tue Aug 26, 2008 10:07 am

And "Stage Name from PX job" means a hard-coded value or something derived? The confusion is the fact that you showed an example of a stage name with a "dot invocation id" tagged on to it and that's not correct.

snt_ds · Post by **snt_ds** » Tue Aug 26, 2008 11:48 am

chulett wrote:And "Stage Name from PX job" means a hard-coded value or something derived?

Yes Craig, The Stage Name and Link Name are hard-coded. But job name is not hard-coded, it just pass through activity variable.

How can I attach snap-shot with this post?

chulett · Post by **chulett** » Tue Aug 26, 2008 1:08 pm

You would need to upload it to any of the many free image servers and then use [img] tags to link to it in a post. Those sites will generate the links for you, typically.

snt_ds · Post by **snt_ds** » Tue Aug 26, 2008 7:39 pm

Following image is for Job Activity:-

Following image is for Rotine Activity:-