Notification Job status show 0 records for underlying jobs

VCInDSX · Post by **VCInDSX** » Mon Jan 28, 2008 5:40 pm

Hi Group,
My job design is as follows

JobActivity1 ----|
JobActivity2-----|--->Sequencer--->Notification
JobActivity3-----|

Each of the Job Activities launch a job that read an input text file and loads into a target SQL Server Table. The underlying job is run as Multi-instance.

The sequence is set to "ALL" mode to pass control to the notification only when all the activities are successful.

When the jobs complete, the notification stage is sending the mail without any issues. However, the status information has all counts reporting as Zero. It creates a section for each of the job activities and the stats for each load job has zero counts in the mail.

When I look at the director log for the individual jobs (Multi-instance id) themselves they report the actual counts of the records that were processed from the file to the database table at each stage level.

Is there any setting that i might be missing to propagate the stats from the individual jobs to the notification stage?

If there is any additional piece of information that would help you help me in troubleshooting this, please let me know.

Thanks in advance for your invaluable time and help,

ray.wurlod · Post by **ray.wurlod** » Mon Jan 28, 2008 9:44 pm

What have you put in the Notification activity? Just checking "include job report" will include the report for the job sequence itself, not for jobs run under control.

VCInDSX · Post by **VCInDSX** » Mon Jan 28, 2008 10:31 pm

Ray,
Thanks for the followup. Appreciate your time.

I reviewed the job once again and can confirm that I have checked "Include job status in email" checkbox in the Notification tab of the Notification stage.

The rest of the SMTP parameters are all setup correctly.

Here is what i got in the notification mail delivered to my inbox.

Code: Select all


**************************************************
STATUS REPORT FOR JOB: DBLoader.TestTable1
Generated: 2008-01-28 20:02:20
   Job start time=2008-01-28 19:56:37
   Job end time=2008-01-28 20:00:07
   Job elapsed time=00:03:30
   Job status=2 (Finished with warnings)
      Stage: SF_RCPInputFile, 0 rows input
      Stage start time=, end time=, elapsed=00:00:00
         Link: SF_To_XFM_Lnk, 0 rows
         Link: SourceRejects_Lnk, 0 rows
      Stage: XFM_RCP_2_Database, 0 rows input
      Stage start time=, end time=, elapsed=00:00:00
         Link: SF_To_XFM_Lnk, 0 rows
         Link: XFM_To_DB_Lnk, 0 rows
      Stage: SF_Source_Rejects, 0 rows input
      Stage start time=, end time=, elapsed=00:00:00
         Link: SourceRejects_Lnk, 0 rows
      Stage: ODB_AsCollected, 0 rows input
      Stage start time=, end time=, elapsed=00:00:00
         Link: XFM_To_DB_Lnk, 0 rows

**************************************************
STATUS REPORT FOR JOB: DBLoader.TestTable2
Generated: 2008-01-28 20:04:51
   Job start time=2008-01-28 19:57:02
   Job end time=2008-01-28 20:00:53
   Job elapsed time=00:03:51
   Job status=2 (Finished with warnings)
      Stage: SF_RCPInputFile, 0 rows input
      Stage start time=, end time=, elapsed=00:00:00
         Link: SF_To_XFM_Lnk, 0 rows
         Link: SourceRejects_Lnk, 0 rows
      Stage: XFM_RCP_2_Database, 0 rows input
      Stage start time=, end time=, elapsed=00:00:00
         Link: SF_To_XFM_Lnk, 0 rows
         Link: XFM_To_DB_Lnk, 0 rows
      Stage: SF_Source_Rejects, 0 rows input
      Stage start time=, end time=, elapsed=00:00:00
         Link: SourceRejects_Lnk, 0 rows
      Stage: ODB_AsCollected, 0 rows input
      Stage start time=, end time=, elapsed=00:00:00
         Link: XFM_To_DB_Lnk, 0 rows

**************************************************
STATUS REPORT FOR JOB: DBLoader.TestTable3
Generated: 2008-01-28 20:05:42
   Job start time=2008-01-28 19:57:38
   Job end time=2008-01-28 20:02:11
   Job elapsed time=00:04:33
   Job status=2 (Finished with warnings)
      Stage: SF_RCPInputFile, 0 rows input
      Stage start time=, end time=, elapsed=00:00:00
         Link: SF_To_XFM_Lnk, 0 rows
         Link: SourceRejects_Lnk, 0 rows
      Stage: XFM_RCP_2_Database, 0 rows input
      Stage start time=, end time=, elapsed=00:00:00
         Link: SF_To_XFM_Lnk, 0 rows
         Link: XFM_To_DB_Lnk, 0 rows
      Stage: SF_Source_Rejects, 0 rows input
      Stage start time=, end time=, elapsed=00:00:00
         Link: SourceRejects_Lnk, 0 rows
      Stage: ODB_AsCollected, 0 rows input
      Stage start time=, end time=, elapsed=00:00:00
         Link: XFM_To_DB_Lnk, 0 rows

Maybe I misundertood the purpose of this option in the Notification stage.

As the underlying multi-instance job is an RCP implementation, i was wondering if RCP is not able to provide the necessary information to the Sequence job. But the director logs for each of the Multi-Instance Id shows the counts correctly.

Please let me know if any additional information will help you help me.

ray.wurlod · Post by **ray.wurlod** » Tue Jan 29, 2008 12:52 am

This is version 8, and there is more in the job report than I expected. TO test whether it's RCP or not should be simple, just disable it and see what happens. Other than that I have no further theories to offer, and am not in a position to test any of them this week even if I did.

But, wait! What do you mean by "is an RCP implementation"? Runtime column propagation is only available in parallel jobs, as far as I am aware. Your job type is marked as server.

VCInDSX · Post by **VCInDSX** » Tue Jan 29, 2008 4:06 pm

Hi Ray,
First the Server Vs Parallel category posting - my apologies for the confusion. Before posting the query, i looked at the description of the forums and saw "Job Sequences" belonged to "IBM(R) DataStage Server Edition" category and posted it there.

I mentioned RCP because I was not sure if that might be playing a role in the status mail counts. The RCP job is what is invoked from the "Job Activities" in the Sequence.

If this topic does not belong here, I will request Admin to move it to the Parallel thread.

As for a Non-RCP job test, I created 2 jobs that have the following design
RowGenerator ==> Transformer ==> Peek
I tried a simple Job sequence that calls these 2 jobs and finally sends a notification. This does not have RCP enabled. It exhibits the same behavior.
The contents of the notification mail from this Non-RCP job are given below.

Code: Select all

**************************************************
STATUS REPORT FOR JOB: SeqJobT1
Generated: 2008-01-29 16:23:07
   Job start time=2008-01-29 16:22:09
   Job end time=2008-01-29 16:23:03
   Job elapsed time=00:00:54
   Job status=2 (Finished with warnings)
      Stage: Row_Generator_0, 0 rows input
      Stage start time=, end time=, elapsed=00:00:00
         Link: DSLink3, 0 rows
      Stage: Transformer_1, 0 rows input
      Stage start time=, end time=, elapsed=00:00:00
         Link: DSLink3, 0 rows
         Link: DSLink4, 0 rows
      Stage: Peek_2, 0 rows input
      Stage start time=, end time=, elapsed=00:00:00
         Link: DSLink4, 0 rows

**************************************************
STATUS REPORT FOR JOB: SeqJobT2
Generated: 2008-01-29 16:24:44
   Job start time=2008-01-29 16:23:41
   Job end time=2008-01-29 16:24:40
   Job elapsed time=00:00:59
   Job status=2 (Finished with warnings)
      Stage: Row_Generator_0, 0 rows input
      Stage start time=, end time=, elapsed=00:00:00
         Link: DSLink3, 0 rows
      Stage: Transformer_1, 0 rows input
      Stage start time=, end time=, elapsed=00:00:00
         Link: DSLink3, 0 rows
         Link: DSLink4, 0 rows
      Stage: Peek_2, 0 rows input
      Stage start time=, end time=, elapsed=00:00:00
         Link: DSLink4, 0 rows

When i reviewed the the log messages of these 2 underlying jobs in the director i can see the actual counts and stats summary.

Thanks in advance for your time Ray. Appreciate you help. I will post any updates if I find anything new in the next few days as well.

VCInDSX · Post by **VCInDSX** » Wed Jan 30, 2008 10:20 am

Hi Ray, Team,
Wanted to give an update on this item. I submitted this to IBM Support late yesterday, just in case, we missed something on the configuration.

Here is what was suggested.
From DOS, go into the "PXEngine\java" directory of the information server datastage server.
Then perform the following (assuming default installation directory) :
C:\IBM\InformationServer\Server\PXEngine\java>sh
$ APT_ORCHHOME="C:/IBM/InformationServer/Server/PXEngine"
$ export APT_ORCHHOME
$ ./jobmoninit start

After starting the job monitor, the same job started populating the stats in the mail.

Here is output

Code: Select all

**************************************************
STATUS REPORT FOR JOB: SeqJobT1
Generated: 2008-01-30 10:40:21
   Job start time=2008-01-30 10:39:32
   Job end time=2008-01-30 10:40:18
   Job elapsed time=00:00:46
   Job status=1 (Finished OK)
      Stage: Row_Generator_0, 10 rows input
      Stage start time=2008-01-30 10:39:54, end time=2008-01-30 10:40:15, elapsed=00:00:21
         Link: DSLink3, 10 rows
      Stage: Transformer_1, 10 rows input
      Stage start time=2008-01-30 10:39:54, end time=2008-01-30 10:40:15, elapsed=00:00:21
         Link: DSLink3, 10 rows
         Link: DSLink4, 10 rows
      Stage: Peek_2, 10 rows input
      Stage start time=2008-01-30 10:39:54, end time=2008-01-30 10:40:15, elapsed=00:00:21
         Link: DSLink4, 10 rows

Thanks a lot Ray for your time to help me on this one amidst your schedules.

I wanted to post this update so that you may focus on other queries that require your invaluable time and also for anyone who might run into this issue in the future.