Problem in Job Report XML File Creation

[Aparna_A]

Dear all,

I am facing an issue while collecting the job Statistics for a set of jobs.

The scenario is like this.
When I run a single job, and later generate a XML file for the same job, all the link counts are correct.

We are using Unix Shell Scripts as the Wrapper for all our parallel jobs (around 20-25 jobs). These scripts have been designed to execute the dsjob -report command after the execution of each job. So, when these 20 odd jobs are being run one after the other using scripts, the link counts for some of the jobs in the XML files are exactly half of the original count or are not equal to the correct count. The jobs are running on 2 nodes. Can anyone pls. help me out on this?.

Thanks Ahead,

[roy]

Hi and Welcome aboard ,
What commands are you using exactly?
Do you wait for the jobs to finish?
Why can't you use Sequence jobs to run the 20 or so jobs and only invoke dsjob once?
What exact OS and DS version do you use?
What is the source of your data?
What do you need to do with the statistics? (perhaps it can be done in DS instead)

[ray.wurlod]

Is the second node being reported separately somewhere else in the XML document? If not, I suspect you've uncovered a bug. What does your support provider suggest?

[kduke]

This maybe be true. Vincent made a change to EtlStats to aggregate the nodes into one number to get the true throughput.

[Aparna_A]

Dear all,

Thanks for all the valuable suggestions, result of which I have found out the solution for the problem. :D

In our process, once the XML is generated, we are using a Server Job to load the counts into the Database Table used to store these statistics. The link count was sometimes being captured in 2 Instance Sets of the XML, whereas in our server job, a filter on Instance ID=0 was being done.

So, we got to modify our job, and thats the solution.

Thanks Again,