How to Write the Statistics into a table?

[skumar]

Hi all,

I want to write the statistics namely when the job started and the End time and the records that has processed at each link and as well as i have to capture the count at each link.I have tried with the Built-in Routine/Before/after/DsJobReport but i am able to write them only into a file and i am not able to wrote them into a table.I heard that METASTAGE will handle these things .Is that True or the normal PX will also handle these capturing of the reocrds?

please let me know how to handle the situation........

Thanks in Advance.......................

Regards,
skumar.

[kduke]

Do a search for EtlStats. They give you routines and dsjob to get the stats but you need to put these together to get the results in a table. You can download EtlStats and it is a serious of jobs that will get row counts and put them in tables. It has examples of how to get log records and email them as attachments.

In the words of Mr. Craig, "Take, Adapt, Shine".

[ray.wurlod]

A "serious of jobs"??? I guess it is. :D

[kduke]

Sorry, series of jobs. I be tired.

[Chuah]

Hi all,

I want to write the statistics namely when the job started and the End time and the records that has processed at each link and as well as i have to capture the count at each link.I have tried with the Built-in Routine/Before/after/DsJobReport but i am able to write them only into a file and i am not able to wrote them into a table.I heard that METASTAGE will handle these things .Is that True or the normal PX will also handle these capturing of the reocrds?

please let me know how to handle the situation........

Thanks in Advance.......................

Regards,
skumar.

Hi,
IF you can get them to a file, I don't see why you can't write them to a table, what you need is to write a simple Unix script to parse the file and format the output into a delimeted file. Any rdbms out there should have loader programs to load a delimeted file.

Chin

[chulett]

IF you can get them to a file, I don't see why you can't write them to a table, what you need is to write a simple Unix script...

Or... yet another DataStage job!

[Chuah]

IF you can get them to a file, I don't see why you can't write them to a table, what you need is to write a simple Unix script...

Or... yet another DataStage job!

of course but a lot depends on the format of the output file; point is the file needs to be formatted into tabular form and DS is not always the best tool to reformat text files.

[bcarlson]

So if you are going to trap statistics like that, do you need to have the job monitor running?

We have not been running it lately, because we don't use the stats except for tracing and debugging, so why add the overhead. However, the numbers would be very valuable for data integrity and count verification.

I am curious for more info.

Brad.

[chulett]

IF you can get them to a file, I don't see why you can't write them to a table, what you need is to write a simple Unix script...

Or... yet another DataStage job!

of course but a lot depends on the format of the output file; point is the file needs to be formatted into tabular form and DS is not always the best tool to reformat text files.

The output of DSJobReport isn't all that hard to parse. Read each record as one long string field and then look for the information you need from each. Timings have both 'start time' and 'end time' literals in them, with what you need just after the = that comes after each literal.

Granted, there are better tools out there, ones that are built to parse 'report' files like Monarch, but it can certainly be done with DataStage. I only commented because that would be my preferred method rather than trying to rip it apart with a shell script. But that's just me.