How to Write the Statistics into a table?
Moderators: chulett, rschirm, roy
How to Write the Statistics into a table?
Hi all,
I want to write the statistics namely when the job started and the End time and the records that has processed at each link and as well as i have to capture the count at each link.I have tried with the Built-in Routine/Before/after/DsJobReport but i am able to write them only into a file and i am not able to wrote them into a table.I heard that METASTAGE will handle these things .Is that True or the normal PX will also handle these capturing of the reocrds?
please let me know how to handle the situation........
Thanks in Advance.......................
Regards,
skumar.
I want to write the statistics namely when the job started and the End time and the records that has processed at each link and as well as i have to capture the count at each link.I have tried with the Built-in Routine/Before/after/DsJobReport but i am able to write them only into a file and i am not able to wrote them into a table.I heard that METASTAGE will handle these things .Is that True or the normal PX will also handle these capturing of the reocrds?
please let me know how to handle the situation........
Thanks in Advance.......................
Regards,
skumar.
Do a search for EtlStats. They give you routines and dsjob to get the stats but you need to put these together to get the results in a table. You can download EtlStats and it is a serious of jobs that will get row counts and put them in tables. It has examples of how to get log records and email them as attachments.
In the words of Mr. Craig, "Take, Adapt, Shine".
In the words of Mr. Craig, "Take, Adapt, Shine".
Mamu Kim
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Re: How to Write the Statistics into a table?
Hi,skumar wrote:Hi all,
I want to write the statistics namely when the job started and the End time and the records that has processed at each link and as well as i have to capture the count at each link.I have tried with the Built-in Routine/Before/after/DsJobReport but i am able to write them only into a file and i am not able to wrote them into a table.I heard that METASTAGE will handle these things .Is that True or the normal PX will also handle these capturing of the reocrds?
please let me know how to handle the situation........
Thanks in Advance.......................
Regards,
skumar.
IF you can get them to a file, I don't see why you can't write them to a table, what you need is to write a simple Unix script to parse the file and format the output into a delimeted file. Any rdbms out there should have loader programs to load a delimeted file.
Chin
Re: How to Write the Statistics into a table?
Or... yet another DataStage job!Chuah wrote:IF you can get them to a file, I don't see why you can't write them to a table, what you need is to write a simple Unix script...
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
Re: How to Write the Statistics into a table?
of course but a lot depends on the format of the output file; point is the file needs to be formatted into tabular form and DS is not always the best tool to reformat text files.chulett wrote:Or... yet another DataStage job!Chuah wrote:IF you can get them to a file, I don't see why you can't write them to a table, what you need is to write a simple Unix script...
So if you are going to trap statistics like that, do you need to have the job monitor running?
We have not been running it lately, because we don't use the stats except for tracing and debugging, so why add the overhead. However, the numbers would be very valuable for data integrity and count verification.
I am curious for more info.
Brad.
We have not been running it lately, because we don't use the stats except for tracing and debugging, so why add the overhead. However, the numbers would be very valuable for data integrity and count verification.
I am curious for more info.
Brad.
Re: How to Write the Statistics into a table?
The output of DSJobReport isn't all that hard to parse. Read each record as one long string field and then look for the information you need from each. Timings have both 'start time' and 'end time' literals in them, with what you need just after the = that comes after each literal.Chuah wrote:of course but a lot depends on the format of the output file; point is the file needs to be formatted into tabular form and DS is not always the best tool to reformat text files.chulett wrote:Or... yet another DataStage job!Chuah wrote:IF you can get them to a file, I don't see why you can't write them to a table, what you need is to write a simple Unix script...
Granted, there are better tools out there, ones that are built to parse 'report' files like Monarch, but it can certainly be done with DataStage. I only commented because that would be my preferred method rather than trying to rip it apart with a shell script. But that's just me.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers