Hi,
We have a set of common metadata tables to record statistics of job runs such as start and end times, status, etc...
I know that these information and more can be obtained from the ds api functions such as dsgetjobinfo etc...
I want to know if there is a way to get these info into oracle tables...straight from a job control...
A detailed response would be much appreciated.
If anybody has implemented this sort of a thing which i am sure people would have, it would be great if it can be shared...
ds job stats
Moderators: chulett, rschirm, roy
-
ray.wurlod
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
I should have mentioned that I did look into etlstats...
and I should have mentioned that I did search before asking...
But, I do not need all the html outputs etc associated with it...And etlstats seems to be kind of tightly integrated to the report generation piece...which our application does not warrant...
Moreover, we cannot install software code from outside with proper licencing and stuff like that ...so, unfortunately, installing etlstats as is would not be feasible...
But, that aside,is it a big task to outout ds api results to a persistant storage if we were to develop it in our application...
When responding, kindly understand that this might be a basic question to experts , but, i am on a learning curve here...
So, please bear with me if this question seems priliminary...
and I should have mentioned that I did search before asking...
But, I do not need all the html outputs etc associated with it...And etlstats seems to be kind of tightly integrated to the report generation piece...which our application does not warrant...
Moreover, we cannot install software code from outside with proper licencing and stuff like that ...so, unfortunately, installing etlstats as is would not be feasible...
But, that aside,is it a big task to outout ds api results to a persistant storage if we were to develop it in our application...
When responding, kindly understand that this might be a basic question to experts , but, i am on a learning curve here...
So, please bear with me if this question seems priliminary...
-
ray.wurlod
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Yes it's a big task. Big, not difficult, just tedious.
You can get MetaStage to do it automatically, but at a (monetary) price. At least it would be properly licensed.
Apart from that, the API is there and documented, and you can examine the source code of ETLStats to see how one person has done it.
You can get MetaStage to do it automatically, but at a (monetary) price. At least it would be properly licensed.
Apart from that, the API is there and documented, and you can examine the source code of ETLStats to see how one person has done it.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Re: ds job stats
Why can't you lookinto macho works driven by Chuck, Duke, Kenneth geants and get your work simplify. Otherwise it will take more time to implement the same in your way. All the best.
In otherwords, we would like to know your requirement in detail as you expected the resonse.
In otherwords, we would like to know your requirement in detail as you expected the resonse.
2 B 1 4 ALL
you can easily modify DSJobReportDb to update any table you like. It is just an example of how to use dsjob -report XML. This job runs this command in the before job routine. Then processes the results to load 3 tables. You can load your table if you want. Do a search there are several posts where people modified it to do something similar. This is a server job but it can get row counts on PX jobs. You could easily convert it to a PX job if you do not like server jobs. It is just an example of how this can be done. It calls a few server routines which can be eliminated as well.
dsjob -report on a PX job reports at the partition level. So you may need to aggregate the row counts to the link level. Links are in there twice. One for each stage it is connected to. So aggregate on stage name as well. You need to add partition to your ETL_ROW_HIST key if you want this level of detail. We have 200 jobs in one project and it writes 40,000 rows every run when we have it at the partition level. This is daily so it really adds up fast. Be careful.
dsjob -report on a PX job reports at the partition level. So you may need to aggregate the row counts to the link level. Links are in there twice. One for each stage it is connected to. So aggregate on stage name as well. You need to add partition to your ETL_ROW_HIST key if you want this level of detail. We have 200 jobs in one project and it writes 40,000 rows every run when we have it at the partition level. This is daily so it really adds up fast. Be careful.
Mamu Kim