Generate Report?
Moderators: chulett, rschirm, roy
Generate Report?
Hey, I am kinda new to Data Stage and there isn't anyone with real knowlage in the company anymore, I took over monitoring the job and need to find a better way to do it. Currently we just go into Director and check each schedualled job. Anyone know a way to generate a daily report stating if the job ran with no errors, or if there were errors? We are currently running DataStage 7.0 on windows platform.
mkeevil,
there are many ways of getting the report information and the best for your site really does depend on what you prefer.
If you want to process the information from outside of DataStage and like writing batch programs you could use the command line interface syntax to extract the log files for the jobs in question into flat files that can be read, or processed or even e-mailed.
My preferred approach would be at the job level, but that would involve modifying all jobs. I think you could start with the dsjob -report option and, if that is not sufficient, graduate to something slicker.
there are many ways of getting the report information and the best for your site really does depend on what you prefer.
If you want to process the information from outside of DataStage and like writing batch programs you could use the command line interface
Code: Select all
dsjob -reportMy preferred approach would be at the job level, but that would involve modifying all jobs. I think you could start with the dsjob -report option and, if that is not sufficient, graduate to something slicker.
</a>I was thinking about something where I could write the status of the job to an oracle table, then create a web page for a monitoring system, but wasn't sure if the best way was to put code in every job or if there was a system table or file I could extract the data from every 10 to 20 min or so. Instead of having so much duplicate code in every job?
Hi,
just like Arnd said - You can always add a piece of job that will log (insert into db table) the status of the job that it runs.
You don't have to modify all Your jobs to do it - You can write a simple batch that will take as a parameter a job to run - the batch will log the start of loading, then start the job and after it's done it will log the status of the job.
just like Arnd said - You can always add a piece of job that will log (insert into db table) the status of the job that it runs.
You don't have to modify all Your jobs to do it - You can write a simple batch that will take as a parameter a job to run - the batch will log the start of loading, then start the job and after it's done it will log the status of the job.
Regards,
Wojciech Nogalski
Wojciech Nogalski
But then the jobs would have to load in order? There are a couple of processes that require the jobs to run in parallel? From what your saying is that I create a batch that runs each job, and then logs the errors if there is any?
wnogalski wrote:Hi,
just like Arnd said - You can always add a piece of job that will log (insert into db table) the status of the job that it runs.
You don't have to modify all Your jobs to do it - You can write a simple batch that will take as a parameter a job to run - the batch will log the start of loading, then start the job and after it's done it will log the status of the job.
You need to download EtlStats.zip from my tips page. It is a series of jobs which run dsjob -report XML and load the data into Oracle tables. The create table scripts are all there. There are lots of reports to give you row counts by sequence or run times of all sequences. There is a way to automatically email these reports at the end of each sequence. Do a search there is a lot there and I do not want to explain it all again but do not mind answering questions if you have problems setting it up. The latest version is a lot easier to install. We have been running most of this in production for months. It is very cool.
Mamu Kim
I'll check it out, I downloaded it and there is a EtlStats_Index.html page, do I just go down the job list one at a time and create them all?
kduke wrote:You need to download EtlStats.zip from my tips page. It is a series of jobs which run dsjob -report XML and load the data into Oracle tables. The create table scripts are all there. There are lots of reports to give you row counts by sequence or run times of all sequences. There is a way to automatically email these reports at the end of each sequence. Do a search there is a lot there and I do not want to explain it all again but do not mind answering questions if you have problems setting it up. The latest version is a lot easier to install. We have been running most of this in production for months. It is very cool.
It is in the zip. You can change the path to wherever you put this file. You need to be able to run the sequence SeqPrepareEtlStats before all of this is ready to add to each job. The email is controlled by a shell script called Email.ksh. It should be in a folder called Scripts below each project you set this up in. You need to modify this script to email to you and not me. I used "Kim" to email to my real email address but you can do this any way you want.
Mamu Kim