Hi,
i need to read the log file for a specified job, after its execution, to display its content in my application.
Any idea? Where is this file? Can i just read the content? If my job is a sequencer, how to access the underlying jobs log file?
Regards,
Christophe
Logs
Moderators: chulett, rschirm, roy
Christophe,
I am not aware of a file for storing job logs but you can access them easily with DataStage Basic.
You can use several Basic functions depending on what you want. In my project, I use a routine to attach the job for which I need the log. After DSAttachJob(). You can attach a job after it has been executed and the handle will get you infos about the last run. Then I use the handle to call one of these function:
DSMakeJobReport(...) : to get a general report of your job(start and end time, link counts, etc.)
DSGetLogEntry(...) : to get specific entries in your log using a log id (see DSGetNewestLogId(...) ) useful for chronological entries.
DSGetLogSummary(...) : to get an array containing the entire log or only the parts you want (Warnings, Fatal, etc.).
When calling for log entries, you can specify start and end time to get the logs you want. To get only the last run. Call DSGetJobInfo(...) asking for Job Start time and use it in your calls.
The functions above apply to jobs as well as job sequencers. For your jobs within a sequencer however, it seems to be a little more difficult. You need a list of your jobs names to call DSAttachJob(...). I asked the question to Ascential support and I haven't had any answer so far. I will start harassing them by phone today and when I get any news I will post it here.
Hope it helps
Mathieu
I am not aware of a file for storing job logs but you can access them easily with DataStage Basic.
You can use several Basic functions depending on what you want. In my project, I use a routine to attach the job for which I need the log. After DSAttachJob(). You can attach a job after it has been executed and the handle will get you infos about the last run. Then I use the handle to call one of these function:
DSMakeJobReport(...) : to get a general report of your job(start and end time, link counts, etc.)
DSGetLogEntry(...) : to get specific entries in your log using a log id (see DSGetNewestLogId(...) ) useful for chronological entries.
DSGetLogSummary(...) : to get an array containing the entire log or only the parts you want (Warnings, Fatal, etc.).
When calling for log entries, you can specify start and end time to get the logs you want. To get only the last run. Call DSGetJobInfo(...) asking for Job Start time and use it in your calls.
The functions above apply to jobs as well as job sequencers. For your jobs within a sequencer however, it seems to be a little more difficult. You need a list of your jobs names to call DSAttachJob(...). I asked the question to Ascential support and I haven't had any answer so far. I will start harassing them by phone today and when I get any news I will post it here.
Hope it helps
Mathieu
-
ray.wurlod
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
DataStage logs are tables in the underlying DataStage Engine database (formerly known as UniVerse).
If you need to access them from an operating system command line, then you need the DataStage bin directory in your PATH, and you need (on UNIX) to have executed the dsenv script in the DataStage Engine direcory. For example:
PATH=$PATH:`cat /.dshome`/bin;export PATH
. `cat /.dshome`/dsenv
(On versions of DataStage earlier than 5.0, use uvhome rather than dshome.)
Next, you need to determine the job number of the job sequencer (which is a DataStage job containing job control code only) or of a particular DataStage job. From the UNIX command line:
uv "SELECT JOBNO FROM DS_JOBS WHERE NAME = 'MyJobName';"
Now that you have the job number, you can construct the name of that job's log table, which is RT_LOGnn, where nn is the job number. To access this table:
uv "SELECT * FROM RT_LOGnn ORDER BY @ID;"
Useful column names are SEVERITY, TIMESTAMP, FULL.TEXT and WAVE.NO (yes, the "." character is permitted in column names). The column @ID contains the event number, padded with leading zeroes. Thus, for example, you could constrain the result set:
WHERE SEVERITY IN ('Warning', 'Fatal', 'Reject')
Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518
Before you do all of that, though, ask yourself whether you really need to do all of this. Process metadata (a record of what happened) can be automatically captured into MetaStage, which has good tools available for reporting on it.
If you need to access them from an operating system command line, then you need the DataStage bin directory in your PATH, and you need (on UNIX) to have executed the dsenv script in the DataStage Engine direcory. For example:
PATH=$PATH:`cat /.dshome`/bin;export PATH
. `cat /.dshome`/dsenv
(On versions of DataStage earlier than 5.0, use uvhome rather than dshome.)
Next, you need to determine the job number of the job sequencer (which is a DataStage job containing job control code only) or of a particular DataStage job. From the UNIX command line:
uv "SELECT JOBNO FROM DS_JOBS WHERE NAME = 'MyJobName';"
Now that you have the job number, you can construct the name of that job's log table, which is RT_LOGnn, where nn is the job number. To access this table:
uv "SELECT * FROM RT_LOGnn ORDER BY @ID;"
Useful column names are SEVERITY, TIMESTAMP, FULL.TEXT and WAVE.NO (yes, the "." character is permitted in column names). The column @ID contains the event number, padded with leading zeroes. Thus, for example, you could constrain the result set:
WHERE SEVERITY IN ('Warning', 'Fatal', 'Reject')
Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518
Before you do all of that, though, ask yourself whether you really need to do all of this. Process metadata (a record of what happened) can be automatically captured into MetaStage, which has good tools available for reporting on it.
Christophe,
To answer your question about accessing logs of jobs within a job sequencer. I talked with Ascential and it seems to be more complicated then I thought at first to get the job names in order to read the logs. The job names are easily viewable through DS Manager if you look at the job dependencies of a sequencer. Unfortunately, accessing the same info with a script or a Basic function is not possible at this time. Ascential told me it was lying in the repository and that the repository structure is proprietary information of Ascential. Meaning they won't tell you how to get it. I filled a request for future enhancement but my hopes are not to good. That was basically my excellent and useful [:(] contact with Ascential support.
Wish I had better news
MAT
To answer your question about accessing logs of jobs within a job sequencer. I talked with Ascential and it seems to be more complicated then I thought at first to get the job names in order to read the logs. The job names are easily viewable through DS Manager if you look at the job dependencies of a sequencer. Unfortunately, accessing the same info with a script or a Basic function is not possible at this time. Ascential told me it was lying in the repository and that the repository structure is proprietary information of Ascential. Meaning they won't tell you how to get it. I filled a request for future enhancement but my hopes are not to good. That was basically my excellent and useful [:(] contact with Ascential support.
Wish I had better news
MAT
-
ray.wurlod
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
That's very interesting, because how to do this is covered in their "Programming with DataStage BASIC" class, specifically archiving a DataStage job log to a text file. It may be that someone in Ascential isn't fully up to date on what's possible! Although accessing logs from BASIC has been possible since version 1.0, so it's not new. And the layout of, and metadata for, job logs has not changed in all that time.
Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518
Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518
-
vmcburney
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
I think Mat was talking about the challenge of taking a Sequence job and retrieving the names of server jobs called by it. From the original post "If my job is a sequencer, how to access the underlying jobs log file?" Since you can't retrieve a list of job names using standard Basic code there are a couple other methods:
- Call all your server jobs from a central routine. Get this routine to output the required job log using the commands suggested by Ray or Mat into a file, give the file the name of the sequence job using the DSJobName macro. When your sequence job finishes get your application to open the text file that matches the sequence job name to display all the logs from all the server jobs run by that sequence job.
- Retrieve all the log messages for the sequence job. Find and parse the log entries with the DSRunJob command:
"SequenceJob..JobControl (DSRunJob): Waiting for job GetCustomers to start"
This means locating the string DSRunJob, then picking out the job name from the remaining text. Use Ray's suggestions for retrieving the job number and job log to get the log for this job.
Vincent McBurney
Data Integration Services
www.intramatix.com
- Call all your server jobs from a central routine. Get this routine to output the required job log using the commands suggested by Ray or Mat into a file, give the file the name of the sequence job using the DSJobName macro. When your sequence job finishes get your application to open the text file that matches the sequence job name to display all the logs from all the server jobs run by that sequence job.
- Retrieve all the log messages for the sequence job. Find and parse the log entries with the DSRunJob command:
"SequenceJob..JobControl (DSRunJob): Waiting for job GetCustomers to start"
This means locating the string DSRunJob, then picking out the job name from the remaining text. Use Ray's suggestions for retrieving the job number and job log to get the log for this job.
Vincent McBurney
Data Integration Services
www.intramatix.com
-
ray.wurlod
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
A late thought.
Surely the job sequence's own log file contains a number of "Job Run Requested" events, each of which will contain not only the name of the job for which the request was made but also the parameter values that were set for it and it's timestamp.
You could process the job sequence's log to determine the names of the controlled jobs, then proceed as above.
(Obtaining the hierarchy from design-time information, before the sequence has executed, is an entirely different can of worms!)
Surely the job sequence's own log file contains a number of "Job Run Requested" events, each of which will contain not only the name of the job for which the request was made but also the parameter values that were set for it and it's timestamp.
You could process the job sequence's log to determine the names of the controlled jobs, then proceed as above.
(Obtaining the hierarchy from design-time information, before the sequence has executed, is an entirely different can of worms!)