Hi,
I have a project where I call a master job sequence which then will call the child sequences, jobs activities, etc.
I want to redirect the logs of all jobs to the same file. This has to be done during the execution of the job (not when it is finished).
I would like to know what is the best way to do it. I'm not asking for all the details or for a script, just an indication of the most elegant wway to do it.
Thanks.
Write DataStage logs to a file (During Execution)
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 30
- Joined: Thu Jun 06, 2013 9:12 am
Do you mean the job log entries seen from the Director? If so, then I'd opine along with Craig that it cannot be done directly within the job without making some very low-level changes to the undocumented system calls.
You can use DSGetLogEventIds(),DSGetLogSummary() and DSGetLogEntry() to get information on log messages for a given job's log file. These messages are not always written to the log immediately and they can be buffered for a long period of time. You can have a background program running which polls the log entries of the jobs which are running and reads those entries and writes them somewhere in near-real-time.
You can use DSGetLogEventIds(),DSGetLogSummary() and DSGetLogEntry() to get information on log messages for a given job's log file. These messages are not always written to the log immediately and they can be buffered for a long period of time. You can have a background program running which polls the log entries of the jobs which are running and reads those entries and writes them somewhere in near-real-time.
-
- Participant
- Posts: 30
- Joined: Thu Jun 06, 2013 9:12 am
It was specifically the whole concept of 'redirection during execution' that I was noting was not really possible. Of course, the log contents can be captured and written elsewhere, plenty of discussions on that topic here... your wrinkle on that is the 'real time' component.
Neither of us mentioned Sequences, however. You certainly could but that would fall into the 'after the fact' camp. Arnd suggested something completely separate, a (possibly generic) process that runs in the background, checks for running jobs and farms their logs in what would be more properly termed 'near real time'. Again, something that certainly could be done but not sure it's really worth all of the effort and support... depends on your requirements, of course.
Neither of us mentioned Sequences, however. You certainly could but that would fall into the 'after the fact' camp. Arnd suggested something completely separate, a (possibly generic) process that runs in the background, checks for running jobs and farms their logs in what would be more properly termed 'near real time'. Again, something that certainly could be done but not sure it's really worth all of the effort and support... depends on your requirements, of course.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers