Write DataStage logs to a file (During Execution)

[meriem_ens]

Hi,

I have a project where I call a master job sequence which then will call the child sequences, jobs activities, etc.

I want to redirect the logs of all jobs to the same file. This has to be done during the execution of the job (not when it is finished).

I would like to know what is the best way to do it. I'm not asking for all the details or for a script, just an indication of the most elegant wway to do it.


Thanks.

[chulett]

Not possible.

[ArndW]

Do you mean the job log entries seen from the Director? If so, then I'd opine along with Craig that it cannot be done directly within the job without making some very low-level changes to the undocumented system calls.

You can use DSGetLogEventIds(),DSGetLogSummary() and DSGetLogEntry() to get information on log messages for a given job's log file. These messages are not always written to the log immediately and they can be buffered for a long period of time. You can have a background program running which polls the log entries of the jobs which are running and reads those entries and writes them somewhere in near-real-time.

[meriem_ens]

Thank you for your answers,

so basically if I want real time logs I need to implement that logic inside my sequences, right ?

I really tought it would be an easy task to redirect the output from the director to a file in real time, I guess i was being too optimistic.


Thank you.

[chulett]

It was specifically the whole concept of 'redirection during execution' that I was noting was not really possible. Of course, the log contents can be captured and written elsewhere, plenty of discussions on that topic here... your wrinkle on that is the 'real time' component.

Neither of us mentioned Sequences, however. You certainly could but that would fall into the 'after the fact' camp. Arnd suggested something completely separate, a (possibly generic) process that runs in the background, checks for running jobs and farms their logs in what would be more properly termed 'near real time'. Again, something that certainly could be done but not sure it's really worth all of the effort and support... depends on your requirements, of course.