Truncating Log Messages from before/after shell commands

[rameshrr3]

Im having a peculiar problem. I have 2 different jobs that execute 2 different Unix executables. One of them is a perl script and the other is a ELF executable. Both are called as Before Job Subroutines via ExecSH.

Whatever the perl script writes to stdout is captured in the datastage director log message, but gets truncated at some point . Usually after 256000 characters. The last line of that detailed log message is given below

Code: Select all

...[Message truncated to 256000 characters, actual length was 300021 characters]...

However the ELF executable also writes a large amount of data to std out , ( sometimes in excess of a few MB) , and when such a thing happens, instead of writing a truncated message to the datastage director log , the job fails with the error message :

Code: Select all

Job control process (pid xxxyz) has failed

.

Im suspecting that the very large size of output written by the ELF executable to stdout could be a problem as it cannot be stored in the fields of the RT_LOGnnnn file. Has anyone faced such a weird situation? Is there anything that i can configure to truncate log messages? Not sure how its happening for the perl script calling job that i mentioned above.

As far as i know no one is trying to stop the second job when its running, nor does it ( the ELF executable) use more than 15% of CPU ( as seen via top command).

[chulett]

Yeesh... why log all that? Why not redirect std out 'somewhere else' when you execute these beasties?

[rameshrr3]

I have already made that suggestion. In our project we also have a seperate 'log file' ( in plain text) that we are generating, a brief summary of the whole sequncer run, along with short messages about all warnings and fatal errors generated in the jobs of that sequencer. The sys admins are familiar with unix and would rather use cat command to see this log file than use director.
In my wildest imagination i did not imagine an otter morphing to an Orca .