Monitoring jobs with UNIX details (memory, space, etc.)

krystlecsy · Post by **krystlecsy** » Thu Aug 24, 2006 10:06 pm

Hi all,

Just want to check with y'all whether DataStage has the capability or program hidden somewhere that is able to monitor the job running with details of the server memory and the space left and all?

kduke · Post by **kduke** » Thu Aug 24, 2006 11:32 pm

You need to create your own. Maybe Ken or someone has something they can share.

ray.wurlod · Post by **ray.wurlod** » Fri Aug 25, 2006 1:40 am

Yes. You enable collection with Boolean environment variables such as APT_STARTUP_SCRIPT, APT_PM_PLAYER_MEMORY, APT_PM_SHOW_PIDS and so on. RTFM.

krystlecsy · Post by **krystlecsy** » Mon Aug 28, 2006 4:52 am

So, by setting those variables to TRUE, the Director log will show all the information on this? Will it cause any performance problem by doing this?

ray.wurlod · Post by **ray.wurlod** » Mon Aug 28, 2006 4:59 am

Of course it will! There's no such thing as a free lunch. You can't have your cake and eat it too. <<Insert favourite cliche here>>.

It's a fundamental fact of performance monitoring that you want to impact the object being measured as little as possible.

But your original question did not pose any reservations about performance, and it was that that I answered.

(The performance impact is not that bad, actually. For a particular definition of "performance".)

vmcburney · Post by **vmcburney** » Mon Aug 28, 2006 6:16 am

The next version of DataStage has an option to turn on performance monitoring in a job and produces 19 different performance graphs on disk I/O, RAM and CPU utilisation. Have a look at the Advanced Parallel Job Developers Guide section on memory, buffering, disk I/O and CPU utilisation for descriptions on how the environment variables work.