Sudden degradation in performance of job collecting logs

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
vaby
Charter Member
Charter Member
Posts: 27
Joined: Tue Feb 15, 2005 1:45 am
Location: Russia

Sudden degradation in performance of job collecting logs

Post by vaby »

All of a sudden a set of jobs collecting DS log data started performing poorly. It now takes 3 and half hours as opposed to few minutes it took before (jobs run normally for the last couple of years). I assume this should have something to do with possibly corrupted or overloaded DS run-time repository (log files, indexes and such). Could you please suggest any techniques and/or methods to clean it up?

Anything else that might be useful to keep DS repository and DS run-time environment in a healthy state?

Thanks!

Vladimir
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Start off with a simple test - go to the project directory and list the files ordered by descending size. The data for the most common file type, DYNAMIC, is contained in a DATA.30 and OVER.30 file inside a directory which is the hashed file name. So, are there very large log files? If yes, you can use the TCL command "LIST DS_JOBS WITH JOBNO EQ {LogFileNumber}" to get the respective job names, then clear those files from the director.

It makes sense to make sure that your autopurge settings are appropriate for each file. The project level settings just give a default value for a job; so it is possible (and common) to have single jobs that have different settings and where the logs can grow indefinately.

If you don't have large log files then you need to look elsewhere for performance bottlenecks.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Those numbers are internal format for date and time; but the contents of that file (it is a sequential file) might contain the job name or number which you can lookup in the DS_JOBS table.
vaby
Charter Member
Charter Member
Posts: 27
Joined: Tue Feb 15, 2005 1:45 am
Location: Russia

Post by vaby »

Thanks again, all this info was very useful.
Post Reply