general maintenance to improve performance

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
dsguyok
Premium Member
Premium Member
Posts: 24
Joined: Thu Jan 21, 2010 10:22 pm

general maintenance to improve performance

Post by dsguyok »

Our Datastage 8 installation has been doing compilation progressively slower in the last month or so. We have been writing more and more jobs and they have been getting bigger but we have split them recently to reduce their size and hopefully their compile time. But the compile time is still too great.

Even simple operations like opening and deleting jobs takes longer than it did before.

Yesterday the GUI was slow to respond (ie drop-down menus not responding quickly to mouse movement).

Please suggest a few areas that can be checked/cleared in order to improve general performance. Maybe something has been accumulating without us noticing. Thanks.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

It could be that your XMETA is filling up an getting slower over time. Ask your DBA to check if general performance can be improved, i.e. through "runstats" on DB2.
At version 8.1 the DataStage log files go into XMETA by default. This can be switched back to the original repository storage by putting the following 2 line values into the DSParams file:

Code: Select all

RTLogging=1
ORLogging=0
setting the switches to 0 and 1 respectively switches logging to XMETA.

It is also worth occasionally going into the Project subdirectory "&PH&" and clearing out all the files.

The compile times and GUI response times are usually very dependant upon network performance. Next time you compile a job, monitor your workstation's network usage and you'll see that quite a bit of data flows back and forth during compilation.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Whenever you compile a job or a routine, the precompiler first performs a check to make sure that the object being compiled is not referred to elsewhere in the project. The more objects that exist in the project, the longer this check will take. I suspect this is the main cause of the degradation you are reporting. Consider splitting the project into two (or more) projects.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply