DataStage blamed again
Posted: Wed Aug 17, 2005 7:47 am
We run DataStage jobs daily. The database (Informix) is on a SAN and the hash files and sequential files are on the AIX server. The jobs are simple seq-->trfm-->seq in most cases with one or two hash file lookups. Hash files are large, most over 1,000,000 rows and at least 20 columns wide (not good I know).
We are rebooting the server every three or four days because it just starts to swap itself to death. The DataStage jobs are being blamed for consuming all the memory on the server. (60 x 1024) blocks of memory as an average. We have been told by the dba 'When DataStage starts, it starts allocating these memory blocks until the server starts swapping because all the memory is consumed'.
We have had a look at our jobs, at log file sizes, at uvconfig settings, dslockd size etc but all seems ok. The Project performance settings are 999 for hash file read and write cache and the inter process buffer size is set at 1024. At any point in time there are probably 10 jobs running concurrently.
Informix has been allocated 4GB of RAM on the server, the rest is available for DataStage and AIX.
We have suggested doing a health check on AIX (making sure all available patches are installed) and then re-installing DataStage. Don't know if this will solve the problem.
If anyone has had similar experiences, I would like to know how you solved them.
![Embarassed :oops:](./images/smilies/icon_redface.gif)
We are rebooting the server every three or four days because it just starts to swap itself to death. The DataStage jobs are being blamed for consuming all the memory on the server. (60 x 1024) blocks of memory as an average. We have been told by the dba 'When DataStage starts, it starts allocating these memory blocks until the server starts swapping because all the memory is consumed'.
![Sad :(](./images/smilies/icon_sad.gif)
We have had a look at our jobs, at log file sizes, at uvconfig settings, dslockd size etc but all seems ok. The Project performance settings are 999 for hash file read and write cache and the inter process buffer size is set at 1024. At any point in time there are probably 10 jobs running concurrently.
Informix has been allocated 4GB of RAM on the server, the rest is available for DataStage and AIX.
We have suggested doing a health check on AIX (making sure all available patches are installed) and then re-installing DataStage. Don't know if this will solve the problem.
![Confused :?](./images/smilies/icon_confused.gif)
If anyone has had similar experiences, I would like to know how you solved them.