High Memory Utilization
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 258
- Joined: Tue Jul 04, 2006 10:35 pm
- Location: Toronto
High Memory Utilization
Hi,
I am just checking the health of our DS servers for a week now.
The reports consistantly show high memory utilization usually around 90%.
The memory on the servers is 128 GB
Is this something to be worried about? like bad design or need of performance tuning?
Regards,
Samyam
I am just checking the health of our DS servers for a week now.
The reports consistantly show high memory utilization usually around 90%.
The memory on the servers is 128 GB
Is this something to be worried about? like bad design or need of performance tuning?
Regards,
Samyam
Well, you need to understand your environment.
Is is it a cluster environment where the data crunching happens on a compute node (thus freeing up memory on your head node).
Is is running a ton of job at the time of monitoring?
Do you have shell scripts doing non (pure) datastage work on the host?
What is the "top" or "topas" telling you?
Have you run nmon to get system stats to tell you who is chewing up your memory?
We can't answer those questions for you.
Is is it a cluster environment where the data crunching happens on a compute node (thus freeing up memory on your head node).
Is is running a ton of job at the time of monitoring?
Do you have shell scripts doing non (pure) datastage work on the host?
What is the "top" or "topas" telling you?
Have you run nmon to get system stats to tell you who is chewing up your memory?
We can't answer those questions for you.
-
- Premium Member
- Posts: 258
- Joined: Tue Jul 04, 2006 10:35 pm
- Location: Toronto
-
- Premium Member
- Posts: 258
- Joined: Tue Jul 04, 2006 10:35 pm
- Location: Toronto
-
- Premium Member
- Posts: 258
- Joined: Tue Jul 04, 2006 10:35 pm
- Location: Toronto
Hi All,
The top -m command gives the below result.
top - 14:02:20 up 40 days, 4:40, 5 users, load average: 3.91, 2.80, 2.71
Tasks: 1117 total, 1 running, 1106 sleeping, 0 stopped, 10 zombie
Cpu(s): 0.2%us, 0.5%sy, 0.0%ni, 99.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 98871628k total, 98052996k used, 818632k free, 1452992k buffers
Swap: 1048568k total, 0k used, 1048568k free, 90826696k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
23244 root 25 0 1140m 192m 10m S 0.3 0.2 425:44.65 java
27380 dsadmin 16 0 911m 75m 16m S 0.0 0.1 17:55.93 osh
23209 root 25 0 743m 75m 7248 S 0.0 0.1 25:50.35 java
27289 dsadmin 16 0 901m 70m 16m S 0.0 0.1 17:36.29 osh
27381 dsadmin 16 0 912m 69m 16m S 0.0 0.1 17:21.65 osh
27372 dsadmin 16 0 901m 69m 16m S 0.0 0.1 20:04.11 osh
27377 dsadmin 18 0 901m 67m 16m S 0.0 0.1 17:53.93 osh
26326 dsadmin 19 0 248m 58m 7812 S 0.0 0.1 0:01.68 osh
26401 dsadmin 18 0 248m 58m 7812 S 0.0 0.1 0:01.46 osh
and more and more osh process say about a 100 more osh processes.
Is it normal to have so many osh running all the time?
Also there are 1106 processes sleeping is that normal too?..
Regards,
Samyam
The top -m command gives the below result.
top - 14:02:20 up 40 days, 4:40, 5 users, load average: 3.91, 2.80, 2.71
Tasks: 1117 total, 1 running, 1106 sleeping, 0 stopped, 10 zombie
Cpu(s): 0.2%us, 0.5%sy, 0.0%ni, 99.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 98871628k total, 98052996k used, 818632k free, 1452992k buffers
Swap: 1048568k total, 0k used, 1048568k free, 90826696k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
23244 root 25 0 1140m 192m 10m S 0.3 0.2 425:44.65 java
27380 dsadmin 16 0 911m 75m 16m S 0.0 0.1 17:55.93 osh
23209 root 25 0 743m 75m 7248 S 0.0 0.1 25:50.35 java
27289 dsadmin 16 0 901m 70m 16m S 0.0 0.1 17:36.29 osh
27381 dsadmin 16 0 912m 69m 16m S 0.0 0.1 17:21.65 osh
27372 dsadmin 16 0 901m 69m 16m S 0.0 0.1 20:04.11 osh
27377 dsadmin 18 0 901m 67m 16m S 0.0 0.1 17:53.93 osh
26326 dsadmin 19 0 248m 58m 7812 S 0.0 0.1 0:01.68 osh
26401 dsadmin 18 0 248m 58m 7812 S 0.0 0.1 0:01.46 osh
and more and more osh process say about a 100 more osh processes.
Is it normal to have so many osh running all the time?
Also there are 1106 processes sleeping is that normal too?..
Regards,
Samyam
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
As a general rule every stage on every node creates an osh process, and there is one extra osh process (the "section leader") per node per job, and one extra osh process (the "conductor") per job. The more nodes you have, and the more stages you have, the more osh processes you will see.
DataStage is not always brilliant at cleaning up osh processes at job end, particularly where jobs abort, but they should be cleaned up eventually.
DataStage is not always brilliant at cleaning up osh processes at job end, particularly where jobs abort, but they should be cleaned up eventually.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Premium Member
- Posts: 258
- Joined: Tue Jul 04, 2006 10:35 pm
- Location: Toronto
Yes, that's probably normal. Even a couple hundred osh processes always running from ISD jobs may be normal. You can verify normal by checking the same counts at various times.
If you're running a default setup on an SMP server, then by default DB2 will allocate 85 to 90% of the physical memory at startup whether it needs it or not. You can override that behavior if you want to and give it a hard upper limit based on a number 4K pages that you specify.
I probably wouldn't worry about it unless you're seeing jobs run unusually slow or jobs or other processes abort due to out of memory errors or the like.
If you're running a default setup on an SMP server, then by default DB2 will allocate 85 to 90% of the physical memory at startup whether it needs it or not. You can override that behavior if you want to and give it a hard upper limit based on a number 4K pages that you specify.
I probably wouldn't worry about it unless you're seeing jobs run unusually slow or jobs or other processes abort due to out of memory errors or the like.
Choose a job you love, and you will never have to work a day in your life. - Confucius
Agree. Why spend money on perfectly good memory and then not use it?qt_ky wrote:I probably wouldn't worry about it unless you're seeing jobs run unusually slow or jobs or other processes abort due to out of memory errors or the like.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Premium Member
- Posts: 258
- Joined: Tue Jul 04, 2006 10:35 pm
- Location: Toronto