How to increase memory usage on HPUX
Moderators: chulett, rschirm, roy
How to increase memory usage on HPUX
I want to use px sort stage on HPUX.but I found something wrong with using memory.
It means that HP Box hase 60GB free memory. so I wanted to increase memory usage to around 1GB. (Parallel Degree = 12)
But When I set Restrict Memory Usage Parameter to 1000 (MB). But this job is aborted with some message. but Message was there is no space.
But this box has a lot of disk storage. and free memory.
so I thought that to use large memory on HPUX. I should set some special parameter for HPUX and Parallel Extender Engine.
For your information.
I aready checked kernal parameters( It statified ascential recommaned parameter(written in Ascenial Menual).
Please Let me know it, how to solve it.
Thanks in advance
Dhwankim
It means that HP Box hase 60GB free memory. so I wanted to increase memory usage to around 1GB. (Parallel Degree = 12)
But When I set Restrict Memory Usage Parameter to 1000 (MB). But this job is aborted with some message. but Message was there is no space.
But this box has a lot of disk storage. and free memory.
so I thought that to use large memory on HPUX. I should set some special parameter for HPUX and Parallel Extender Engine.
For your information.
I aready checked kernal parameters( It statified ascential recommaned parameter(written in Ascenial Menual).
Please Let me know it, how to solve it.
Thanks in advance
Dhwankim
Dhwankim,
could you post the exact error message - I think that the message you are getting is referring to temporary disk space (see your $APT_CONFIG values) and not virtual memory - although your system might have a lot of disk space left free, it might not be on the partition that is being used by Px.
could you post the exact error message - I think that the message you are getting is referring to temporary disk space (see your $APT_CONFIG values) and not virtual memory - although your system might have a lot of disk space left free, it might not be on the partition that is being used by Px.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
ArndW
I checked Scratch space for each node. But each pointed disk area has a lot of free space.
Each Scratched Area has 300GB left more. Is not It enough?
If so, Please give how to calculate the amount of needed disk space.
and Other thing is, When I decreased Restirict memory limit to around 60 (MG), than Job was well.
Please give your hands more.
Thanks in Advance.
I checked Scratch space for each node. But each pointed disk area has a lot of free space.
Each Scratched Area has 300GB left more. Is not It enough?
If so, Please give how to calculate the amount of needed disk space.
and Other thing is, When I decreased Restirict memory limit to around 60 (MG), than Job was well.
Please give your hands more.
Thanks in Advance.
ArndW wrote:Dhwankim,
could you post the exact error message - I think that the message you are getting is referring to temporary disk space (see your $APT_CONFIG values) and not virtual memory - although your system might have a lot of disk space left free, it might not be on the partition that is being used by Px.
dhwankim,
what I think is happening is that the stage is trying to access 1Gb of virtual memory for the process, but the OS will not allow the process so much space. What is your ulimit -a displayed value for memory? You either need to specify an amount in the Restrict memory limit that is smaller and/or you need to increase your process' virtual memory limit.
what I think is happening is that the stage is trying to access 1Gb of virtual memory for the process, but the OS will not allow the process so much space. What is your ulimit -a displayed value for memory? You either need to specify an amount in the Restrict memory limit that is smaller and/or you need to increase your process' virtual memory limit.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
Hi dhwankim,
you can increase memory usage on HP-UX using memory windows.
Here the issue:
- In the file /etc/hosts: at the line
<IP> <hostname> add <host2> <host3> <host4> ...
- In /etc/services.window add the lines:
<hostname> 50
<host2> 51
<host3> 52
<host4> 53
...
- In .rhosts of your user add:
<hostname> <your user>
<host2> <your user>
<host3> <your user>
<host4> <your user>
- In the Config file:
the first node may have "fastname <hostname>"
and the others may have the aliases.
Example:
{
node "node1"
{
fastname "hostname"
pools ""
resource disk "/dstageeetl3/Dataset" {pools ""}
resource scratchdisk "/dstageeetl3/Scratch" {pools ""}
}
node "node2"
{
fastname "host2"
pools ""
resource disk "/dstageeetl3/Dataset" {pools ""}
resource scratchdisk "/dstageeetl3/Scratch" {pools ""}
}
node "node3"
{
fastname "host3"
pools ""
resource disk "/dstageeetl3/Dataset" {pools ""}
resource scratchdisk "/dstageeetl3/Scratch" {pools ""}
}
...
In this way you can use 1,7 GB per node.
By
Matteo.
you can increase memory usage on HP-UX using memory windows.
Here the issue:
- In the file /etc/hosts: at the line
<IP> <hostname> add <host2> <host3> <host4> ...
- In /etc/services.window add the lines:
<hostname> 50
<host2> 51
<host3> 52
<host4> 53
...
- In .rhosts of your user add:
<hostname> <your user>
<host2> <your user>
<host3> <your user>
<host4> <your user>
- In the Config file:
the first node may have "fastname <hostname>"
and the others may have the aliases.
Example:
{
node "node1"
{
fastname "hostname"
pools ""
resource disk "/dstageeetl3/Dataset" {pools ""}
resource scratchdisk "/dstageeetl3/Scratch" {pools ""}
}
node "node2"
{
fastname "host2"
pools ""
resource disk "/dstageeetl3/Dataset" {pools ""}
resource scratchdisk "/dstageeetl3/Scratch" {pools ""}
}
node "node3"
{
fastname "host3"
pools ""
resource disk "/dstageeetl3/Dataset" {pools ""}
resource scratchdisk "/dstageeetl3/Scratch" {pools ""}
}
...
In this way you can use 1,7 GB per node.
By
Matteo.
Hi
I tested this functionality on SMP.
but I used memorywindow concept for HPUX 11.x
This concept is announced at releasing for HPUX 11.x .
How to overcome limit of memory usage for 32 bit application.
(I just read an article.)
Anyway, It could be applied for HPUX box.
so I had a enterprise job used some gb memory.
I tested this functionality on SMP.
but I used memorywindow concept for HPUX 11.x
This concept is announced at releasing for HPUX 11.x .
How to overcome limit of memory usage for 32 bit application.
(I just read an article.)
Anyway, It could be applied for HPUX box.
so I had a enterprise job used some gb memory.
T42 wrote:Oh, so this was a MPP setup?
increase memory usage on HP-UX using memory windows
Hi mmanes
We are implementing memory window option in HP unix server
Just changing max_mem_window will work or do i need to logically divide Hostname and assign each memory window
Thanks
Sanjay
We are implementing memory window option in HP unix server
Just changing max_mem_window will work or do i need to logically divide Hostname and assign each memory window
Thanks
Sanjay
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Even so, your tsort operators will still be constrained by any limit set explicitly in the Sort stage or by APT_TSORT_STRESS_BLOCKSIZE environment variable. You will probably need to increase one of these also.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Hi Ray,ray.wurlod wrote:Even so, your tsort operators will still be constrained by any limit set explicitly in the Sort stage or by APT_TSORT_STRESS_BLOCKSIZE environment variable. You will probably need to increase one of these also.
We want to use APT_TSORT_STRESS_BLOCKSIZE to increase the memory usage and hence performance.
Can you please tell us what is the default value for APT_TSORT_STRESS_BLOCKSIZE variable and upto what value we can increase the size of this variable? We could not find the details about this variable in product manual
Thanks for the help in advance
vishal
Hi Ray,ray.wurlod wrote:Even so, your tsort operators will still be constrained by any limit set explicitly in the Sort stage or by APT_TSORT_STRESS_BLOCKSIZE environment variable. You will probably need to increase one of these also.
We want to use APT_TSORT_STRESS_BLOCKSIZE to increase the memory usage and hence performance.
Can you please tell us what is the default value for APT_TSORT_STRESS_BLOCKSIZE variable and upto what value we can increase the size of this variable? We could not find the details about this variable in product manual
Thanks for the help in advance
vishal
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: