Hi all,
I faced a strang behavior in the configration file , which i need to understand
we are using datastage 7.5.2 on IBM AIX Server with 4 processors and 24 GB ram
i've made 8 nodes pointing to the same file system , i.e. the same node and copied it 8 times with diffrent name and i have some questions now :
1- There are warrning while checking configration file appears only on node1 the warning is
Warning: Config file checker on node node1 can only create 413 processes. Ascential Software recommends that ORCHESTRATE users be able to create at least 64 processes per logical node. This node is one of 8 on the physical node csdsp1. Therefore, ORCHESTRATE users should be able to create 512 processes on this node
although the 8 nodes refer to the same file system i.e. the datasets & scratch folders are the same for the 8 nodes , so i think that it is strang behavior to warn in one node only
2- does making 8 file systems , i.e. mount points and making each node point to different file system will improve the performance of parallel jobs ?
3- what is the difference between physical nodes and logical nodes , i really need to understand that
i tried to search dsxchange for these questions , i found really some usefull information but i need more illustrated answers for my questuions
i really appreciate your effort and support
Nodes Configration strange warning
Moderators: chulett, rschirm, roy
Nodes Configration strange warning
Mohmmed Elgendy
Senior System Analyst
Data IntegrationTeam
Etisalat Egypt
+20 1118511161
Senior System Analyst
Data IntegrationTeam
Etisalat Egypt
+20 1118511161
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
dear rayray.wurlod wrote:Check the UNIX kernel limit on the number of processes any process can start. This is usually called something like NPROC. ...
thanks a lot for your interest , i already checkd the maximum number of processes , and i can change them , but the question is , why this warninng appears only for one node , although all the nodes are mounted to the same filesystem , and the sam limits.
kindly ray , if you could illustrate the difference between logical nodes and phisical nodes , it will be totaly appreciated
Gendy
Mohmmed Elgendy
Senior System Analyst
Data IntegrationTeam
Etisalat Egypt
+20 1118511161
Senior System Analyst
Data IntegrationTeam
Etisalat Egypt
+20 1118511161
Physical nodes (or, in our context, physical CPUs) is what hardware you have. The Datastage concept of logical nodes defines how manz concurrent or parallel
DataStage uses logical nodes for defining levels of parallelism. The physical layout of a machine and the number and type of logical nodes defined in an APT_CONFIG file are not related. The operating system takes care of mapping the DataStage processes onto the physically available CPUs and other resources.
While you can declare a 16 node APT_CONFIG file and run it on a 1 CPU system, it wouldn't be a very efficient use of resources. There is no simple and straightforward answer to the relationship between logical and physical nodes.
DataStage uses logical nodes for defining levels of parallelism. The physical layout of a machine and the number and type of logical nodes defined in an APT_CONFIG file are not related. The operating system takes care of mapping the DataStage processes onto the physically available CPUs and other resources.
While you can declare a 16 node APT_CONFIG file and run it on a 1 CPU system, it wouldn't be a very efficient use of resources. There is no simple and straightforward answer to the relationship between logical and physical nodes.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
Completely agree with what Arnd posted, with one clarification in my mind.
In my context, a "physical node" would be the box, the physical server regardless of the number of CPUs on the box. You can further sub-divide a node into logical collections of resources, pools - scratch and resource disk spaces for instance, so that processes running on the same box can spread their loads across the different physical resources there.
In my context, a "physical node" would be the box, the physical server regardless of the number of CPUs on the box. You can further sub-divide a node into logical collections of resources, pools - scratch and resource disk spaces for instance, so that processes running on the same box can spread their loads across the different physical resources there.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
Nodes Configration strange warning
unfortunatly i couldn't read these posts because i'm a basic member , but anyway many thanks for ArndW , chulett
i'll mark the topic as resolved , for premium users
Gendy
i'll mark the topic as resolved , for premium users
Gendy
Mohmmed Elgendy
Senior System Analyst
Data IntegrationTeam
Etisalat Egypt
+20 1118511161
Senior System Analyst
Data IntegrationTeam
Etisalat Egypt
+20 1118511161