Processing on SMP & MPP architectures

mdbatra · Post by **mdbatra** » Sun Nov 23, 2008 1:48 pm

How actually the job processing is carried out in SMP & MPP/Cluster environments.

For SMP, say i have 3 processors & a N node configuration file defined. Then how the conductor node, section leaders, players are created.

For MPP, say, there are 3 CPUs ,each with a single processor. Curious to know the nature of configuration file(s) + creation of conductor nodes, section leaders & players.

Thanks!

ray.wurlod · Post by **ray.wurlod** » Sun Nov 23, 2008 1:52 pm

Exactly the same except:

(a) the fastnames are different in an MPP configuration file

(b) the mechanisms used for inter-process communication are different between SMP and MPP

mdbatra · Post by **mdbatra** » Mon Nov 24, 2008 5:31 am

Thanks Ray..
Can you elaborate a bit more on these 2 points.

attu · Post by **attu** » Mon Nov 24, 2008 1:56 pm

There will be only 1 controller process for the Job, for 3 node configuration there will be 1 section leader per node, Players would be created 1 for each stage per node. You will get the number of processes after you do the Math.
The Conductor starts the osh when a job is kicked off and it takes the required info from your configuration file, where you define your nodes and filesystem allocation.
Section leader runs osh on each node and it communicates with Conductor and Player processes.

As Ray mentioned (b) the mechanisms used for inter-process communication are different between SMP and MPP

SMP uses shared memory and MPP uses TCP.