CPU its Cores and Nodes

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
srireddypunuru
Premium Member
Premium Member
Posts: 40
Joined: Thu Jul 10, 2008 12:45 pm

CPU its Cores and Nodes

Post by srireddypunuru »

Guys.

We have a 2 CPUs, and each CPU has 6 cores with 24 GB RAM. The CPUs are also hyperthreaded and running on windows 64 bit box with IS 8.5.

I was trying to understand if each CPU has 6 Cores can i have 12 node Config file. I know more nodes more process but trying to understand what is the best Config file i can design in this Setup. We have all Parallel jobs reading and writing into Datasets and SQL Server Tables.

Really appreciate anyones Thought on this. Really confused with Cores and Nodes.

Below is the Config File we are using at this point of time.

{
node "node1"
{
fastname "ms979"
pools ""
resource disk "D:/IBM/InformationServer/Server/Datasets" {pools ""}
resource scratchdisk "D:/IBM/InformationServer/Server/Scratch" {pools ""}
}
node "node2"
{
fastname "ms979"
pools ""
resource disk "D:/IBM/InformationServer/Server/Datasets" {pools ""}
resource scratchdisk "D:/IBM/InformationServer/Server/Scratch" {pools ""}
}
}


A
Srikanth Reddy
Integration Consultant
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

There is no relationship between CPUs or "cores" to nodes.
-craig

"You can never have too many knives" -- Logan Nine Fingers
srireddypunuru
Premium Member
Premium Member
Posts: 40
Joined: Thu Jul 10, 2008 12:45 pm

CPU Cores Node

Post by srireddypunuru »

Thanks

But what would be the Ideal Config File setup for Hardware setup then
Srikanth Reddy
Integration Consultant
jhmckeever
Premium Member
Premium Member
Posts: 301
Joined: Thu Jul 14, 2005 10:27 am
Location: Melbourne, Australia
Contact:

Post by jhmckeever »

The number of logical nodes determines how many 'copies' of each OSH process are created. (Sorry -That's a terrible description, but you get the idea)

The number of concurrent processes suitable for your system can depend upon the number of jobs you're running concurrently, and the complexity of your jobs, along with many other factors. I've often found the ideal node configuration is best determined by trial and error.
<b>John McKeever</b>
Data Migrators
<b><a href="https://www.mettleci.com">MettleCI</a> - DevOps for DataStage</b>
<a href="http://www.datamigrators.com/"><img src="https://www.datamigrators.com/assets/im ... l.png"></a>
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

There is no single ideal setup. But, if there were, it would not have disk and scratchdisk resource specified in the default locations; it would have these spread over as many disks as possible and those disks would have plenty of free space.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
srireddypunuru
Premium Member
Premium Member
Posts: 40
Joined: Thu Jul 10, 2008 12:45 pm

Post by srireddypunuru »

Thanks Ray !
ray.wurlod wrote:There is no single ideal setup. But, if there were, it would not have disk and scratchdisk resource specified in the default locations; it would have these spread over as many disks as possible and those disks would have plenty of free space.
Srikanth Reddy
Integration Consultant
srireddypunuru
Premium Member
Premium Member
Posts: 40
Joined: Thu Jul 10, 2008 12:45 pm

Post by srireddypunuru »

Thanks John !
jhmckeever wrote:The number of logical nodes determines how many 'copies' of each OSH process are created. (Sorry -That's a terrible description, but you get the idea)

The number of concurrent processes suitable for your system can depend upon the number of jobs you're running concurrently, and the complexity of your jobs, along with many other factors. I've often found the ideal node configuration is best determined by trial and error.
Srikanth Reddy
Integration Consultant
Post Reply