Datastage configuration file

[chandra.shekhar@tcs.com]

Dear All,

Need some information about datastage configuration file.

We have 2 DS+QS engine . Engine 1 containing 10 core(CPU) and engine 2 containing 9 core (CPU) with 40 gb ram on each server.

While running job what configuration file we have create so that we can take the advantages of processing of both server .

Basically my configuration sample file as follows:

{
node "node1"
{
fastname "Engine1"
pools ""
resource disk "/resource1" {pools ""}
resource scratchdisk "/scratch1" {pools ""}
}
node "node2"
{
fastname "Engine2"
pools ""
resource disk "/resource2" {pools ""}
resource scratchdisk "/scratch2" {pools ""}
}

}


Can anybody explain me configuration file wrt to cpu (10 +9 CPU)

Thanks.

[arvind_ds]

You can start with below.

No of nodes = Half the number of cores.

[chandra.shekhar@tcs.com]

You can start with below.

No of nodes = Half the number of cores.

Means that I can use following combination.

10 cpu =20 Nodes and 9 cpu =18 nodes.

So can I use 20 +18=38 nodes for loading data ? or It can hampered the performance while using 38 nodes. How can I get the information that how many nodes I have to use for a perticular job ?

Thanks

[chanaka]

That depends on the complexity of the jobs that you run. If you have the grid version of the InfoSphere then its taken care of by the resource manager. Else you have to use different configuration files based on the complexity of the job.

From your explanation it sounds like an SMP cluster. Checkout the link below. It may help you further.
http://publib.boulder.ibm.com/infocente ... n_SMP.html

[PaulVL]

You also have to balance it out with how much data you are processing.

Is your job CPU bound or IO bound?

A bigger job (# of nodes) is not always going to be faster than a smaller one.

[ray.wurlod]

You can start with below.

No of nodes = Half the number of cores.

Means that I can use following combination.

10 cpu =20 Nodes and 9 cpu =18 nodes.

No it doesn't. It means
10 cpu = 5 nodes and 9 cpu = 5 nodes.