Number of CPU used by DS PX/EE

Dralair · Post by **Dralair** » Fri Jul 29, 2005 9:27 am

Hi,

does someone knows if it's possible to reduce the number of CPU used by DS ? My server has 8 CPU but I don't want the 8 of them to be used.
I don't want to change the number of nodes in the .apt that's not the solution ; it uses the 8 CPU whatever I'm in 2, 4 or 8 nodes.

Thanks,
Dralair

ArndW · Post by **ArndW** » Fri Jul 29, 2005 9:32 am

Dralair,

DS doesn't really know how many CPUs a system has - it's only interface to the "real" world is through the configuration file. The actual distribution of threads and processes across the CPUs is done by UNIX and cannot be controlled externally; a physical machine can, depending on HW and OS, be broken into virtual machines with their own CPU allocation, but that is not what you seem to be trying to do.

That was the long answer, the short one is "no, it is not possible"

Dralair · Post by **Dralair** » Fri Jul 29, 2005 9:46 am

That's what I was afraid of...

Thanks anyway for your time.

ray.wurlod · Post by **ray.wurlod** » Fri Jul 29, 2005 8:12 pm

One job creates one conductor process, one section leader process per processing node, and up to one process for each operator. You can reduce the total number of processes by allowing operators to combine into the same process. You can reduce the total number of processes by starting fewer jobs at a time. But that's about it.

DataStage starts processes; it has no inbuilt mechanism for limiting those processes to a particular subset of available CPUs. Indeed, its design philosophy is to grab as many resources as it needs to do the task.

Perhaps an enhancement request? If so, you would also need to specify how your proposed mechanism would be implemented in an MPP (cluster) environment.

jack_dcy · Post by **jack_dcy** » Sat Jul 30, 2005 2:43 am

I heard that in the newest edition of DataStage(Maybe 8.0), using a tool of another company, let you to specify the cpus your job want to run on.

ray.wurlod · Post by **ray.wurlod** » Sat Jul 30, 2005 5:38 pm

Where'd you hear that?

Some operating system vendors provide utilities (for example Affinity) that can tie a particular process to a particular CPU, but there's nothing within DataStage. Nor have I heard any rumour to that effect about the Hawk release. But I don't hear everything, so do let us know your source.