Equal Usage of CPU

rcil · Post by **rcil** » Wed Feb 08, 2006 3:25 pm

Hello All,

We have a 8 CPU production machine where EDW and DWH jobs will run. Can I know how we do a load balance so that both the environments EDW and DWH share the same amount of CPU usage?

Any ideas will be appreciated.

Thanks

ArndW · Post by **ArndW** » Wed Feb 08, 2006 5:05 pm

If you have a UNIX system you can never guarantee equal usage. I don't know how you differentiate or define your "EDW" and "DWH" - are they different DataSTage projects?

Assuming your jobs are CPU bound (which isn't always the case) then if you start n processes they will each get 1/nth of the resources; having 8 CPUs just increases the resources. So if "DWH" and "EDW" each start roughly the same number of processes and do similar type of work then they will share the system's resources evenly.

There is little other load balancing available to you. Playing with "nice" values can shift things back and forth - but only with regards to CPU and if won't necessarily affect database activities.

ray.wurlod · Post by **ray.wurlod** » Wed Feb 08, 2006 11:17 pm

You might look at separate logical partitions (LPARs) but these are totally separate, so you may end up having to have duplicated components.

rcil · Post by **rcil** » Thu Feb 09, 2006 12:23 pm

ArndW wrote:If you have a UNIX system you can never guarantee equal usage. I don't know how you differentiate or define your "EDW" and "DWH" - are they different DataSTage projects?

Thank you for the inputs. Yes, the EDW and DWH are two different projects.

thanks,
rcil

kcbland · Post by **kcbland** » Thu Feb 09, 2006 7:47 pm

If your OS allows, you can "domain" the server into two logical servers. But, if you requirement is that you can dynamically shift resources between separate processes, you could consider "processor affinity", which allows processes to be specific for use on a given cpu. By reserving one set of processors for one userid, and the other processors for the other, there will be separation of cpus.

ArndW · Post by **ArndW** » Fri Feb 10, 2006 1:59 am

Ken - IBM's use of dynamic LPARs is a good example of quickly and seamlessly changing CPUs from one UNIX instance to another. It takes seconds and is great for load balancing. Unfortunately, shifting the memory back and forth is an entirely different matter. It can take a large part of an hour to shift memory around, and in order to work efficiently the applications need to be coded so that they respond to memory re-allocation requests. Since DataStage is a memory hog just shifting around CPUs might not really balance load as much as people originally think.

kcbland · Post by **kcbland** » Sat Feb 11, 2006 8:55 am

The "load balancing" issue is a tricky question. Is the poster asking for the ability to control and arbitrarily shift cpu resources around, or are they simply wanting to keep one set of processes from hogging the machine.

I guessed the poster wanted to be able to "halve" the machine based on the "equal usage of cpu" subject line, rather than do true loading balancing. If one set of jobs can only use 1/3 of the machine, and another use 2/3 of the machine, then processor affinity or domaining might be the answer. If the question is to have either process consume the whole machine when executing along, but when timesharing then only consume to their limit, that's the trickier part. I'd have to defer the poster to their onsite unix admin because hardware and OS limitations will frame the response.