No of Jobs

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Munish
Participant
Posts: 89
Joined: Sun Nov 19, 2006 10:34 pm

No of Jobs

Post by Munish »

Hi All,
I got to execute 30 jobs which are infact loading the data into fact and summary tables.
Configuration: 12 CPU, each CPU duo core solaris
Logical nodes: 12

I was just wondering that
1. If the larger number of job running parallel will effect the overall performance

or

2, Should I restrict number of jobs.

Please let me know of your thoughts.

Cheers,
Munish
MK
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Monitor your system, using a tool like top or sar or similar. If you have spare capacity, you're OK. If you have no spare capacity, you should reduce the number of concurrent jobs until you do have some spare capacity, or add more capacity.

It's probable that some of the smaller jobs could be run on fewer nodes. More of these could then run concurrently.

Ultimately your key performance indicator is whether you can meet the available time windows with a safety margin to spare.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
us1aslam1us
Charter Member
Charter Member
Posts: 822
Joined: Sat Sep 17, 2005 5:25 pm
Location: USA

Post by us1aslam1us »

It fairly depends on how much data (being processed) you are talking about?

Hey Ray i have seen that sar tool but it seems to be bit complicated as we need to install some other thing on the maching.
(i don't remember what else can be used but i feel top and iostat are best bet with ps)

Sam
Post Reply