Tuning Solaris for Datstage 7.5.2
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 10
- Joined: Thu May 24, 2007 12:34 am
- Location: India
Tuning Solaris for Datstage 7.5.2
Hi,
We have migrated or DS server Jobs from Solaris 10 to Solaris 10 with additional hardware .
Since we have included additional RAM and processor,we are expecting our jobs to finish soon. But the jobs are running slowly when to our previous configuration.
I have searched the forum and found two links but im not satisfactoy with that.
viewtopic.php?t=115718&highlight=tuning ... +datastage
viewtopic.php?t=115407&highlight=tuning ... +datastage
Is the problem with Configuration of DS server or with solaris 10?
IS there any way to increase the performance by changing either DS engine files or kernel parameters?
Is there any values to be set up during DS Server installation?
Thanks,
Mohana Krishnan
We have migrated or DS server Jobs from Solaris 10 to Solaris 10 with additional hardware .
Since we have included additional RAM and processor,we are expecting our jobs to finish soon. But the jobs are running slowly when to our previous configuration.
I have searched the forum and found two links but im not satisfactoy with that.
viewtopic.php?t=115718&highlight=tuning ... +datastage
viewtopic.php?t=115407&highlight=tuning ... +datastage
Is the problem with Configuration of DS server or with solaris 10?
IS there any way to increase the performance by changing either DS engine files or kernel parameters?
Is there any values to be set up during DS Server installation?
Thanks,
Mohana Krishnan
You cannot tune DataStage at all unless you know what your bottleneck is.
The first step to tuning is understanding what is slowing your jobs down. Are you close to 100% CPU when jobs are running? If not, then CPU is most likely not your bottleneck. Are your I/O channels at maximum throughput? Is your system paging or swapping?
The first step to tuning is understanding what is slowing your jobs down. Are you close to 100% CPU when jobs are running? If not, then CPU is most likely not your bottleneck. Are your I/O channels at maximum throughput? Is your system paging or swapping?
-
- Participant
- Posts: 10
- Joined: Thu May 24, 2007 12:34 am
- Location: India
ArndW wrote:You cannot tune DataStage at all unless you know what your bottleneck is.
The first step to tuning is understanding what is slowing your jobs down. Are you close to 100% CPU when jobs are running? ...
Same jobs ,accessing same database ,almost same load but is it taking much longer time when compared to previous environment.
Number of records processed in each stage is also reduced in new environemnt .
Im not sure whether is problem is with DS server or Solaris kernel parameters.
Please give ur suggestions.
Thanks,
Mohana krishnan
I would tend to look at your SOLARIS configuration first. Tuning the DataStage engine is often counterproductive. Almost all of the tuning possible in DataStage is done with the uvconfig file, and odds are you haven't changed that in the old installation and thus it shouldn't be changed in the new, either.
-
- Participant
- Posts: 10
- Joined: Thu May 24, 2007 12:34 am
- Location: India
Hi kcbland,kcbland wrote:run prstat -a and watch your jobs ...
thanks for ur suggestion the real fact is i could not able to understand the syntax.
i searched the forum and found viewtopic.php?t=108451&highlight=prstat
But how it will help in my case, can u please tell me what prstat command stands for.
Normally we will be using ps -ef | grep -i 'RUN' and this will be showing all the jobs which are running,
And i need to find out where the issue lies,is a Database problem or Datastage server problem or solaris configuration problem.
Im not sure who is the culprit of above three.
But what we have discussed is DBA has no performance issues,so it must be either DS server or solaris im not sure about that.
can you please help me out.
Thanks,
Mohana krishnan
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
There are four major resource classes on a computer; processing, memory, disk I/O throughput and network bandwidth. Commands for monitoring all of these exist, for example prstat, vmstat for processors and virtual memory usage. More general tools, such as top, also exist.
These should be run for a substantial period (a couple of business cycles) to get statistically meaningful samples of resource usage. They should be run at "normal" times, to get a baseline measure of the system, then again when times are tough. Comparing the two sets of measures will identify the most likely location of the bottleneck.
Never react to a single sample of data.
Beware also that a CPU bottleneck does not mean that you don't also have a memory or disk bottleneck; the CPU bottleneck may hide (mask) the fact that demand for the other resources would be too great if only the processes making those demands could get enough CPU.
Strange as it may seem, tuning operating systems is not a skill you can learn in a day or two. It's something typically undertaken as "post-graduate" work by experienced System Administrators.
A final word: . in system tuning there are no shortcuts and there is no magic bullet.
These should be run for a substantial period (a couple of business cycles) to get statistically meaningful samples of resource usage. They should be run at "normal" times, to get a baseline measure of the system, then again when times are tough. Comparing the two sets of measures will identify the most likely location of the bottleneck.
Never react to a single sample of data.
Beware also that a CPU bottleneck does not mean that you don't also have a memory or disk bottleneck; the CPU bottleneck may hide (mask) the fact that demand for the other resources would be too great if only the processes making those demands could get enough CPU.
Strange as it may seem, tuning operating systems is not a skill you can learn in a day or two. It's something typically undertaken as "post-graduate" work by experienced System Administrators.
A final word: . in system tuning there are no shortcuts and there is no magic bullet.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.