Server jobs performance on unix vs windows
Moderators: chulett, rschirm, roy
Server jobs performance on unix vs windows
We are migrating DataStage 7.5 server on Windows to 8.5 Enterprise on Solaris. This is a phased project. At phase one, we only do migration from 7.5 server jobs on windows to 8.5 server jobs on solaris, without any change of job design. At phase two, we will do performance tuning and rewrite server jobs into parallel jobs.
Now it's the end of phase one and we are suffering from worse performance of server jobs on solaris compared to windows. The unix server is Sun M4000, 64GB RAM, 1 * CPU Board (2 * 4 core, 2.66GHz SPARC64 VII+ CPUs). Windows server has 8 cores AMD opteron 2.4GHz CPUs on HP Proliant 880, 16GB RAM, a 5-year model. Our unix admin comments that because the jobs we run are mostly single-thread design, hence they are not taking advantage of parallel power from unix server. Furthermore, x64 CPUs perform better than Sun CPUs on single-thread jobs, even it's a 5-year model, regardless of CPU speed. It sounds like CISK vs RISC issue,
According to unix admin's comments, we are making wrong decision of migration, or says, IBM & consultants gave us wrong recommendation of migration before we started this project. We shouldn't do phase one becaues there is no benefit, however, due to limitation of resource, we do phase one as IBM & consultants said new Solaris server must be faster than windows server in terms of running server jobs.
Can we say, DataStage server jobs run faster on windows than solaris, based on similar spec of hardware, because of single-thread job natural?
Now it's the end of phase one and we are suffering from worse performance of server jobs on solaris compared to windows. The unix server is Sun M4000, 64GB RAM, 1 * CPU Board (2 * 4 core, 2.66GHz SPARC64 VII+ CPUs). Windows server has 8 cores AMD opteron 2.4GHz CPUs on HP Proliant 880, 16GB RAM, a 5-year model. Our unix admin comments that because the jobs we run are mostly single-thread design, hence they are not taking advantage of parallel power from unix server. Furthermore, x64 CPUs perform better than Sun CPUs on single-thread jobs, even it's a 5-year model, regardless of CPU speed. It sounds like CISK vs RISC issue,
According to unix admin's comments, we are making wrong decision of migration, or says, IBM & consultants gave us wrong recommendation of migration before we started this project. We shouldn't do phase one becaues there is no benefit, however, due to limitation of resource, we do phase one as IBM & consultants said new Solaris server must be faster than windows server in terms of running server jobs.
Can we say, DataStage server jobs run faster on windows than solaris, based on similar spec of hardware, because of single-thread job natural?
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Have you tried to narrow down exactly why your performance is poor on the Solaris box? What aspect(s) of the jobs you're running have been affected - read speed? Write speed? It is specific to your hashed files, for instance or has access times to remote relational databases taken a hit? You really need to quantify the nature of the 'performance issue' as (as Ray notes) there are ton of factors at play here.
There is also the significant architectural change with the 8.5 release, so realize you're not just simply comparing 'Windows v. UNIX' here.
There is also the significant architectural change with the 8.5 release, so realize you're not just simply comparing 'Windows v. UNIX' here.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
Have you compared any settings across servers, like the uvconfig file or the DataStage Administrator settings per project? If someone tweaked settings on Windows and you have default settings on Unix, seems like that is another factor that could slow down server jobs, not because of the OS difference.
Choose a job you love, and you will never have to work a day in your life. - Confucius
I understand it's not simply windows vs unix scenario. We have done heaps of testing and monitoring. All stats show that disk /IO or performance is great, network performace is great, and other indicators say unix server is in very good condition. However, it just runs slower than windows in terms of server jobs.
Oracle(Sun) also help us to diagnose but there is no encouraging result. Only thing we assure is CPU utilisation is 100% on 1 single CPU while other 7 ones are nearly idle when we are running testing jobs.
The same jobs run on windows show the same CPU utlisation but runing faster. And it's very hard for us to convince management we are making the right decision.
Oracle(Sun) also help us to diagnose but there is no encouraging result. Only thing we assure is CPU utilisation is 100% on 1 single CPU while other 7 ones are nearly idle when we are running testing jobs.
The same jobs run on windows show the same CPU utlisation but runing faster. And it's very hard for us to convince management we are making the right decision.
Thanks for all replies. This issue has bothered us for couple weeks. We did any necessary tweak and also raised PMR to IBM, but so far, no much improvement. We did get some performance gain from switching ZFS to UFS for hashed files location. However, the improvement is not good enough to fill the gap.
Now we don't expect server jobs can run faster on solaris than windows but at least, shuoldn't be worse. It's embarrassing that we spend big money on new hardware and software but it results in worse performance.
Now we don't expect server jobs can run faster on solaris than windows but at least, shuoldn't be worse. It's embarrassing that we spend big money on new hardware and software but it results in worse performance.
Yes, that's me. In the meanwhile, we start to convert some jobs in critical path into parallel and also get IBM to our site to investigate.chulett wrote:Ah... I was wondering if this was you but I hadn't checked the older posts yet.azens wrote:We did get some performance gain from switching ZFS to UFS for hashed files location.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
So you are seeing a CPU bottleneck?
Is that normal for all server jobs to run on a single CPU or core? If you start a second server job while the first server job is running on one CPU at 100%, then would you expect the second server job to land on a different CPU or core?
Is that normal for all server jobs to run on a single CPU or core? If you start a second server job while the first server job is running on one CPU at 100%, then would you expect the second server job to land on a different CPU or core?
Choose a job you love, and you will never have to work a day in your life. - Confucius
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Have you analyzed what part of the overall job process is slower in the new environment?
How do you run your jobs? Are there any scripts, before job/after job routines, etc., which may call ds server commands such as dsadmin?
Regards,
How do you run your jobs? Are there any scripts, before job/after job routines, etc., which may call ds server commands such as dsadmin?
Regards,
- james wiles
All generalizations are false, including this one - Mark Twain.
All generalizations are false, including this one - Mark Twain.