IS8.1 --> Linux 64 Bit - Performance issues

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
shareeman
Charter Member
Charter Member
Posts: 39
Joined: Tue Sep 23, 2003 6:09 am

IS8.1 --> Linux 64 Bit - Performance issues

Post by shareeman »

Hello All
I've had a quick look at the forums to pick up on something related but couldn't find anything specific, so here goes.....

We are having horrendous performance issues with an IS8.1 deployment on Red Hat Linux 5.x (64 bit) and by that i mean, the simple tasks of logging in, opening and compiling jobs, navigating from one folder to another, the lot. it's excruciatingly and painfully slow. this is even before we actually get to the job runs.

This is in addition to the several 'bugs' /issues we've found with various stages for which IBM are providing patches as we go along.

It seems to us that, although this is a 'recommended' platform, there is simply not enough QA to back this up.

After having ploughed through the various 'bugs' and finally got IS functional, we are hit with the performance issue.

Has anyone out there got a stable, robust and relatively bug-free 8.1 installation on a Red Hat Linux 64 bit platform. (We are on VMware)

Before you ask, i'm very well acquainted with IBM Support, I will shortly be camping outside their offices demanding better human rights!

To be fair, i think they do the best they can (at our expense!) but i reckon the Product on this platform is simply too flaky for the $$$ it costs.

Compare this with our 7.5.x Windows installation which works like a dream and we are seriously contemplating a rollback and a sizeable refund!

As you can tell, I'm ever so slightly miffed!
lstsaur
Participant
Posts: 1139
Joined: Thu Oct 21, 2004 9:59 pm

Post by lstsaur »

So, what do you think how much is the fair price for IIS 8.1.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

I don't think price really is the issue here. My current client is running 8.1 on a 32bit version of RedHat and had some interesting experiences getting it up and running before I got here.

Their biggest complaint was that the Linux port seemed to get short shrift, meaning things that had been fixed quite some time ago in other UNIX flavors were still broken in the Linux port. Sure, they were able to get patches but only after they reported the issue, it was found to have been reported and resolved in other flavors and then a patch created for Linux. Or so I hear. Still, it is running fairly well here but they're in no rush to 'productionalize' this version quite yet. They are happily running 32bit 7.5.2 on a 64bit Linux server, although I hear the "64bit" part was an accident. :wink:

So... I'm wondering how much the 64bit-ness is your factor versus RH itself. Or how much of a role VMWare plays here. Do you have any idea which of them are your main nemesis or do you think everything lies squarely on DataStage shoulders? Is your DataStage 64bit as well?
-craig

"You can never have too many knives" -- Logan Nine Fingers
shareeman
Charter Member
Charter Member
Posts: 39
Joined: Tue Sep 23, 2003 6:09 am

Post by shareeman »

I'm not sure if 32 bit Linux installations have the same teething problems as our 64 bit installation does but it ain't pretty i can tell you that, especially having spent the moolah to get your hands on a product that should be a cut above the rest.

Craig, I missed almost all of your post due to your Premium Membership

:(

Is there any chance i can get visibility of that post without having to cough up for the premium membership?

Is the Linux (64 bit) deployment even a popular choice or does everyone just opt for the Windows version instead?
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

You show as a charter premium subscriber, did you let it lapse? Regardless, nothing worth marking that way so I unset it.
-craig

"You can never have too many knives" -- Logan Nine Fingers
shareeman
Charter Member
Charter Member
Posts: 39
Joined: Tue Sep 23, 2003 6:09 am

Post by shareeman »

Thanks for that Craig, Appreciate it

I have no evidence to claim this has anything to do with RHL or VMWare, if anything, we have surplus firepower on the Server itself. We've also had IBM cast aspersions on the OS but they've found nothing upon investigation and have gone away satisfied with it.

For me this is entirely an IIS 8.1 issue

I concur with the view about Linux flavour patches having to be prepared by IBM for every issue that we stumble upon. it just seems to me that IBM support aren't acquainted well enough with this flavor and rely on us to keep them informed.

(IMHO) The IIS architecture is not primed for Linux deployment and i'm also reliably informed by IBM that we should expect a drop in performance due to a 'thick' client with this release, due to the JVM that is bundled along with the client.

We've found bugs every step of the way and have had to hassle IBM for patches for each and everyone of them.

As you can appreciate, the patch generation process takes time which we can ill-afford and this has caused absolute mayhem with our 'go-live' dates.

Funnily enough, we went with a Windows 64 bit deployment initially (as advised by IBM) but were subsequently informed by IBM that this platform is not supported yet and the recommended platform would be Linux 64 bit. Go Figure!

My biggest gripe would be that IBM do not have a dedicated Unix/Linux Support desk that can focus solely on this OS and related issues.
bobyon
Premium Member
Premium Member
Posts: 200
Joined: Tue Mar 02, 2004 10:25 am
Location: Salisbury, NC

Post by bobyon »

Our servers are AIX 5.3 and we are having similar performance issues. It takes several minutes to start DS clients, often a minute or more to open a job and sometimes up to a minute just to open a stage.

I've been told that 8.x version are much more network intensive than the 7.5 versions.

I've also noticed that our performance issues only happen on our dev and QA systems. The production system is much more responsive. I don't think it has anything to do with the horsepower differences of the servers. I believe it has to do with the distance between client and server. But that is merely a suspicion. I have not had time to research this fully yet.

Our dev and QA systems are physically half way cross the country at a Disaster Recovery site while our production system is not directly on site but fairly local (within 100 miles).

I've also heard that others have had similar performance troubles and have been using a workaround. They place a client system local to the server and remote desktop to that client cross-country. They state performance is much better that way.

I know this is no solution and may send you on a totally wild goose chase but I thought the additional information may help.
Bob
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

bobyon wrote:Our dev and QA systems are physically half way cross the country at a Disaster Recovery site while our production system is not directly on site but fairly local (within 100 miles).

I've also heard that others have had similar performance troubles and have been using a workaround. They place a client system local to the server and remote desktop to that client cross-country. They state performance is much better that way.
This can happen with any version, geographic distance is a killer for a 'fat client' application. I've seen that in 7.5 with clients being halfway around the globe from the server or even halfway across the States here. And yes, something like a co-resident Citrix server that you then connect to from wherever can make a world of difference.
-craig

"You can never have too many knives" -- Logan Nine Fingers
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

bobyon,

please take a look at your uvconfig file. I've noticed that the MFILES setting at 8.0.1 defaults to 12 - a ridiculously low number. If you have several jobs running and several users developing the system will quickly use up all 12 file units and start "swapping" them. In our developement environment the change from 12 to 512 brought the time necessary to open a job from 3-5 minutes back down to 10 seconds!

In addition, it is worth raising the T30FILES which is also set restrictively low. Times have changed and the couple of Kb reserved by these parameters no longer play a role in potentially decreasing system performance.
shareeman
Charter Member
Charter Member
Posts: 39
Joined: Tue Sep 23, 2003 6:09 am

Post by shareeman »

I was going to do something similar actually to see if it helps the performance. All of our users are experiencing performance issues, irrespective of their proximity to the Server and the number of concurrent users on the system.

I found this link

http://www-01.ibm.com/support/docview.w ... tf-8&lang=

that asks for the following parameters to be changed as follows in the uvconfig file.

MFILES=64, T30FILE=512, GLTABSZ=100 and RLTABSZ=100
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Don't ever change uvconfig without understanding why. You can break DataStage if you're not careful.

Involve your support provider to supply explanations.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
shareeman
Charter Member
Charter Member
Posts: 39
Joined: Tue Sep 23, 2003 6:09 am

Post by shareeman »

Hi Ray
Point taken, The more we delve into the nitty gritty of the uvconfig, the more reluctant we are to meddle with it. We are in touch with IBM Support regarding this.
alans
Participant
Posts: 1
Joined: Wed Jun 25, 2008 9:03 am

Re: IS8.1 --> Linux 64 Bit - Performance issues

Post by alans »

FYI... Took IBM DataStage lab folks 3 months ONSITE and numerous patches to get DS 8.0.1 running acceptably well on RH Linux 5, 64bit. The re-architecting introduced in DS 8.0.1 along with poor QA resulted in a less than stable release. The developer intensive environments (dev, test and qa) were the most painful in terms of bugs and performance. Production had less visable issues because of the limited developer access.
Post Reply