7.5.2 Windows to 8.1 Linux Performance Discrepancy

kdg · Post by **kdg** » Fri Nov 05, 2010 1:51 pm

We are moving from Windows to Linux and from DS Server 7.5.2 to 8.1. We have a simple job that reads a sequential file and loads an ODBC stage. The Wintel servers are 5+ years old. The Linux server is a SUSE VM on our IBM z10. The performance on the Linux/8.1 server is not good despite all indications of the server having more than enough resources: RAM, CPU, I/O and network.

# of ------- |-------------- | ------------ | Average -- | Records
Records -- | DataStage | Database | Processing | per
Processed | Host ------ | Host ------ | Time ------- | Second
----------------------------------------------------------------------
739,837 | Win01 | Win02 | 14.5 min. | 850.4

739,837 | Win01 | Linux01 | 40.8 min. | 302.2

739,837 | Linux01 | Linux01 | 35.5 min. | 347.3

739,837 | Linux01 | Win02 | 38.7 min. | 318.6

Has anyone had a similar situation? Are we missing something? Is there a solution?

mobashshar · Post by **mobashshar** » Fri Nov 05, 2010 1:55 pm

Are you using Server or Parallel jobs?

kdg · Post by **kdg** » Fri Nov 05, 2010 2:28 pm

Server.

Sreenivasulu · Post by **Sreenivasulu** » Sun Nov 07, 2010 3:24 am

Is you linux running on a VM? Is the hardware configuration of your linux box same as the windows box ?
There might also be a case of multiple installation of server(datastage server, oracle client) being done in the linux box.

Regards
Sreeni

kdg · Post by **kdg** » Fri Nov 12, 2010 5:07 pm

The hardware specifications are as follows:

Linux01 (VM on IBM z10)
-------------------------------------------
CPU: 2 vCPU model 2097 (z10)
Mem: 4096MB / 3071MB page (vdisk)
Net: 10Gb Ethernet
Disk: Ficon attached

Win01 - DataStage Host
------------------------------------------
CPU: 4 CPU AMD 285 (2.6GHz)
Mem: 4096MB / 2046MB pagefile
Net: 100Mb Ethernet
Disk: Direct attached

Win02 - Oracle Host
-----------------------------------------
CPU: 4 CPU AMD 285 (2.6GHz)
Mem: 4096MB / 2046MB pagefile
Net: 100Mb Ethernet
Disk: Direct attached

Thank you for your time!

kdg · Post by **kdg** » Mon Nov 22, 2010 4:10 pm

Are there configuration settings during the DataStage 8.1 installation that would affect throughput? I would bet that our installation is a "default" installation.

ray.wurlod · Post by **ray.wurlod** » Mon Nov 22, 2010 5:29 pm

No, it's mainly about there being a radically different architecture. Version 8 is totally a service-oriented architecture, whereas version 7 is not. There's a lot more happening to support DataStage (the rest of the Information Server suite) in version 8, all of which consume some resources. The price of progress!

kdg · Post by **kdg** » Mon Nov 22, 2010 5:40 pm

Ray,

Sorry, I work for a small, cash-strapped government agency and therefore can't access Premium Content. All I can see is what is quoted above.

Thanks for your post.

ray.wurlod · Post by **ray.wurlod** » Mon Nov 22, 2010 7:59 pm

There wasn't much more, so I unblocked it.

ray.wurlod · Post by **ray.wurlod** » Mon Nov 22, 2010 8:01 pm

Actually there's another impact on server jobs, that they have to interact with the parallel engine (and therefore data have to be converted from typeless to typed or vice versa when things like process metadata are being collected).

kdg · Post by **kdg** » Tue Nov 23, 2010 9:41 am

Ray,

Thank you for your posts. Are you saying that the parallel engine is running and using resources even though we are not licensed for it?

ray.wurlod · Post by **ray.wurlod** » Tue Nov 23, 2010 1:28 pm

Yes, and you have paid for and licensed that part of it that supports communication between the engine and services tiers.