Migration to different host became speed 10 times lower

saikrishna · Post by **saikrishna** » Wed Sep 10, 2008 12:21 am

Hi

We recently migrated our dev project from one linux box "A" to other linux box "B".
There was a job, which creates a hash file from a table as follows.

DB Stage -> Hash File

The number of records in DB : 2,471,947 records
It took 10 mins in our earlier "A" system (at 4510 rows/sec), where as it took 1 hr 40 mins in new "B" system at (at 406 rows/sec).

Both are having the following versions:
DataStage server 7.5.2

Any inputs on this would be great.

Thanks
Sai

chulett · Post by **chulett** » Wed Sep 10, 2008 12:24 am

What's different between the two servers?

saikrishna · Post by **saikrishna** » Wed Sep 10, 2008 12:39 am

A system's top command output as follows
top - 23:34:59 up 54 days, 4:29, 2 users, load average: 0.00, 0.00, 0.00
Tasks: 123 total, 1 running, 122 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0% us, 0.0% sy, 0.0% ni, 99.9% id, 0.0% wa, 0.0% hi, 0.1% si
Mem: 8166112k total, 8137588k used, 28524k free, 50252k buffers
Swap: 2048276k total, 23880k used, 2024396k free, 7890272k cached

B system's top command ouptut as follows
104 processes: 101 sleeping, 2 running, 1 zombie, 0 stopped
CPU states: cpu user nice system irq softirq iowait idle
total 18.4% 0.0% 2.9% 0.0% 0.0% 7.7% 70.9%
cpu00 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 100.0%
cpu01 0.1% 0.0% 0.0% 0.0% 0.0% 0.0% 99.8%
cpu02 65.0% 0.0% 10.5% 0.0% 0.0% 15.1% 9.1%
cpu03 8.3% 0.0% 1.1% 0.0% 0.0% 15.7% 74.6%
Mem: 5907468k av, 5886096k used, 21372k free, 0k shrd, 521072k buff
2691660k actv, 1740296k in_d, 129928k in_c
Swap: 5907432k av, 0k used, 5907432k free 3382448k cached

Please let me know specifically you need anything about the differences

Thanks
Sai

WoMaWil · Post by **WoMaWil** » Wed Sep 10, 2008 12:42 am

on which machine and how connected is you DB.

saikrishna · Post by **saikrishna** » Wed Sep 10, 2008 12:50 am

Oracle client is installed on both of the machines.
added tnsentries of the server and connected to DB same way using the same plugins.

Infact we just made an export copy and then imported back to the B server from A server.

Thanks
Sai

ray.wurlod · Post by **ray.wurlod** » Wed Sep 10, 2008 1:32 am

Are the hashed files identically configured (see Options in Hashed File stage)? Is the 64BIT_FILES setting in uvconfig the same in both machines?

saikrishna · Post by **saikrishna** » Wed Sep 10, 2008 2:39 am

Hi Ray

In both the places it is as follows

64BIT_FILES 0

Hemant_Kulkarni · Post by **Hemant_Kulkarni** » Wed Sep 10, 2008 3:56 am

Are the tunables (Read /Write cache size for Hashed file stage & Active-Active link Performance settings) The same for both the projects ?

saikrishna · Post by **saikrishna** » Wed Sep 10, 2008 6:09 am

Read/Write cache size is 128 MB for both 'A' system and 'B' system.

Earlier A and B both dont have in-process row buffer or inter-process row buffer.

To improve the system I enabled B system's in-process row buffering for 128 KB.

Thanks
Sai

balajisr · Post by **balajisr** » Wed Sep 10, 2008 6:39 am

Try removing DB stage and replace it with sequential file stage and verify whether the performance improves.Also, verify whether you are connecting to same db server.