Number of Readers per node

arun@sinmax · Post by **arun@sinmax** » Thu Jun 15, 2006 3:51 am

Hi,
When I worked with sequensial file I have tried 'No of Readers per Node' option to make the extraction process faster.But the result I have got is just against

.
Following are result of my trails.

Start Time - Production Run Time (in Sec) - No of Nodes

03 - 08 - 1
03 - 09 - 2
04 - 09 - 4
07 - 10 - 8
09 - 13 - 16

Can you help me in this issue that how to apply this to make it faster.

gbusson · Post by **gbusson** » Thu Jun 15, 2006 3:57 am

how many rows do you want to read?

kumar_s · Post by **kumar_s** » Thu Jun 15, 2006 4:41 am

For a lengthier record, sometime it may not be effecient to increate the number of reader per node.

ArndW · Post by **ArndW** » Thu Jun 15, 2006 4:45 am

Never use anything less than a couple of minutes runtime for any timing comparison purposes! Your startup-time accounts for almost all your activity, so of course using more readers is going to add significantly to your times.

Don't even think about using multiple processes before your runtime goes over 5 or 10 minutes; it will just cause more overall system load and make your server run slower. This means that even though you might get your job to run in 30 seconds instead of 60 seconds the system will need to work 3 times as hard so your overall performance has gone down - unless you are not running in a multiuser/multijob environment.

kumar_s · Post by **kumar_s** » Thu Jun 15, 2006 5:54 am

Arnd - The sample shows the increase in elapse time of Runtime. Shouldnt it get shorter.
arun@sinmax - If you have large volume of data to run, pls do it and produce the result, which will be optimal to analyze.

ArndW · Post by **ArndW** » Thu Jun 15, 2006 6:07 am

The times depend upon repartitioning after the read as well, and we don't know what has been done with the data.