DSXchange

Good morning all,
I am having performance problems with a datastage flow. The rows per second keeps decreacing for no obvious reason (3000 per second to about 300 per second). What I'm looking for is some kind of receipe of things to look at to maintain the initial performance.

Here is information on the job I'm looking at now:
Flat file into a single transformer, 3 hash file lookups , dumping the result in a hash file.

1.6 Million rows in the input and in each of the lookups.
Granted the hardware is not 'top notch': (I'll write the detail if needed)

What can I look at, the software install is cookie cutter, nothing changed or adapted in the apt files... Anyone have any insights? Could the size of the project have anything with it?

thank you for your time

If you use a dynamic hash file (default) the rows per second decrease and later increase, that is normal due to administration tasks. Look at the total performance. If you are happy leave it so. If not check the manual and use static hash file.

Are you using write cache for the target hashed file? If so, rows/sec screams along while rows are being written to memory, then appear to reduce. Actually at this time all rows have been processed into memory and are being flushed to disk; because the clock keeps running the rows/sec figure appears to be falling. But it's an artifact, and one of many reasons you should ignore rows/sec as a performance metric.

Good morning again, the problem seems to be resolved. The job ran during the night with no problems.

<turns red> the end file did not have 'write cache' checked. </turns red>

Ty all for the help.