Good morning all,
I am having performance problems with a datastage flow. The rows per second keeps decreacing for no obvious reason (3000 per second to about 300 per second). What I'm looking for is some kind of receipe of things to look at to maintain the initial performance.
Here is information on the job I'm looking at now:
Flat file into a single transformer, 3 hash file lookups , dumping the result in a hash file.
1.6 Million rows in the input and in each of the lookups.
Granted the hardware is not 'top notch': (I'll write the detail if needed)
What can I look at, the software install is cookie cutter, nothing changed or adapted in the apt files... Anyone have any insights? Could the size of the project have anything with it?
thank you for your time
Performance problem
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Are you using write cache for the target hashed file? If so, rows/sec screams along while rows are being written to memory, then appear to reduce. Actually at this time all rows have been processed into memory and are being flushed to disk; because the clock keeps running the rows/sec figure appears to be falling. But it's an artifact, and one of many reasons you should ignore rows/sec as a performance metric.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.