Hi all,
I am using 5 hashed files in single job and my hash file contains records around 50000 and my design looks like below.
seqfile---->TFM1----->TFM2----->TFM3----->Target
| | |
| | |
| | |
Lkp1 Lkp2 Lkp2
when i ran the job the first part i.e. from seqfile to TFM1 link was fetching at good speed around 3500 rows/sec and LKP1 file conatins around one million records.Now when it was running from TFM2 part the performance of reading and writing the records is too slow even though my Lkp2 file conatins only 50000 rows.The speed of reading and writing was only 50 rows/second(important thing here is that i am using the same LKP2
lookup at both TFM2 and TFM3)due to some business logic........
Can some one please guide me why the perfomance is too poor when it was reading the data and writing the data into the next link.
Thanks in advance.........
Regards,
skumar.
Poor performance of Hash file?
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Rafik----I am not doing any transformations in the TFM2 & TFM3 . i am doing only the lookup based on one key which it is meant to be used for the same..
Ray,
Could you please tell me some thing more about the read cahche file size because i don't know that......How can we check whether the file is read cached or not???
Thanks,
skumar.
Ray,
Could you please tell me some thing more about the read cahche file size because i don't know that......How can we check whether the file is read cached or not???
Thanks,
skumar.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
There is a field in each Hsahed File stage in which caching can be enabled or disabled, if the stage's output is connected to a reference link.
Size of the hashed file cache for the project is set in the Administrator client on the Performance tab.
Size of the hashed file cache for the project is set in the Administrator client on the Performance tab.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.