How to estimate sratchdisk space for sorting
Moderators: chulett, rschirm, roy
How to estimate sratchdisk space for sorting
I have to sort data with volume of 900 million records. The sort is on two keys with 26 bytes in total. The sort is not stable sort. If I caluclate 900 million multiple by 26, it should be about 23 G of data. But it consumes more than 200G of scratch disk space. What does the other data come from? Does it come from the buffer overflow? We do not specify buffer resource in the configure file. Thanks
Regards
Jie
Jie
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
As Paul noted, the entire Data Set has to be sorted, not just the keys. Estimate not less than two times the entire volume of the Data Set as the requirement for scratch space when sorting. More is always better.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Yes, I was being simplistic. For example are your VarChar data types bounded (have a maximum length) or unbounded? There are also other overheads associated with storage in a Data Set - approximately 80 bits per row last time I looked (version 7.x).
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.