Can we have hash file whose size exceeds 2GB by any means.
Need expert comments.
Hash file Size greater than 2 GB
Moderators: chulett, rschirm, roy
Hash file Size greater than 2 GB
----------------
Rgds,
Anupam
----------------
The future is not something we enter. The future is something we create.
Rgds,
Anupam
----------------
The future is not something we enter. The future is something we create.
Re: Hash file Size greater than 2 GB
No expert on hashfiles, but it can be done. The "size problem" is because of 32bit pointers used internally in hashfiles which is usually the default compared to using 64bit pointers.anupam wrote:Can we have hash file whose size exceeds 2GB by any means.
Need expert comments.
You can specify this when creating a hash file (but not via the DataStage GUI). You can also specify that the default pointer size is 64 bit using the 64BIT_FILES in the uvconfig file.
I usually stop using hash files when they reach a size between 100 and 200Mb. If this is for a job in production recreate the hash-file using 64 bit pointers and go with it, if it's for a new job:
* Only load the data which is absolutely required in the hashfile;
* Consider reconsidering using a hashfile: what's the point of using hashfiles when you can't load the hash file completely in memory.
For more information on hash files do a search on this site, it has been covered numerous times.
Ogmios
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
STRONG ADVICE
Don't enable the 64BIT_FILES configuration parameter. This sets ALL your hashed files to >2GB capability. You don't need it.
The syntax for creating hashed files with >2GB capacity (that is, with 64-bit internal pointers) is to use the appropriate option:
64BIT with the CREATE.FILE or CREATE TABLE verb
-64bit with the mkdbfile utility
The Hashed File Calculator lets you choose 32-bit or 64-bit via option buttons, and generates the appropriate commands.
![Exclamation :!:](./images/smilies/icon_exclaim.gif)
Don't enable the 64BIT_FILES configuration parameter. This sets ALL your hashed files to >2GB capability. You don't need it.
The syntax for creating hashed files with >2GB capacity (that is, with 64-bit internal pointers) is to use the appropriate option:
64BIT with the CREATE.FILE or CREATE TABLE verb
-64bit with the mkdbfile utility
The Hashed File Calculator lets you choose 32-bit or 64-bit via option buttons, and generates the appropriate commands.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
An existing hashed file, provided there is a VOC entry pointing to it, can be converted to 64-bit internal addressing with the following command.
Code: Select all
RESIZE hashedfilename * * * 64BIT
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.