Hash file Size greater than 2 GB

[anupam]

Can we have hash file whose size exceeds 2GB by any means.

Need expert comments.

[ogmios]

Can we have hash file whose size exceeds 2GB by any means.

Need expert comments.

No expert on hashfiles, but it can be done. The "size problem" is because of 32bit pointers used internally in hashfiles which is usually the default compared to using 64bit pointers.

You can specify this when creating a hash file (but not via the DataStage GUI). You can also specify that the default pointer size is 64 bit using the 64BIT_FILES in the uvconfig file.

I usually stop using hash files when they reach a size between 100 and 200Mb. If this is for a job in production recreate the hash-file using 64 bit pointers and go with it, if it's for a new job:
* Only load the data which is absolutely required in the hashfile;
* Consider reconsidering using a hashfile: what's the point of using hashfiles when you can't load the hash file completely in memory.

For more information on hash files do a search on this site, it has been covered numerous times.

Ogmios

[ray.wurlod]

STRONG ADVICE
Don't enable the 64BIT_FILES configuration parameter. This sets ALL your hashed files to >2GB capability. You don't need it.

The syntax for creating hashed files with >2GB capacity (that is, with 64-bit internal pointers) is to use the appropriate option:
64BIT with the CREATE.FILE or CREATE TABLE verb
-64bit with the mkdbfile utility

The Hashed File Calculator lets you choose 32-bit or 64-bit via option buttons, and generates the appropriate commands.

[ray.wurlod]

An existing hashed file, provided there is a VOC entry pointing to it, can be converted to 64-bit internal addressing with the following command.

Code: Select all

RESIZE hashedfilename * * * 64BIT