Hash Size Limit

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
New2DS
Participant
Posts: 33
Joined: Sat Jun 26, 2004 9:58 am

Hash Size Limit

Post by New2DS »

Hi All,
We have a static hash file that we clear and re-create every month in our production cycle. I think the hash file limit is up to 2GB. I am wondering if we are close to the size of hash file or did we exceed already? What will happen if we exceed 2GB does the job fail or does it overwrite the existing data.

Below is the current file size we have.

1900496896 DATA.30
778000384 OVER.30

There are many jobs in our production that depend on this hash file. Any kind of help is appreciated.

Thanks,
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Closer to 2.2GB and writes will fail (and it may corrupt) when the limit is exceeded. Time to switch to -64BIT or investigate the "resize" command to convert an existing 32bit hashed file.
-craig

"You can never have too many knives" -- Logan Nine Fingers
New2DS
Participant
Posts: 33
Joined: Sat Jun 26, 2004 9:58 am

Post by New2DS »

chulett wrote:Closer to 2.2GB and writes will fail (and it may corrupt) when the limit is exceeded. Time to switch to -64BIT or investigate the "resize" command to convert an existing 32bit hashed file. ...
Thanks for quick response. Are we already at 2.2GB or the limit of hash file is 2.2GB? Do you add the size of both files data and over to calculate the size? How much risk are we in now?

Can you please explain or point me to a post where I know more about resize command.

Appreciate all your help.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

The limit is governed by the largest address (offest) within a file that can be addressed by a signed 32-bit integer. Hashed files use 32-bit pointers internally to link the records in each group. The RESIZE command can, among other things, change the size of those pointers to 64-bit. The largest address that can then be represented is approximately 19 million TB.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

The limit. And an exact search for "resize" should turn up quite a number of discussions on this topic.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply