Enable hash file cache sharing

[Thomas.Nel]

Hi PPL,

I've need someone to please set me straight on Hash file cache sharing as I have been getting conflicting answers.

The option to "Enable hash file cache sharing" in a server job's properties means that a memory resident hash file will only be available to stages in that job and multiple invocations of that same job. Is my understanding of that correct

If that is the case, does that mean that when multiple DIFFERENT jobs running at same time referring to the same hash file, will each create a copy of the hash in memory This seems to be the case when I take a look at the memory usage of my machine when these jobs are executing

If that is the case is there any way to make multiple different jobs use the same in memory hash file

Thanks,
Tom.

[peternolan9]

All,
for what it's worth. In INFA the hash file equivalent is a 'lookup' which is simply a sharable icon that contains an SQL statement. The data can be read from the database or from memory. In memory I believe it is binary searched.

If read from memory any job using that lookup will share the one copy of the file. You know this to be the case because the log for the job issues a message to say that the lookup is already running in memory and will be shared. Also, I don't know if INFA is now 64 bit but it seems like the lookups are only limited by size of memory of the machine. I've never had a really big machine to try it on.

Hash files have always been a huge advantage for DS because there is so much you can do with them. I'd be surprised if you can't share them in cache between multiple jobs....

Hi PPL,


The option to "Enable hash file cache sharing" in a server job's properties means that a memory resident hash file will only be available to stages in that job and multiple invocations of that same job. Is my understanding of that correct

Thanks,
Tom.

[kcbland]

With release 5.1 a new daemon was added, the "Hash Cache Daemon". You can run a command that requests the daemon to put a hash file into a shared cache. All jobs that use preload to memory for a hash file will use the hash cache daemon data first before loading its own copy. When you're done, you "de-cache" the file.

So to recap, create your hash file and populate it. Then, run the command that requests the daemon to cache the hash file. All jobs using preload to memory on that hash file will use the cached copy. When you're done processing for the day, de-cache the copy.

Now, a hash file used repeatedly within the same job can preload one copy and share it among stages, but this is different. Please read your manuals on the Start button, as the disk caching guide spells it allll out in the full gory/glory details.

[ariear]

viewtopic.php?t=84710&start=0&postdays= ... highlight=