Hash file question

rafik2k · Post by **rafik2k** » Thu Jun 07, 2007 4:19 am

One basic question related to hash file:
In which scenario should we set Pre-load file to Memory-Enabled.

I mean to say, is it wise to set above property when hash file is big?

In my case, in the sequence jobs are running parallel and some of the jobs using hash files as well.

Need to know best practice on same.

Thanks in advance.

chulett · Post by **chulett** » Thu Jun 07, 2007 6:39 am

If you 'know' it will always be too big to cache, don't set it - it will save a little startup time. Will it hurt to set it? No. It will try and you'll get a message it could not be cached.

ray.wurlod · Post by **ray.wurlod** » Fri Jun 08, 2007 1:22 am

Did you mean "hashed" file?

ArndW · Post by **ArndW** » Fri Jun 08, 2007 2:10 am

It will load the file to memory until it determines that it won't fit; the program isn't smart enough to detect from the file size that it cannot fit. This load causes IO and CPU and can take a long time on many systems, so it is best to do as Craig has already suggested and don't set the switch.

rameshrr3 · Post by **rameshrr3** » Fri Jun 08, 2007 2:21 am

The setting you mentioned will be used when you need the link private caching facility. More details are found in the Disk Caching Guide. Problem is , you rarely can predict the size of a hashed file that you will use.

rafik2k · Post by **rafik2k** » Fri Jun 08, 2007 7:55 am

rameshrr3 wrote:The setting you mentioned will be used when you need the link private caching facility. More details are found in the Disk Caching Guide. Problem is , you rarely can predict the size of a hashed file that you will use.

Ramesh: Can you please explain little bit more about link private caching facility. I don't have any idea on same.

rafik2k · Post by **rafik2k** » Fri Jun 08, 2007 8:07 am

Thanks everyone for valuable input!

Craig: Will it give error message or warning message if data can not fit in the memory, if I set Pre-load file to Memory-Enabled ?

chulett · Post by **chulett** » Fri Jun 08, 2007 8:44 am

No, it will just be informational. Here, one from my logs:

Code: Select all

Project:XXX
Job name:YYYY
Event #:3187
Timestamp:5/27/2007 9:56:28 AM
Event type:Info
User:dsuser
Message:
Job.Link.HashedFileName: Read caching disabled, file size exceeds cache size

rafik2k · Post by **rafik2k** » Mon Jun 11, 2007 2:09 am

chulett wrote:No, it will just be informational. Here, one from my logs:
Code: Select all
Project:XXX
Job name:YYYY
Event #:3187
Timestamp:5/27/2007 9:56:28 AM
Event type:Info
User:dsuser
Message:
Job.Link.HashedFileName: Read caching disabled, file size exceeds cache size

Thanks a lot Craig!

ray.wurlod · Post by **ray.wurlod** » Mon Jun 11, 2007 5:10 am

You might like to read the Technical Bulletin dsdskche.pdf to gain an in-depth knowledge of different possibilities for hashed file cache, such as private link cache, public link cache and shared cache.