Hash File Qtd Limits

ArndW · Post by **ArndW** » Thu Apr 13, 2006 5:23 am

I've never encountered that error before, but have created many hashed files simultaneously. I wonder if the error might be due to Windows drives. Is the "S" drive local to the DataStage server or on a mapped network drive? Could you test the hashed file creation on a local drive to see if the error is the same?

Loobian · Post by **Loobian** » Thu Apr 13, 2006 5:35 am

Hi ArndW,

At the time this error occurs, i've already created about 200 hash in the same directory/with various method (standard/not standard - altering the min modulus atribute with various values).

So i'm not suspecting of window drives... could it be cache limits?

Thanks

ArndW wrote:I've never encountered that error before, but have created many hashed files simultaneously. I wonder if the error might be due to Windows drives. Is the "S" drive local to the DataStage server or on ...

ArndW · Post by **ArndW** » Thu Apr 13, 2006 6:57 am

Creating files doesn't get cached in DataStage. Is your "S" drive on the network? Can you try a test job with all the file to a local drive?

Loobian · Post by **Loobian** » Thu Apr 13, 2006 7:22 am

Hi,

My "S" directory is on the network.

The job is about loading from one hash file at the same directory(!), then it performs a couple of lookups and output the result to both informix table (works fine) and another hash in that (same) directory (abort).

I find it hard to understand, my first guess was limit of total hash files was reached (if there's one)...

I've foward this issue to the network team so they can check the ODBC connections and drivers.

I'll provide feedback of the resolution when it's done (if there's one

).

Thanks

ArndW wrote:Creating files doesn't get cached in DataStage. Is your "S" drive on the network? Can you try a test job with all the file to a local drive? ...

kcbland · Post by **kcbland** » Thu Apr 13, 2006 7:33 am

That's your problem. You should ALWAYS use physical drives local to the server for hashed files.

ray.wurlod · Post by **ray.wurlod** » Thu Apr 13, 2006 3:34 pm

The system-wide limit on the number of simultaneously open dynamic (Type 30) hashed files is set by the T30FILE configuration parameter. If this value is 200, then you will fail to open (a job that creates it probably tries to open it) the 201st.

It is always a bad idea to perform lookups - of whatever kind - over a network. It's horribly inefficient. You have to put the key into a packet, send that to the other machine, wait for the other machine to do whatever it has to do, it then puts the resulting row into a packet and sends it back where you need to get it out of its packet. All this is unnecessary overhead, and the network latency doesn't help either.

crang · Post by **crang** » Mon Aug 14, 2006 12:09 pm

I am wondering if anyone found a solution to this issue. I have encountered it a couple of times now. We have tried to delete the temp file and rerun but the errors seems to screw something up in the project that will not get past the error. So the way we have gotten around it is by changing the name of the target file. The limit of 200 hash files was not a factor in any of the times this has happened.

ray.wurlod · Post by **ray.wurlod** » Mon Aug 14, 2006 10:22 pm

There might also be an operating system limit; the number of subdirectories in a directory. On Solaris it's 32765.

And it's "hashed" file, not "hash" file.

asitagrawal · Post by **asitagrawal** » Thu Apr 05, 2007 4:35 am

Hi,

Today, I also faced the same problem...
and the only soln turned out was name change...

I am yet to try for using a local disk and not a network disc...

Regards,
Asit

ray.wurlod · Post by **ray.wurlod** » Thu Apr 05, 2007 7:36 am

When considering the "limit is not an issue" remember that the Repository tables are mostly hashed files, and you get five of these per job.

sb_akarmarkar · Post by **sb_akarmarkar** » Fri Apr 06, 2007 5:38 am

Hi,

Are you using delete/create hashed file option while creating lookup some time this wont work. Better to go with clear hash file and then loads option

Thank You,
Anupam