Problem in Hash file

MOHAMMAD.ISSAQ · Post by **MOHAMMAD.ISSAQ** » Fri Feb 22, 2008 12:22 am

I've one sequential file having one column of around 20000 length.
I want to extract data from seq. file to a hashed file(Direct mapping).
The single column itself i've set as akey in hashed file.

My job is working fine but it's not fetching any rows from input.
My source file has 1000 rows and each row has around 10000 charcters.
If i'm selecting only a few characters i'm getting the output.
E.g:
Hash.SingleRow->Input.SingleRow[1,100] /* 831 rows extracted */
Hash.SingleRow->Input.SingleRow /* 0 rows extracted */
Do i need to change the attributes of the hash file?
If it' so then please tell me what attributes i need to change and why?

Please help me out..

ray.wurlod · Post by **ray.wurlod** » Fri Feb 22, 2008 2:26 am

The correct terminology is hashed file, not "hash" file.

Are there any warnings in your job log?

What is the MAXKEYLENGTH setting for hashed files at your installation? (The default is 768, which would definitely rule out 20000-character keys.)

MOHAMMAD.ISSAQ · Post by **MOHAMMAD.ISSAQ** » Fri Feb 22, 2008 3:23 am

I'm extremely sorry ray for doing the same mistake again.
Thanks for the correction again.

Yes i'm getting warnings as:
1) "IP_DB..Hashed_File_4.DSLink5:ds_uvput()-record ID + 0000+006082312..........." (Some data is present).
2) " 941 DataStage job Phantom 29072
Program "JOB.1623772262.DT.1466257407.TRANS1": Line 46, Attempted WRITE with record ID larger than file/table maximum "

I don't know how to find the MAXKEYLENGTH for a hashed file.
Can you please tell me where to check and how much length i should give for my job?

Please help me out...

ArndW · Post by **ArndW** » Fri Feb 22, 2008 3:46 am

cd to your $DSHOME directory, then "cat uvconfig | grep MAXKEY" to get the current value.

MOHAMMAD.ISSAQ · Post by **MOHAMMAD.ISSAQ** » Fri Feb 22, 2008 3:56 am

Thanks Arndw, It's 768 characters. My key column length is 20000 characters.

Will i able to change the MaxKey length?

If it's so then please tell me how to do it?

ArndW · Post by **ArndW** » Fri Feb 22, 2008 4:58 am

The documentation will tell you how to do this, but the steps are relatively simple:

1. Edit the "uvconfig" file
2. while attached to $DSHOME, execute "bin/uvregen" to parse the text file into the one that is actually used by DataStage
3. bounce DataStage

MOHAMMAD.ISSAQ · Post by **MOHAMMAD.ISSAQ** » Fri Feb 22, 2008 5:35 am

Thanks Arndw

chulett · Post by **chulett** » Fri Feb 22, 2008 7:30 am

Are you sure this is good advice? Up the MAXKEY value to 20000 ?

Usually I would have expected to see dire warnings about the consequences / side effects of this, or questions on why someone would really need a hashed key that large. [shrug] Whatever.

ArndW · Post by **ArndW** » Fri Feb 22, 2008 7:47 am

Craig - that is what I though as well ... the "shrug" part, I mean. Some performance degradation in all hashed files can be expected as well as growth in file sizes.

ray.wurlod · Post by **ray.wurlod** » Fri Feb 22, 2008 7:48 am

The answer is no. The maximum permitted value of MAXKEYSIZE (my error earlier, it's not MAXKEYLENGTH) is nowhere near 20000. From the manual:

Administering UniVerse wrote:Specifies the maximum number of characters for a primary key. It must be any multiple of 64 between 256 and 2048. The full record ID is stored in the record lock entry. The default value is 255. Do not change MAXKEYSIZE from the default value without understanding its effect on the record lock table entries.

To which I'd add you also need to understand its effect on group size (separation) in hashed files.

The UniVerse default is 255, the DataStage default is 768.

As Craig notes, a key size of 20000 in any database is generally a Bad Idea.

MOHAMMAD.ISSAQ · Post by **MOHAMMAD.ISSAQ** » Fri Feb 22, 2008 8:22 am

Thanks for the correction.

That means i can't reply on hashed file if the key size is greater than 768.

chulett · Post by **chulett** » Fri Feb 22, 2008 8:36 am

2048. Tell us why you think you would need a key size that large. What are you trying to accomplish?

MOHAMMAD.ISSAQ · Post by **MOHAMMAD.ISSAQ** » Fri Feb 22, 2008 8:58 am

I posted one thread as "Joining the hash file".You can check it here

"viewtopic.php?t=116624"

In that source file is a sequential file and it's has one key k2 and hashed file as key k1 and k2.

I thought of extracting data from sequential file and putting in a hashed file.
So my source now will become Hashed file with key k2.

Instead of extracting data into hashed file i'm loading into a sequential file with keys k1 and k2.Now my source file has keys k1 and k2 and hashed file has key k2.And i'm creating one dummy hashed file with key k1.

Altogether i have one source file with keys k1 and k2,two hashed files with keys k1 and k2 each.Now i'll join the two hashed files with the source file.In my earlier post i'm not able to join so i reversed the process.