Problem in Hash file
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 78
- Joined: Fri Mar 02, 2007 4:54 am
- Location: CHENNAI
Problem in Hash file
I've one sequential file having one column of around 20000 length.
I want to extract data from seq. file to a hashed file(Direct mapping).
The single column itself i've set as akey in hashed file.
My job is working fine but it's not fetching any rows from input.
My source file has 1000 rows and each row has around 10000 charcters.
If i'm selecting only a few characters i'm getting the output.
E.g:
Hash.SingleRow->Input.SingleRow[1,100] /* 831 rows extracted */
Hash.SingleRow->Input.SingleRow /* 0 rows extracted */
Do i need to change the attributes of the hash file?
If it' so then please tell me what attributes i need to change and why?
Please help me out..
I want to extract data from seq. file to a hashed file(Direct mapping).
The single column itself i've set as akey in hashed file.
My job is working fine but it's not fetching any rows from input.
My source file has 1000 rows and each row has around 10000 charcters.
If i'm selecting only a few characters i'm getting the output.
E.g:
Hash.SingleRow->Input.SingleRow[1,100] /* 831 rows extracted */
Hash.SingleRow->Input.SingleRow /* 0 rows extracted */
Do i need to change the attributes of the hash file?
If it' so then please tell me what attributes i need to change and why?
Please help me out..
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
The correct terminology is hashed file, not "hash" file.
Are there any warnings in your job log?
What is the MAXKEYLENGTH setting for hashed files at your installation? (The default is 768, which would definitely rule out 20000-character keys.)
Are there any warnings in your job log?
What is the MAXKEYLENGTH setting for hashed files at your installation? (The default is 768, which would definitely rule out 20000-character keys.)
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 78
- Joined: Fri Mar 02, 2007 4:54 am
- Location: CHENNAI
I'm extremely sorry ray for doing the same mistake again.
Thanks for the correction again.
Yes i'm getting warnings as:
1) "IP_DB..Hashed_File_4.DSLink5:ds_uvput()-record ID + 0000+006082312..........." (Some data is present).
2) " 941 DataStage job Phantom 29072
Program "JOB.1623772262.DT.1466257407.TRANS1": Line 46, Attempted WRITE with record ID larger than file/table maximum "
I don't know how to find the MAXKEYLENGTH for a hashed file.
Can you please tell me where to check and how much length i should give for my job?
Please help me out...
Thanks for the correction again.
Yes i'm getting warnings as:
1) "IP_DB..Hashed_File_4.DSLink5:ds_uvput()-record ID + 0000+006082312..........." (Some data is present).
2) " 941 DataStage job Phantom 29072
Program "JOB.1623772262.DT.1466257407.TRANS1": Line 46, Attempted WRITE with record ID larger than file/table maximum "
I don't know how to find the MAXKEYLENGTH for a hashed file.
Can you please tell me where to check and how much length i should give for my job?
Please help me out...
cd to your $DSHOME directory, then "cat uvconfig | grep MAXKEY" to get the current value.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 78
- Joined: Fri Mar 02, 2007 4:54 am
- Location: CHENNAI
The documentation will tell you how to do this, but the steps are relatively simple:
1. Edit the "uvconfig" file
2. while attached to $DSHOME, execute "bin/uvregen" to parse the text file into the one that is actually used by DataStage
3. bounce DataStage
1. Edit the "uvconfig" file
2. while attached to $DSHOME, execute "bin/uvregen" to parse the text file into the one that is actually used by DataStage
3. bounce DataStage
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 78
- Joined: Fri Mar 02, 2007 4:54 am
- Location: CHENNAI
Are you sure this is good advice? Up the MAXKEY value to 20000 ?
Usually I would have expected to see dire warnings about the consequences / side effects of this, or questions on why someone would really need a hashed key that large. [shrug] Whatever.
Usually I would have expected to see dire warnings about the consequences / side effects of this, or questions on why someone would really need a hashed key that large. [shrug] Whatever.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
Craig - that is what I though as well ... the "shrug" part, I mean. Some performance degradation in all hashed files can be expected as well as growth in file sizes.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
The answer is no. The maximum permitted value of MAXKEYSIZE (my error earlier, it's not MAXKEYLENGTH) is nowhere near 20000. From the manual:
The UniVerse default is 255, the DataStage default is 768.
As Craig notes, a key size of 20000 in any database is generally a Bad Idea.
To which I'd add you also need to understand its effect on group size (separation) in hashed files.Administering UniVerse wrote:Specifies the maximum number of characters for a primary key. It must be any multiple of 64 between 256 and 2048. The full record ID is stored in the record lock entry. The default value is 255. Do not change MAXKEYSIZE from the default value without understanding its effect on the record lock table entries.
The UniVerse default is 255, the DataStage default is 768.
As Craig notes, a key size of 20000 in any database is generally a Bad Idea.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 78
- Joined: Fri Mar 02, 2007 4:54 am
- Location: CHENNAI
-
- Participant
- Posts: 78
- Joined: Fri Mar 02, 2007 4:54 am
- Location: CHENNAI
I posted one thread as "Joining the hash file".You can check it here
"viewtopic.php?t=116624"
In that source file is a sequential file and it's has one key k2 and hashed file as key k1 and k2.
I thought of extracting data from sequential file and putting in a hashed file.
So my source now will become Hashed file with key k2.
Instead of extracting data into hashed file i'm loading into a sequential file with keys k1 and k2.Now my source file has keys k1 and k2 and hashed file has key k2.And i'm creating one dummy hashed file with key k1.
Altogether i have one source file with keys k1 and k2,two hashed files with keys k1 and k2 each.Now i'll join the two hashed files with the source file.In my earlier post i'm not able to join so i reversed the process.
"viewtopic.php?t=116624"
In that source file is a sequential file and it's has one key k2 and hashed file as key k1 and k2.
I thought of extracting data from sequential file and putting in a hashed file.
So my source now will become Hashed file with key k2.
Instead of extracting data into hashed file i'm loading into a sequential file with keys k1 and k2.Now my source file has keys k1 and k2 and hashed file has key k2.And i'm creating one dummy hashed file with key k1.
Altogether i have one source file with keys k1 and k2,two hashed files with keys k1 and k2 each.Now i'll join the two hashed files with the source file.In my earlier post i'm not able to join so i reversed the process.