Hash File Limit

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
hemant
Participant
Posts: 67
Joined: Mon Dec 15, 2003 6:43 am

Hash File Limit

Post by hemant »

Hi!
Can i increase the limit of hash file size cause after 2 GB my job get hanged and i couldn't procure desired result .
I am giving a separate space to it specific for my application.

Kindly suggest .


Regards:

Hemant Krishnatrey
roy
Participant
Posts: 2598
Joined: Wed Jul 30, 2003 2:05 am
Location: Israel

Post by roy »

Hi,
This was covered here before use search.
in general you can using the HFC unsupported utility get the syntax to manually create the file.
you can't get it using the regular create option in the hash stage.

IHTH,
Roy R.
Time is money but when you don't have money time is all you can afford.

Search before posting:)

Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Image
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

If the hashed file already exists, and is not corrupted, you can use the resize utility to switch it to 64-bit addressing.
Use the following commands at the TCL prompt or in the Administrator client command window.

Use SETFILE to create a VOC pointer to the hashed file.

Code: Select all

SETFILE pathname hashedfile OVERWRITING
The OVERWRITING keyword is optional.

Code: Select all

RESIZE hashedfile * * * 64BIT
You will get a message indicating that the date/time stamp in the file header has been modified. This message is terminated by an exclamation point (!) but really is just an informative message. Don't worry about it.

Enough space for a second copy of the hashed file is required for this operation. If this does not exist where the project or the hashed file reside, add a USING clause referring to a directory where there is plenty of space (that is, at least as much as the hashed file currently occupies).

Code: Select all

RESIZE hashedfile * * * 64BIT USING pathname
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
hemant
Participant
Posts: 67
Joined: Mon Dec 15, 2003 6:43 am

Error in executing command

Post by hemant »

ray.wurlod wrote:If the hashed file already exists, and is not corrupted, you can use the resize utility to switch it to 64-bit addressing.
Use the following commands at the TCL prompt or in the Administrator client command window.

Use SETFILE to create a VOC pointer to the hashed file.

Code: Select all

SETFILE pathname hashedfile OVERWRITING
The OVERWRITING keyword is optional.

Code: Select all

RESIZE hashedfile * * * 64BIT
You will get a message indicating that the date/time stamp in the file header has been modified. This message is terminated by an exclamation point (!) but really is just an informative message. Don't worry about it.

Enough space for a second copy of the hashed file is required for this operation. If this does not exist where the project or the hashed file reside, add a USING clause referring to a directory where there is plenty of space (that is, at least as much as the hashed file currently occupies).

Code: Select all

RESIZE hashedfile * * * 64BIT USING pathname
Hi! Ray

There is two entities created when i run my job for creating an hash file > 2 GB
1. D_H_CUST_SUM_ID - Directory
2. H_CUST_SUM_ID - File

now when i executed yur given code
SETFILE /u50/PRODDSS/data2/ffamily/Hashfile H_CUST_SUM_ID OVERWRITING
The command gives some warning.
Secondly when i execute
RESIZE H_CUST_SUM_ID* * * 64BIT
The command gives an error as
RESIZE : Invalid file name,sizing parameter, or option on command line .
Kindly suggest .

Regards
Hemant
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Re: Error in executing command

Post by chulett »

hemant wrote:Secondly when i execute
RESIZE H_CUST_SUM_ID* * * 64BIT
The command gives an error as
RESIZE : Invalid file name,sizing parameter, or option on command line .
I don't believe you can use this command on an external (i.e. 'pathed') hash file, which you seem to have. You may need to add a VOC entry for it, at least temporarily, before running this command.
-craig

"You can never have too many knives" -- Logan Nine Fingers
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

Search the forum for mkdbfile command. You can manually created the hash file with the 64bit keyword so that you don't have to mess around with all of these afterwards commands.
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

That it gave "some warning" doesn't help to diagnose what happened.
If your SETFILE command failed, then you don't have the VOC pointer that RESIZE needs.

If you post what the warning message was, we will be able to be more helpful.

Given the names of the file system objects that are out there, I expect your command should have been

Code: Select all

SETFILE /u50/PRODDSS/data2/ffamily/Hashfile/H_CUST_SUM_ID  H_CUST_SUM_ID OVERWRITING 
Also, note that there is a space between the hashed file name and the first asterisk in the RESIZE command. Yours did not have one.

Code: Select all

RESIZE H_CUST_SUM_ID * * * 64BIT USING /lots_of_space_dir
While it is true that you could create a 64-bit hashed file from scratch, I suspect you would prefer not to have to move the approaching 2GB of data from one to the other.

Incidentally, I think you'll find that H_CUST_SUM_ID is a directory and that D_H_CUST_SUM_ID is a file.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
hemant
Participant
Posts: 67
Joined: Mon Dec 15, 2003 6:43 am

Post by hemant »

Hi!

Thanks Ray ! Its been very great of you for solving our problem both explicitly and instantaneously .
We are still working on it hope we will achieve the desired output soon. amidst can i have the exhaustive
list of the commands at one place if u can provide i will be greatful .

Thanks & Regards
Hemant Krishnatrey
Post Reply