CREATE.FILE usage for 64 bit hashed files

clarcombe · Post by **clarcombe** » Wed Dec 05, 2007 9:30 am

I have to create a group of 64 bit hashed files via a script that I can pass to the production team

I have no access to the server via telnet (so dvsh is out) so I can create them via a job or an excel script (and paste the commands into DS Administrator)

I can do the following

SETFILE /applis/psoft/EPMV5_APPL/DataStage/Projects/dsfichseq/FDM84_90_PERF/KOMODO/LOOKUP/HF_T84_PS_DEMAND_INF_INV HF_T84_PS_DEMAND_INF_INV OVERWRITING
RESIZE HF_T84_PS_DEMAND_INF_INV * * * 64BIT

but I can't find the command to create the file first off i.e. CREATE.FILE. Nor have I found the syntax in the manuals or on the board.

Can anyone help ?

TIA

PS.Note that the HF is NOT being created in the project directory

PPS. I tried CREATE.FILE /applis/psoft/EPMV5_APPL/DataStage/Projects/dsfichseq/FDM84_90_PERF/KOMODO/LOOKUP/HF_T84_PS_DEMAND_INF_INV but it didnt like it

ArndW · Post by **ArndW** » Wed Dec 05, 2007 10:21 am

The UNIX command you are looking for is mkdbfile; you can always call that from a job or sequence using either the DS/BASIC "EXECUTE" command or the DSExecute function
The command will not create a VOC entry for the file, and in order to create the DICTionary file you will have to call it again with the appropriate settings.

I don't have DS on this system, but the command CREATE.FILE will let you specify a remote path for the data file. You can get the syntax by typing "HELP CREATE.FILE" from the UVSH or Administrator command line.

Code: Select all

mkdbfile /your/file/path/and/name 30 999999 4 20 20 80 1628  -64BIT

This will create a 64 bit type 30 file with a group size of 4, 20% merge, 80% split and group size of 1628 {which I seem to remember are the defaults, it might be 8 for the group size}

CharlesNagy · Post by **CharlesNagy** » Wed Dec 05, 2007 10:25 am

clarcombe wrote:I have to create a group of 64 bit hashed files via a script that I can pass to the production team

I have no access to the server via telnet (so dvsh is out) so I can create them via a job or an excel script (and paste the commands into DS Administrator)

I can do the following

SETFILE /applis/psoft/EPMV5_APPL/DataStage/Projects/dsfichseq/FDM84_90_PERF/KOMODO/LOOKUP/HF_T84_PS_DEMAND_INF_INV HF_T84_PS_DEMAND_INF_INV OVERWRITING
RESIZE HF_T84_PS_DEMAND_INF_INV * * * 64BIT

but I can't find the command to create the file first off i.e. CREATE.FILE. Nor have I found the syntax in the manuals or on the board.

Can anyone help ?

TIA

PS.Note that the HF is NOT being created in the project directory

PPS. I tried CREATE.FILE /applis/psoft/EPMV5_APPL/DataStage/Projects/dsfichseq/FDM84_90_PERF/KOMODO/LOOKUP/HF_T84_PS_DEMAND_INF_INV but it didnt like it

You could just do a job to create the Files. If instead of running the job, you select "validate", it will create the files for you without putting anything in them....

ray.wurlod · Post by **ray.wurlod** » Wed Dec 05, 2007 2:22 pm

The Hashed File Calculator (on your DataStage CD, in the unsupported utilities folder) will generate either CREATE.FILE or mkdbfile command.

clarcombe · Post by **clarcombe** » Thu Dec 06, 2007 4:43 am

Given that I don't have access via telnet to the server I thought I would go with the CREATE.FILE via HFC

So I ran CREATE.FILE /applis/psoft/FSCM4_APPL/CBLKOM/Datastage/Projects/FDM84_90_CBLKOM/HF_T84_PS_SHIP_INF_INV DYNAMIC MINIMUM.MODULUS 533353 LARGE.RECORD 682 64BIT

from the command prompt but it failed. It appeared not to like the /

So I then ran

CREATE.FILE \applis\psoft\FSCM4_APPL\CBLKOM\Datastage\Projects\FDM84_90_CBLKOM\HF_TEST DYNAMIC MINIMUM.MODULUS 533353 LARGE.RECORD 682 64BIT

but received the error
Invalid Type entered the file type must be an integer in the range 1-19 or 30

Any pointers ?

TIA

ArndW · Post by **ArndW** » Thu Dec 06, 2007 5:12 am

I'm not at a DataStage box so cannot confirm your CREATE.FILE syntax, in my opinion you would be creating a local hashed file called "/applis/psoft/FSCM4_APPL/CBLKOM/Datastage/Projects/FDM84_90_CBLKOM/HF_T84_PS_SHIP_INF_INV " and that would cause issues when creating the actual files since they contain path delimiters.

Have you checked the CREATE.FILE syntax?

clarcombe · Post by **clarcombe** » Thu Dec 06, 2007 5:46 am

Arnd

The create file syntax I copied from the HFC results. I am surprised that I am unable to copy the command directly from HFC but the text is greyed out.

Having failed with CREATE.FILE, I tried mkdbfile on our development box and got the following error

Code: Select all

mkdbfile /applis/psoft/FSCM4_APPL/CBLKOM/Datastage/Projects/FDM84_90_CBLKOM/KOMO
DO/LOOKUP/HF_TEST 30 999999 4 20 20 80 1628 -64BIT

Code: Select all

mkdbfile: cannot create file /applis/psoft/FSCM4_APPL/CBLKOM/Datastage/Projects/
FDM84_90_CBLKOM/KOMODO/LOOKUP/HF_TEST

I also tried to run this via a Before/after function but got a 127 error. I assume it was because from the routine I wasn't in the $DSHOME/bin directory.

How do I ensure that the mkdbfile works from a routine. Must I put $DSHOME/bin before the command ?
Will $DSHOME be recognised by the routine ?

TIA

ArndW · Post by **ArndW** » Thu Dec 06, 2007 8:40 am

What does "ls applis/psoft/FSCM4_APPL/CBLKOM/Datastage/Projects/
FDM84_90_CBLKOM/KOMODO/LOOKUP/HF_TEST" and/or "touch applis/psoft/FSCM4_APPL/CBLKOM/Datastage/Projects/
FDM84_90_CBLKOM/KOMODO/LOOKUP/HF_TEST" return (remember to delete the file after the touch. Actually, instead of touch it really should be "mkdir")

clarcombe · Post by **clarcombe** » Thu Dec 06, 2007 8:45 am

I just tried changing to the HF directory and then running

$DSHOME/bin/mkdbfile HF_TEST 30 999999 4 20 20 1628 -64BIT and this appeared to work

I don't understand why the path before the HF_TEST didnt work previously.

However in the directory HF_TEST I find the files DATA.30 and OVER.30 but the D_HF_TEST file one level above has not been created.

Thus when I try and create the VOC entry using DSExecute("UV","SETFILE <path>/HF_TEST HF_TEST OVERWRITING") I get an error saying D_HF_TEST does not exist.

Any suggestions ?

clarcombe · Post by **clarcombe** » Thu Dec 06, 2007 9:18 am

Code: Select all

touch applis/psoft/FSCM4_APPL/CBLKOM/Datastage/Projects/ 
FDM84_90_CBLKOM/KOMODO/LOOKUP/HF_TEST"

touch: applis/psoft/FSCM4_APPL/CBLKOM/Datastage/Projects/FDM84_90_CBLKOM/KOMODO/LOOKUP/HF_TEST cannot create

I am assuming it is a rights issue and have asked the unix admin to investigate

clarcombe · Post by **clarcombe** » Thu Dec 06, 2007 9:46 am

Oops. Forgot the / at the beginning.

I can create files in this directory.

ArndW · Post by **ArndW** » Thu Dec 06, 2007 10:01 am

What about the mkdir - can you do that?

kcbland · Post by **kcbland** » Thu Dec 06, 2007 10:04 am

When you create a hashed file by hand, you'll need to create the dictionary (the D_ file) yourself. Setting up a VOC pointer is one thing, but without a valid dictionary you're going to have issues running UV/SQL against the file. The dictionary is the file that describes columns within the hashed file.

clarcombe · Post by **clarcombe** » Mon Dec 10, 2007 2:22 am

How do you create the D_File. Doesn't mkdbfile do that ?

clarcombe · Post by **clarcombe** » Mon Dec 10, 2007 3:18 am

As I am on a tight timescale, I think the easiest solution is to have a job which creates all the hashed files but using sql which returns no rows.

I will then create an after job routine, which runs the SETFILE and RESIZE commands on the new files.

As the sql for each job still has to be evaluated and this obviously takes time, is it possible to create a hashed file without the correct structure for example select x from dual in order to speed up the process ?

Thanks

DSXchange

CREATE.FILE usage for 64 bit hashed files

CREATE.FILE usage for 64 bit hashed files

Re: CREATE.FILE usage for 64 bit hashed files