CREATE.FILE usage for 64 bit hashed files

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

No, in order to use the SQL DML you need a DICTtionary. But that does not mean that each file has to have it's own DICTionary file, you can point the VOC entry to a common DICTionary file if you wish.

When using a hashed file using SQL type commands there are 3 things that need to be in place, a VOC entry, a data file and a dictionary file. The VOC entry has 3 lines, the first must begin with an "F", the second is a path to the data file (type 30 files point to the directory containing the 3 component files), and line 3 is a path to the DICTionary file (usually called D_{filename} but not restricted to that.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

You can create the hashed file and its dictionary with a single CREATE TABLE command.

Code: Select all

CREATE TABLE hashedfilename
DATA pathname_of_hashedfilename
DICT pathname_of_D_hashedfilename
(
    column_definitions...
) ;

RESIZE hashedfilename * * * 64BIT
(You can probably include the 64BIT option in the CREATE TABLE statement - I can't check at the moment.)
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
clarcombe
Premium Member
Premium Member
Posts: 515
Joined: Wed Jun 08, 2005 9:54 am
Location: Europe

Post by clarcombe »

You can create the hashed file and its dictionary with a single CREATE TABLE command.
For future reference how would this be done in a job? From a series of DSExecute commands via a routine?

Finally, I rang Ascential and they said I didn't need a dictionary file if I specify a directory under Use Directory Path for the HF, so I have created an after job routine which creates the HF using mkdbfile.

The problem is if the file already exists the mkdbfile fails. This surprised me as I thought this would be overwritten. This may cause issues later as the file may have been originally created as a 32 bit HF

Am I going to have to test for the existence of the DATA.30 file before running the mkdbfile ?

Is there a command to remove the HF beforehand (unmkdbfile ?)
Colin Larcombe
-------------------

Certified IBM Infosphere Datastage Developer
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

clarcombe wrote:Is there a command to remove the HF beforehand (unmkdbfile ?)
Yes, unmkdbfile is also known as 'rm'. :wink:

Or more specifically, for the entire directory and its contents:

rm -rf <directory>
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

To do the CREATE TABLE method, you would use a UV stage and edit the DDL that it creates.

To delete a hashed file (not a table) created in a directory you have to remove not only the data portion but also the dictionary portion.

Code: Select all

rm -rf hashedfilename ; rm -f D_hashedfilename
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
clarcombe
Premium Member
Premium Member
Posts: 515
Joined: Wed Jun 08, 2005 9:54 am
Location: Europe

Post by clarcombe »

Gentlemen,

Thanks for your contributions.

I never thought of using a UV stage to create a hashed file but I will now add that to the toolbox
Colin Larcombe
-------------------

Certified IBM Infosphere Datastage Developer
clarcombe
Premium Member
Premium Member
Posts: 515
Joined: Wed Jun 08, 2005 9:54 am
Location: Europe

Post by clarcombe »

Gentlemen,

Thanks for your contributions.

I never thought of using a UV stage to create a hashed file but I will now add that to the toolbox
Colin Larcombe
-------------------

Certified IBM Infosphere Datastage Developer
CharlesNagy
Premium Member
Premium Member
Posts: 66
Joined: Mon Feb 21, 2005 10:40 am
Location: Paris

Re Flag in UVconfig

Post by CharlesNagy »

Looking in UVconfig, there is a flag

64BIT_FILES 0

The comments there say that if you set this to 1 then 64 Bit files will be created. It also says that it is only valid on 64 bit capable platforms. I had taken this to mean that you needed a 64 bit operating system for this, but apparently not.

Still, it is a bit confusing.... :roll:
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

And if you search the forums for that option, you'll see it comes with HUGE caveats, so make sure you understand exactly what it does before you enable it!
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

NEVER enable 64BIT_FILES !!!!

This means that every hashed file you create will have (wasteful) 64-bit internal pointers, unless you specify the 32BIT option when creating them.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

That would be the huge caveat I mentioned. :lol:
-craig

"You can never have too many knives" -- Logan Nine Fingers
CharlesNagy
Premium Member
Premium Member
Posts: 66
Joined: Mon Feb 21, 2005 10:40 am
Location: Paris

Post by CharlesNagy »

Thanks for clearing that up guys. In future I will use the 64BIT option when using CREATE.FILE, however, I have searched the Universe documentation, but there does not seem to be a 64BIt option for CREATE TABLE... So I will have to issue a RESIZE command to create a 64 Bit TABLE.

Also, I did a test CREATE.FILE Type 18 Mod 1995 Sep 2, to create a 32 Bit file, and issued a resize command to 64BIt, then did an ANALYZE.FILE on it, but the file appeared to be exactly the same size before and after (2043904 Bytes). I had expect it to be somewhat larger due to the above mentioned internal pointers...

Is there a command that will tell you wether a file is 32 BIT or 64 BIT?
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Yup, and Ray has laid it all out in the past, let me see if I can find it... ah:

viewtopic.php?t=86639
-craig

"You can never have too many knives" -- Logan Nine Fingers
CharlesNagy
Premium Member
Premium Member
Posts: 66
Joined: Mon Feb 21, 2005 10:40 am
Location: Paris

Post by CharlesNagy »

chulett wrote:Yup, and Ray has laid it all out in the past, let me see if I can find it... ah:

viewtopic.php?t=86639
Wow, "UVFIXFILE filename VLEVEL 0 TRACE 1", and if it doesn't work - then it is 64 BIT, thats so obvious, why didn't I think of that? :wink:

Thanks a bunch, I would never have discovered that....
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

:lol:

I prefer the 'check the magic number' bit myself as it is a little bit more intuitive, but either way works.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply