I have a following scenario which I would like to get an expert openion before spending time on it. Please provide osme help in this regard
Q1 . Can I use the universe database provided by the Datastage for my local calculation and some storage to ease the joins?
Q2. Is there any storage limit for this uuniverse datbase? In my scenario i may need to store 30 million records maximum?
Q3. I am in a shared enviorment, if I use this universe database, will it impact prfomance for other guys anyway?
Q4. Where is the universe database stored.? I mean is it in $DSHOME?
universe database
Moderators: chulett, rschirm, roy
-
- Charter Member
- Posts: 299
- Joined: Wed Nov 13, 2002 5:38 pm
- Location: USA
All the "Experts" are laughing and playing at IOD with their fancy two drink minimums and dancing girls of easy virture so I'll answer your questions. I think you are mis-understanding the purpose of the universe database(s). It is a multi-valued database whose sole purpose should be to support the "app" called DataStage. There is no way I would use it for ETL.
Q1:There is a thing called hashed files that is appropriate for the volatile, non-permanent storage of data to aid you in your ETL. As a best practice these should not be defined in the account, rather they should be defined and stored elsewhere. A simple reason for this is that if you store them in the uv account and fill up the disk in that location, you'll risk corrupting the DataStage project.
Q2: The limit is the limit of your OS for file size
Q3: Don't do it
Q4: Each DataStage Project is a separate instance of the universe database.
Q1:There is a thing called hashed files that is appropriate for the volatile, non-permanent storage of data to aid you in your ETL. As a best practice these should not be defined in the account, rather they should be defined and stored elsewhere. A simple reason for this is that if you store them in the uv account and fill up the disk in that location, you'll risk corrupting the DataStage project.
Q2: The limit is the limit of your OS for file size
Q3: Don't do it
Q4: Each DataStage Project is a separate instance of the universe database.
-
- Premium Member
- Posts: 238
- Joined: Fri Jul 25, 2008 8:55 am
Incorrect! But thanks for playing, here's a lovely consolation prize. Now if you'll please exit to the right of the stage...myukassign wrote:i well aware of hash files, but the thing is that my data size is more than 2 gb. So i am not able to store in hash.
![Wink :wink:](./images/smilies/icon_wink.gif)
While that may be the limit for 32bit hashed files, no such limit exists for 64BIT hashed files that's worth mentioning. Search the forums for "64BIT" to find all of the exciting conversations on the subject.
ps. Not quite all of the "experts", throbinson.
![Crying or Very sad :cry:](./images/smilies/icon_cry.gif)
Last edited by chulett on Wed Oct 29, 2008 9:10 am, edited 1 time in total.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
There IS a limit with 64-bit hashed files, it's probably more than the operating system file size limit. The theoretical limit is 19 million TB (approximately) but that all has to be on the one file system.
Please note also that UniVerse is a separate database product. The DataStage Engine has not been UniVerse since version 5.2 of DataStage (version 9.6 of UniVerse). However, they remain quite similar.
Expanding the answer to your Q3, everything you do will impact other users of the server. That's simple supply and demand.
Q4: hashed files can be stored anywhere on the file system; they do not need to be in the DataStage project.
Please note also that UniVerse is a separate database product. The DataStage Engine has not been UniVerse since version 5.2 of DataStage (version 9.6 of UniVerse). However, they remain quite similar.
Expanding the answer to your Q3, everything you do will impact other users of the server. That's simple supply and demand.
Q4: hashed files can be stored anywhere on the file system; they do not need to be in the DataStage project.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Premium Member
- Posts: 238
- Joined: Fri Jul 25, 2008 8:55 am