We have a data set which has the following columns and data type
The structure of the data set is
Dset1
C1 char(40)
C2 smallint
C3 char(6)
C4 decimal(9,2)
C5 integer
C6 decimal(9,2)
C7 integer
C8 decimal(9,2)
C9 integer
This data set will have 150 million rows so how much will be the size of the Dataset.
Will the datastage can handle the size of the data set?
Amount of data a dataset can handle
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 31
- Joined: Tue Jul 13, 2004 5:26 am
- Location: chennai
- Contact:
Hi Lakshmi,
Pls look into this post and check if that helps
viewtopic.php?t=88791
According to the post they have a dataset which can hold 4.5GB of data.
If I were to do this I will load the dataset with different volumes and check how much size the Dataset is taking.
Rgds
--Rich
Pls look into this post and check if that helps
viewtopic.php?t=88791
According to the post they have a dataset which can hold 4.5GB of data.
I think Dataset Management Tool available in Datastage Manager can give you the details you want.Lakshmi wrote:This data set will have 150 million rows so how much will be the size of the Dataset.
If I were to do this I will load the dataset with different volumes and check how much size the Dataset is taking.
Rgds
--Rich
-
- Participant
- Posts: 25
- Joined: Thu Oct 02, 2003 8:57 am
Hi
I guess the maximum size of the dataset will be 150 million * the row width, plus a small amount for the 'file pointer', but the size of each of the actual files making up the dataset will depend on the number of nodes you are running.
We have datasets in excess of 20 Gb, but we run with 8 nodes so the size of each file is around 2.5 Gb.
Hope this helps
I guess the maximum size of the dataset will be 150 million * the row width, plus a small amount for the 'file pointer', but the size of each of the actual files making up the dataset will depend on the number of nodes you are running.
We have datasets in excess of 20 Gb, but we run with 8 nodes so the size of each file is around 2.5 Gb.
Hope this helps