Hi,
In our process,First job will load approx. 4-5 GB of data in a dataset.And in the second job, i have to use the above data set as source.
Will Dataset handle that much data or any limitations to that ?
Thanks in Advance,
Kishore
Dataset Limitations
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 222
- Joined: Tue Aug 30, 2005 2:07 am
- Location: pune
- Contact:
Hi,
The amount of data that can be stored in each destination data file is limited by the characteristics of the file system and the amount of free disk space available.
In your case, there is no problem until and unless, if there is an issue with space. You can access that Dataset as a source in other job also.
The amount of data that can be stored in each destination data file is limited by the characteristics of the file system and the amount of free disk space available.
In your case, there is no problem until and unless, if there is an issue with space. You can access that Dataset as a source in other job also.
NageshSunkoji
If you know anything SHARE it.............
If you Don't know anything LEARN it...............
If you know anything SHARE it.............
If you Don't know anything LEARN it...............
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Even if your operating system limits file sizes to 2GB there is no effective limit. Data Sets' data will be stored in multiple files per partition. So, if you had 10000 files per partition and 1000 partitions, that's 2.0E9 GB of data. And you're allowed more than that; I only chose those numbers for illustrative purposes.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.