Hi,
Actually there is a issue occured in our project...We want to store the cleanse data into intermedate stage..same as staging table ...initially we were extracting the data from the source and directly loading it into the target tables...but now we have to store the data into some intermediate stage( that is datasets) ....ideally the best option would be to store it into staging tables...but since there is lack of time....we decide to store the data into datasets....So is there any limit to use datasets, how many datasets which we can use in a particular project...is there any way to find out the free size in a particular project ??
The data volume is pretty huge and number of datasets which we are going to create will also be very huge...
If we use may datasets....it won't effect performance of other proejects on same datastage servers... I don't know if its really very foolish questions...but gurus ...please shade some light on the same....
Thanks in Advance!!
Number of datasets which we can use in a single project
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 334
- Joined: Fri Dec 01, 2006 5:17 am
- Location: Texas
Number of datasets which we can use in a single project
http://findingjobsindatastage.blogspot.com/
Theory is when you know all and nothing works. Practice is when all works and nobody knows why. In this case we have put together theory and practice: nothing works. and nobody knows why! (Albert Einstein)
Theory is when you know all and nothing works. Practice is when all works and nobody knows why. In this case we have put together theory and practice: nothing works. and nobody knows why! (Albert Einstein)
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
-
- Participant
- Posts: 334
- Joined: Fri Dec 01, 2006 5:17 am
- Location: Texas
Thanks for the reply!!chulett wrote:This has nothing to do with individual projects and everything to do with the amount of disk space you have allocated for datasets in your config files. There's no limit other than your hardware. ...
So if the number of datasets are increased, then it will hamper the job performance.. ..ok that's why now one of our job which is using lookup stage is getting aborted because of the space issue.( as we created around 127 datasets)
http://findingjobsindatastage.blogspot.com/
Theory is when you know all and nothing works. Practice is when all works and nobody knows why. In this case we have put together theory and practice: nothing works. and nobody knows why! (Albert Einstein)
Theory is when you know all and nothing works. Practice is when all works and nobody knows why. In this case we have put together theory and practice: nothing works. and nobody knows why! (Albert Einstein)
-
- Premium Member
- Posts: 1735
- Joined: Thu Mar 01, 2007 5:44 am
- Location: Troy, MI