Maximum Records can be stored in a DataSet

uegodawa · Post by **uegodawa** » Tue Jul 31, 2007 9:13 am

Hi
Does any one knows about the storage limitation about a DataSet. I've designed a job for a Dataset with 398 columns and 1,000,000 source records. When I executed the job I can see only 521327 records stored in DataSet with 4,270,718,976 bytes.
So, What happened to rest of my source records ?
Is this because of a limitation of O/S or DataSet stage itself ?
Thanks
uegodawa

surendra.v · Post by **surendra.v** » Tue Jul 31, 2007 3:11 pm

How many configuration nodes are being used to create this data set.

Based on the file size in bytes, it appears to me like it is running under 2 node configuration and created two 2GB files.

ray.wurlod · Post by **ray.wurlod** » Tue Jul 31, 2007 4:34 pm

How are you determining the number of records in the Data Set?