Page 1 of 1
sequential file in 32 bit windows system larger than 2G byte
Posted: Wed Jan 31, 2007 8:40 pm
by xinhuang66
I want to use sequential file to store temporary data during my Datastage ETL process.
I am not sure how datastage deal with the sequential file the the file's size will be larger than 2 G bytes..
thanks
Posted: Wed Jan 31, 2007 9:35 pm
by chulett
Datastage doesn't care. Your operating system on the other hand, may. NTFS has no file size limit, by the way.
http://www.ntfs.com/ntfs_vs_fat.htm
Posted: Wed Jan 31, 2007 9:40 pm
by xinhuang66
So if file size exceed partition limitation, it will cause OS error, and also hang up Datastage job ?
For example in FAT32, if the file size will be larger than 4G bytes. what will happen ?
Posted: Wed Jan 31, 2007 10:40 pm
by ray.wurlod
DataStage job will abort when it receives the signal from the operating system that the file size limit has been exceeded.
Your data will be truncated at 4GB and may not be recoverable, as no end-of-file mark will have been written.
That's one of the reasons DataStage can only be installed on NTFS partitions.
If you choose to use FAT32 partitions, aware of their limitations, then it's your responsibility to ensure that the restrictions are not violated.