Page 2 of 2
Posted: Wed Jul 18, 2007 5:27 am
by abhilashnair
Well..I just checked the size of DATA.30 file inside the hash file dir...It has crossed 4GB
Posted: Wed Jul 18, 2007 1:24 pm
by abhilashnair
What perplexes me is that when Stage Write Cache is allowed...hash file can only accomodate 2Gb...when it is not allowed it accomodates more than that...I know it is very slow and the rate at which rows gets populated decreases as time passes...but still it does not abort...
Posted: Wed Jul 18, 2007 2:52 pm
by ray.wurlod
There is no a priori knowledge of the size of the hashed file when "allow cache" is set at design time. If the hashed file is too big, then a warning is issued, and the hashed file is accessed on disk. But this can not be ascertained until run time.
Posted: Thu Jul 19, 2007 8:55 am
by abhilashnair
I just came to know that the OS is limiting hash file size. But the admin is not ready to increase this limit saying that it may affect other hash files which are fine now...Is this really an issue?
Posted: Thu Jul 19, 2007 3:12 pm
by ray.wurlod
No.
No hashed file can exceed 2GB without being created/resized to use 64-bit addressing internally. Increasing ulimit or a kernel file size parameter will have no impact whatsoever on existing or newly-created hashed files.
Posted: Fri Jul 20, 2007 1:03 am
by abhilashnair
We already have 64BIT hash file...But once it reaches 2 GB limit the job aborts...If a 64bIT hashed file can hold data in terabytes..why is this happening?. It is TYPE30 file.
Urgent Issue in Production
Posted: Fri Jul 20, 2007 6:21 am
by abhilashnair
We have a urgent issue in Production...We have 184 million records which have come from source. The size is 18 GB. We want them to be loaded into a hashed file. We have 64BIT Type 30 Hash file. But still after 2Gb it is aborting...Please Help
Posted: Fri Jul 20, 2007 6:42 am
by chulett
abhilashnair wrote:I just came to know that the OS is limiting hash file size.
This would be your 'urgent' issue, I'd wager. Have your SA remove the file size limitation for your DataStage user.
Posted: Fri Jul 20, 2007 10:38 pm
by ray.wurlod
If they won't, use a Distributed file of at least ten parts.