Page 1 of 1
clearing scratch space
Posted: Sun Mar 04, 2007 8:46 am
by madhukar
Hi All,
How can i clear the files under scratch directory
can i just remove with rm command?
Thx
Posted: Sun Mar 04, 2007 10:29 am
by DSguru2B
Do an all term search on key words 'scratch space clean up'.
Posted: Sun Mar 04, 2007 10:33 am
by ray.wurlod
Are any jobs running that might be using the scratch space? Scratch space should be cleaned up automatically - if it's not you have jobs that are aborting, and should do something about that.
Posted: Sun Mar 04, 2007 12:15 pm
by madhukar
yes ray, you got it right all my jobs were fine during development.
when testing with large volume of data jobs are aborting with SIGBUS errors.
and scratch space full.
Is there any way i can clean up those files (aborted jobs) and run the jobs again.
Posted: Sun Mar 04, 2007 8:41 pm
by ray.wurlod
Unless you're running out of space there's no rush to clean them up. But, if you want to, simply use rm when there's no jobs running.
Posted: Sun Mar 04, 2007 11:43 pm
by kumar_s
As noted by Ray, you can go ahead and clear the files avialble in TEMP dir or Scratch disk when the jobs are not running. Those are temporay files used by intermediate stages available in the jobs like SORT, Aggregator etc.,
It can be isolated to different node pool via Config file.
Posted: Mon Mar 05, 2007 4:50 am
by madhukar
removed the files in scratch space but still it gives 99% full (previously also)
Am i missing anything?
Posted: Mon Mar 05, 2007 8:02 am
by DSguru2B
You need more space. As Ray puts it, supply and demand.
Posted: Mon Mar 05, 2007 8:10 am
by ray.wurlod
madhukar wrote:removed the files in scratch space but still it gives 99% full (previously also)
Am i missing anything?
Yes, the 99%.
Use
du -s to identify which directories are taking the resources.
Posted: Mon Mar 05, 2007 5:08 pm
by kumar_s
Also check the config file, if you have given a different path for Sort node pool and scratch space.