Page 1 of 1
Regaining Scaratch disk space
Posted: Thu Nov 17, 2011 12:49 am
by pandeesh
Currently my scratch disk is 73% utilized.
I am able to see lot of temp files in side the scratch path.
Can i simply remove all the files ?
What those files are exactly?
The issue is at this time, if i run any job which processes more than 2 billion records, the job gets aborted with write failed,file full errors.
The following errrors are:
Code: Select all
Unable to allocate communication process.
Short read
Socket on non socket
All those reveal the scratch disk space issue.
My question is, i need to increase the size of scratch disk or can i regain the space which is already used?
Thanks
Posted: Thu Nov 17, 2011 2:02 am
by chandra.shekhar@tcs.com
What is the size of the scratch disk?
Posted: Thu Nov 17, 2011 2:08 am
by pandeesh
This is the size of scratch disk
Code: Select all
FileSystem 1K-blocks used Availability USE%
/dev/mxse/vshared-vl1 20642428 14213512 5380340 73%
How we can regain the used space??
Thanks
Posted: Thu Nov 17, 2011 3:28 am
by pandeesh
How I can find that?
In my previous post I have posted the size of the path which is mentioned as scratch disk in my configuration file.
Thanks
Posted: Thu Nov 17, 2011 7:23 am
by chulett
Of course, you could always increase scratch space - more is better. And clearing it can be simple if it is dedicated scratch space. Does DataStage share that space with other applications? If not, if all that plays there are DataStage jobs, whenever there is a quiet moment when nothing is running you know you can scrape out any leftover debris from that space.
Posted: Thu Nov 17, 2011 8:17 am
by pandeesh
Thanks Craig!!
As per my understanding, if none of the jobs are running in the environment , then scratch disk directory should 0% used.
Am I correct?
Thanks
Posted: Thu Nov 17, 2011 9:35 am
by chulett
Ideally, yes, in a perfect world.
Posted: Thu Nov 17, 2011 9:39 am
by kduke
Start with the oldest files first. Delete them first.
Posted: Thu Nov 17, 2011 10:12 am
by pandeesh
Thanks for your inputs!!
But in my case,the jobs are running in grid environment,
So, i am not able to see the actual scratch disk path.
In the configuration file, the path is mentioned as "/a/Datastage/Scratch"
But actually there is no such path in my unix server.
I believe that directory is located in some other server.
Thanks
Posted: Thu Nov 17, 2011 12:53 pm
by PaulVL
Contact your datastage administrator and have them look at the mounts. They can see what jobs are currently running and the age of the left-over files in scratch.
An aborted job does not always clean up scratch files.
You admin team would have access to the compute nodes in your grid.
Posted: Thu Nov 17, 2011 2:33 pm
by ray.wurlod
Write jobs that never abort.