Regaining Scaratch disk space

pandeesh · Post by **pandeesh** » Thu Nov 17, 2011 12:49 am

Currently my scratch disk is 73% utilized.
I am able to see lot of temp files in side the scratch path.
Can i simply remove all the files ?
What those files are exactly?
The issue is at this time, if i run any job which processes more than 2 billion records, the job gets aborted with write failed,file full errors.
The following errrors are:

Code: Select all

Unable to allocate communication process.
Short read
Socket on non socket

All those reveal the scratch disk space issue.
My question is, i need to increase the size of scratch disk or can i regain the space which is already used?

Thanks

chandra.shekhar@tcs.com · Thu Nov 17, 2011 2:02 am

What is the size of the scratch disk?

pandeesh · Post by **pandeesh** » Thu Nov 17, 2011 2:08 am

This is the size of scratch disk

Code: Select all

FileSystem                              1K-blocks   used         Availability     USE%
/dev/mxse/vshared-vl1          20642428    14213512   5380340         73%

How we can regain the used space??

Thanks

pandeesh · Post by **pandeesh** » Thu Nov 17, 2011 3:28 am

How I can find that?
In my previous post I have posted the size of the path which is mentioned as scratch disk in my configuration file.

Thanks

chulett · Post by **chulett** » Thu Nov 17, 2011 7:23 am

Of course, you could always increase scratch space - more is better. And clearing it can be simple if it is dedicated scratch space. Does DataStage share that space with other applications? If not, if all that plays there are DataStage jobs, whenever there is a quiet moment when nothing is running you know you can scrape out any leftover debris from that space.

pandeesh · Post by **pandeesh** » Thu Nov 17, 2011 8:17 am

Thanks Craig!!
As per my understanding, if none of the jobs are running in the environment , then scratch disk directory should 0% used.
Am I correct?

Thanks

chulett · Post by **chulett** » Thu Nov 17, 2011 9:35 am

Ideally, yes, in a perfect world.

kduke · Post by **kduke** » Thu Nov 17, 2011 9:39 am

Start with the oldest files first. Delete them first.

pandeesh · Post by **pandeesh** » Thu Nov 17, 2011 10:12 am

Thanks for your inputs!!
But in my case,the jobs are running in grid environment,
So, i am not able to see the actual scratch disk path.
In the configuration file, the path is mentioned as "/a/Datastage/Scratch"
But actually there is no such path in my unix server.
I believe that directory is located in some other server.
Thanks

PaulVL · Post by **PaulVL** » Thu Nov 17, 2011 12:53 pm

Contact your datastage administrator and have them look at the mounts. They can see what jobs are currently running and the age of the left-over files in scratch.

An aborted job does not always clean up scratch files.

You admin team would have access to the compute nodes in your grid.

ray.wurlod · Post by **ray.wurlod** » Thu Nov 17, 2011 2:33 pm

Write jobs that never abort.