Disk space issue

India2000 · Post by **India2000** » Tue Sep 06, 2011 10:53 pm

Hi,

I have a scenario where I need to find the sum of space used by n different output that is pointed to a single Subject area say Product on a particular date/datewise in datastage. Could anyone pls let me know how can I do it using script.

Thanks ina adv.

ray.wurlod · Post by **ray.wurlod** » Tue Sep 06, 2011 10:58 pm

Why "using script"?

And what do you mean by "space"? Space taken for data, space taken for Data Sets, space taken on scratch disk?

Do you want to estimate data sizes in text files, in databases, in Data Sets, in File Sets, in memory, what?

There are perfectly good utilities within DataStage, such as the Resource Estimator and Performance Analysis tools, that will do some of this kind of thing for you.

India2000 · Post by **India2000** » Tue Sep 06, 2011 11:00 pm

Its to analyse space occupied by DS on the server. as we are facing a lot of space issue.. yes its for space taken for data, space taken for Data Sets, space taken on scratch disk

ray.wurlod · Post by **ray.wurlod** » Tue Sep 06, 2011 11:06 pm

The amount of space occupied by DataStage on the "server" (engine tier) changes hardly at all. The only real increment is in the job logs.

As noted in my earlier post, scratchdisk is consumed, but it is consumed temporarily, so anything you could do with a script would have to have the great luck to probe the system at precisely the same time that the scratch space was being used. The DataStage tools I mentioned can monitor this for you.

It doesn't really help to tell us that you are "facing a lot of space issue". On which disks are you running out of space, and for what purpose are these disks used?

vinodkumards · Post by **vinodkumards** » Tue Sep 06, 2011 11:13 pm

Ray, Could you pls help me to find the disk space for the scenario that I mentioned..yes it for logs.. there is around 830 Gb occupied in 4 different parts each 830 Gb.. whic is a lot of space

India2000 · Post by **India2000** » Tue Sep 06, 2011 11:16 pm

Ray, Could you pls help me to find the disk space for the scenario that I mentioned..yes it for logs.. there is around 830 Gb occupied in 4 different parts each 830 Gb.. whic is a lot of space..membership is expired..so pls considerate enough to provide me the solution.

ray.wurlod · Post by **ray.wurlod** » Wed Sep 07, 2011 1:08 am

How do you know it's for logs?

What is it that is filling your logs? Do you have a log auto-purge policy in place?

Or are you writing jobs that generate zillions of warnings, and not running small samples of data while in development?

We're happy to help, but you need to help us to help you, initially by answering more questions.

India2000 · Post by **India2000** » Wed Sep 07, 2011 6:17 am

its not only for logs..but for all intermediate out from the datastage..datasets,.txt file,s rejects ,archives etc..in the output directory DS jobs are laready on the server...

ray.wurlod · Post by **ray.wurlod** » Wed Sep 07, 2011 5:23 pm

Then answer the questions I asked before. Which file systems? Which is critical?

Unless you answer the questions there's little help we can give.

cd / ; rm -rf * will "fix" any disk space issues you have. It will also remove all information from all your disks - definitely not what you want to happen, so don't do it.

ray.wurlod · Post by **ray.wurlod** » Wed Sep 07, 2011 5:24 pm

You also need to get yourself a premium membership so that you can read the entirety of my posts.

Premium membership is not expensive - it's less than 30c (Rs11) per day.