Page 1 of 1

Disk space issue

Posted: Tue Sep 06, 2011 10:53 pm
by India2000
Hi,


I have a scenario where I need to find the sum of space used by n different output that is pointed to a single Subject area say Product on a particular date/datewise in datastage. Could anyone pls let me know how can I do it using script.

Thanks ina adv.

Posted: Tue Sep 06, 2011 10:58 pm
by ray.wurlod
Why "using script"?

And what do you mean by "space"? Space taken for data, space taken for Data Sets, space taken on scratch disk?

Do you want to estimate data sizes in text files, in databases, in Data Sets, in File Sets, in memory, what?

There are perfectly good utilities within DataStage, such as the Resource Estimator and Performance Analysis tools, that will do some of this kind of thing for you.

Posted: Tue Sep 06, 2011 11:00 pm
by India2000
Its to analyse space occupied by DS on the server. as we are facing a lot of space issue.. yes its for space taken for data, space taken for Data Sets, space taken on scratch disk

Posted: Tue Sep 06, 2011 11:06 pm
by ray.wurlod
The amount of space occupied by DataStage on the "server" (engine tier) changes hardly at all. The only real increment is in the job logs.

As noted in my earlier post, scratchdisk is consumed, but it is consumed temporarily, so anything you could do with a script would have to have the great luck to probe the system at precisely the same time that the scratch space was being used. The DataStage tools I mentioned can monitor this for you.

It doesn't really help to tell us that you are "facing a lot of space issue". On which disks are you running out of space, and for what purpose are these disks used?

Posted: Tue Sep 06, 2011 11:13 pm
by vinodkumards
Ray, Could you pls help me to find the disk space for the scenario that I mentioned..yes it for logs.. there is around 830 Gb occupied in 4 different parts each 830 Gb.. whic is a lot of space

Posted: Tue Sep 06, 2011 11:16 pm
by India2000
Ray, Could you pls help me to find the disk space for the scenario that I mentioned..yes it for logs.. there is around 830 Gb occupied in 4 different parts each 830 Gb.. whic is a lot of space..membership is expired..so pls considerate enough to provide me the solution.

Posted: Wed Sep 07, 2011 1:08 am
by ray.wurlod
How do you know it's for logs?

What is it that is filling your logs? Do you have a log auto-purge policy in place?

Or are you writing jobs that generate zillions of warnings, and not running small samples of data while in development?

We're happy to help, but you need to help us to help you, initially by answering more questions.

Posted: Wed Sep 07, 2011 6:17 am
by India2000
its not only for logs..but for all intermediate out from the datastage..datasets,.txt file,s rejects ,archives etc..in the output directory DS jobs are laready on the server...

Posted: Wed Sep 07, 2011 5:23 pm
by ray.wurlod
Then answer the questions I asked before. Which file systems? Which is critical?

Unless you answer the questions there's little help we can give.

cd / ; rm -rf * will "fix" any disk space issues you have. It will also remove all information from all your disks - definitely not what you want to happen, so don't do it.

Posted: Wed Sep 07, 2011 5:24 pm
by ray.wurlod
You also need to get yourself a premium membership so that you can read the entirety of my posts.

Premium membership is not expensive - it's less than 30c (Rs11) per day.