compress and delete dataset

peep · Post by **peep** » Thu Sep 20, 2012 12:50 am

Can i compress all unwanted datasets gzip them and delete using orchadmin command?

orchadmin rm ".*gzip"

if not how can i delete gzip file ?

ArndW · Post by **ArndW** » Thu Sep 20, 2012 1:22 am

Dataset consist of 2 components, the file "abc.ds" that you reference in jobs which is a descriptor file, then the actual data files that reside in locations specified by the APT_CONFIG file. For this reason one cannot just zip up datasets the way you envision. The names and numbers of files in a datasets are not easy to determine.

peep · Post by **peep** » Thu Sep 20, 2012 1:31 am

I have to delete dataset descriptor files which are of 10 gb.
how can i delete them ..
Its too hard to delete name by name ( each one at a time).

Any other option?

ArndW · Post by **ArndW** » Thu Sep 20, 2012 2:07 am

Dataset descriptor files (the ones you name in jobs to access the datasets) contain no data and cannot be that large. You are probably talking about the actual data files and you should not manipulate them singly at all. If on Windows you can specify that the directory and its contents are to be compressed in order to save space.
Alternatively you can use "orchadmin dump" to dump the contents of a dataset into a sequential file, then delete the dataset using "orchadmin rm" and then gzip that sequential file.

peep · Post by **peep** » Thu Sep 20, 2012 2:14 am

I am talking about descriptor files (3-5 mb each) .They are occupying resource disk on all 3 nodes. it needs clean up. there are many files. so by using orchadmin rm can i delete good number of files at a single entry ?

jerome_rajan · Post by **jerome_rajan** » Thu Sep 20, 2012 2:39 am

Not sure descriptor files that big are possible. A descriptor file contains the metadata and a copy of the configuration file.

peep · Post by **peep** » Thu Sep 20, 2012 3:37 am

yes .. is there a way to delete?

peep · Post by **peep** » Thu Sep 20, 2012 3:56 am

I m referring to .ds files..
Which are stored in nodes/datasets on resource disk.
How to delete them in bulk?

jerome_rajan · Post by **jerome_rajan** » Thu Sep 20, 2012 4:07 am

orchadmin rm "*.ds" should work

peep · Post by **peep** » Thu Sep 20, 2012 4:14 am

It deletes all the .ds files in the folder rite?
Wat if I want to delete all .ds files which are created before 9/10/2012?

ArndW · Post by **ArndW** » Thu Sep 20, 2012 4:16 am

I'm not sure that using "*.ds" is a good idea unless the OP wishes to delete all of the datasets.

peep · Post by **peep** » Thu Sep 20, 2012 4:20 am

Can I run any shell script and compress all those .ds files which were created before mm.dd.yyyy into on file or zip file n then use orchadmin rm ?
Do you think it's possible?

ArndW · Post by **ArndW** » Thu Sep 20, 2012 4:23 am

NO, YOU CANNOT COMPRESS THE FILES and keep the dataset usable. By renaming them the original dataset becomes corrupt.

peep · Post by **peep** » Thu Sep 20, 2012 4:25 am

Ok. So there is only way to delete is better delete *.ds all of them If they are not using ?

pnpmarques · Post by **pnpmarques** » Thu Sep 20, 2012 5:26 am

If you don't need those datasets why zip them? Just use the orchadmin command to delete them, but be careful when using *.
If you want to keep that data for later use, it's better to save it to a sequential file, that you can easily zip/unzip/move/read.