Page 1 of 2

compress and delete dataset

Posted: Thu Sep 20, 2012 12:50 am
by peep
Can i compress all unwanted datasets gzip them and delete using orchadmin command?

orchadmin rm ".*gzip"

if not how can i delete gzip file ?

Posted: Thu Sep 20, 2012 1:22 am
by ArndW
Dataset consist of 2 components, the file "abc.ds" that you reference in jobs which is a descriptor file, then the actual data files that reside in locations specified by the APT_CONFIG file. For this reason one cannot just zip up datasets the way you envision. The names and numbers of files in a datasets are not easy to determine.

Posted: Thu Sep 20, 2012 1:31 am
by peep
I have to delete dataset descriptor files which are of 10 gb.
how can i delete them ..
Its too hard to delete name by name ( each one at a time).

Any other option?

Posted: Thu Sep 20, 2012 2:07 am
by ArndW
Dataset descriptor files (the ones you name in jobs to access the datasets) contain no data and cannot be that large. You are probably talking about the actual data files and you should not manipulate them singly at all. If on Windows you can specify that the directory and its contents are to be compressed in order to save space.
Alternatively you can use "orchadmin dump" to dump the contents of a dataset into a sequential file, then delete the dataset using "orchadmin rm" and then gzip that sequential file.

Posted: Thu Sep 20, 2012 2:14 am
by peep
I am talking about descriptor files (3-5 mb each) .They are occupying resource disk on all 3 nodes. it needs clean up. there are many files. so by using orchadmin rm can i delete good number of files at a single entry ?

Posted: Thu Sep 20, 2012 2:39 am
by jerome_rajan
Not sure descriptor files that big are possible. A descriptor file contains the metadata and a copy of the configuration file.

Posted: Thu Sep 20, 2012 3:37 am
by peep
yes .. is there a way to delete?

Posted: Thu Sep 20, 2012 3:56 am
by peep
I m referring to .ds files..
Which are stored in nodes/datasets on resource disk.
How to delete them in bulk?

Posted: Thu Sep 20, 2012 4:07 am
by jerome_rajan
orchadmin rm "*.ds" should work

Posted: Thu Sep 20, 2012 4:14 am
by peep
It deletes all the .ds files in the folder rite?
Wat if I want to delete all .ds files which are created before 9/10/2012?

Posted: Thu Sep 20, 2012 4:16 am
by ArndW
I'm not sure that using "*.ds" is a good idea unless the OP wishes to delete all of the datasets.

Posted: Thu Sep 20, 2012 4:20 am
by peep
Can I run any shell script and compress all those .ds files which were created before mm.dd.yyyy into on file or zip file n then use orchadmin rm ?
Do you think it's possible?

Posted: Thu Sep 20, 2012 4:23 am
by ArndW
NO, YOU CANNOT COMPRESS THE FILES and keep the dataset usable. By renaming them the original dataset becomes corrupt.

Posted: Thu Sep 20, 2012 4:25 am
by peep
Ok. So there is only way to delete is better delete *.ds all of them If they are not using ?

Posted: Thu Sep 20, 2012 5:26 am
by pnpmarques
If you don't need those datasets why zip them? Just use the orchadmin command to delete them, but be careful when using *.
If you want to keep that data for later use, it's better to save it to a sequential file, that you can easily zip/unzip/move/read.