clean up memory for datasets which is deleted manually

mspanda · Post by **mspanda** » Fri Jan 23, 2009 11:15 am

Hi,

I am creating all dataset files through my datastage job in a working Location:/Path1/datasetfiles/
File path for datasets in congiguration file :/path2

Could someone please tell me how to clear the memory
of the datasets location mensioned in configuration file(/path2) ,if the original dataset files are
already deleted (/Path1/datasetfiles) through unix command manually.

I have created a unix script with orchadmin command to delete the dataset file from both
/Path1/datasetfiles/ and /path2/ and it is working fine.

I am creating dataset through datastage with the same name which is already deleted .
But this new dataset file is poniting to a different memory location.When i am running the script ,
the new one is being deleted both from /Path1/datasetfiles/ and /path2 but old file with
same name still exist.

Appriciate for you suggestions how to clean this memery .
And Is there any problem if i will delete this file manually from (path mensioned in congiguration
file for datasetspath ) :/path2

Thanks,
mspanda

peddidsx · Post by **peddidsx** » Fri Jan 23, 2009 12:22 pm

There are 2 ways of cleaning up datasets:

1)Go to datastage Management Utility through director and clean up.

2)Search for orchadmin command in datastage.

mspanda · Post by **mspanda** » Fri Jan 23, 2009 1:26 pm

peddidsx wrote:There are 2 ways of cleaning up datasets:

1)Go to datastage Management Utility through director and clean up.

2)Search for orchadmin command in datastage.

Thanks for the Answer. but

If source file is there then there is no problem ,i can delete from orchadmin command or
through dataset manager .

In my previous post:i have mensioned about the path.

File present in /Path1/datasetfiles/ (path mensioned in config file)
original deleted from /path2/ (path mensioned for working directory)

Problem is the source file *.ds file is deleted.

i tried to delete from /Path1/datasetfiles/ even with the command and dataset Management in director and even dataset management
in designer (ver 8.0) .it is giving error because the original file is not there which is deleted manually in unix command.

Thanks,
mspanda

ray.wurlod · Post by **ray.wurlod** » Fri Jan 23, 2009 3:08 pm

Restore the *.ds file from backup and delete the Data Set properly.

If you don't have a backup then shame on you.

mspanda · Post by **mspanda** » Fri Jan 23, 2009 3:44 pm

ray.wurlod wrote:Restore the *.ds file from backup and delete the Data Set properly.

If you don't have a backup then shame on you. ...

Sorry for the misinterpretation

*.ds is the intermediate file. While running datastage it is created.

ray.wurlod · Post by **ray.wurlod** » Fri Jan 23, 2009 4:06 pm

And it contains the only accurate guide to where the data files of the Data Set are stored. So, if someone has deleted it, you need to get it back from somewhere. And castigate them severely!

mk_ds09 · Post by **mk_ds09** » Mon Jan 26, 2009 3:06 am

i tried to delete from /Path1/datasetfiles/ even with the command and dataset Management in director and even dataset management
in designer (ver 8.0) .it is giving error because the original file is not there which is deleted manually in unix command.

If the .ds file is deleted, there is no way to retrive the data files, as they become orphan now..

I guess to delete the data file from the path /Path1/datasetfiles/ is one of the options that you are having...

I think u need to manully delete those files from the given directoy..
which command and login you are using to delete those files..
can you paste the error message that you are getting when you are trying to delete those files. ?

As Datastage management wont find the descriptor file, you cant delete those using this tool..so i guess there is only one way to do it !

hope this helps...

-----------------
MK

ray.wurlod · Post by **ray.wurlod** » Mon Jan 26, 2009 4:36 am

... apart from restoring the *.ds file from backup, of course.

mk_ds09 · Post by **mk_ds09** » Mon Jan 26, 2009 5:15 am

i dont think..people are storing the backups for the .ds file..as they are only the intermediate files..!

and ys..unix commands can be used to restore the deleted file..
I am not sure the what exact command is..!
however it can be used to restore .ds file and then it can be deleted using ds management tool..

---------------------------

MK

mspanda · Post by **mspanda** » Mon Jan 26, 2009 10:58 am

Thanks MK,
I got an idea from your answer why this error i am getting now.
as it is an orphan it is giving error and no way to recover it.
but my goal is to clear the memory.

Here is the command you are asking for: ( i have used this command in the script)

previously before deleting the .ds file from /path2/ , i was using following command in a script
and it is working fine and clear the memory from both /path2/ and /Path1/datasetfiles/ location.

orchadmin delete /path2/out_Dataset.ds

For creating one file in /path2/out_Dataset.ds ,datastage is creating 4 files in
/Path1/datasetfiles/

out_Dataset.ds.dsadm.servername.0000.0003.0000.2c82.cb422922.0003.afff8327
out_Dataset.ds.dsadm.servername.0000.0002.0000.2c82.cb422922.0002.3c8979f6
out_Dataset.ds.dsadm.servername.0000.0001.0000.2c82.cb422922.0001.dde27080
out_Dataset.ds.dsadm.servername.0000.0000.0000.2c82.cb422922.0000.b5c0490b

now after manually deleting the .ds file from /path2/ i am using the following command to delete all files from /Path1/datasetfiles/

orchadmin delete out_Dataset.ds.dsadm.servername.0000.0003.0000.2c82.cb422922.0003.afff8327
orchadmin delete out_Dataset.ds.dsadm.servername.0000.0002.0000.2c82.cb422922.0002.3c8979f6
orchadmin delete out_Dataset.ds.dsadm.servername.0000.0001.0000.2c82.cb422922.0001.dde27080
orchadmin delete out_Dataset.ds.dsadm.servername.0000.0000.0000.2c82.cb422922.0000.b5c0490b

It is giving me the following Error:

""

##I IIS-DSEE-TCOA-00021 11:38:46(001) <main_program>
WARNING: could not delete out_Dataset.ds.dsadm.servername.0000.0003.0000.2c82.cb422922.0003.afff8327
because it does not exist.

""

Thanks,
msp

chulett · Post by **chulett** » Mon Jan 26, 2009 11:17 am

Once you start down the "manual deletion" path (and thus destroyed any automated mechanism) it seems you'll have to walk that manual path to its end. At this point you would just use rm to "clear the memory", I assume.

mspanda · Post by **mspanda** » Mon Jan 26, 2009 12:45 pm

chulett wrote:Once you start down the "manual deletion" path (and thus destroyed any automated mechanism) it seems you'll have to walk that manual path to its end. At this point you would just use rm to "cle ...

That means , Only solution to clear the memory in the above case is i.e :we must have to use unix command : rm ?

Example:
rm /Path1/datasetfiles/ out_Dataset.ds.dsadm.servername.0000.0003.0000.2c82.cb422922.0003.afff8327

Thanks for your suggestions.....

mspanda · Post by **mspanda** » Mon Jan 26, 2009 2:35 pm

sorry i have missed one more Error Line :

##E IIS-DSEE-TFDM-00093 15:19:42(001) <main_program> The file identified as '/Path1/datasetfiles/out_Dataset.ds.dsadm.servername.0000.0003.0000.2c82.cb422922.0003.afff8327' is not an ORCHESTRATE Data Set descriptor.
##I IIS-DSEE-TCOA-00022 15:19:42(002) <main_program> WARNING: could not delete /Path1/datasetfiles/out_Dataset.ds.dsadm.servername.0000.0003.0000.2c82.cb422922.0003.afff8327
dataset descriptor file has unrecognizeable contents.