How to delete fileset

richdhan · Post by **richdhan** » Mon Feb 28, 2005 7:55 am

Hi All,

I have used the Dataset Management Tool to delete datasets and filesets.

If I have to do it in a job I have been using routine stage in a sequencer and calling orchadmin delete command to delete the datasets. I was wondering the Dataset Management Tool under the covers was using orchadmin delete to delete the datasets.

I used the same orchadmin delete command to delete a fileset but it is not getting deleted. But the same fileset is being identified by Dataset Management Tool.

1. How is Dataset Management Tool able to identify the fileset and delete it.

2. How do I delete the fileset. Are there any other orchadmin commands for fileset.

TIA
Rich

dsxuserrio · Post by **dsxuserrio** » Mon Feb 28, 2005 4:43 pm

I tried orchadmin delete test.lfs. It worked.
However you cannot look at the data using orchadmin dump. Same way cant use dsrecords to get count.

richdhan · Post by **richdhan** » Tue Mar 01, 2005 5:52 am

Hi,

dsxuserrio - Thanks for the reply. The job creates both a dataset as well as a fileset. I did use orchadmin delete from the unix command line for both the dataset and fileset. These are the results.

For dataset

Code: Select all

##I TUTL 000031 06:14:44(001) <main_program> The open files limit is 2000; raising to 2147483647. 
##I TCOA 000024 06:14:44(002) <main_program> deleting /root/richdhan/testing.ds                   
##I TFSC 000001 06:14:44(003) <main_program> APT configuration file: /tmp/aptoa57358853352f56     
##I TFSR 000010 06:14:45(000) <main_program> Step execution finished with status = OK.            
##I TCOA 000025 06:14:45(001) <main_program> deleted /root/richdhan/testing.ds

The dataset is deleted successfully.

For fileset

Code: Select all

##I TUTL 000031 06:09:47(001) <main_program> The open files limit is 2000; raising to 2147483647. 
##I TCOA 000018 06:09:47(002) <main_program> deleting fileset /root/richdhan/testing.fs           
##E TFDM 000028 06:09:47(003) <main_program> Invalid fileset /root/richdhan/testing.fs: problem with partitioning
##E TFDM 000000 06:09:47(004) <delete file set data files in APT_DeleteFilesetOperator> Error when checking operator: Node name "node3671" not in config file    
##E TFSR 000019 06:09:47(005) <main_program> Could not check all operators because of previous error(s)
##I TCOA 000020 06:09:47(006) <main_program> delete FAILED for fileset /root/richdhan/testing.fs

The fileset is not deleted.

Iam able to delete the dataset and not the fileset. It says there is a problem with partitioning but the job executes OK and the fileset is created. Iam able to view the contents of the fileset. But Iam getting a warning in the job

Code: Select all

File_Set_5: When checking operator: No nodes with disk in "export" resource pool; using default disk pool instead

Can this be a problem why the fileset was not deleted.

Secondly what does it mean by Node name "node3671" not in config file. The configuration I use for the job does have a node with this name. Is it referencing the configuration file defined in the unix environment variable APT_CONFIG_FILE.

TIA
Rich

Sainath.Srinivasan · Post by **Sainath.Srinivasan** » Tue Mar 01, 2005 7:10 am

The export pool is the defn in your configuration file.

If you do not have it, then you can add the word "export" (in double quotes) following the pool defn (preferably to those with default - blank - pool name).

I do not think this can cause any problem.

Sainath.Srinivasan · Post by **Sainath.Srinivasan** » Tue Mar 01, 2005 7:14 am

By having a look in your message log, it appears that your config file has been changed / is different compared to the one used to create the file set.

You can try to manually delete the fileset (central control and the corresponding partition files).

dsxuserrio · Post by **dsxuserrio** » Tue Mar 01, 2005 7:15 am

Rich
I ried again to delete some more filesets using orchadmin command. I could delete many of them but not all. I am getting the same errors as you were getting.

It is not documented anywhere that filesets can be deleted using orchadmin, but I thought it would work because I could delete some of the filesets. I am sorry.

I will let you know if there is any way you could delete the filestes by more testing.

dsxuserrio · Post by **dsxuserrio** » Tue Mar 01, 2005 4:31 pm

Rich
OK. When you cat your fileset or lookupfileset file in unix you can see exactly the location of the data. If your intention is to free space by deleting unwanted filesets and lookup filesets then you can delete the actual file which are pointed by the fileset

--Orchestrate File Set v2
--LFile
node1:/opt/Ascential/DataStage/Datasets/export.cdr.P000000_F0000
--Schema
record {final_delim=end, delim=",", quote=double}
( FIELD1: int32;
CUST_ID: int32;
SECTION_ID: int32;
)

In the above example the actual file you need to delete would be

:/opt/Ascential/DataStage/Datasets/export.cdr.P000000_F0000

If multiple files are listed you need to delete everything. After that ypu can delete the file with extension .fs or .lfs also.

You can view the data in filesets using view data but not for lookup filesets.

richdhan · Post by **richdhan** » Wed Mar 02, 2005 1:53 am

Hi,

Sai & dsxuserrio - Thanks for the response. I have already done what you both have suggested. I manually deleted the control file(.fs) and the data files associated with it.

But my problem is that I would like to do it in a sequencer using ExecSH routine and using orchadmin delete command. Thats how I would delete datasets and it gets deleted. But when I try to delete filesets it is not happening. That is the reason I tried orchadmin delete from Unix command file and found that the fileset is not being deleted.

My question is that the same fileset is being identfied by Dataset Management Tool and able to delete it sucessfully. Why not the orchadmin delete command from the unix command line not able to delete the fileset.

I think the problem might be because of this - The configuration file used by the job is different from that which is defined in APT_CONFIG_FILE environment variable.

dsxuserio wrote:I ried again to delete some more filesets using orchadmin command. I could delete many of them but not all. I am getting the same errors as you were getting.

You mentioned some of the filesets were not getting deleted. Is it due to the fact that these filesets were created with a different configuration file other than what is defined in APT_CONFIG_FILE environment variable. Pls confirm.

TIA
Rich

dsxuserrio · Post by **dsxuserrio** » Thu Mar 03, 2005 11:27 am

Rich

For the problem about the scratch disk

File_Set_5: When checking operator: No nodes with disk in "export" resource pool; using default disk pool instead

You can add LFS or FS in your resource line in the config.apt file

Code: Select all

resource disk "/opt/apps/Ascential/DataStage/Datasets" {pools "DS" "LFS" "FS" ""}

I think you are right about the usage of different config files. The same thing will happen for datasets too. So you need to use the same config file that was used to create the filesets.

I was unsure becuase your node number was a 4 digit number. Can you please double check if you have node3671 in one of our config files please?

Thanks

richdhan · Post by **richdhan** » Fri Mar 04, 2005 1:33 am

Hi,

dsxuserri - Thanks for the info.

Code: Select all

resource disk "/opt/apps/Ascential/DataStage/Datasets" {pools "DS" "LFS" "FS" ""}

dsxuserrio wrote:I was unsure becuase your node number was a 4 digit number. Can you please double check if you have node3671 in one of our config files please?

Sorry just wanted to give dummy info instead of real one. But Iam still confused about 3 things.

1. How am I able to delete the datasets and not the filesets using orchadmin command. I created both the dataset and fileset in the same job with the same configuration file ?

2. How is Dataset Management Tool able to identify both the Dataset and Fileset that have been created and not the orchadmin command(for fileset). What is happening under the covers when using Dataset Management Tool?

3. What is the command to be used at runtime(sequencer) in order to delete fileset as well as the associated exported files. Is there any documentation available

I have also posted these in Ascential Developernet. If there is any feedback I will let you know.

Thanks
Rich

ds_teg · Post by **ds_teg** » Wed Aug 26, 2009 9:30 am

Any anybody help me answering above questions ??

Sainath.Srinivasan · Post by **Sainath.Srinivasan** » Wed Aug 26, 2009 9:37 am

What is your issue?

Post in a seperate thread.

ds_teg · Post by **ds_teg** » Wed Aug 26, 2009 9:39 am

Just i am going though some posts and found that those questions regarding how to delete the filesets has not been answered .So , Out of curiosity i once again bumped that old post .

chulett · Post by **chulett** » Wed Aug 26, 2009 10:19 am

Don't. Start a new post with whatever question you have.