How to delete fileset
Moderators: chulett, rschirm, roy
How to delete fileset
Hi All,
I have used the Dataset Management Tool to delete datasets and filesets.
If I have to do it in a job I have been using routine stage in a sequencer and calling orchadmin delete command to delete the datasets. I was wondering the Dataset Management Tool under the covers was using orchadmin delete to delete the datasets.
I used the same orchadmin delete command to delete a fileset but it is not getting deleted. But the same fileset is being identified by Dataset Management Tool.
1. How is Dataset Management Tool able to identify the fileset and delete it.
2. How do I delete the fileset. Are there any other orchadmin commands for fileset.
TIA
Rich
I have used the Dataset Management Tool to delete datasets and filesets.
If I have to do it in a job I have been using routine stage in a sequencer and calling orchadmin delete command to delete the datasets. I was wondering the Dataset Management Tool under the covers was using orchadmin delete to delete the datasets.
I used the same orchadmin delete command to delete a fileset but it is not getting deleted. But the same fileset is being identified by Dataset Management Tool.
1. How is Dataset Management Tool able to identify the fileset and delete it.
2. How do I delete the fileset. Are there any other orchadmin commands for fileset.
TIA
Rich
-
- Participant
- Posts: 82
- Joined: Thu Dec 02, 2004 10:27 pm
- Location: INDIA
Hi,
dsxuserrio - Thanks for the reply. The job creates both a dataset as well as a fileset. I did use orchadmin delete from the unix command line for both the dataset and fileset. These are the results.
For dataset
The dataset is deleted successfully.
For fileset
The fileset is not deleted.
Iam able to delete the dataset and not the fileset. It says there is a problem with partitioning but the job executes OK and the fileset is created. Iam able to view the contents of the fileset. But Iam getting a warning in the job
Can this be a problem why the fileset was not deleted.
Secondly what does it mean by Node name "node3671" not in config file. The configuration I use for the job does have a node with this name. Is it referencing the configuration file defined in the unix environment variable APT_CONFIG_FILE.
TIA
Rich
dsxuserrio - Thanks for the reply. The job creates both a dataset as well as a fileset. I did use orchadmin delete from the unix command line for both the dataset and fileset. These are the results.
For dataset
Code: Select all
##I TUTL 000031 06:14:44(001) <main_program> The open files limit is 2000; raising to 2147483647.
##I TCOA 000024 06:14:44(002) <main_program> deleting /root/richdhan/testing.ds
##I TFSC 000001 06:14:44(003) <main_program> APT configuration file: /tmp/aptoa57358853352f56
##I TFSR 000010 06:14:45(000) <main_program> Step execution finished with status = OK.
##I TCOA 000025 06:14:45(001) <main_program> deleted /root/richdhan/testing.ds
For fileset
Code: Select all
##I TUTL 000031 06:09:47(001) <main_program> The open files limit is 2000; raising to 2147483647.
##I TCOA 000018 06:09:47(002) <main_program> deleting fileset /root/richdhan/testing.fs
##E TFDM 000028 06:09:47(003) <main_program> Invalid fileset /root/richdhan/testing.fs: problem with partitioning
##E TFDM 000000 06:09:47(004) <delete file set data files in APT_DeleteFilesetOperator> Error when checking operator: Node name "node3671" not in config file
##E TFSR 000019 06:09:47(005) <main_program> Could not check all operators because of previous error(s)
##I TCOA 000020 06:09:47(006) <main_program> delete FAILED for fileset /root/richdhan/testing.fs
Iam able to delete the dataset and not the fileset. It says there is a problem with partitioning but the job executes OK and the fileset is created. Iam able to view the contents of the fileset. But Iam getting a warning in the job
Code: Select all
File_Set_5: When checking operator: No nodes with disk in "export" resource pool; using default disk pool instead
Secondly what does it mean by Node name "node3671" not in config file. The configuration I use for the job does have a node with this name. Is it referencing the configuration file defined in the unix environment variable APT_CONFIG_FILE.
TIA
Rich
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
-
- Participant
- Posts: 82
- Joined: Thu Dec 02, 2004 10:27 pm
- Location: INDIA
Rich
I ried again to delete some more filesets using orchadmin command. I could delete many of them but not all. I am getting the same errors as you were getting.
It is not documented anywhere that filesets can be deleted using orchadmin, but I thought it would work because I could delete some of the filesets. I am sorry.
I will let you know if there is any way you could delete the filestes by more testing.
I ried again to delete some more filesets using orchadmin command. I could delete many of them but not all. I am getting the same errors as you were getting.
It is not documented anywhere that filesets can be deleted using orchadmin, but I thought it would work because I could delete some of the filesets. I am sorry.
I will let you know if there is any way you could delete the filestes by more testing.
dsxuserrio
Kannan.N
Bangalore,INDIA
Kannan.N
Bangalore,INDIA
-
- Participant
- Posts: 82
- Joined: Thu Dec 02, 2004 10:27 pm
- Location: INDIA
Rich
OK. When you cat your fileset or lookupfileset file in unix you can see exactly the location of the data. If your intention is to free space by deleting unwanted filesets and lookup filesets then you can delete the actual file which are pointed by the fileset
You can view the data in filesets using view data but not for lookup filesets.
OK. When you cat your fileset or lookupfileset file in unix you can see exactly the location of the data. If your intention is to free space by deleting unwanted filesets and lookup filesets then you can delete the actual file which are pointed by the fileset
In the above example the actual file you need to delete would be--Orchestrate File Set v2
--LFile
node1:/opt/Ascential/DataStage/Datasets/export.cdr.P000000_F0000
--Schema
record {final_delim=end, delim=",", quote=double}
( FIELD1: int32;
CUST_ID: int32;
SECTION_ID: int32;
)
If multiple files are listed you need to delete everything. After that ypu can delete the file with extension .fs or .lfs also.:/opt/Ascential/DataStage/Datasets/export.cdr.P000000_F0000
You can view the data in filesets using view data but not for lookup filesets.
dsxuserrio
Kannan.N
Bangalore,INDIA
Kannan.N
Bangalore,INDIA
Hi,
Sai & dsxuserrio - Thanks for the response. I have already done what you both have suggested. I manually deleted the control file(.fs) and the data files associated with it.
But my problem is that I would like to do it in a sequencer using ExecSH routine and using orchadmin delete command. Thats how I would delete datasets and it gets deleted. But when I try to delete filesets it is not happening. That is the reason I tried orchadmin delete from Unix command file and found that the fileset is not being deleted.
My question is that the same fileset is being identfied by Dataset Management Tool and able to delete it sucessfully. Why not the orchadmin delete command from the unix command line not able to delete the fileset.
I think the problem might be because of this - The configuration file used by the job is different from that which is defined in APT_CONFIG_FILE environment variable.
TIA
Rich
Sai & dsxuserrio - Thanks for the response. I have already done what you both have suggested. I manually deleted the control file(.fs) and the data files associated with it.
But my problem is that I would like to do it in a sequencer using ExecSH routine and using orchadmin delete command. Thats how I would delete datasets and it gets deleted. But when I try to delete filesets it is not happening. That is the reason I tried orchadmin delete from Unix command file and found that the fileset is not being deleted.
My question is that the same fileset is being identfied by Dataset Management Tool and able to delete it sucessfully. Why not the orchadmin delete command from the unix command line not able to delete the fileset.
I think the problem might be because of this - The configuration file used by the job is different from that which is defined in APT_CONFIG_FILE environment variable.
You mentioned some of the filesets were not getting deleted. Is it due to the fact that these filesets were created with a different configuration file other than what is defined in APT_CONFIG_FILE environment variable. Pls confirm.dsxuserio wrote:I ried again to delete some more filesets using orchadmin command. I could delete many of them but not all. I am getting the same errors as you were getting.
TIA
Rich
-
- Participant
- Posts: 82
- Joined: Thu Dec 02, 2004 10:27 pm
- Location: INDIA
Rich
For the problem about the scratch disk
I think you are right about the usage of different config files. The same thing will happen for datasets too. So you need to use the same config file that was used to create the filesets.
I was unsure becuase your node number was a 4 digit number. Can you please double check if you have node3671 in one of our config files please?
Thanks
For the problem about the scratch disk
You can add LFS or FS in your resource line in the config.apt fileFile_Set_5: When checking operator: No nodes with disk in "export" resource pool; using default disk pool instead
Code: Select all
resource disk "/opt/apps/Ascential/DataStage/Datasets" {pools "DS" "LFS" "FS" ""}
I was unsure becuase your node number was a 4 digit number. Can you please double check if you have node3671 in one of our config files please?
Thanks
dsxuserrio
Kannan.N
Bangalore,INDIA
Kannan.N
Bangalore,INDIA
Hi,
dsxuserri - Thanks for the info.
1. How am I able to delete the datasets and not the filesets using orchadmin command. I created both the dataset and fileset in the same job with the same configuration file ?
2. How is Dataset Management Tool able to identify both the Dataset and Fileset that have been created and not the orchadmin command(for fileset). What is happening under the covers when using Dataset Management Tool?
3. What is the command to be used at runtime(sequencer) in order to delete fileset as well as the associated exported files. Is there any documentation available
I have also posted these in Ascential Developernet. If there is any feedback I will let you know.
Thanks
Rich
dsxuserri - Thanks for the info.
Code: Select all
resource disk "/opt/apps/Ascential/DataStage/Datasets" {pools "DS" "LFS" "FS" ""}
Sorry just wanted to give dummy info instead of real one. But Iam still confused about 3 things.dsxuserrio wrote:I was unsure becuase your node number was a 4 digit number. Can you please double check if you have node3671 in one of our config files please?
1. How am I able to delete the datasets and not the filesets using orchadmin command. I created both the dataset and fileset in the same job with the same configuration file ?
2. How is Dataset Management Tool able to identify both the Dataset and Fileset that have been created and not the orchadmin command(for fileset). What is happening under the covers when using Dataset Management Tool?
3. What is the command to be used at runtime(sequencer) in order to delete fileset as well as the associated exported files. Is there any documentation available
I have also posted these in Ascential Developernet. If there is any feedback I will let you know.
Thanks
Rich
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom