Deleting a dataset
Moderators: chulett, rschirm, roy
Deleting a dataset
Hi everybody,
I have one requirement where we need to delete DataSet through UNIX.
My Observation is:
DataSet stage creates a .ds file which is like a pointer to Data file(s) (which are binary) stored at particular location.
So if I remove the .ds file and the data file will that be enough.
Also I guess there can be many datafiles will be created for one dataset so how tho find all the related datafiles.
(Note: These Data files are in binary format.)
There is one utitlity in DataStage which can do these things i.e. DataSet Management Utitlity but this is a client based utility. And the requirement is we need to do it on server/ using UNIX shell script.
Also we need to count number of records stored in a dataset.
I found one utility dsrecords but it works on something called OSH and that doesne't work on my UNIX server. Is there any other way?
Thanks,
Manish
___________________
Manish Kulkarni
I have one requirement where we need to delete DataSet through UNIX.
My Observation is:
DataSet stage creates a .ds file which is like a pointer to Data file(s) (which are binary) stored at particular location.
So if I remove the .ds file and the data file will that be enough.
Also I guess there can be many datafiles will be created for one dataset so how tho find all the related datafiles.
(Note: These Data files are in binary format.)
There is one utitlity in DataStage which can do these things i.e. DataSet Management Utitlity but this is a client based utility. And the requirement is we need to do it on server/ using UNIX shell script.
Also we need to count number of records stored in a dataset.
I found one utility dsrecords but it works on something called OSH and that doesne't work on my UNIX server. Is there any other way?
Thanks,
Manish
___________________
Manish Kulkarni
As you've noted, the .ds file doesn't contain the data, so removing it iwll leave orphaned datafiles. You should use the UNIX command orchadmin rm {dataset.ds} to remove both the descriptor as well as the associated data files. The orchadmin command also has a desc option to let you get such information as the number of rows in your dataset files.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
Hi Arndw,ArndW wrote:As you've noted, the .ds file doesn't contain the data, so removing it iwll leave orphaned datafiles. You should use the UNIX command orchadmin rm {dataset.ds} to remove bo ...
I have also tried same thing.
first i have done the settings :-
DSHOME=/opt/Ascential/DataStage/DSEngine
. $DSHOME/dsenv
DSBIN=$DSHOME/bin
export APT_CONFIG_FILE=/opt/Ascential/DataStage/Configurations/default.apt
export APT_ORCHHOME=/opt/Ascential/DataStage/PXEngine
export PATH=$PATH:$APT_ORCHHOME/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$APT_ORCHHOME/lib
export ORCHBIN=$APT_ORCHHOME/bin
After that I used that command in the same script
but it is giving error
"Could not load program orchadmin:
Dependent module liborchgeneralaix3.so could not be loaded.
Could not load module liborchgeneralaix3.so.
System error: No such file or directory"
Command used is Orchadmin delete filename
this file liborchgeneralaix3.so is their in /opt/Ascential/DataStage/PXEngine/bin
And a request we can't read your post ,can you give a post in way we can read that.
Hi Arndw,ArndW wrote:As you've noted, the .ds file doesn't contain the data, so removing it iwll leave orphaned datafiles. You should use the UNIX command orchadmin rm {dataset.ds} to remove bo ...
I have also tried same thing.
first i have done the settings :-
DSHOME=/opt/Ascential/DataStage/DSEngine
. $DSHOME/dsenv
DSBIN=$DSHOME/bin
export APT_CONFIG_FILE=/opt/Ascential/DataStage/Configurations/default.apt
export APT_ORCHHOME=/opt/Ascential/DataStage/PXEngine
export PATH=$PATH:$APT_ORCHHOME/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$APT_ORCHHOME/lib
export ORCHBIN=$APT_ORCHHOME/bin
After that I used that command in the same script
but it is giving error
"Could not load program orchadmin:
Dependent module liborchgeneralaix3.so could not be loaded.
Could not load module liborchgeneralaix3.so.
System error: No such file or directory"
Command used is Orchadmin delete filename
this file liborchgeneralaix3.so is their in /opt/Ascential/DataStage/PXEngine/bin
And a request we can't read your post ,can you give a post in way we can read that.
It is on AIX.After changing LD_LIBRARY_PATH to LIBPATH it is giving error "Could not load program orchadmin:kumar_s wrote:Is it for HP-UX or AIX?
Dependent module /home/apps/db2c/ddb2018/sqllib/lib/librwtool.a(librwtoo
l.o) could not be loaded.
Member librwtool.o is not found in archive"
And also i couldn't locate this file l"ibrwtool.o" but .a file is there
-
- Premium Member
- Posts: 72
- Joined: Thu Sep 04, 2003 5:01 am
- Location: UK & Europe
This is a fairly common requirement - 'housekeeping' after a run that creates large datasets for example. Note that 'orchadmin rm' can accept multiple filenames as arguments.
Here is the business part of my script to delete a defined list of datasets, you will need to modify the path names for your installation ...
. /DataStageApp/Ascential/DataStage/DSEngine/dsenv
export APT_CONFIG_FILE=/DataStageApp/Ascential/DataStage/Configurations/default.apt
export LIBPATH=/DataStageApp751_adc/Ascential/DataStage/PXEngine/lib:$LIBPATH
# Build lists of datasets to delete
filelist="PROD/ValIntermediate1"
filelist="$filelist PROD/ValIntermediate2"
filelist="$filelist PROD/ValIntermediate3"
filelist="$filelist PROD/Intermediate3P"
filelist="$filelist PROD/Intermediate4D"
filelist="$filelist SummaryTableFinal"
(etc)
# Run Orchadmin and delete the datasets
/DataStageApp751_adc/Ascential/DataStage/PXEngine/bin/orchadmin rm $filelist
Here is the business part of my script to delete a defined list of datasets, you will need to modify the path names for your installation ...
. /DataStageApp/Ascential/DataStage/DSEngine/dsenv
export APT_CONFIG_FILE=/DataStageApp/Ascential/DataStage/Configurations/default.apt
export LIBPATH=/DataStageApp751_adc/Ascential/DataStage/PXEngine/lib:$LIBPATH
# Build lists of datasets to delete
filelist="PROD/ValIntermediate1"
filelist="$filelist PROD/ValIntermediate2"
filelist="$filelist PROD/ValIntermediate3"
filelist="$filelist PROD/Intermediate3P"
filelist="$filelist PROD/Intermediate4D"
filelist="$filelist SummaryTableFinal"
(etc)
# Run Orchadmin and delete the datasets
/DataStageApp751_adc/Ascential/DataStage/PXEngine/bin/orchadmin rm $filelist
-
- Premium Member
- Posts: 72
- Joined: Thu Sep 04, 2003 5:01 am
- Location: UK & Europe
This is a fairly common requirement - 'housekeeping' after a run that creates large temporary or intermediate datasets for example.
Note that 'orchadmin rm' can accept multiple filenames as arguments.
Here is the business part of my script to delete a defined list of datasets, you will need to modify the path names for your installation ...
. /DataStageApp/Ascential/DataStage/DSEngine/dsenv
export APT_CONFIG_FILE=/DataStageApp/Ascential/DataStage/Configurations/default.apt
export LIBPATH=/DataStageApp751_adc/Ascential/DataStage/PXEngine/lib:$LIBPATH
# Build lists of datasets to delete
filelist="PROD/ValIntermediate1"
filelist="$filelist PROD/ValIntermediate2"
filelist="$filelist PROD/ValIntermediate3"
filelist="$filelist PROD/Intermediate3P"
filelist="$filelist PROD/Intermediate4D"
filelist="$filelist SummaryTableFinal"
(etc)
# Run Orchadmin and delete the datasets
/DataStageApp751_adc/Ascential/DataStage/PXEngine/bin/orchadmin rm $filelist
Hope this is useful
Note that 'orchadmin rm' can accept multiple filenames as arguments.
Here is the business part of my script to delete a defined list of datasets, you will need to modify the path names for your installation ...
. /DataStageApp/Ascential/DataStage/DSEngine/dsenv
export APT_CONFIG_FILE=/DataStageApp/Ascential/DataStage/Configurations/default.apt
export LIBPATH=/DataStageApp751_adc/Ascential/DataStage/PXEngine/lib:$LIBPATH
# Build lists of datasets to delete
filelist="PROD/ValIntermediate1"
filelist="$filelist PROD/ValIntermediate2"
filelist="$filelist PROD/ValIntermediate3"
filelist="$filelist PROD/Intermediate3P"
filelist="$filelist PROD/Intermediate4D"
filelist="$filelist SummaryTableFinal"
(etc)
# Run Orchadmin and delete the datasets
/DataStageApp751_adc/Ascential/DataStage/PXEngine/bin/orchadmin rm $filelist
Hope this is useful
-
- Premium Member
- Posts: 72
- Joined: Thu Sep 04, 2003 5:01 am
- Location: UK & Europe
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: