Deleting a dataset

manishsk · Post by **manishsk** » Mon Mar 27, 2006 12:20 am

Hi everybody,

I have one requirement where we need to delete DataSet through UNIX.

My Observation is:
DataSet stage creates a .ds file which is like a pointer to Data file(s) (which are binary) stored at particular location.

So if I remove the .ds file and the data file will that be enough.
Also I guess there can be many datafiles will be created for one dataset so how tho find all the related datafiles.
(Note: These Data files are in binary format.)

There is one utitlity in DataStage which can do these things i.e. DataSet Management Utitlity but this is a client based utility. And the requirement is we need to do it on server/ using UNIX shell script.

Also we need to count number of records stored in a dataset.
I found one utility dsrecords but it works on something called OSH and that doesne't work on my UNIX server. Is there any other way?

Thanks,
Manish
___________________
Manish Kulkarni

ArndW · Post by **ArndW** » Mon Mar 27, 2006 12:39 am

As you've noted, the .ds file doesn't contain the data, so removing it iwll leave orphaned datafiles. You should use the UNIX command orchadmin rm {dataset.ds} to remove both the descriptor as well as the associated data files. The orchadmin command also has a desc option to let you get such information as the number of rows in your dataset files.

anujgarg · Post by **anujgarg** » Mon Mar 27, 2006 12:57 am

ArndW wrote:As you've noted, the .ds file doesn't contain the data, so removing it iwll leave orphaned datafiles. You should use the UNIX command orchadmin rm {dataset.ds} to remove bo ...

Hi Arndw,

I have also tried same thing.
first i have done the settings :-
DSHOME=/opt/Ascential/DataStage/DSEngine
. $DSHOME/dsenv
DSBIN=$DSHOME/bin

export APT_CONFIG_FILE=/opt/Ascential/DataStage/Configurations/default.apt
export APT_ORCHHOME=/opt/Ascential/DataStage/PXEngine
export PATH=$PATH:$APT_ORCHHOME/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$APT_ORCHHOME/lib
export ORCHBIN=$APT_ORCHHOME/bin
After that I used that command in the same script
but it is giving error

"Could not load program orchadmin:
Dependent module liborchgeneralaix3.so could not be loaded.
Could not load module liborchgeneralaix3.so.
System error: No such file or directory"

Command used is Orchadmin delete filename

this file liborchgeneralaix3.so is their in /opt/Ascential/DataStage/PXEngine/bin

And a request we can't read your post ,can you give a post in way we can read that.

anujgarg · Post by **anujgarg** » Mon Mar 27, 2006 1:00 am

ArndW wrote:As you've noted, the .ds file doesn't contain the data, so removing it iwll leave orphaned datafiles. You should use the UNIX command orchadmin rm {dataset.ds} to remove bo ...

Hi Arndw,

I have also tried same thing.
first i have done the settings :-
DSHOME=/opt/Ascential/DataStage/DSEngine
. $DSHOME/dsenv
DSBIN=$DSHOME/bin

export APT_CONFIG_FILE=/opt/Ascential/DataStage/Configurations/default.apt
export APT_ORCHHOME=/opt/Ascential/DataStage/PXEngine
export PATH=$PATH:$APT_ORCHHOME/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$APT_ORCHHOME/lib
export ORCHBIN=$APT_ORCHHOME/bin
After that I used that command in the same script
but it is giving error

"Could not load program orchadmin:
Dependent module liborchgeneralaix3.so could not be loaded.
Could not load module liborchgeneralaix3.so.
System error: No such file or directory"

Command used is Orchadmin delete filename

this file liborchgeneralaix3.so is their in /opt/Ascential/DataStage/PXEngine/bin

And a request we can't read your post ,can you give a post in way we can read that.

kumar_s · Post by **kumar_s** » Mon Mar 27, 2006 1:28 am

Is it for HP-UX or AIX?

anujgarg · Post by **anujgarg** » Mon Mar 27, 2006 1:46 am

kumar_s wrote:Is it for HP-UX or AIX?

It is on AIX.After changing LD_LIBRARY_PATH to LIBPATH it is giving error "Could not load program orchadmin:
Dependent module /home/apps/db2c/ddb2018/sqllib/lib/librwtool.a(librwtoo
l.o) could not be loaded.
Member librwtool.o is not found in archive"

And also i couldn't locate this file l"ibrwtool.o" but .a file is there

SettValleyConsulting · Mon Mar 27, 2006 8:36 am

This is a fairly common requirement - 'housekeeping' after a run that creates large datasets for example. Note that 'orchadmin rm' can accept multiple filenames as arguments.

Here is the business part of my script to delete a defined list of datasets, you will need to modify the path names for your installation ...

. /DataStageApp/Ascential/DataStage/DSEngine/dsenv
export APT_CONFIG_FILE=/DataStageApp/Ascential/DataStage/Configurations/default.apt
export LIBPATH=/DataStageApp751_adc/Ascential/DataStage/PXEngine/lib:$LIBPATH

# Build lists of datasets to delete

filelist="PROD/ValIntermediate1"
filelist="$filelist PROD/ValIntermediate2"
filelist="$filelist PROD/ValIntermediate3"
filelist="$filelist PROD/Intermediate3P"
filelist="$filelist PROD/Intermediate4D"
filelist="$filelist SummaryTableFinal"

(etc)

# Run Orchadmin and delete the datasets
/DataStageApp751_adc/Ascential/DataStage/PXEngine/bin/orchadmin rm $filelist

SettValleyConsulting · Mon Mar 27, 2006 8:39 am

This is a fairly common requirement - 'housekeeping' after a run that creates large temporary or intermediate datasets for example.

Note that 'orchadmin rm' can accept multiple filenames as arguments.

Here is the business part of my script to delete a defined list of datasets, you will need to modify the path names for your installation ...

. /DataStageApp/Ascential/DataStage/DSEngine/dsenv
export APT_CONFIG_FILE=/DataStageApp/Ascential/DataStage/Configurations/default.apt
export LIBPATH=/DataStageApp751_adc/Ascential/DataStage/PXEngine/lib:$LIBPATH

# Build lists of datasets to delete

filelist="PROD/ValIntermediate1"
filelist="$filelist PROD/ValIntermediate2"
filelist="$filelist PROD/ValIntermediate3"
filelist="$filelist PROD/Intermediate3P"
filelist="$filelist PROD/Intermediate4D"
filelist="$filelist SummaryTableFinal"

(etc)

# Run Orchadmin and delete the datasets
/DataStageApp751_adc/Ascential/DataStage/PXEngine/bin/orchadmin rm $filelist

Hope this is useful

SettValleyConsulting · Mon Mar 27, 2006 8:41 am

Apologies for the duplicate post ... I thought that the first version had disappeared into the bit bucket ...

chulett · Post by **chulett** » Mon Mar 27, 2006 9:25 am

No problem. Keep in mind the fact that, until someone replies, you can delete the last post in a thread if it is yours. Now that we've both replied since then, you are stuck with them.

vigneshra · Post by **vigneshra** » Tue Jun 19, 2007 7:38 am

Is it necessary that the exact configuration file name used to create dataset is set to APT_CONFIG_FILE, while deleting that dataset using orchadmin command? What will happen if any other file name is set?

balajisr · Post by **balajisr** » Tue Jun 19, 2007 7:47 am

Is it necessary that the exact configuration file name used to create dataset is set to APT_CONFIG_FILE, while deleting that dataset using orchadmin command?

Yes.

vigneshra · Post by **vigneshra** » Tue Jun 19, 2007 7:51 am

You can call me dumb :D
But why do orchadmin command require APT_CONFIG_FILE set with the appropriate config file name, when the dataset descriptor file having the entire node configuration information?

mandyli · Post by **mandyli** » Tue Jun 19, 2007 10:29 am

Hi vigneshra

You are correct for running orchadmin APT_CONFIG_FILE is not needed.

Thanks
Man

ray.wurlod · Post by **ray.wurlod** » Tue Jun 19, 2007 3:16 pm

Oh yes it is - the executable checks that both APT_CONFIG_FILE and APT_ORCHHOME are set.