Dataset Mgmt.

gopskrish · Post by **gopskrish** » Mon Jun 13, 2005 5:31 am

Hello Everybody,

I want to rename and delete dataset files which are 7 days old ?
Can i use Unix shell scripts to do that or Is there a better way of handling it ? I heard from one of my friend that Datastage automatically deletes the files in a periodical basis in the latest version. Is that true ? Let me know.

kcshankar · Post by **kcshankar** » Mon Jun 13, 2005 5:56 am

Hi,
You can do that in Datastage Manager.

It is given in the Datastage Documentation that,we cannot use Unix cp or rm commands to copy or delete a dataset becuse,Datastage represents a single data set with multiple files.Using rm simply removes the descriptor file,leaving the much larger files behind.

gopskrish · Post by **gopskrish** » Mon Jun 13, 2005 6:04 am

Hi,

Thanks a lot. I would like to know the equivalent commands in Datastage if i dont want to use Unix commands, and how to specify the value (number of days , like 7 in my case) for deletion or renaming.

If u do have some scripts based on that, do share it with me so that i can see if i can implement the same from my end.

cheers,
gops

ArndW · Post by **ArndW** » Mon Jun 13, 2005 6:52 am

kcshankar wrote:Hi,
You can do that in Datastage Manager.

It is given in the Datastage Documentation that,we cannot use Unix cp or rm commands to copy or delete a dataset becuse,Datastage represents a single data set with multiple files.Using rm simply removes the descriptor file,leaving the much larger files behind.

what you would do from UNIX is "orchadmin rm {yourfile}", this will remove the dataset data files as well as the dataset descriptor.

elavenil · Post by **elavenil** » Mon Jun 13, 2005 7:08 am

Hi Arnd,

Is "orchadmin rm <the descriptor file>" equivalent to deleting the dataset from Dataset Management.

How would you set up OrchAdmin in Unix?

Pls share with us.

Regards
Saravanan

kcshankar · Post by **kcshankar** » Mon Jun 13, 2005 7:08 am

Hi ,
Sorry gops,i have not done this in Shell Scripts.

Even I have few doubts.

1.Is the Blocks size of a file given in Datastage Manager and Unix are same.
For a same Dataset file,
in Unix it is showing 16 Blocks
in Ds manager it is showing 1 Block.

2.What is happening when we delete .ds file.
First I deleted .ds file in a particular directory then I checked the size of the directory,I found that the file is deleted completely.
But in the document it is given that,using rm simply removes the descriptor file,leaving the much larger files behind.

Thanks in advance
kcs

gopskrish · Post by **gopskrish** » Mon Jun 13, 2005 7:08 am

Hi Arnd,

Thanks a lot. I hope that i need administrative privilage to execute that piece of code. Correct me if i am wrong.

Best Regards,
Gops

ArndW · Post by **ArndW** » Mon Jun 13, 2005 7:14 am

elavenil wrote:Hi Arnd,

Is "orchadmin rm <the descriptor file>" equivalent to deleting the dataset from Dataset Management.

How would you set up OrchAdmin in Unix?

Pls share with us.

Regards
Saravanan

No magic to that - if you set up your user's environment with the correct settings for PX then the orchadmin command will be in your search path. Just call up orchadmin and see what happens.

ray.wurlod · Post by **ray.wurlod** » Mon Jun 13, 2005 3:10 pm

A persistent DataSet comprises the control file and one or more data files. You can cat the control file to determine the location of the data files. Then you can process this list using find (with -ls option) to determine which ones are old enough. Then use appropriate commands to move or remove all parts of the DataSet.

If all you do is remove the control file (the one with a ".ds" suffix) then all the data files will hang around forever and you won't know to which DataSet each belongs.

gopskrish · Post by **gopskrish** » Tue Jun 14, 2005 2:23 am

Hi Ray,

Can you please help me out on how to find the location of datafile using the cat ? I am more of a beginner in Unix. Please share your thoughts.

Best Regards,
Gops

ArndW · Post by **ArndW** » Tue Jun 14, 2005 2:30 am

gopskrish,

if you do a cat <mydataset.ds> or more <mydataset.ds> at unix you will, apart from quite a bit of other text, see one or more UNIX paths and filenames. You can then attach to that directory to delete the datafiles.

Your PX installation will also contain default locations for these dataset files; but in the DataSet management tool you can select your dataset and then click on each segment file to see where on the UNIX system that particular node has been placed.

gopskrish · Post by **gopskrish** » Tue Jun 14, 2005 2:34 am

Hi ArndW,

Thanks a lot for an immediate response. I will proceed accordingly. Your timely help is verymuch appreciated.

cheers,
gopskrish

ray.wurlod · Post by **ray.wurlod** » Tue Jun 14, 2005 3:05 pm

Not much I can add to Arnd's response. The default location is a directory called Datasets.

gopskrish · Post by **gopskrish** » Wed Jun 15, 2005 2:25 am

Hi,

I can able to delete old files using unix command based on the number passed but when i replace it with orchadmin delete option it is not deleting files. Herewith I am attaching the command which i use for testing. Correct me if it is wrong.

It is executing fine if the command is
find /detld1/etl/ascential/ascential/DataStage/Projects/CTI_London/IAM/staging/ -name "*.ds" -mtime +5 -print -exec compress {} \;

But when i use orchadmin in place of that it is not doing the job

find /detld1/etl/ascential/ascential/DataStage/Projects/CTI_London/IAM/staging/ -name "*.ds" -mtime +5 -print -exec orchadmin delete {} \; Looking for an earlier reply.

cheers,
gopskrish

gopskrish · Post by **gopskrish** » Wed Jun 15, 2005 2:57 am

Hi ,

I am very sorry for sending my previous mail. The same command is executing fine and I had a small typo error when i ran it lasttime.
Sorry onceagain to all.

cheers,
gopskrish