DataStage Admin tasks

[DS4DRIVER]

Hi,
I want to know the Tasks of a person who is a DataSTage Administrator for a Data Warehouse Implementation in an Organization.
Aditionally i want to know the things to be taken care of before and after upgrading from Ver 4 to 6.

Thanks.

[ray.wurlod]

The Administrator's primary role is installation and setting project-wide defaults, basically the tasks on the Administrator client's tabs. After that, the DataStage engine is designed to look after itself reasonably well, though there are a few minor housekeeping tasks, such as deleting old files from the &PH& directory, that will benefit performance.
As for the upgrade, the main thing is to make sure you have everything backed up, and have verified that the backup was successful (!), before proceeding. Installing the new version of the server will upgrade any components that require it. It may be beneficial to re-index the repository (Cleanup Project from Administrator) before upgrading.

[vmcburney]

When it comes to the support of a production DataStage data warehouse environment a lot depends on how well the DataStage jobs were designed. If you've followed an approach often talked about on this forum that uses best practices the administrator should have just a small number of tasks outlined by Ray. If you are missing some of these best practices, such as comprehensive error trapping and success and failure email notification, your DataStage administrator and/or Operator may find themselves spending time each day logging onto the Director and checking job logs, logging onto the database and checking row counts and even staying up late to make sure an important extract finishes without errors.

The first step in upgrading from version 4 to version 6 is to exchange your version 6 CD for a version 7 CD.

[ariear]

The DS administrator should know to use DS.TOOLS (Unlock locked resources and so on..), take care of regular backup procedures ,Monitor the projects disk space, Creating projects/permissions and even sometimes he's in charge of the CM (Version control).
probably I forgot some issues that other's will remember

[Teej]

After that, the DataStage engine is designed to look after itself reasonably well, though there are a few minor housekeeping tasks, such as deleting old files from the &PH& directory, that will benefit performance.

When I saw the word "performance" and "benefit" in the same sentence, I had to put together a script. Just tested it (carefully), and ran it (several times) and now running full scale run on dev box.

Code: Select all

find /...../Ascential/DataStage/Projects/ -type d -name "&PH&" -print |\
while read obj
do
    find $obj -type f -atime +10 -exec \
        rm -f {} \;
done

You will need to replace the "....." with the appropriate directory. You could improve portability by using $dshome or whatever flavor you choose.

What this does is find all "&PH&" on all project folders, then pull all files that are older than 10 days (when I say older -- I mean it have not been touched, looked, changed, toyed with, or even maimed. Yes, Virginia, not even maimed.) Those file are promptly sent stage left to the /dev/null heaven in the sky. Everything else SHOULD be left alone.

One disadvantage of this -- find is VERY slow on large installations (due to heapload of files and logs beind created), so if you have a better alternative to suggest, I am open to it.

I must warn everyone -- this is dangerous stuff. If you do it wrong, you lose your company's lifeblood, and likely get fired if there's no backup. Test it carefully on a safe directory.

The above code works on Tru64 5.1a using KSH.

-T.J.