automate CLEAR.FILE &PH&

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
luca
Participant
Posts: 81
Joined: Wed May 14, 2003 11:34 pm

automate CLEAR.FILE &PH&

Post by luca »

I would like to automate the deletion of the files in the &PH& folder.
First I wanted to write a job which would execute a "CLEAR.FILE &PH&" and I would have run this job once a day, when every job is finished and before any other job starts. I have done this job and it works fine. Then I have read it is not safe to do it when DS jobs are running and it is better to do it from the OS (AT command for example). Obviously, the job that do the clear command is running when the command is launched but I have been testing the job successfully as it would not allow to delete the files that correspond to process pending.
If it is safe, I would prefer to keep the job solution as it would be launched by a "daily job" and would run before any other job, what ever time this daily job is launched.
If we go for the OS solution, we would then have to synchronize the time when this command would be executed with the time when the DS jobs would start.
Thanks in advance for your advises.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

You're in luck! Because you're on a Windows platform. Windows will not allow you to delete a file that's in use - you get a "sharing violation". So if there are jobs running, your CLEAR.FILE &PH& command will generate sharing violation warnings, but things will be safe otherwise.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
luca
Participant
Posts: 81
Joined: Wed May 14, 2003 11:34 pm

Post by luca »

Thanks a lot for your answer, Ray.
For once that it is lucky to be on Windows !
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Cynicism 101

Post by ray.wurlod »

May as well post this thought before one of the other cynics does.

Once you start relying on something like this, they'll change it! :x
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

I don't think this is such a bad idea on UNIX. If you do it when no jobs are running. All you lose is debuging a job that fails hard. I would think this directory does not get filled up that much. I would do it once a week. I would resize &PH& to type 19 as well. If you have weekly jobs which run over the weekend then you might want to run this on Monday after you have checked the logs.
Mamu Kim
Teej
Participant
Posts: 677
Joined: Fri Aug 08, 2003 9:26 am
Location: USA

Post by Teej »

On Unix, do this:

Code: Select all

# Clean up Project's junk.
find /[path]/Projects/ -type d -name "&PH&" -print |\
while read obj
do
    find $obj -type f -atime +10 -exec \
        rm -f {} \;
done
You will need to edit the [path] into the path to your Projects folder.

What this does is search for all files that has not been touched for over 10 days, and show them the door discretely. Don't want to disturb the party, after all.

I have it set as part of a running cron job cleaning up everything else (scratch space, datasets, temp, removing CORE dumps, et cetera...)

It's one of the few things I will carry with me to all of my jobs. :)
The goal I'm trying to achieve -- minimize maintenance so much that I could focus on developing! :)

-T.J.
Developer of DataStage Parallel Engine (Orchestrate).
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

TJ

This works well if you RESIZE &PH& 19. The default for the &PH& and &SAVEDLISTS& hash files or directories is type 1. The difference is this, type 1 is based on a short name filesystem. In real old versions of UNIX the longest name you could have was 14 characters. So what they did was create directory every 14 characters and start with the rest of the name. So A23456789012345 becomes something like &PH&/A2345678901234/5. Ugly huh? Type 19 uses longnames.

So what your script does is leave &PH&/A2345678901234 behind. It deletes the filenamed 5 in this directory. This is usually not a problem. Just wanted you to know type 19 is cleaner. The format of &PH& ids is (process name) underscore (date started internal format) underscore (time started). You could create dictionary items to delete these as well. I hope this helps someone.
Mamu Kim
Post Reply