We have a sequence calling a number of jobs which create several unique Hashed and text files for each file that is passed to the sequence.
We have been "asked" by production management to follow the McDonalds maxim and "Clean as we go".
From what I have read on this, executing a simple "CLEAR.FILE" at the end of the sequence will not be sufficient as the disk space is retained (even though the DATA.30 and OVER.30 are set to minimal limits).
I have thought about creating a routine at the end of the sequence which executes a series of Universe and Unix commands to delete the hashed files and text files.
Is this the best way to approach this ?
Deleting work (Hashed and Seq files) after job execution
Moderators: chulett, rschirm, roy
Deleting work (Hashed and Seq files) after job execution
Colin Larcombe
-------------------
Certified IBM Infosphere Datastage Developer
-------------------
Certified IBM Infosphere Datastage Developer
If you do a CLEAR.FILE and then a "RESIZE {fname} * * *" the OVER.30 will also get set back to it's original size. If you have a MINIMUM.MODULO set your files will not shrink to a couple of Kb, either.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
An after-job subroutine, or a routine or command (shell script) invoked from a job sequence, are the obvious choices. In either case you can pass a list of names of things to be deleted. Don't forget, in the case of dynamic hashed files, that you'll need a recursive deletion command.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 9
- Joined: Tue Jan 08, 2008 7:21 am
- Location: Chennai
Re: Deleting work (Hashed and Seq files) after job execution
Hope you are running in UNIX platform. If so use "rm" command with options available and place all the commands in a file. So the file will have the coding to remove all the files that are created. In Sequence you have an activity called Execute Command. Use that activity and make a call the the Shell script that is created.
Regards,
Raja Soundararajan
Regards,
Raja Soundararajan
clarcombe wrote:We have a sequence calling a number of jobs which create several unique Hashed and text files for each file that is passed to the sequence.
We have been "asked" by production management to follow the McDonalds maxim and "Clean as we go".
From what I have read on this, executing a simple "CLEAR.FILE" at the end of the sequence will not be sufficient as the disk space is retained (even though the DATA.30 and OVER.30 are set to minimal limits).
I have thought about creating a routine at the end of the sequence which executes a series of Universe and Unix commands to delete the hashed files and text files.
Is this the best way to approach this ?
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Deleting Hashed Files
If the hashed file was created in a project (account), use CREATE.FILE from within the DataStage environment.
If the hashed file was created as a dynamic hashed file in a directory, use a recursive delete command (rm -rf in UNIX, DEL /S in Windows). This deletes the three files DATA.30, OVER.30 and .Type30, as well as the directory that "is" the hashed file.
If the hashed file was created as a static hashed file in a directory you can use a non-recursive delete command, but the recursive one will work just as well.
If you have used SETFILE or any other means to create a VOC pointer to the deleted hashed file you need to delete that pointer. This is a regular query of the form
Caution: Never used DELETE on VOC without a WHERE clause! Your project will immedately become unusable. The constraint on "TYPE" above is not strictly necessary - I include it as a safety measure.
If the hashed file was created as a dynamic hashed file in a directory, use a recursive delete command (rm -rf in UNIX, DEL /S in Windows). This deletes the three files DATA.30, OVER.30 and .Type30, as well as the directory that "is" the hashed file.
If the hashed file was created as a static hashed file in a directory you can use a non-recursive delete command, but the recursive one will work just as well.
If you have used SETFILE or any other means to create a VOC pointer to the deleted hashed file you need to delete that pointer. This is a regular query of the form
Code: Select all
DELETE FROM VOC WHERE "TYPE" in ('F','Q') AND "@ID" = 'hashedfilename';
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.