Clearing the temporary files in Datastage

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
arun_im4u
Premium Member
Premium Member
Posts: 104
Joined: Mon Nov 08, 2004 8:42 am

Clearing the temporary files in Datastage

Post by arun_im4u »

Hi,

When I run a job which has a hashed file stage or a sorter stage it creates a lot of temporary files under the datastage home directory. How do i clear them. It takes up a lot of my disk space. Can i delete these files from the client side???

ThankYou.
changming
Participant
Posts: 68
Joined: Wed Oct 13, 2004 3:35 am

delete temp file

Post by changming »

in transformer, after stage rountine, excute sh script,
rm file_path/file_name
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Well... :?

The Sort stage by default puts the temporary files it uses in the current Project, which is not the best place for them. Jobs that generate a large volume of temp files can fill your Project partition and then you'll find yourself in all kinds of hurt.

On the General / Properties tab of the stage there is a Temporary Directory option. Change it to something appropriate to your system like /tmp and it will put all of its files there. Normally the stage cleans up after itself, but when it doesn't it is best to have the leftovers off in someplace like /tmp.

As to the Hash files, they too by default go into your Project, but do not make use of any temporary files. The files you are seeing are (more than likely) the hash files themselves, so I'd be a little careful about just going out and deleting things. :wink:

BTW, all of the temporary sort files start with 'soa', from what I recall. Those would be safe to delete if the job that generated them is not running.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply