Sequential File: Overwrite problem

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
new_dsuser
Participant
Posts: 5
Joined: Sat Apr 17, 2010 10:56 am

Sequential File: Overwrite problem

Post by new_dsuser »

Hello,

I have a job which writes some data into a table as well as a sequential file.
The property 'file update mode' is set to 'Overwrite'.

There is a check in one of the transformers to abort the job if a certain constraint condition succeeds. This check is done to abort the job when it is run twice. Now my problem is, even when the job aborts, the file which was created by the last run gets overwritten . Is there a way to not overwrite the file in case the job aborts?

Any help would be much appreciated.

If the issue is not clear/if you would like to know more details, please let me know

Cheers!
priyadarshikunal
Premium Member
Premium Member
Posts: 1735
Joined: Thu Mar 01, 2007 5:44 am
Location: Troy, MI

Post by priyadarshikunal »

you can use a before job subroutine to rename that file and in case it aborts, use a after job subroutine to rename it back to orignal name.
Priyadarshi Kunal

Genius may have its limitations, but stupidity is not thus handicapped. :wink:
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

In other words, no. The moment the job starts and the stage 'opens' the file is overwritten.
-craig

"You can never have too many knives" -- Logan Nine Fingers
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

One suggestion would be to write to a different file name, then in the after-job subroutine call (which can be set to only execute when the job completes successfully) you execute a shell command to move the temporary file to the permanent file name and overwrite it.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Right, in some fashion you'd need to preserve the original file. You could use an After Job routine to check the INTERIM status and, if the job was aborting, swap old back for new. Or, as noted, write to a new name and if the job was going to finish normally, rename the new name to the old name.
-craig

"You can never have too many knives" -- Logan Nine Fingers
new_dsuser
Participant
Posts: 5
Joined: Sat Apr 17, 2010 10:56 am

Post by new_dsuser »

Priyadarshi, Craig and ArndW,

Thanks! I used a cp command as an after job sub-routine to replace the actual file which would get executed only when the job succeeds.

Really appreciate the help

Thanks!
Post Reply