how to rename files after datastage job completed?
Moderators: chulett, rschirm, roy
how to rename files after datastage job completed?
Hi All,
Q1
My requirement is to rername file with date timestamp appended after Datastage job completed successfully.
I know I can use a unix script but howto call unix script in Datastage job?
Or is there any way to do this in datastage job itself?
one option i have is
Q2
If this is successfull,can i attach this genearated file as an attchment ?
I have added this in the sequencer, @ the job parameter level - 'genegatedFile_':Oconv(DATE(),"D-YMD[4,2,2]" : @VM : "MCN"):OCONV(TIME(),"MTS": @VM : "MCN"):'.csv', it is generating the file with the current timestamp(For Example: generatedfile_20121119034051.csv). but the problem here is while attaching the files in the sequencer, it is not considering the timestamp. I have tried with different options but its not working.
Q1
My requirement is to rername file with date timestamp appended after Datastage job completed successfully.
I know I can use a unix script but howto call unix script in Datastage job?
Or is there any way to do this in datastage job itself?
one option i have is
Q2
If this is successfull,can i attach this genearated file as an attchment ?
I have added this in the sequencer, @ the job parameter level - 'genegatedFile_':Oconv(DATE(),"D-YMD[4,2,2]" : @VM : "MCN"):OCONV(TIME(),"MTS": @VM : "MCN"):'.csv', it is generating the file with the current timestamp(For Example: generatedfile_20121119034051.csv). but the problem here is while attaching the files in the sequencer, it is not considering the timestamp. I have tried with different options but its not working.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
1. Not in the DataStage job, because the file is still open while the job is running. However, you can rename the file in an after-job subroutine (for example ExecSH) providing the timestamp portion as a job parameter or in the command itself (using backquotes on the date command).
2. In the sequence grab the date from the upstream job parameter value.
2. In the sequence grab the date from the upstream job parameter value.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Hi qutesanju,
For this And
Check the option- Only Run after-job subroutine on successfull job completion in the Job properties.
For this
As Ray has posted ExecSh option is avalable in the afterjob subroutine of a individual datastage job. Choose ExecSh option and in input value-in individual job?
Code: Select all
mv generatedfile generatedfile_`date +%Y%m%d%H%M%S`
Check the option- Only Run after-job subroutine on successfull job completion in the Job properties.
globally renaming all files in a folder
I got the rename to work for 1 file but can you generically rename all files in the folder (adding the timestamp) and moving to a new folder such as an Archive folder?
Thank you,
Robin
Thank you,
Robin
Robin
It can really depend on your O/S and release as some add 'rename' or 'mmv' as a command line function (in addition to mv) that are multiple file rename friendly. For a series of steps like renaming all files (which generally takes a for loop) and then archiving to a new folder, I'd suggest a script rather than trying to cram them all into one command line property.
A quick search for "unix rename multiple files" turned up a ton of discussions and code examples on how to code this. FYI.
A quick search for "unix rename multiple files" turned up a ton of discussions and code examples on how to code this. FYI.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
globally renaming all files in a folder
I got the rename to work for 1 file but can you generically rename all files in the folder (appending the timestamp) and moving to a new folder such as an Archive folder? OR do you have to create separate Execute_Command stages to do separately. I'm not UNIX savy so need a little help here.
Thank you,
Robin
Thank you,
Robin
Robin
So a script is out of the question? Then probably best to move it from the 'After Job' area to Execute Command stages in a Sequence job as long as those are allowed at your site. And yes, I'd suggest separate Execute Command stages so you can check the execution of the first and only trigger the second if the first succeeds.
Did you do the search I noted? Lots of ways to rename multiple files but we'd really need to know the flavor of *nix you are on to provide more specific help. Can you enlist the help of someone with UNIX experience where you work? Very little about this task is specific to DataStage.
Did you do the search I noted? Lots of ways to rename multiple files but we'd really need to know the flavor of *nix you are on to provide more specific help. Can you enlist the help of someone with UNIX experience where you work? Very little about this task is specific to DataStage.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
The UNIX command executed via ExecSH can be anything you can do on the UNIX command line. Therefore, for example, you could construct a pipeline based around a for loop picking up all the files in the directory.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Premium Member
- Posts: 1735
- Joined: Thu Mar 01, 2007 5:44 am
- Location: Troy, MI