how to rename files after datastage job completed?

[qutesanju]

Hi All,
Q1
My requirement is to rername file with date timestamp appended after Datastage job completed successfully.

I know I can use a unix script but howto call unix script in Datastage job?
Or is there any way to do this in datastage job itself?
one option i have is

Q2
If this is successfull,can i attach this genearated file as an attchment ?

I have added this in the sequencer, @ the job parameter level - 'genegatedFile_':Oconv(DATE(),"D-YMD[4,2,2]" : @VM : "MCN"):OCONV(TIME(),"MTS": @VM : "MCN"):'.csv', it is generating the file with the current timestamp(For Example: generatedfile_20121119034051.csv). but the problem here is while attaching the files in the sequencer, it is not considering the timestamp. I have tried with different options but its not working.

[ray.wurlod]

1. Not in the DataStage job, because the file is still open while the job is running. However, you can rename the file in an after-job subroutine (for example ExecSH) providing the timestamp portion as a job parameter or in the command itself (using backquotes on the date command).

2. In the sequence grab the date from the upstream job parameter value.

[qutesanju]

i was looking for options of ExecSH ,but where shlud i add it in SEQ? or in individual job?

also i'm not sure how to use after job subroutine in DSv 8.5

[vamsi.4a6]

If you are using sequence use execute command activity to trigger the unix script.In unix script you can rename the file

[bhasds]

Hi qutesanju,
For this

in individual job?

As Ray has posted ExecSh option is avalable in the afterjob subroutine of a individual datastage job. Choose ExecSh option and in input value-

Code: Select all

mv generatedfile   generatedfile_`date +%Y%m%d%H%M%S`

And
Check the option- Only Run after-job subroutine on successfull job completion in the Job properties.

[robinlg33]

I got the rename to work for 1 file but can you generically rename all files in the folder (adding the timestamp) and moving to a new folder such as an Archive folder?

Thank you,
Robin

[chulett]

It can really depend on your O/S and release as some add 'rename' or 'mmv' as a command line function (in addition to mv) that are multiple file rename friendly. For a series of steps like renaming all files (which generally takes a for loop) and then archiving to a new folder, I'd suggest a script rather than trying to cram them all into one command line property.

A quick search for "unix rename multiple files" turned up a ton of discussions and code examples on how to code this. FYI.

[robinlg33]

I got the rename to work for 1 file but can you generically rename all files in the folder (appending the timestamp) and moving to a new folder such as an Archive folder? OR do you have to create separate Execute_Command stages to do separately. I'm not UNIX savy so need a little help here.

Thank you,
Robin

[chulett]

So a script is out of the question? Then probably best to move it from the 'After Job' area to Execute Command stages in a Sequence job as long as those are allowed at your site. And yes, I'd suggest separate Execute Command stages so you can check the execution of the first and only trigger the second if the first succeeds.

Did you do the search I noted? Lots of ways to rename multiple files but we'd really need to know the flavor of *nix you are on to provide more specific help. Can you enlist the help of someone with UNIX experience where you work? Very little about this task is specific to DataStage.

[ray.wurlod]

The UNIX command executed via ExecSH can be anything you can do on the UNIX command line. Therefore, for example, you could construct a pipeline based around a for loop picking up all the files in the directory.

[priyadarshikunal]

or find command with -exec option should do that in one line. If I understood it correctly.