Passing pattern as a Job parameter

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
news78
Participant
Posts: 44
Joined: Fri Jul 07, 2006 1:37 pm

Passing pattern as a Job parameter

Post by news78 »

I have a datastage job that reads a file using a Sequential File stage and processes it. Now the file name is dynamic and follows a pattern: feedata_<month>_1_<yyyy>.txt
e.g.
feedata_January_1_2007.txt


Does datastage provide any option so that I can provide feedata_<month>_1_<yyyy>.txt as job parameter and it replaces the date part and proceeds ahead?

I checked the folder stage but seems its only for wildcards and processing multiple files. I searched the forums, and one option is to replace(in a shell script) the pattern before invoking the datastage job.

Thanks for your help.
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Do you want to replace the Month part and the yyyy part automatically or provide parameter values to them. The latter is easy. But for the former you will have to do OCONV() on Date() to get Month as you require and the OCONV() on Date() with conversion code "DY[4]" for yyyy, in the derivation of job parameter in a job activity in a sequence job.
I hope I was'nt confusing :?
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

It ought to be possible, but you would need to guarantee that only one file matches the regular expression if you are planning to use it as the filename property directly.

Otherwise, specify a filter command of cat feedata_*_1_????.txt and use /dev/null as the file name. The stage will read stdout from the filter command, and ignore the file name property.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
news78
Participant
Posts: 44
Joined: Fri Jul 07, 2006 1:37 pm

Post by news78 »

ray.wurlod wrote:It ought to be possible, but you would need to guarantee that only one file matches the regular expression if you are planning to use it as the filename property directly.

Otherwise, specify a filt ...
Yes only one file will match the pattern. If so is there any way to achive this?
In case I use a filter, can I use a sed command somthing like:

#jobparam_filename#|sed -e "s/month/date/"

where date will be appropriate pattern.
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Ray already did give you a much simpler solution.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
Post Reply