Hi,
I have 2 questions and hope that some of you has already met this kind of problem
1. How can a sequencer job of datastage wait for a file that always change his name.
For example I receive sequential files from a system that give me files named with the creation date and an id,
like this: filename_yyyymmdd_hhmmss_nnnnn, where nnnnn is a progressive id.
The filename does not change but yyyymmdd_hhmmss_nnnnn change everytime.
2.In case the system send me 2 or more files at the same time, i have to process the one with the smaller id first.
Datastage Version 8.1
Thanks a lot to everybody,
Angelo.
How to wait for and process multiple and changing name file
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 16
- Joined: Fri Mar 28, 2003 4:36 am
- Location: Italy - Roma
Re: How to wait for and process multiple and changing name f
Hi Angelo,angelovaira wrote:Hi,
I have 2 questions and hope that some of you has already met this kind of problem
1. How can a sequencer job of datastage wait for a file that always change his name.
For example I receive sequential files from a system that give me files named with the creation date and an id,
like this: filename_yyyymmdd_hhmmss_nnnnn, where nnnnn is a progressive id.
The filename does not change but yyyymmdd_hhmmss_nnnnn change everytime.
2.In case the system send me 2 or more files at the same time, i have to process the one with the smaller id first.
Datastage Version 8.1
Thanks a lot to everybody,
Angelo.
1) I think we can use job parameters for the 'Wait for file' activity stage.
( i dont have access to datastage right now to double check, but will check and confirm)
2) If you have multiple files to process and you want to process them as per the order of ID, you can follow this:
-> write a unix script that will return the list of
files in that source dir in the order you want and get the list as comma separated values.
( i assume that only unprocessed files are in source dir)
->You can call this script in the sequence using execute command stage.
->Use a loop activity in the sequence to call DS job in loop, with the counter value given by the comma seperated list of file names i.e output of the unix script.
let me know if it helps or you need more info.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
You can use an Execute Command activity to execute an ls -1 command and pipe its results through sort. Then use a User Variables activity to change the field marks in the command output to commas, and the result becomes your "list of things" list (comma delimited) in a Start Loop activity. Use a Nested Condition activity to answer the question are there any files there at all?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 16
- Joined: Fri Mar 28, 2003 4:36 am
- Location: Italy - Roma
-
- Participant
- Posts: 16
- Joined: Fri Mar 28, 2003 4:36 am
- Location: Italy - Roma
In execute command activity in a sequencer
Try sh-c with argument
to get a comma seperated list, I use it my process, The first value in the output list is the oldest created file.
You may want to sort further on the sequence number- maybe using a routine or a sort command.
Regards
Ramesh
Try sh-c with argument
Code: Select all
'ls -trx #FILEDIR##FileNamePattern# | tr "\n" ","'
You may want to sort further on the sequence number- maybe using a routine or a sort command.
Regards
Ramesh