I have a job which needs to loop through the files in a directory ,read all the files sequentially in the order which they were created,These files have a time stamp in their name .
Does any body has an idea about it.
Thanks for the help
Arif
Looping through the files in directory
Moderators: chulett, rschirm, roy
-
- Charter Member
- Posts: 87
- Joined: Sat Mar 18, 2006 11:45 pm
There are at least two basic approaches. One way is to use a UNIX command one liner to get the filelist and cat all the results to stdin for the DataStage PX job. Another is to work through a DataStage Sequence and get the directory contents and call up your processing job once for each input file.
I like the Job Sequence method, but a PX job will allow you execute the input stream command easily. Also, you can use a file pattern to automatically load all the files in the sequetial stage, but you cannot order the files.
I like the Job Sequence method, but a PX job will allow you execute the input stream command easily. Also, you can use a file pattern to automatically load all the files in the sequetial stage, but you cannot order the files.
-
- Charter Member
- Posts: 87
- Joined: Sat Mar 18, 2006 11:45 pm
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
The StartLoop activity can be set to process a "list of things". The list can be generated by any means, such as a regular expression or a script. Read the manual chapter or on-line help on StartLoop activity.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Use an Execute Command activity to generate a delimited list of file names.
Use its output as the delimited list in the Start Loop activity, and specify there that it's a "list of things" loop and the appropriate delimiter.
Downstream of that, the index variable ($Control, or something like that) will contain one file name that can be delivered as a parameter. Put in all your other required activities, bring them into an End Loop activity and link that back to the Start Loop activity to complete the loop. An output trigger (unconditional) from the End Loop activity is where control goes once the loop has been completed.
Use its output as the delimited list in the Start Loop activity, and specify there that it's a "list of things" loop and the appropriate delimiter.
Downstream of that, the index variable ($Control, or something like that) will contain one file name that can be delivered as a parameter. Put in all your other required activities, bring them into an End Loop activity and link that back to the Start Loop activity to complete the loop. An output trigger (unconditional) from the End Loop activity is where control goes once the loop has been completed.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.