Page 1 of 1

How could I process all files from a directory?

Posted: Sun May 14, 2006 2:23 pm
by tiagogen
Hi there!
I am newbie in DataStage and in this forum as well.
My doubt is how can I get all files from a directory with a determinate standard.
For example, I want to process all files from a directory that starts with "PCBR*".
This directory comes from a parameter.

The way I found to do that, is using the job sequence.
I execute a find(unix command) , and I call the job once for file.
Anyone has a better way to do that?

thanks a bunch!
(sorry my English mistakes)

Posted: Sun May 14, 2006 2:58 pm
by ray.wurlod
Welcome aboard. :D

A job sequence is the best way. A StartLoop activity can set up a "list of things" loop based on a regular expression.

Code: Select all

  Pre-loop activity ---->  StartLoop  ---->  Within-loop activities  ---->  EndLoop  ---->  Post-loop activity
                             ^                                                 |
                             |                                                 |
                             +-------------------------------------------------+

Posted: Sun May 14, 2006 11:04 pm
by kumar_s
I dont have the access to Server Edition now. If you have "File Pattern" pattern option as available in PX editon you job will become easiar.

Posted: Sun May 14, 2006 11:25 pm
by ray.wurlod
That's a strange response in the server forum! :?

Posted: Sun May 14, 2006 11:34 pm
by chulett
kumar_s wrote:If you have "File Pattern" pattern option as available in PX...
What Ray means is... you don't.

Posted: Mon May 15, 2006 8:07 am
by kaps
I believe that the sequential file stage accepts wild cards in file names.
we can mention like Directory/PCBR*

Regards

Posted: Mon May 15, 2006 8:11 am
by DSguru2B
Wild cards is not supported in server. I believe its supported in PX.

Posted: Mon May 15, 2006 8:19 am
by chulett
Final score in the 'Wildcards in the Sequential File stage' Bowl Game: PX 1, Server 0. :wink:

Posted: Mon May 15, 2006 8:27 am
by DSguru2B
Thanks for confirming that Craig, in a very humourous manner :wink:

Posted: Mon May 15, 2006 3:02 pm
by ray.wurlod
For a not too large number of not-too-large files, the Folder stage does support wildcards.

The restriction on not-too-large files is that the Folder stage reads each entire file contents into a single field - it's intended to be used with XML documents rather than data files.