How could I process all files from a directory?

tiagogen · Post by **tiagogen** » Sun May 14, 2006 2:23 pm

Hi there!
I am newbie in DataStage and in this forum as well.
My doubt is how can I get all files from a directory with a determinate standard.
For example, I want to process all files from a directory that starts with "PCBR*".
This directory comes from a parameter.

The way I found to do that, is using the job sequence.
I execute a find(unix command) , and I call the job once for file.
Anyone has a better way to do that?

thanks a bunch!
(sorry my English mistakes)

ray.wurlod · Post by **ray.wurlod** » Sun May 14, 2006 2:58 pm

Welcome aboard. :D

A job sequence is the best way. A StartLoop activity can set up a "list of things" loop based on a regular expression.

Code: Select all

  Pre-loop activity ---->  StartLoop  ---->  Within-loop activities  ---->  EndLoop  ---->  Post-loop activity
                             ^                                                 |
                             |                                                 |
                             +-------------------------------------------------+

kumar_s · Post by **kumar_s** » Sun May 14, 2006 11:04 pm

I dont have the access to Server Edition now. If you have "File Pattern" pattern option as available in PX editon you job will become easiar.

ray.wurlod · Post by **ray.wurlod** » Sun May 14, 2006 11:25 pm

That's a strange response in the server forum!

chulett · Post by **chulett** » Sun May 14, 2006 11:34 pm

kumar_s wrote:If you have "File Pattern" pattern option as available in PX...

What Ray means is... you don't.

kaps · Post by **kaps** » Mon May 15, 2006 8:07 am

I believe that the sequential file stage accepts wild cards in file names.
we can mention like Directory/PCBR*

Regards

DSguru2B · Post by **DSguru2B** » Mon May 15, 2006 8:11 am

Wild cards is not supported in server. I believe its supported in PX.

chulett · Post by **chulett** » Mon May 15, 2006 8:19 am

Final score in the 'Wildcards in the Sequential File stage' Bowl Game: PX 1, Server 0.

DSguru2B · Post by **DSguru2B** » Mon May 15, 2006 8:27 am

Thanks for confirming that Craig, in a very humourous manner

ray.wurlod · Post by **ray.wurlod** » Mon May 15, 2006 3:02 pm

For a not too large number of not-too-large files, the Folder stage does support wildcards.

The restriction on not-too-large files is that the Folder stage reads each entire file contents into a single field - it's intended to be used with XML documents rather than data files.