Process several files in a folder in sequence.

[semja0]

Hi,

The scenario is as follows.
In a folder on the DataStage server there is a bunch of files of the same type waiting to be processed by a job in DataStage. What I have not figured out yet is how one defines in a job to work with sequential files with arbitrary names. As fare as I can se one has to define the file name in the Sequential file stage. I tried the folder stage as well but I could not se how I should use that.
As you already understand by now I am quit new with this tool/product. (DataStage version 5.2)


I hope someone can help me with what I guess is a quit simple problem.

----------------------------------------------------------------------

Tank you all for your reply.

I read the OPENPATH statement in the BASIC Guide
However I do not quit understand what the result is of this statement/function

OPENPATH pathname [TO file.variable] [ON ERROR statements]
{THEN statements [ELSE statements] | ELSE statements}

Let say I got following files:

TestFile1
TestFile2
TestFile3

In a folder with following path /home/me/test

What should pathname contain?
In what format/structure can I expect to find the information in the file.variable?
How is file.variable defined in a program/routine?



Edited by - semja0 on 11/22/2002 02:18:56

[vmcburney]

You can set up the file name as a job parameter and use this parameter in the file name field in the sequential file stage.

Eg.:
FileName: #FileDir##FileName#
Two job parameters are used, one for the directory location and one for the file name. You can then call this job up multiple times and pass the file names through as a parameter each time.

Have a look at the OpenPath command in your DataStage BASIC manual, it retrieves a list of file names from a directory and the DSSetParam command will let you pass those file names to the job each time you call it.

OpenPath FileDir
...
For each file name
Attach the job
Set the parameter
Run the job
Wait for it to finish
Loop

I can't give you the exact code because I don't have access to the DataStage manuals at the moment.

regards
Vincent

[ray.wurlod]

Sequential File stage is the correct stage type to use and, as Vincent suggested, you use a parameter for the file name.
You can set up the job to run as multiple instances and run the jobs in parallel, providing the appropriate file name to each instance.

[semja0]

Tank you all for your reply.

I read the OPENPATH statement in the BASIC Guide
However I do not quit understand what the result is of this statement/function

OPENPATH pathname [TO file.variable] [ON ERROR statements]
{THEN statements [ELSE statements] | ELSE statements}

Let say I got following files:

TestFile1
TestFile2
TestFile3

In a folder with following path /home/me/test

What should pathname contain?
In what format/structure can I expect to find the information in the file.variable?
How is file.variable defined in a program/routine?

Hi,

The scenario is as follows.
In a folder on the DataStage server there is a bunch of files of the same type waiting to be processed by a job in DataStage. What I have not figured out yet is how one defines in a job to work with sequential files with arbitrary names. As fare as I can se one has to define the file name in the Sequential file stage. I tried the folder stage as well but I could not se how I should use that.
As you already understand by now I am quit new with this tool/product. (DataStage version 5.2)

I hope someone can help me with what I guess is a quit simple problem.



[/quote]

[raviyn]

In case of multiple files having same processing.U can use the first job the folder Stage and in this job call other job (using the UtilityRunJob with the filename as the parameter) which is processing these files.Hope this helps.