DSXchange

Hi there,

Does anyone know of a DataStage function that will read the file names inside of a predefined directory, like a "dir" or "ls" command, then pass each file name into a variable in a given job.

Thanks much!

There isn't one.
What exactly are you trying to accomplish? DataStage generally only works with one file at a time. Do you want to run a job N times, passing each time the name of one of N files?
This is done with a sequence or Job Control code.
I won't elaborate until I get a better feel for what you're trying to do.


Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518

It has been brought to my attention that Ascential consultants have a function called GetNextFile in their toolkit. You might like to request a copy from them. It was written explicitly for HP-UX but ought to be easily adapted. Here's the description.

Scan a directory and return the name of a file. The file name will match the passed in pattern and only files that are not open by other processes will be returned. This makes the routine ideal for use with FTP as a file name will only be returned once the FTP file has arrived.
Returns - the file name or the character "/" to indicate that nothing was available.
Arg 1 - Directory to scan for new files
Arg 2 - Once a file is found it is moved to this directory for processing
Arg 3 - The file name to look file e.g. "SA*" will only find files starting SA.


Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518

Hi there,

You can use Folder stage to get all the filename from your directory, and stored in a hash file (Job A).
And then, use another job (Job B) to read the content of the files and process it, use job parameters for all the filename variables. Then you need another job control (Job C) to do a looping based on the number of files in the directory, and pass the paramereter to the processing job (Job B).

I used this approach for my problem, and it works nicely...

HtH


Regards

ND

Yes, but the poster asked for a function! [:)]

Hi guys,

Thanks so much for all of your valuable input.
The route i've opted to take is to use the folder stage to read file names and their contents (as one large string) into the oracle backend.
Then write a seperate chunk of plsql to parse through the comma delimitted string and process each value accordingly into another oracle table.

Thanks gang! Ray I will look into the GetNextFile for the next piece that I'll be doing. cheers