Page 1 of 1

Process Multiple files:Process Each file in each run

Posted: Sun May 10, 2009 2:14 pm
by krishna81
I have a list of filenames in a file and i want to pass these file names in parameter and want to run my job for each file by keep in loop using job sequencer.

in each files header is record count and and following are data.
example:

file1timestamp.txt
-----------------------
3
1 rec1
2 rec2
3 rec3

file2timestamp.txt
--------------------
4
1 rec1
3 rec2
4 rec3
2 rec5

i need to run my job for each file.
i guess file pattern doesn't works in this scenario.And i need to process each file in each run.
Though i am invoking my DS job thru KSH, want to avoid loop step in my korn shell.
please advise me in this matter.

Thanks in Advance
kris

Posted: Sun May 10, 2009 2:19 pm
by krishna81
example:
file contains list of filenames

Filenamelist.txt
----------------
file1timestamp.txt
file2timestamp.txt

Posted: Sun May 10, 2009 6:18 pm
by chulett
So, what have you tried? The UserVariables stage can be used to fetch the list of filenames and then the Start Loop stage can take one entry off that list for each iteration of the loop and pass it as a job parameter to your processing job.

The only 'tricky' part would be building a routine that returns the filenames from your file in a delimited fashion that the stage supports. You could, for example, 'cat' the file capturing the output and then Convert() the delimiters from field marks to commas. Or perhaps script the "pivoting" of the names.

Posted: Mon May 11, 2009 7:29 am
by krishna81
what exactly i have to give in expression editor in UserVariables stage.

Posted: Mon May 11, 2009 7:47 am
by chulett
The name of the routine you wrote, as one example.