Hi, Everyone
Can anyone suggest a solution to the following scenario.
I have more than 100 csv files as my sources. all of them are of same structure.
I need to extract the data from all these files into a table with the file name as an added field.
Number of files in the folder are not constant. and the file names may change every time .
flat file---->transformer----->table.
Can I put this job into a nested loop and pass the file name as parameter into it. and the loop should run till all the files in the folder are extracted(I need to use a job after sub routine to move the files after extraction).
Is this works? If works, How can we pass the file name parameters into the job?.
Thanks in Advance,
Kalpana.
Passing parameters to a nested loop
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Yes, but why would you? If you can match the file names with a regular expression you can read all files with a single Sequential File stage, using File Pattern as the method and the regular expression as the pattern.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Re: Passing parameters to a nested loop
Ray,
I have two files with same structure, File names are test1.txt, test2.txt
test1.txt file structure is
ID Integer
Desc Varchar(100)
test2.txt file structure is
ID Integer
Desc Varchar(100)
Can I read these two files data in one sequetial file stage?
Could you please give us more explanation with the example.
Thanks
Sreeni
I have two files with same structure, File names are test1.txt, test2.txt
test1.txt file structure is
ID Integer
Desc Varchar(100)
test2.txt file structure is
ID Integer
Desc Varchar(100)
Can I read these two files data in one sequetial file stage?
Could you please give us more explanation with the example.
Thanks
Sreeni
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
When I use the this method in version 8.0.1.x the results is the file pattern in the column instead of the actual file name.OddJob wrote:If both files have the same structure...
In the sequential file stage add two 'File' properties, one for each file. Or, you can change the 'Read Method' to 'File Pattern' and use a value of 'test*.txt'
If you need to know which file the data came from add option 'File Name Column'