Hi All,
I need to pick two 0r more than two files for which file name starts with similar word say file 1 name : HFDC_0101.csv,HDFC_08082012.csv.
Can we able to process these two files in a single job run and load into the target table?
Loading two or more files in a single job run
Moderators: chulett, rschirm, roy
Yes, take a look at the server "folder file" stage, that will let you use wildcards and other rules to select files to process from a directory.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
ssreeni3 - This is posted in the server forum, and the server sequential file stage does not offer this option.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
There are 'issues' with the Folder stage as it was built for XML processing and passes in the file contents in one lump. You could leverage the Filter option in the Sequential File stage to 'type' those files to standard out using a wildcard pattern.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Participant
- Posts: 20
- Joined: Tue Feb 02, 2010 4:49 am
- Location: United Kingdom
The Answer to your above question is NO. Sequential file Stage cannot handle multiple input streams
However, you can combine the multiple streams of data (multiple sequential files) into a single stream and feed that into the Sequential File Stage, provided they have consistent Metadata. In this way you can handle multiple Sequential files
To acheive the above please follow the posts from ssreeni3 and chulett and use the Filter command option. This will create a single data stream from multiple inputs and hence can be parsed through the Sequential file stage
Thanks
However, you can combine the multiple streams of data (multiple sequential files) into a single stream and feed that into the Sequential File Stage, provided they have consistent Metadata. In this way you can handle multiple Sequential files
To acheive the above please follow the posts from ssreeni3 and chulett and use the Filter command option. This will create a single data stream from multiple inputs and hence can be parsed through the Sequential file stage
Thanks
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
The Sequential File stage can readily handle multiple input files, provided that they have the same record structure.
The Sequential File stage can use a Filter command. In this case the filter command should be TYPE file1 file2 (or COPY file1 file2). (If you were on UNIX, the command would be cat file1 file2.) The stage then reads stdout from the command rather than trying to read the file named in the File property (which is still required, even though ignored).
The Sequential File stage can use a Filter command. In this case the filter command should be TYPE file1 file2 (or COPY file1 file2). (If you were on UNIX, the command would be cat file1 file2.) The stage then reads stdout from the command rather than trying to read the file named in the File property (which is still required, even though ignored).
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.