File Pattern problems !
Moderators: chulett, rschirm, roy
File Pattern problems !
Hi,
just playing around with the functionality of 'file pattern'which you can set in sequential files to read multiple files in a folder.
i have four files in a folder a.txt , b.txt , c.txt and d.txt in folder E:\files
when i read them individually they work all ok
but when i change the property to file pattern and state E:\files\*.txt as the pattern and view the data . it says no files found at host XYZ E:\files\*.txt or
when i run the job it says :
Missing record delimiter "\n", saw EOF instead
i only have one value in all the files and that is 1 i.e.
a.txt has only one col column1 with value 1 in the file and same is the case for other files
Any ideas ?
just playing around with the functionality of 'file pattern'which you can set in sequential files to read multiple files in a folder.
i have four files in a folder a.txt , b.txt , c.txt and d.txt in folder E:\files
when i read them individually they work all ok
but when i change the property to file pattern and state E:\files\*.txt as the pattern and view the data . it says no files found at host XYZ E:\files\*.txt or
when i run the job it says :
Missing record delimiter "\n", saw EOF instead
i only have one value in all the files and that is 1 i.e.
a.txt has only one col column1 with value 1 in the file and same is the case for other files
Any ideas ?
this is exactly what i did before and how i created the test files. i have read them individually with the same options for all and they work fine. it just gives me a problem when i try to read them all at once. i have done all the basic checks, but it does'nt work. and gives me the same errors. it however does process only the first file.
Add a final \n to your file a.txt and see if the error wanders to file 2...
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
thanks. stil does'nt work. when i click on view data on seq file stage it gives me this:
Code: Select all
##I TOSH 000002 09:31:35(001) <main_program> orchgeneral: loaded
##I TOSH 000002 09:31:35(002) <main_program> orchsort: loaded
##I TOSH 000002 09:31:35(003) <main_program> orchstats: loaded
##W TCOS 000049 09:31:35(004) <main_program> Parameter specified but not used in flow: DSProjectMapName
##W TOIX 000000 09:31:35(005) <main_program> createFilesetFromPattern(): Couldn't find any files on host with pattern E:/test/*.txt.
>##E TOIX 000138 09:31:36(001) <Sequential_File_0> At least one filename or data source must be set in APT_FileImportOperator before use.
>##E TFSR 000019 09:31:36(005) <main_program> Could not check all operators because of previous error(s)
>##E TCOS 000029 09:31:36(006) <main_program> Creation of step finished with status = FAILED
-
- Participant
- Posts: 467
- Joined: Tue Mar 20, 2007 6:36 am
- Location: Chennai
- Contact:
Try changing the backslashes in your path to forward slashes. It's a bug in the parallel edition.
E:\test/*.txt
E:\test/*.txt
Minhajuddin
<a href="http://feeds.feedburner.com/~r/MyExperi ... ~6/2"><img src="http://feeds.feedburner.com/MyExperienc ... lrow.3.gif" alt="My experiences with this DLROW" border="0"></a>
<a href="http://feeds.feedburner.com/~r/MyExperi ... ~6/2"><img src="http://feeds.feedburner.com/MyExperienc ... lrow.3.gif" alt="My experiences with this DLROW" border="0"></a>
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
APT_FileImportOperator is the class from which Import objects (such as the operator generated by a Sequential File stage with an output link generates) are instantiated.
The most recent error message indicates that the wildcard handling is not finding any files matching the E:/test/*.txt pattern. What happen when you change back to Specific File and use four File properties to mention the four files explicitly?
We'll get back to the missing line terminator later; we need to solve one problem at a time.
The most recent error message indicates that the wildcard handling is not finding any files matching the E:/test/*.txt pattern. What happen when you change back to Specific File and use four File properties to mention the four files explicitly?
We'll get back to the missing line terminator later; we need to solve one problem at a time.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
thanks ray i have done that and it works like a charm when i use specific files and add four files by using file property .
i don't know why when i change it to file pattern it's not working with the same files.
also i used the property first name column but its giving me the whole path of the file along with the file name and not just the file name in the target file..
i don't know why when i change it to file pattern it's not working with the same files.
also i used the property first name column but its giving me the whole path of the file along with the file name and not just the file name in the target file..
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
With File Pattern try using a job parameter for the directory part of the pathname, for example #jpFileDir#/*.txt
The behaviour of the File Name Column property (to return the complete pathname of the file) is as documented.
The behaviour of the File Name Column property (to return the complete pathname of the file) is as documented.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Thanks Ray! . it works fine when i specify the path as a parameter and not a hardcorded value. strange !
although when i try to view the data it still doesnt show it to me and still says that no files found on host and then about APT_FileImportOperator.
apart from this when i sepcify the file name property. it doesnt run the job properly. when i take it out it does. i think it sees that as a column in the file too and that why messes everything up when it comes to reading the data
Any ideas ?
although when i try to view the data it still doesnt show it to me and still says that no files found on host and then about APT_FileImportOperator.
apart from this when i sepcify the file name property. it doesnt run the job properly. when i take it out it does. i think it sees that as a column in the file too and that why messes everything up when it comes to reading the data
Any ideas ?
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
The File Name Column property must name a column that occurs on the stage's output link. This column should be a VarChar large enough to take a full pathname, for example VarChar(128). And, needless to say, the column must appear on the output link.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.