Hi,
Is there any limit to the file size that I can read using sequential file stage using read method as Specific file OR is there any limit to the number of files/file size when I user read method as File Pattern?
Regards
Om
Sequential File size limit
Moderators: chulett, rschirm, roy
Thanks a lot or the reply....ArndW wrote:DataStage doesn't have one, but many file systems default to a 2Gb limit for sequential files. If your OS or installation doesn't have a limit then DataStage doesn't have one, either. ...
I'm strugling at one more point ..when I use read method as file pattern and I want the name of the file also, we use File column Name but what we get is the File pattern what we pass.
Suppose we pass "/dir/filepattern*.txt" in File Pattern we get the same thing back in the File Name column and not the file name the data being sourced from.
Om Prakash
"There are things that are known, and there are things that are unknown, and in between there are doors"
"There are things that are known, and there are things that are unknown, and in between there are doors"
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
If you just want to get the file name, just pass the path name alone, it will read all the filename present in the directory.opdas wrote:How do I get the file names? Do I have to do some changes in setings?
Coz I get File Column Name as "/dir/filepattern*.txt" the same thing I pass in File Pattern.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
I have tried passing just the path name as "/dir/" but thats not even trying to read the content of the file leave alone reading the file name.kumar_s wrote:If you just want to get the file name, just pass the path name alone, it will read all the filename present in the directory.opdas wrote:How do I get the file names? Do I have to do some changes in setings?
Coz I get File Column Name as "/dir/filepattern*.txt" the same thing I pass in File Pattern.
Om Prakash
"There are things that are known, and there are things that are unknown, and in between there are doors"
"There are things that are known, and there are things that are unknown, and in between there are doors"
A workaround.
Add this in before job subroutine.
and read xxx.txt as 'specific file'. Now the first line of the xxx.txt will have the filename. In transformer @INROWNUM = 1 can be used to grasp the first line.
Code: Select all
ls -1 /dir/filepattern*.txt > xxx.txt; cat /dir/filepattern*.txt >> xxx.txt
and read xxx.txt as 'specific file'. Now the first line of the xxx.txt will have the filename. In transformer @INROWNUM = 1 can be used to grasp the first line.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'