File Name property in Sequential File Stage
Moderators: chulett, rschirm, roy
File Name property in Sequential File Stage
I have to read many .txt files from a directory and output to one single file. The output will be as follows ; First column is the name of the appropriate file and the remaining is the contents from each file.
In the sequential stage I have set
propertie -> Options -> File Name Column = 'FileNameCol1'
properties -> Source -> File = '/opt/datastage/*.txt'
properties -> Source -> Read Method = File Pattern
When I run the Job the FileNameCol1 column contains ''/opt/datastage/*.txt'' instead of the individual file names as follows
''/opt/datastage/one.txt'' xxx yyy
''/opt/datastage/one.txt'' zzz hhh
''/opt/datastage/two.txt'' aaa bbb
Any light ?
In the sequential stage I have set
propertie -> Options -> File Name Column = 'FileNameCol1'
properties -> Source -> File = '/opt/datastage/*.txt'
properties -> Source -> Read Method = File Pattern
When I run the Job the FileNameCol1 column contains ''/opt/datastage/*.txt'' instead of the individual file names as follows
''/opt/datastage/one.txt'' xxx yyy
''/opt/datastage/one.txt'' zzz hhh
''/opt/datastage/two.txt'' aaa bbb
Any light ?
Django
If a simple dsx project requires such intelligence how much more is required to create this Cosmic System. Who dares to say there is no intelligence behind this creation i.e. no GOD .....
If a simple dsx project requires such intelligence how much more is required to create this Cosmic System. Who dares to say there is no intelligence behind this creation i.e. no GOD .....
Re: File Name property in Sequential File Stage
Hi,
enable this option in seqeuntial file stage
First line is column is 'True'
Default is 'False'
Regards Naveen.K
enable this option in seqeuntial file stage
First line is column is 'True'
Default is 'False'
Regards Naveen.K
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Any workaround ?
So whats the solution ?
If not through sequential stage properties is there any other way of reading the name of the files (of pattern *.txt) being used in the source stage.
If not through sequential stage properties is there any other way of reading the name of the files (of pattern *.txt) being used in the source stage.
Django
If a simple dsx project requires such intelligence how much more is required to create this Cosmic System. Who dares to say there is no intelligence behind this creation i.e. no GOD .....
If a simple dsx project requires such intelligence how much more is required to create this Cosmic System. Who dares to say there is no intelligence behind this creation i.e. no GOD .....
There are other posts here of people with the same question that a search for "file pattern" would turn up. Best as I can tell, you need to set the APT_IMPORT_PATTERN_USES_FILESET environment variable to TRUE to get this to work as one would expect.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
Resolve
Craig
Setting the APT_IMPORT_PATTERN_USES_FILESET to True resolved the issue.
Thx
Setting the APT_IMPORT_PATTERN_USES_FILESET to True resolved the issue.
Thx
Django
If a simple dsx project requires such intelligence how much more is required to create this Cosmic System. Who dares to say there is no intelligence behind this creation i.e. no GOD .....
If a simple dsx project requires such intelligence how much more is required to create this Cosmic System. Who dares to say there is no intelligence behind this creation i.e. no GOD .....
-
- Premium Member
- Posts: 783
- Joined: Mon Jan 16, 2006 10:17 pm
- Location: Sydney, Australia
-
- Participant
- Posts: 48
- Joined: Tue Jan 26, 2010 4:05 am
- Location: Indonesia
hi,
I'm facing the same problem here.. and I have changed the parameters into above. when I run the job, it has solved the problem. but suddenly, when I want to try to run the job again, it produces an error when I try to view data:
what happen? and what should I do?
Thanks in advance
I'm facing the same problem here.. and I have changed the parameters into above. when I run the job, it has solved the problem. but suddenly, when I want to try to run the job again, it produces an error when I try to view data:
Code: Select all
##E IIS-DSEE-TFOP-00020 13:59:23(007) <rlgmfagun> Error occurred during initializeFromArgs().
##E IIS-DSEE-TOIX-00118 13:59:23(008) <rlgmfagun> In file set "/tmp/import_tmp_954582894fa4bc.fs": Parsing a dataset, expecting a fileset..
##E IIS-DSEE-TCOS-00029 13:59:23(009) <main_program> Creation of a step finished with status = FAILED.
Thanks in advance
-
- Participant
- Posts: 48
- Joined: Tue Jan 26, 2010 4:05 am
- Location: Indonesia
-
- Participant
- Posts: 48
- Joined: Tue Jan 26, 2010 4:05 am
- Location: Indonesia
-
- Participant
- Posts: 48
- Joined: Tue Jan 26, 2010 4:05 am
- Location: Indonesia
nope, my job only has 3 stages.
1. sequential file as source file
which read file with pattern *customer.txt
2. transformer to map the columns
3. ODBC stage as the database target
if I set the $APT_IMPORT_PATTERN_USES_FILESET = False, it works and I can view data n run the job.. but the problem is, the file name column contains * and I still need the real file name for some reason.
thanks,
Caterin
1. sequential file as source file
which read file with pattern *customer.txt
2. transformer to map the columns
3. ODBC stage as the database target
if I set the $APT_IMPORT_PATTERN_USES_FILESET = False, it works and I can view data n run the job.. but the problem is, the file name column contains * and I still need the real file name for some reason.
thanks,
Caterin
-
- Participant
- Posts: 48
- Joined: Tue Jan 26, 2010 4:05 am
- Location: Indonesia
thanks God, everything has been solved..
if anyone find this post and facing the same problem, the solution is please check the directory and file name whether it is at the correct location. In my case, I have an additional / at my file path, and it makes the DS can't find the right file.
Why the error sounds like that? perhaps when DS import pattern use fileset,it will create a temporary fileset. and if DS can't find the correct filepath, it also can't parse the temporary fileset.
Thanks,
Caterin
if anyone find this post and facing the same problem, the solution is please check the directory and file name whether it is at the correct location. In my case, I have an additional / at my file path, and it makes the DS can't find the right file.
Why the error sounds like that? perhaps when DS import pattern use fileset,it will create a temporary fileset. and if DS can't find the correct filepath, it also can't parse the temporary fileset.
Thanks,
Caterin