DSXchange

Posted: **Sun Aug 10, 2008 10:23 pm**

I have to read many .txt files from a directory and output to one single file. The output will be as follows ; First column is the name of the appropriate file and the remaining is the contents from each file.

In the sequential stage I have set

propertie -> Options -> File Name Column = 'FileNameCol1'
properties -> Source -> File = '/opt/datastage/*.txt'
properties -> Source -> Read Method = File Pattern

When I run the Job the FileNameCol1 column contains ''/opt/datastage/*.txt'' instead of the individual file names as follows

''/opt/datastage/one.txt'' xxx yyy
''/opt/datastage/one.txt'' zzz hhh
''/opt/datastage/two.txt'' aaa bbb

Any light ?

Posted: **Sun Aug 10, 2008 11:43 pm**

Hi,

enable this option in seqeuntial file stage

First line is column is 'True'

Default is 'False'

Regards Naveen.K

Posted: **Sun Aug 10, 2008 11:54 pm**

Would you care to explain how this affects the File Name property?

Posted: **Mon Aug 11, 2008 6:36 pm**

So whats the solution ?

If not through sequential stage properties is there any other way of reading the name of the files (of pattern *.txt) being used in the source stage.

Posted: **Mon Aug 11, 2008 9:08 pm**

There are other posts here of people with the same question that a search for "file pattern" would turn up. Best as I can tell, you need to set the APT_IMPORT_PATTERN_USES_FILESET environment variable to TRUE to get this to work as one would expect.

Posted: **Mon Aug 11, 2008 10:50 pm**

Craig

Setting the APT_IMPORT_PATTERN_USES_FILESET to True resolved the issue.

Thx

Posted: **Thu Mar 26, 2009 6:08 pm**

set parameters
$APT_IMPORT_PATTERN_USES_FILESET = True
$APT_IMPORT_PATTERN_USES_FILESET_MOUNTED = True

Posted: **Thu Mar 18, 2010 7:53 am**

hi,
I'm facing the same problem here.. and I have changed the parameters into above. when I run the job, it has solved the problem. but suddenly, when I want to try to run the job again, it produces an error when I try to view data:

Code: Select all

##E IIS-DSEE-TFOP-00020 13:59:23(007) <rlgmfagun> Error occurred during initializeFromArgs().
##E IIS-DSEE-TOIX-00118 13:59:23(008) <rlgmfagun> In file set "/tmp/import_tmp_954582894fa4bc.fs": Parsing a dataset, expecting a fileset..
##E IIS-DSEE-TCOS-00029 13:59:23(009) <main_program> Creation of a step finished with status = FAILED.

what happen? and what should I do?

Thanks in advance

Posted: **Thu Mar 18, 2010 8:00 am**

I've never seen the "MOUNTED" one mentioned before and have no idea what it means or does. What happens if you remove it from the job?

Posted: **Thu Mar 18, 2010 8:04 am**

if I removed the "MOUNTED", it also produces the same error..
thanks

Posted: **Thu Mar 18, 2010 8:07 am**

OK... your filename ends in ".fs" which is for a fileset but it found a dataset where it pointed to instead. Any idea how that happened? Naming collision?

You really should start your own thread for something like this.

Posted: **Thu Mar 18, 2010 8:09 am**

no, my filename is ended with .txt and there is only .txt file at the source folder

fyi, my file location is /transport/*tblcustomer.txt
thanks,
Caterin

Posted: **Thu Mar 18, 2010 8:19 am**

Not according to the error you posted. Perhaps your issue is now coming from elsewhere in the job, target rather than source side?

Posted: **Thu Mar 18, 2010 10:51 am**

nope, my job only has 3 stages.
1. sequential file as source file
which read file with pattern *customer.txt

2. transformer to map the columns
3. ODBC stage as the database target

if I set the $APT_IMPORT_PATTERN_USES_FILESET = False, it works and I can view data n run the job.. but the problem is, the file name column contains * and I still need the real file name for some reason.

thanks,
Caterin

Posted: **Thu Mar 18, 2010 8:28 pm**

thanks God, everything has been solved..

if anyone find this post and facing the same problem, the solution is please check the directory and file name whether it is at the correct location. In my case, I have an additional / at my file path, and it makes the DS can't find the right file.

Why the error sounds like that? perhaps when DS import pattern use fileset,it will create a temporary fileset. and if DS can't find the correct filepath, it also can't parse the temporary fileset.

Thanks,
Caterin

DSXchange

File Name property in Sequential File Stage

File Name property in Sequential File Stage

Re: File Name property in Sequential File Stage

Any workaround ?

Resolve