DSXchange

Hi,

I have to read N input files all with the same schema.Their complete path along with the filenames are stored in file A. Now i have to concatenate all these files to create one input file.

Right now I have a job which reads the file A from a external source stage using the unix command

"cat A | xargs zcat -f"

And then links the output of the stage to a sequential file.

My question is , is there a better approach to handle the above scenario?

Thanks
~Abhi

Hi

u can use fileset .

Thanks
Sanjay

Abhijeet wrote:Hi,

I have to read N input files all with the same schema.Their complete path along with the filenames are stored in file A. Now i have to concatenate all these files to create one input file.

Right now I have a job which reads the file A from a external source stage using the unix command

"cat A | xargs zcat -f"

And then links the output of the stage to a sequential file.

My question is , is there a better approach to handle the above scenario?

Thanks
~Abhi

File set cannot read file names from a sequential file

Do you mean creating a .fs file{which will need some scripting} and then read the file using fileset stage ?

use file pattern option of sequential file stage and load all the file in fileset.

Thanks
Sanjay

Abhijeet wrote:File set cannot read file names from a sequential file

Do you mean creating a .fs file{which will need some scripting} and then read the file using fileset stage ?

Could you generate the stream of data using something like cat `cat FileA`?