multiple files into a single seq file???

[ushasunkara]

Hello all,
What stage is best to use, for joining or add multiple xml files into a single seq file,
in parallel jobs(there is no folder stage) ,
from where this seq file -> xml input stage...
i thought it was file set, but i've never used file set stage, if that is the stage, how do i go about it...
thanks a lot in advance
Usha

[kwwilliams]

So you have multiple files that you want to use to create one data set, like a union? If so you would use the funnel. If that's not what you want could you clarify what you are trying to accomplish. Describe how many files you have and what you would liek to happen to the data in those files.

[Raog]

Usha:

Can u give more details about file content ie. all are having same xml type or different xml schema.

I think, when all are having different schema definition, u better to go for JOINER or MERGE.

Rgds,
Rao.

Hello all,
What stage is best to use, for joining or add multiple xml files into a single seq file,
in parallel jobs(there is no folder stage) ,
from where this seq file -> xml input stage...
i thought it was file set, but i've never used file set stage, if that is the stage, how do i go about it...
thanks a lot in advance
Usha

[ameyvaidya]

Hi Usha,
The Sequential File Stage supports Reading both a Single file and multiple files (Given a file pattern).
Property to look for is:

Read Method, specify whether to read specific files (the
default) or all files whose name fits a pattern.

Parjdev.pdf Page 144

The Caveat here is that all files should have the same metadata.
IHTH

[ushasunkara]

Hi All,
thanks for all your opinions,
and ameyvaidya, i'm looking for a wildcard option, there, so in the property - read method - file pattern - what way can i give this expression?
for specific file - i give the pathname of the file location,
then for file pattern - what path do i give? the path of the folder location??
Correct me if i am wrong, what i understood is in Server edition - folder stage can have many files, and when u link to a flat file, u get the names of the files,
but i need here, in Parallel edition,
i've 3 xml files of the same type, but jus for different sources, columns would be same, i need all the 3 xml files to write into one single seq file,
so after the seq file, the xml input stage can take it easily - the whole xml file to go ahead with other transformations...

So, the design is seqfile --> xml input stage --> diff odbc stages....

this seq file - should take 3 xml files - with same content but jus diff source systems(company1, company2, company3)... into one single seq file....

Plz let me know, what do i need to use in this case...
thank you so much....

[DSguru2B]

You can develop a job in PX using multiple instances. If the target is same and the source file is of same TYPE(ie, seq,xml,etc).
Then load the records schema file that overwrites any settings in the format and columns tab. You can have multiple record schema files of your source in your unix dir.
This design would save you mutliple jobs of the same kind.

[ameyvaidya]

I havent had to work with XML Files.. but heres how to go about it:

Directory /home/dsadm/infiles

File 1 name: DB_Cust_XML_1.xml
File 2 name: DB_Cust_XML_2.xml
File 3 name: DB_Cust_XML_3.xml
File 4 name: DB_Cust_XML_4.xml
File 5 name: DB_Cust_XML_5.xml
File 6 name: DB_Cust_XML_6.xml
File 7 name: DB_Cust_XML_7.xml

Set sequential file stage to read from a file pattern.
set file property to:
/home/dsadm/infiles/DB_Cust_XML_*.xml

Also if your file names do not fit any pattern; while reading up on the Sequential File Stage I came across this:

File
This property defines the flat file that data will be read from. You can
type in a pathname, or browse for a file. You can specify multiple files
by repeating the File property. Do this by selecting the Properties
item at the top of the tree, and clicking on File in the Available
properties to add window. Do this for each extra file you want to
specify.

[ray.wurlod]

Tip: If you use multiple File property values, the technique is Specific File, not Pattern.

[ushasunkara]

sorry for the late reply...
thank you so much...ameyvaidya and ray...
it works :D
thank you...