FOLDER Stage-Read multiple input and get multiple output

[dprasanth]

Hi
I have a problem with using folder stage. I am using a folder stage to pick up multiple xml files and convert them to CSV files.
So this is my job
FOLDER STAGE --->XML Reader---> Sequential File stage
My problem is, I have multiple XML files in the folder that I am using. So I need to read each xml file and have seperate output for each file
Say I have xml files A.XML and B.XML in the folder stage, now I need XML reader to read the A.XML and then produce A.txt, then B.XML and produce b.txt.
Right now, what is happening is the output is merged.
So can you please let me know how to put a constraint or some sort of derivation saying produce one output file for each input file read?
I did search the forum, but couldn't get relevant answers. Hope some one can guide me?If there are any relevant threads regarding this, you can point me towards the thread.
Any help much appreciated.

Regards,
Dprasanth

[Kirtikumar]

If you are having 2 cols in folder stage: 1st as filename and 2nd as column which will contain the XML doc then...

After the folder stage instead of putting XML stage, put a transformer.
Folder stage has 2 columns as output: 1st filename and 2nd column which will contain the XML data.

In the transformer, put 2 output links to 2 XML stages. First link will have constraint : Filename = A and second will have Filename = B.

Send only 2nd column on the output links to XML stages.

Design will be as follows:

Code: Select all

Folder-------->Transformer------>XML(A)------>Seq(A)
                       |
                        ------------------> XML(B)----->seq(B)

[dprasanth]

If you are having 2 cols in folder stage: 1st as filename and 2nd as column which will contain the XML doc then...

After the folder stage instead of putting XML stage, put a transformer.
Folder stage has 2 columns as output: 1st filename and 2nd column which will contain the XML data.

In the transformer, put 2 output links to 2 XML stages. First link will have constraint : Filename = A and second will have Filename = B.

Send only 2nd column on the output links to XML stages.

Design will be as follows:

Code: Select all

Folder-------->Transformer------>XML(A)------>Seq(A)
                       |
                        ------------------> XML(B)----->seq(B)

Thanks a lot for your suggestion. But my problem is, I will not know the name of the file. I just know that there can be more than one file.For eg,in folder temp, I can have three files with extension .XML. But I really wouldn't know what the name of the file. Sometimes, there might be one file, sometimes two or three. Is there any way that I can deal with such a situation?
Thanks once again
Regards,
dprasanth

[Kirtikumar]

If the format of the file is same, then following can be done:

The job will take the file name as parameter and will use this job parameter while reading the file thro' folder stage and at a time will process only one file.
Write a shell script, which should be called to check if any file is in dir /tmp with extension XML.
If the file is there, it will call the DS job with the file name as job parameter. Job will then pick this file as job param will be the filename and do the processing. If there are multiple XML files, the script will call the job once for each and every file.
The only imp thing here is, as the script will be called, you should delete the file or move it elsehwhere after processing the file sothat during next call the already processed file wont be considered

[dprasanth]

If the format of the file is same, then following can be done:

The job will take the file name as parameter and will use this job parameter while reading the file thro' folder stage and at a time will process only one file.
Write a shell script, which should be called to check if any file is in dir /tmp with extension XML.
If the file is there, it will call the DS job with the file name as job parameter. Job will then pick this file as job param will be the filename and do the processing. If there are multiple XML files, the script will call the job once for each and every file.
The only imp thing here is, as the script will be called, you should delete the file or move it elsehwhere after processing the file sothat during next call the already processed file wont be considered

That is a nice idea Kirtikumar. I will surely try this and will let you know the results. Thanks for your help