Page 1 of 1
Read text files from a folder
Posted: Mon Dec 20, 2004 10:06 pm
by rajeev_prabhuat
Hi,
I have to read multiple text files from a folder and do the processing with the data present in it, in Server jobs we have folder stage. In Parallel, do we have any stages equivalent to it. If not what is the other option for me to read each file from the folder and process data.
Regards,
Rajeev Prabhu
Posted: Tue Dec 21, 2004 12:18 am
by richdhan
Hi Rajeev,
I don't think there is any equivalent in PX for Folder stage. How about using a shared container which contains the server job functionality. The shared container can then be used in your PX job.
--Rich
Be quick to listen, slow to speak and slow to become angry
Posted: Tue Dec 21, 2004 1:25 am
by ray.wurlod
You've posted in the Parallel Extender forum.
The Parallel Sequential File stage does indeed have these properties.
Now you're complaining that the same properties are not there in server jobs.
It is NOT THE SAME STAGE TYPE!
Posted: Tue Dec 21, 2004 2:22 am
by rajeev_prabhuat
Hi Ray,
That is fine in Parallel there is no folder stage, i am not complainting that there is no folder stage in server jobs.
You mean to say that Sequential stage acts as Folder stage in Parallel Jobs.
Regards,
Rajeev Prabhu
ray.wurlod wrote:You've posted in the Parallel Extender forum.
The Parallel Sequential File stage does indeed have these properties.
Now you're complaining that the same properties are not there in server jobs.
It is NOT THE SAME STAGE TYPE!
Posted: Tue Oct 18, 2005 4:03 pm
by ray.wurlod
Welcome aboard! :D
Do you just want the file names, or the contents of those files?
If the former, simply execute the command - perhaps as a before-stage subroutine ExecSH - then use a Sequential File stage to read from the file myfile.txt.
If the latter, and you really are in a parallel job, use the file pattern mechanism outlined earlier.
If you're in a server job you really should have posted on that forum, but you can use the Folder stage.
Posted: Wed Oct 19, 2005 5:11 am
by balki
Hi,
I dont think that we can give wild cards in file pattern option. It can point to a file that contains a list of files which can be used as source provided all the files share the same metadata.
Balki
Posted: Wed Oct 19, 2005 4:14 pm
by ray.wurlod
Welcome aboard. :D
You can, indeed, use wildcard characters. Read the on-line help.
Posted: Wed Oct 19, 2005 9:55 pm
by balki
balki wrote:Hi,
I dont think that we can give wild cards in file pattern option. It can point to a file that contains a list of files which can be used as source provided all the files share the same metadata.
Balki
Sorry about this post. Yes wild cards can be used in the file pattern.