Read text files from a folder

rajeev_prabhuat · Post by **rajeev_prabhuat** » Mon Dec 20, 2004 10:06 pm

Hi,

I have to read multiple text files from a folder and do the processing with the data present in it, in Server jobs we have folder stage. In Parallel, do we have any stages equivalent to it. If not what is the other option for me to read each file from the folder and process data.

Regards,
Rajeev Prabhu

richdhan · Post by **richdhan** » Tue Dec 21, 2004 12:18 am

Hi Rajeev,

I don't think there is any equivalent in PX for Folder stage. How about using a shared container which contains the server job functionality. The shared container can then be used in your PX job.

--Rich

Be quick to listen, slow to speak and slow to become angry

ray.wurlod · Post by **ray.wurlod** » Tue Dec 21, 2004 1:25 am

You've posted in the Parallel Extender forum.

The Parallel Sequential File stage does indeed have these properties.

Now you're complaining that the same properties are not there in server jobs.

It is NOT THE SAME STAGE TYPE!

rajeev_prabhuat · Post by **rajeev_prabhuat** » Tue Dec 21, 2004 2:22 am

Hi Ray,

That is fine in Parallel there is no folder stage, i am not complainting that there is no folder stage in server jobs.

You mean to say that Sequential stage acts as Folder stage in Parallel Jobs.

Regards,
Rajeev Prabhu

ray.wurlod wrote:You've posted in the Parallel Extender forum.

The Parallel Sequential File stage does indeed have these properties.

Now you're complaining that the same properties are not there in server jobs.

It is NOT THE SAME STAGE TYPE!

ray.wurlod · Post by **ray.wurlod** » Tue Oct 18, 2005 4:03 pm

Welcome aboard! :D

Do you just want the file names, or the contents of those files?

If the former, simply execute the command - perhaps as a before-stage subroutine ExecSH - then use a Sequential File stage to read from the file myfile.txt.

If the latter, and you really are in a parallel job, use the file pattern mechanism outlined earlier.

If you're in a server job you really should have posted on that forum, but you can use the Folder stage.

balki · Post by **balki** » Wed Oct 19, 2005 5:11 am

Hi,

I dont think that we can give wild cards in file pattern option. It can point to a file that contains a list of files which can be used as source provided all the files share the same metadata.

Balki

ray.wurlod · Post by **ray.wurlod** » Wed Oct 19, 2005 4:14 pm

Welcome aboard. :D

You can, indeed, use wildcard characters. Read the on-line help.

balki · Post by **balki** » Wed Oct 19, 2005 9:55 pm

balki wrote:Hi,

I dont think that we can give wild cards in file pattern option. It can point to a file that contains a list of files which can be used as source provided all the files share the same metadata.

Balki

Sorry about this post. Yes wild cards can be used in the file pattern.