File Pattern

[manjuak.boya1@gmail.com]

in sequential stage we give file patters but it reads multiple files at a time but how to read one by one

[chulett]

I've been deleting these wayward posts up till now but took pity on this one for some reason, so split it out to here.

[chulett]

If "one by one" means a separate run of the processing job for each file, that's what a Sequence job with the Start/End Loop stages will do. Any particular problem with doing them all at once?

[asorrell]

Have you tried setting the option "Keep file partitions" to "True"? I haven't used it in a long time, but I think that may do what you are looking for - keep the data from each file together in a single partition.

For example if you have four files that are read, the data would be output in four partitions, one for each file. You'd still have to be concerned about re-partitioning later in the job by other stages.