Merging Data

vskr72 · Post by **vskr72** » Mon Mar 14, 2005 11:52 am

I have a folder which has few Seqeuntial file names. The number of files names can vary every time. I need to merge data from all these files and store it in a new Seq file for further processing.

Can anybody help me on how to handle this? Thank you.

Kumar

kcbland · Post by **kcbland** » Mon Mar 14, 2005 12:01 pm

Any problem using the unix command cat?

ArndW · Post by **ArndW** » Mon Mar 14, 2005 12:16 pm

vskr72,

how about using a folder stage to selectively get the filenames you wish to process, then a simple SeqFile->SeqFile type job with the output file NOT being overwritten, but appended to. It lacks the intrinsic simplicity and elegance of Ken's solution [I'd recommend his suggestion, execute a single efficient UNIX command].

Sainath.Srinivasan · Post by **Sainath.Srinivasan** » Mon Mar 14, 2005 3:44 pm

....make sure that your target file is not created in the same directory.