auto split file
Posted: Thu May 19, 2016 4:29 am
Hi
is there a function in the sequtialFile stage, to auto split file.
e.g.
I output records to the sequtial file. set the file's max record number is 1000000.
then the stage will help me to output F1.txt, F2.txt, F3.txt,etc. every file contains 1000000 records.
because the output file is about 30 million records, I don't hope to split it in another job, it takes time.
is there a function in the sequtialFile stage, to auto split file.
e.g.
I output records to the sequtial file. set the file's max record number is 1000000.
then the stage will help me to output F1.txt, F2.txt, F3.txt,etc. every file contains 1000000 records.
because the output file is about 30 million records, I don't hope to split it in another job, it takes time.