Sorted output Sequential File

rroy2 · Post by **rroy2** » Mon Sep 21, 2009 3:39 pm

Hi,

I have a parallel job which generates output in an Sequential File with fileds say A, B, C and D

The volume in the file can go upto 1 million.

I need the data to be sorted on C and D when the file is created by the job.

Will selecting the SORT MERGE option in the Partitioning tab of File stage, and selecting C and D in sort fields, ensure that all 1 million records are sorted on C and D.

OR do I need to do something extra to ensure this (like using some setting, placing any stage before the file stage etc)

Please suggest.

Thanks

ray.wurlod · Post by **ray.wurlod** » Mon Sep 21, 2009 6:03 pm

The Sort/Merge collector only works if the data are already sorted on those keyse upstream of the Sequential stage. If this is not the case, you need to add sorting to the input stream of the Sequential stage.