Sort with Join stages

devidotcom · Post by **devidotcom** » Wed Jul 11, 2007 6:51 am

Hi All,

I have a scenerio where I sort and hash partition data on column 1 and column 2 also I would have to rename these columns to col 1 and col 2.

The output link then is an input to the join stage which has same partitioning on this link. The other input link to the join would have the same names col 1 and col 2. Will this work? Would it have the same hash partitioning done on the sort stage.

Thanks
Devi

Maveric · Post by **Maveric** » Wed Jul 11, 2007 7:08 am

Yes it would propagate the sort order and partitioning. specify the partitioning as SAME in the join stage for this link. Ideally for renaming you should use a copy or Modify or transformer stages. So it would be Sort -> copy or Modify or transformer stages -> Join. And partitioning is SAME for the Rename stage and join stage.

Maveric · Post by **Maveric** » Wed Jul 11, 2007 7:12 am

Forgot. If you are using Transformer stage to rename, then it will not propagate the sort order by default. You need to check the property "preserve sort order" on the output tab of transformer.

kumar_s · Post by **kumar_s** » Wed Jul 11, 2007 11:26 pm

Also make sure the other link of the join is also partitioned and sorted on the same columns.