How Preserve Partitioning Works

[kamaldeep.kaur]

Hi All,

If we keep Preserve Partitioning as Default(Propagate) in all the stages, Will it be able to repartitioning the data in next stage.
For example:

If i have 2 seq file ----- Join -------- Aggregator -------- Target Table.

What will happen if Preserve Partition is Default(Propagate) .Join and Aggregator are using different Partitioning methods on different key columns.

Will it partition the data in Aggregator stage properly?

As i never change the preserve partition tab in my jobs.

Thanks in advance
Kamal

[ray.wurlod]

Propagate is advisory (Set is mandatory). So, with Propagate, you will be able to re-partition downstream. One occasion where it is desirable to do so is where the downstream stage executes sequentially (and so can not preserve partitioning). You set Preserve Partitioning to Clear in this case to prevent the warning about inability to preserve partitioning.