Use of auto partitioning method

[NigeGriff]

When defining guidelines for enterprise developers is it best to recommend using auto partitioning method or for developers to explicitly select the partioning method required for the processing being performed.


Ascential recommend auto partitioning but are there issues with this recommendation.

[ray.wurlod]

Auto will always work. Somewhere else on DSXchange I have posted a list of what you get in various situations.

Auto may not, however, be optimal in all cases, particularly where it sets Entire as the algorithm on a reference input to a Lookup stage in a multi-machine environment.

[NigeGriff]

Thanks for the quick reply Ray.

[sharantheboss]

Hi,

Auto is not recomended always! use diffrent kind of partitions depending on the scenarios like Hash partition for key based opertaions,Entire for look-up,round robin for non-key operations etc

Regards
Boss

When defining guidelines for enterprise developers is it best to recommend using auto partitioning method or for developers to explicitly select the partioning method required for the processing being performed.


Ascential recommend auto partitioning but are there issues with this recommendation.

[ray.wurlod]

(Auto) does give "Hash" on inputs to those stage types that require key-based partitioning.

Therefore I say again, (Auto) will always work.

Again, this may be sub-optimal. Modulus may be more appropriate (it is more efficient, but requires a single integer key column).