When defining guidelines for enterprise developers is it best to recommend using auto partitioning method or for developers to explicitly select the partioning method required for the processing being performed.
Ascential recommend auto partitioning but are there issues with this recommendation.
Use of auto partitioning method
Moderators: chulett, rschirm, roy
Use of auto partitioning method
Thanks
Nigel
Nigel
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Auto will always work. Somewhere else on DSXchange I have posted a list of what you get in various situations.
Auto may not, however, be optimal in all cases, particularly where it sets Entire as the algorithm on a reference input to a Lookup stage in a multi-machine environment.
Auto may not, however, be optimal in all cases, particularly where it sets Entire as the algorithm on a reference input to a Lookup stage in a multi-machine environment.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 14
- Joined: Mon Mar 23, 2009 12:57 am
- Location: INDIA
Re: Use of auto partitioning method
Hi,
Auto is not recomended always! use diffrent kind of partitions depending on the scenarios like Hash partition for key based opertaions,Entire for look-up,round robin for non-key operations etc
Regards
Boss
Auto is not recomended always! use diffrent kind of partitions depending on the scenarios like Hash partition for key based opertaions,Entire for look-up,round robin for non-key operations etc
Regards
Boss
NigeGriff wrote:When defining guidelines for enterprise developers is it best to recommend using auto partitioning method or for developers to explicitly select the partioning method required for the processing being performed.
Ascential recommend auto partitioning but are there issues with this recommendation.
DS Info
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
(Auto) does give "Hash" on inputs to those stage types that require key-based partitioning.
Therefore I say again, (Auto) will always work.
Again, this may be sub-optimal. Modulus may be more appropriate (it is more efficient, but requires a single integer key column).
Therefore I say again, (Auto) will always work.
Again, this may be sub-optimal. Modulus may be more appropriate (it is more efficient, but requires a single integer key column).
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.