Partioning techniques

ramesh_inform · Post by **ramesh_inform** » Tue Jan 22, 2008 12:02 am

Best partioning techniques for different stages..........
Can anyone list out different partioning techniques which suits best for different stages?

ramesh_inform · Post by **ramesh_inform** » Tue Jan 22, 2008 1:18 am

I think Hash partitioning is best to use with join stage,aggregator stage and change capture stage
and entire for lukup[reference link]

any opinions....

ray.wurlod · Post by **ray.wurlod** » Tue Jan 22, 2008 1:38 am

The best is (Auto) in general. For a large reference table into a Lookup stage you may get some benefit from hash partitioning both the stream and the reference input based upon the lookup key.
Any stage that requires key adjacency (the previous poster mentioned some of them) requires partitioning on the join/grouping/remdup key using a key-based partitioning algorithm. Modulus is more efficient than hash, but is only available for a single-column integer key.

ramesh_inform · Post by **ramesh_inform** » Wed Jan 23, 2008 1:02 am

Partitioning depends on the business logic u apply in ur project
So what I think is rather than allowing DS to select the partitioning technique it will be useful if we mention the partitioning technique for different stages.

ray.wurlod · Post by **ray.wurlod** » Wed Jan 23, 2008 8:03 am

Partitioning does NOT depend on business logic. Can you please justify that claim?

As to appropriate partitioning algorithms, I believe I covered that in my previous post on this thread.

subrat · Post by **subrat** » Fri Jan 25, 2008 2:23 am

Hi,
I also have same impression as the partioning does not depend upon the business logic. But it has a big impact on the column data on which u mentioned the partion key.

Hope u got it....

Subrat

ramesh_inform wrote:Partitioning depends on the business logic u apply in ur project
So what I think is rather than allowing DS to select the partitioning technique it will be useful if we mention the partitioning technique for different stages.

RavishankarMS · Post by **RavishankarMS** » Fri Jan 25, 2008 8:20 am

Hi,

What i believe is neither the stage nor business logic is as important as the Data itself is. The deeper you understand the data easier is to find partitioning technique.

ray.wurlod · Post by **ray.wurlod** » Sat Jan 26, 2008 12:54 am

subrat wrote:Hi,
I also have same impression as the partioning does not depend upon the business logic. But it has a big impact on the column data on which u mentioned the partion key.

Hope u got it....

Subrat

U hasn't posted in a white. Why do you hope U got it? As far as I can tell U never mentioned anything of the kind (U hasn't posted very often, so a search by author was quite short).

Incidentally, the second person personal pronoun in English is "you".