When should I use "Entire" partitioning in a looku

[pattemk]

Hi,

Is it mandatory to specify entire as the partitioning method when using normal lookup. Is there any chance of losing data if we just leave it as auto partitioning?

Kindly advice

Thanks

**Note: Subject made more descriptive - Content Editor **

[betterthanever]

no it is not mandatory

[pattemk]

[quote="betterthanever"]no it is not mandatory[/quote]

Thanks for your prompt reply, a quick question.

what would be the cases or scenarios where we must specify entire partitiong method when doing normal lookup?

Kindly advice

Thanks

[betterthanever]

i don't think any

[Mike]

Specify entire partitioning whenever you are unable or unwilling to partition the reference input exactly the same as the stream input. One example of unable that I can think of: multiple reference links into a single lookup stage where the stream input can only be partitioned to match one of the reference inputs. An example of unwilling: a very small reference table where you don't want the overhead of repartitioning the stream input.

Mike

[ray.wurlod]

On a single machine ("SMP" environment) you may as well use Entire, because it comes at no cost, via shared memory.

In a multiple machine environment ("MPP", cluster, grid) there can be a substantial cost moving records to all nodes, so you tend to avoid Entire (other than for small Data Sets) and use the same key-based partitioning as is used for the stream input.

[pattemk]

[quote="Mike"]Specify entire partitioning whenever you are unable or unwilling to partition the reference input exactly the same as the stream input. One example of unable that I can think of: multiple reference links into a single lookup stage where the stream input can only be partitioned to match one of the reference inputs. An example of unwilling: a very small reference table where you don't want the overhead of repartitioning the stream input.

Mike[/quote]

Thanks for your prompt reply.
my reference data is very small, i believe specifying entire will not result in lose of data or performance, so i am assuming specifying entire is better practice and mostly like mandatory when doing normal lokup with small reference data.