What is the method of partitioning ideal for my JOB?

ray.wurlod · Post by **ray.wurlod** » Mon Apr 07, 2008 3:19 pm

Round Robin will preserve the generated order. However, they will be spread over the nodes mentioned in your configuration file. For example if you have four nodes your Data Set will receive:

Code: Select all

node #0:  1  5  9  13  ...
node #1:  2  6  10  14  ...
node #2:  3  7  11  15  ...
node #3:  4  8  12  16  ...

When you use View Data to look at these the order may be scrambled - it depends on which node is able to report first. But the rows in the Data Set will be as I have described.

bkumar103 · Post by **bkumar103** » Tue Apr 08, 2008 4:55 am

There is no hard and fast rule which decides which partition you have to use except for some specific stages like entire for lookup, Hash for aggregator, sort etc. Its all depends on you requirement and the input data set you get. Based on the input data set you get and the stages you use, you have to take the call on partioning. Auto is the default in most case which leaves the partitioning option to the DSEngine.

bkumar103 · Post by **bkumar103** » Tue Apr 08, 2008 4:56 am

There is no hard and fast rule which decides which partition you have to use except for some specific stages like entire for lookup, Hash for aggregator, sort etc. Its all depends on you requirement and the input data set you get. Based on the input data set you get and the stages you use, you have to take the call on partioning. Auto is the default in most case which leaves the partitioning option to the DSEngine.

sunayan_pal · Post by **sunayan_pal** » Tue Apr 08, 2008 6:09 am

what happen in case when required to do operation to database.
let say if the database is "Teradata" and if the partition is set to Auto then how do DSEngine will decide as because the information about the database activities like collect statistics or join index had been performed or not.