how to distribute the records amoung the nodes..

[kannan_ayyanar]

i am new to the concept of partition. please help me...

4 nodes are defined in server...

how could i distribute the input records to the 4 nodes... for example 1000 rows are coming from input, i would like to distribute 250 rows to each node... is it possible to achieve this.. if this kind of question is already raised in this forum... please give me the link.. i searched in this forum but i couldn't find that..

thanks in advance...

[kannan_ayyanar]

if any one having good document which explains very basics about partitioning. let me know... i searched in google.. i didnt get what i expected....


thanks in advance...

[dsa]

parallel job dev guide explains types of partitioning. Depending upon your requirement, you might want to go for round robin or hash or modulus or random partitioning to achieve the same.

[kannan_ayyanar]

job is simple... sequential file as input and output... and transformer in between.. ok if i go for a round robin partition in transformer.. how can i check the count of record processed in each node... ?

[Sainath.Srinivasan]

From monitor.

[Marley777]

where in monitor can you see how many records were processed for each node?

Wondering if there is a break down per stage?

Example

Node1 processed 4 records
Node2 processed 7 records
etc...

[ray.wurlod]

Right click in Monitor background area and enable "Show instances". Parallel stages will show as x 4 if collapsed, or as four separate instances if expanded.