Page 1 of 1

how to distribute the records amoung the nodes..

Posted: Fri Oct 15, 2010 3:38 am
by kannan_ayyanar
i am new to the concept of partition. please help me...

4 nodes are defined in server...

how could i distribute the input records to the 4 nodes... for example 1000 rows are coming from input, i would like to distribute 250 rows to each node... is it possible to achieve this.. if this kind of question is already raised in this forum... please give me the link.. i searched in this forum but i couldn't find that..

thanks in advance...

document...

Posted: Fri Oct 15, 2010 3:58 am
by kannan_ayyanar
if any one having good document which explains very basics about partitioning. let me know... i searched in google.. i didnt get what i expected....


thanks in advance...

Posted: Fri Oct 15, 2010 4:39 am
by dsa
parallel job dev guide explains types of partitioning. Depending upon your requirement, you might want to go for round robin or hash or modulus or random partitioning to achieve the same.

Posted: Fri Oct 15, 2010 5:06 am
by kannan_ayyanar
job is simple... sequential file as input and output... and transformer in between.. ok if i go for a round robin partition in transformer.. how can i check the count of record processed in each node... ?

Posted: Fri Oct 15, 2010 5:55 am
by Sainath.Srinivasan
From monitor.

Posted: Wed Jan 05, 2011 12:41 pm
by Marley777
where in monitor can you see how many records were processed for each node?

Wondering if there is a break down per stage?

Example

Node1 processed 4 records
Node2 processed 7 records
etc...

Posted: Wed Jan 05, 2011 2:14 pm
by ray.wurlod
Right click in Monitor background area and enable "Show instances". Parallel stages will show as x 4 if collapsed, or as four separate instances if expanded.