Problem while using Head stage

[datisaq]

From the source i have 20 records say 1 to 20.And i'm having a 4 node configuration file.I used the round robin partitioning mechanism, so my output should be

node0 node1 node2 node3
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
17 18 19 20
I given the output to a head stage, i specified the following options:-

No of rows per partition=10
All Partitions=True
I got the proper output as mentioned above.

If i specify the property:-
No of rows per partition=10
All Partitions=False
Partition Number=1

I got the output as:-
5 6 7 8
But the output shouldbe
2 6 10 14 18
since this are the records passed to the node1.

Please let me know if i'm wrong..

Need your valuable inputs...

[ArndW]

Sounds like you've discovered a potential bug in the stage.

[datisaq]

Thanks Arndw for your prompt reply.
Could you please check this at your end and confirm this?

[ArndW]

Sorry, no time at the moment but perhaps someone else might have time. I've been here at work 12 hours and am working on other problems (for which I'm being paid)

[Sainath.Srinivasan]

Try hash or modulus partition and check the result.

[datisaq]

For Hash and modulus partitioning it's working fine but for round robin method the output is incorrect.
Following are the results:-

For Hash:-

node0 node1 node2 node3
9 5 6 1
18 12 8 2
19 14 7 10
16 4
13 20
3
17
11
15

Output of Head stage(Correct Result):-
5 12 14 16 13 3 17 11 15


For Modulus:-

node0 node1 node2 node3
4 1 2 3
8 5 6 7
12 9 10 11
16 13 14 15
20 17 18 19

Output of Head stage(Correct Result):-
1 5 7 9 13 17

For Round Robin:-
node0 node1 node2 node3
1 2 3 4 ---> Row0
5 6 7 8 ---> Row1
9 10 11 12 ---> Row2
13 14 15 16 ---> Row3
17 18 19 20 ---> Row0

Output of Head stage(InCorrect Result):-
5 6 7 8
Expecting output from node1 that's:- 2 6 10 14 18

If i change the partition number=2 then the output is:-
9 10 11 12
If partition number=0 then the output is:-
1 2 3 4
17 18 19 20

So it's working rowwise(horizontally) for round robin based on the configuration file.

Please be cautious while using round robin method with Head stage.

Code: Select all

[quote][/quote]

[ethelvina]

It is working fine.
I used Rowgenerator-Sortstage(RoundRobin Partition)-Head Stage
Try this

[datisaq]

But the stage shlould work for all the scenarios..

[datisaq]

The problem is resolved now.
Actually my job design is :

rowgen---->copy--->Headstage--->peek

At copy stage i given earlier partioning method as "Auto" since it's a sequential to parallel process from row generator to copy stage and at Head Stage it's "round robin".Because of "Auto" partioning i got in correct output.

When i changed the partioning method as "Same" at input of copy stage i got correct output.

Thanks all for your help...