partition data not showing on one node

[vinsashi]

Hi,
i am running one job on two nodes but it not showing records on two nodes..

Peek_7: When checking operator: Operator of type "APT_PeekOperator": will partition despite the
preserve-partitioning flag on the data set on input port 0.

Code: Select all

main_program: This step has 2 datasets:
ds0: {op0[1p] (sequential Row_Generator_0)
      eOther(APT_HashPartitioner { key={ value=eid, 
        subArgs={ cs }
      }
})<>eCollectAny
      op1[2p] (parallel Sort_11)}
ds1: {op1[2p] (parallel Sort_11)
      [pp] eOther(APT_HashPartitioner { key={ value=eid, 
        subArgs={ cs }
      }
})#>eCollectAny
      op2[2p] (parallel Peek_7)}
It has 3 operators:
op0[1p] {(sequential Row_Generator_0)
    on nodes (
      node1[op0,p0]
    )}
op1[2p] {(parallel Sort_11)
    on nodes (
      node1[op1,p0]
      node2[op1,p1]
    )}
op2[2p] {(parallel Peek_7)
    on nodes (
      node1[op2,p0]
      node2[op2,p1]
    )}
It runs 5 processes on 2 nodes.

but job running on two nodes..


thanks
vi.

[nikhilanshuman]

The operators are running on two nodes.

The sequential_row_generator is a sequential stage.It will not run on two nodes.The sort and peek stage are running on two nodes as shown in bold below.(node1 and node2)

op0[1p] {(sequential Row_Generator_0)
on nodes (
node1[op0,p0]
)}
op1[2p] {(parallel Sort_11)
on nodes (
node1[op1,p0]
node2[op1,p1]
)}
op2[2p] {(parallel Peek_7)
on nodes (
node1[op2,p0]
node2[op2,p1]
)}

The message :

Peek_7: When checking operator: Operator of type "APT_PeekOperator": will partition despite the
preserve-partitioning flag on the data set on input port 0.


says that although in the previous stage,the preserve partitioning flag is set,it is going to partition the data.

To remove this warning,go to the stage which is prior to the peek stage.Now,go to stage->Adavenced and set the value of "Preserve Partioning" to "Clear"