Partition method behind Auto Partition.
Posted: Wed Dec 29, 2010 1:02 am
My Job design is as follows.
Sequential File --> Transformer --> Sequential File
At Transformer level the partition method set as Auto Partition. is it possible to find what partition method was used for parallel execution by using SCORE or Suggest any alternatives. Please find the Job score as follows.
ds0: {op0[1p] (sequential Sequential_File_0)
eAny<>eCollectAny
op1[4p] (parallel APT_TransformOperatorImplV0S2_Functions_TEST_Transformer_2 in Transformer_2)}
ds1: {op1[4p] (parallel APT_TransformOperatorImplV0S2_Functions_TEST_Transformer_2 in Transformer_2)
>>eCollectAny
op2[1p] (sequential APT_RealFileExportOperator in Sequential_File_1)}
It has 3 operators:
op0[1p] {(sequential Sequential_File_0)
on nodes (
node1[op0,p0]
)}
op1[4p] {(parallel APT_TransformOperatorImplV0S2_Functions_TEST_Transformer_2 in Transformer_2)
on nodes (
node1[op1,p0]
node2[op1,p1]
node3[op1,p2]
node4[op1,p3]
)}
op2[1p] {(sequential APT_RealFileExportOperator in Sequential_File_1)
on nodes (
node2[op2,p0]
)}
It runs 6 processes on 4 nodes.
Sequential File --> Transformer --> Sequential File
At Transformer level the partition method set as Auto Partition. is it possible to find what partition method was used for parallel execution by using SCORE or Suggest any alternatives. Please find the Job score as follows.
ds0: {op0[1p] (sequential Sequential_File_0)
eAny<>eCollectAny
op1[4p] (parallel APT_TransformOperatorImplV0S2_Functions_TEST_Transformer_2 in Transformer_2)}
ds1: {op1[4p] (parallel APT_TransformOperatorImplV0S2_Functions_TEST_Transformer_2 in Transformer_2)
>>eCollectAny
op2[1p] (sequential APT_RealFileExportOperator in Sequential_File_1)}
It has 3 operators:
op0[1p] {(sequential Sequential_File_0)
on nodes (
node1[op0,p0]
)}
op1[4p] {(parallel APT_TransformOperatorImplV0S2_Functions_TEST_Transformer_2 in Transformer_2)
on nodes (
node1[op1,p0]
node2[op1,p1]
node3[op1,p2]
node4[op1,p3]
)}
op2[1p] {(sequential APT_RealFileExportOperator in Sequential_File_1)
on nodes (
node2[op2,p0]
)}
It runs 6 processes on 4 nodes.