Hi,
My Job has 3 input datasets, one join stage, 1 transformer stage and a sequential file. I added the Environment variable APT_DUMP_SCORE in my job. I got below information in the log
......
ds9: {op6[2p] (parallel buffer(0))
[pp] eSame=>eCollectAny
op9[2p] (parallel APT_JoinSubOperator(0) in jnDli56Data)}
ds10: {op7[2p] (parallel buffer(1))
[pp] eSame=>eCollectAny
op9[2p] (parallel APT_JoinSubOperator(0) in jnDli56Data)}
ds11: {op8[2p] (parallel buffer(2))
[pp] eSame=>eCollectAny
op11[2p] (parallel APT_JoinSubOperator(1) in jnDli56Data)}
ds12: {op9[2p] (parallel APT_JoinSubOperator(0) in jnDli56Data)
[pp] eSame=>eCollectAny
op10[2p] (parallel buffer(3))}
ds13: {op10[2p] (parallel buffer(3))
[pp] eSame=>eCollectAny
op11[2p] (parallel APT_JoinSubOperator(1) in jnDli56Data)}
ds14: {op11[2p] (parallel APT_JoinSubOperator(1) in jnDli56Data)
[pp] eSame=>eCollectAny
op12[2p] (parallel APT_CombinedOperatorController(3):stJnKeyCngCol)}
.........
From the above log my doubt is,why many datasets has been assigned to a single join stage (jnDli56Data)?
Generally each stage will be assigned with single datastage but this join has been assigned to more datasets why?
Could any one please explain?
Thnaks
APT_DUMP_SCORE?
Moderators: chulett, rschirm, roy
Do you have a 4-node configuration file?
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
That's not how it works, folks. Irrespective of the degree of parallelism you only get one data set between each pair of operators. Remember that the Score is distributed separately to the section leader process on each node.
What you need to look at here is the fact that DataStage has inserted some buffer operators (to account for differences in processing speeds on the input links to the Join stage), and has generated two Join sub-operators in a composite - which is indicated by the "in" keyword.
The Operators section of the Score will make this clearer.
What you need to look at here is the fact that DataStage has inserted some buffer operators (to account for differences in processing speeds on the input links to the Join stage), and has generated two Join sub-operators in a composite - which is indicated by the "in" keyword.
The Operators section of the Score will make this clearer.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.