Hi,
I want to know, how internally Aggregator stage does count opertaion?
does it do the count on each partition and sum it out to the output?
or does it do it in a sequential mode?
Aggregator stage internal processing
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
No, it means just as Ray indicated: It does a count on each node that Agreggator is executing on.
If you need a single count across all nodes, you must run an Aggregator (or other stage) in a single node (i.e. sequential) in order to combine the counts. Aggregator is no different from other parallel stages in this regard.
Regards,
If you need a single count across all nodes, you must run an Aggregator (or other stage) in a single node (i.e. sequential) in order to combine the counts. Aggregator is no different from other parallel stages in this regard.
Regards,
- james wiles
All generalizations are false, including this one - Mark Twain.
All generalizations are false, including this one - Mark Twain.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
While the job is running in parallel (let's assume on four nodes) you will have four counts. Nothing will sum those together unless you create more stages to do so, or unless you run your Aggregator on a single node (that is, in Sequential mode).
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.