Aggregator stage Inconsistent output
Moderators: chulett, rschirm, roy
Aggregator stage Inconsistent output
Hi,
The requirement is to aggregate the Amount column based on Amount Type.
I am dealing with the data of around 45 million and currently it has only 2 amount types.
I am getting different results each time i run the job.
I used to Sort method with a preceding Sort stage and Sort on Amount type with Hash partition. I also tried using a single node configuration and sequential node also but i am getting the same result. Is it a bug in dstage?
Any help regarding this will be highly appreciated.
The requirement is to aggregate the Amount column based on Amount Type.
I am dealing with the data of around 45 million and currently it has only 2 amount types.
I am getting different results each time i run the job.
I used to Sort method with a preceding Sort stage and Sort on Amount type with Hash partition. I also tried using a single node configuration and sequential node also but i am getting the same result. Is it a bug in dstage?
Any help regarding this will be highly appreciated.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Please confirm:
(a) the names of your grouping columns
(b) the names of your hash key columns
(c) the names of your sort key columns
(d) the Aggregator operation method (hash or sort)
(a) the names of your grouping columns
(b) the names of your hash key columns
(c) the names of your sort key columns
(d) the Aggregator operation method (hash or sort)
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
(a) the names of your grouping columns -Amount Type
(b) the names of your hash key columns - Amount Type
(c) the names of your sort key columns - Amount Type
(d) the Aggregator operation method (hash or sort) - Sort ...
And Amount Type has only 2 values in my data sample.
The data is static, I am reading it from a flat file
(b) the names of your hash key columns - Amount Type
(c) the names of your sort key columns - Amount Type
(d) the Aggregator operation method (hash or sort) - Sort ...
And Amount Type has only 2 values in my data sample.
The data is static, I am reading it from a flat file
The answers to the other questions are important as well. Also, how are you detecting that the output is different? Are there just 2 output rows and values are different? If so, what is the output column type?
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
I have given all the required information for the questions...ArndW wrote:The answers to the other questions are important as well. Also, how are you detecting that the output is different? Are there just 2 output rows and values are different? If so, what is the output col ...
if i run for the first time i am getting say 10,000 amount but when i run it the next time it changes to 10,001. this one is just an example the real values are huge...
I see that you went back and re-edited an earlier post to add that missing information.nvuradi wrote:I have given all the required information for the questions...
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 2
- Joined: Wed Dec 03, 2008 10:44 pm
so how is it?
is it just because of floating point?
what if the output difference is huge in ammount.
let say, 1st run, the job return the total value 10,000
2nd run, the job return 12,000.
any ideas?
i'm using Sort operation method in Aggregator. The source is static, and from Oracle DB using Oracle Stage.
The output is to Oracle Stage.
is this a bug in Oracle Stage?
Thanks
is it just because of floating point?
what if the output difference is huge in ammount.
let say, 1st run, the job return the total value 10,000
2nd run, the job return 12,000.
any ideas?
i'm using Sort operation method in Aggregator. The source is static, and from Oracle DB using Oracle Stage.
The output is to Oracle Stage.
is this a bug in Oracle Stage?
Thanks