miwinter,
The qty column is decimal 19,6 and the amount column is decimal 22,6 in the source and in the aggregator. There are no conversions on these columns before or after the aggregator.
Thx!
aggregator sort method issue
Moderators: chulett, rschirm, roy
What are the inputs to the job using the aggregator(s)? If they are datasets, and the qty and amount attributes are being output as decimal, have they been converted from any other datatype further upstream?
I've just discovered the issue behind my problem, hence my line of questioning over the last day or so...
The field we were summing on is derived on input (from a dataset) as a string. However, in the preceding job, this is a floating point, ultimately converted to a string for flat file output. As such, we output the data in string format, depicted as a number with an exponent. In the job using the aggregator, a conversion takes place from string to decimal. The conversion of a string to decimal, when the input 'string' is actually an exponential number, appears to be flawed. I've further proven this by converting the string back to floating point and then performing the aggregation, observing that the aggregated results were consistent across six subsequent iterations of running the job - same results with both hash and sort method too, no less. As such, I'm concluding that the use of decimal is the issue here, where it started life as floating point. NB: I've also removed the aggregator option to output as decimal and allowed it to default output as floating point.
I've just discovered the issue behind my problem, hence my line of questioning over the last day or so...
The field we were summing on is derived on input (from a dataset) as a string. However, in the preceding job, this is a floating point, ultimately converted to a string for flat file output. As such, we output the data in string format, depicted as a number with an exponent. In the job using the aggregator, a conversion takes place from string to decimal. The conversion of a string to decimal, when the input 'string' is actually an exponential number, appears to be flawed. I've further proven this by converting the string back to floating point and then performing the aggregation, observing that the aggregated results were consistent across six subsequent iterations of running the job - same results with both hash and sort method too, no less. As such, I'm concluding that the use of decimal is the issue here, where it started life as floating point. NB: I've also removed the aggregator option to output as decimal and allowed it to default output as floating point.
Mark Winter
<i>Nothing appeases a troubled mind more than <b>good</b> music</i>
<i>Nothing appeases a troubled mind more than <b>good</b> music</i>
In the test job we have set up, it starts with a sequential file, goes into a copy, then one output from the copy goes to the aggregator 'hash' method to xfm to ora, and second output from the copy goes to the aggregator 'sort' method to xfm to ora.
When we compare the ora's, the sums of the QTY and AMT fields are different. (the ora's are 2 separate tables) We have the same input file, carry the same decimal type through to the tables, and have diffs.
That is fantastic that you figured out your issue so that you can get around it.
Thank you for sharing what you found in your testing. This is good information to have.
When we compare the ora's, the sums of the QTY and AMT fields are different. (the ora's are 2 separate tables) We have the same input file, carry the same decimal type through to the tables, and have diffs.
That is fantastic that you figured out your issue so that you can get around it.
Thank you for sharing what you found in your testing. This is good information to have.