I am in the process of moving to an 8.1 FP1 environment from 7.5.1a. During data validation, I noticed that my decimal and float results were differen in 8.1 vs the same job in 7.5.1a.
Jobs are the same and I am using a clone of my production Data Warehouse for a specific date and time (copied out my staging tables before the cycle ran for comparison purposes).
Significant figures issue seen on qty column.
Example
7.5.1a value = 0.18
8.1 value = 0.17999999999999999
My Environment Variable settings for the APT_DECIMAL_PRECISION, ROUNDMODE, and SCALE match between environments.
Is this a bug in 7.5.1a we have been living with for 6 years?
Thanks
Bill
float/decimal differences in 8.1 vs 7.5.1a
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 46
- Joined: Tue Jul 20, 2010 1:26 pm
- Location: USA
- Contact:
Re: float/decimal differences in 8.1 vs 7.5.1a
7.5.1a value = 0.18
8.1 value = 0.17999999999999999
These variation in the value......
There being two fields in the metadata definition of the column one being length and the other being display.
Put the display as 2 more than the length. The more here reflects one for the precision and the other being for the decimal point.
So if the datatype being defined for 10 characters length define the display as 12.
Make the changes and verify the result.
8.1 value = 0.17999999999999999
These variation in the value......
There being two fields in the metadata definition of the column one being length and the other being display.
Put the display as 2 more than the length. The more here reflects one for the precision and the other being for the decimal point.
So if the datatype being defined for 10 characters length define the display as 12.
Make the changes and verify the result.
-
- Premium Member
- Posts: 1735
- Joined: Thu Mar 01, 2007 5:44 am
- Location: Troy, MI
Prior to my 8.1 cycle run, I copied out my staging tables, then run an 8.1 cycle, then compare the 8.1 staging tables to the 7.5.1a version of the staging tables. It is here, in the Oracle DB (same DB instance/same schema - copy of the table with a different name) that I see the difference.
I am not using view data to compare the data.
I am not using view data to compare the data.
-
- Premium Member
- Posts: 1735
- Joined: Thu Mar 01, 2007 5:44 am
- Location: Troy, MI
still, can you check in source, what the exact value is?
As you might already know that computer systems have problems with storing/processing float/double.
Even if it complies to IEEE 754 standards it can change the data by rounding in case of bit overflow.
17 decimal points??? seems a bit large for float and even for double. In case its defined as float, i am not surprised to see it rounded.
Also, Please compare UVConfig file to see the differences.
As you might already know that computer systems have problems with storing/processing float/double.
Even if it complies to IEEE 754 standards it can change the data by rounding in case of bit overflow.
17 decimal points??? seems a bit large for float and even for double. In case its defined as float, i am not surprised to see it rounded.
Also, Please compare UVConfig file to see the differences.
Priyadarshi Kunal
Genius may have its limitations, but stupidity is not thus handicapped.
Genius may have its limitations, but stupidity is not thus handicapped.