float/decimal differences in 8.1 vs 7.5.1a

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Bill_G
Premium Member
Premium Member
Posts: 74
Joined: Thu Oct 20, 2005 9:34 am

float/decimal differences in 8.1 vs 7.5.1a

Post by Bill_G »

I am in the process of moving to an 8.1 FP1 environment from 7.5.1a. During data validation, I noticed that my decimal and float results were differen in 8.1 vs the same job in 7.5.1a.

Jobs are the same and I am using a clone of my production Data Warehouse for a specific date and time (copied out my staging tables before the cycle ran for comparison purposes).

Significant figures issue seen on qty column.
Example

7.5.1a value = 0.18
8.1 value = 0.17999999999999999

My Environment Variable settings for the APT_DECIMAL_PRECISION, ROUNDMODE, and SCALE match between environments.

Is this a bug in 7.5.1a we have been living with for 6 years?

Thanks

Bill
creatingfusion
Participant
Posts: 46
Joined: Tue Jul 20, 2010 1:26 pm
Location: USA
Contact:

Re: float/decimal differences in 8.1 vs 7.5.1a

Post by creatingfusion »

7.5.1a value = 0.18
8.1 value = 0.17999999999999999

These variation in the value......
There being two fields in the metadata definition of the column one being length and the other being display.
Put the display as 2 more than the length. The more here reflects one for the precision and the other being for the decimal point.

So if the datatype being defined for 10 characters length define the display as 12.

Make the changes and verify the result.
Bill_G
Premium Member
Premium Member
Posts: 74
Joined: Thu Oct 20, 2005 9:34 am

Post by Bill_G »

I'd prefer not to modify all the jobs in my daily cycle.

I should also note I am writing to Oracle - using the Enterprise Stage from 7.5.1a.

Thanks
priyadarshikunal
Premium Member
Premium Member
Posts: 1735
Joined: Thu Mar 01, 2007 5:44 am
Location: Troy, MI

Post by priyadarshikunal »

Are you using "View Data" to derive the conclusion or its same in database too?
Priyadarshi Kunal

Genius may have its limitations, but stupidity is not thus handicapped. :wink:
Bill_G
Premium Member
Premium Member
Posts: 74
Joined: Thu Oct 20, 2005 9:34 am

Post by Bill_G »

Prior to my 8.1 cycle run, I copied out my staging tables, then run an 8.1 cycle, then compare the 8.1 staging tables to the 7.5.1a version of the staging tables. It is here, in the Oracle DB (same DB instance/same schema - copy of the table with a different name) that I see the difference.

I am not using view data to compare the data.
priyadarshikunal
Premium Member
Premium Member
Posts: 1735
Joined: Thu Mar 01, 2007 5:44 am
Location: Troy, MI

Post by priyadarshikunal »

still, can you check in source, what the exact value is?

As you might already know that computer systems have problems with storing/processing float/double.

Even if it complies to IEEE 754 standards it can change the data by rounding in case of bit overflow.

17 decimal points??? seems a bit large for float and even for double. In case its defined as float, i am not surprised to see it rounded.


Also, Please compare UVConfig file to see the differences.
Priyadarshi Kunal

Genius may have its limitations, but stupidity is not thus handicapped. :wink:
Post Reply