precision problem from sequential file to DB2

yimwai · Post by **yimwai** » Sun Sep 23, 2012 6:40 pm

source data:sequential file
source value:2324.6199999999999
destination database: db2
data type :decimal(23,10)

We use float datatype to read the file and transform it to decimal(23,10), then the value in the db is 2324.6201171875
why?
how can i make the db2 value is also 2324.6199999999

ray.wurlod · Post by **ray.wurlod** » Mon Sep 24, 2012 12:02 am

Try making the source data type VarChar or Char, rather than float.

Floating point numbers can not be fully accurately represented in digital computers; you can read about the IEEE 754 standard to understand why this is.

Conversion from string to decimal is far more likely to preserve the actual digits in the number represented in the text file.

yimwai · Post by **yimwai** » Mon Sep 24, 2012 8:39 pm

i use varchar to read the file and use StringtoDecimal Function, partly solve the problem.
but in file there is some data with scientific notation such as 2.34567E-2.
varchar cannot help

chulett · Post by **chulett** » Mon Sep 24, 2012 10:35 pm

An exact search here for "scientific notation" turned up a number of matches, many marked as Resolved. Check those out.

ray.wurlod · Post by **ray.wurlod** » Tue Sep 25, 2012 4:41 pm

Perhaps your derivation can include an If..Then..Else test looking for the E and using a different format string for StringToDecimal() function.