Page 1 of 1

precision problem from sequential file to DB2

Posted: Sun Sep 23, 2012 6:40 pm
by yimwai
source data:sequential file
source value:2324.6199999999999
destination database: db2
data type :decimal(23,10)

We use float datatype to read the file and transform it to decimal(23,10), then the value in the db is 2324.6201171875
why?
how can i make the db2 value is also 2324.6199999999

Posted: Mon Sep 24, 2012 12:02 am
by ray.wurlod
Try making the source data type VarChar or Char, rather than float.

Floating point numbers can not be fully accurately represented in digital computers; you can read about the IEEE 754 standard to understand why this is.

Conversion from string to decimal is far more likely to preserve the actual digits in the number represented in the text file.

Posted: Mon Sep 24, 2012 8:39 pm
by yimwai
i use varchar to read the file and use StringtoDecimal Function, partly solve the problem.
but in file there is some data with scientific notation such as 2.34567E-2.
varchar cannot help

Posted: Mon Sep 24, 2012 10:35 pm
by chulett
An exact search here for "scientific notation" turned up a number of matches, many marked as Resolved. Check those out.

Posted: Tue Sep 25, 2012 4:41 pm
by ray.wurlod
Perhaps your derivation can include an If..Then..Else test looking for the E and using a different format string for StringToDecimal() function.