Mathematical Function FABS return value
Posted: Tue Jun 24, 2008 11:13 am
FABS(00000067767332.29 * 100) returning me 6776733228 when the target is BigInt, where as it is returning 6776733229 when target is decimal.
It is giving correct values for most of values but for the following values it is giving wrong values.
00000067767332.29
00000067117893.84
00000065365551.43
00000057435593.04
For ths following one , it is giving correct value.
000000234168268.89
I am using this function to capture the decimal part of the incoming value, by multiplying it with 100 and then applying FABS to get absolute value( if i apply the FABS before multiplying with 100, it is truncating the decimal part) and then devide the result with 100 and capture the reminder using Modulus function(Mod). It is behaving strange for some values and most of the time it is goiving correct values.
I modified the code to multiply with -100 when input is < 0 otherwise multiply with 100 to get the desired result by avoiding the FABS function.
It is giving correct values for most of values but for the following values it is giving wrong values.
00000067767332.29
00000067117893.84
00000065365551.43
00000057435593.04
For ths following one , it is giving correct value.
000000234168268.89
I am using this function to capture the decimal part of the incoming value, by multiplying it with 100 and then applying FABS to get absolute value( if i apply the FABS before multiplying with 100, it is truncating the decimal part) and then devide the result with 100 and capture the reminder using Modulus function(Mod). It is behaving strange for some values and most of the time it is goiving correct values.
I modified the code to multiply with -100 when input is < 0 otherwise multiply with 100 to get the desired result by avoiding the FABS function.