Hi,
I am reading "amount" from a fixed width seq file as a Char e.g (2259 or 36153.35) and then the only transformation I do before writting to an Oracle table is do type conversion using StringToDecimal.
For some strange reason it stores these values as 2259.580000000000 and 36153.349999999999 respectively. (oracle data type is number 24,12). The first one is correct where as the second one is incorrect.
any idea why?
Reg,
NV
StringToDecimal Conversion Error
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 783
- Joined: Mon Jan 16, 2006 10:17 pm
- Location: Sydney, Australia
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
The conversion is attempting to give you 12 decimal places. In the second case you are getting rounding errors to an acceptable degree of precision (which you can prevent with coding).
The first one looks like a bug; maybe you need to test for the absence of a decimal place holder and add a decimal place and at least one decimal place digit before converting. For example:
The first one looks like a bug; maybe you need to test for the absence of a decimal place holder and add a decimal place and at least one decimal place digit before converting. For example:
Code: Select all
If Index(InLink.TheString, ".", 1) = 0 Then StringToDecimal(InLink.TheString : ".00") Else StringToDecimal(InLink.TheString)
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.