Hello,
I have to do a simple calculation with 2 field.
The first field is a decimal(30,8 ) the second one is a decimal(30,8 ) too.
The value of the first field is : 1324414646
The value of the second field is : 1301128551.1
The result is : 23286094.9000001
Edit : Oups.. The calculation is field one - field two
Why the result isn't : 23286094.9000000
Thanks
Problem with calculation in transformer
Moderators: chulett, rschirm, roy
Problem with calculation in transformer
Last edited by caltog on Mon Oct 08, 2007 6:28 am, edited 1 time in total.
The calculation is a simple minus.chulett wrote:Any 'calculation' in particular or do we get to figure that out?
I try to make a simple job with this query :
SELECT 1324414646, 1301128551.1 FROM DUAL
I send the result in a flat file then i made the minus in the transformer and i send to another flat file with one more field calculation who has the result 23286094.9000001
When I try with SELECT 1324414646.3, 1301128551.2 FROM DUAL
the result is 23286095.0999999
Any idea ?
I use DS 7.0
Thanks
-
- Premium Member
- Posts: 503
- Joined: Wed Jun 29, 2005 8:14 am
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
There is a configurable limit to the PRECISION (number of significant digits) in DataStage BASIC.
What happens when you use SSub(string1,string2) function, where the two arguments are your two numbers? This is an example of DataStage's "string math" functions, which can be used to get around precision limits for simple arithmetic calculation.
You could, of course, alter the precision.
What happens when you use SSub(string1,string2) function, where the two arguments are your two numbers? This is an example of DataStage's "string math" functions, which can be used to get around precision limits for simple arithmetic calculation.
You could, of course, alter the precision.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Hi people thanks for your help.ray.wurlod wrote:There is a configurable limit to the PRECISION (number of significant digits) in DataStage BASIC.
What happens when you use SSub(string1,string2) function, where the two arguments are your two numbers? This is an example of DataStage's "string math" functions, which can be used to get around precision limits for simple arithmetic calculation.
You could, of course, alter the precision.
I made some test and when the precision is less than 7 the result is correct.
I try the SSub function and It's working fine thanks a lot.
Have nice day.