Error on subtraction of two decimal(19,3) number
Moderators: chulett, rschirm, roy
Error on subtraction of two decimal(19,3) number
Hi, all
I have experienced that when I subtract two numbers,
datastage always give wrong results in the decimal places.
e.g. 123456789012345.123 - 0.000 ==> 123456789012345.xxx
I have changed related environment variables in Project but seems no
effect.
Does anyone know how to solve it ?
I have experienced that when I subtract two numbers,
datastage always give wrong results in the decimal places.
e.g. 123456789012345.123 - 0.000 ==> 123456789012345.xxx
I have changed related environment variables in Project but seems no
effect.
Does anyone know how to solve it ?
alphac
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
I doubt very much that it gives "xxx" as the fractional digits. Please advise one or two specific cases of incorrect subtraction, and whether it's all subtractions that give a wrong result, or only the occasional one.
I presume you are using a parallel Transformer stage here? Have you inspected the generated code?
I presume you are using a parallel Transformer stage here? Have you inspected the generated code?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Hi, ray
As mentioned, it always gives wrong answer in decimal places
e.g.
123456789012345.123 - 0.000 ==> 123456789012345.058
123456789012345.123 - 0.001 ==> 123456789012345.121
...
Not only the subtraction encountered this problem but also
using parallel function "NullToZero()".
I have tested that if a decimal number have 15 digits (exclude those decimal places ), the results will be wrong.
I am wondering it is the data type declaration of parallel function is not large enough to store the decimal number ...
(
As mentioned, it always gives wrong answer in decimal places
e.g.
123456789012345.123 - 0.000 ==> 123456789012345.058
123456789012345.123 - 0.001 ==> 123456789012345.121
...
Not only the subtraction encountered this problem but also
using parallel function "NullToZero()".
I have tested that if a decimal number have 15 digits (exclude those decimal places ), the results will be wrong.
I am wondering it is the data type declaration of parallel function is not large enough to store the decimal number ...
(
alphac
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
If what you say is true, and I'm not doubting you, then you may have uncovered a bug? What are the data type declarations in the generated C++ code?
I think your next step must be to report this suspected bug through your support provider. They will need a clearly documented reproducible case.
I think your next step must be to report this suspected bug through your support provider. They will need a clearly documented reproducible case.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
May I know what is the values for APT_DECIMAL_INTERM_SCALE and
APT_DECIMAL_INTERM_PRECISION?
Have you changed it, if so what is the new value?
Have you tried to set APT_DECIMAL_INTERM_ROUND_MODE to "Discard any unrepresentable fractional digits"
APT_DECIMAL_INTERM_PRECISION?
Have you changed it, if so what is the new value?
Have you tried to set APT_DECIMAL_INTERM_ROUND_MODE to "Discard any unrepresentable fractional digits"
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
Hi, kumar_s
Lets for sharing.
I have change setting of APT_DECIMAL_INTERM_SCALE and APT_DECIMAL_INTERM_PRECISION but never change on APT_DECIMAL_INTERM_ROUND_MODE.
It is now APT_DECIMAL_INTERM_SCALE = 38 and
APT_DECIMAL_INTERM_PRECISION = 10
Make APT_DECIMAL_INTERM_SCALE and APT_DECIMAL_INTERM_PRECISION larger could not fix the captioned issue.
What is the purpose of changing parameter "APT_DECIMAL_INTERM_ROUND_MODE" ?
Lets for sharing.
I have change setting of APT_DECIMAL_INTERM_SCALE and APT_DECIMAL_INTERM_PRECISION but never change on APT_DECIMAL_INTERM_ROUND_MODE.
It is now APT_DECIMAL_INTERM_SCALE = 38 and
APT_DECIMAL_INTERM_PRECISION = 10
Make APT_DECIMAL_INTERM_SCALE and APT_DECIMAL_INTERM_PRECISION larger could not fix the captioned issue.
What is the purpose of changing parameter "APT_DECIMAL_INTERM_ROUND_MODE" ?
alphac
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
By my understanding the ...INTERM... environment variables are for intermediate values, for example results within expressions that are not the final result. I believe that in the case that prompted this post we are looking at a final result, and that therefore the ...INTERM... environment variables will have no effect. But I'm happy to learn if it turns out not to be the case.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.