Page 1 of 1
Error on subtraction of two decimal(19,3) number
Posted: Tue Jul 25, 2006 9:25 pm
by alphac
Hi, all
I have experienced that when I subtract two numbers,
datastage always give wrong results in the decimal places.
e.g. 123456789012345.123 - 0.000 ==> 123456789012345.xxx
I have changed related environment variables in Project but seems no
effect.
Does anyone know how to solve it ?
Posted: Tue Jul 25, 2006 11:23 pm
by ray.wurlod
I doubt very much that it gives "xxx" as the fractional digits. Please advise one or two specific cases of incorrect subtraction, and whether it's all subtractions that give a wrong result, or only the occasional one.
I presume you are using a parallel Transformer stage here? Have you inspected the generated code?
Posted: Tue Jul 25, 2006 11:38 pm
by alphac
Hi, ray
As mentioned, it always gives wrong answer in decimal places
e.g.
123456789012345.123 - 0.000 ==> 123456789012345.058
123456789012345.123 - 0.001 ==> 123456789012345.121
...
Not only the subtraction encountered this problem but also
using parallel function "NullToZero()".
I have tested that if a decimal number have 15 digits (exclude those decimal places ), the results will be wrong.
I am wondering it is the data type declaration of parallel function is not large enough to store the decimal number ...
(
Posted: Tue Jul 25, 2006 11:53 pm
by ray.wurlod
If what you say is true, and I'm not doubting you, then you may have uncovered a bug? What are the data type declarations in the generated C++ code?
I think your next step must be to report this suspected bug through your support provider. They will need a clearly documented reproducible case.
Posted: Wed Jul 26, 2006 12:06 am
by alphac
Thank you, ray.
Let me report this issue and kindly share my experience to the others.
Posted: Wed Jul 26, 2006 2:37 am
by kumar_s
May I know what is the values for APT_DECIMAL_INTERM_SCALE and
APT_DECIMAL_INTERM_PRECISION?
Have you changed it, if so what is the new value?
Have you tried to set APT_DECIMAL_INTERM_ROUND_MODE to "Discard any unrepresentable fractional digits"
Posted: Wed Jul 26, 2006 3:14 am
by alphac
Hi, kumar_s
Lets for sharing.
I have change setting of APT_DECIMAL_INTERM_SCALE and APT_DECIMAL_INTERM_PRECISION but never change on APT_DECIMAL_INTERM_ROUND_MODE.
It is now APT_DECIMAL_INTERM_SCALE = 38 and
APT_DECIMAL_INTERM_PRECISION = 10
Make APT_DECIMAL_INTERM_SCALE and APT_DECIMAL_INTERM_PRECISION larger could not fix the captioned issue.
What is the purpose of changing parameter "APT_DECIMAL_INTERM_ROUND_MODE" ?
Posted: Wed Jul 26, 2006 3:23 am
by kumar_s
By default the value of "APT_DECIMAL_INTERM_ROUND_MODE" would be "Round towards nearest representable value". You can experiment whether, change in this prarameter causes any difference in your issue.
Posted: Wed Jul 26, 2006 4:01 pm
by ray.wurlod
By my understanding the ...INTERM... environment variables are for intermediate values, for example results within expressions that are not the final result. I believe that in the case that prompted this post we are looking at a final result, and that therefore the ...INTERM... environment variables will have no effect. But I'm happy to learn if it turns out not to be the case.
Posted: Wed Jul 26, 2006 9:49 pm
by alphac
I've tested but seems these three ...INTERM... parameters do not affected the results.
Thanks for your suggestions again.