Error on subtraction of two decimal(19,3) number

alphac · Post by **alphac** » Tue Jul 25, 2006 9:25 pm

Hi, all

I have experienced that when I subtract two numbers,
datastage always give wrong results in the decimal places.

e.g. 123456789012345.123 - 0.000 ==> 123456789012345.xxx

I have changed related environment variables in Project but seems no
effect.

Does anyone know how to solve it ?

ray.wurlod · Post by **ray.wurlod** » Tue Jul 25, 2006 11:23 pm

I doubt very much that it gives "xxx" as the fractional digits. Please advise one or two specific cases of incorrect subtraction, and whether it's all subtractions that give a wrong result, or only the occasional one.

I presume you are using a parallel Transformer stage here? Have you inspected the generated code?

alphac · Post by **alphac** » Tue Jul 25, 2006 11:38 pm

Hi, ray

As mentioned, it always gives wrong answer in decimal places

e.g.
123456789012345.123 - 0.000 ==> 123456789012345.058
123456789012345.123 - 0.001 ==> 123456789012345.121
...

Not only the subtraction encountered this problem but also
using parallel function "NullToZero()".

I have tested that if a decimal number have 15 digits (exclude those decimal places ), the results will be wrong.

I am wondering it is the data type declaration of parallel function is not large enough to store the decimal number ...
(

ray.wurlod · Post by **ray.wurlod** » Tue Jul 25, 2006 11:53 pm

If what you say is true, and I'm not doubting you, then you may have uncovered a bug? What are the data type declarations in the generated C++ code?

I think your next step must be to report this suspected bug through your support provider. They will need a clearly documented reproducible case.

alphac · Post by **alphac** » Wed Jul 26, 2006 12:06 am

Thank you, ray.

Let me report this issue and kindly share my experience to the others.

kumar_s · Post by **kumar_s** » Wed Jul 26, 2006 2:37 am

May I know what is the values for APT_DECIMAL_INTERM_SCALE and
APT_DECIMAL_INTERM_PRECISION?
Have you changed it, if so what is the new value?
Have you tried to set APT_DECIMAL_INTERM_ROUND_MODE to "Discard any unrepresentable fractional digits"

alphac · Post by **alphac** » Wed Jul 26, 2006 3:14 am

Hi, kumar_s

Lets for sharing.
I have change setting of APT_DECIMAL_INTERM_SCALE and APT_DECIMAL_INTERM_PRECISION but never change on APT_DECIMAL_INTERM_ROUND_MODE.

It is now APT_DECIMAL_INTERM_SCALE = 38 and
APT_DECIMAL_INTERM_PRECISION = 10

Make APT_DECIMAL_INTERM_SCALE and APT_DECIMAL_INTERM_PRECISION larger could not fix the captioned issue.

What is the purpose of changing parameter "APT_DECIMAL_INTERM_ROUND_MODE" ?

kumar_s · Post by **kumar_s** » Wed Jul 26, 2006 3:23 am

By default the value of "APT_DECIMAL_INTERM_ROUND_MODE" would be "Round towards nearest representable value". You can experiment whether, change in this prarameter causes any difference in your issue.

ray.wurlod · Post by **ray.wurlod** » Wed Jul 26, 2006 4:01 pm

By my understanding the ...INTERM... environment variables are for intermediate values, for example results within expressions that are not the final result. I believe that in the case that prompted this post we are looking at a final result, and that therefore the ...INTERM... environment variables will have no effect. But I'm happy to learn if it turns out not to be the case.

alphac · Post by **alphac** » Wed Jul 26, 2006 9:49 pm

I've tested but seems these three ...INTERM... parameters do not affected the results.

Thanks for your suggestions again.