Page 1 of 1

Rounding of Decimals places

Posted: Tue Jul 15, 2008 7:05 pm
by cooperjv
I have a stage variable which I defined as Decimal(38,28). If in this stage variable I divide 1.000000000000000000000000000 by 3.0000000000000000000000000000. I get an output of 0000000000.3333333333000000000000000000 instead of 0000000000.333333333333333333333333333. Some how the last 18 digits of the output are all zeros. Could any one let me know how I can get the correct output

The input and output to this stage variable are Decimal(38,28)

Posted: Tue Jul 15, 2008 7:16 pm
by ray.wurlod
There is an environment variable that governs intermediate precision of decimal calculations. I'm not at a site where I can check, but its name is something like APT_DECIMAL_INTERM_PRECISION. This may help.