Decimal Scale on Calculation

hchellham · Post by **hchellham** » Mon Jun 28, 2010 5:41 am

Hi Members,

The Data Types and the values are:
InCol (Decimal 24,6) = 000000000000100000.000000
OutCol (Decimal 24,16)

The requirement is to perform the following calculation:
OutCol = ( (InCol * 0.05) /12 ) (Also tried by substituting 12.00)

The output of our job is OutCol = 00000416.6666666667000000

The Desired Output is OutCol = 00000416.6666666666666666 (Please note that the last 16th digit is not rounded too).

Could anyone suggest some methods to acheive the desired output? Thanks in advance!

Sainath.Srinivasan · Post by **Sainath.Srinivasan** » Mon Jun 28, 2010 7:22 am

There is a setting for the default decimal scale which is set to 10 (by default).

Try increasing the value from Admin.

Also try decimalToDecimal.

SBSA_DW2 · Post by **SBSA_DW2** » Mon Jun 28, 2010 7:32 am

the setting mentioned is:

APT_DECIMAL_INTERM_SCALE
Specifies the default scale value for any decimal intermediate
variables required in calculations. Default value is 10.
(page 6-14 of the parallel job advanced user guide)

ray.wurlod · Post by **ray.wurlod** » Mon Jun 28, 2010 3:26 pm

Add environment variable APT_DECIMAL_INTERM_SCALE to your job as a parameter.

hchellham · Post by **hchellham** » Tue Jun 29, 2010 4:21 am

ray.wurlod wrote:Add environment variable APT_DECIMAL_INTERM_SCALE to your job as a parameter. ...

Hi,

Thanks everyone!

Setting the value of APT_DECIMAL_INTERM_SCALE as 17 had produced the desired result. When using the value as 16, the last digit came with a rounded off value, hence went for value 17 and declared the target column meta data as (24,16).