Difference in calculation for decimal (38,10) fields.

[yousuff1710]

Hi All,

I am doing calculation for decimal fields: (A-1)/(A*B)*C
In datastage the result is differing.
All input fileds i.e A,B,C are of datatype Numeric (38,10) & also o/p field.
Below are the values of i/p fields used for calculating;
A = 2.0789281794
B = 0.05
C = 49579862.4
The result datastage giving through Transformer = 514622017.1393123702
But the actual result should be = 514622017.2894397446
There is difference of 0.15; difference is in decimal field.

Any ideas or solutions are welcome.

Thanks in advance,
Yusuf.

[ShaneMuir]

The only thing I can think of is a rounding error. If datastage was rounding off each time after 10 decimal places its quite possible that you would end up 0.15 out when multiplying by 50mill.

[miwinter]

I'd agree. Look at an explicit decimal conversion so you have greater control over rounding.

[yousuff1710]

Hi Shane,

Thanks for reply.

I am passing all these values as Numeric (38,10) without any rounding function.
Is datastage rounding these decimals internally.
If yes, then how to get correct result.

Please give a solution for this.

Regards,
Yusuf.

[go4it]

There are environment variables for handling decimal data:

APT_DECIMAL_INTERM_ROUND_MODE, descrption is it specifies the default rounding mode for any decimal intermediate variables required in calculations. The default is round_inf.

APT_DECIMAL_INTERM_SCALE Specifies the default scale value for any decimal intermediate variables required in calculations. Default value is 10.

Check the above env variables and set it accordingly

M

[ShaneMuir]

I am passing all these values as Numeric (38,10) without any rounding function.
Is datastage rounding these decimals internally.
If yes, then how to get correct result.

Please give a solution for this.

Regards,
Yusuf.

I am not sure if datastage is rounding. But if you do those calculations step by step you get up to 20 decimal places. If datastage process the output each step as (38,10) then yes it will do some rounding to put it into that format before taking the next step. Though i always thought that datastage did the calcuation with no rounding then rounded the output to the specified format.

My only suggestion is to allow for more decimal places so that implicit rounding will not occur.

[chulett]

There are environment variables for handling decimal data:

APT_DECIMAL_INTERM_ROUND_MODE, descrption is it specifies the default rounding mode for any decimal intermediate variables required in calculations. The default is round_inf.

APT_DECIMAL_INTERM_SCALE Specifies the default scale value for any decimal intermediate variables required in calculations. Default value is 10.

Check the above env variables and set it accordingly

M

M - APT environment variables are not applicable in a Server question.

[chulett]

I agree with Shane, your intermediate calculations go well over your scale of 10.

[miwinter]

M - APT environment variables are not applicable in a Server question.

I was also misled by the OP by this post being at cross-purposes. Stated as a server job issue yet posted in the parallel forum...

[chulett]

I never go by the forum any more, God Knows where posts end up nowadays.

[miwinter]

Roger that :D

[ray.wurlod]

Is that because folks can also put anything anywhere in the Repository tree, do you think?

[yousuff1710]

Sorry guys, for posting it as Server job.
I forgot to select it as Parallel job in the dropdown list while posting.

Yusuf.

[yousuff1710]

Thanks Guys.....

I used environment variable APT_DECIMAL_INTERM_SCALE = 12.
Now the job is giving correct results.

Thanks again for all your precious time.