Decimal variable scale issue
Posted: Wed Oct 05, 2005 7:59 am
I have a decimal[24,6] value 'D1' and a decimal[20,2] value 'D2' going into a parallel transformer. Within that transformer I have a stage variable 'SV' declared as decimal[20,2].
In the transformer, I set SV = D1 * D2. I then convert SV to a string using DecimalToString( SV, "fix_zero,suppress_zero" ) and print the result to a file.
The problem is that I am seeing more than 2 decimal places in the output. For example,
which is the correct numerical value, except I want the result (and expected it to be) rounded to 184558.13.
Does this occur because DecimalToString() is displaying an underlying value which is actually stored with more than two decimal places? Or am I confused about the meaning of "precision" and "scale" in DataStage decimal variable declarations?
TIA
In the transformer, I set SV = D1 * D2. I then convert SV to a string using DecimalToString( SV, "fix_zero,suppress_zero" ) and print the result to a file.
The problem is that I am seeing more than 2 decimal places in the output. For example,
Code: Select all
D1 = 0.71449
D2 = 258307.5
0.71449 * 258307.5 = 184558.125675
Does this occur because DecimalToString() is displaying an underlying value which is actually stored with more than two decimal places? Or am I confused about the meaning of "precision" and "scale" in DataStage decimal variable declarations?
TIA