Problem with calculation in transformer

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
caltog
Premium Member
Premium Member
Posts: 39
Joined: Thu May 22, 2003 7:32 am

Problem with calculation in transformer

Post by caltog »

Hello,

I have to do a simple calculation with 2 field.
The first field is a decimal(30,8 ) the second one is a decimal(30,8 ) too.
The value of the first field is : 1324414646

The value of the second field is : 1301128551.1

The result is : 23286094.9000001

Edit : Oups.. The calculation is field one - field two

Why the result isn't : 23286094.9000000


Thanks
Last edited by caltog on Mon Oct 08, 2007 6:28 am, edited 1 time in total.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Any 'calculation' in particular or do we get to figure that out? :?
-craig

"You can never have too many knives" -- Logan Nine Fingers
caltog
Premium Member
Premium Member
Posts: 39
Joined: Thu May 22, 2003 7:32 am

Post by caltog »

chulett wrote:Any 'calculation' in particular or do we get to figure that out? :?
The calculation is a simple minus.

I try to make a simple job with this query :
SELECT 1324414646, 1301128551.1 FROM DUAL

I send the result in a flat file then i made the minus in the transformer and i send to another flat file with one more field calculation who has the result 23286094.9000001

When I try with SELECT 1324414646.3, 1301128551.2 FROM DUAL
the result is 23286095.0999999

Any idea ?

I use DS 7.0

Thanks
Ronetlds
Participant
Posts: 28
Joined: Thu Mar 30, 2006 12:48 pm

Post by Ronetlds »

My in file:
Filed1,Field2

SELECT 1301128551.1,1324414646 FROM DUAL

My out file
"23286094.9"

all fields dec(30,8)

deriv: DSLink77.Two- DSLink77.One

Can't recreate error - mine works fine
DeepakCorning
Premium Member
Premium Member
Posts: 503
Joined: Wed Jun 29, 2005 8:14 am

Post by DeepakCorning »

What query tool are you using to see the data?? Try multiplying the numbers by 1 and get those results , means --

select fld1 * 1 , fld2 * 1 from dual

and let us know the results.
Thanks
Deepak Patil

Convince Them Confuse Them .. What's the difference?
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

There is a configurable limit to the PRECISION (number of significant digits) in DataStage BASIC.

What happens when you use SSub(string1,string2) function, where the two arguments are your two numbers? This is an example of DataStage's "string math" functions, which can be used to get around precision limits for simple arithmetic calculation.

You could, of course, alter the precision.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
caltog
Premium Member
Premium Member
Posts: 39
Joined: Thu May 22, 2003 7:32 am

Post by caltog »

ray.wurlod wrote:There is a configurable limit to the PRECISION (number of significant digits) in DataStage BASIC.

What happens when you use SSub(string1,string2) function, where the two arguments are your two numbers? This is an example of DataStage's "string math" functions, which can be used to get around precision limits for simple arithmetic calculation.

You could, of course, alter the precision.
Hi people thanks for your help.

I made some test and when the precision is less than 7 the result is correct.
I try the SSub function and It's working fine thanks a lot.

Have nice day.
Post Reply