Page 1 of 1

double datatype weirdness

Posted: Mon Nov 23, 2015 4:50 pm
by hsahay
Something weird is happening and i can't tell why.

I send in two values in a comma separated sequential file -

9999999999, 111 (that is ten 9s)

All columns and the stage variable in the job are defined as double 20,5.

In the transformer i set a stage variable V = max(val1, val2)

Then i output the stage variable to another sequential file. Data type through out the job is double 20,5.

Now,

In LINUX, i get the following output in the file -

1.11000000000000000E+02

So basically Linux datastage thinks that between 9999999999 and 111, 111 is the max.

In AIX i get the following output -

2.14748364700000000E+09

Which is neither 111 nor 9999999999.

So any idea what is going on ?


Please note that if i use just nine 9s instead of ten ...it works okay giving me 9.99999999000000000E+08

Which means when i use a value that has ten 9s, some kind of overflow is occurring turning the value into a negative number perhaps ??

Posted: Mon Nov 23, 2015 5:12 pm
by ray.wurlod
Without addressing your particular problem, I observe that 2.14748364700000000E+09 is the largest 32-bit signed integer value.

And also that 999999999 can be represented as a 32-bit integer byt 9999999999 cannot.

Posted: Mon Nov 23, 2015 5:19 pm
by hsahay
Yes sir !

I noticed it too.

But what does that signify. Why is it happening only on AIX ?
Why is LINUX putting out 111 ?

What can we try to fix this behavior ? Is there anything bigger than double that we can use ?

Is double restricted to signed 32 bit in datastage ?

Posted: Mon Nov 23, 2015 6:15 pm
by ray.wurlod
Why are you using the ",5" piece? Your numbers do not appear to have any decimal places.

Have you tried using Decimal as the data type? According to IEEE standards, Double can never be assured to stored with perfect accuracy, even though you've clearly specified a large enough mantissa.

May I further suggest that you dump the score from each job (the one on Linux and the one on AIX) and try to discern any difference between them? Possibly even post them here.

Posted: Tue Nov 24, 2015 1:39 pm
by sjfearnside
Could this have any thing to do with a difference in endianness? I converted many years back from a Linux to an AIX system and ran into this difference.

Posted: Tue Nov 24, 2015 2:12 pm
by qt_ky
Look what the function is expecting and returning vs data types you are passing and using as stage variables.

The Max() function returns the greater of the two argument values.
- Input: number1 (int32),number2(int32)
- Output: result (int32)

Have you tried If Col1 > Col2 Then ...?

Posted: Tue Nov 24, 2015 7:30 pm
by hsahay
qt_ky - thank you ...i think that explains it.

I will mark this thread as resolved.

Posted: Tue Nov 24, 2015 10:22 pm
by ray.wurlod
On a twos-complement machine 9999999999 would have wrapped around, setting the sign bit for an Integer, which is why it would be interpreted as being less than 111.