double datatype weirdness

hsahay · Post by **hsahay** » Mon Nov 23, 2015 4:50 pm

Something weird is happening and i can't tell why.

I send in two values in a comma separated sequential file -

9999999999, 111 (that is ten 9s)

All columns and the stage variable in the job are defined as double 20,5.

In the transformer i set a stage variable V = max(val1, val2)

Then i output the stage variable to another sequential file. Data type through out the job is double 20,5.

Now,

In LINUX, i get the following output in the file -

1.11000000000000000E+02

So basically Linux datastage thinks that between 9999999999 and 111, 111 is the max.

In AIX i get the following output -

2.14748364700000000E+09

Which is neither 111 nor 9999999999.

So any idea what is going on ?

Please note that if i use just nine 9s instead of ten ...it works okay giving me 9.99999999000000000E+08

Which means when i use a value that has ten 9s, some kind of overflow is occurring turning the value into a negative number perhaps ??

ray.wurlod · Post by **ray.wurlod** » Mon Nov 23, 2015 5:12 pm

Without addressing your particular problem, I observe that 2.14748364700000000E+09 is the largest 32-bit signed integer value.

And also that 999999999 can be represented as a 32-bit integer byt 9999999999 cannot.

hsahay · Post by **hsahay** » Mon Nov 23, 2015 5:19 pm

Yes sir !

I noticed it too.

But what does that signify. Why is it happening only on AIX ?
Why is LINUX putting out 111 ?

What can we try to fix this behavior ? Is there anything bigger than double that we can use ?

Is double restricted to signed 32 bit in datastage ?

ray.wurlod · Post by **ray.wurlod** » Mon Nov 23, 2015 6:15 pm

Why are you using the ",5" piece? Your numbers do not appear to have any decimal places.

Have you tried using Decimal as the data type? According to IEEE standards, Double can never be assured to stored with perfect accuracy, even though you've clearly specified a large enough mantissa.

May I further suggest that you dump the score from each job (the one on Linux and the one on AIX) and try to discern any difference between them? Possibly even post them here.

sjfearnside · Post by **sjfearnside** » Tue Nov 24, 2015 1:39 pm

Could this have any thing to do with a difference in endianness? I converted many years back from a Linux to an AIX system and ran into this difference.

qt_ky · Post by **qt_ky** » Tue Nov 24, 2015 2:12 pm

Look what the function is expecting and returning vs data types you are passing and using as stage variables.

The Max() function returns the greater of the two argument values.
- Input: number1 (int32),number2(int32)
- Output: result (int32)

Have you tried If Col1 > Col2 Then ...?

hsahay · Post by **hsahay** » Tue Nov 24, 2015 7:30 pm

qt_ky - thank you ...i think that explains it.

I will mark this thread as resolved.

ray.wurlod · Post by **ray.wurlod** » Tue Nov 24, 2015 10:22 pm

On a twos-complement machine 9999999999 would have wrapped around, setting the sign bit for an Integer, which is why it would be interpreted as being less than 111.