issue with implicit casting during field comparison

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
hsahay
Premium Member
Premium Member
Posts: 175
Joined: Wed Mar 21, 2007 9:35 am

issue with implicit casting during field comparison

Post by hsahay »

I have two DataStage installations, one on AIX and one on LINUX, displaying the below described behavior.

DataStage version - 11.3 .2
AIX version - 7100-03-03-1415
Linux version - 2.6.32-504.30.3.el6.x86_64

Problem description -
------------------------

I have two variables

var1 integer
var2 double

I assign the value 1234 to var1 and 9999999999 to var2

So

Var1 = 1234
Var2 = 9999999999 (those are ten 9s).

Now I execute the below comparison -

if var1 < var 2 then 1 else 0

Problem is, DataStage is doing an implicit conversion of VAR2 to int32.
Since the value 9999999999 is too large for an int32 variable, it is overflowing.

On our AIX system, var2 is becoming 2147483647
but
on the linux system, var2 is becoming -2147483647.

Due to which on the AIX system, the comparison statement still works but on linux system it does not.

What is worse is that the system is not generating any warning or error in the director logs.

We don't know how many places in our huge system something like this is happening. Who knows what it has been doing to the data for all these years.

Any one has any suggestion to help us isolate where we have a code like this ?

What we would like to happen is -

1. Either the system ABORTS when a casting conversion like this happens or
2. At least generate a specific warning so that we can code around it to raise abort ourselves.

Please advise.
vishal
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

From the documentation...
The type casting functions help you when you perform mathematical calculations using numeric fields. For example, if you have a calculation using an output column of type float derived from an input column of type integer in a Parallel Transformer stage the result is derived as an integer regardless of its float type. If you want a non-integral result for a calculation using integral operands, you can use the type casting functions to cast the integer operands into non-integral operands.
I know this is talking about "results", but it makes sense that if the output is going to be integer, it probably converts everything to integer for the comparison. So - use "AsDouble()" on the integer prior to the comparison (making both elements "double") and see if it works...
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
Post Reply