Data Type issue

ariear · Post by **ariear** » Tue Feb 11, 2003 3:45 pm

Hello everyone,

Does anyone knows why ???!!!
A routine that does
Ans=Oconv("2638722939","MB")
Ans=Iconv(Ans,"MB")

returns -1656244357 instead of 2638722939 again

(Which is exactly the difference between long and unsigned long)
If the string of bits "10011101010001111011011101111011" generated by the "MB" option is copies into MS CALC the ansear will be 2638722939

Thanks All

ray.wurlod · Post by **ray.wurlod** » Tue Feb 11, 2003 5:54 pm

Iconv with MB seeks to return a (four-byte signed) integer.
This is buried somewhere in the BASIC manual.
To get unsigned, add 2**32 to the result if the result is negative.

Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518

ariear · Post by **ariear** » Wed Feb 12, 2003 1:06 pm

Thanks Ray - You're number one.[:)]
I hate to admit but I should have known the answear - (If the MSB in long is 1 it turns the number into negative .i.e. subtracts 2**32)

But what do I do with a number that's more than 32 bit ?
Oconv("45517928323990080","MB")= 10001100101010011100001001000000 instead of 10100001101101100101000010001100101010011100001001000000
How do we work with those numbers [:0]
Thanks Again

ray.wurlod · Post by **ray.wurlod** » Wed Feb 12, 2003 5:04 pm

Let me repeat: Iconv with "MB" seeks to return a four-byte signed integer.
What exactly are you trying to accomplish?
Certainly numbers larger than 32 bits can be handled, although not by Iconv with "MB", but they are stored internally as floating point, which may defeat your purpose.

Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518

ariear · Post by **ariear** » Thu Feb 13, 2003 12:30 am

Ray,

I'm trying to find a way to swap bytes in datastage i.e. from Big Indian to Little indian and the first thing I tried is Iconv with "MB"/"MX" and manipulating the string and Oconving it again - It worked until - unsigned long

Regards

ray.wurlod · Post by **ray.wurlod** » Thu Feb 13, 2003 1:51 am

Work a byte at a time! Adapt the following code. The result is hex, which you must Iconv a word (four bytes) at a time, multiplying by an appropriate constant (e.g. 2**32).

0001: FUNCTION BYTESWAP(BUFFER)
0002: * Returns input value as two hex characters per byte, byte-swapped.
0003: *
0004: * Date Who Version Details of modification
0005: * --------- ----- ------- ------------------------------------------
0006: * 02 Feb 02 RayW 1.0 Initial coding
0007: *
0008: * Arguments
0009: * ---------
0010: * BUFFER INPUT the string to be processed
0011: * (big-endian byte order assumed)
0012: * RESULT OUTPUT BUFFER converted to two hex characters
0013: * per byte, byte-swapped if MACHINE.CLASS
0014: * is odd.
0015:
0016: * Generate two hex characters per byte
0017: RESULT = OCONV(BUFFER, "MX0C")
0018:
0019: * Byteswap the string if little-endian byte order required
0020: TEMP = RESULT
0021: RESULT = ""
0022: LOOP
0023: WORD = TEMP[1,8]
0024: FOR J = 7 TO 1 STEP -2
0025: RESULT := WORD[J,2]
0026: NEXT J
0027: TEMP = TEMP[9,LEN(TEMP)-8]
0028: WHILE LEN(TEMP)
0029: REPEAT
0030:
0031: RETURN(RESULT)
0032:
0033: END

Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518

ariear · Post by **ariear** » Thu Feb 13, 2003 3:11 pm

Thanks Ray -
I did almost the same by swapping strings of 8 bits at time but still
I didn't get the needed results because:
1- I had to use a stage (can't read the the data from a routine/batch - the data is mixed with chars -so I must work at a column level
2.The only stage that I found appropriat was CFF - define the column binary 8 and redefine it with 2 4 bytes binaries.
3.take each 4 byte binary, swap it (check for sign in the first one - Thanks again)concat them both (as hex) and then run a loop (Xtd(char)*16**Place in loop) - It still doesn't work in all cases.

I did't find a way to mathematically add those 2 separate hex strings

Regards

ray.wurlod · Post by **ray.wurlod** » Fri Feb 14, 2003 1:04 am

Umm... how about:
Oconv(Iconv(str1,"MX") + Iconv(str2,"MX")),"MX")

ariear · Post by **ariear** » Fri Feb 14, 2003 11:36 am

Tried it but... then again the oconv will return only a 32-bit number and mine's bigger [:)]

The numbers are s1=10596554 s2=1719702016 in that order (it's an 8 byte number)

The result should be 45511854600000000 [:0]

ray.wurlod · Post by **ray.wurlod** » Fri Feb 14, 2003 5:27 pm

Well, it's doable.

>SELECT EVAL "2**32 * 10596554 + 1719702016" FROM VOC FIRST 1;

2**32 * 10596554 + 1
719702016...........

45511854600000000

Sample of 1 records listed.
>

All we need to do is to be more careful with the sign of the first string. And put the thing into a Transform or Routine.

ariear · Post by **ariear** » Sat Feb 15, 2003 1:17 am

Yes , That's it
The swaping is done before the arithymetic so it deals with the sign of the first string
Thanks