Re: SQR Math Oddity...1=1...or does it?

["Johnson, Dan" <Dan_Johnson@WRIGHTEXPRESS.COM>]

Rounding error. Somewhere out at the nth decimal point these values are not
the same. Use an edit mask to truncate the values to only the number of
decimal places that you need.

-----Original Message-----
From: Marcellus, Scott [mailto:SMarcellus@AMILINK.COM]
Sent: Wednesday, July 18, 2001 10:54 AM
To: SQR-USERS@list.iex.net
Subject: SQR Math Oddity...1=1...or does it?


I have tried the following snippet of code in SQR (6.1.4) and I am a little
baffled at the results when it evaluates that the values are not equal:
Oddly enough, converting the number with the to_char function properly
evaluates these statements correctly. I have tried this on several instances
of SQR that we have and found the results strange. Does anyone have any
ideas what may cause this? Can anyone else verify if this error occurs? Does
anyone have any ideas on how to resolve it?

Thanks,

Scott Marcellus

!Begin Code Snippet

let #x = 231.00
let #y = 16.17
let #z = 247.17
if to_char(#x + #y) = to_char(#z)
    print 'This is correct' (+1,20,0)
else
    print 'This is NOT correct' (+1,20,0)
end-if

if #x + #y = #z
    print 'This is correct' (+1,20,0)
else
    print 'This is NOT correct' (+1,20,0)
end-if

Let #y = 16.16
Let #z = 247.16
if #x + #y = #z
    print 'This is correct' (+1,20,0)
else
    print 'This is NOT correct' (+1,20,0)
end-if

let #x = 427667.1
let #y = -34.83
let #z = 427632.27
if #x + #y = #z
    print 'This is correct' (+1,20,0)
else
    print 'This is NOT correct' (+1,20,0)
let #tmpxy = #x + #y
    print #tmpxy                (+1,20,0)
    print #z                    (+1,20,0)

end-if

!End Code Snippet