[Date Prev][Date Next][Thread Prev][Thread Next]
[SQRUG Home Page]
precision of numeric variables
- Subject: precision of numeric variables
- From: Raymond Yip <MCYIP@MTRC.COM.HK>
- Date: Wed, 17 Apr 1996 11:53:47 +0800
I encountered a strange thing in SQR3 with Sybase. My program select
a numeric column from a table. When I display it it shows -442.11.
However when I insert the value into a numeric column of another table,
it raises error 241 "Scale error during implicit conversion of NUMERIC
value '-442.110000000000010' to a NUMERIC field". Somehow SQR
passes the numeric value as -442.110000000000010 instead of -442.11
to Sybase and causes the error.
I know that SQR stores numeric data as floating point number. I was
wondering if this kind of precision lost is intrinsic in SQR and what can I
do for it?