[Date Prev][Date Next][Thread Prev][Thread Next]
[Author Index] [Date Index] [Thread Index]
[SQR-USERS Info] [SQRUG Home Page]

precision of numeric variables

I encountered a strange thing in SQR3 with Sybase.  My program select
a numeric column from a table.  When I display it it shows -442.11.
However when I insert the value into a numeric column of another table,
it raises error 241 "Scale error during implicit conversion of NUMERIC
value '-442.110000000000010' to a NUMERIC field".  Somehow SQR
passes the numeric value as -442.110000000000010 instead of -442.11
to Sybase and causes the error.

I know that SQR stores numeric data as floating point number.  I was
wondering if this kind of precision lost is intrinsic in SQR and what can I
do for it?

Raymond Yip