"Joel Barsotti" <jo**********@hotmail.com> wrote in message news:<u3**************@TK2MSFTNGP10.phx.gbl>...
But when I go to do more arithmetic on the numbers I'm getting this strange
anomaly where it just comes out wrong. Watching the values in the debugger
show that the values are in fact 39 and 32.2 but when I subtract 32.2 from
39 I get 7.7999999999.
What should I be doing to avoid this?
You are a victim of internal binary representation of decimal numbers.
This is a problem with all computers that use binary represenation of
floating point data. Simply, some decimal finite numbers, cannot be
converted to finite binary representation. Therefore computer cuts the
translation at the predetermined binary length creating binary
floating point that is very close but not exactly the decimal number
that you started with. It is exactly the same type of problem when you
for example divide decimal 1 by decimal 3. In decimal notation 1/3
cannot be represented by finite string. In other words 1/3 =
0.333333... (infinite). Therefore, even of you would use decimal type
data you still can encounter this problem once you introduce division
into your calculations. Most of the handheld calculators are using
decimal (BCD) internal representation and they still suffer from the
simple problem that when you take 1, divide it by 3 and then multiply
by 3 you will get 0.999999.... but not 1.0 as you would otherwise
expect.
There are several solutions to your particular problem. The first one
is to "ignore" the problem and use binary. Even when binary is not
exact it is in most cases good enough to hold practical precision for
your needs, that even after thousands of calculations the error is
still within acceptable range. All you have to do then to avoid
cofusing display is to display the outcome using round-off and
formatting to display only what you want from your calculations. For
example, if you use financial calculations, you probably want only two
decimal places after period to represent cents. Nobody needs fractions
of the cent. In such case 7.7999999... will be displayed as 7.80 and
you should be OK as long as 7.79999.. is not used later millions of
times in a loop which could magnify error to the unacceptable level.
In most cases this is not the case and you can live with binary OK. In
rare cases, when it is not OK you could use decimal type data or
integer type data. This solution is perfect for monetary as long as
you don't use division. The integer or decimal representation is
translated perfectly from decimal input and preserves perfect value
for multiplication, subtraction and addition as long as you don't
overflow the data capacity. The same trouble with limited
representation however is exposed when you introduce division. I
already mentiond 1/3 as a problem for decimal (or integer for that
matter) representation. If you program uses division then your
calculations suffer again from close but not exact representation
syndrome. Again the solution here is to ignore small error and take
care of it during display using round-off and formatting. But ignoring
this carries the same risk, that if you use intermediate calculations
furhter in a long loops then the intial small error can get magnified
to even nonsensical values.
For example if you take a fraction of 1/3, multiply it by 10 and
subtract 3, you should get back exactly the same 1/3. So in theory
when you start with 1/3, you could multiply it by 10 and then subtract
3 thousands of times in a loop. Try it on you calculator using
numerical accuracy (don't use modern algebraic fractional
representation that is available in some advanced models) and you will
find out nonsense only after ten or so loops. So even decimal internal
representation is no good, when you start with number like 1/3
represented imperfectly in decimal as 0.33333... internally.
If you still insisit on perfection here, you could use fractional
representation. Such data type however is not built in to the C#
natively and you would have to write your own classes. In such
representation all numbers are represented as fractions of two integer
numbers a/b. For eample 3 is represented as (3,1) pair of integers,
1/3 is represented as (1,3) pair of integers and so on. In such
represenrtaion all four basic math operators are preserved perfectly
(addition, subtraction, multiplication and division) as long as you
don't overflow the data capacity. Such approach is used by the
Microsoft Windows calculator.
Of course, once you ask for functions such as square root or
trigonometric the ugly problem is back no matter what above
representation you use :-)
JM
