Basic calculation returns incorrect values

[Dale Reed]

We're having a lot of problems getting visual studio to calculate even the
most basic arithmetic correctly.

For example:

Dim MyValue AS Double = 2.155 * 100

This should give me 215.5 right? Wrong. It gives me 215.49999999999 9999997

Change it to 1.155, it works. 3.155 it works. 2.155 it doesn't work.
We've found several values that don't seem to work correctly. I'm staggered
at this, we're trying to develop accounts based systems and yet visual
studio can't even multiply an item by 100 correctly?! It's not like I'm
doing anything complicated!

Has anyone else come across this and is there some kind of hot fix or patch
that can be applied to sort this out?

We've already been having problems with the round function in .net 1.1 and
had to write our own. The above problem occurs in 1.1 and 2.

Thanks in advance...

[Ebbe Kristensen]

Dale Reed wrote:

Dim MyValue AS Double = 2.155 * 100

This should give me 215.5 right? Wrong. It gives me
215.49999999999 9999997

Welcome to the "wonderful" world of floating point computing.

For more information on the reasons for the above-mentioned problem, please
see:

http://www.lahey.com/float.htm

or

http://docs.sun.com/source/806-3568/ncg_goldberg.html

Ebbe

[Flinky Wisty Pomm]

I doubt there's a hot fix for it; the problem, in essence, is that .Net
doesn't know that (215.5 * 100) is an "easier" sum than (214.9871 *
99.31048) - because for a computer, it isn't.

If the latter was wrong by 0.0000000000000 003 you'd be hard pressed to
notice. You only notice the specific case because humans think in base
10.

That's an aside though, and not very useful. If you need to know that
your numbers are accurate, avoid floating points. Stick to longs and
divide 'em at the end. That's generally how you are recommended to
store money values in databases, too, for exactly the same reason -
floating point arithmetic is not 100% accurate, and accounting
departments are.
Dale Reed wrote:

We're having a lot of problems getting visual studio to calculate even the
most basic arithmetic correctly.

For example:

Dim MyValue AS Double = 2.155 * 100

This should give me 215.5 right? Wrong. It gives me 215.49999999999 9999997

Change it to 1.155, it works. 3.155 it works. 2.155 it doesn't work.
We've found several values that don't seem to work correctly. I'm staggered
at this, we're trying to develop accounts based systems and yet visual
studio can't even multiply an item by 100 correctly?! It's not like I'm
doing anything complicated!

Has anyone else come across this and is there some kind of hot fix or patch
that can be applied to sort this out?

We've already been having problems with the round function in .net 1.1 and
had to write our own. The above problem occurs in 1.1 and 2.

Thanks in advance...

[Lukas Kurka]

why trivial calc.exe in windows calculate 2.155*100 right but .NET doesn't?

"Flinky Wisty Pomm" <Pa********@gma il.comwrote in message
news:11******** *************@c 28g2000cwb.goog legroups.com...

>I doubt there's a hot fix for it; the problem, in essence, is that .Net
doesn't know that (215.5 * 100) is an "easier" sum than (214.9871 *
99.31048) - because for a computer, it isn't.

If the latter was wrong by 0.0000000000000 003 you'd be hard pressed to
notice. You only notice the specific case because humans think in base
10.

That's an aside though, and not very useful. If you need to know that
your numbers are accurate, avoid floating points. Stick to longs and
divide 'em at the end. That's generally how you are recommended to
store money values in databases, too, for exactly the same reason -
floating point arithmetic is not 100% accurate, and accounting
departments are.
Dale Reed wrote:

>We're having a lot of problems getting visual studio to calculate even
the
most basic arithmetic correctly.

For example:

Dim MyValue AS Double = 2.155 * 100

This should give me 215.5 right? Wrong. It gives me
215.4999999999 99999997

Change it to 1.155, it works. 3.155 it works. 2.155 it doesn't work.
We've found several values that don't seem to work correctly. I'm
staggered
at this, we're trying to develop accounts based systems and yet visual
studio can't even multiply an item by 100 correctly?! It's not like I'm
doing anything complicated!

Has anyone else come across this and is there some kind of hot fix or
patch
that can be applied to sort this out?

We've already been having problems with the round function in .net 1.1
and
had to write our own. The above problem occurs in 1.1 and 2.

Thanks in advance...