473,386 Members | 1,846 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,386 software developers and data experts.

Error in Simple Math

In my program I am trying to do a simple calculation of 5.1 * 100 which
should =510 if my gradeschool level math is correct. My problem is vb.net
is returning 509.99999999999994 as the answer. Am I doing something wrong?
Try it for yourself in your Command Window in the .Net IDE.
Type ?5.1 * 100 in the command window during debugging.
Thanks,

Chris Baxter
cb*****@pilotair.com

Nov 20 '05 #1
10 1679
* "Chris Baxter" <cb*****@baxternet.net> scripsit:
In my program I am trying to do a simple calculation of 5.1 * 100 which
should =510 if my gradeschool level math is correct. My problem is vb.net
is returning 509.99999999999994 as the answer. Am I doing something wrong?
Try it for yourself in your Command Window in the .Net IDE.
Type ?5.1 * 100 in the command window during debugging.


<http://research.microsoft.com/~hollasch/cgindex/coding/ieeefloat.html>

--
Herfried K. Wagner
MVP · VB Classic, VB.NET
<http://www.mvps.org/dotnet>
Nov 20 '05 #2
Hi Chris,

5.1 * 100 is so obviously 510 to us because we think in decimal.

Convert 5.1 to a binary floating point number, however, and the digits
after the 'binary point' go on somewhat longer than the few bytes available.
Thus the very fact of <storing> 5.1 is going to introduce an error - let alone
multiplying it.

You'd think that it wouldn't need many bits to store 5.1 given that 51
only needs 6!

51 = 32 + 16 + 2 + 1 = 110011

But 5.1 is more complicated than this as it is built up from fractional
powers of two:

5.1 = 4 + 1 + 1/16 + 1/32 + 1/256 + 1/512 + 1/2048 + ...

= 11.00011001101000 ...

We've already got to 14 digits after the 'binary point' and it's still not
accurate, being only 5.09999765625.

Some numbers just don't like being converted to sums of powers of two.

Regards,
Fergus
Nov 20 '05 #3
Hi again Chris,

I forgot to add:

If you want to overdose on this, ;-) , here's a good slideshow explaining
floating point using very clear examples.
http://www.cs.ualberta.ca/~amaral/co...loatingPoint/s
ld003.htm

Regards
Fergus
Nov 20 '05 #4
Nak
Hi Fergus,

Excellently explained, very good indeed.

Nick.

--
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
"No matter. Whatever the outcome, you are changed."

Fergus - September 5th 2003
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
Nov 20 '05 #5
Cheers Nick :-))
Nov 20 '05 #6
Cor
Hi Fergus,

I don't know what you mean I have 10 fingers not 2.

Cor
Nov 20 '05 #7
Hi Cor,

10 fingers?!! With peas and chips as well, I'm full up after only 4. I bet
you can eat more than 3 Shredded Wheat as well.

Regards,
Fergus
Nov 20 '05 #8
* "Fergus Cooney" <fi*****@post.com> scripsit:
10 fingers?!! With peas and chips as well, I'm full up after only 4. I bet
you can eat more than 3 Shredded Wheat as well.


ROFL again.

--
Herfried K. Wagner
MVP · VB Classic, VB.NET
<http://www.mvps.org/dotnet>
Nov 20 '05 #9
Cor
Fergus,

10 fingers?!! With peas and chips as well, I'm full up after only 4. I bet you can eat more than 3 Shredded Wheat as well.


Maybe this looks all clear for you, but to eat more than 3 Shredded Wheat,
you first have to dimension it. There are 3 dimensions, which will not fit
in a single bit structure.

In a single bit structure you can only have 1 piece or nothing sometimes
called "null".

So if you want to eat more than 1 piece you have to add an overload bit.

When the overload bit is on, you can bring that to an extra one bit
register.

In that case you can eat 3 pieces, although you have only 2 single bit
registers.
The first one has the human value 2 and the second one the value 1. When you
add that together you get the human value 3.

It is a total different way humans are calculating. Humans have ten fingers
and they don't say on or off, but just one finger, two fingers, three
fingers and with 10 they are at the end.
So with this example of you, they would use 3 fingers.

Humans do use a virtual calculating system that is in almost every culture
the same although there are too who use also there toes. But most say in
virtual thinking 11 means 2 full hands of fingers plus one finger. For a
computer this would be 3. Therefore when I can eat 3 peas, you eat 11 peas
and that is a lot I think.

But Humans can calculate with half fingers but do not cut off them further.

So in there mind, when they have to use half or partial fingers, they bring
it first back to a value that does not need half fingers, so 5.1 * 100
becomes the same as 1 * 510.

If the computer software would do that (and as far as I know some do or
did), the results would be the same as humans calculate even if the computer
uses binary values for calculating. That is what I remind me from the past
when the system of binary calculating was developed the system was build on.

But maybe this has to do because that the importance of the rule is
shifting, first were humans important, but now he or she is only the one who
enters values using the keyboard, the computer rules.

This means, when you think you can eat 1000 chips, you maybe only get 999 or
something. But who cares

I thought you did not understand what I did mean, so I made a little
explanation.

:-)) Have fun be happy

Cor
Nov 20 '05 #10
Hi Cor,

LOL, and LOL again and again. ;-))

I understand much better now and so I realise that you don't have 10
fingers - you have 10.

If I had 1000 chips, I'd go to a casino!! Do you want to come?

Best Regards,
Fergus
Nov 20 '05 #11

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Guy Robinson | last post by:
I have the code below which parses an expression string and creates tokens. Can anyone suggest the best of error checking for things like: Valid variable only obj.attribute -whitespace allowed...
3
by: Rob B | last post by:
Hello, I am just starting to learn Python and was writing a simple script on my machine (Mac OS X 10.3.4), but I can't seem to import the random module: #!/usr/bin/env python import random
67
by: Steven T. Hatton | last post by:
Some people have suggested the desire for code completion and refined edit-time error detection are an indication of incompetence on the part of the programmer who wants such features. ...
10
by: John | last post by:
Hi all... Either I'm doing really something really stupid - or maybe there is some bug somewhere (optimizing?). In a function I have: int x1, y1, x2, y2; float dR, dX, dY; dR =...
6
by: ng_mr | last post by:
No, not a question about "banker's rounding" or whatever it's called. I want to round a double to the nearest 100th, so I perform the following: // original is a double double result =...
9
by: Peter Oliphant | last post by:
For some reson my code is generating a LNK1215 error, which 'suggests' I re-install VS C++. So I did. which did NOT solve the problem. The weid part is it seems to be caused by my one CPP file, but...
2
by: saurabh | last post by:
Hey i created a simple page to chk the custom validation control....It checks if the number is prime or not....I get the following error whn i run it A Runtime Error has occured.Do you wish to...
6
by: sadegh | last post by:
Hi I have a problem with my program in VC++6 When I compile it, the following errors are listed. I spend a lot of time on the groups.google.com to find its reason, but none of comments could...
3
by: gokulagopal | last post by:
I am sending the code please help me in finding the error and correct it.The code is below: import java.awt.*; import java.awt.event.*; class CalcFrame extends Frame { ...
22
by: giordan | last post by:
Hi all! I've wrote this code: <script type="text/javascript"> var largImg; var altImg; var txtTop = '<b>Ottima scelta!</b> Ora compila il form e premi "Ricevi banner". Il...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.