473,513 Members | 2,658 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Math issue?

I'm doing the below calculating in VB.net. All variable are declared as
double and their contents are below.
mdblMidtermWt 0.2 Double
mdblFinalWt 0.2 Double
mdblAssignWt 0.2 Double
mdblGroupWt 0.3 Double
mdblPartWt 0.1 Double

temp 1.0000000000000002 Double
temp = mdblMidtermWt + mdblFinalWt + mdblAssignWt + mdblGroupWt + mdblPartWt

Last I check 0.2 + 0.2 + 0.2+0.3 + 0.1 was equal to 1 so why am I getting
1.0000000000000002 ?

Nov 21 '05 #1
3 1011
"David A. Osborn" <do********@hotmail.com> schrieb:
temp = mdblMidtermWt + mdblFinalWt + mdblAssignWt + mdblGroupWt +
mdblPartWt

Last I check 0.2 + 0.2 + 0.2+0.3 + 0.1 was equal to 1 so why am I getting
1.0000000000000002 ?


That's a characteristic of floating point numbers.

<URL:http://research.microsoft.com/~hollasch/cgindex/coding/ieeefloat.html>
<URL:http://www.math.grin.edu/~stone/courses/fundamentals/IEEE-reals.html>
<URL:http://support.microsoft.com/?scid=kb;[LN];42980>

--
M S Herfried K. Wagner
M V P <URL:http://dotnet.mvps.org/>
V B <URL:http://classicvb.org/petition/>

Nov 21 '05 #2
use the decimal dataype
"David A. Osborn" <do********@hotmail.com> wrote in message
news:pnYse.83260$nG6.13100@attbi_s22...
I'm doing the below calculating in VB.net. All variable are declared as
double and their contents are below.
mdblMidtermWt 0.2 Double
mdblFinalWt 0.2 Double
mdblAssignWt 0.2 Double
mdblGroupWt 0.3 Double
mdblPartWt 0.1 Double

temp 1.0000000000000002 Double
temp = mdblMidtermWt + mdblFinalWt + mdblAssignWt + mdblGroupWt +
mdblPartWt

Last I check 0.2 + 0.2 + 0.2+0.3 + 0.1 was equal to 1 so why am I getting
1.0000000000000002 ?

Nov 21 '05 #3
use the decimal dataype
"David A. Osborn" <do********@hotmail.com> wrote in message
news:pnYse.83260$nG6.13100@attbi_s22...
I'm doing the below calculating in VB.net. All variable are declared as
double and their contents are below.
mdblMidtermWt 0.2 Double
mdblFinalWt 0.2 Double
mdblAssignWt 0.2 Double
mdblGroupWt 0.3 Double
mdblPartWt 0.1 Double

temp 1.0000000000000002 Double
temp = mdblMidtermWt + mdblFinalWt + mdblAssignWt + mdblGroupWt +
mdblPartWt

Last I check 0.2 + 0.2 + 0.2+0.3 + 0.1 was equal to 1 so why am I getting
1.0000000000000002 ?

Nov 21 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

89
5019
by: Radioactive Man | last post by:
In python 2.3 (IDLE 1.0.3) running under windows 95, I get the following types of errors whenever I do simple arithmetic: 1st example: >>> 12.10 + 8.30 20.399999999999999 >>> 1.1 - 0.2...
1
2704
by: limelight | last post by:
I have discovered a math error in the .NET framework's Log function. It returns incorrect results for varying powers of 2 that depend on whether the program is run from within the IDE or from the...
17
3589
by: cwdjrxyz | last post by:
Javascript has a very small math function list. However there is no reason that this list can not be extended greatly. Speed is not an issue, unless you nest complicated calculations several levels...
15
12208
by: Ingmar | last post by:
Simple comparison tests we have performed show that System.Math functions in C# are much slower than corresponding functions in C++. Extreme examples are System.Math.Exp() and System.Math.Tan(),...
110
8466
by: Gregory Pietsch | last post by:
I'm writing a portable implementation of the C standard library for http://www.clc-wiki.net and I was wondering if someone could check the functions in math.h for sanity/portability/whatever. I'm...
1
1350
by: Andy | last post by:
Hi, I noticed when migrating a 2002 application to 2003 that the system changed to a different usage for pow(2,-15). In 2002, it used double pow(double,double). In 2003, it is using double...
2
4872
by: igotyourdotnet | last post by:
How can I perform a math function in C# example: int sales = 25 int goal = 225 i want to devide sales by goal so the sale person knows how much of his goal he's at
1
211
by: Jerry Hill | last post by:
On Thu, May 22, 2008 at 12:32 PM, Dutton, Sam <Sam.Dutton@itn.co.ukwrote: I believe this is an issue with the precision of IEEE floating point numbers. Those are the types of numbers used by the...
0
1927
by: Akira Kitada | last post by:
Hi Marc-Andre, Thanks for the suggestion. I opened a ticket for this issue: http://bugs.python.org/issue4204 Now I understand the state of the multiprocessing module, but it's too bad to see...
0
1788
by: M.-A. Lemburg | last post by:
On 2008-10-25 20:19, Akira Kitada wrote: Thanks. The errors you are getting appear to be related to either some missing header files or a missing symbol definition to enable these - looking...
0
7257
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
7157
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
7379
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
7535
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
1
7098
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
7521
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
5682
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
0
4745
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and...
0
1591
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated ...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.