469,275 Members | 1,482 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,275 developers. It's quick & easy.

Adding double values gives incorrect result

I have a section of code that adds some double values and gives an incorrect
result. This occurs with data that isn't really waht I would call high
precision. An example is the following code snippet:

---------------
double a = 2.7;
double b = 2.7;
double c = 0.001;

double result=0;
result=result+a;
result=result+b;
result=result+c;
------------------

After running this code the value of the result variable in the quickwatch
(or watch) window is 5.4010000000000007. How is this possible??

If I do the following:
System.Diagnostics.Debug.WriteLine("Value of result = " + result);

it comes out correctly as 5.401. But I need to be able to compare this to
another variable and the result is wrong.

Any ideas what is going on here
Jul 21 '05 #1
2 2149
On Wed, 25 Aug 2004 02:39:04 -0700, sw******@mapinfo.com wrote:
After running this code the value of the result variable in the quickwatch
(or watch) window is 5.4010000000000007. How is this possible??
Any ideas what is going on here


The result you're getting is because decimal numbers cannot accurately be
stored in binary. Try using the Decimal data type instead which, I
believe, compensates for this.

--
Chris

dunawayc[AT]sbcglobal_lunchmeat_[DOT]net

To send me an E-mail, remove the "[", "]", underscores ,lunchmeat, and
replace certain words in my E-Mail address.
Jul 21 '05 #2
sw******@mapinfo.com <sw******@mapinfo.com> wrote:
I have a section of code that adds some double values and gives an incorrect
result. This occurs with data that isn't really waht I would call high
precision. An example is the following code snippet:

---------------
double a = 2.7;
double b = 2.7;
double c = 0.001;

double result=0;
result=result+a;
result=result+b;
result=result+c;
------------------

After running this code the value of the result variable in the quickwatch
(or watch) window is 5.4010000000000007. How is this possible??

If I do the following:
System.Diagnostics.Debug.WriteLine("Value of result = " + result);

it comes out correctly as 5.401. But I need to be able to compare this to
another variable and the result is wrong.

Any ideas what is going on here


See http://www.pobox.com/~skeet/csharp/floatingpoint.html and
http://www.pobox.com/~skeet/csharp/decimal.html

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Jul 21 '05 #3

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

reply views Thread by Danny Winslow | last post: by
5 posts views Thread by DAVID SCHULMAN | last post: by
6 posts views Thread by James Thurley | last post: by
17 posts views Thread by Sri | last post: by
14 posts views Thread by cj | last post: by
116 posts views Thread by Dilip | last post: by
3 posts views Thread by =?Utf-8?B?RmFiaW8=?= | last post: by
5 posts views Thread by =?Utf-8?B?anVzdGluc2FyYWNlbm8=?= | last post: by
reply views Thread by suresh191 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.