473,320 Members | 1,921 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Help! Numbers not adding properly

I am having problems adding two numbers. I am trying to add 1.005 and
1.007 to come up with 2.012. Should be easy enough right? Problem is that
I keep getting 2.0119999999999996 as my result. Why is this happening? Is
there a way that I can get it to return the right result? Any help is
appreciated.

Thanks,
David J. Ricker II
Nov 20 '05 #1
10 1426
What data types are you using?

"David Ricker" <da*************@hotmail.com> wrote in message
news:eW**************@TK2MSFTNGP12.phx.gbl...
I am having problems adding two numbers. I am trying to add 1.005 and
1.007 to come up with 2.012. Should be easy enough right? Problem is that I keep getting 2.0119999999999996 as my result. Why is this happening? Is there a way that I can get it to return the right result? Any help is
appreciated.

Thanks,
David J. Ricker II

Nov 20 '05 #2
Doubles. I have also tried simply typing 1.005+1.007 into the watch window
and it gives the same result.

Thanks,
David J. Ricker II

"CJ Taylor" <no****@blowgoats.com> wrote in message
news:vs************@corp.supernews.com...
What data types are you using?

"David Ricker" <da*************@hotmail.com> wrote in message
news:eW**************@TK2MSFTNGP12.phx.gbl...
I am having problems adding two numbers. I am trying to add 1.005 and 1.007 to come up with 2.012. Should be easy enough right? Problem is

that
I keep getting 2.0119999999999996 as my result. Why is this happening?

Is
there a way that I can get it to return the right result? Any help is
appreciated.

Thanks,
David J. Ricker II


Nov 20 '05 #3
What the he.....

Try a decimal. see what happens.
"David Ricker" <da*************@hotmail.com> wrote in message
news:uX**************@TK2MSFTNGP10.phx.gbl...
Doubles. I have also tried simply typing 1.005+1.007 into the watch window and it gives the same result.

Thanks,
David J. Ricker II

"CJ Taylor" <no****@blowgoats.com> wrote in message
news:vs************@corp.supernews.com...
What data types are you using?

"David Ricker" <da*************@hotmail.com> wrote in message
news:eW**************@TK2MSFTNGP12.phx.gbl...
I am having problems adding two numbers. I am trying to add 1.005 and 1.007 to come up with 2.012. Should be easy enough right? Problem is

that
I keep getting 2.0119999999999996 as my result. Why is this
happening? Is
there a way that I can get it to return the right result? Any help is
appreciated.

Thanks,
David J. Ricker II



Nov 20 '05 #4
Strange, a decimal does work. Does anyone have any idea why a double won't
work in this situation... is it a bug?

Thanks CJ, I can use the decimals to get the desired results in my program.

Thanks Again,
David J. Ricker II
"CJ Taylor" <no****@blowgoats.com> wrote in message
news:vs************@corp.supernews.com...
What the he.....

Try a decimal. see what happens.
"David Ricker" <da*************@hotmail.com> wrote in message
news:uX**************@TK2MSFTNGP10.phx.gbl...
Doubles. I have also tried simply typing 1.005+1.007 into the watch

window
and it gives the same result.

Thanks,
David J. Ricker II

"CJ Taylor" <no****@blowgoats.com> wrote in message
news:vs************@corp.supernews.com...
What data types are you using?

"David Ricker" <da*************@hotmail.com> wrote in message
news:eW**************@TK2MSFTNGP12.phx.gbl...
> I am having problems adding two numbers. I am trying to add 1.005
and
> 1.007 to come up with 2.012. Should be easy enough right? Problem
is that
> I keep getting 2.0119999999999996 as my result. Why is this

happening? Is
> there a way that I can get it to return the right result? Any help is > appreciated.
>
> Thanks,
> David J. Ricker II
>
>



Nov 20 '05 #5
Cor
Hi David,

I think that this will help you.

I asume your decimal separator is a dot and not a comma

http://msdn.microsoft.com/library/de...roundtopic.asp

I hope this helps a little bit.

Cor
I am having problems adding two numbers. I am trying to add 1.005 and
1.007 to come up with 2.012. Should be easy enough right? Problem is that
I keep getting 2.0119999999999996 as my result. Why is this happening? Isthere a way that I can get it to return the right result? Any help is
appreciated.

Nov 20 '05 #6
One would think so. But I don't know the honest answer. My best guess
would be back to high school chemistry days and understanding the use of Sig
Figs... Could be something to do with that, even though we only have 3 to
work with so I don't know what the big deal is.

I've had the same issue before, and something to do with decimal having a
precision scale is the reason I've heard before. But I quesiton everything.
=)

Then again... it could be a bug, but I'm probably wrong... but if it was,
that sounds like it would be processor level instead of code level.

-CJ

"David Ricker" <da*************@hotmail.com> wrote in message
news:ed**************@tk2msftngp13.phx.gbl...
Strange, a decimal does work. Does anyone have any idea why a double won't
work in this situation... is it a bug?

Thanks CJ, I can use the decimals to get the desired results in my program.
Thanks Again,
David J. Ricker II
"CJ Taylor" <no****@blowgoats.com> wrote in message
news:vs************@corp.supernews.com...
What the he.....

Try a decimal. see what happens.
"David Ricker" <da*************@hotmail.com> wrote in message
news:uX**************@TK2MSFTNGP10.phx.gbl...
Doubles. I have also tried simply typing 1.005+1.007 into the watch window
and it gives the same result.

Thanks,
David J. Ricker II

"CJ Taylor" <no****@blowgoats.com> wrote in message
news:vs************@corp.supernews.com...
> What data types are you using?
>
> "David Ricker" <da*************@hotmail.com> wrote in message
> news:eW**************@TK2MSFTNGP12.phx.gbl...
> > I am having problems adding two numbers. I am trying to add 1.005 and
> > 1.007 to come up with 2.012. Should be easy enough right?
Problem is > that
> > I keep getting 2.0119999999999996 as my result. Why is this happening?
> Is
> > there a way that I can get it to return the right result? Any
help is > > appreciated.
> >
> > Thanks,
> > David J. Ricker II
> >
> >
>
>



Nov 20 '05 #7
Bob
http://www.math.grin.edu/~stone/cour...EEE-reals.html

Bob

"David Ricker" <da*************@hotmail.com> wrote in message
news:eW**************@TK2MSFTNGP12.phx.gbl...
I am having problems adding two numbers. I am trying to add 1.005 and
1.007 to come up with 2.012. Should be easy enough right? Problem is that
I keep getting 2.0119999999999996 as my result. Why is this happening? Is
there a way that I can get it to return the right result? Any help is
appreciated.

Thanks,
David J. Ricker II


Nov 20 '05 #8
David,
In addition to CJ & Cor's posts.

http://msdn.microsoft.com/library/de...egralTypes.asp

1.007 & 2.012 are by default a Double. Double & Single are base 2 floating
point numbers.

Decimal is a base 10 floating point number. (technally base 2, but think of
it as base 10 ;-))

Decimals are stored as a whole number (1234.) that is scaled, where as
Single & Double are stored as a fraction number (1.1234) that is scaled.
Seeing as both are represented as binary number internally, a binary whole
number can store all decimal numbers, where as a binary fraction cannot
store all decimal fractions. Hence the rounding issue you are seeing.

Hope this helps
Jay

"David Ricker" <da*************@hotmail.com> wrote in message
news:eW**************@TK2MSFTNGP12.phx.gbl...
I am having problems adding two numbers. I am trying to add 1.005 and
1.007 to come up with 2.012. Should be easy enough right? Problem is that I keep getting 2.0119999999999996 as my result. Why is this happening? Is there a way that I can get it to return the right result? Any help is
appreciated.

Thanks,
David J. Ricker II

Nov 20 '05 #9
Thanks Jay,

At least now I know that it is by design.
"Jay B. Harlow [MVP - Outlook]" <Ja************@msn.com> wrote in message
news:%2***************@tk2msftngp13.phx.gbl...
David,
In addition to CJ & Cor's posts.

http://msdn.microsoft.com/library/de...egralTypes.asp
1.007 & 2.012 are by default a Double. Double & Single are base 2 floating
point numbers.

Decimal is a base 10 floating point number. (technally base 2, but think of it as base 10 ;-))

Decimals are stored as a whole number (1234.) that is scaled, where as
Single & Double are stored as a fraction number (1.1234) that is scaled.
Seeing as both are represented as binary number internally, a binary whole
number can store all decimal numbers, where as a binary fraction cannot
store all decimal fractions. Hence the rounding issue you are seeing.

Hope this helps
Jay

"David Ricker" <da*************@hotmail.com> wrote in message
news:eW**************@TK2MSFTNGP12.phx.gbl...
I am having problems adding two numbers. I am trying to add 1.005 and 1.007 to come up with 2.012. Should be easy enough right? Problem is

that
I keep getting 2.0119999999999996 as my result. Why is this happening?

Is
there a way that I can get it to return the right result? Any help is
appreciated.

Thanks,
David J. Ricker II


Nov 20 '05 #10
* "David Ricker" <da*************@hotmail.com> scripsit:
I am having problems adding two numbers. I am trying to add 1.005 and
1.007 to come up with 2.012. Should be easy enough right? Problem is that
I keep getting 2.0119999999999996 as my result. Why is this happening? Is
there a way that I can get it to return the right result? Any help is
appreciated.


<http://research.microsoft.com/~hollasch/cgindex/coding/ieeefloat.html>
<http://support.microsoft.com/default.aspx?scid=kb;de;42980>

--
Herfried K. Wagner [MVP]
<http://www.mvps.org/dotnet>
Nov 20 '05 #11

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

9
by: Tom | last post by:
A question for gui application programmers. . . I 've got some GUI programs, written in Python/wxPython, and I've got a help button and a help menu item. Also, I've got a compiled file made with...
4
by: Sarir Khamsi | last post by:
Is there a way to get help the way you get it from the Python interpreter (eg, 'help(dir)' gives help on the 'dir' command) in the module cmd.Cmd? I know how to add commands and help text to...
2
by: Sudheer Kareem | last post by:
Dear All Please tell me how to assosiate help files with my Vb.net Project. Regards Sudheer
6
by: wukexin | last post by:
Help me, good men. I find mang books that introduce bit "mang header files",they talk too bit,in fact it is my too fool, I don't learn it, I have do a test program, but I have no correct doing...
3
by: Colin J. Williams | last post by:
Python advertises some basic service: C:\Python24>python Python 2.4.1 (#65, Mar 30 2005, 09:13:57) on win32 Type "help", "copyright", "credits" or "license" for more information. >>> With...
7
by: Corepaul | last post by:
Missing Help Files When I enter "recordset" as the keyword and search the Visual Basic Help index, I get many topics of interest in the resulting list. But there isn't any information available...
5
by: Steve | last post by:
I have written a help file (chm) for a DLL and referenced it using Help.ShowHelp My expectation is that a developer using my DLL would be able to access this help file during his development time...
8
by: Mark | last post by:
I have loaded Visual Studio .net on my home computer and my laptop, but my home computer has an abbreviated help screen not 2% of the help on my laptop. All the settings look the same on both...
10
by: JonathanOrlev | last post by:
Hello everybody, I wrote this comment in another message of mine, but decided to post it again as a standalone message. I think that Microsoft's Office 2003 help system is horrible, probably...
0
by: hitencontractor | last post by:
I am working on .NET Version 2003 making an SDI application that calls MS Excel 2003. I added a menu item called "MyApp Help" in the end of the menu bar to show Help-> About. The application...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.