440,905 Members | 902 Online
Need help? Post your question and get tips & solutions from a community of 440,905 IT Pros & Developers. It's quick & easy.

 P: n/a I am having problems adding two numbers. I am trying to add 1.005 and 1.007 to come up with 2.012. Should be easy enough right? Problem is that I keep getting 2.0119999999999996 as my result. Why is this happening? Is there a way that I can get it to return the right result? Any help is appreciated. Thanks, David J. Ricker II Nov 20 '05 #1
10 Replies

 P: n/a What data types are you using? "David Ricker" wrote in message news:eW**************@TK2MSFTNGP12.phx.gbl... I am having problems adding two numbers. I am trying to add 1.005 and 1.007 to come up with 2.012. Should be easy enough right? Problem is that I keep getting 2.0119999999999996 as my result. Why is this happening? Is there a way that I can get it to return the right result? Any help is appreciated. Thanks, David J. Ricker II Nov 20 '05 #2

 P: n/a Doubles. I have also tried simply typing 1.005+1.007 into the watch window and it gives the same result. Thanks, David J. Ricker II "CJ Taylor" wrote in message news:vs************@corp.supernews.com... What data types are you using? "David Ricker" wrote in message news:eW**************@TK2MSFTNGP12.phx.gbl... I am having problems adding two numbers. I am trying to add 1.005 and 1.007 to come up with 2.012. Should be easy enough right? Problem is that I keep getting 2.0119999999999996 as my result. Why is this happening? Is there a way that I can get it to return the right result? Any help is appreciated. Thanks, David J. Ricker II Nov 20 '05 #3

 P: n/a What the he..... Try a decimal. see what happens. "David Ricker" wrote in message news:uX**************@TK2MSFTNGP10.phx.gbl... Doubles. I have also tried simply typing 1.005+1.007 into the watch window and it gives the same result. Thanks, David J. Ricker II "CJ Taylor" wrote in message news:vs************@corp.supernews.com... What data types are you using? "David Ricker" wrote in message news:eW**************@TK2MSFTNGP12.phx.gbl... I am having problems adding two numbers. I am trying to add 1.005 and 1.007 to come up with 2.012. Should be easy enough right? Problem is that I keep getting 2.0119999999999996 as my result. Why is this happening? Is there a way that I can get it to return the right result? Any help is appreciated. Thanks, David J. Ricker II Nov 20 '05 #4

 P: n/a Strange, a decimal does work. Does anyone have any idea why a double won't work in this situation... is it a bug? Thanks CJ, I can use the decimals to get the desired results in my program. Thanks Again, David J. Ricker II "CJ Taylor" wrote in message news:vs************@corp.supernews.com... What the he..... Try a decimal. see what happens. "David Ricker" wrote in message news:uX**************@TK2MSFTNGP10.phx.gbl... Doubles. I have also tried simply typing 1.005+1.007 into the watch window and it gives the same result. Thanks, David J. Ricker II "CJ Taylor" wrote in message news:vs************@corp.supernews.com... What data types are you using? "David Ricker" wrote in message news:eW**************@TK2MSFTNGP12.phx.gbl... > I am having problems adding two numbers. I am trying to add 1.005 and > 1.007 to come up with 2.012. Should be easy enough right? Problem is that > I keep getting 2.0119999999999996 as my result. Why is this happening? Is > there a way that I can get it to return the right result? Any help is > appreciated. > > Thanks, > David J. Ricker II > > Nov 20 '05 #5

 P: n/a Hi David, I think that this will help you. I asume your decimal separator is a dot and not a comma http://msdn.microsoft.com/library/de...roundtopic.asp I hope this helps a little bit. Cor I am having problems adding two numbers. I am trying to add 1.005 and1.007 to come up with 2.012. Should be easy enough right? Problem is thatI keep getting 2.0119999999999996 as my result. Why is this happening? Isthere a way that I can get it to return the right result? Any help isappreciated. Nov 20 '05 #6

 P: n/a One would think so. But I don't know the honest answer. My best guess would be back to high school chemistry days and understanding the use of Sig Figs... Could be something to do with that, even though we only have 3 to work with so I don't know what the big deal is. I've had the same issue before, and something to do with decimal having a precision scale is the reason I've heard before. But I quesiton everything. =) Then again... it could be a bug, but I'm probably wrong... but if it was, that sounds like it would be processor level instead of code level. -CJ "David Ricker" wrote in message news:ed**************@tk2msftngp13.phx.gbl... Strange, a decimal does work. Does anyone have any idea why a double won't work in this situation... is it a bug? Thanks CJ, I can use the decimals to get the desired results in my program. Thanks Again, David J. Ricker II "CJ Taylor" wrote in message news:vs************@corp.supernews.com... What the he..... Try a decimal. see what happens. "David Ricker" wrote in message news:uX**************@TK2MSFTNGP10.phx.gbl... Doubles. I have also tried simply typing 1.005+1.007 into the watch window and it gives the same result. Thanks, David J. Ricker II "CJ Taylor" wrote in message news:vs************@corp.supernews.com... > What data types are you using? > > "David Ricker" wrote in message > news:eW**************@TK2MSFTNGP12.phx.gbl... > > I am having problems adding two numbers. I am trying to add 1.005 and > > 1.007 to come up with 2.012. Should be easy enough right? Problem is > that > > I keep getting 2.0119999999999996 as my result. Why is this happening? > Is > > there a way that I can get it to return the right result? Any help is > > appreciated. > > > > Thanks, > > David J. Ricker II > > > > > > Nov 20 '05 #7

 P: n/a http://www.math.grin.edu/~stone/cour...EEE-reals.html Bob "David Ricker" wrote in message news:eW**************@TK2MSFTNGP12.phx.gbl... I am having problems adding two numbers. I am trying to add 1.005 and 1.007 to come up with 2.012. Should be easy enough right? Problem is that I keep getting 2.0119999999999996 as my result. Why is this happening? Is there a way that I can get it to return the right result? Any help is appreciated. Thanks, David J. Ricker II Nov 20 '05 #8

 P: n/a David, In addition to CJ & Cor's posts. http://msdn.microsoft.com/library/de...egralTypes.asp 1.007 & 2.012 are by default a Double. Double & Single are base 2 floating point numbers. Decimal is a base 10 floating point number. (technally base 2, but think of it as base 10 ;-)) Decimals are stored as a whole number (1234.) that is scaled, where as Single & Double are stored as a fraction number (1.1234) that is scaled. Seeing as both are represented as binary number internally, a binary whole number can store all decimal numbers, where as a binary fraction cannot store all decimal fractions. Hence the rounding issue you are seeing. Hope this helps Jay "David Ricker" wrote in message news:eW**************@TK2MSFTNGP12.phx.gbl... I am having problems adding two numbers. I am trying to add 1.005 and 1.007 to come up with 2.012. Should be easy enough right? Problem is that I keep getting 2.0119999999999996 as my result. Why is this happening? Is there a way that I can get it to return the right result? Any help is appreciated. Thanks, David J. Ricker II Nov 20 '05 #9

 P: n/a Thanks Jay, At least now I know that it is by design. "Jay B. Harlow [MVP - Outlook]" wrote in message news:%2***************@tk2msftngp13.phx.gbl... David, In addition to CJ & Cor's posts. http://msdn.microsoft.com/library/de...egralTypes.asp 1.007 & 2.012 are by default a Double. Double & Single are base 2 floating point numbers. Decimal is a base 10 floating point number. (technally base 2, but think of it as base 10 ;-)) Decimals are stored as a whole number (1234.) that is scaled, where as Single & Double are stored as a fraction number (1.1234) that is scaled. Seeing as both are represented as binary number internally, a binary whole number can store all decimal numbers, where as a binary fraction cannot store all decimal fractions. Hence the rounding issue you are seeing. Hope this helps Jay "David Ricker" wrote in message news:eW**************@TK2MSFTNGP12.phx.gbl... I am having problems adding two numbers. I am trying to add 1.005 and 1.007 to come up with 2.012. Should be easy enough right? Problem is that I keep getting 2.0119999999999996 as my result. Why is this happening? Is there a way that I can get it to return the right result? Any help is appreciated. Thanks, David J. Ricker II Nov 20 '05 #10

 P: n/a * "David Ricker" scripsit: I am having problems adding two numbers. I am trying to add 1.005 and 1.007 to come up with 2.012. Should be easy enough right? Problem is that I keep getting 2.0119999999999996 as my result. Why is this happening? Is there a way that I can get it to return the right result? Any help is appreciated. -- Herfried K. Wagner [MVP] Nov 20 '05 #11