By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,905 Members | 902 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,905 IT Pros & Developers. It's quick & easy.

Help! Numbers not adding properly

P: n/a
I am having problems adding two numbers. I am trying to add 1.005 and
1.007 to come up with 2.012. Should be easy enough right? Problem is that
I keep getting 2.0119999999999996 as my result. Why is this happening? Is
there a way that I can get it to return the right result? Any help is
appreciated.

Thanks,
David J. Ricker II
Nov 20 '05 #1
Share this Question
Share on Google+
10 Replies


P: n/a
What data types are you using?

"David Ricker" <da*************@hotmail.com> wrote in message
news:eW**************@TK2MSFTNGP12.phx.gbl...
I am having problems adding two numbers. I am trying to add 1.005 and
1.007 to come up with 2.012. Should be easy enough right? Problem is that I keep getting 2.0119999999999996 as my result. Why is this happening? Is there a way that I can get it to return the right result? Any help is
appreciated.

Thanks,
David J. Ricker II

Nov 20 '05 #2

P: n/a
Doubles. I have also tried simply typing 1.005+1.007 into the watch window
and it gives the same result.

Thanks,
David J. Ricker II

"CJ Taylor" <no****@blowgoats.com> wrote in message
news:vs************@corp.supernews.com...
What data types are you using?

"David Ricker" <da*************@hotmail.com> wrote in message
news:eW**************@TK2MSFTNGP12.phx.gbl...
I am having problems adding two numbers. I am trying to add 1.005 and 1.007 to come up with 2.012. Should be easy enough right? Problem is

that
I keep getting 2.0119999999999996 as my result. Why is this happening?

Is
there a way that I can get it to return the right result? Any help is
appreciated.

Thanks,
David J. Ricker II


Nov 20 '05 #3

P: n/a
What the he.....

Try a decimal. see what happens.
"David Ricker" <da*************@hotmail.com> wrote in message
news:uX**************@TK2MSFTNGP10.phx.gbl...
Doubles. I have also tried simply typing 1.005+1.007 into the watch window and it gives the same result.

Thanks,
David J. Ricker II

"CJ Taylor" <no****@blowgoats.com> wrote in message
news:vs************@corp.supernews.com...
What data types are you using?

"David Ricker" <da*************@hotmail.com> wrote in message
news:eW**************@TK2MSFTNGP12.phx.gbl...
I am having problems adding two numbers. I am trying to add 1.005 and 1.007 to come up with 2.012. Should be easy enough right? Problem is

that
I keep getting 2.0119999999999996 as my result. Why is this
happening? Is
there a way that I can get it to return the right result? Any help is
appreciated.

Thanks,
David J. Ricker II



Nov 20 '05 #4

P: n/a
Strange, a decimal does work. Does anyone have any idea why a double won't
work in this situation... is it a bug?

Thanks CJ, I can use the decimals to get the desired results in my program.

Thanks Again,
David J. Ricker II
"CJ Taylor" <no****@blowgoats.com> wrote in message
news:vs************@corp.supernews.com...
What the he.....

Try a decimal. see what happens.
"David Ricker" <da*************@hotmail.com> wrote in message
news:uX**************@TK2MSFTNGP10.phx.gbl...
Doubles. I have also tried simply typing 1.005+1.007 into the watch

window
and it gives the same result.

Thanks,
David J. Ricker II

"CJ Taylor" <no****@blowgoats.com> wrote in message
news:vs************@corp.supernews.com...
What data types are you using?

"David Ricker" <da*************@hotmail.com> wrote in message
news:eW**************@TK2MSFTNGP12.phx.gbl...
> I am having problems adding two numbers. I am trying to add 1.005
and
> 1.007 to come up with 2.012. Should be easy enough right? Problem
is that
> I keep getting 2.0119999999999996 as my result. Why is this

happening? Is
> there a way that I can get it to return the right result? Any help is > appreciated.
>
> Thanks,
> David J. Ricker II
>
>



Nov 20 '05 #5

P: n/a
Cor
Hi David,

I think that this will help you.

I asume your decimal separator is a dot and not a comma

http://msdn.microsoft.com/library/de...roundtopic.asp

I hope this helps a little bit.

Cor
I am having problems adding two numbers. I am trying to add 1.005 and
1.007 to come up with 2.012. Should be easy enough right? Problem is that
I keep getting 2.0119999999999996 as my result. Why is this happening? Isthere a way that I can get it to return the right result? Any help is
appreciated.

Nov 20 '05 #6

P: n/a
One would think so. But I don't know the honest answer. My best guess
would be back to high school chemistry days and understanding the use of Sig
Figs... Could be something to do with that, even though we only have 3 to
work with so I don't know what the big deal is.

I've had the same issue before, and something to do with decimal having a
precision scale is the reason I've heard before. But I quesiton everything.
=)

Then again... it could be a bug, but I'm probably wrong... but if it was,
that sounds like it would be processor level instead of code level.

-CJ

"David Ricker" <da*************@hotmail.com> wrote in message
news:ed**************@tk2msftngp13.phx.gbl...
Strange, a decimal does work. Does anyone have any idea why a double won't
work in this situation... is it a bug?

Thanks CJ, I can use the decimals to get the desired results in my program.
Thanks Again,
David J. Ricker II
"CJ Taylor" <no****@blowgoats.com> wrote in message
news:vs************@corp.supernews.com...
What the he.....

Try a decimal. see what happens.
"David Ricker" <da*************@hotmail.com> wrote in message
news:uX**************@TK2MSFTNGP10.phx.gbl...
Doubles. I have also tried simply typing 1.005+1.007 into the watch window
and it gives the same result.

Thanks,
David J. Ricker II

"CJ Taylor" <no****@blowgoats.com> wrote in message
news:vs************@corp.supernews.com...
> What data types are you using?
>
> "David Ricker" <da*************@hotmail.com> wrote in message
> news:eW**************@TK2MSFTNGP12.phx.gbl...
> > I am having problems adding two numbers. I am trying to add 1.005 and
> > 1.007 to come up with 2.012. Should be easy enough right?
Problem is > that
> > I keep getting 2.0119999999999996 as my result. Why is this happening?
> Is
> > there a way that I can get it to return the right result? Any
help is > > appreciated.
> >
> > Thanks,
> > David J. Ricker II
> >
> >
>
>



Nov 20 '05 #7

P: n/a
Bob
http://www.math.grin.edu/~stone/cour...EEE-reals.html

Bob

"David Ricker" <da*************@hotmail.com> wrote in message
news:eW**************@TK2MSFTNGP12.phx.gbl...
I am having problems adding two numbers. I am trying to add 1.005 and
1.007 to come up with 2.012. Should be easy enough right? Problem is that
I keep getting 2.0119999999999996 as my result. Why is this happening? Is
there a way that I can get it to return the right result? Any help is
appreciated.

Thanks,
David J. Ricker II


Nov 20 '05 #8

P: n/a
David,
In addition to CJ & Cor's posts.

http://msdn.microsoft.com/library/de...egralTypes.asp

1.007 & 2.012 are by default a Double. Double & Single are base 2 floating
point numbers.

Decimal is a base 10 floating point number. (technally base 2, but think of
it as base 10 ;-))

Decimals are stored as a whole number (1234.) that is scaled, where as
Single & Double are stored as a fraction number (1.1234) that is scaled.
Seeing as both are represented as binary number internally, a binary whole
number can store all decimal numbers, where as a binary fraction cannot
store all decimal fractions. Hence the rounding issue you are seeing.

Hope this helps
Jay

"David Ricker" <da*************@hotmail.com> wrote in message
news:eW**************@TK2MSFTNGP12.phx.gbl...
I am having problems adding two numbers. I am trying to add 1.005 and
1.007 to come up with 2.012. Should be easy enough right? Problem is that I keep getting 2.0119999999999996 as my result. Why is this happening? Is there a way that I can get it to return the right result? Any help is
appreciated.

Thanks,
David J. Ricker II

Nov 20 '05 #9

P: n/a
Thanks Jay,

At least now I know that it is by design.
"Jay B. Harlow [MVP - Outlook]" <Ja************@msn.com> wrote in message
news:%2***************@tk2msftngp13.phx.gbl...
David,
In addition to CJ & Cor's posts.

http://msdn.microsoft.com/library/de...egralTypes.asp
1.007 & 2.012 are by default a Double. Double & Single are base 2 floating
point numbers.

Decimal is a base 10 floating point number. (technally base 2, but think of it as base 10 ;-))

Decimals are stored as a whole number (1234.) that is scaled, where as
Single & Double are stored as a fraction number (1.1234) that is scaled.
Seeing as both are represented as binary number internally, a binary whole
number can store all decimal numbers, where as a binary fraction cannot
store all decimal fractions. Hence the rounding issue you are seeing.

Hope this helps
Jay

"David Ricker" <da*************@hotmail.com> wrote in message
news:eW**************@TK2MSFTNGP12.phx.gbl...
I am having problems adding two numbers. I am trying to add 1.005 and 1.007 to come up with 2.012. Should be easy enough right? Problem is

that
I keep getting 2.0119999999999996 as my result. Why is this happening?

Is
there a way that I can get it to return the right result? Any help is
appreciated.

Thanks,
David J. Ricker II


Nov 20 '05 #10

P: n/a
* "David Ricker" <da*************@hotmail.com> scripsit:
I am having problems adding two numbers. I am trying to add 1.005 and
1.007 to come up with 2.012. Should be easy enough right? Problem is that
I keep getting 2.0119999999999996 as my result. Why is this happening? Is
there a way that I can get it to return the right result? Any help is
appreciated.


<http://research.microsoft.com/~hollasch/cgindex/coding/ieeefloat.html>
<http://support.microsoft.com/default.aspx?scid=kb;de;42980>

--
Herfried K. Wagner [MVP]
<http://www.mvps.org/dotnet>
Nov 20 '05 #11

This discussion thread is closed

Replies have been disabled for this discussion.