By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
455,161 Members | 1,301 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 455,161 IT Pros & Developers. It's quick & easy.

Comparing different floats fails

P: n/a
In the following piece of code, the == operator is saying that the two
different float values are the same. What's going on here?
float testValueA = float.MaxValue;
float testValueB = float.MaxValue - (float)1.0;
if (testValueA == testValueB)
{
Console.WriteLine("Test Failed");
}

Feb 4 '07 #1
Share this Question
Share on Google+
2 Replies


P: n/a
O.B. wrote:
In the following piece of code, the == operator is saying that the two
different float values are the same. What's going on here?

float testValueA = float.MaxValue;
float testValueB = float.MaxValue - (float)1.0;
if (testValueA == testValueB)
{
Console.WriteLine("Test Failed");
}
That is how floating point works.

The C# data type float has only about 7 decimal digits
precision.

float.MaxValue is in the 10^38 magnitude.

Adding 1 to that in 7 decimal digits precision gives
the same.

Arne
Feb 4 '07 #2

P: n/a
O.B. <fu******@bellsouth.netwrote:
In the following piece of code, the == operator is saying that the two
different float values are the same. What's going on here?
There aren't two different float values there. When you subtract one
from the largest possible float value, it's still closer to the largest
possible float value than to any other value, so testValueB is the same
as testValueA.

For a more concrete understanding of this - suppose you are writing
down numbers to three significant decimal digits. Now work out the
result of this calculation:

123000 - 0.0000987

The "perfct" result is 122999.9999013. However, we've got to represent
it to three significant figures - which is 123000 again.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Feb 4 '07 #3

This discussion thread is closed

Replies have been disabled for this discussion.