471,325 Members | 1,663 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,325 software developers and data experts.

Strange behaviour when using floats

Hi,

I have the following very simple code snippet:

line 1: float f;
line 2: f = 100000.99f;

Why is f: 100000.992 after line 2? I would assume it to be 100000.99.

Cheers
Henrik.
Sep 16 '08 #1
8 1227
on 16-9-2008, Henrik Skak Pedersen supposed :
Hi,

I have the following very simple code snippet:

line 1: float f;
line 2: f = 100000.99f;

Why is f: 100000.992 after line 2? I would assume it to be 100000.99.

Cheers
Henrik.
See http://www.yoda.arachsys.com/csharp/floatingpoint.html
Sep 16 '08 #2
Hi Hans,

Thank you for your reply. So I guess that decimal is my choice then.

Cheers
Henrik.

"Hans Kesting" <ne*********@spamgourmet.comwrote in message
news:uz****************@TK2MSFTNGP03.phx.gbl...
on 16-9-2008, Henrik Skak Pedersen supposed :
>Hi,

I have the following very simple code snippet:

line 1: float f;
line 2: f = 100000.99f;

Why is f: 100000.992 after line 2? I would assume it to be 100000.99.

Cheers
Henrik.

See http://www.yoda.arachsys.com/csharp/floatingpoint.html


Sep 16 '08 #3
Henrik Skak Pedersen wrote:
Hi Hans,

Thank you for your reply. So I guess that decimal is my choice then.

Cheers
Henrik.

"Hans Kesting" <ne*********@spamgourmet.comwrote in message
news:uz****************@TK2MSFTNGP03.phx.gbl...
>on 16-9-2008, Henrik Skak Pedersen supposed :
>>Hi,

I have the following very simple code snippet:

line 1: float f;
line 2: f = 100000.99f;

Why is f: 100000.992 after line 2? I would assume it to be 100000.99.

Cheers
Henrik.
See http://www.yoda.arachsys.com/csharp/floatingpoint.html

You can use a double if you just want better precision, but still want
the performance of a type that is supported by the ALU of the processor.

A decimal still doesn't represent values exactly, but you get something
like 100000.990000002 rather than 100000.992.

--
Göran Andersson
_____
http://www.guffa.com
Sep 16 '08 #4
On Sep 16, 12:49*pm, Göran Andersson <gu...@guffa.comwrote:
You can use a double if you just want better precision, but still want
the performance of a type that is supported by the ALU of the processor.

A decimal still doesn't represent values exactly, but you get something
like 100000.990000002 rather than 100000.992.
Both double and decimal represent exact values - just different sets
of values. And decimal certainly *can* represent 10000.992 exactly.

Jon
Sep 16 '08 #5
On Tue, 16 Sep 2008 06:22:40 -0700, Jon Skeet [C# MVP] <sk***@pobox.com>
wrote:
>A decimal still doesn't represent values exactly, but you get something
like 100000.990000002 rather than 100000.992.

Both double and decimal represent exact values - just different sets
of values. And decimal certainly *can* represent 10000.992 exactly.
Or even 100000.99, as seen in the original post. :)
Sep 16 '08 #6
Thank you all for your replies.

"Peter Duniho" <Np*********@nnowslpianmk.comwrote in message
news:op***************@petes-computer.local...
On Tue, 16 Sep 2008 06:22:40 -0700, Jon Skeet [C# MVP] <sk***@pobox.com>
wrote:
>>A decimal still doesn't represent values exactly, but you get something
like 100000.990000002 rather than 100000.992.

Both double and decimal represent exact values - just different sets
of values. And decimal certainly *can* represent 10000.992 exactly.

Or even 100000.99, as seen in the original post. :)

Sep 17 '08 #7
Jon Skeet [C# MVP] wrote:
On Sep 16, 12:49 pm, Göran Andersson <gu...@guffa.comwrote:
>You can use a double if you just want better precision, but still want
the performance of a type that is supported by the ALU of the processor.

A decimal
Obviously I meant a double.
still doesn't represent values exactly, but you get something
>like 100000.990000002 rather than 100000.992.

Both double and decimal represent exact values - just different sets
of values.
What do you mean by that? A double is a floating point type, and
approximates most values.
And decimal certainly *can* represent 10000.992 exactly.
A decimal stores 100000.99 as 100000.99, not as 100000.992.

--
Göran Andersson
_____
http://www.guffa.com
Sep 17 '08 #8
Göran Andersson <gu***@guffa.comwrote:
Jon Skeet [C# MVP] wrote:
On Sep 16, 12:49 pm, Göran Andersson <gu...@guffa.comwrote:
You can use a double if you just want better precision, but still want
the performance of a type that is supported by the ALU of the processor.

A decimal
Obviously I meant a double.
Sorry, it wasn't obvious to me. I thought you really were casting
aspersions on decimal :)
still doesn't represent values exactly, but you get something
like 100000.990000002 rather than 100000.992.
Both double and decimal represent exact values - just different sets
of values.
What do you mean by that? A double is a floating point type, and
approximates most values.
Decimal is a floating point type too - just a floating decimal point
instead of a floating binary point. Both types have a set of numbers
they can represent exactly. You can't represent 1/3 exactly in decimal
any more than you can represent 1/5 exactly in binary.

The reason that decimal "feels" precise whereas binary "feels"
approximate is that humans tend to use a decimal system - we tend to
think that 1/5 is a more "normal" number than 1/3. When you look at it
in a mathematical sense, decimal has more precision and less range than
double, but both are just sets of numbers with conversions and
operations which will approximate the accurate value to the nearest
value in the target set.
And decimal certainly *can* represent 10000.992 exactly.
A decimal stores 100000.99 as 100000.99, not as 100000.992.
Indeed - I didn't check the original post, just yours.

--
Jon Skeet - <sk***@pobox.com>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com
Sep 17 '08 #9

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

45 posts views Thread by Jordan Rastrick | last post: by
1 post views Thread by David Ehmer | last post: by
3 posts views Thread by Sebastian C. | last post: by
11 posts views Thread by Marlene Stebbins | last post: by
5 posts views Thread by Vedran FuraÄŤ | last post: by
8 posts views Thread by Dox33 | last post: by
reply views Thread by rosydwin | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.