By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
443,302 Members | 1,816 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 443,302 IT Pros & Developers. It's quick & easy.

Working with floating point values

P: n/a
I'm working on a project that uses floating-point values (doubles), and I'm
being driven crazy by something pretty basic. I understand that it's in the
nature of floating-point calculations to produce values like
0.10000000000000003, when what I really want is 0.1. But is there any way to
eliminate that digit at the end? I've tried rounding, but that simply moves
the digit to the least significant position, such as 0.1000003.

Failing that, can anyone recommend any good online articles for dealing with
floating-point issues, beyond using Epsilon for zero comparisons? Thanks in
advance.

David Veeneman
Foresight Systems
Nov 17 '05 #1
Share this Question
Share on Google+
9 Replies


P: n/a
David,

I've had a quick go at your problem and the Math.Round method does seem
to work.

static void Main(string[] args)
{

double TestNumber = 0.100000000003;

Console.WriteLine ( TestNumber.ToString() );

Console.WriteLine ( Math.Round ( TestNumber, 1 ) );
Console.WriteLine ( Math.Round ( TestNumber, 5 ) );
Console.ReadLine();
}

OUTPUT:

0.100000000003
0.1
0.1

Hope this helps

David Veeneman wrote:
I'm working on a project that uses floating-point values (doubles), and I'm
being driven crazy by something pretty basic. I understand that it's in the
nature of floating-point calculations to produce values like
0.10000000000000003, when what I really want is 0.1. But is there any way to
eliminate that digit at the end? I've tried rounding, but that simply moves
the digit to the least significant position, such as 0.1000003.

Failing that, can anyone recommend any good online articles for dealing with
floating-point issues, beyond using Epsilon for zero comparisons? Thanks in
advance.

David Veeneman
Foresight Systems


Nov 17 '05 #2

P: n/a
Just use Decimal.
--
Brian Delahunty
Ireland

http://briandela.com/blog
"David Veeneman" wrote:
I'm working on a project that uses floating-point values (doubles), and I'm
being driven crazy by something pretty basic. I understand that it's in the
nature of floating-point calculations to produce values like
0.10000000000000003, when what I really want is 0.1. But is there any way to
eliminate that digit at the end? I've tried rounding, but that simply moves
the digit to the least significant position, such as 0.1000003.

Failing that, can anyone recommend any good online articles for dealing with
floating-point issues, beyond using Epsilon for zero comparisons? Thanks in
advance.

David Veeneman
Foresight Systems

Nov 17 '05 #3

P: n/a
you could format the number like
String.Format("{0:#,##0.00;#,##0.00;0.00}", myDecimal.ToString()) ;
(assuming you want a string at the end)

Ranjeet.
Nov 17 '05 #4

P: n/a
That's what I've always done in the past, but decimals are *much* slower to
process than floating-points. In this project, I'm running some fairly
complex calculations thousands of times each, so the difference between
decimals and doubles is quite noticeable. So, I'm pretty much stuck with
doubles.

"Brian Delahunty" <Br************@discussions.microsoft.com> wrote in
message news:FA**********************************@microsof t.com...
Just use Decimal.
--
Brian Delahunty
Ireland

http://briandela.com/blog
"David Veeneman" wrote:
I'm working on a project that uses floating-point values (doubles), and
I'm
being driven crazy by something pretty basic. I understand that it's in
the
nature of floating-point calculations to produce values like
0.10000000000000003, when what I really want is 0.1. But is there any way
to
eliminate that digit at the end? I've tried rounding, but that simply
moves
the digit to the least significant position, such as 0.1000003.

Failing that, can anyone recommend any good online articles for dealing
with
floating-point issues, beyond using Epsilon for zero comparisons? Thanks
in
advance.

David Veeneman
Foresight Systems

Nov 17 '05 #5

P: n/a
Thanks-- I'd thought about that, but I need to maintain the values as
doubles. And the time it would take to convert to string, then back to
double, would slow my calculations down noticeably.

"ranjeet" <ra************@gmail.com> wrote in message
news:op**************@athwalrt40.castlewood.co.uk. ..
you could format the number like
String.Format("{0:#,##0.00;#,##0.00;0.00}", myDecimal.ToString()) ;
(assuming you want a string at the end)

Ranjeet.

Nov 17 '05 #6

P: n/a
KH
Rounding won't help because the some floating point numbers simply can't be
exactly represented in binary. ...
http://docs.sun.com/source/806-3568/...dberg.html#680

If you can't/won't use decimal, you can test conditionals by seeing if a
number is within a range:

double dd = 0.10000000000000003;

if (dd > 0.09 && dd < 0.11) // true, although the number could be 0.091
0.109 ...
// but you can increase the precision in the conditional

If you doing arithmitic with floating point numbers usually you just do the
calculations and round before displaying.

"David Veeneman" wrote:
I'm working on a project that uses floating-point values (doubles), and I'm
being driven crazy by something pretty basic. I understand that it's in the
nature of floating-point calculations to produce values like
0.10000000000000003, when what I really want is 0.1. But is there any way to
eliminate that digit at the end? I've tried rounding, but that simply moves
the digit to the least significant position, such as 0.1000003.

Failing that, can anyone recommend any good online articles for dealing with
floating-point issues, beyond using Epsilon for zero comparisons? Thanks in
advance.

David Veeneman
Foresight Systems

Nov 17 '05 #7

P: n/a
Well, I found the canonical 1991 article on the subject: "What Every
Computer Scientist Should Know About Floating-Point Arithmetic", by David
Goldberg. Here is a link to the article in PDF format:

http://www.physics.ohio-state.edu/~d...point_math.pdf
Nov 17 '05 #8

P: n/a
Thanks-- the reference to the Goldberg article is very helpful.
Nov 17 '05 #9

P: n/a
If precision is an issue in your calculation, then you have to use decimal.
otherwise, keep using doubles is fine, and use the right output format to
trim off the rounding error when displaying the results.

"David Veeneman" wrote:
Thanks-- I'd thought about that, but I need to maintain the values as
doubles. And the time it would take to convert to string, then back to
double, would slow my calculations down noticeably.

"ranjeet" <ra************@gmail.com> wrote in message
news:op**************@athwalrt40.castlewood.co.uk. ..
you could format the number like
String.Format("{0:#,##0.00;#,##0.00;0.00}", myDecimal.ToString()) ;
(assuming you want a string at the end)

Ranjeet.


Nov 17 '05 #10

This discussion thread is closed

Replies have been disabled for this discussion.