443,302 Members | 1,816 Online
Need help? Post your question and get tips & solutions from a community of 443,302 IT Pros & Developers. It's quick & easy.

# Working with floating point values

 P: n/a I'm working on a project that uses floating-point values (doubles), and I'm being driven crazy by something pretty basic. I understand that it's in the nature of floating-point calculations to produce values like 0.10000000000000003, when what I really want is 0.1. But is there any way to eliminate that digit at the end? I've tried rounding, but that simply moves the digit to the least significant position, such as 0.1000003. Failing that, can anyone recommend any good online articles for dealing with floating-point issues, beyond using Epsilon for zero comparisons? Thanks in advance. David Veeneman Foresight Systems Nov 17 '05 #1
9 Replies

 P: n/a David, I've had a quick go at your problem and the Math.Round method does seem to work. static void Main(string[] args) { double TestNumber = 0.100000000003; Console.WriteLine ( TestNumber.ToString() ); Console.WriteLine ( Math.Round ( TestNumber, 1 ) ); Console.WriteLine ( Math.Round ( TestNumber, 5 ) ); Console.ReadLine(); } OUTPUT: 0.100000000003 0.1 0.1 Hope this helps David Veeneman wrote: I'm working on a project that uses floating-point values (doubles), and I'm being driven crazy by something pretty basic. I understand that it's in the nature of floating-point calculations to produce values like 0.10000000000000003, when what I really want is 0.1. But is there any way to eliminate that digit at the end? I've tried rounding, but that simply moves the digit to the least significant position, such as 0.1000003. Failing that, can anyone recommend any good online articles for dealing with floating-point issues, beyond using Epsilon for zero comparisons? Thanks in advance. David Veeneman Foresight Systems Nov 17 '05 #2

 P: n/a Just use Decimal. -- Brian Delahunty Ireland http://briandela.com/blog "David Veeneman" wrote: I'm working on a project that uses floating-point values (doubles), and I'm being driven crazy by something pretty basic. I understand that it's in the nature of floating-point calculations to produce values like 0.10000000000000003, when what I really want is 0.1. But is there any way to eliminate that digit at the end? I've tried rounding, but that simply moves the digit to the least significant position, such as 0.1000003. Failing that, can anyone recommend any good online articles for dealing with floating-point issues, beyond using Epsilon for zero comparisons? Thanks in advance. David Veeneman Foresight Systems Nov 17 '05 #3

 P: n/a you could format the number like String.Format("{0:#,##0.00;#,##0.00;0.00}", myDecimal.ToString()) ; (assuming you want a string at the end) Ranjeet. Nov 17 '05 #4

 P: n/a That's what I've always done in the past, but decimals are *much* slower to process than floating-points. In this project, I'm running some fairly complex calculations thousands of times each, so the difference between decimals and doubles is quite noticeable. So, I'm pretty much stuck with doubles. "Brian Delahunty" wrote in message news:FA**********************************@microsof t.com... Just use Decimal. -- Brian Delahunty Ireland http://briandela.com/blog "David Veeneman" wrote: I'm working on a project that uses floating-point values (doubles), and I'm being driven crazy by something pretty basic. I understand that it's in the nature of floating-point calculations to produce values like 0.10000000000000003, when what I really want is 0.1. But is there any way to eliminate that digit at the end? I've tried rounding, but that simply moves the digit to the least significant position, such as 0.1000003. Failing that, can anyone recommend any good online articles for dealing with floating-point issues, beyond using Epsilon for zero comparisons? Thanks in advance. David Veeneman Foresight Systems Nov 17 '05 #5

 P: n/a Thanks-- I'd thought about that, but I need to maintain the values as doubles. And the time it would take to convert to string, then back to double, would slow my calculations down noticeably. "ranjeet" wrote in message news:op**************@athwalrt40.castlewood.co.uk. .. you could format the number like String.Format("{0:#,##0.00;#,##0.00;0.00}", myDecimal.ToString()) ; (assuming you want a string at the end) Ranjeet. Nov 17 '05 #6

 P: n/a Rounding won't help because the some floating point numbers simply can't be exactly represented in binary. ... http://docs.sun.com/source/806-3568/...dberg.html#680 If you can't/won't use decimal, you can test conditionals by seeing if a number is within a range: double dd = 0.10000000000000003; if (dd > 0.09 && dd < 0.11) // true, although the number could be 0.091 0.109 ... // but you can increase the precision in the conditional If you doing arithmitic with floating point numbers usually you just do the calculations and round before displaying. "David Veeneman" wrote: I'm working on a project that uses floating-point values (doubles), and I'm being driven crazy by something pretty basic. I understand that it's in the nature of floating-point calculations to produce values like 0.10000000000000003, when what I really want is 0.1. But is there any way to eliminate that digit at the end? I've tried rounding, but that simply moves the digit to the least significant position, such as 0.1000003. Failing that, can anyone recommend any good online articles for dealing with floating-point issues, beyond using Epsilon for zero comparisons? Thanks in advance. David Veeneman Foresight Systems Nov 17 '05 #7

 P: n/a Well, I found the canonical 1991 article on the subject: "What Every Computer Scientist Should Know About Floating-Point Arithmetic", by David Goldberg. Here is a link to the article in PDF format: http://www.physics.ohio-state.edu/~d...point_math.pdf Nov 17 '05 #8

 P: n/a Thanks-- the reference to the Goldberg article is very helpful. Nov 17 '05 #9

 P: n/a If precision is an issue in your calculation, then you have to use decimal. otherwise, keep using doubles is fine, and use the right output format to trim off the rounding error when displaying the results. "David Veeneman" wrote: Thanks-- I'd thought about that, but I need to maintain the values as doubles. And the time it would take to convert to string, then back to double, would slow my calculations down noticeably. "ranjeet" wrote in message news:op**************@athwalrt40.castlewood.co.uk. .. you could format the number like String.Format("{0:#,##0.00;#,##0.00;0.00}", myDecimal.ToString()) ; (assuming you want a string at the end) Ranjeet. Nov 17 '05 #10

### This discussion thread is closed

Replies have been disabled for this discussion.