473,320 Members | 1,691 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Working with floating point values

I'm working on a project that uses floating-point values (doubles), and I'm
being driven crazy by something pretty basic. I understand that it's in the
nature of floating-point calculations to produce values like
0.10000000000000003, when what I really want is 0.1. But is there any way to
eliminate that digit at the end? I've tried rounding, but that simply moves
the digit to the least significant position, such as 0.1000003.

Failing that, can anyone recommend any good online articles for dealing with
floating-point issues, beyond using Epsilon for zero comparisons? Thanks in
advance.

David Veeneman
Foresight Systems
Nov 17 '05 #1
9 1760
David,

I've had a quick go at your problem and the Math.Round method does seem
to work.

static void Main(string[] args)
{

double TestNumber = 0.100000000003;

Console.WriteLine ( TestNumber.ToString() );

Console.WriteLine ( Math.Round ( TestNumber, 1 ) );
Console.WriteLine ( Math.Round ( TestNumber, 5 ) );
Console.ReadLine();
}

OUTPUT:

0.100000000003
0.1
0.1

Hope this helps

David Veeneman wrote:
I'm working on a project that uses floating-point values (doubles), and I'm
being driven crazy by something pretty basic. I understand that it's in the
nature of floating-point calculations to produce values like
0.10000000000000003, when what I really want is 0.1. But is there any way to
eliminate that digit at the end? I've tried rounding, but that simply moves
the digit to the least significant position, such as 0.1000003.

Failing that, can anyone recommend any good online articles for dealing with
floating-point issues, beyond using Epsilon for zero comparisons? Thanks in
advance.

David Veeneman
Foresight Systems


Nov 17 '05 #2
Just use Decimal.
--
Brian Delahunty
Ireland

http://briandela.com/blog
"David Veeneman" wrote:
I'm working on a project that uses floating-point values (doubles), and I'm
being driven crazy by something pretty basic. I understand that it's in the
nature of floating-point calculations to produce values like
0.10000000000000003, when what I really want is 0.1. But is there any way to
eliminate that digit at the end? I've tried rounding, but that simply moves
the digit to the least significant position, such as 0.1000003.

Failing that, can anyone recommend any good online articles for dealing with
floating-point issues, beyond using Epsilon for zero comparisons? Thanks in
advance.

David Veeneman
Foresight Systems

Nov 17 '05 #3
you could format the number like
String.Format("{0:#,##0.00;#,##0.00;0.00}", myDecimal.ToString()) ;
(assuming you want a string at the end)

Ranjeet.
Nov 17 '05 #4
That's what I've always done in the past, but decimals are *much* slower to
process than floating-points. In this project, I'm running some fairly
complex calculations thousands of times each, so the difference between
decimals and doubles is quite noticeable. So, I'm pretty much stuck with
doubles.

"Brian Delahunty" <Br************@discussions.microsoft.com> wrote in
message news:FA**********************************@microsof t.com...
Just use Decimal.
--
Brian Delahunty
Ireland

http://briandela.com/blog
"David Veeneman" wrote:
I'm working on a project that uses floating-point values (doubles), and
I'm
being driven crazy by something pretty basic. I understand that it's in
the
nature of floating-point calculations to produce values like
0.10000000000000003, when what I really want is 0.1. But is there any way
to
eliminate that digit at the end? I've tried rounding, but that simply
moves
the digit to the least significant position, such as 0.1000003.

Failing that, can anyone recommend any good online articles for dealing
with
floating-point issues, beyond using Epsilon for zero comparisons? Thanks
in
advance.

David Veeneman
Foresight Systems

Nov 17 '05 #5
Thanks-- I'd thought about that, but I need to maintain the values as
doubles. And the time it would take to convert to string, then back to
double, would slow my calculations down noticeably.

"ranjeet" <ra************@gmail.com> wrote in message
news:op**************@athwalrt40.castlewood.co.uk. ..
you could format the number like
String.Format("{0:#,##0.00;#,##0.00;0.00}", myDecimal.ToString()) ;
(assuming you want a string at the end)

Ranjeet.

Nov 17 '05 #6
KH
Rounding won't help because the some floating point numbers simply can't be
exactly represented in binary. ...
http://docs.sun.com/source/806-3568/...dberg.html#680

If you can't/won't use decimal, you can test conditionals by seeing if a
number is within a range:

double dd = 0.10000000000000003;

if (dd > 0.09 && dd < 0.11) // true, although the number could be 0.091
0.109 ...
// but you can increase the precision in the conditional

If you doing arithmitic with floating point numbers usually you just do the
calculations and round before displaying.

"David Veeneman" wrote:
I'm working on a project that uses floating-point values (doubles), and I'm
being driven crazy by something pretty basic. I understand that it's in the
nature of floating-point calculations to produce values like
0.10000000000000003, when what I really want is 0.1. But is there any way to
eliminate that digit at the end? I've tried rounding, but that simply moves
the digit to the least significant position, such as 0.1000003.

Failing that, can anyone recommend any good online articles for dealing with
floating-point issues, beyond using Epsilon for zero comparisons? Thanks in
advance.

David Veeneman
Foresight Systems

Nov 17 '05 #7
Well, I found the canonical 1991 article on the subject: "What Every
Computer Scientist Should Know About Floating-Point Arithmetic", by David
Goldberg. Here is a link to the article in PDF format:

http://www.physics.ohio-state.edu/~d...point_math.pdf
Nov 17 '05 #8
Thanks-- the reference to the Goldberg article is very helpful.
Nov 17 '05 #9
If precision is an issue in your calculation, then you have to use decimal.
otherwise, keep using doubles is fine, and use the right output format to
trim off the rounding error when displaying the results.

"David Veeneman" wrote:
Thanks-- I'd thought about that, but I need to maintain the values as
doubles. And the time it would take to convert to string, then back to
double, would slow my calculations down noticeably.

"ranjeet" <ra************@gmail.com> wrote in message
news:op**************@athwalrt40.castlewood.co.uk. ..
you could format the number like
String.Format("{0:#,##0.00;#,##0.00;0.00}", myDecimal.ToString()) ;
(assuming you want a string at the end)

Ranjeet.


Nov 17 '05 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

687
by: cody | last post by:
no this is no trollposting and please don't get it wrong but iam very curious why people still use C instead of other languages especially C++. i heard people say C++ is slower than C but i can't...
3
by: Mark L Pappin | last post by:
<puts on Compiler Vendor hat> I've recently discovered that our compilers don't make any attempt to handle floating point overflow in add/subtract/ multiply/divide, with the result that...
17
by: Jonathan Burd | last post by:
Greetings everyone, Reading about the int64_t types added by C'99, I decided to implement a 64-bit version of atoi for my own library. However, for the test number provided, it isn't doing what...
5
by: Steffen | last post by:
Hi, is it possible to have two fractions, which (mathematically) have the order a/b < c/d (a,b,c,d integers), but when (correctly) converted into floating point representation just have the...
15
by: michael.mcgarry | last post by:
Hi, I have a question about floating point precision in C. What is the minimum distinguishable difference between 2 floating point numbers? Does this differ for various computers? Is this...
5
by: Peteroid | last post by:
I know how to use rand() to generate random POSITIVE-INTEGER numbers. But, I'd like to generate a random DOUBLE number in the range of 0.0 to 1.0 with resolution of a double (i.e., every possible...
7
by: thisismyidentity | last post by:
Hi all, I am trying to predict the behaviour of floating point load and store operations on integer locations. I ve written a small piece of code having some inline assembly code which I am...
15
by: Mukesh_Singh_Nick | last post by:
Why does floating point have a rounding error? How to work around it? For example, the following: flaot f = 1234.12345678F; printf("%2f\n", f) //prints 1234.123413 and
6
by: shira | last post by:
Hi, Looking to see if someone might have an explanation for this behavior. Is it a bug? Corruption? I have been able to reproduce the problem with only 2 rows and 1 field. Here is the table:...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.