473,385 Members | 1,798 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

floating point rounding

hg
Hi,

Here is my issue:

f = 1.5 * 0.01
f
>0.014999999999999999
'%f' % f
>>'0.015000'

But I really want to get 0.02 as a result ... is there a way out ?

Thanks,

hg

Mar 9 '07 #1
4 1214
hg
hg wrote:
Hi,

Here is my issue:

f = 1.5 * 0.01
f
>>0.014999999999999999
'%f' % f
>>>'0.015000'


But I really want to get 0.02 as a result ... is there a way out ?

Thanks,

hg
round

Mar 9 '07 #2
On Mar 9, 5:45 am, hg <h...@nospam.orgwrote:
hg wrote:
Hi,
Here is my issue:
f = 1.5 * 0.01
f
>0.014999999999999999
'%f' % f
>>'0.015000'
But I really want to get 0.02 as a result ... is there a way out ?
Thanks,
hg

round

Or more precisely:

round(0.014999999999999999,2)

if that's what you wish to do.

Mar 9 '07 #3
hg
John Henry wrote:
On Mar 9, 5:45 am, hg <h...@nospam.orgwrote:
>hg wrote:
Hi,
Here is my issue:
f = 1.5 * 0.01
f
0.014999999999999999
'%f' % f
'0.015000'
But I really want to get 0.02 as a result ... is there a way out ?
Thanks,
hg

round


Or more precisely:

round(0.014999999999999999,2)

if that's what you wish to do.
Indeed.

hg

Mar 9 '07 #4
John Henry wrote:
Or more precisely:

round(0.014999999999999999,2)
No, that *won't* solve the problem. Using a slightly
different example,
>>x = 1.5 * 0.1
x
0.15000000000000002
>>round(x, 2)
0.14999999999999999

The problem is that floats are stored internally in
binary, not decimal, and numbers like 0.1 and 0.01
have no exact representation as a binary float.
Using round() doesn't help, because the result is
still a binary float, and the result you're after
still can't be represented.

The best you can do is to *display* it rounded
to the number of digits you want using formatting,
e.g.
>>"%.2f" % x
'0.15'

Alternatively, use the Decimal module, which stores
numbers as decimal and does arithmetic in ways that
will match your calculator. It's slower, though.

--
Greg
Mar 10 '07 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: Dave | last post by:
Hi folks, I am trying to develop a routine that will handle sphere-sphere and sphere-triangle collisions and interactions. My aim is to develop a quake style collision engine where a player can...
31
by: JS | last post by:
We have the same floating point intensive C++ program that runs on Windows on Intel chip and on Sun Solaris on SPARC chips. The program reads the exactly the same input files on the two platforms....
9
by: Klaus Bonadt | last post by:
I have found strange behaviour in casting floating point values in C++ 6.0 to int: If I enter in the watch window while debugging in version 6.0 the following term: (1.4 - 1.0) * 10.0 the...
7
by: Hiten | last post by:
Hi please check ffollowing conditon variable float1 and float2 holds user entered value..... Answer=float1 * float2; //output must be 8.5 output must to be 8.5 but it has 8.500002, i am...
9
by: David Veeneman | last post by:
I'm working on a project that uses floating-point values (doubles), and I'm being driven crazy by something pretty basic. I understand that it's in the nature of floating-point calculations to...
4
by: jacob navia | last post by:
Hi people I continue to work in the tutorial for lcc-win32, and started to try to explain the floating point flags. Here is the relevant part of the tutorial. Since it is a difficult part, I...
7
by: thisismyidentity | last post by:
Hi all, I am trying to predict the behaviour of floating point load and store operations on integer locations. I ve written a small piece of code having some inline assembly code which I am...
18
by: n.torrey.pines | last post by:
I understand that with floats, x*y == y*x, for example, might not hold. But what if it's the exact same operation on both sides? I tested this with GCC 3.4.4 (Cygwin), and it prints 0 (compiled...
15
by: Mukesh_Singh_Nick | last post by:
Why does floating point have a rounding error? How to work around it? For example, the following: flaot f = 1234.12345678F; printf("%2f\n", f) //prints 1234.123413 and
31
by: Mark Dickinson | last post by:
On SuSE 10.2/Xeon there seems to be a rounding bug for floating-point addition: dickinsm@weyl:~python Python 2.5 (r25:51908, May 25 2007, 16:14:04) on linux2 Type "help", "copyright",...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.