473,327 Members | 2,065 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,327 software developers and data experts.

rounding problem

It is on Windows, Linux, Python 2.3:

[GCC 3.3.2 (Mandrake Linux 10.0 3.3.2-6mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
a=1.1
a 1.1000000000000001

Is it normal?

Andy
Jul 18 '05 #1
7 1676
In article <gu********************@comcast.com>,
Andy Leszczynski <le*********@nospamyahoo.com> wrote:
It is on Windows, Linux, Python 2.3:

[GCC 3.3.2 (Mandrake Linux 10.0 3.3.2-6mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
a=1.1
a1.1000000000000001

Is it normal?


http://www.python.org/doc/current/tut/node16.html
--
Aahz (aa**@pythoncraft.com) <*> http://www.pythoncraft.com/

"The joy of coding Python should be in seeing short, concise, readable
classes that express a lot of action in a small amount of clear code --
not in reams of trivial code that bores the reader to death." --GvR
Jul 18 '05 #2
In article <gu********************@comcast.com>, Andy Leszczynski wrote:
a=1.1
a 1.1000000000000001

Is it normal?


Yes, for floating-point numbers. This is due to inherent imprecision
in how floats are represented in hardware. If you can live with being
a touch off that many decimal places out, you can just clean up upon
printing with something like:
print "%0.1f" % (a) 1.1

for your value of a above. If you don't want to deal with this cruft,
you may want to give the Decimal class a gander:
from Decimal import *
setcontext(ExtendedContext)
a = Decimal('1.1')
a 'Decimal("1.1")' print a

1.1
--
zoerhoff(AT)sdf.lonestar.org
kristian.zoerhoff(AT)gmail.com
Jul 18 '05 #3
> Is it normal?

Yes. The interpreter prints back the repr of a, which reflects the
imprecision inherent in floats. If you want '1.1', use the string
returned by the str function.
a = 1.1
a 1.1000000000000001 repr(a) '1.1000000000000001' str(a)

'1.1'

Michael

--
Michael D. Hartl, Ph.D.
CTO, Quark Sports LLC
http://quarksports.com/

Jul 18 '05 #4
tom
On Wed, 23 Feb 2005 19:04:47 -0600, Andy Leszczynski wrote:
It is on Windows, Linux, Python 2.3:

[GCC 3.3.2 (Mandrake Linux 10.0 3.3.2-6mdk)] on linux2 Type "help",
"copyright", "credits" or "license" for more information.
>>> a=1.1
>>> a 1.1000000000000001 >>>
>>>
>>> Is it normal?


Yes. Quite normal.

Floating point numbers are by nature approximations. When you enter your
variable "a" at the prompt, you are asking for a very raw (i.e totally
unformatted!) dump of what your variable "a" contains. It is the
most precise representation of the float 1.1 that your computer can store.
It is probably the best precision your hardware math processor subsystem
has to offer. And, incidentally, that ain't too shabby since a 1 in the
16th decimal place ain't worth spit!

Try putting a print statement in front of your variable.
print a
1.1


The print statement ,by itself, adds the most minimial formatting, which
amounts to dropping the last significant digit your math processor holds.
That last digit will *always* contain some arithmetic slop.

If you do any serious work with floats, you will want to learn about
significant figures and the use of the very essential round() function in
whatever language you are using. If you do, you will find that this
situation is really a math thing and has little to do with computers or
any particular language. It's the exact same problem one runs into when
using a slide rule.

If anyone remembers those things :-)
Thomas Bartkus

Jul 18 '05 #5
tom wrote:
On Wed, 23 Feb 2005 19:04:47 -0600, Andy Leszczynski wrote:
It is on Windows, Linux, Python 2.3:

[GCC 3.3.2 (Mandrake Linux 10.0 3.3.2-6mdk)] on linux2 Type "help",
"copyright", "credits" or "license" for more information.
>>> a=1.1
>>> a 1.1000000000000001 .... Is it normal?


Yes. Quite normal.

.... The print statement ,by itself, adds the most minimial formatting, which amounts to dropping the last significant digit your math processor holds. That last digit will *always* contain some arithmetic slop.


Your statement is misleading, because it suggests that your processor
stores digits. It doesn't; it stores *bits*.

The distinction is important because 1.1 does not have a finite
representation in binary. Instead, it's the repeating bit sequence 1.0
0011 0011 0011...
This is analogous to 1/3 = 0.333... in base ten.

IEEE 754 requires that all normalized numbers be rounded to 53
significant bits, so "1.1" is actually stored as 0x1.199999999999A,
which is equivalent to the decimal number
1.100000000000000088817841970012523233890533447265 625. Python's repr
function rounds this to 17 significant digits, or "1.1000000000000001".

Jul 18 '05 #6
"Dan Bishop" <da*****@yahoo.com> wrote in message
news:11**********************@f14g2000cwb.googlegr oups.com...
tom wrote:

<snip>
That last digit will *always* contain some arithmetic slop.


Your statement is misleading, because it suggests that your processor
stores digits. It doesn't; it stores *bits*.


Your explanation is much clearer and more than I knew. And I am sorry you
found my statement a bit misleading. But I never implied details about how
things were stored internally.

No matter what computer, calculater, slide rule or programming language - a
floating point number will always exhibit "arithmetic slop" at the last
significant digit. This is a property inherent to floating point numbers and
has nothing to do with how it is stored on any machine.

Thomas Bartkus
Jul 18 '05 #7
Dan Bishop wrote:
Your statement is misleading, because it suggests that your processor
stores digits. It doesn't; it stores *bits*.


And where does the word 'bit' come from, hmm? It couldn't possibly be an
abbreviation of Binary digIT, could it?

Cheers,
Nick.

--
Nick Coghlan | nc******@email.com | Brisbane, Australia
---------------------------------------------------------------
http://boredomandlaziness.skystorm.net
Jul 18 '05 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Caesar Augustus | last post by:
Hello Tom, Long time listener, first time caller... I have been working with vb code to automate a salary increase process that not only rounds to the nearest penny but also rounds to the...
0
by: Rick Gabel | last post by:
Trying to use the decimal datatype with SQL Server2000 and C#.Net and having a rounding problem. I am using SqlCommand/SqlParameter process to retrieve and insert a record into the table. One...
5
by: Jason | last post by:
I am having a rounding problem in a value i am trying to display in VB.NET. If I have the following code: Dim x As Single = 2726.795 Dim y As Single = Math.Round(2726.795, 2) Dim s as String =...
11
by: cj | last post by:
Lets assume all calculations are done with decimal data types so things are as precise as possible. When it comes to the final rounding to cut a check to pay dividends for example in VB rounding...
4
by: Altman | last post by:
I am having a rounding problem all over the place and am struggling to fix it. My problem is that I have an unit price and a quantity. When I multiply them together they come out to 3.705. I...
29
by: Marco | last post by:
Hello, I have : float f = 36.09999999; When I do : char cf; sprintf(cf,"%0.03lf", f); I get : 36.100
18
by: jdrott1 | last post by:
i'm trying to round my currency string to end in 9. it's for a pricing application. this is the function i'm using to get the item in currency: FormatCurrency(BoxCost, , , , TriState.True) if...
206
by: md | last post by:
Hi Does any body know, how to round a double value with a specific number of digits after the decimal points? A function like this: RoundMyDouble (double &value, short numberOfPrecisions) ...
20
by: jacob navia | last post by:
Hi "How can I round a number to x decimal places" ? This question keeps appearing. I would propose the following solution #include <float.h> #include <math.h>
30
by: bdsatish | last post by:
The built-in function round( ) will always "round up", that is 1.5 is rounded to 2.0 and 2.5 is rounded to 3.0. If I want to round to the nearest even, that is my_round(1.5) = 2 # As...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.