473,396 Members | 2,002 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,396 software developers and data experts.

FAQ Topic - Why does simple decimal arithmetic give strange results? (2007-12-16)

-----------------------------------------------------------------------
FAQ Topic - Why does simple decimal arithmetic give strange
results?
-----------------------------------------------------------------------

For example, 5*1.015 does not give exactly 5.075 and 0.06+0.01 does
not give exactly 0.07 in javascript.

Javascript numbers are represented in binary as IEEE-754 (IEC 559)
Doubles, with a resolution of 53 bits, giving an accuracy of
15-16 decimal digits; integers up to about 9e15 are precise, but
few decimal fractions are. Given this, arithmetic is as exact
as possible, but no more. Operations on integers are exact if
the true result and all intermediates are integers within that
range.

In particular, non-integer results should not normally be
compared for equality; and non-integer computed results
commonly need rounding; see 4.6.

http://msdn.microsoft.com/library/de...6dbf483b41.asp

Otherwise, use Math.round on the results of expressions which
should be of integer value.
--
Postings such as this are automatically sent once a day. Their
goal is to answer repeated questions, and to offer the content to
the community for continuous evaluation/improvement. The complete
comp.lang.javascript FAQ is at http://jibbering.com/faq/index.html.
The FAQ workers are a group of volunteers. The sendings of these
daily posts are proficiently hosted by http://www.pair.com.

Dec 16 '07 #1
2 1561
In comp.lang.javascript message <47***********************@news.sunsite.
dk>, Sun, 16 Dec 2007 00:00:01, FAQ server <ja********@dotinternet.be>
posted:
>
Javascript numbers are represented in binary as IEEE-754 (IEC 559)
Doubles, with a resolution of 53 bits, giving an accuracy of
15-16 decimal digits; integers up to about 9e15 are precise, but
few decimal fractions are.
You could add a link to
<URL:http://www.merlyn.demon.co.uk/js-misc0.htm#DW4which shows
exactly what one gets.

5.015*5 gives +5.07499999999999928945726423989981412887573242187 5 which
displays as "5.074999999999999".

--
(c) John Stockton, Surrey, UK. *@merlyn.demon.co.uk / ??*********@physics.org
Web <URL:http://www.merlyn.demon.co.uk/- FAQish topics, acronyms, & links.
Correct <= 4-line sig. separator as above, a line precisely "-- " (SoRFC1036)
Do not Mail News to me. Before a reply, quote with ">" or "" (SoRFC1036)
Dec 16 '07 #2
Dr J R Stockton said the following on 12/16/2007 6:25 PM:
In comp.lang.javascript message <47***********************@news.sunsite.
dk>, Sun, 16 Dec 2007 00:00:01, FAQ server <ja********@dotinternet.be>
posted:
>Javascript numbers are represented in binary as IEEE-754 (IEC 559)
Doubles, with a resolution of 53 bits, giving an accuracy of
15-16 decimal digits; integers up to about 9e15 are precise, but
few decimal fractions are.

You could add a link to
<URL:http://www.merlyn.demon.co.uk/js-misc0.htm#DW4which shows
exactly what one gets.

5.015*5 gives +5.07499999999999928945726423989981412887573242187 5 which
displays as "5.074999999999999".
Added locally.

--
Randy
comp.lang.javascript FAQ - http://jibbering.com/faq/index.html
FAQ Notes: http://www.jibbering.com/faq/faq_notes/faq_notes.html
ECMAScript Language Specification via FAQ2.6
Dec 17 '07 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Neculai Macarie | last post by:
Hi! Using Union and Order By gives strange behaviour in the following test-case: drop table if exists gallery; drop table if exists gallery_categ; # create test tables create table gallery...
1
by: FAQ server | last post by:
----------------------------------------------------------------------- FAQ Topic - Why does simple decimal arithmetic give strange results?...
2
by: FAQ server | last post by:
----------------------------------------------------------------------- FAQ Topic - Why does simple decimal arithmetic give strange results?...
1
by: FAQ server | last post by:
----------------------------------------------------------------------- FAQ Topic - Why does simple decimal arithmetic give strange results?...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.