By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
459,904 Members | 1,439 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 459,904 IT Pros & Developers. It's quick & easy.

FAQ Topic - Why does simple decimal arithmetic give strange results? (2007-12-16)

P: n/a
-----------------------------------------------------------------------
FAQ Topic - Why does simple decimal arithmetic give strange
results?
-----------------------------------------------------------------------

For example, 5*1.015 does not give exactly 5.075 and 0.06+0.01 does
not give exactly 0.07 in javascript.

Javascript numbers are represented in binary as IEEE-754 (IEC 559)
Doubles, with a resolution of 53 bits, giving an accuracy of
15-16 decimal digits; integers up to about 9e15 are precise, but
few decimal fractions are. Given this, arithmetic is as exact
as possible, but no more. Operations on integers are exact if
the true result and all intermediates are integers within that
range.

In particular, non-integer results should not normally be
compared for equality; and non-integer computed results
commonly need rounding; see 4.6.

http://msdn.microsoft.com/library/de...6dbf483b41.asp

Otherwise, use Math.round on the results of expressions which
should be of integer value.
--
Postings such as this are automatically sent once a day. Their
goal is to answer repeated questions, and to offer the content to
the community for continuous evaluation/improvement. The complete
comp.lang.javascript FAQ is at http://jibbering.com/faq/index.html.
The FAQ workers are a group of volunteers. The sendings of these
daily posts are proficiently hosted by http://www.pair.com.

Dec 16 '07 #1
Share this Question
Share on Google+
2 Replies


P: n/a
In comp.lang.javascript message <47***********************@news.sunsite.
dk>, Sun, 16 Dec 2007 00:00:01, FAQ server <ja********@dotinternet.be>
posted:
>
Javascript numbers are represented in binary as IEEE-754 (IEC 559)
Doubles, with a resolution of 53 bits, giving an accuracy of
15-16 decimal digits; integers up to about 9e15 are precise, but
few decimal fractions are.
You could add a link to
<URL:http://www.merlyn.demon.co.uk/js-misc0.htm#DW4which shows
exactly what one gets.

5.015*5 gives +5.07499999999999928945726423989981412887573242187 5 which
displays as "5.074999999999999".

--
(c) John Stockton, Surrey, UK. *@merlyn.demon.co.uk / ??*********@physics.org
Web <URL:http://www.merlyn.demon.co.uk/- FAQish topics, acronyms, & links.
Correct <= 4-line sig. separator as above, a line precisely "-- " (SoRFC1036)
Do not Mail News to me. Before a reply, quote with ">" or "" (SoRFC1036)
Dec 16 '07 #2

P: n/a
Dr J R Stockton said the following on 12/16/2007 6:25 PM:
In comp.lang.javascript message <47***********************@news.sunsite.
dk>, Sun, 16 Dec 2007 00:00:01, FAQ server <ja********@dotinternet.be>
posted:
>Javascript numbers are represented in binary as IEEE-754 (IEC 559)
Doubles, with a resolution of 53 bits, giving an accuracy of
15-16 decimal digits; integers up to about 9e15 are precise, but
few decimal fractions are.

You could add a link to
<URL:http://www.merlyn.demon.co.uk/js-misc0.htm#DW4which shows
exactly what one gets.

5.015*5 gives +5.07499999999999928945726423989981412887573242187 5 which
displays as "5.074999999999999".
Added locally.

--
Randy
comp.lang.javascript FAQ - http://jibbering.com/faq/index.html
FAQ Notes: http://www.jibbering.com/faq/faq_notes/faq_notes.html
ECMAScript Language Specification via FAQ2.6
Dec 17 '07 #3

This discussion thread is closed

Replies have been disabled for this discussion.