FAQ Topic - Why does simple decimal arithmetic give strange results? (2008-10-30)

[FAQ server]

-----------------------------------------------------------------------
FAQ Topic - Why does simple decimal arithmetic give strange
results?
-----------------------------------------------------------------------

For example, ` 5 * 1.015 ` does not give exactly
` 5.075 ` and ` 0.06+0.01 ` does
not give exactly ` 0.07 ` in javascript.

ECMAScript numbers are represented in binary as IEEE-754 (IEC 559)
Doubles, with a resolution of 53 bits, giving an accuracy of
15-16 decimal digits; integers up to about ` 9e15 ` are precise, but
few decimal fractions are. Given this, arithmetic is as exact
as possible, but no more. Operations on integers are exact if
the true result and all intermediates are integers within that
range.

In particular, non-integer results should not normally be
compared for equality; and non-integer computed results
commonly need rounding; see 4.6.

http://msdn2.microsoft.com/en-us/library/7wkd9z69.aspx

http://www.merlyn.demon.co.uk/js-misc0.htm#DW4

Otherwise, use ` Math.round ` on the results of expressions which
should be of integer value.
--
Postings such as this are automatically sent once a day. Their
goal is to answer repeated questions, and to offer the content to
the community for continuous evaluation/improvement. The complete
comp.lang.javascript FAQ is at http://jibbering.com/faq/index.html.
The FAQ workers are a group of volunteers. The sendings of these
daily posts are proficiently hosted by http://www.pair.com.

[Dr J R Stockton]

In comp.lang.javascript message <49***********************@news.sunsite.
dk>, Thu, 30 Oct 2008 00:00:03, FAQ server <ja********@dotinternet.be>
posted:

>ECMAScript numbers are represented in binary as IEEE-754 (IEC 559)
Doubles, with a resolution of 53 bits, giving an accuracy of
15-16 decimal digits; integers up to about ` 9e15 ` are precise, but

Within the line above, please change "about" to "just over".

>In particular, non-integer results should not normally be
compared for equality; and non-integer computed results
commonly need rounding; see 4.6.

The value of 4.6 has changed; the link should be to "How do I convert a
Number into a String with exactly 2 decimal places?".

And that section title is wrong, since the Number is not converted but
its value is used to determine the String, and the input can be an
expression (or a Boolean; but I doubt whether it is ever useful to give
/true/ to two decimal places) (or a suitable String). I suggest "How do
I show a value as a String with exactly 2 decimal places?".

--
(c) John Stockton, nr London UK. ?@merlyn.demon.co.uk Turnpike v6.05 MIME.
<URL:http://www.merlyn.demon.co.uk/TP/BP/Delphi/&c., FAQqy topics & links;
<URL:http://www.merlyn.demon.co.uk/clpb-faq.txt RAH Prins : c.l.p.b mFAQ;
<URL:ftp://garbo.uwasa.fi/pc/link/tsfaqp.zipTimo Salmi's Turbo Pascal FAQ.