By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
460,036 Members | 1,142 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 460,036 IT Pros & Developers. It's quick & easy.

A problem with decimals...

P: n/a
Try this code, and tell me how can I fix it, please.

var num1=new Number(parseFloat("118.18"))
var num2=new Number(parseFloat("50"))
var num3=new Number(parseFloat("50"))
alert(num1-num2-num3)

This is very strange...
--
Posted via Mailgate.ORG Server - http://www.Mailgate.ORG
Jul 20 '05 #1
Share this Question
Share on Google+
5 Replies


P: n/a
Cristian Martinello wrote:
Try this code, and tell me how can I fix it, please.

var num1=new Number(parseFloat("118.18"))
var num2=new Number(parseFloat("50"))
var num3=new Number(parseFloat("50"))
alert(num1-num2-num3)

This is very strange...

No its not. Javascript, like most programming languages, use floating
point arithmetic. Fractional numbers which are finite in one base may be
'repeating decimals' in another base. Floating point converts the
numbers to base 2, does the arithmetic, then converts the answer to base 10.

Rounding the answer to the precision of the operands is usually
sufficient. If not, convert the numbers to integers, perform the math,
then convert back.

Jul 20 '05 #2

P: n/a
"Cristian Martinello" <ca*********@tiscali.it> writes:
Try this code, and tell me how can I fix it, please.

var num1=new Number(parseFloat("118.18"))
var num2=new Number(parseFloat("50"))
var num3=new Number(parseFloat("50"))
alert(num1-num2-num3)
Just
alert(118.18-100)
will do the trick.
This is very strange...


This is completely normal when calculating with finite precission
binary-based floating point numbers.

The computer cannot represent 118.18 exactly. It finds a
representation that is close enough that when output, it is rounded to
118.18. Now you subtract 100 from it. That means that the integer part
of the number gets smaller, so there are more bits available for
the fractional part.

Imagine the number is stored as 15 decimal digits, but not exactly:
118.180000000001
When you output it, the least significant digit is rounded away.
Now subtract 100 and still use 15 decimal digits:
18.1800000000010
Suddently the error on the least significant bit is amplified, so
it won't go away by rounding.
The same problem happens with binary representation, typically when
making the number smaller.

<URL:http://jibbering.com/faq/#FAQ4_7>

/L
--
Lasse Reichstein Nielsen - lr*@hotpop.com
Art D'HTML: <URL:http://www.infimum.dk/HTML/randomArtSplit.html>
'Faith without judgement merely degrades the spirit divine.'
Jul 20 '05 #3

P: n/a
"Cristian Martinello" <ca*********@tiscali.it> wrote in message
news:7d*************************************@mygat e.mailgate.org...
Try this code, and tell me how can I fix it, please.

var num1=new Number(parseFloat("118.18"))
var num2=new Number(parseFloat("50"))
var num3=new Number(parseFloat("50"))
alert(num1-num2-num3)

This is very strange...


It is not strange, it is normal. It is a manifestation of the inability
of an IEEE double precision floating point number (as used by JavaScript
for its one number type) to accurately represent all fractions. What you
are seeing is the nearest approximation of 18.18 that the number format
can accommodate. It is not a problem unique to computer floating point
number formats, how would you write 1/3 as a decimal fraction?
(precisely?)

<URL: http://www.jibbering.com/faq/#FAQ4_7 >

What is strange is wrapping a - new Number - call around a -
parseFloat - call. You would get the same results from:-

var num1="118.18"
var num2="50"
var num3="50"
alert(num1-num2-num3)

- as the subtraction operator will force the required string to number
type conversion.

Richard.
Jul 20 '05 #4

P: n/a
A typical trick to solve that is the following:

var result=Math.round((num1-num2-num3)*10000)/10000;
alert(result)

that is, you round something that you also m,ultiplicate by some power of
ten 8depending on how many floats you want to keep) and then you divide
exactly by that same amount of power of ten out of the rounding process -
that fixes it in a way that many use.

http://www.unitedscripters.com/

"Cristian Martinello" <ca*********@tiscali.it> ha scritto nel messaggio
news:7d*************************************@mygat e.mailgate.org...
Try this code, and tell me how can I fix it, please.

var num1=new Number(parseFloat("118.18"))
var num2=new Number(parseFloat("50"))
var num3=new Number(parseFloat("50"))
alert(num1-num2-num3)

This is very strange...
--
Posted via Mailgate.ORG Server - http://www.Mailgate.ORG

Jul 20 '05 #5

P: n/a
TrueBlue wrote:
A typical trick to solve that is the following:

var result=Math.round((num1-num2-num3)*10000)/10000;
alert(result)

that is, you round something that you also m,ultiplicate by some power of
ten 8depending on how many floats you want to keep) and then you divide
exactly by that same amount of power of ten out of the rounding process -
that fixes it in a way that many use.
Where's the point? Math.round(num1 - num2 - num3) will also do the trick.
[Fullquote below the text]


Please don't do this, bandwidth is precious.
PointedEars

Jul 20 '05 #6

This discussion thread is closed

Replies have been disabled for this discussion.