460,036 Members | 1,142 Online
Need help? Post your question and get tips & solutions from a community of 460,036 IT Pros & Developers. It's quick & easy.

# A problem with decimals...

 P: n/a Try this code, and tell me how can I fix it, please. var num1=new Number(parseFloat("118.18")) var num2=new Number(parseFloat("50")) var num3=new Number(parseFloat("50")) alert(num1-num2-num3) This is very strange... -- Posted via Mailgate.ORG Server - http://www.Mailgate.ORG Jul 20 '05 #1
5 Replies

 P: n/a Cristian Martinello wrote: Try this code, and tell me how can I fix it, please. var num1=new Number(parseFloat("118.18")) var num2=new Number(parseFloat("50")) var num3=new Number(parseFloat("50")) alert(num1-num2-num3) This is very strange... No its not. Javascript, like most programming languages, use floating point arithmetic. Fractional numbers which are finite in one base may be 'repeating decimals' in another base. Floating point converts the numbers to base 2, does the arithmetic, then converts the answer to base 10. Rounding the answer to the precision of the operands is usually sufficient. If not, convert the numbers to integers, perform the math, then convert back. Jul 20 '05 #2

 P: n/a "Cristian Martinello" writes: Try this code, and tell me how can I fix it, please. var num1=new Number(parseFloat("118.18")) var num2=new Number(parseFloat("50")) var num3=new Number(parseFloat("50")) alert(num1-num2-num3) Just alert(118.18-100) will do the trick. This is very strange... This is completely normal when calculating with finite precission binary-based floating point numbers. The computer cannot represent 118.18 exactly. It finds a representation that is close enough that when output, it is rounded to 118.18. Now you subtract 100 from it. That means that the integer part of the number gets smaller, so there are more bits available for the fractional part. Imagine the number is stored as 15 decimal digits, but not exactly: 118.180000000001 When you output it, the least significant digit is rounded away. Now subtract 100 and still use 15 decimal digits: 18.1800000000010 Suddently the error on the least significant bit is amplified, so it won't go away by rounding. The same problem happens with binary representation, typically when making the number smaller. /L -- Lasse Reichstein Nielsen - lr*@hotpop.com Art D'HTML: 'Faith without judgement merely degrades the spirit divine.' Jul 20 '05 #3

 P: n/a "Cristian Martinello" wrote in message news:7d*************************************@mygat e.mailgate.org... Try this code, and tell me how can I fix it, please. var num1=new Number(parseFloat("118.18")) var num2=new Number(parseFloat("50")) var num3=new Number(parseFloat("50")) alert(num1-num2-num3) This is very strange... It is not strange, it is normal. It is a manifestation of the inability of an IEEE double precision floating point number (as used by JavaScript for its one number type) to accurately represent all fractions. What you are seeing is the nearest approximation of 18.18 that the number format can accommodate. It is not a problem unique to computer floating point number formats, how would you write 1/3 as a decimal fraction? (precisely?) What is strange is wrapping a - new Number - call around a - parseFloat - call. You would get the same results from:- var num1="118.18" var num2="50" var num3="50" alert(num1-num2-num3) - as the subtraction operator will force the required string to number type conversion. Richard. Jul 20 '05 #4

 P: n/a A typical trick to solve that is the following: var result=Math.round((num1-num2-num3)*10000)/10000; alert(result) that is, you round something that you also m,ultiplicate by some power of ten 8depending on how many floats you want to keep) and then you divide exactly by that same amount of power of ten out of the rounding process - that fixes it in a way that many use. http://www.unitedscripters.com/ "Cristian Martinello" ha scritto nel messaggio news:7d*************************************@mygat e.mailgate.org... Try this code, and tell me how can I fix it, please. var num1=new Number(parseFloat("118.18")) var num2=new Number(parseFloat("50")) var num3=new Number(parseFloat("50")) alert(num1-num2-num3) This is very strange... -- Posted via Mailgate.ORG Server - http://www.Mailgate.ORG Jul 20 '05 #5

 P: n/a TrueBlue wrote: A typical trick to solve that is the following: var result=Math.round((num1-num2-num3)*10000)/10000; alert(result) that is, you round something that you also m,ultiplicate by some power of ten 8depending on how many floats you want to keep) and then you divide exactly by that same amount of power of ten out of the rounding process - that fixes it in a way that many use. Where's the point? Math.round(num1 - num2 - num3) will also do the trick. [Fullquote below the text] Please don't do this, bandwidth is precious. PointedEars Jul 20 '05 #6

### This discussion thread is closed

Replies have been disabled for this discussion.