Javascript Floating Point Precision Problem

Hello everyone! I'm new here but have been programming for the web in various languages for 15 years or so. I'm certainly no "expert" but can keep myself out of trouble (or in it?) most of the time.

This particular problem has plagued me for years; it is making me very, very, old. :-( It deals with the way Javascript's method of floating point precision takes the simplest math calculations and steals a penny - in the example below, a simple 636.35 is turned into 636.34 (636.34999999999999 !)

I know the truncate function below is not only lame, it turns the number into a string - but that is not the issue, as you can see the problem exists before truncate gets to it, and persists even if it is taken out of the script. I won't bore you will all the various methods I've tried, the most obvious being multiply everything by 100 then divide by 100 at the end - all failed

PLEASE can someone paste and test this code, offer a solution? Just load it, hit calculate, watch the alerts. I'm turning gray by the minute. :-( THANK YOU! -- Bill

Expand|Select|Wrap|Line Numbers

 
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<html>

<head>

    <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">

    <title>Javascript Floating Point Precision Problem</title>

</head>
 
<body>

<h2>Javascript Floating Point Precision Problem</h2>

<form name="po_form" id="po_form" onSubmit="return false;">

<input type="hidden" name="number_of_items" id="number_of_items" value="3">

<table>

 <tr><td><input type="text" name="pn_1" id="pn_1" size="15" maxlength="20" value="3H00NT"  tabindex="18"></td>

     <td> <input type="text" name="description_1" id="description_1" size="18" maxlength="255" value="Fuser Roller for HP4200DT" tabindex="19"></td>

     <td><input type="text" onChange="calcForm(this.form);" name="quantity_1" id="quantity_1" size="2" maxlength="12" value="1" tabindex="21"></td>

     <td> $ <input type="text" onChange="calcForm(this.form);" name="price_1" id="price_1" size="6" maxlength="12" value="540.80" tabindex="22"></td>

     <td> $ <input type="text" onChange="calcForm(this.form);" name="shipping_1" id="shipping_1" size="6" maxlength="12" value="0.00" tabindex="23"></td>

     <td> <strong>$</strong> <input type="text" name="subtotal_1" id="subtotal_1" onFocus="blur();" size="6" maxlength="12" value="540.80" tabindex="5018"></td>

 </tr>

 <tr><td><input type="text" name="pn_2" id="pn_2" size="15" maxlength="20" value="3H0006"  tabindex="24"></td>

  <td> <input type="text" name="description_2" id="description_2" size="18" maxlength="255" value="Feed Roller" tabindex="25"></td>

  <td><input type="text" onChange="calcForm(this.form);" name="quantity_2" id="quantity_2" size="2" maxlength="12" value="1" tabindex="27"></td>

  <td> $ <input type="text" onChange="calcForm(this.form);" name="price_2" id="price_2" size="6" maxlength="12" value="48.75" tabindex="28"></td>

  <td> $ <input type="text" onChange="calcForm(this.form);" name="shipping_2" id="shipping_2" size="6" maxlength="12" value="0.00" tabindex="29"></td>

  <td> <strong>$</strong> <input type="text" name="subtotal_2" id="subtotal_2" onFocus="blur();" size="6" maxlength="12" value="48.75" tabindex="5024"></td>

 </tr>

 <tr><td><input type="text" name="pn_3" id="pn_3" size="15" maxlength="20" value="3H007B"  tabindex="30"></td>

   <td> <input type="text" name="description_3" id="description_3" size="18" maxlength="255" value="Roller" tabindex="31"></td>

   <td><input type="text" onChange="calcForm(this.form);" name="quantity_3" id="quantity_3" size="2" maxlength="12" value="1" tabindex="33"></td>

   <td> $ <input type="text" onChange="calcForm(this.form);" name="price_3" id="price_3" size="6" maxlength="12" value="46.80" tabindex="34"></td>

   <td> $ <input type="text" onChange="calcForm(this.form);" name="shipping_3" id="shipping_3" size="6" maxlength="12" value="0.00" tabindex="35"></td>

   <td> <strong>$</strong> <input type="text" name="subtotal_3" id="subtotal_3" onFocus="blur();" size="6" maxlength="12" value="46.80" tabindex="5030"></td>

 </tr>

 <tr>

  <td colspan="4">&nbsp;</td>

  <td> TOTAL:</td>

  <td> <strong>$</strong> <input type="text" name="po_total" id="po_total" onFocus="blur();" size="6" maxlength="12" value="636.34" tabindex="5500"> </td>

 </tr>

 <tr><td colspan="6"> <hr width="100%" size="1"></td></tr>

 <tr>

  <td><input type="submit" name="addLineItem" id="addLineItem" value="Add New Item" tabindex="2001" onClick="alert('Different function, ignore.'); return false;"></td>

  <td> &nbsp; </td>

  <td colspan="4">

    <input type="button" id="recalcButton" onClick="calcForm(this.form);" value="Recalculate Total" tabindex="2002">

    <input type="submit" name="submitButton" id="submitButton" onClick="checkForm(this.form); return false;" value="Submit PO" tabindex="2003">

   </td>

</tr>

</table>

</form>
 
       <script type="text/javascript">
 
            function calcForm (form) {
 
                   var subtotal=quan=price=ship=total=0;

                   var sub,cancelled;

                   var tot       = document.getElementById('po_total');

                   var num_items = document.getElementById('number_of_items').value;
 
                   for (i=1;i<=num_items;i++) {

                      subtotal = 0;
 
                      sub   = document.getElementById('subtotal_' + i);

                      ship  = document.getElementById('shipping_' + i);

                      q     = document.getElementById('quantity_' + i);

                      p     = document.getElementById('price_' + i);

                      subtotal = (q.value * p.value) + (ship.value * 1);

                      total += subtotal;

                      alert(subtotal + ' ' + total);

                      sub.value = truncate(subtotal);
 
                   }

                   tot.value = truncate(total);

            }
 
            function checkForm (form) {

                 calcForm(form);

                 alert('various checks, then submit form.');

            }
 
            function truncate(num) {

                 string = "" + num;

                 if (string.indexOf('.') == -1)

                 return string + '.00';

                 seperation = string.length - string.indexOf('.');

                 if (seperation > 3)

                 return string.substring(0,string.length-seperation+3);

                 else if (seperation == 2)

                 return string + '0';

                 return string;

            }

          </script>
 
</body>

</html>

Aug 1 '06 #1

Subscribe Post Reply

13055

iam_clint

1,208

Expert 1GB

This might make you mad my very simple solution if this isn't what your looking for let me know and i will give you another solution!

num.toFixed(2)
:)

Expand|Select|Wrap|Line Numbers

 
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<html>

<head>

    <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">

    <title>Javascript Floating Point Precision Problem</title>

</head>
 
<body>

<h2>Javascript Floating Point Precision Problem</h2>

<form name="po_form" id="po_form" onSubmit="return false;">

<input type="hidden" name="number_of_items" id="number_of_items" value="3">

<table>

 <tr><td><input type="text" name="pn_1" id="pn_1" size="15" maxlength="20" value="3H00NT"  tabindex="18"></td>

     <td> <input type="text" name="description_1" id="description_1" size="18" maxlength="255" value="Fuser Roller for HP4200DT" tabindex="19"></td>

     <td><input type="text" onChange="calcForm(this.form);" name="quantity_1" id="quantity_1" size="2" maxlength="12" value="1" tabindex="21"></td>

     <td> $ <input type="text" onChange="calcForm(this.form);" name="price_1" id="price_1" size="6" maxlength="12" value="540.80" tabindex="22"></td>

     <td> $ <input type="text" onChange="calcForm(this.form);" name="shipping_1" id="shipping_1" size="6" maxlength="12" value="0.00" tabindex="23"></td>

     <td> <strong>$</strong> <input type="text" name="subtotal_1" id="subtotal_1" onFocus="blur();" size="6" maxlength="12" value="540.80" tabindex="5018"></td>

 </tr>

 <tr><td><input type="text" name="pn_2" id="pn_2" size="15" maxlength="20" value="3H0006"  tabindex="24"></td>

  <td> <input type="text" name="description_2" id="description_2" size="18" maxlength="255" value="Feed Roller" tabindex="25"></td>

  <td><input type="text" onChange="calcForm(this.form);" name="quantity_2" id="quantity_2" size="2" maxlength="12" value="1" tabindex="27"></td>

  <td> $ <input type="text" onChange="calcForm(this.form);" name="price_2" id="price_2" size="6" maxlength="12" value="48.75" tabindex="28"></td>

  <td> $ <input type="text" onChange="calcForm(this.form);" name="shipping_2" id="shipping_2" size="6" maxlength="12" value="0.00" tabindex="29"></td>

  <td> <strong>$</strong> <input type="text" name="subtotal_2" id="subtotal_2" onFocus="blur();" size="6" maxlength="12" value="48.75" tabindex="5024"></td>

 </tr>

 <tr><td><input type="text" name="pn_3" id="pn_3" size="15" maxlength="20" value="3H007B"  tabindex="30"></td>

   <td> <input type="text" name="description_3" id="description_3" size="18" maxlength="255" value="Roller" tabindex="31"></td>

   <td><input type="text" onChange="calcForm(this.form);" name="quantity_3" id="quantity_3" size="2" maxlength="12" value="1" tabindex="33"></td>

   <td> $ <input type="text" onChange="calcForm(this.form);" name="price_3" id="price_3" size="6" maxlength="12" value="46.80" tabindex="34"></td>

   <td> $ <input type="text" onChange="calcForm(this.form);" name="shipping_3" id="shipping_3" size="6" maxlength="12" value="0.00" tabindex="35"></td>

   <td> <strong>$</strong> <input type="text" name="subtotal_3" id="subtotal_3" onFocus="blur();" size="6" maxlength="12" value="46.80" tabindex="5030"></td>

 </tr>

 <tr>

  <td colspan="4">&nbsp;</td>

  <td> TOTAL:</td>

  <td> <strong>$</strong> <input type="text" name="po_total" id="po_total" onFocus="blur();" size="6" maxlength="12" value="636.34" tabindex="5500"> </td>

 </tr>

 <tr><td colspan="6"> <hr width="100%" size="1"></td></tr>

 <tr>

  <td><input type="submit" name="addLineItem" id="addLineItem" value="Add New Item" tabindex="2001" onClick="alert('Different function, ignore.'); return false;"></td>

  <td> &nbsp; </td>

  <td colspan="4">

    <input type="button" id="recalcButton" onClick="calcForm(this.form);" value="Recalculate Total" tabindex="2002">

    <input type="submit" name="submitButton" id="submitButton" onClick="checkForm(this.form); return false;" value="Submit PO" tabindex="2003">

   </td>

</tr>

</table>

</form>
 
       <script type="text/javascript">
 
            function calcForm (form) {
 
                   var subtotal=quan=price=ship=total=0;

                   var sub,cancelled;

                   var tot       = document.getElementById('po_total');

                   var num_items = document.getElementById('number_of_items').value;
 
                   for (i=1;i<=num_items;i++) {

                      subtotal = 0;
 
                      sub   = document.getElementById('subtotal_' + i);

                      ship  = document.getElementById('shipping_' + i);

                      q     = document.getElementById('quantity_' + i);

                      p     = document.getElementById('price_' + i);

                      subtotal = (q.value * p.value) + (ship.value * 1);

                      total += subtotal;

                      alert(subtotal + ' ' + total);

                      sub.value = truncate(subtotal);
 
                   }

                   tot.value = truncate(total);

            }
 
            function checkForm (form) {

                 calcForm(form);

                 alert('various checks, then submit form.');

            }
 
            function truncate(num) {

                 string = num.toFixed(2);

                 return string;

            }

          </script>
 
</body>

</html>

Aug 1 '06 #2

rocknbil

Clint, if anything on the Internet makes one mad, it's time to back away from the computer! :-) Thanks for your reply!

Funny you should mention toFixed() - I had discovered it through other means (today.) But the problem with the numbers exists long before it gets to truncate() - in fact, by using toFixed() I was able to eliminate that stupid function altogether. Hereis my revised Javascript in its entirety, notice that in the alerts it still displays 636.349999999999 but that toFixed() properly rounds it up to .35. WOO HOO!

Many thanks for looking, i have been fighting this Javascript "feature" off and on for years . . . . .

Expand|Select|Wrap|Line Numbers

 
       <script type="text/javascript">
 
            function calcForm (form) {
 
                   var subtotal,q,p,ship;

                   var total = 0;
 
                   var sub,cancelled;

                   var tot       = document.getElementById('po_total');

                   var num_items = document.getElementById('number_of_items').value;
 
                   for (i=1;i<=num_items;i++) {

                      subtotal = 0;

                      sub = document.getElementById('subtotal_' + i);

                      ship = document.getElementById('shipping_' + i).value;

                      q = document.getElementById('quantity_' + i).value;

                      p = document.getElementById('price_' + i).value;
 
                      subtotal = (q * p) + (ship * 1);

                      total += (subtotal);

                      alert(subtotal + ' ' + total);

                      sub.value = subtotal.toFixed(2); 
 
                   }

                   total = total.toFixed(2);

                   tot.value = total;

            }
 
            function checkForm (form) {

                 calcForm(form);

                 alert('various checks, then submit form.');

            }

          </script>

Aug 2 '06 #3

iam_clint

1,208

Expert 1GB

Yeah i didn't think about what you really said i just got the proper number to display but yes for your answer .toFixed(2) is probably the best way to fix your problem -- i didn't really worry about the alert... all you gotta do is look what i did and change the variable that the alert is coming from and do the toFixed(2) but glad you got it working if you have any more question give us a post.

Aug 2 '06 #4

Banfa

9,065

Expert Mod 8TB

This isn't actually a Javascript feature but a feature of the way the IEEE define how floating point numbers are stored in memory. Any language using this definaition suffers from this "feature", for instance C does as well and since javascript is often implemented in C so does Javascript.

The problem is that Javascript is using 32 bits of data which have 4294967296 different combinations to hold any value in the range

1.7976931348623158e+308 to 2.2250738585072014e–308

It does this by using a smaller presision value and using some of the bits as a exponent (this can also be done in 16 bits with a smaller range and less presision) which results in it being able to approximate any value in the range but not exactly represent them all (because in real number terms the are an infinaite number of values between any 2 given values).

As part of your calculcation clearly the internal representation of the value is going outside the available presision (15 digits for a 32 bit number but only 6 digits for a 16 digit number so perhaps Javascript uses 16 bit floating point values) and you are ending up with an approximation to the value instead of an exact value.

We all have to code round this.

Sometimes it is easier (particularly with money) when you don't really need the huge range of a floating point value to write your code so that internally it deals with cents (or pence) and only converts to dollars (pounds) for display to the user.

This results in all your calculations being performed in integer arithmatic which does not have the imprecisions of floating point arrithmatic. Of course this wont work if you are anticipating many orders greater than $42,949,672.9 :D

Aug 2 '06 #5

iam_clint

1,208

Expert 1GB

LOL thanks for the explanation Banfa, i wasn't completely clear on toFixed but now I am. I just knew of several times i have used it to fix the same problem so thats what i gave him.

Aug 2 '06 #6

by: Dave | last post by:

Hi folks, I am trying to develop a routine that will handle sphere-sphere and sphere-triangle collisions and interactions. My aim is to develop a quake style collision engine where a player can...

C / C++

small numerical differences in floating point result between wintel and Sun/SPARC

by: JS | last post by:

We have the same floating point intensive C++ program that runs on Windows on Intel chip and on Sun Solaris on SPARC chips. The program reads the exactly the same input files on the two platforms....

C / C++

Floating Point Bug in Visual Studio 2003

by: Anton Noll | last post by:

We are using Visual Studio 2003.NET (C++) for the development of our software in the fields digital signal processing and numerical acoustics. One of our programs was working correctly if we are...

C / C++

687

why still use C?

by: cody | last post by:

no this is no trollposting and please don't get it wrong but iam very curious why people still use C instead of other languages especially C++. i heard people say C++ is slower than C but i can't...

C / C++

fractions and floating point

by: Steffen | last post by:

Hi, is it possible to have two fractions, which (mathematically) have the order a/b < c/d (a,b,c,d integers), but when (correctly) converted into floating point representation just have the...

C / C++

Another problem about javascript number!

by: chanma | last post by:

code1:var x=0xf0000000; alert(x); output:4026531840 code2:var b=0xf<<28; alert(b); output:-268435456

Javascript

Floating Point Exception

by: Shhnwz.a | last post by:

Hi, I have a problem regarding handling floating point error, specially Denormalisation error. I want to know is there any trick or technique to find bit length of the resultant before using and...

C / C++

137

Multi precision floating point

by: mathieu.dutour | last post by:

Dear all, I want to do multiprecision floating point, i.e. I want to go beyond single precision, double precision and have quadruple precision, octuple precision and the like, and possibly with...

C / C++

Trouble with integer floating point conversion

by: rembremading | last post by:

Hi all! The following piece of code has (for me) completely unexpected behaviour. (I compile it with gcc-Version 4.0.3) Something goes wrong with the integer to float conversion. Maybe somebody...

C / C++

Benefits of Blockchain App Development for Enterprise Businesses

by: Kemmylinns12 | last post by:

Blockchain technology has emerged as a transformative force in the business world, offering unprecedented opportunities for innovation and efficiency. While initially associated with cryptocurrencies...

General

Basic concepts of WebLogic Admin training?

by: Naresh1 | last post by:

What is WebLogic Admin Training? WebLogic Admin Training is a specialized program designed to equip individuals with the skills and knowledge required to effectively administer and manage Oracle...

Oracle Database

How to layout php header location with a variable as the location?

by: Matthew3360 | last post by:

Hi there. I have been struggling to find out how to use a variable as my location in my header redirect function. Here is my code. header("Location:".$urlback); Is this the right layout the...

PHP

How to use http requests (Get function) to get variable value?

by: Matthew3360 | last post by:

Hi, I have a python app that i want to be able to get variables from a php page on my webserver. My python app is on my computer. How would I make it so the python app could use a http request to get...

Python

HOW CAN I CREATE AN AI with an .executable file that would suck all files in the folder and on my computer

by: AndyPSV | last post by:

HOW CAN I CREATE AN AI with an .executable file that would suck all files in the folder and on my computerHOW CAN I CREATE AN AI with an .executable file that would suck all files in the folder and...

General

by: WisdomUfot | last post by:

It's an interesting question you've got about how Gmail hides the HTTP referrer when a link in an email is clicked. While I don't have the specific technical details, Gmail likely implements measures...

General

PHP Curl Not wanting to connect to localhost.

by: Matthew3360 | last post by:

Hi, I have been trying to connect to a local host using php curl. But I am finding it hard to do this. I am doing the curl get request from my web server and have made sure to enable curl. I get a...

PHP

FPGA Question

by: Oralloy | last post by:

Hello Folks, I am trying to hook up a CPU which I designed using SystemC to I/O pins on an FPGA. My problem (spelled failure) is with the synthesis of my design into a bitstream, not the C++...

General

Join our open community on Discord to learn programming, SEO, and marketing

by: BLUEPANDA | last post by:

At BluePanda Dev, we're passionate about building high-quality software and sharing our knowledge with the community. That's why we've created a SaaS starter kit that's not only easy to use but also...

General

Javascript Floating Point Precision Problem

Similar topics