Steve Horsley wrote:

Ben O'Steen wrote: On Mon, October 31, 2005 10:23, Sybren Stuvel said: Ben O'Steen enlightened us with:

Using decimal as opposed to float sorts out this error as floats are

not built to handle the size of number used here.

They can handle the size just fine. What they can't handle is 1/1000th

precision when using numbers in the order of 1e10.

I used the word 'size' here incorrectly, I intended to mean 'length'

rather than numerical value. Sorry for the confusion :)

Sybren is right. The problem is not the length or the size, it's

the fact that 0.039 cannot be represented exactly in binary, in

just the same way that 1/3 cannot be represented exactly in

decimal. They both give recurring numbers. If you truncate those

recurring numbers to a finite number of digits, you lose

precision. And this shows up when you convert the inaccurate

number from binary into decimal representation where an exact

representation IS possible.

That's A source of error, but it's only part of the story. The

double-precision binary representation of 0.039 is 5620492334958379 *

2**(-57), which is in error by 1/18014398509481984000. By contrast,

Johnny Lee's answer is in error by 9/262144000, which is more than 618

billion times the error of simply representing 0.039 in floating point

-- a loss of 39 bits.

The problem here is catastrophic cancellation.

1130748744.500 ~= 4742703982051328 * 2**(-22)

1130748744.461 ~= 4742703981887750 * 2**(-22)

Subtracting gives 163578 * 2**(-22), which has only 18 significant bits.