446,193 Members | 843 Online Need help? Post your question and get tips & solutions from a community of 446,193 IT Pros & Developers. It's quick & easy.

# Division precision issue

 P: n/a Hello all! I just stumbled across a weird problem with precision of a division operation. I am on Mac OS X, GCC 4.0.1. Say I have two float or double numbers, and I want to divide one by another, e.g. float A, B, C; C = A / B; The problem is that the result my program gives for two different A and B values are the same: 1.002323 / 11.025554 = 0.090909090909 1.036174 / 11.397911 = 0.090909090909 But with a calculator application, I get: 1.002323 / 11.025554 = 0.0909090826637827 1.036174 / 11.397911 = 0.0909091148369205 Does anyone have any suggestions as what I may be doing wrong? Is there any special division function I should be using? Thanks very much, Artemiy. Mar 8 '07 #1
10 Replies

 P: n/a Artemio wrote: Hello all! I just stumbled across a weird problem with precision of a division operation. I am on Mac OS X, GCC 4.0.1. Say I have two float or double numbers, and I want to divide one by another, e.g. float A, B, C; C = A / B; The problem is that the result my program gives for two different A and B values are the same: 1.002323 / 11.025554 = 0.090909090909 1.036174 / 11.397911 = 0.090909090909 But with a calculator application, I get: 1.002323 / 11.025554 = 0.0909090826637827 1.036174 / 11.397911 = 0.0909091148369205 Does anyone have any suggestions as what I may be doing wrong? Is there any special division function I should be using? How can you call this weird? You specify C float data type, defined in std C as supporting 6 or more equivalent decimal digits precision, then you compare to a calculator which has 15 decimal digits precision. YOur comparison should be much closer if you use C double data type. But, if your data are accurate only to 6 or 7 decimal digits, you may not care how digits beyond those are generated. Mar 8 '07 #2

 P: n/a Well but with double the results are exactly the same... Mar 8 '07 #3

 P: n/a Artemio wrote: Well but with double the results are exactly the same... Show code. -- Chris "electric hedgehog" Dollin "It was the first really clever thing the King had said that day." /Alice in Wonderland/ Mar 8 '07 #4

 P: n/a Artemio wrote: Well but with double the results are exactly the same... Generally calculator programs use platform specific types or multiprecision libraries to provide a high level of precision. Plain C has a long double type which might be sufficient for most purposes, but it makes little sense to compare it with specialised mathematical programs. PS. Please quote the message to which you reply. Some people download new messages and read, and respond, to them offline. This becomes difficult when messages are not fairly standalone, in terms of context. Mar 8 '07 #5

 P: n/a Okay guys, sorry for not being more clear with the issue. Let's say I have this code: double A, B; printf("%0.12f / %0.12f = %0.12f\n", A, B, A / B ); For two different A and B it gives: 1.002323075899 / 11.025553834888 = 0.090909090909 and 1.048065059110 / 11.528715650212 = 0.090909090909 E.g. 1/11 and 1/11.5 are not that precise as you guess but the result is the same! How in the world can that be? :-( Artemiy. On Mar 8, 5:51 pm, "santosh"

 P: n/a Artemio wrote: Okay guys, sorry for not being more clear with the issue. Let's say I have this code: double A, B; printf("%0.12f / %0.12f = %0.12f\n", A, B, A / B ); For two different A and B it gives: 1.002323075899 / 11.025553834888 = 0.090909090909 and 1.048065059110 / 11.528715650212 = 0.090909090909 E.g. 1/11 and 1/11.5 are not that precise as you guess but the result is the same! How in the world can that be? :-( muntyan@munt10:~\$ calc C-style arbitrary precision calculator (version 2.12.1.5) Calc is open software. For license details type: help copyright [Type "exit" to exit, or "help" for help.] ; 1.002323075899 / 11.025553834888 ~0.09090909090909915440 ; 1.048065059110 / 11.528715650212 ~0.09090909090907513819 ; 1.002323075899*11.528715650212 - 1.048065059110*11.025553834888 ~0.00000000000305271121 So maybe results are actually good and your estimates aren't? Best regards, Yevgen Mar 8 '07 #7

 P: n/a "Artemio" int main() { double a = 1.002323075899; double b = 11.025553834888; double c = a / b; double x = 1.048065059110; double y = 11.528715650212; double z = x / y; printf("a == %.20f\n" "b == %.20f\n" "a / b == %.20f\n\n" "x == %.20f\n" "y == %.20f\n" "x / y == %.20f\n", a, b, c, x, y, z); return 0; } Output (MSVC 2005 Express, Pentium 4): a == 1.00232307589900000000 b == 11.02555383488799900000 a / b == 0.09090909090909916900 x == 1.04806505911000000000 y == 11.52871565021200000000 x / y == 0.09090909090907514600 "What Every Computer Scientist Should Know About Floating-Point Arithmetic": http://docs.sun.com/source/806-3568/ncg_goldberg.html -Mike Mar 8 '07 #8

 P: n/a Thanks very much Mike, I'll give that one a thorough read! Artemiy. On Mar 8, 7:46 pm, "Mike Wahler" int main() { double a = 1.002323075899; double b = 11.025553834888; double c = a / b; double x = 1.048065059110; double y = 11.528715650212; double z = x / y; printf("a == %.20f\n" "b == %.20f\n" "a / b == %.20f\n\n" "x == %.20f\n" "y == %.20f\n" "x / y == %.20f\n", a, b, c, x, y, z); return 0; } Output (MSVC 2005 Express, Pentium 4): a == 1.00232307589900000000 b == 11.02555383488799900000 a / b == 0.09090909090909916900 x == 1.04806505911000000000 y == 11.52871565021200000000 x / y == 0.09090909090907514600 "What Every Computer Scientist Should Know About Floating-Point Arithmetic":http://docs.sun.com/source/806-3568/ncg_goldberg.html -Mike Mar 8 '07 #9

 P: n/a Artemio wrote: Say I have two float or double numbers, and I want to divide one by another, e.g. float A, B, C; C = A / B; The problem is that the result my program gives for two different A and B values are the same: 1.002323 / 11.025554 = 0.090909090909 1.036174 / 11.397911 = 0.090909090909 But with a calculator application, I get: 1.002323 / 11.025554 = 0.0909090826637827 1.036174 / 11.397911 = 0.0909091148369205 Does anyone have any suggestions as what I may be doing wrong? The problem is loss of precision, but the important question is where. The actual results are 6 and 5 significant digits, worse than expected for most single precision floating point implementations. You get much closer to the results indicated earlier with 1.002323 / 11.025553 1.036174 / 11.397914 I suspect this is due to rounding error with limited precision of conversion on the denominators. Try increasing the precision of the conversion and storing all values in double, or long double, if the results you get above is not accurate enough for your application. -- Thad Mar 9 '07 #10

 P: n/a Artemio wrote: > Thanks very much Mike, I'll give that one a thorough read! You have already been asked to correct the top-posting. Do so, or you are likely to be ignored by many here. Your answer belongs after (or intermixed with) the *snipped* material to which you reply. The snipping removes anything not germane to that reply. See the following links: -- Some informative links:

### This discussion thread is closed

Replies have been disabled for this discussion. 