By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
446,148 Members | 1,250 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 446,148 IT Pros & Developers. It's quick & easy.

floating-point precision

P: n/a
Hi guys,

I use LUTs to accelerate certain parts of my app. The LUTs contain
float numbers, and I generated the numbers very precisely, up to 10
decimals. All seems fine, however, when I printf the numbers in the
LUT, I do not get the same exact numbers. The first 6 decimals are ok,
but it got beserk after that. To illustrate further:

float myLUT = [0.123456789, 0.123456789, 0.123456789, ................] ;

printf("myLUT: .10f", myLUT[1]);

##### output

myLUT: 0.123456111
I am not quite sure why that is the case, although I believe float do
not have enough precision, so the compiler (VC 8) gives the closest
approximation that can be had with float. Is that so? Can somebody
enlighten me on that issue?

Thanks in advance,
Enrique

Mar 15 '07 #1
Share this Question
Share on Google+
6 Replies


P: n/a
Enrique Cruiz <jn*************@jetable.orgwrites:
I use LUTs to accelerate certain parts of my app. The LUTs contain
float numbers, and I generated the numbers very precisely, up to 10
decimals. All seems fine, however, when I printf the numbers in the
LUT, I do not get the same exact numbers. The first 6 decimals are ok,
but it got beserk after that. To illustrate further:

float myLUT = [0.123456789, 0.123456789, 0.123456789, ................] ;
float is only guaranteed to have 6 decimal digits worth of
accuracy. If you need more, use double, which has at least 10
decimal digits worth of accuracy. Also, read what the C FAQ has
to say about floating point numbers.
--
Peter Seebach on C99:
"[F]or the most part, features were added, not removed. This sounds
great until you try to carry a full-sized printout of the standard
around for a day."
Mar 15 '07 #2

P: n/a
Enrique Cruiz <jn*************@jetable.orgwrites:
I use LUTs to accelerate certain parts of my app. The LUTs contain
float numbers, and I generated the numbers very precisely, up to 10
decimals. All seems fine, however, when I printf the numbers in the
LUT, I do not get the same exact numbers. The first 6 decimals are ok,
but it got beserk after that. To illustrate further:

float myLUT = [0.123456789, 0.123456789, 0.123456789, ................] ;

printf("myLUT: .10f", myLUT[1]);
What the heck is a "LUT", and what language are you using? ("LookUp
Table", maybe?) The above is a syntax error in C, even if you replace
the "................" with something meaningful. It's probably some
compiler-specific extension.

Aside from that, type float just doesn't provide the precision you
need. Type double or long double might. Consult your system's
documentation.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
Mar 16 '07 #3

P: n/a
On Mar 15, 5:18 pm, Enrique Cruiz <jni6l03mdo6n...@jetable.orgwrote:
Hi guys,

I use LUTs to accelerate certain parts of my app. The LUTs contain
float numbers, and I generated the numbers very precisely, up to 10
decimals. All seems fine, however, when I printf the numbers in the
LUT, I do not get the same exact numbers. The first 6 decimals are ok,
but it got beserk after that. To illustrate further:

float myLUT = [0.123456789, 0.123456789, 0.123456789, ................] ;

printf("myLUT: .10f", myLUT[1]);

##### output

myLUT: 0.123456111

I am not quite sure why that is the case, although I believe float do
not have enough precision,
Why? Float is only guaranteed to have 6 digits of precision, which
is exactly what you are seeing.

If you need more than 6 digits of precision, then, use double
(And unless you have a reason to use float, use double).

Why do you think you need this precision? (Are you trying
to check the results using the LUT against those not
using a LUT?)

Have you checked to see if using the LUT actually does
increase execution speed? (It is quite possible for
the opposite to happen). Have you checked whether
the putative incease in speed is significant?

- William Hughes

Mar 16 '07 #4

P: n/a
Enrique Cruiz wrote:
>
I use LUTs to accelerate certain parts of my app. The LUTs contain
float numbers, and I generated the numbers very precisely, up to 10
decimals. All seems fine, however, when I printf the numbers in the
LUT, I do not get the same exact numbers. The first 6 decimals are
ok, but it got beserk after that. To illustrate further:

float myLUT = [0.123456789, 0.123456789, 0.123456789, ..........] ;

printf("myLUT: .10f", myLUT[1]);

##### output

myLUT: 0.123456111

I am not quite sure why that is the case, although I believe float
do not have enough precision, so the compiler (VC 8) gives the
closest approximation that can be had with float. Is that so? Can
somebody enlighten me on that issue?
Look in <float.hfor the number of digits available. Consider
using double rather than float.

--
Chuck F (cbfalconer at maineline dot net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net>
mm
--
Posted via a free Usenet account from http://www.teranews.com

Mar 16 '07 #5

P: n/a
William Hughes wrote:
>
.... snip ...
>
Have you checked to see if using the LUT actually does increase
execution speed? (It is quite possible for the opposite to
happen). Have you checked whether the putative incease in speed
is significant?
What is a LUT? A capitalized abbreviation for a lute, perhaps. If
so, who is playing it.

--
Chuck F (cbfalconer at maineline dot net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net>

--
Posted via a free Usenet account from http://www.teranews.com

Mar 16 '07 #6

P: n/a
CBFalconer <cb********@yahoo.comwrites:
What is a LUT?
Look-Up Table.
--
"I should killfile you where you stand, worthless human." --Kaz
Mar 16 '07 #7

This discussion thread is closed

Replies have been disabled for this discussion.