floating-point precision

[Enrique Cruiz]

Hi guys,

I use LUTs to accelerate certain parts of my app. The LUTs contain
float numbers, and I generated the numbers very precisely, up to 10
decimals. All seems fine, however, when I printf the numbers in the
LUT, I do not get the same exact numbers. The first 6 decimals are ok,
but it got beserk after that. To illustrate further:

float myLUT = [0.123456789, 0.123456789, 0.123456789, ................] ;

printf("myLUT: .10f", myLUT[1]);

##### output

myLUT: 0.123456111
I am not quite sure why that is the case, although I believe float do
not have enough precision, so the compiler (VC 8) gives the closest
approximation that can be had with float. Is that so? Can somebody
enlighten me on that issue?

Thanks in advance,
Enrique

[Ben Pfaff]

Enrique Cruiz <jn*************@jetable.orgwrites:

I use LUTs to accelerate certain parts of my app. The LUTs contain
float numbers, and I generated the numbers very precisely, up to 10
decimals. All seems fine, however, when I printf the numbers in the
LUT, I do not get the same exact numbers. The first 6 decimals are ok,
but it got beserk after that. To illustrate further:

float myLUT = [0.123456789, 0.123456789, 0.123456789, ................] ;

float is only guaranteed to have 6 decimal digits worth of
accuracy. If you need more, use double, which has at least 10
decimal digits worth of accuracy. Also, read what the C FAQ has
to say about floating point numbers.
--
Peter Seebach on C99:
"[F]or the most part, features were added, not removed. This sounds
great until you try to carry a full-sized printout of the standard
around for a day."

[Keith Thompson]

Enrique Cruiz <jn*************@jetable.orgwrites:

I use LUTs to accelerate certain parts of my app. The LUTs contain
float numbers, and I generated the numbers very precisely, up to 10
decimals. All seems fine, however, when I printf the numbers in the
LUT, I do not get the same exact numbers. The first 6 decimals are ok,
but it got beserk after that. To illustrate further:

float myLUT = [0.123456789, 0.123456789, 0.123456789, ................] ;

printf("myLUT: .10f", myLUT[1]);

What the heck is a "LUT", and what language are you using? ("LookUp
Table", maybe?) The above is a syntax error in C, even if you replace
the "................" with something meaningful. It's probably some
compiler-specific extension.

Aside from that, type float just doesn't provide the precision you
need. Type double or long double might. Consult your system's
documentation.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

[William Hughes]

On Mar 15, 5:18 pm, Enrique Cruiz <[e-mail]:

Hi guys,

I use LUTs to accelerate certain parts of my app. The LUTs contain
float numbers, and I generated the numbers very precisely, up to 10
decimals. All seems fine, however, when I printf the numbers in the
LUT, I do not get the same exact numbers. The first 6 decimals are ok,
but it got beserk after that. To illustrate further:

float myLUT = [0.123456789, 0.123456789, 0.123456789, ................] ;

printf("myLUT: .10f", myLUT[1]);

##### output

myLUT: 0.123456111

I am not quite sure why that is the case, although I believe float do
not have enough precision,

Why? Float is only guaranteed to have 6 digits of precision, which
is exactly what you are seeing.

If you need more than 6 digits of precision, then, use double
(And unless you have a reason to use float, use double).

Why do you think you need this precision? (Are you trying
to check the results using the LUT against those not
using a LUT?)

Have you checked to see if using the LUT actually does
increase execution speed? (It is quite possible for
the opposite to happen). Have you checked whether
the putative incease in speed is significant?

- William Hughes

[CBFalconer]

Enrique Cruiz wrote:

>
I use LUTs to accelerate certain parts of my app. The LUTs contain
float numbers, and I generated the numbers very precisely, up to 10
decimals. All seems fine, however, when I printf the numbers in the
LUT, I do not get the same exact numbers. The first 6 decimals are
ok, but it got beserk after that. To illustrate further:

float myLUT = [0.123456789, 0.123456789, 0.123456789, ..........] ;

printf("myLUT: .10f", myLUT[1]);

##### output

myLUT: 0.123456111

I am not quite sure why that is the case, although I believe float
do not have enough precision, so the compiler (VC 8) gives the
closest approximation that can be had with float. Is that so? Can
somebody enlighten me on that issue?

Look in <float.hfor the number of digits available. Consider
using double rather than float.

--
Chuck F (cbfalconer at maineline dot net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net>
mm
--
Posted via a free Usenet account from http://www.teranews.com

[CBFalconer]

William Hughes wrote:

>

.... snip ...

>
Have you checked to see if using the LUT actually does increase
execution speed? (It is quite possible for the opposite to
happen). Have you checked whether the putative incease in speed
is significant?

What is a LUT? A capitalized abbreviation for a lute, perhaps. If
so, who is playing it.

--
Chuck F (cbfalconer at maineline dot net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net>

--
Posted via a free Usenet account from http://www.teranews.com

[Ben Pfaff]

CBFalconer <cb********@yahoo.comwrites:

What is a LUT?

Look-Up Table.
--
"I should killfile you where you stand, worthless human." --Kaz