473,883 Members | 2,212 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Why does 28.08 show up as 28.080000000000 002 in DataGridView

I'm baffled. I have a column in a SQL Server Express database called
"Longitude, " which is a float. When I view the table in a DataGridView,
some of the numbers, which only have two decimal places in the database
show up with *15* decimal places and are ever so slightly off (in the
example in the subject line, by about 2E-15).

I'm not doing any operations on this column. It's just running a stored
procedure which performs a pretty basic SELECT on the table. If I run
the stored procedure in Management Studio Express, all numbers show up
fine (just two decimal places).

What's going on here?

Thanks,

-Dan

Jan 9 '07
130 6667
Jon Skeet [C# MVP] <sk***@pobox.co mwrote:

<snip>

Having failed to sleep properly due to nuances about this kind of thing
floating (no pun intended) round my head, I think it's probably best
for everyone if I sign off from the thread at this point. That's not in
any way meant to try to tell people not to reply to the posts I made a
short while ago - I'm not trying to get the last word.

I've enjoyed the debate, and I hope you all have too. Back to bed for a
couple of hours, hopefully to get some rest...

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Jan 10 '07 #21
Hi guys,

It is very interesting, but I see in this long thread not any question from
a VB.Net guy/girl, they know this probably already, so will you next time
disconnect the not relevant newsgroups as you recognise this.

Thanks in advance,

Cor
Jan 10 '07 #22

"Jon Skeet [C# MVP]" <sk***@pobox.co mwrote in message
news:MP******** **************@ msnews.microsof t.com...
Mike C# <xy*@xyz.comwro te:
>"Jon Skeet [C# MVP]" <sk***@pobox.co mwrote in message
The SQL Decimal and Numeric data types are designed to store every single
last number in the range of valid values in an *EXACT* representation.

1/3 is in the range, but can't be stored exactly. Which part of that
statement do you disagree with?
Just as an example: Consider the SQL NUMERIC(2, 1) data type. 1/3 is
*absolutely not* a member of the set of valid values for this type. The
complete set of valid values for this type is:

{ -9.9, -9.8, -9.7, -9.6, -9.5, -9.4, -9.3, -9.2, -9.1, -9.0, -8.9, -8.8, -8.7,
-8.6, -8.5, -8.4, -8.3, -8.2, -8.1, -8.0, -7.9, -7.8, -7.7, -7.6, -7.5, -7.4,
-7.3, -7.2, -7.1, -7.0, -6.9, -6.8, -6.7, -6.6, -6.5, -6.4, -6.3, -6.2, -6.1,
-6.0, -5.9, -5.8, -5.7, -5.6, -5.5, -5.4, -5.3, -5.2, -5.1, -5.0, -4.9, -4.8,
-4.7, -4.6, -4.5, -4.4, -4.3, -4.2, -4.1, -4.0, -3.9, -3.8, -3.7, -3.6, -3.5,
-3.4, -3.3, -3.2, -3.1, -3.0, -2.9, -2.8, -2.7, -2.6, -2.5, -2.4, -2.3, -2.2,
-2.1, -2.0, -1.9, -1.8, -1.7, -1.6, -1.5, -1.4, -1.3, -1.2, -1.1, -1.0, -0.9,
-0.8, -0.7, -0.6, -0.5, -0.4, -0.3, -0.2, -0.1, 0.0, 0.1, 0.2, 0.3, 0.4,
0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9,
2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9, 3.0, 3.1, 3.2, 3.3, 3.4,
3.5, 3.6, 3.7, 3.8, 3.9, 4.0, 4.1, 4.2, 4.3, 4.4, 4.5, 4.6, 4.7, 4.8, 4.9,
5.0, 5.1, 5.2, 5.3, 5.4, 5.5, 5.6, 5.7, 5.8, 5.9, 6.0, 6.1, 6.2, 6.3, 6.4,
6.5, 6.6, 6.7, 6.8, 6.9, 7.0, 7.1, 7.2, 7.3, 7.4, 7.5, 7.6, 7.7, 7.8, 7.9,
8.0, 8.1, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7, 8.8, 8.9, 9.0, 9.1, 9.2, 9.3, 9.4,
9.5, 9.6, 9.7, 9.8, 9.9 }

Likewise with NUMERIC (10, 2), DECIMAL (24, 18), or any other valid
precision and scale settings for the NUMERIC and DECIMAL types.

The fact that you cannot accurately store a value that is not a member of
the set of valid values for a type is of no significance. Since by
definition 1/3 is not a member of the set of valid values for the DECIMAL or
NUMERIC types, why do you find it surprising that 1/3 cannot be stored
exactly using these types?
>Are we talking about information loss from the simple act of converting
from
one base to another, which is what the Decimal and Numeric types address;
or
are you talking about information loss from performing operations on data
which goes back to your 1/3 example earlier?

They're the same thing. Converting 0.1 base 3 (i.e. 1/3) into a decimal
*is* a conversion from one base to another, and the Decimal and Numeric
types *do not* store that number exactly.
No, they are two different arguments. As explained, 0.1 is a non-repeating
decimal, whereas 1/3 is an infinitely repeating decimal which cannot be
represented exactly in decimal form, period. The fact that an infinitely
repeating decimal cannot be represented *exactly* using a DECIMAL or NUMERIC
type should come as no surprise to you or anyone else for that matter. The
fact that a non-repeating decimal with a scale of 1 or 2 (just one or two
digits after the decimal point) cannot be stored *exactly* using FLOAT and
REAL data types comes as a surprise to many people.
>You need to present your arguments about this to ANSI, since these are
the
definitions they put forth.

I'm happy to disagree with any number of people, if what they're saying
doesn't make sense.
That's perfectly fine; however, as long as the standard calls INT, DECIMAL,
NUMERIC, etc., "exact" numeric data types and it calls FLOAT and REAL
"approximat e" numeric data types, you may as well prepare yourself to
disagree with every SQL database programmer and designer in the world.
>They can store *every single valid value in their defined set of values*
in
an exact representation. With Numeric and Decimal types precision and
scale
that you specify define the valid set of values.

In that case the same can be said for Float. Every Float value is an
exact value - you can write down the *exact* binary string it
represents, and even convert it into an exact decimal representation.
Sounds great! Prove it: Store 28.08 as a FLOAT value and write doen the
*exact* binary string it represents. Then convert it into an exact decimal
representation. Here's something to get you started:

DECLARE @r FLOAT
SELECT @r = 28.08
SELECT @r

--Result in QA: 28.079999999999 998
>You might want to re-read that, because you seem to have picked out just
enough of a single sentence to support your argument, but completely
ignored
the part that doesn't. Here's the full sentence for you (the emphasis is
mine):

"WHEN MAXIMUM PRECISION IS USED, valid values are from - 10^38 +1 through
10^38 - 1."

That also defines a default scale of "0". If, however, you define a
different precision and/or scale, the valid set of values changes.

I don't think that makes any difference - when maximum precision is
used, the range of valid values is given in terms of a minimum and a
maximum, and not every value in that range is exactly representable. If
you're going to treat it as "every value that can be exactly
represented can be exactly represented" then as I say, the same is true
for Float.
Of course you don't think it makes any difference, or you would have quoted
the full quote instead of just the last part that appears to substantiate
your argument. The point is that when the precision and scale are turned
down the set of valid values changes for a given NUMERIC or DECIMAL data
type declaration. See the example above for NUMERIC (2, 1) for an example.
The range of valid values for NUMERIC (2, 1) is nowhere near -10^38 + 1 nor
+10^38 - 1. The minimum and maximum values in this case are -9.9 and +9.9,
and the set of valid values includes only decimal numbers that consist of
two digits where one of those digits is after the decimal point. Your
example of 1/3 does not fit in that set since it has more than 1 digit after
the decimal point (in fact, it has infinitely more digits after the decimal
point).
>Of course I'm biased to decimal numbers, as are most of the people here
who
were born with 10 fingers and 10 toes. Now let's talk about your bias
against irrational numbers, since you seem to be tip-toeing all around
them
here.

No, I'm recognising the mathematical reality that the existence (and
value) of a number doesn't depend on what base you represent it in.
The mathematical reality is that very few people through the course of human
history have needed to use Base 3. The reality is that NUMERIC and DECIMAL
types do not represent every single number "in existence", nor every single
number between the minimum and maximum values specified by the precision and
scale set by the user. The real reality is that the user defines the set of
*valid values* as well as implicit upper and lower bounds when they declare
a DECIMAL or NUMERIC type. The reality is that every single *valid value*
in this set can be stored *exactly*; that is, you get out of it exactly what
you put into it.
>There is no "fraction" data type in SQL, which you might consider a
shortcoming of the language. There is also no "fraction" data type built
into C#, C++, or most other languages. The reason for that might be a
huge
bias or conspiracy against vulgar fractions, or it might be the lack of a
useful vinculum in the original ASCII character set, or who it might just
be
that most people just don't find it all that useful.

I wasn't talking about usefulness, I was talking about existence.
Suggesting that 1/3 isn't a number is silly, IMO.
1/3 is a number, and I for one did not say otherwise. Now let me tell you
what is really silly (IMO):

- Suggesting that because you can't store an exact representation of the
infinitely repeating fraction 1/3 in a NUMERIC or DECIMAL (or any data type
for that matter) makes it inexact is ludicrous. As mentioned above, 1/3 is
not defined as a *valid value* for any NUMERIC or DECIMAL precision and
scale.

- Saying that trying to store the infinitely repeating decimal number 1/3
and getting less than an infinitely repeating decimal number is *the same
thing* as storing the non-repeating decimal number 1/10 and getting
something other than 0.1 back as a result is absolutely ridiculous.

Maybe a couple of examples will help, but let's get away from numbers and
bases for a minute and talk about character data (so we don't have to worry
about my decimal bias, or your love for "Base 3"):

0. Consider the VARCHAR data type on SQL 2000. It maxes out at 8,000
characters. You absolutely cannot store an infinite number of 'A'
characters in a VARCHAR(8000); therefore it is not exact by your reasoning.
By your reasoning, the only "exact" representation would be one in which you
could store every possible combination of characters ever, in an infinitely
long character string; even though a character string with 9,000 characters
is not a member of the set of valid values for a VARCHAR(8000). Likewise
you cannot store an infinitely repeating decimal (like 1/3) which, by your
reasoning, makes the NUMERIC and DECIMAL data types "inexact" as well;
despite the fact that 1/3 is not a member of the set of *valid values*
defined for the type.

1. Consider an imaginary data type similar to VARCHAR. We'll call it
APPROXIMATE VARCHAR. The APPROXIMATE VARCHAR will store basically the
strings we pass it; however, it may lose some accuracy in the least
significant characters (right-hand side of the string) due to the way it is
stored. If you were to store 8000 'A' characters in an APPROXIMATE
VARCHAR(8000), but then select the value back you might get 7,999 'A'
characters and one 'B' character. This might be no big deal if you really
only ever need to use the first 10 characters anyway, which appears to be
IEEE's rational behind their representation of floating point numbers. The
analogy is, of course, storing 28.08 in a FLOAT and retrieving 28.07999...
or 28.080000002 or some such number that *approximates* the exact value you
stored originally.

By your reasoning, both of these situations are the same: i.e., trying to
store an infinite number of characters in a data type that has a
well-defined limitation on the number of characters that can be stored *is
the same thing* as storing an approximation of the character string you pass
in. This is pretty silly, IMO.
>Regardless of the reason, 1/10 is not a repeating decimal in decimal
(yes,
I'm showing my "bias" for decimal again). It has a finite representation
in
decimal. 1/3 is a repeating decimal in decimal. Since it's decimal
representati on repeats infinitely, it's impossible to store it exactly
anywhere. However, with regards to Decimal and Numeric data types in
SQL,
"0.33333" is not a member of the set of valid values for a DECIMAL (10,
2)
or a NUMERIC (10, 3) [for instance].

Indeed.
Glad we agree on something.
>The definitions of SQL DECIMAL and NUMERIC specifically allow you to
define
the precision and scale. By doing so you define the number of total
digits
and the number of digits after the decimal point. Any numbers that are
in
the set of valid values for the precision and scale that you define will
be
stored *exactly*. Any numbers that are not in the set of valid values
for
that precision and scale will not be stored exactly. SQL REAL and FLOAT
types do not come with this guarantee.

There's no reason why they shouldn't - why would it not be able to
guarantees tha the number is essentially reproducible, which is all I
can understand that we're talking about? The binary string "0.010101"
represents an exact number and no others.
SQL REAL and FLOAT types are based on the IEEE standard which guarantees
that it will store an approximation of a value you feed to it. Since we're
dealing with binary computers (as opposed to analog computers) we can expect
that once our exact value is converted to an approximation the internal
representation *of that approximation* will be exact (1's and 0's).
However, the exact value you feed it is still converted to an approximation,
as shown by the OP. NUMERIC and DECIMAL types do not suffer from this
approximation. They store only values from the set of *valid values*
defined for their precision and scale, and they store them exactly. In your
example of 1/3, storing it as a NUMERIC (2, 1) results in "0.3" being
stored, and that rounding occurs during casting, not during the storage
operation. Basically it is attempting to cast 1/3, which is not a member of
the valid set of numbers for NUMERIC(2, 1), to a valid member of that set.
It has nothing to do with the exactness with which "0.3" is actually stored;
whereas that is the entire issue with FLOAT and REAL.
>Quite simply because with NUMERIC and DECIMAL types I can define the
precision and scale, and every valid value in that range will be stored
*EXACTLY*. REAL and FLOAT come with pre-defined ranges, but have no
guarantees concerning how the exactness with which your numeric data will
be
stored. As we saw, even a number as simple as 28.08 is stored as an
approximatio n of 28.08 (or 2808/100 if you prefer your fractions).
Whereas
with a NUMERIC (10, 2) it would be stored as -- well, "28.08" *EXACTLY*.

Do you regard 28.08 as in the "range" for FLOAT? It seems unfair to
demand that it is while saying that 1/3 isn't in the "RANGE" for
NUMERIC, but that's what you've basically got to do in order to claim
that FLOAT is "approximat e" but DECIMAL is "exact".
Life is not fair; however, your argument here is lacking.

FLOAT is a floating point type and 28.08 is a non-repeating decimal. The
fact that FLOAT cannot accurately store 28.08 exactly but DECIMAL and
NUMERIC can proves the point very well.

DECIMAL and NUMERIC are fixed-point types with a guarantee of storage
"exactness" . As mentioned, you define the precision (total number of
digits) and scale (number of digits after the decimal point) for DECIMAL and
NUMERIC, and in the process you implicitly define the upper and lower
bounds, as well as the set of *valid values* for the DECIMAL and NUMERIC
types. The guarantee is that any value that is a member of the set of
*valid values* for those types will be stored exactly. If you want to store
1/3 as DECIMAL or NUMERIC you have to cast it (implicitly or explicitly) to
a member of the set of *valid values* defined for the precision and scale
you have chosen. 1/3 is not a *valid value* from the set of *valid values*
for any NUMERIC or DECIMAL type (since they all have a finite scale).
Consider a DECIMAL (5, 4). You can store only 4 digits after the decimal
point; therefore you cannot store an infinite number of 3's after the
decimal point. Therefore 1/3 is not in the set of valid values for that
type (see previous for example).

Personally I don't find your argument about not being able to store an exact
representation of an infinitely repeating decimal on machines with a finite
amount of memory to be all that compelling a reason to call DECIMAL and
NUMERIC data types "inexact".
Jan 10 '07 #23

"Jon Skeet [C# MVP]" <sk***@pobox.co mwrote in message
news:MP******** *************** *@msnews.micros oft.com...
Mike C# <xy*@xyz.comwro te:
>"Jon Skeet [C# MVP]" <sk***@pobox.co mwrote in message
news:MP******* *************** **@msnews.micro soft.com...
The "approximat e" nature of binary floating point isn't because it's
floating point, and isn't changed by using a decimal format. It's just
the inability to exactly represent all *decimal* numbers, which isn't
the same thing as all numbers.

That seems like hair-splitting, for real.

I see nothing hair-splitting about recognising that 1/3 is a number.
Listen to some of your arguments:

Argument A
========
0. "All numeric data types are *approximate* and none are *exact* because
you can't represent every single combination, including infinitely repeating
combinations, using them."

By extension (and we may need to enter the realm of philosophy here), your
argument says there is no such thing as an exact data type of *any kind*
since you can't represent every combination of whatever their base type is
using them; including infinitely repeating series. VARCHAR(8000) is now
*approximate* because I can't store 8,001 'Z' characters in it. INT is
*approximate* because I can't store 999999999999999 999999999999999 999999999
in it. Personally I find that silly, IMO. An exact data type is not
*exact* based on being able to store every infinite combination; it is exact
because the valid values it does store are stored using *exact*
representations instead of approximations.

Argument B
========
0. "I type in the FLOAT value 10.0,"
1. "The computer stores it as an approximate value of 9.999999,"
2. "The binary representation of 9.999999 is *exact*,"
3. "Therefore FLOAT is an *exact* data types."

Really...
>People in this newsgroup refer to binary floating point numbers as
"approximate " because that's what the ANSI SQL standard calls them.

You may notice that there are multiple newsgroups involved here...
Yes I did notice, and I'm responding from the sqlserver newsgroup, just FYI.
SQL folks call floating point numbers "approximat e" because that's what the
standard says.
>"The data types NUMERIC, DECIMAL, INTEGER, and SMALLINT are collectively
referred to as exact numeric types. The data types FLOAT, REAL, and
DOUBLE
PRECISION are collectively referred to as approximate numeric types.
Exact
numeric types and approximate numeric types are collectively referred to
as
numeric types. Values of numeric type are referred to as numbers." --
ANSI
SQL-92

Just because ANSI says something doesn't mean it's right, IMO.
I agree with that. However, they did just fine with this one. Unless you
can find a better argument than "You can't store an infinitely repeating
decimal exactly in a fixed-decimal point type, on a machine with a finite
amount of memory, therefore the fixed-decimal point type is not an *exact*
type", you're not going to convince ANSI or just about anyone else of the
justness of your cause.
Jan 10 '07 #24
Mike C# wrote:
[...] The fact that an infinitely
repeating decimal cannot be represented *exactly* using a DECIMAL or NUMERIC
type should come as no surprise to you or anyone else for that matter. The
fact that a non-repeating decimal with a scale of 1 or 2 (just one or two
digits after the decimal point) cannot be stored *exactly* using FLOAT and
REAL data types comes as a surprise to many people. [...]
I agree with both statements above. But you have to consider that the
number 0.2, for example (as well as 28.08), is an "infinitely repeating
binary number" (although is not an "infinitely repeating decimal").

The representation of 0.2 in base 2 is:
0.0011001100110 011...
For easier reading, the same thing in base-16 (hexadecimal), would be:
0.3333...

The representation of 0.08 in base 2 is:
0.0001010001111 0101110 000101000111101 01110 000101000111101 01110 ...
For easier reading, in hex that would be:
0.147AE147AE147 AE...

This may come as surprise to many people, but if you really think about
it, it is normal that some numbers which can be represented in base-10
with a finit number of digits after the decimal point, will be
represented in base-2 with an infinite number of digits. Obviously,
these numbers cannot be accurately stored in SQL using a float or real
data type (which uses base 2).

To avoid problems when displaying such a number, in the conversion from
base-2 back to base-10, the last bits are sometimes ignored (because
they cannot form an entire decimal digit anyway) and the result
sometimes happens to be the expected result (e.g. 0.2), but not always.
Razvan

Jan 10 '07 #25

"Razvan Socol" <rs****@gmail.c omwrote in message
news:11******** **************@ p59g2000hsd.goo glegroups.com.. .
Mike C# wrote:
I agree with both statements above. But you have to consider that the
number 0.2, for example (as well as 28.08), is an "infinitely repeating
binary number" (although is not an "infinitely repeating decimal").
Absolutely.
This may come as surprise to many people, but if you really think about
it, it is normal that some numbers which can be represented in base-10
with a finit number of digits after the decimal point, will be
represented in base-2 with an infinite number of digits. Obviously,
these numbers cannot be accurately stored in SQL using a float or real
data type (which uses base 2).
That's exactly the point I was discussing with Mr. Skeet. The
"approximat ion" comes into play precisely because of the conversion from
base-10 to base-2 in the FLOAT and REAL types, whereas with DECIMAL and
NUMERIC types presumably use some form of Binary Coded Decimal (possibly
packed BCD or some similar algorithm) to represent the number, which
prevents data loss during the storage process.
Jan 10 '07 #26
Mike C# <xy*@xyz.comwro te:

<snip>
Just because ANSI says something doesn't mean it's right, IMO.

I agree with that. However, they did just fine with this one. Unless you
can find a better argument than "You can't store an infinitely repeating
decimal exactly in a fixed-decimal point type, on a machine with a finite
amount of memory, therefore the fixed-decimal point type is not an *exact*
type", you're not going to convince ANSI or just about anyone else of the
justness of your cause.
Just *one* point I can't resist here: I've never claimed that decimal
isn't exact. I've claimed that float is exact too.

Once you've got a float value, you can always convert it to an exact
decimal string too, and indeed I've got C# code to do it, referenced
from
http://www.pobox.com/~skeet/csharp/floatingpoint.html
There's also an online version where you can type in the decimal value
you want to find the closest double to, and then show the exact value
of the double. (These are C# floats and doubles, but the principle is
the same). For example, the closest double to 28.08 is *exactly*
28.079999999999 998294697434175 759553909301757 8125

My point (which I thought I'd made again and again, but clearly not
enough) is that it's the conversion from a decimal literal to a
floating point value which is an "approximat e" conversion, *not* the
float value itself. The point about base 3 is that the same would be
true - convert a base 3 number to a decimal and you'd end up with an
exact value which is the closest decimal to that number, but that may
not be exactly the same as the original value. (eg 1/3 becoming 0.333).
In both cases the conversion is "lossy" but the resultant value is
exact.

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Jan 10 '07 #27
Just *one* point I can't resist here: I've never claimed that decimal
isn't exact. I've claimed that float is exact too.
The problem is, if I hard-code a value like 28.08, and store it in a FLOAT,
when I ask for that value back, I may not get that EXACT value. It doesn't
matter to me if that approximation occurred during conversion, or
translation, or storage, or divine intervention. The value I thought I had
sent to the database has been changed on me.

A
Jan 10 '07 #28
Aaron Bertrand [SQL Server MVP] <te*****@dnartr eb.noraawrote:
Just *one* point I can't resist here: I've never claimed that decimal
isn't exact. I've claimed that float is exact too.

The problem is, if I hard-code a value like 28.08, and store it in a FLOAT,
when I ask for that value back, I may not get that EXACT value.
Indeed - and that's why you want to use DECIMAL in such a situation.
I've never denied the usefulness of DECIMAL in the slightest.
It doesn't matter to me if that approximation occurred during
conversion, or translation, or storage, or divine intervention. The
value I thought I had sent to the database has been changed on me.
It may not matter to you, but accuracy in description of types matters
to me. It's a bit like people claiming that "objects are passed by
reference by default" in C#, or that "value types live on the stack and
reference types live on the heap" - both statements sound okay on first
hearing, but dig a little deeper and they're inaccurate. I like to be
accurate from the start (where I can, at least). Why claim that it's
the type which is approximate rather than the conversion? Where's the
benefit?

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too
Jan 10 '07 #29

"Jon Skeet [C# MVP]" <sk***@pobox.co mwrote in message
news:MP******** *************** *@msnews.micros oft.com...
Mike C# <xy*@xyz.comwro te:

<snip>
Just because ANSI says something doesn't mean it's right, IMO.

I agree with that. However, they did just fine with this one. Unless
you
can find a better argument than "You can't store an infinitely repeating
decimal exactly in a fixed-decimal point type, on a machine with a finite
amount of memory, therefore the fixed-decimal point type is not an
*exact*
type", you're not going to convince ANSI or just about anyone else of the
justness of your cause.

Just *one* point I can't resist here: I've never claimed that decimal
isn't exact. I've claimed that float is exact too.
I pointed this out in Argument B above: You enter 10.0, the computer
converts it to an approximate FLOAT base-2 representation of "9.999999" and
stores that value. Since "9.999999" can be stored "exactly" using the IEEE
floating point standard, it follows that the FLOAT data type must be exact
too.

Hogwash. The reason it is "approximat e" is because it is converted to an
Base-2 approximate representation since the exact number can't be
represented in Base-2. The reason the other types are exact is because they
don't convert to Base-2 but instead store the decimal digits in a packed
format so no conversion to a different base needs to take place.
Once you've got a float value, you can always convert it to an exact
decimal string too, and indeed I've got C# code to do it, referenced
from
http://www.pobox.com/~skeet/csharp/floatingpoint.html
There's also an online version where you can type in the decimal value
you want to find the closest double to, and then show the exact value
of the double. (These are C# floats and doubles, but the principle is
the same). For example, the closest double to 28.08 is *exactly*
28.079999999999 998294697434175 759553909301757 8125
But 28.079999999999 998294697434175 759553909301757 8125 is not *EXACTLY*
28.08. It is an approximation. Hence the word "approximat e".

Perhaps not surprisingly, the closest NUMERIC(4, 2) to 28.08 is *EXACTLY*
28.08! And the *EXACT* string representation of the NUMERIC(4, 2) value
28.08 is *EXACTLY* 28.08.
My point (which I thought I'd made again and again, but clearly not
enough) is that it's the conversion from a decimal literal to a
floating point value which is an "approximat e" conversion, *not* the
float value itself. The point about base 3 is that the same would be
true - convert a base 3 number to a decimal and you'd end up with an
exact value which is the closest decimal to that number, but that may
not be exactly the same as the original value. (eg 1/3 becoming 0.333).
In both cases the conversion is "lossy" but the resultant value is
exact.
"You'd end up with an exact value which is the closest decimal to that
number". Again you're talking about base conversions here, which is good as
it happens to be the root of the problem with FLOAT. However, DECIMAL and
NUMERIC types were introduced because sometimes - that is to say every once
in a while - you actually want the EXACT value back that you originally fed
into the computer.

Every once in a while when you deposit $100 in the bank, you might want to
later take $100 out. You obviously can't do that if the bank "approximat es"
the value of your deposit down to $92 every time you deposit. It would be
interesting to watch a bank manager explain to his customer that $92 is the
EXACT value of their deposit since it is an EXACT number, even though it's
not EXACTLY the amount they deposited. After all 92 is an EXACT number, it
also happens to be EXACTLY *not* 100...

The loss of information from conversion to a different base is what ANSI had
in mind when they applied the terms "approximat e" and "exact". The loss of
information from truncating an infinitely repeating decimal is neither here
nor there no matter what media you are storing data on. You can expect
truncation under any circumstances in which you have an inifinitely long
repeating decimal and a finite amount of storage - whether it's computer
memory or a college-ruled notebook. The fact that FLOAT can't represent
many simple non-repeating decimals without corruption is what makes it
"approximat e". I can't say it any more plainly than that.
Jan 10 '07 #30

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

79
5514
by: Daniel Manes | last post by:
I'm baffled. I have a column in a SQL Server Express database called "Longitude," which is a float. When I view the table in a DataGridView, some of the numbers, which only have two decimal places in the database show up with *15* decimal places and are ever so slightly off (in the example in the subject line, by about 2E-15). I'm not doing any operations on this column. It's just running a stored procedure which performs a pretty basic...
0
9798
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10766
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10863
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
10422
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
9588
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
7136
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
6007
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4622
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
3241
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.