473,883 Members | 2,141 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Why does 28.08 show up as 28.080000000000 002 in DataGridView

I'm baffled. I have a column in a SQL Server Express database called
"Longitude, " which is a float. When I view the table in a DataGridView,
some of the numbers, which only have two decimal places in the database
show up with *15* decimal places and are ever so slightly off (in the
example in the subject line, by about 2E-15).

I'm not doing any operations on this column. It's just running a stored
procedure which performs a pretty basic SELECT on the table. If I run
the stored procedure in Management Studio Express, all numbers show up
fine (just two decimal places).

What's going on here?

Thanks,

-Dan

Jan 9 '07
130 6668
I think Mr. Skeet has an excellent idea in wrapping up this thread. As
such, I'd like to provide an opposing view to the question which Mr. Skeet
implied early on; in a nutshell: "Why are FLOAT and REAL called
'approximate' types while INT, DECIMAL, etc., types called 'exact'?"

I believe that the following simple definition follows the intent of the
standards put forth by ANSI (SQL), Microsoft (C#, VB, etc.) and other
standards bodies in referring to data types based on the IEEE floating
standards as "approximat e" and other data types as "exact":

"Exact data types are those data types which can store and retrieve single
values which conform to the following rules without loss of data, and
without the introduction of error:

1) The value must be composed of digits/characters that are defined as
valid, in the proper format defined for the data type,
2) For numeric types, the scale (digits after the decimal point) of the
value needs to be less than or equal to the scale of the data type,
3) For numeric types, the difference between precision (total number of
digits) and scale of the value needs to be less than or equal to the
difference between the precision and scale of the data type,
4) For numeric types, the value must be greater than or equal to the lower
bound defined for the data type, and less than or equal to the upper bound
defined for the data type,
5) For non-numeric types, the length of the value must be less than or
equal to the length of the data type"

This definition follows the intent of the standards bodies, including their
specification of floating point literals as Base 10. This definition also
accounts for truncation and rounding errors in irrational numbers and
infinitely repeating decimal values since their scale is infinitely beyond
what can be accurately represented by any built-in finite SQL, C#, or VB
types.

While Mr. Skeet also expressed an issue with calling FLOAT and REAL data
types "approximat e", these labels are entirely accurate, as they succinctly
express the position that when you try to store a numeric literal that
follows the above rules, an "approximat ion" of that value may be stored
instead of the "exact" value.

Like Mr. Skeet, I too feel this thread has run its course, and also feel
that arguments about items that have no real world applications or relevance
(quite possibly why they were left outside the scope of the adopted
standards: i.e., "Base 3", etc.) has no real bearing on the issues at hand.
This has been an interesting discussion despite having to deal with side
issues that have no basis in reality, and the ignoring of accepted standards
as well as the intent of the standards bodies in defining these types and in
labeling them as "approximat e" or "exact".
Jan 13 '07 #131

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

79
5514
by: Daniel Manes | last post by:
I'm baffled. I have a column in a SQL Server Express database called "Longitude," which is a float. When I view the table in a DataGridView, some of the numbers, which only have two decimal places in the database show up with *15* decimal places and are ever so slightly off (in the example in the subject line, by about 2E-15). I'm not doing any operations on this column. It's just running a stored procedure which performs a pretty basic...
0
9798
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10767
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
9588
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
7980
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
7137
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5807
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
6008
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4623
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
3242
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.