473,322 Members | 1,307 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

What Algorithm Do Single and Double Use when Storing Imprecise Values?

I know that Single and Double can only store values that can be stored as
x/2^y, such as 0.5 and 0.125. But when they attempt to store values such as
0.1 or 0.2 whose exact value cannot be stored, what algorithm do they use to
determine what value to store? Thanks.
--
Nathan Sokalski
nj********@hotmail.com
http://www.nathansokalski.com/
Oct 19 '08 #1
6 1454
"Nathan Sokalski" <nj********@hotmail.comwrote in message
news:OL**************@TK2MSFTNGP03.phx.gbl...
>I know that Single and Double can only store values that can be stored as
x/2^y, such as 0.5 and 0.125. But when they attempt to store values such as
0.1 or 0.2 whose exact value cannot be stored, what algorithm do they use
to determine what value to store? Thanks.
They use a mantissa and an exponent, so that a number is expressed as
(mantissa)*10^(exponent). We do this frequently in everyday engineering work
when we say for instance, that something took "1.23*10^-6 seconds". However,
when speaking about Single and Double, the "10" has to be interpreted in
base 2 (meaning that it is a "2" in base 10). The exponent and mantissa are
also in base 2. The precission of the mantissa is limited, because the
mantissa and exponent and their signs have to fit into 4 bytes (Single) or 8
bytes (Double). The numbers that can be represented exactly are those that
can be converted into base 2 within that number of bits. For instance, 0.5
(base10) is 0.1 (base 2), but some numbers that have a small number of
decimals in base 10 have an infinite number of decimal places once converted
to base 2, so they will be truncated when assigned to the mantissa, and
therefore they will not be "exact".

Oct 19 '08 #2
"Nathan Sokalski" <nj********@hotmail.comschrieb
>I know that Single and Double can only store values that can be stored as
x/2^y, such as 0.5 and 0.125. But when they attempt to store values such as
0.1 or 0.2 whose exact value cannot be stored, what algorithm do they use
to determine what value to store? Thanks.
As the documentation
http://msdn.microsoft.com/en-us/libr...em.single.aspx
says, the types comply with IEEE 754 which also defines rounding rules.
Search for it; you'll find wikipedia etc.

Armin

Oct 19 '08 #3
"Nathan Sokalski" <nj********@hotmail.comwrote in message
news:OL**************@TK2MSFTNGP03.phx.gbl...
>I know that Single and Double can only store values that can be stored as
x/2^y, such as 0.5 and 0.125. But when they attempt to store values such as
0.1 or 0.2 whose exact value cannot be stored, what algorithm do they use
to determine what value to store? Thanks.
--
Nathan Sokalski
nj********@hotmail.com
http://www.nathansokalski.com/
What Every Computer Scientist Should Know About Floating-Point Arithmetic
http://docs.sun.com/source/806-3568/ncg_goldberg.html

Here's a PDF version
http://www.physics.ohio-state.edu/~d...point_math.pdf
Oct 19 '08 #4
Ken Halter wrote:
What Every Computer Scientist Should Know About Floating-Point
Arithmetic http://docs.sun.com/source/806-3568/ncg_goldberg.html

Here's a PDF version
http://www.physics.ohio-state.edu/~d...point_math.pdf

Interesting link, thanks. I can use such a .pdf from time to time. It
is amazing that CS people know so little about it.

--
Rudy Velthuis http://rvelthuis.de

"I once heard two ladies going on and on about the pains of
childbirth and how men don't seem to know what real pain is. I
asked if either of them ever got themselves caught in a zipper."
-- Emo Philips.
Oct 19 '08 #5
I understand how they are stored, that is very simple. My question was what
algorithm they use to determine what value to store. Let me rephrase what I
meant by this. When a value is assigned to a variable that cannot be stored
as an exact value, such as the following:

Dim x As Byte = 1.2

Since the exact value cannot be stored, what algorithm is used to determine
what value will be stored? Since the value is obviously not exact, it must
be rounded up or down, so how do I know which it is?
--
Nathan Sokalski
nj********@hotmail.com
http://www.nathansokalski.com/

"Alberto Poblacion" <ea******************************@poblacion.orgwro te
in message news:%2****************@TK2MSFTNGP02.phx.gbl...
"Nathan Sokalski" <nj********@hotmail.comwrote in message
news:OL**************@TK2MSFTNGP03.phx.gbl...
>>I know that Single and Double can only store values that can be stored as
x/2^y, such as 0.5 and 0.125. But when they attempt to store values such
as 0.1 or 0.2 whose exact value cannot be stored, what algorithm do they
use to determine what value to store? Thanks.

They use a mantissa and an exponent, so that a number is expressed as
(mantissa)*10^(exponent). We do this frequently in everyday engineering
work when we say for instance, that something took "1.23*10^-6 seconds".
However, when speaking about Single and Double, the "10" has to be
interpreted in base 2 (meaning that it is a "2" in base 10). The exponent
and mantissa are also in base 2. The precission of the mantissa is
limited, because the mantissa and exponent and their signs have to fit
into 4 bytes (Single) or 8 bytes (Double). The numbers that can be
represented exactly are those that can be converted into base 2 within
that number of bits. For instance, 0.5 (base10) is 0.1 (base 2), but some
numbers that have a small number of decimals in base 10 have an infinite
number of decimal places once converted to base 2, so they will be
truncated when assigned to the mantissa, and therefore they will not be
"exact".

Oct 20 '08 #6
Since the exact value cannot be stored, what algorithm is used to
determine what value will be stored? Since the value is obviously not
exact, it must be rounded up or down, so how do I know which it is?
But you've already been told: IEEE 754.

SteveT

Oct 20 '08 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: towers | last post by:
Hello, I've got a bit of experience in C++, but I'm writing my first app that is dependent on relatively precise math functions. The app requires that I get a time stamp based on s sample...
116
by: Dilip | last post by:
Recently in our code, I ran into a situation where were stuffing a float inside a double. The precision was extended automatically because of that. To make a long story short, this caused...
4
by: IceMan | last post by:
Hello I am getting some odd answers when i use doubles in c# 0.55-0.5 = 0.050000000000000044 why am i not getting 0.05
6
by: Nathan Sokalski | last post by:
I know that Single and Double can only store values that can be stored as x/2^y, such as 0.5 and 0.125. But when they attempt to store values such as 0.1 or 0.2 whose exact value cannot be stored,...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.