473,387 Members | 1,700 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,387 software developers and data experts.

Is this a bug: 2.2 - 0.4 = 1.8 but 1.2 - 0.4 = 0.8000001 ?

Hi,

Dim a, b As Single
a = 2.2
b = 0.4
Response.Write(a - b)

This gives: 1,8 (rendered according my regional settings)
But this gives: 0,8000001 (rendered according my regional settings):
Dim a, b As Single
a = 1.2
b = 0.4
Response.Write(a - b)
Any explantion for this, and how to solve this?? ( i rounded on two digits)
Thanks
Dave
Jul 15 '06 #1
4 1068
Dave,
Try:

| Dim a, b As Decimal
| a = 2.2D
| b = 0.4D
| Response.Write(a - b)

| But this gives: 0,8000001 (rendered according my regional settings):
Is given as that is the approximate value that a Single can hold. A Single
holds base 2 floating point numbers, some (most) base 10 floating point
numbers cannot be exactly represented

For details see:

http://www.yoda.arachsys.com/csharp/floatingpoint.html

http://www.yoda.arachsys.com/csharp/decimal.html
NOTE: The D in 2.2D indicates it is a Decimal literal as opposed to 2.2
which is a Double literal, as opposed to 2.2F which is a Single literal.
--
Hope this helps
Jay B. Harlow [MVP - Outlook]
..NET Application Architect, Enthusiast, & Evangelist
T.S. Bradley - http://www.tsbradley.net
"Dave" <sd***@fgn.qcwrote in message
news:Od**************@TK2MSFTNGP05.phx.gbl...
| Hi,
|
| Dim a, b As Single
| a = 2.2
| b = 0.4
| Response.Write(a - b)
|
| This gives: 1,8 (rendered according my regional settings)
|
|
| But this gives: 0,8000001 (rendered according my regional settings):
| Dim a, b As Single
| a = 1.2
| b = 0.4
| Response.Write(a - b)
|
|
| Any explantion for this, and how to solve this?? ( i rounded on two
digits)
| Thanks
| Dave
|
|
Jul 15 '06 #2
Thanks,

suppose i use variables a and b which are defined as decimals and gets their
values from anywhere else.
How can i put the "D" after the variable :
c=a-b D


"Jay B. Harlow [MVP - Outlook]" <Ja************@tsbradley.netschreef in
bericht news:eh**************@TK2MSFTNGP03.phx.gbl...
Dave,
Try:

| Dim a, b As Decimal
| a = 2.2D
| b = 0.4D
| Response.Write(a - b)

| But this gives: 0,8000001 (rendered according my regional settings):
Is given as that is the approximate value that a Single can hold. A Single
holds base 2 floating point numbers, some (most) base 10 floating point
numbers cannot be exactly represented

For details see:

http://www.yoda.arachsys.com/csharp/floatingpoint.html

http://www.yoda.arachsys.com/csharp/decimal.html
NOTE: The D in 2.2D indicates it is a Decimal literal as opposed to 2.2
which is a Double literal, as opposed to 2.2F which is a Single literal.
--
Hope this helps
Jay B. Harlow [MVP - Outlook]
.NET Application Architect, Enthusiast, & Evangelist
T.S. Bradley - http://www.tsbradley.net
"Dave" <sd***@fgn.qcwrote in message
news:Od**************@TK2MSFTNGP05.phx.gbl...
| Hi,
|
| Dim a, b As Single
| a = 2.2
| b = 0.4
| Response.Write(a - b)
|
| This gives: 1,8 (rendered according my regional settings)
|
|
| But this gives: 0,8000001 (rendered according my regional settings):
| Dim a, b As Single
| a = 1.2
| b = 0.4
| Response.Write(a - b)
|
|
| Any explantion for this, and how to solve this?? ( i rounded on two
digits)
| Thanks
| Dave
|
|


Jul 15 '06 #3
| suppose i use variables a and b which are defined as decimals and gets
their
| values from anywhere else.
If you define the variable as Decimal, the variable will hold a Decimal.

| c=a-b D

a is a variable of type Decimal, it will return a Decimal value,
literals are not involved here, so you don't need the "literal type
character D".

The D is part of a Literal (constant if you will) value.

2.2 is a literal

a is a variable

b is a variable

c is a variable

2.2 is a Double literal

2.2D is a Decimal literal

2.2F is a Single literal (the F stands for Float).
--
Hope this helps
Jay B. Harlow [MVP - Outlook]
..NET Application Architect, Enthusiast, & Evangelist
T.S. Bradley - http://www.tsbradley.net
"Dave" <sd***@fgn.qcwrote in message
news:uq**************@TK2MSFTNGP03.phx.gbl...
| Thanks,
|
| suppose i use variables a and b which are defined as decimals and gets
their
| values from anywhere else.
| How can i put the "D" after the variable :
| c=a-b D
|
|
|
|
| "Jay B. Harlow [MVP - Outlook]" <Ja************@tsbradley.netschreef in
| bericht news:eh**************@TK2MSFTNGP03.phx.gbl...
| Dave,
| Try:
| >
| | Dim a, b As Decimal
| | a = 2.2D
| | b = 0.4D
| | Response.Write(a - b)
| >
| | But this gives: 0,8000001 (rendered according my regional settings):
| Is given as that is the approximate value that a Single can hold. A
Single
| holds base 2 floating point numbers, some (most) base 10 floating point
| numbers cannot be exactly represented
| >
| For details see:
| >
| http://www.yoda.arachsys.com/csharp/floatingpoint.html
| >
| http://www.yoda.arachsys.com/csharp/decimal.html
| >
| >
| NOTE: The D in 2.2D indicates it is a Decimal literal as opposed to 2.2
| which is a Double literal, as opposed to 2.2F which is a Single literal.
| >
| >
| --
| Hope this helps
| Jay B. Harlow [MVP - Outlook]
| .NET Application Architect, Enthusiast, & Evangelist
| T.S. Bradley - http://www.tsbradley.net
| >
| >
| "Dave" <sd***@fgn.qcwrote in message
| news:Od**************@TK2MSFTNGP05.phx.gbl...
| | Hi,
| |
| | Dim a, b As Single
| | a = 2.2
| | b = 0.4
| | Response.Write(a - b)
| |
| | This gives: 1,8 (rendered according my regional settings)
| |
| |
| | But this gives: 0,8000001 (rendered according my regional settings):
| | Dim a, b As Single
| | a = 1.2
| | b = 0.4
| | Response.Write(a - b)
| |
| |
| | Any explantion for this, and how to solve this?? ( i rounded on two
| digits)
| | Thanks
| | Dave
| |
| |
| >
| >
|
|
Jul 15 '06 #4
Thanks again

"Jay B. Harlow [MVP - Outlook]" <Ja************@tsbradley.netschreef in
bericht news:%2****************@TK2MSFTNGP03.phx.gbl...
>| suppose i use variables a and b which are defined as decimals and gets
their
| values from anywhere else.
If you define the variable as Decimal, the variable will hold a Decimal.

| c=a-b D

a is a variable of type Decimal, it will return a Decimal value,
literals are not involved here, so you don't need the "literal type
character D".

The D is part of a Literal (constant if you will) value.

2.2 is a literal

a is a variable

b is a variable

c is a variable

2.2 is a Double literal

2.2D is a Decimal literal

2.2F is a Single literal (the F stands for Float).
--
Hope this helps
Jay B. Harlow [MVP - Outlook]
.NET Application Architect, Enthusiast, & Evangelist
T.S. Bradley - http://www.tsbradley.net
"Dave" <sd***@fgn.qcwrote in message
news:uq**************@TK2MSFTNGP03.phx.gbl...
| Thanks,
|
| suppose i use variables a and b which are defined as decimals and gets
their
| values from anywhere else.
| How can i put the "D" after the variable :
| c=a-b D
|
|
|
|
| "Jay B. Harlow [MVP - Outlook]" <Ja************@tsbradley.netschreef
in
| bericht news:eh**************@TK2MSFTNGP03.phx.gbl...
| Dave,
| Try:
| >
| | Dim a, b As Decimal
| | a = 2.2D
| | b = 0.4D
| | Response.Write(a - b)
| >
| | But this gives: 0,8000001 (rendered according my regional settings):
| Is given as that is the approximate value that a Single can hold. A
Single
| holds base 2 floating point numbers, some (most) base 10 floating
point
| numbers cannot be exactly represented
| >
| For details see:
| >
| http://www.yoda.arachsys.com/csharp/floatingpoint.html
| >
| http://www.yoda.arachsys.com/csharp/decimal.html
| >
| >
| NOTE: The D in 2.2D indicates it is a Decimal literal as opposed to
2.2
| which is a Double literal, as opposed to 2.2F which is a Single
literal.
| >
| >
| --
| Hope this helps
| Jay B. Harlow [MVP - Outlook]
| .NET Application Architect, Enthusiast, & Evangelist
| T.S. Bradley - http://www.tsbradley.net
| >
| >
| "Dave" <sd***@fgn.qcwrote in message
| news:Od**************@TK2MSFTNGP05.phx.gbl...
| | Hi,
| |
| | Dim a, b As Single
| | a = 2.2
| | b = 0.4
| | Response.Write(a - b)
| |
| | This gives: 1,8 (rendered according my regional settings)
| |
| |
| | But this gives: 0,8000001 (rendered according my regional settings):
| | Dim a, b As Single
| | a = 1.2
| | b = 0.4
| | Response.Write(a - b)
| |
| |
| | Any explantion for this, and how to solve this?? ( i rounded on two
| digits)
| | Thanks
| | Dave
| |
| |
| >
| >
|
|


Jul 15 '06 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: K. Shier | last post by:
when attempting to edit code in a class file, i see the bug "Visual Basic ..NET compiler is unable to recover from the following error: System Error &Hc0000005&(Visual Basic internal compiler...
8
by: Rene | last post by:
The following code: float someFloat = 0; for(int x = 1; x <= 10; x++) { someFloat += .1F; Console.WriteLine(someFloat.ToString()); }
102
by: Xah Lee | last post by:
i had the pleasure to read the PHP's manual today. http://www.php.net/manual/en/ although Pretty Home Page is another criminal hack of the unix lineage, but if we are here to judge the quality...
16
by: Edward Diener | last post by:
After spending more than a day reducing a complicated compiler bug to a simple case I reported it to the MSDN Product Feedback Center as a bug just now. However this bug is completely stymying my...
26
by: Patient Guy | last post by:
The code below shows the familiar way of restricting a function to be a method of a constructed object: function aConstructor(arg) { if (typeof(arg) == "undefined") return (null);...
158
by: Giovanni Bajo | last post by:
Hello, I just read this mail by Brett Cannon: http://mail.python.org/pipermail/python-dev/2006-October/069139.html where the "PSF infrastracture committee", after weeks of evaluation, recommends...
0
by: Kurt B. Kaiser | last post by:
Patch / Bug Summary ___________________ Patches : 385 open (+21) / 3790 closed (+21) / 4175 total (+42) Bugs : 1029 open (+43) / 6744 closed (+43) / 7773 total (+86) RFE : 262 open...
0
by: LiveTecs | last post by:
http://www.livetecs.com TimeLive Web Collaboration Suite is an integrated suite that allows you to manage project life cycle including tasks, issues, bugs, timesheet, expense, attendance. ...
12
by: Juan T. Llibre | last post by:
re: !I found an MSDN document that explains why what I'm trying to do should work Lee, From : http://www.w3.org/TR/REC-xml/ "A special attribute named xml:lang may be inserted in...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.