473,320 Members | 2,024 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Number conversion problem

If I do this:

Console.WriteLine(Convert.ToString((int)(0.7f * 100)));
Console.WriteLine(Convert.ToString((int)(0.7d * 100)));

The application output is:
69
70

Can you help me to understand how conversion in c# is working?
Regards
Ovindo
Nov 16 '05 #1
3 1235
Ovindo <oe*@rocketmail.com> wrote:
If I do this:

Console.WriteLine(Convert.ToString((int)(0.7f * 100)));
Console.WriteLine(Convert.ToString((int)(0.7d * 100)));

The application output is:
69
70

Can you help me to understand how conversion in c# is working?


Sure. First, look at
http://www.pobox.com/~skeet/csharp/floatingpoint.html

Then use DoubleConverter.cs (linked in the article).

Let's look at what 0.7d and 0.7f *really* are:

using System;

public class Test
{
static void Main()
{
Console.WriteLine (DoubleConverter.ToExactString(0.7f));
Console.WriteLine (DoubleConverter.ToExactString(0.7d));
}
}

Results:

0.699999988079071044921875
0.699999999999999955591079014993738383054733276367 1875

Now let's look at the actual value of the results you got when
multiplying by 100 (before the cast):

0.7f*100 = 69.9999988079071044921875
0.7d*100 = 70

Now in fact, those calculations are being done at compile time by the
C# compiler, but you get the same results if you make it happen at
runtime.

Casting to an int rounds towards zero, hence why you're getting the
results you are.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Nov 16 '05 #2
"Jon Skeet [C# MVP]" <sk***@pobox.com> wrote in message
news:MP************************@msnews.microsoft.c om...
Ovindo <oe*@rocketmail.com> wrote:
If I do this:

Console.WriteLine(Convert.ToString((int)(0.7f * 100)));
Console.WriteLine(Convert.ToString((int)(0.7d * 100)));

The application output is:
69
70

Can you help me to understand how conversion in c# is working?


Sure. First, look at
http://www.pobox.com/~skeet/csharp/floatingpoint.html

Then use DoubleConverter.cs (linked in the article).

Let's look at what 0.7d and 0.7f *really* are:

using System;

public class Test
{
static void Main()
{
Console.WriteLine (DoubleConverter.ToExactString(0.7f));
Console.WriteLine (DoubleConverter.ToExactString(0.7d));
}
}

Results:

0.699999988079071044921875
0.699999999999999955591079014993738383054733276367 1875

Now let's look at the actual value of the results you got when
multiplying by 100 (before the cast):

0.7f*100 = 69.9999988079071044921875
0.7d*100 = 70

Now in fact, those calculations are being done at compile time by the
C# compiler, but you get the same results if you make it happen at
runtime.

Casting to an int rounds towards zero, hence why you're getting the
results you are.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too


I still have some problems to understand this.
If
0.7f*100 = 69.9999988079071044921875
why is
0.7d*100 = 70
and not 69.99999999999999555910790149937383830547332763671 875?

Regards
Ovindo
Nov 16 '05 #3
Ovindo <oe*@rocketmail.com> wrote:
I still have some problems to understand this.
If
0.7f*100 = 69.9999988079071044921875
why is
0.7d*100 = 70
and not 69.99999999999999555910790149937383830547332763671 875?


Because the closest exactly representable double to

69.99999999999999555910790149937383830547332763671 875

is 70.

(I recognise that it's odd. You'd expect to be able to keep the same
level of precision just through multiplying by an integer, but I
suspect if you look down at the binary level it makes sense.)

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Nov 16 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

16
by: atse | last post by:
Hi, How can I know the number of columns of the .csv files? I want to import them to the database. Thanks. Atse
6
by: John Bentley | last post by:
John Bentley writes at this level: If we think about our savings accounts then division never comes in (as far as I can see). We deposit and withdraw exact amounts most of the time. Occasionaly...
7
by: astro | last post by:
Anyone have suggestions on where to troubleshoot this error? Background: -Access 2k v. 9.0.6926 sp3 - front and backend on production server (wiindows 2k) -accessed via Citrix -front-end is...
9
by: fooboo | last post by:
Does anyone know if a easier way (built in function, or something) that can verify that a string is an alphanumeric number? Here is what I am doing now: for(i=0; i < strlen(temp); i++){...
11
by: Leroy | last post by:
Hello, I have a question regarding the maximum number of parameters that can be passed to a procedure. In VB 6 the max was 60. What is the max for Dot Net? please and thanks.
5
by: Pavils Jurjans | last post by:
Hello, I am somewhat lost in the implicit/expicit possible/impossible type casting in C#... I need to write a class, which among other things, must have wat to read a numeric value type, and...
4
by: dba_222 | last post by:
Dear Experts, Ok, I hate to ask such a seemingly dumb question, but I've already spent far too much time on this. More that I would care to admit. In Sql server, how do I simply change a...
1
by: FAQ server | last post by:
----------------------------------------------------------------------- FAQ Topic - Why does 1+1 equal 11? or How do I convert a string to a number?...
2
by: FAQ server | last post by:
----------------------------------------------------------------------- FAQ Topic - Why does 1+1 equal 11? or How do I convert a string to a number?...
23
by: neha_chhatre | last post by:
which is the best format specifier(data type) if i have to work with decimal number. also please tell me the syntax for truncating a decimal number please reply as soon as possible
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.