469,929 Members | 1,443 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,929 developers. It's quick & easy.

Strange Conversion Error from double to int

The following code compiled using MSVC++6.0 -

double di=0.0;
for(;di<=512.0;)
{
string.Format("%li",int(di));
pDC-> TextOut( left +int(di), 100,string);

di += 51.2;
}

gives the following unexpected output -

0 51 102 153 204 256 307 358 409 460 511

Why 511 and not 512??

Replacing with -

string.Format("%lf",di);

gives 512.00000

so why is the casting of 512.0 {int(di)} giving 511??

Thanks,

Barry.
Jul 22 '05 #1
6 4515
Barry posted:
The following code compiled using MSVC++6.0 -

double di=0.0;
for(;di<=512.0;)
{
string.Format("%li",int(di));
pDC-> TextOut( left +int(di), 100,string);

di += 51.2;
}

double di = 0.0;

do
{
string.Format( "%li", int(di) );

pDC->TextOut( left +int(di), 100, string );

di += 51.2;

} while ( di <= 512.0 )
Now isn't that pretty.
-JKop
Jul 22 '05 #2

"Barry" <bg***@yahoo.com> wrote in message
news:73**************************@posting.google.c om...
The following code compiled using MSVC++6.0 -

double di=0.0;
for(;di<=512.0;)
{
string.Format("%li",int(di));
pDC-> TextOut( left +int(di), 100,string);

di += 51.2;
}

gives the following unexpected output -

0 51 102 153 204 256 307 358 409 460 511

Why 511 and not 512??

Replacing with -

string.Format("%lf",di);

gives 512.00000

so why is the casting of 512.0 {int(di)} giving 511??

Thanks,

Barry.


Because floating point arithmetic is not exact, and unless you really,
really know what you are doing you shouldn't assume that it is going to be.

In your case there is no exact floating point representation of the number
51.2 and so errors accumulate.

john
Jul 22 '05 #3
Barry wrote:
The following code compiled using MSVC++6.0 -

double di=0.0;
for(;di<=512.0;)
{
string.Format("%li",int(di));
pDC-> TextOut( left +int(di), 100,string);

di += 51.2;
}

gives the following unexpected output -

0 51 102 153 204 256 307 358 409 460 511

Why 511 and not 512??

Replacing with -

string.Format("%lf",di);

gives 512.00000

so why is the casting of 512.0 {int(di)} giving 511??


Some numbers cannot be exactly represented in floating point format.
Therefore adding 51.2 ten times does necessarily not yield exactly 512.
It may in fact result in 511.999999999999999 being stored in di. Since
string.Format() is not a standard C++ function you will have to lookup
the documentation of that function yourself. Apparently this function
does do some rounding. Considering it displays only five digits behind
the decimal point (the accuracy a double is much higher than that)
512.00000 is probably a closer representation of what is in the di
variable than 511.99999.

If you need exact results don't use floating point numbers.

--
Peter van Merkerk
peter.van.merkerk(at)dse.nl
Jul 22 '05 #4
Barry wrote:

The following code compiled using MSVC++6.0 -

double di=0.0;
for(;di<=512.0;)
{
string.Format("%li",int(di));
pDC-> TextOut( left +int(di), 100,string);

di += 51.2;
}

gives the following unexpected output -

0 51 102 153 204 256 307 358 409 460 511

Why 511 and not 512??

Replacing with -

string.Format("%lf",di);

gives 512.00000


replace that with

string.Format("%.14lf",di);

and see for yourself

You might also want to study
http://www.petebecker.com/js200006.html

--
Karl Heinz Buchegger
kb******@gascad.at
Jul 22 '05 #5
Peter van Merkerk wrote:
Some numbers cannot be exactly represented in floating point format.
Therefore adding 51.2 ten times does necessarily not yield exactly 512.


Correction: does *not* necessarily yield exactly 512.

--
Peter van Merkerk
peter.van.merkerk(at)dse.nl
Jul 22 '05 #6

"Barry" <bg***@yahoo.com> wrote in message
news:73**************************@posting.google.c om...
The following code compiled using MSVC++6.0 -

double di=0.0;
for(;di<=512.0;)
{
string.Format("%li",int(di));
pDC-> TextOut( left +int(di), 100,string);

di += 51.2;
}

gives the following unexpected output -

0 51 102 153 204 256 307 358 409 460 511

Why 511 and not 512??

Replacing with -

string.Format("%lf",di);

gives 512.00000

so why is the casting of 512.0 {int(di)} giving 511??
That should be expected. since one can argue that any double where d >=
511.0 and d < 512.0 is converted to an integer with the value of 511. Since
floating numbers are designed to express such values as 2/3 or
0.666666666667, its perfectly legal for 512.0 to be represented by
511.999999999999.

The problem here is not the representation or conversion of the double. The
problem lies in the code. For starters, a floating number should never be
accompanied with a comparison operator of any kind (d <= 512.0) in the
context of a loop condition. In this case, if d were an int, your processor
would be saved from a tremendous amount of useless computations.

The comparison in following code is guarenteed (and expected) to fail:

int main()
{
double d = 0.0;

for (int i = 0; i < 10; i++)
{
d += 51.2;
}

double dx = 512.0;

if (dx != d)
{
std::cout << "comparison failure, d = " << d << std::endl;
}

return 0;
}

Thanks,

Barry.

Jul 22 '05 #7

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

24 posts views Thread by David | last post: by
8 posts views Thread by grundmann | last post: by
31 posts views Thread by Bjørn Augestad | last post: by
11 posts views Thread by Steve Gough | last post: by
6 posts views Thread by Dhirendra Singh | last post: by
4 posts views Thread by Valeriu Catina | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.