float to DWORD conversion bug

Gabest

Running this piece of code while having the sse optimization turned on
(vcnet2003), something really strange happens I cannot explain. Without sse
it is giving me the right results.

float f = 0.8;
f *= UINT_MAX;
DWORD dw = (DWORD)f;

Basically the result in dw should be 0xcccccccc (or 0xcccccd00 because of
float's inaccuracy), but instead I get 0x80000000 for anything above f =
0.5!

Here is the same code with the assembly instructions inlined copied from the
debugger's view. According to the watch window, f still holds the right
value after f *= UINT_MAX, and only becomes wrong after the third line.

float f = 0.8;

movss xmm0,dword ptr [__real@3f4ccccd (102261C4h)]
movss dword ptr [f],xmm0

f *= UINT_MAX;

movss xmm0,dword ptr [f]
mulss xmm0,dword ptr [__real@4f800000 (102260ECh)]
movss dword ptr [f],xmm0

DWORD dw = (DWORD)f;

fld dword ptr [f]
fnstcw word ptr [ebp-10Ah]
movzx eax,word ptr [ebp-10Ah]
or ah,0Ch
mov dword ptr [ebp-110h],eax
fldcw word ptr [ebp-110h]
fistp dword ptr [ebp-114h]
fldcw word ptr [ebp-10Ah]
mov eax,dword ptr [ebp-114h]
mov dword ptr [dw],eax

Nov 17 '05 #1

Subscribe Post Reply

4680

Gabest

Hm, no one has commented on this yet?

Anyway, I just noticed the problem is not there with the sse opt, but only
with sse2. The sse code seems to call __ftol2 which still works. So as it
looks, the current sse2 option in vc2k3 is quite useless and _dangerous_ if
someone wants to use floating point calculations in his program too.
DWORD dw = (DWORD)f;

sse:

fld dword ptr [f]
call @ILT+1375(__ftol2) (411564h)
mov dword ptr [dw],eax

sse2

fld dword ptr [f]
fnstcw word ptr [ebp-0DAh]
movzx eax,word ptr [ebp-0DAh]
or ah,0Ch
mov dword ptr [ebp-0E0h],eax
fldcw word ptr [ebp-0E0h]
fistp dword ptr [ebp-0E4h]
fldcw word ptr [ebp-0DAh]
mov eax,dword ptr [ebp-0E4h]
mov dword ptr [dw],eax

Nov 17 '05 #2

Similar topics

Is it legal to type cast to DWORD* ???

by: __PPS__ | last post by:

Actually what I mean is that - if I have some memory buffer, lets say char a; and then I do like this: DWORD num = 0x1234; *(DWORD*)a = num; (1) *(DWORD*)(a+1) = num; (2) either...

C / C++

Clashes between type conversion and operator[]

by: Arne Schmitz | last post by:

I guess this has been asked before, but I cannot find any answer to this problem. I have program like this: ---SNIP--- #include <cassert> #include <cstdlib> class C { public:

C / C++

32-bit IEEE float multiplication

by: Andy | last post by:

Hi, I don't know if this is the correct group to post this, but when I multiply a huge floating point value by a really small (non-zero) floating point value, I get 0 (zero) for the result. This...

C / C++

how will two objects of type float perform when adding one to another

by: van6 | last post by:

code frag: int main() { float fval1 = 1.0f; float fval2 = 2.0f; double dsum =0.0; // here is the point ; // Are fval1 and fval2 both promoted to type of double prior to adding

C / C++

converting string to float

by: pipe.jack | last post by:

I'm trying to convert string to float and my float after conversion is 0 (zero). Here is my code: $c = $currencies->format($cart->show_total()); echo gettype($c); echo (float)$c; $c = 39.59...

PHP

conversion warning

by: Alexandru Mosoi | last post by:

Hi Recently I've made a stupid bug. Here is a snippet: long long a, b; .... int c = min(a, b); But this bug could have been prevented if g++ would warn me about loss of precision. Do you...

C / C++

Conversion between float and long

by: d major | last post by:

I was very puzzled about the conversion between float and long, I cann't understand why a long val can convert to a float, as the below codes show: typedef unsigned long u_long; float val =...

C / C++

Re: Comparing float and decimal

by: Terry Reedy | last post by:

Gerhard Häring wrote: The new fractions module acts differently, which is to say, as most would want. True Traceback (most recent call last): File "<pyshell#20>", line 1, in <module> F(1.0)...

Python

Basic Javascript concepts

by: aa123db | last post by:

Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...

Javascript

Migrating Website to Cloud - Emmanuel Katto

by: emmanuelkatto | last post by:

Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel

General

Navigating the Data Structures and Algorithms (DSA)

by: BarryA | last post by:

What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...

Algorithms / Advanced Math

Is that possible of reading the .csv file in column wise and the column have different lengths ?

by: Sonnysonu | last post by:

This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...

C / C++

How to build RAID in BIOS?

by: Hystou | last post by:

There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

Computer Hardware

What is ONU?

by: marktang | last post by:

ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...

General

Changing the language in Windows 10

by: Hystou | last post by:

Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...

Windows Server

Problem With Comparison Operator <=> in G++

by: Oralloy | last post by:

Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...

C / C++

Maximizing Business Potential: The Nexus of Website Design and Digital Marketing

by: jinu1996 | last post by:

In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...

Online Marketing