Problem with VC++ signed and unsigned __int64 << and >> operators?

[Peter Bromley]

The following code snippet does not seem to work correcly

unsigned __int64 result = 0xFFFFFFFFFFFFFFFF;
result = result << 64;
Debugger.WriteLine(System::String::Format(S"{0:X16 }", __box(result));
result = 0xFFFFFFFFFFFFFFFF;
result = result >> 64;
Debugger.WriteLine(System::String::Format(S"{0:X16 }", __box(result));

__int64 result = -1;
result = result << 64;
Debugger.WriteLine(System::String::Format(S"{0:X16 }", __box(result));

I am expecting output to read

0000000000000000
FFFFFFFFFFFFFFFF
0000000000000000
FFFFFFFFFFFFFFFF

but output reads

FFFFFFFF00000000 << incorrect
00000000FFFFFFFF << incorrect
FFFFFFFF00000000 << incorrect
FFFFFFFFFFFFFFFF << correct

All other shift values ( < 64) appear to work correctly.

Can someone from MS confirm this is a bug with the int64 shift
operators, please?

Peter

--
If you wish to reply to me directly, my addres is spam proofed as:

pbromley at adi dot co dot nz

Or if you prefer - no****@nowhere.com :-)

[Doug Harrison [MVP]]

Peter Bromley wrote:

The following code snippet does not seem to work correcly

unsigned __int64 result = 0xFFFFFFFFFFFFFFFF;
result = result << 64;
Debugger.WriteLine(System::String::Format(S"{0:X1 6}", __box(result));
result = 0xFFFFFFFFFFFFFFFF;
result = result >> 64;
Debugger.WriteLine(System::String::Format(S"{0:X1 6}", __box(result));

__int64 result = -1;
result = result << 64;
Debugger.WriteLine(System::String::Format(S"{0:X1 6}", __box(result));

I am expecting output to read

0000000000000000
FFFFFFFFFFFFFFFF
0000000000000000
FFFFFFFFFFFFFFFF

but output reads

FFFFFFFF00000000 << incorrect
00000000FFFFFFFF << incorrect
FFFFFFFF00000000 << incorrect
FFFFFFFFFFFFFFFF << correct

All other shift values ( < 64) appear to work correctly.

It's undefined to shift a value by an amount >= its length in bits, so you
need to avoid doing that.

--
Doug Harrison
Microsoft MVP - Visual C++

[Gabest]

> It's undefined to shift a value by an amount >= its length in bits, so you

need to avoid doing that.

This thru for mc++ only, right? Shifting in x86 is well defined, for
unsigned 0s are shifted in, for signed the sign bit is extended when needed.
I wonder what can be the reason why the same code compiled for .net
shouldn't give the same results as its native counterpart.

[Gabest]

"Gabest" <ga****@freemail.hu> wrote in message
news:e6**************@TK2MSFTNGP14.phx.gbl...

It's undefined to shift a value by an amount >= its length in bits, so
you
need to avoid doing that.

This thru

"is true" I mean. It's really late here :)

[Carl Daniel [VC++ MVP]]

Gabest wrote:

It's undefined to shift a value by an amount >= its length in bits,
so you need to avoid doing that.

This thru for mc++ only, right? Shifting in x86 is well defined, for
unsigned 0s are shifted in, for signed the sign bit is extended when
needed. I wonder what can be the reason why the same code compiled
for .net shouldn't give the same results as its native counterpart.

No, it's true for C and C++ in general - shifting by an amount >= the
variable size is undefined behavior according to the standards.

-cd

[Gabest]

Well, after digging up intel's manual on these shifting instructions, it is
defined but not in the way I though to be:

"The count operand can be an immediate value or register CL. The count is
masked to 5 bits, which limits the count range to 0 to 31"

I believe amd can have something similar for the x86-64 ISA in its manuals
too, and also that in 32 bit mode the compiler inserts such a piece of code
what still gives a good result for its builtin __int64 type, but only for
the native code.

[Doug Harrison [MVP]]

Gabest wrote:

It's undefined to shift a value by an amount >= its length in bits, so you
need to avoid doing that.

This thru for mc++ only, right? Shifting in x86 is well defined, for
unsigned 0s are shifted in, for signed the sign bit is extended when needed.
I wonder what can be the reason why the same code compiled for .net
shouldn't give the same results as its native counterpart.

What I said comes from the C++ Standard, 5.8/1. (To be completely accurate,
s/value/promoted left operand/, but note that a 64-bit integer undergoes no
promotion in VC++.) The VC docs say the same thing:

http://msdn.microsoft.com/library/de..._operators.asp
<q>
The results are undefined if the right operand of a shift expression is
negative or if the right operand is greater than or equal to the number of
bits in the (promoted) left operand.
</q>

I'm unable to find any information on the shift operators in MC++, but even
if the result were well-defined, I'd hesitate to take advantage of it.

--
Doug Harrison
Microsoft MVP - Visual C++

[Ioannis Vranos]

Gabest wrote:

This thru for mc++ only, right? Shifting in x86 is well defined, for
unsigned 0s are shifted in, for signed the sign bit is extended when needed.
I wonder what can be the reason why the same code compiled for .net
shouldn't give the same results as its native counterpart.

I suppose this happens because in managed applications you do not
program against your x86 but the .NET VM which has its own assembly
language etc.

So your x86 system-specific code does not work the same to this
different "machine".

To see detailed technical info for .NET (CLI) specific code, you may
download the latest CLI standard:

http://www.ecma-international.org/pu...s/Ecma-335.htm

which also contains the specification of its assembly language.
A book on this assembly language:

http://www.amazon.com/exec/obidos/tg...glance&s=books

--
Ioannis Vranos

[Hendrik Schober]

Gabest <ga****@freemail.hu> wrote:

Well, after digging up intel's manual on these shifting instructions, it is
defined but not in the way I though to be:

"The count operand can be an immediate value or register CL. The count is
masked to 5 bits, which limits the count range to 0 to 31"

I believe amd can have something similar for the x86-64 ISA in its manuals
too, and also that in 32 bit mode the compiler inserts such a piece of code
what still gives a good result for its builtin __int64 type, but only for
the native code.

The problem is that you're not programming
in assembler/machine code. If you're in
C/C++/MC++ land, you have to follow the
rules as spelled out there. For C++ (and
most likely for C, too), what you do is
undefined. As for MC++ -- I don't know
that.

Schobi

--
Sp******@gmx.de is never read
I'm Schobi at suespammers dot org

"The presence of those seeking the truth is infinitely
to be prefered to those thinking they've found it."
Terry Pratchett

[Gabest]

> The problem is that you're not programming

in assembler/machine code.
Yea, you are right, I was believing too much that the compiler would just
use the existing assembly instructions for shifting, while it has the right
to translate it to anything else of course.

There is still one thing but bothers me though. This means you cannot be
sure about shifting by a+b always gives the same result as shifting by a and
then b separately. Something like saying 1+2+3 equals to 6, but the result
of 1+5 is undefined. I can understand shl/shr/sar can have some limitations,
since they are just low level instructions translated directly to machine
code, but in c/c++ this sounds pretty wierd to me.
If you're in
C/C++/MC++ land, you have to follow the
rules as spelled out there. For C++ (and
most likely for C, too), what you do is
undefined. As for MC++ -- I don't know
that.

Schobi

[Bo Persson]

"Gabest" <ga****@freemail.hu> skrev i meddelandet
news:uN**************@TK2MSFTNGP12.phx.gbl...

The problem is that you're not programming
in assembler/machine code.

Yea, you are right, I was believing too much that the compiler would
just use the existing assembly instructions for shifting, while it has
the right to translate it to anything else of course.

But the limitations are there just so that the compiler should be able
to use the machine instructions. :-)

On recent x86 machines, the shift count is limited to be less than the
number of bits in a machine register. On other processors it is not. So,
the C and C++ standards conveniently state that we cannot know what will
happen - undefined behavior.
On early x86 processors, there was no specific limit on the shift count,
but also no limit on how long it would take to execute. Luckily they
were limited to 16 bits. Now consider a 32 bit model, where

1 << UINT_MAX;

would actually shift the bit left 4 billion times. How long would that
take? What would the interrupt response time be like during the
shifting?

Bo Persson

[Peter Bromley]

>>>It's undefined to shift a value by an amount >= its length in bits, so you

need to avoid doing that.

This thru for mc++ only, right? Shifting in x86 is well defined, for
unsigned 0s are shifted in, for signed the sign bit is extended when needed.
I wonder what can be the reason why the same code compiled for .net
shouldn't give the same results as its native counterpart.

What I said comes from the C++ Standard, 5.8/1. (To be completely accurate,
s/value/promoted left operand/, but note that a 64-bit integer undergoes no
promotion in VC++.) The VC docs say the same thing:

http://msdn.microsoft.com/library/de..._operators.asp
<q>
The results are undefined if the right operand of a shift expression is
negative or if the right operand is greater than or equal to the number of
bits in the (promoted) left operand.
</q>

I'm unable to find any information on the shift operators in MC++, but even
if the result were well-defined, I'd hesitate to take advantage of it.

So, that makes things very clear. I should have consulted MSDN before
posting.

It is interesting though that __int64 << and >> use function calls (I
presume into clr code). Perhaps a debug version of these functions which
throws ArgumentException would be useful. Just a thought.

Anyway, I have restructured my code so that 64 is never passed to the <<
operator (which is what triggered my inquiry).

Thanks to all,

Peter

--
If you wish to reply to me directly, my addres is spam proofed as:

pbromley at adi dot co dot nz

Or if you prefer - no****@nowhere.com :-)

[Ioannis Vranos]

Hendrik Schober wrote:

The problem is that you're not programming
in assembler/machine code. If you're in
C/C++/MC++ land, you have to follow the
rules as spelled out there. For C++ (and
most likely for C, too), what you do is
undefined. As for MC++ -- I don't know
that.

This is true concerning portability. However it has not anything to do
with system-specific code.
I think the main factor for the OP issue, is that the target machine is
not his x86 but the CLR when he uses managed code.

--
Ioannis Vranos

[Gabest]

> 1 << UINT_MAX;

would actually shift the bit left 4 billion times. How long would that
take? What would the interrupt response time be like during the shifting?

Hehe, good point, though smart hardware could still clamp it to the width of
the register.

[Hendrik Schober]

Ioannis Vranos <iv*@remove.this.grad.com> wrote:

[...]

This is true concerning portability. However it has not anything to do
with system-specific code.
True.
However, nowadays, when you target std C++
/and/ MC++, you're alreday coding to two
different platforms. I think not very many
developers here realize that they actually
are porting their code between two platforms
hwen they do that.
[...]

Schobi

--
Sp******@gmx.de is never read
I'm Schobi at suespammers dot org

"The presence of those seeking the truth is infinitely
to be prefered to those thinking they've found it."
Terry Pratchett

[Ioannis Vranos]

Hendrik Schober wrote:

True.
However, nowadays, when you target std C++
/and/ MC++, you're alreday coding to two
different platforms.

What do you mean. When you use /clr you are actually targeting CLR.

I think not very many
developers here realize that they actually
are porting their code between two platforms
hwen they do that.

Well, let me provide another example. In .NET
(http://msdn.microsoft.com/netframework) long is System::Int32.

However this is not required for all CLI environments, so for example in
DotGNU (http://www.gnu.org/projects/dotgnu) or Mono
(http://www.mono-project.com) or some other CLI VM, it can be otherwise.
We can say that code assuming long being System::Int32 is .NET specific
and this assumption is not portable (causes undefined behaviour in a CLI
system having long as another type).

--
Ioannis Vranos

[Hendrik Schober]

Ioannis Vranos <iv*@remove.this.grad.com> wrote:

Hendrik Schober wrote:

True.
However, nowadays, when you target std C++
/and/ MC++, you're alreday coding to two
different platforms.
What do you mean. When you use /clr you are actually targeting CLR.

Yes. And when you don't use /clr, you are
targetting x86. If you do both with the
same code base, you have code that ports
between two platforms.
What I mean is, porting isn't as esotheric
and rare as many think. Even what they now
view as Windows-only (i.e., one platform)
code often was ported from DOS to Win16 to
Win32 and probably will get ported to .NET
or Win64 or both. IME, porting is rather
common and the attitude to do something
based on the fact that some code runs only
on one platform causes trouble later more
often than people realize.
[...]

Schobi

--
Sp******@gmx.de is never read
I'm Schobi at suespammers dot org

"The presence of those seeking the truth is infinitely
to be prefered to those thinking they've found it."
Terry Pratchett

[Ioannis Vranos]

Hendrik Schober wrote:

Yes. And when you don't use /clr, you are
targetting x86. If you do both with the
same code base, you have code that ports
between two platforms.
What I mean is, porting isn't as esotheric
and rare as many think. Even what they now
view as Windows-only (i.e., one platform)
code often was ported from DOS to Win16 to
Win32 and probably will get ported to .NET
or Win64 or both. IME, porting is rather
common and the attitude to do something
based on the fact that some code runs only
on one platform causes trouble later more
often than people realize.

Yes I agree with you, we should try to be as much portable as possible.
I think I was not much comprehensible. :-) What I meant is that the
specific reason for OP problem was that his unportable/system-specific
code was not running against his x86 but against the CLR, and that's why
it did not execute as he expected.
In any case as you said, strictly ISO C++ speaking, his code was
unportable and invokes undefined behaviour.

--
Ioannis Vranos