Request

[jacob navia]

As a consequence of a heavy discussion in another thread,
I wrote this program:
#include <stdlib.h>
int main(void)
{
char *p=calloc(65521 ,65552);
if (p)
printf("BUG!!!! \n");
}

The multiplication will give an erroneus result
in 32 bit size_t systems. I have tested this
in several systems with several compilers.

Maybe people with other compilers/implementations
would compile this program and send the results?

The results I have so far are:

LINUX
GCC 2.95
HAS THE BUG

GCC-4.02
NO BUGS

GCC-3.x
NO BUGS

WINDOWS
cygwin gcc 3.4.4
NO bugs
pelles c:
HAS THE BUG
Microsoft
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42
for 80x86
NO BUGS (If the multiplication overflows it will trap. Very good
behavior)

AIX
IBM-cc (compiling in 32 bit mode)
NO BUGS
gcc 4.0.0 (compiling in 32 bit mode)
NO BUGS
Thanks in advance for your cooperation

[user923005]

jacob navia wrote:

user923005 a écrit :

jacob navia wrote:

>No, this is impossible in 32 bit size_t systems!!!

Why? The calloc() call has two parameters. There is no reason that
the total allocation cannot exceed the value of a size_t.

BTW, there are definitely 32 bit systems that will allocate memory in
excess of 32 bits. For example:
http://support.microsoft.com/kb/274750

This is irrelevant. Those links refer as to how
to use certain flags in the boot.ini to allow
*3GB* if user space. (!!!!!!!)

That is either 8 GB user space or 32 GB, not 3 GB. I suggest you read
again -- this time with comprehension enabled... Here is a quote from
that page that you may find helpful:
"SQL Server 2000 Enterprise Edition introduces support for the use of
Microsoft Windows 2000 Address Windowing Extensions (AWE) to address
approximately 8 GB of memory for instances that run on Microsoft
Windows 2000 Advanced Server, and approximately 32 GB for instances
that run on Microsoft Windows 2000 Datacenter."

P.S.
These operating systems are 32 bits.

And 3GB is well below a 32 bit size_t.

You seem to have a bit of a comprehension problem.

With a 32 bit size_t and pointer size how can you
address more than 4GB?

There are lots of ways to do it.

But this is a well known tactic.

Since I present the code, I do the effort of making all those
measurements, it is so easy to just sit by and say

"A system with 32 bit size_t could allocate more than 4GB
memory. A system like that COULD exist."

So WHAT?

So that is all that is needed to show that your test is flawed.

Can you please tell me where that system is????

I mentioned one above that uses 32 bit addressing lines and yet can
address 32 GB or RAM. The way it does it is with memory models at the
low level. Just like back in the bad old days when MS-DOS programmers
had to model with large, medium or small memory models because of 16
bit addressing, in a similar way you can use 32 bit segments and
offsets to address (potentially) 64 GB of RAM or any other scheme you
like.

If not, just do not participate. Thanks for your
correction. A system where this test gives a false
positive COULD exist. It is conceivable.

Your test has no merit that I can imagine.
Systems that will allocate 2^64 pages of ram could return a failure
result if the user space is less than 4GB. So what have we learned
from your code? Nothing.

[jacob navia]

user923005 a écrit :

That is either 8 GB user space or 32 GB, not 3 GB. I suggest you read
again -- this time with comprehension enabled... Here is a quote from
that page that you may find helpful:
"SQL Server 2000 Enterprise Edition introduces support for the use of
Microsoft Windows 2000 Address Windowing Extensions (AWE) to address
approximately 8 GB of memory for instances that run on Microsoft
Windows 2000 Advanced Server, and approximately 32 GB for instances
that run on Microsoft Windows 2000 Datacenter."

P.S.
These operating systems are 32 bits.

So what?

I do not talk about SQL server!

I am talking about calloc, quite a different beast.

Maybe SQL server can allocate a lot of memory but
this is not relevant to calloc I am sure.

>>With a 32 bit size_t and pointer size how can you
address more than 4GB?

There are lots of ways to do it.

No. If you mean using segmented 32:48 bit pointers they
are no longer 32 bit!

I mentioned one above that uses 32 bit addressing lines and yet can
address 32 GB or RAM. The way it does it is with memory models at the
low level. Just like back in the bad old days when MS-DOS programmers
had to model with large, medium or small memory models because of 16
bit addressing, in a similar way you can use 32 bit segments and
offsets to address (potentially) 64 GB of RAM or any other scheme you
like.

But those aren't 32 bit pointers but 48 bit pointers.

>

>>If not, just do not participate. Thanks for your
correction. A system where this test gives a false
positive COULD exist. It is conceivable.

Your test has no merit that I can imagine.

OK

Systems that will allocate 2^64 pages of ram could return a failure
result if the user space is less than 4GB. So what have we learned
from your code? Nothing.

OK

Thanks for your input

[Richard Tobin]

In article <11************ *********@v33g2 000cwv.googlegr oups.com>,
user923005 <dc*****@connx. comwrote:

>BTW, there are definitely 32 bit systems that will allocate memory in
excess of 32 bits. For example:

There are many 32-bit systems that allow more than 2^32 bytes of
memory to be used, but none that I know of that allow a block that big
to be accessed through calls to C standard allocation functions.

>http://support.microsoft.com/kb/274750

And as far as I can see, this is not a counter-example.

>Why? The calloc() call has two parameters. There is no reason that
the total allocation cannot exceed the value of a size_t.

Obviously if that were the case, pointers would have to be larger than
32 bits. Even if Jacob is not precise, he obviously means systems
with 32-bit pointers. Constantly attacking for imprecision, when he's
trying to make a serious point about faulty implementations , seems
pointless.

-- Richard
--
"Considerat ion shall be given to the need for as many as 32 characters
in some alphabets" - X3.4, 1963.

[jacob navia]

Richard Tobin a écrit :

In article <11************ *********@v33g2 000cwv.googlegr oups.com>,
user923005 <dc*****@connx. comwrote:

>>BTW, there are definitely 32 bit systems that will allocate memory in
excess of 32 bits. For example:

There are many 32-bit systems that allow more than 2^32 bytes of
memory to be used, but none that I know of that allow a block that big
to be accessed through calls to C standard allocation functions.

>>http://support.microsoft.com/kb/274750

And as far as I can see, this is not a counter-example.

SQL server surely doesn't use calloc for allocating 32GB of
RAM, since it needs 48 bit pointers (16:32 pointers with
a segmented architecture like in the msdos days of 16:16)

>

>>Why? The calloc() call has two parameters. There is no reason that
the total allocation cannot exceed the value of a size_t.

Obviously if that were the case, pointers would have to be larger than
32 bits. Even if Jacob is not precise, he obviously means systems
with 32-bit pointers. Constantly attacking for imprecision, when he's
trying to make a serious point about faulty implementations , seems
pointless.

Thanks. this is precisely what I meant.

sizeof returns a size_t. If an object larger than 4GB
could exist then size_t can't be 32 bits.

[Spiros Bousbouras]

jacob navia wrote:

Spiros Bousbouras a écrit :

jacob navia wrote:

>As a consequence of a heavy discussion in another thread,
I wrote this program:
#include <stdlib.h>
int main(void)
{
char *p=calloc(65521 ,65552);
if (p)
printf("BUG!!!! \n");
}

What bug are you claiming exists ? If you're
saying that only 65296 bytes have been allocated
how do you know ?

if size_t is 32 bits, it would be highly surprising that
an allocator returns an object of size 33 bits at least.

You are just saying here that such a system *could* exist.
This doesn't make it any more real.

OK. In those hypothetical systems my test would fail.

But how do you know that the systems that you tried
are not one of those "hypothetic al" systems ?

>The multiplication will give an erroneus result
in 32 bit size_t systems. I have tested this
in several systems with several compilers.

How do you know what the calloc implementation does
upon receiving the arguments 65521 and 65552 ? It is
reasonable to assume that it multiplies them but how
do you know that the result of the multiplication is limited
to 32 bits ? I don't see what relevance the width of size_t
has.

because if an object of size 4GB could exist, sizeof
(that returns a size_t) would not work.

You could not declare an array which has size 4GB
but it doesn't necessarily impose restrictions on what
calloc can return. But I agree it would be surprising.

>

>Maybe people with other compilers/implementations
would compile this program and send the results?

I can't be bothered to be honest because I don't see the
point.

You can't be bothered to be honest?

I hope you mean

To be honest, I can't be bothered...

:-)

I meant "I can't be bothered , to be honest , ..."
;-)

>
BUG!!!!

As far as I can see , even if calloc returns a non NULL
pointer you can't know how much memory has been allocated
unless you have seen the code of the calloc implementation.
Note that even if you know for sure that the system cannot
allocate 65521*65552 bytes and calloc returns non NULL you
still cannot be sure that it is calloc which has the bug. It may
be that calloc uses some platform specific system call to get
the memory and it is the system call which has the bug.

Excuse me but what calloc does is not measured. I just
test the result. If it is the system's fault or whatever
this is not relevant.

This answers my second point but not my first , that your
test doesn't tell you for sure how much memory has been
allocated.

[Richard Tobin]

In article <45************ ***********@new s.orange.fr>,
jacob navia <ja***@jacob.re mcomp.frwrote:

>Maybe people with other compilers/implementations
would compile this program and send the results?

To my surprise, it has the bug on MacOS X Tiger with gcc 4.0.1 (x86)
and 4.0.0 (ppc). It also has the bug on Dragonfly 1.2 with gcc 2.95,
but that's a rather out-of-date system. It behaves correctly on
FreeBSD 6.1 x86 with gcc 3.4.4. I have a vague recollection of seeing
discussions about fixing this very problem in *BSD within the last
year or so.

In all cases I think the version of gcc is irrelevant; rather it is
the version of the C library that matter and the OS version determines
that. (The situation is likely to be more complicated for Linux,
where I believe people mix and match kernel and glibc versions.) All
these systems have sizeof(char *) == 4, sizeof(size_t) == 4, and
CHAR_BIT == 8.

-- Richard
--
"Considerat ion shall be given to the need for as many as 32 characters
in some alphabets" - X3.4, 1963.

[Spiros Bousbouras]

Richard Tobin wrote:

In article <11************ *********@v33g2 000cwv.googlegr oups.com>,
user923005 <dc*****@connx. comwrote:

Why? The calloc() call has two parameters. There is no reason that
the total allocation cannot exceed the value of a size_t.

Obviously if that were the case, pointers would have to be larger than
32 bits. Even if Jacob is not precise, he obviously means systems
with 32-bit pointers. Constantly attacking for imprecision, when he's
trying to make a serious point about faulty implementations , seems
pointless.

Well it wasn't obvious to me from his first post. Even
if some people here were out to get him, so to speak, he
could still try to offer them less possibilities by being
precise and clear.

[Richard Tobin]

In article <45************ ***********@new s.orange.fr>,
jacob navia <ja***@jacob.re mcomp.frwrote:

>sizeof returns a size_t. If an object larger than 4GB
could exist then size_t can't be 32 bits.

To be pedantic, that doesn't quite follow. You can't apply sizeof to
the block returned by calloc(). I'm not sure whether you can legally
have a system in which calloc() can return an object with more bytes
than size_t can count.

-- Richard

--
"Considerat ion shall be given to the need for as many as 32 characters
in some alphabets" - X3.4, 1963.

[Spiros Bousbouras]

jacob navia wrote:

P.S.
I forgot to spell out clearly the objective of this tests.

My objective is to

1) make a list of all the buggy systems that
is here in clc and everybody can see.

2) Make people aware of this bug in their implementations .
(if any)

Ok , now we're getting somewhere.

So the central question is:

If some code contains a call of the form calloc(n,m)
where n*m 2**( sizeof(void*) * CHAR_BIT) then is it
the obligation of the calloc implementation to detect
this and return NULL ?

If the answer to the question is yes then you can say
that the implementations which don't do it are buggy.

>From the practical point of view , if I ever needed to

write something like calloc(n,m) where n*m >= 2**31
on a 32 bits system , alarm bells would start to ring
in my mind and I would wonder if it would work. I
would probably write some small test programme to
see if the allocation would succeed. I'm far from infallible
but my view is that a competent C programmer should
be alert to such issues and not rely on protection from
the calloc implementation *even* if the standard demands
that calloc detects this sort of thing. That's because this
is the kind of thing that an implementor can get wrong.
So I would write a test programme which not only tests
that the allocation succeeded but also assign some random
values to the allocated memory and make sure that everything
went ok.

[Keith Thompson]

jacob navia <ja***@jacob.re mcomp.frwrites:
[...]

But this is a well known tactic.

Since I present the code, I do the effort of making all those
measurements, it is so easy to just sit by and say

"A system with 32 bit size_t could allocate more than 4GB
memory. A system like that COULD exist."

So WHAT?

Can you please tell me where that system is????

If not, just do not participate. Thanks for your
correction. A system where this test gives a false
positive COULD exist. It is conceivable.

So you acknowledge that a false positive could exist, but when someone
points it out you ask him not to participate in the discussion.

Try being just a little bit less arrogant.

--
Keith Thompson (The_Other_Keit h) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.