gcc knows about malloc()

[James Dow Allen]

The gcc compiler treats malloc() specially! I have no
particular question, but it might be fun to hear from
anyone who knows about gcc's special behavior.

Some may find this post interesting; some may find it
off-topic or confusing. Disclaimers at end.

The code samples are intended to be nearly minimal
demonstrations. They are *not* related to any
actual application code.

(1)
gcc assumes that malloc()'s return value is not an
alias for another pointer. This can be confirmed by compiling
and running the following code twice, once with a
-Dgetnode=malloc
option. (The special behavior will not be observed if
getnode is declared static, or the '-O' option is omitted.)

Compile and run the following code, first with
gcc -Wall -O
then with
gcc -Wall -O -Dgetnode=malloc

=== Begin sample code
#include <stdio.h>

struct node {
struct node *next;
} A[10];

void *getnode(int x)
{
return &A[x];
}

/* Insert A[x] into 3rd list slot */
void insert(x)
{
struct node *newp;
struct node *head = A;

newp = getnode(x);
newp->next = head->next->next;
/* will compiler recompute head->next, or use old value? */
head->next->next = newp;
}

int main(int argc, char **argv)
{
/* Create a circular list 0 -> 1 -> 2 -> 3 -> 0 */
A->next = A;
insert(1);
insert(3);
insert(2);
printf("Third element in list is A[%d]\n", A->next->next - A);

/* Next will "damage" the list, but should be deterministic! */
insert(0);
printf("Third element in list is A[%d]\n", A->next->next - A);
return 0;
}
=== End sample code

(2)
Speaking of malloc(), a misleading answer is sometimes given in
this ng, when coders neglect to include <stdlib.h>, and then
attempt to type-cast malloc().

While omitting the system header file is a mistake, strictly
speaking the resultant problems aren't caused by casting the
malloc() function incorrectly, but by failing to cast it at all!

In an expression such as
foo = (void *)malloc(TWELVE _MILLION);
it is malloc's *return value* that is cast, not malloc() itself.
This is another case where C syntax has different effect in
expression vs declaration.

To avoid the problem (assuming the <stdlib.h> file has been
accidentally erased, but one knows malloc's correct declaration)
one could write:
void *malloc();
...
foo = malloc(TWELVE_M ILLION);

Or even (if your employer docks your pay for explicitly declaring
library functions) :

if (0) malloc(); /* tell gcc that malloc is a function */
foo = ((void *(*)())malloc)( TWELVE_MILLION) ;

(Please forgive the peculiar way to declare that malloc is a function!
It seemed like an approach for "minimum compilable example"!)

Please don't accuse me of encouraging perversion (!), but some
prior discussions *were* misleading.

(3)
While playing around, I discovered more bizarre behavior by
gcc (GNU C version 3.2) concerning malloc().

Compile the following code, e.g. with
gcc -c

=== Begin sample code
struct foo { int x; };

int main(int argc, char **argv)
{
if (0) malloc(); /* tell gcc malloc() is a function */
return (((struct foo(*)())malloc )(sizeof(struct foo))).x;
}
=== End sample code

When I compile it, I get "Segmentati on fault"!
Before flaming, please note:
It never occurred to me to *run* this bizarre faulty program;
the "Segmentati on fault" occurs during compilation.

Disclaimers:

(a) No one is advocating faulty type-casting, or faulty
replacements for malloc().

(b) No one is claiming 'gcc' is defective. To the contrary,
sample (1) demonstrates a clever efficiency that gcc is able to
apply due to malloc()'s semantics. (The fault in (3), I imagine,
would be easy to avoid and FSF probably would if it knew of it.)

(c) This post may be off-topic because gcc isn't standard C,
and some readers use other compilers. But this is a general forum
for issues relating to C.

(BTW, why *doesn't* everyone use the gcc compiler?)

(d) Standard C specifies what a compiler *may* do and what it
*must* do, not what it *might* do. Still, I wonder if the Standard
specifically condones gcc's optimization in (1).

(e) Both code samples above *violate* the rules of Standard C,
and instead of the "faulty" optimization or Segmentation fault,
]> ... it would be proper for a compliant Standard C compiler
]> to trash [my] hard disk, or launch a nuclear strike....
Such comments are amusing, but do become repetitious.
Frankly, anyone who has no interest in the above C fragments
beyond observing that they are non-compliant, strikes me
as lacking imagination.

For code sample (1) it would be easy to produce Standard C
code where the interesting difference emerges from a timing
measurement or examining object code, but it seemed more
straightforward to demonstrate gcc's clever optimization
with a substituted malloc() that invalidates the optimization.
James Dow Allen

- - - - - - - - - - -

Tell us, Senator, what is your opinion of alcohol?

``Well, if by alcohol you mean that hearty spirit which brings laughter
and
livelihood to social gatherings, which lubricates the gears of
business
transactions, which makes life's hardships easier to bear, then I say
hurrah for it, and I support its legal sale and consumption.

``However, if by alcohol you mean that evil essence which robs the
family
budget, which is a toxin to the body and the mind, which causes men
to
assault one another and to fail in their professions, then I say fie
on
it, and I will work with all my power to prevent its sale and
consumption.''

- - - - - - - - - - -

[jjf]

James Dow Allen wrote:

Speaking of malloc(), a misleading answer is sometimes given in
this ng, when coders neglect to include <stdlib.h>, and then
attempt to type-cast malloc().

While omitting the system header file is a mistake, strictly
speaking the resultant problems aren't caused by casting the
malloc() function incorrectly, but by failing to cast it at all!
No, the resultant problems are caused by failing to include the header.
In an expression such as
foo = (void *)malloc(TWELVE _MILLION);
it is malloc's *return value* that is cast, not malloc() itself.
No. Without a prior correct declaration, it's the location in which
integer return values get placed which gets cast. This may or may not
be where malloc() returns its value. It's obviously "not malloc()
itself" though - what does that even mean?
This is another case where C syntax has different effect in
expression vs declaration.
What do you mean?
To avoid the problem (assuming the <stdlib.h> file has been
accidentally erased, but one knows malloc's correct declaration)
one could write:
void *malloc();
...
foo = malloc(TWELVE_M ILLION);
If that happens to be the correct declaration in your implementation,
though it's unlikely to be the case in any modern implementation.
Or even (if your employer docks your pay for explicitly declaring
library functions) :

if (0) malloc(); /* tell gcc that malloc is a function */
foo = ((void *(*)())malloc)( TWELVE_MILLION) ;

(Please forgive the peculiar way to declare that malloc is a function!
You don't declare that malloc is a function anywhere here.
It seemed like an approach for "minimum compilable example"!)
The correct approach would be to include the appropriate header.
Please don't accuse me of encouraging perversion (!), but some
prior discussions *were* misleading.

Possibly, but nowhere near as misleading as the remarkable stuff you
wrote above.

[Mark McIntyre]

On 12 Jun 2006 00:57:37 -0700, in comp.lang.c , "James Dow Allen"
<jd*********@ya hoo.com> wrote:

Speaking of malloc(), a misleading answer is sometimes given in
this ng, when coders neglect to include <stdlib.h>, and then
attempt to type-cast malloc().

While omitting the system header file is a mistake, strictly
speaking the resultant problems aren't caused by casting the
malloc() function incorrectly, but by failing to cast it at all!
No. If you omit the header, the compiler must assume malloc returns an
int. It is a common mistake to think that casting the int to a pointer
"fixes" this. Not so. As a trivial counterexample, some real-world
operating systems use different registers for pointers and integers,
and casting the garbage in the integer register won't help.
In an expression such as
foo = (void *)malloc(TWELVE _MILLION);
it is malloc's *return value* that is cast, not malloc() itself.
Of course, but then once compiled, malloc() is a bunch of machine
language instructions, it would be fairly meaningless to cast that !
This is another case where C syntax has different effect in
expression vs declaration.
This doesn't make any sense - what do you mean?
To avoid the problem (assuming the <stdlib.h> file has been
accidentally erased, but one knows malloc's correct declaration)
one could write:
void *malloc();
This is guaranteed by the Standard, section 7.1.4
if (0) malloc(); /* tell gcc that malloc is a function */
foo = ((void *(*)())malloc)( TWELVE_MILLION) ;
this is NOT guaranteed, is exceptionally bad practice, and leads to
undefined behaviour. On many implementations , such code would lead to
unexplained crashes at runtime.
Please don't accuse me of encouraging perversion (!), but some
prior discussions *were* misleading.
I think you have some misunderstandin gs, probably based on only
experience on 'friendly' hardware.
While playing around, I discovered more bizarre behavior by
gcc (GNU C version 3.2) concerning malloc().
This is offtopic here, you need to take C bugs to the gcc newsgroups
and fora.
When I compile it, I get "Segmentati on fault"!
so you found a compiler bug - please report it to gcc via their usual
method.
(c) This post may be off-topic because gcc isn't standard C,
and some readers use other compilers. But this is a general forum
for issues relating to C.
Please read the Welcome message, and about a zillion messages
discussion topicality
(BTW, why *doesn't* everyone use the gcc compiler?)

Because fish don't have feet.

--
Mark McIntyre

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan

[Jordan Abel]

Mark McIntyre <ma**********@s pamcop.net> wrote:

On 12 Jun 2006 00:57:37 -0700, in comp.lang.c , "James Dow Allen"
<jd*********@ya hoo.com> wrote:

if (0) malloc(); /* tell gcc that malloc is a function */
foo = ((void *(*)())malloc)( TWELVE_MILLION) ;

this is NOT guaranteed, is exceptionally bad practice, and leads to
undefined behaviour. On many implementations , such code would lead to
unexplained crashes at runtime.

Where's the undefinedd behavior? Code that's not executed and contains
no syntax errors doesn't cause undefined behavior. All function pointer
types are supposed to have the same representation. It's not clear
what's undefined here.

[SuperKoko]

Jordan Abel wrote:

Mark McIntyre <ma**********@s pamcop.net> wrote:

On 12 Jun 2006 00:57:37 -0700, in comp.lang.c , "James Dow Allen"
<jd*********@ya hoo.com> wrote:

if (0) malloc(); /* tell gcc that malloc is a function */
foo = ((void *(*)())malloc)( TWELVE_MILLION) ;
this is NOT guaranteed, is exceptionally bad practice, and leads to
undefined behaviour. On many implementations , such code would lead to
unexplained crashes at runtime.

Where's the undefinedd behavior?

When the pointer-to-function is used to call the function.

In the C99 standard (the C89 standard has similar statments, see
below):

6.3.2.3-8 says
"
A pointer to a function of one type may be converted to a pointer to a
function of another
type and back again; the result shall compare equal to the original
pointer. If a converted
pointer is used to call a function whose type is not compatible with
the pointed-to type,
the behavior is undefined.
"
void* and int are not compatible.
Code that's not executed and contains
no syntax errors doesn't cause undefined behavior. AFAIK "foo = ((void *(*)())malloc)( TWELVE_MILLION) " will be evaluated.
All function pointer
types are supposed to have the same representation. Could you quote a paragraph from the standard?

I have only found:
6.7.5.3-15
"
For two function types to be compatible, both shall specify compatible
return types.
Moreover, the parameter type lists, if both are present, shall agree in
the number of
parameters and in use of the ellipsis terminator; corresponding
parameters shall have
compatible types. [...]
"
Since void* and int are not compatible types, int() and void*() are
incompatible function types.

6.7.5.1-2
"
For two pointer types to be compatible, both shall be identically
qualified and both shall
be pointers to compatible types.
"
Thus int(*)() and void*(*)() are not compatible pointer types.

Note also that "same representation" doesn't mean "compatible types".
For instance:
6.7.3-9
"
For two qualified types to be compatible, both shall have the
identically qualified version
of a compatible type; the order of type qualifiers within a list of
specifiers or qualifiers
does not affect the specified type.
"
Thus const int and int are not compatible types while they have the
same representation.
6.2.5-25
"
Any type so far mentioned is an unqualified type. Each unqualified type
has several
qualified versions of its type,38) corresponding to the combinations of
one, two, or all
three of the const, volatile, and restrict qualifiers. The qualified or
unqualified
versions of a type are distinct types that belong to the same type
category and have the
same representation and alignment requirements.39 ) A derived type is
not qualified by the
qualifiers (if any) of the type from which it is derived.
"
Note also that, since int() and void*() are not compatible types,
pointers to these function types can have a different representation:
6.5.5-26
"
A pointer to void shall have the same representation and alignment
requirements as a
pointer to a character type.39) Similarly, pointers to qualified or
unqualified versions of
compatible types shall have the same representation and alignment
requirements. All
pointers to structure types shall have the same representation and
alignment requirements
as each other. All pointers to union types shall have the same
representation and
alignment requirements as each other. Pointers to other types need not
have the same representation or alignment requirements.
"

Now, I see that your code uses C89, not C99 (implicit declaration have
been banned in C99):
Anyway, I can quote equivalent statements from the C89 standard:
3.3.4 (Cast operators)
"
A pointer to a function of one type may be converted to a pointer to a
function of another type and back again; the result shall compare equal
to the original pointer. If a converted pointer is used to call a
function that has a type that is not compatible with the type of the
called function, the behavior is undefined.
"

3.5.4.3 Function declarators (including prototypes)
"
For two function types to be compatible, both shall specify compatible
return types.63 Moreover, the parameter type lists, if both are
present, shall agree in the number of parameters and in use of the
ellipsis terminator; corresponding parameters shall have compatible
types.
"
3.5.4.1 Pointer declarators
"
For two pointer types to be compatible, both shall be identically
qualified and both shall be pointers to compatible types.
"

Furthermore, even the non-evaluated expression "malloc()" will give
another undefined behavior (which might give a link-time error for
instance).
3.3.2.2 Function calls
"
If the expression that precedes the parenthesized argument list in a
function call consists solely of an identifier, and if no declaration
is visible for this identifier, the identifier is implicitly declared
exactly as if, in the innermost block containing the function call, the
declaration

extern int identifier();

appeared
"

3.5 Declarations (or 6.7-4 in the C99 standard)
"
All declarations in the same scope that refer to the same object or
function shall specify compatible types.
"

3.1.2.6 Compatible type and composite type (6.2.7-2 in the C99
standard)
"
All declarations that refer to the same object or function shall have
compatible type; otherwise the behavior is undefined.
"
It's not clear
what's undefined here.

There are two undefined behaviors : one for each line.

[Morris Dovey]

James Dow Allen (in
11************* ********@i40g20 00...legro ups.com) said:

| The gcc compiler treats malloc() specially! I have no
| particular question, but it might be fun to hear from
| anyone who knows about gcc's special behavior.

You seem surprised. Gcc implementations can sometimes be enhanced to
take advantage of target processor architectural features. Consider it
an advantage (and/or disadvantage) of open-source software.

--
Morris Dovey
DeSoto Solar
DeSoto, Iowa USA
http://www.iedu.com/DeSoto

[Thomas J. Gritzan]

SuperKoko schrieb:

Jordan Abel wrote:

Mark McIntyre <ma**********@s pamcop.net> wrote:

On 12 Jun 2006 00:57:37 -0700, in comp.lang.c , "James Dow Allen"
<jd*********@ya hoo.com> wrote:

if (0) malloc(); /* tell gcc that malloc is a function */
foo = ((void *(*)())malloc)( TWELVE_MILLION) ;
this is NOT guaranteed, is exceptionally bad practice, and leads to
undefined behaviour. On many implementations , such code would lead to
unexplained crashes at runtime.

Where's the undefinedd behavior?

When the pointer-to-function is used to call the function.

In the C99 standard (the C89 standard has similar statments, see
below):

6.3.2.3-8 says
"
A pointer to a function of one type may be converted to a pointer to a
function of another
type and back again; the result shall compare equal to the original
pointer. If a converted
pointer is used to call a function whose type is not compatible with
the pointed-to type,
the behavior is undefined.
"
void* and int are not compatible.

[snip]

But the converted pointer type is void*(*)() and the pointed-to type is
not int(*)() but the type malloc is implemented in the library, and
malloc does not return int but void*. So where is the undefined behaviour?

Thomas

[Keith Thompson]

"James Dow Allen" <jd*********@ya hoo.com> writes:
[...]

Speaking of malloc(), a misleading answer is sometimes given in
this ng, when coders neglect to include <stdlib.h>, and then
attempt to type-cast malloc().

While omitting the system header file is a mistake, strictly
speaking the resultant problems aren't caused by casting the
malloc() function incorrectly, but by failing to cast it at all!
No, the undefined behavior is caused by calling malloc() without a
visible prototype. Casting the result may change the way that
undefined behavior is manifested, and may prevent the compiler from
warning you about it.
In an expression such as
foo = (void *)malloc(TWELVE _MILLION);
it is malloc's *return value* that is cast, not malloc() itself.
Of course.
This is another case where C syntax has different effect in
expression vs declaration.
I'll join several other people in not understanding what you mean by
this. If you're drawing a parallel between
(void*)malloc(T WELVE_MILLION)
and
void *malloc(size_t size);
that's not the same syntax.

This is a case where one piece of C syntax in an expression has a
different effect than a different piece of C syntax in a declaration.
To avoid the problem (assuming the <stdlib.h> file has been
accidentally erased, but one knows malloc's correct declaration)
one could write:
void *malloc();
...
foo = malloc(TWELVE_M ILLION);
One could, but one would very likely invoke undefined behavior.
Given
void *malloc();
the compiler doesn't know the type, or even number, of malloc()'s
argument(s). Unless TWELVE_MILLION happens to be promoted to
a type compatible with size_t, you're lying to the compiler.
Or even (if your employer docks your pay for explicitly declaring
library functions) :

if (0) malloc(); /* tell gcc that malloc is a function */
foo = ((void *(*)())malloc)( TWELVE_MILLION) ;
Undefined behavior.

By converting malloc (a function name which has been implicitly
converted to a pointer-to-function) to type (void *(*)(), you're
trying to avoid the problem with malloc()'s return type, but you're
still lying to the compiler. The compiler assumes that malloc() is a
function returning int. You're *assuming* that this is the case (it
isn't), and converting the bogus pointer to a pointer to function
returning void*. It's the same problem you get with the simpler
(void*)malloc(T WELVE_MILLION)
where you take something that's really a void*, forcing the compiler
to assume it's an int, and converting it from int to void*.

You're probably assuming that all pointer-to-function types have the
same representation. That's probably true in most or all
implementations , but it's not guaranteed. The standard guarantees
that any pointer-to-function type can be converted to any other
pointer-to-function type and back again, yielding the original pointer
value -- but the conversions don't necessarily just reinterpret the
bits of the pointer. C99 6.2.5 specifically guarantees that all
pointers to structure types have the same representation and alignment
requirements; likewise for pointers to union types. It makes no such
guarantess for pointers to function types.

In all your function declarations and casts, you're omitting the
parameter type. If you must provide your own declaration of malloc()
for some reason, the correct declaration is
void *malloc(size_t) ;
or
void *malloc(size_t size);

There is, of course, no good reason to use any method other than
#include <stdlib.h>
(unless your compilation system is broken and you're unable to repair
or replace it -- but if it's that badly broken it's not clear that
even a correct declaration will be an effective workaround).

[snip]
Please don't accuse me of encouraging perversion (!), but some
prior discussions *were* misleading.
Some of them undoubtedly were, but not all of them. I think you've
just misunderstood them.

[...] When I compile it, I get "Segmentati on fault"!
Before flaming, please note:
It never occurred to me to *run* this bizarre faulty program;
the "Segmentati on fault" occurs during compilation. [...] (b) No one is claiming 'gcc' is defective. To the contrary,
sample (1) demonstrates a clever efficiency that gcc is able to
apply due to malloc()'s semantics. (The fault in (3), I imagine,
would be easy to avoid and FSF probably would if it knew of it.)
<OT>
If gcc gets a segmentation fault during compilation, regardless of how
bad the compiled code might be, of course it's defective. In other
words, you've found a bug. You should report it to the gcc
maintainers. See <http://gcc.gnu.org/> for information on how to do
this.
</OT>
(BTW, why *doesn't* everyone use the gcc compiler?)

On some platforms, gcc doesn't exist. On others, another compiler
generates better code. For those who need C99, there are compilers
that support it better than gcc does. Many C compilers have features
that gcc lacks. I'm sure there are other reasons.

Why *should* everyone use the same compiler? The point of a standard
is that it's a contract between the implementer and the programmer; if
I write conforming code, I can use any conforming compiler I like.

--
Keith Thompson (The_Other_Keit h) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

[Martin Ambuhl]

James Dow Allen wrote:

Speaking of malloc(), a misleading answer is sometimes given in
this ng, when coders neglect to include <stdlib.h>, and then
attempt to type-cast malloc().

While omitting the system header file is a mistake, strictly
speaking the resultant problems aren't caused by casting the
malloc() function incorrectly, but by failing to cast it at all!
This is completely nuts. Without a suitable declaration of malloc() in
scope, the return value is assumed to be an int. Casting an int to a
pointer is not guaranteed to be meaningful. Your last sentence has no
intelligible meaning, since there is no way that castint that int to a
pointer helps at all.

In an expression such as
foo = (void *)malloc(TWELVE _MILLION);
it is malloc's *return value* that is cast, not malloc() itself.

Yeah. And who doesn't know that? Apparently, it's you, to judge from
your own nonsensical use of "casting the malloc() function" above.

[Keith Thompson]

Martin Ambuhl <ma*****@earthl ink.net> writes:

James Dow Allen wrote:

Speaking of malloc(), a misleading answer is sometimes given in
this ng, when coders neglect to include <stdlib.h>, and then
attempt to type-cast malloc().
While omitting the system header file is a mistake, strictly
speaking the resultant problems aren't caused by casting the
malloc() function incorrectly, but by failing to cast it at all!

This is completely nuts. Without a suitable declaration of malloc() in
scope, the return value is assumed to be an int. Casting an int to a
pointer is not guaranteed to be meaningful.

You understate the problem. The int being converted is not guaranteed
to be meaningful, whether you convert it or not. The call itself
invokes undefined behavior before you even look at the result.

(On some systems, where int and void* happen to be the same size, and
function results of those types happen to be returned by the same
mechanism, casting the phony int result to a pointer type might happen
to "work". This is merely one of the infinitely many possible
consequences of undefined behavior. In fact, it's the worst one,
since the compiler is failing to tell you about your error.)

--
Keith Thompson (The_Other_Keit h) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.