is NULL-checking redundant in accessor-functions?

[Mikhail Teterin]

Hello!

Consider the following simple accessor function:

typedef struct {
int i;
char name[32];
} MY_TYPE;

const char *
getName(const MY_TYPE *p)
{
return p->name;
}

I always assumed, that checking the passed pointer `p' against NULL is
redundant, because, if it happens to be NULL at run-time (as a result of
programming error), trying to dereference it would result in SEGFAULT
anyway.

I'm now looking at a situation, where an accessor function, instead of
crashing, returns the offset of the field `name' from 0 (0x4 in the above
example). As if there is a legal value 0x00000000 at the NULL pointer...

Any comments? This is on a Sparc machine under Solaris-9. Thanks!

-mi
--
comp.lang.c.moderated - moderation address: cl**@plethora.net

[Jens M Andreasen]

On Fri, 21 Jan 2005 03:59:55 +0000, Chris Torek wrote:

Jens M Andreasen wrote:
free(p);
if(p) // Segfault here?

On Sat, 15 Jan 2005 00:38:25 +0000, Douglas A. Gwyn wrote:

Quite possibly a segmentation (mapping) violation. The value of p (not
just *p) becomes indeterminate after the call to free. free might map
that range of addresses out of the program's virtual address space.
Most implementations of free don't, but it is allowed.

In article <cl****************@plethora.net> Jens M Andreasen
<[e-mail]> wrote:

What you say is correct, but in context wrong. So ...

Mmmm ... no! You would need a 'move' to get there and not a mere 'add'.
And again, it is *p that becomes indeterminate, not p.

You are wrong and Doug Gwyn is right, at least formally speaking.

OK, I'll buy 'formally speaking' ...
(Implementations that actually catch the "if (p)" error after the free(p)
call are somewhere between rare and nonexistent.)

Perhaps even touching nonexistent?
This issue is quite tricky, but it may become clear if you can answer me
this question: what is the value represented by the following bit pattern?

11000011 11110101 01001000 01000000
(in hex: 0xc3 0xf5 0x48 0x40)

I will give you two possible answers (though there are more):

3.14 (a float)
1078523331 (an int)

and will I tell you that at least one of them is correct (and the machine
in question has a Pentium III CPU).

Depends on what register your bitpattern resides in. Given the option
between float and int, my guess is one of the SSE registers?

The answer depends on what instruction you are to issue next, no?

The latencies though, between SSE and adress registers prohibits practical
use of SSE to calculate offsets.

(and yes, I am aware of the wonderful weirdness of Intel pointer
bitpatterns, but even in asm they are hard to actually get at)
How will you decide whether those bytes represent a float or an int?
More importantly, whether I tell you the truth or lie -- it does not
really matter which -- how can it be that this same bit pattern
apparently represents two different numbers?

The answer lies in the *interpretation* of the bits. The bits themseves
are just bits. By themselves they have no deeper meaning. It is only
when I tell you "these are the bits of a float" that they *mean* 3.14,
or "these are the bits of an int" that they *mean* 1078523331.

OK ...
What if I tell you, instead, that the machine is a Deathstation 9900,
and the bits represent a pointer? What is the value of that pointer?
How will you interpret them?
I wouldn't ...

You are now saying that:

free(p)
if(p)
...

.... *could* result in a cast to float whith a possible raise of an
exception (NaN or whatever)

That means free() would deliberately have to set the register where it
found p to an evil pattern. And if p was on the stack (or some such),
nobody would even ever notice?

This is longwinded.

The value of p was returned by malloc()
Originally, yes.

and is valid for the implemntation of p.

Valid until free()d, yes.

Formally ...
But what if free() changed something in the hardware so that the very
same -- unchanged -- bits in p, that *used* to mean "the contents of the
memory bank located in the south wing of the building", now mean "please
trigger the nuclear warhead"? (This is, after all, the Deathstation.
:-) )
OK, on the Deathstation perhaps. I think you have now touched the
unimplemented part of the world, no?
;-)

There is nothing in the C language that prohibits free() from changing
the way the machine interprets the bits. If certain pointer bit
patterns were like "int" before (always valid), and are like "float" now
(have trap representations), the simple act of testing the value of "p"
can fault.
I would buy this if we have had a construct like:

p = free(p);

.... but this is not the case.


*This* is how p can become invalid after free(): not by having its
stored bit pattern change, but rather by having the *meaning* of those
bits change.

Ehrmm, p is still declared as a pointer and not a float. I assume my
compiler to still evaluate it as a pointer. How then can the meaning of
the (unchanged) bitpattern have changed (in an implementation other than
the imaginary evil Deathstation?)
mvh // Jens M Andreasen
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Jens M Andreasen]

On Fri, 21 Jan 2005 04:02:43 +0000, Stephen Sprunk wrote:

p is indeterminate after free(p), not just *p. Any operation on p, other
than storing a new value in it, is undefined. IIRC, AS/400 systems in
particular will segfault (or whatever they call it) on "if (p)" or "q=p",
and that's perfectly legal -- it's just not common.

OK, now we are getting somewhere.

Can anybody with current access to an AS/400 confirm that if(p) indeed
fails after free?

mvh // Jens M Andreasen
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Jens M Andreasen]

On Fri, 21 Jan 2005 04:02:14 +0000, Richard Bos wrote:

If, OTOH, the segment that p pointed to has just been deallocated, it
could result in

LD ADDR1, null (ok, ADDR1 contains a null pointer) LD ADDR2, (p) (not
ok. The processor detects an attempt to load a
pointer into ADDR2 which is not valid, and causes a trap. The OS
reacts to this by terminating your program with extreme prejudice.)
This is perhaps the best description of the situation so far

I would have thought that it is the following MOVE that raises the
exception. Afterall we haven't done anything nasty (yet.)

But the implementation, as described, makes sense in context of
implementing an efficient MMU.

Realworld examples, please?
mvh // Jens M Andreasen

Richard

--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Dave Hansen]

On Fri, 21 Jan 2005 04:02:41 -0000, "Stephen Sprunk"
<st*****@sprunk.org> wrote:

[...]

That's the biggest problem programmers seem to have with C; undefined
behavior is exactly that -- undefined.

An implementation is free to do whatever it wants with UB, and it's not
required to document it nor be consistent. It might silently fail one time,
and the next time it might call a hitman on you -- both are perfectly legal
since the behavior is _undefined_.

Even worse, it might do exactly what you expect. Every time. Leading
one to the (incorrect) conclusion that the behavior in that case isn't
undefined. "It works on _my_ system, so it _must_ be OK."

Regards,

-=Dave
--
Change is inevitable, progress is not.
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Christian Bau]

In article <cl****************@plethora.net>,
Chris Torek <no****@torek.net> wrote:

There is nothing in the C language that prohibits free() from
changing the way the machine interprets the bits. If certain
pointer bit patterns were like "int" before (always valid), and
are like "float" now (have trap representations), the simple act
of testing the value of "p" can fault.

*This* is how p can become invalid after free(): not by having its
stored bit pattern change, but rather by having the *meaning* of
those bits change.

There is another possibility: An optimising compiler can be quite
clever. For example, an optimising compiler can know that the result of
malloc (), if it is not a null pointer, cannot equal any other pointer.
Or a compiler can know that a variable containing a pointer that is
free'd has an indeterminate value after the call to free (), and
therefore cannot be equal to any other pointer.

Example:

void test (void *p)
{
if (p != NULL) {
void* q = p;
free (p);
if (p != q) printf ("Gotcha!!!\n");
}
}

The compiler can _know_ that the value of p is indeterminate after the
call free (p), and therefore p != q must be true. (A compiler can also
_know_ that p != q must be false, or it can compare p and q and anything
can happen. That is undefined behavior for you).
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Christian Bau]

In article <cl****************@plethora.net>,
"Stephen Sprunk" <st*****@sprunk.org> wrote:

That's the biggest problem programmers seem to have with C; undefined
behavior is exactly that -- undefined.

An implementation is free to do whatever it wants with UB, and it's not
required to document it nor be consistent. It might silently fail one time,
and the next time it might call a hitman on you -- both are perfectly legal
since the behavior is _undefined_.

And if you don't believe that: Most so-called computer viruses take
advantage of undefined behavior in C programs; a relatively common
consequence of undefined behavior is that your computer uses its modem
to dial a phone sex line which runs up a phone bill of a few hundred
pound in short time.
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Keith Thompson]

Jens M Andreasen <[e-mail]> writes:

On Fri, 21 Jan 2005 04:00:00 +0000, Keith Thompson wrote:

Jens M Andreasen <[e-mail]> writes:

It is not my intention to be rude, but I just have not the imagination
to figure out what you mean? Chapter and verse please?

I already provided chapter and verse in a previous article. See
<http://groups-beta.google.com/group/comp.lang.c.moderated/msg/f457a236c0af9e0e>

Yes I read the article, but it appeared to be on if(*p) which I believe is
different from if(p)

You're right, my mistake. In the referenced article I was explaining
that the undefined behavior caused by an attempt to dereference a null
pointer doesn't necessarily cause a trap and terminate the program.
I shall continue my crusade against 'p == NULL considered dangerous!'
We're not necessarily saying it's dangerous; we're saying the standard
doesn't guarantee that it isn't dangerous.

To re-state the context:

int *p;
p = malloc(sizeof *p); /* assume malloc() succeeds */
free(p);
if (p == NULL) /* undefined behavior here */
{ ... }

I think this has been explained at length in this thread.

Again, undefined behavior is behavior for which the standard imposes
no requirements. Even if every actual implementation does the same
thing, or even if every *possible* implementation does the same thing,
the behavior is still undefined as far as the standard is concerned.

The comparison (p == NULL) might always evaluate to false; the point
is that the standard doesn't guarantee it. (And in any case, since
the standard doesn't guarantee the behavior, an optimizing compiler
can "cheat" in any way it likes.)
If nothing else, it has entertaining value ...

Um, not really.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Douglas A. Gwyn]

Jens M Andreasen wrote:

Ehrmm, p is still declared as a pointer and not a float. I assume my
compiler to still evaluate it as a pointer. How then can the meaning of
the (unchanged) bitpattern have changed (in an implementation other than
the imaginary evil Deathstation?)

The object is still of pointer type, but it no longer
has the same "value" (meaning) despite the bit pattern
being unchanged. How this can actually happen has been
explained in this thread, more than once.
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Douglas A. Gwyn]

Jens M Andreasen wrote:

I would have thought that it is the following MOVE that raises the
exception. Afterall we haven't done anything nasty (yet.)

Yes you have: you tried to load an unmapped value
into an address register. A trap is highly appropriate.
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Gordon Burditt]

>Ehrmm, p is still declared as a pointer and not a float. I assume my

compiler to still evaluate it as a pointer. How then can the meaning of
the (unchanged) bitpattern have changed (in an implementation other than
the imaginary evil Deathstation?)

How can the usability of a credit card change when it is reported
stolen? The number on the card is unchanged.

Memory maps can change during the life of a process. Segments can
become valid or invalid (and having this happen during calls to
malloc() or free() is not unreasonable). Loading a "large" pointer
with an invalid segment into an i386 segment register will cause a
trap. No dereference is required (you won't get a chance to try).
A NULL pointer is treated as a special case here: you can load the
pointer, but not dereference.

Gordon L. Burditt
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Brian Inglis]

On Sat, 22 Jan 2005 02:47:00 -0000 in comp.lang.c.moderated, Jens M
Andreasen <[e-mail]> wrote:

On Fri, 21 Jan 2005 03:59:55 +0000, Chris Torek wrote:

Jens M Andreasen wrote:
> free(p);
> if(p) // Segfault here?

On Sat, 15 Jan 2005 00:38:25 +0000, Douglas A. Gwyn wrote:
Quite possibly a segmentation (mapping) violation. The value of p (not
just *p) becomes indeterminate after the call to free. free might map
that range of addresses out of the program's virtual address space.
Most implementations of free don't, but it is allowed.

In article <cl****************@plethora.net> Jens M Andreasen
<[e-mail]> wrote:

What you say is correct, but in context wrong. So ...

Mmmm ... no! You would need a 'move' to get there and not a mere 'add'.
And again, it is *p that becomes indeterminate, not p.

You are wrong and Doug Gwyn is right, at least formally speaking.

OK, I'll buy 'formally speaking' ...

(Implementations that actually catch the "if (p)" error after the free(p)
call are somewhere between rare and nonexistent.)

Perhaps even touching nonexistent?

This issue is quite tricky, but it may become clear if you can answer me
this question: what is the value represented by the following bit pattern?

11000011 11110101 01001000 01000000
(in hex: 0xc3 0xf5 0x48 0x40)

I will give you two possible answers (though there are more):

3.14 (a float)
1078523331 (an int)

and will I tell you that at least one of them is correct (and the machine
in question has a Pentium III CPU).

Depends on what register your bitpattern resides in. Given the option
between float and int, my guess is one of the SSE registers?

The answer depends on what instruction you are to issue next, no?

The latencies though, between SSE and adress registers prohibits practical
use of SSE to calculate offsets.

(and yes, I am aware of the wonderful weirdness of Intel pointer
bitpatterns, but even in asm they are hard to actually get at)

How will you decide whether those bytes represent a float or an int?
More importantly, whether I tell you the truth or lie -- it does not
really matter which -- how can it be that this same bit pattern
apparently represents two different numbers?

The answer lies in the *interpretation* of the bits. The bits themseves
are just bits. By themselves they have no deeper meaning. It is only
when I tell you "these are the bits of a float" that they *mean* 3.14,
or "these are the bits of an int" that they *mean* 1078523331.

OK ...

What if I tell you, instead, that the machine is a Deathstation 9900,
and the bits represent a pointer? What is the value of that pointer?
How will you interpret them?

I wouldn't ...

You are now saying that:

free(p)
if(p)
...

... *could* result in a cast to float whith a possible raise of an
exception (NaN or whatever)

That means free() would deliberately have to set the register where it
found p to an evil pattern. And if p was on the stack (or some such),
nobody would even ever notice?

This is longwinded.

The value of p was returned by malloc()

Originally, yes.

and is valid for the implemntation of p.

Valid until free()d, yes.

Formally ...

But what if free() changed something in the hardware so that the very
same -- unchanged -- bits in p, that *used* to mean "the contents of the
memory bank located in the south wing of the building", now mean "please
trigger the nuclear warhead"? (This is, after all, the Deathstation.
:-) )

OK, on the Deathstation perhaps. I think you have now touched the
unimplemented part of the world, no?
;-)

There is nothing in the C language that prohibits free() from changing
the way the machine interprets the bits. If certain pointer bit
patterns were like "int" before (always valid), and are like "float" now
(have trap representations), the simple act of testing the value of "p"
can fault.

I would buy this if we have had a construct like:

p = free(p);

... but this is not the case.

*This* is how p can become invalid after free(): not by having its
stored bit pattern change, but rather by having the *meaning* of those
bits change.

Ehrmm, p is still declared as a pointer and not a float. I assume my
compiler to still evaluate it as a pointer. How then can the meaning of
the (unchanged) bitpattern have changed (in an implementation other than
the imaginary evil Deathstation?)

The Deathstation could change the tag bits defining the type from
pointer to float in it's type TLB. AFAIK there's nothing in the spec
to prohibit free() from being implemented as an intrinsic which sets p
to NULL or BAD bitpattern.

--
Thanks. Take care, Brian Inglis Calgary, Alberta, Canada

Br**********@CSi.com (Brian[dot]Inglis{at}SystematicSW[dot]ab[dot]ca)
fake address use address above to reply
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Bart van Ingen Schenau]

On Sat, 22 Jan 2005 02:47:00 -0000, Jens M Andreasen <[e-mail]>
wrote:

On Fri, 21 Jan 2005 03:59:55 +0000, Chris Torek wrote:

This issue is quite tricky, but it may become clear if you can answer me
this question: what is the value represented by the following bit pattern?

11000011 11110101 01001000 01000000
(in hex: 0xc3 0xf5 0x48 0x40)

I will give you two possible answers (though there are more):

3.14 (a float)
1078523331 (an int)

and will I tell you that at least one of them is correct (and the machine
in question has a Pentium III CPU).

Depends on what register your bitpattern resides in. Given the option
between float and int, my guess is one of the SSE registers?

And what if the bitpattern resides in main memory?

<snip>

How will you decide whether those bytes represent a float or an int?
More importantly, whether I tell you the truth or lie -- it does not
really matter which -- how can it be that this same bit pattern
apparently represents two different numbers?

The answer lies in the *interpretation* of the bits. The bits themseves
are just bits. By themselves they have no deeper meaning. It is only
when I tell you "these are the bits of a float" that they *mean* 3.14,
or "these are the bits of an int" that they *mean* 1078523331.

OK ...

What if I tell you, instead, that the machine is a Deathstation 9900,
and the bits represent a pointer? What is the value of that pointer?
How will you interpret them?

I wouldn't ...

You are now saying that:

free(p)
if(p)
...

... *could* result in a cast to float whith a possible raise of an
exception (NaN or whatever)

No, but it could result in a hardware exception due to loading an
invalid address.

That means free() would deliberately have to set the register where it
found p to an evil pattern. And if p was on the stack (or some such),
nobody would even ever notice?
There is no need to alter p itself. It is sufficient to mark the
address contained inh p as being invalid. (This could be done by
unmapping the segment of p from the segment table).

<snip>

But what if free() changed something in the hardware so that the very
same -- unchanged -- bits in p, that *used* to mean "the contents of the
memory bank located in the south wing of the building", now mean "please
trigger the nuclear warhead"? (This is, after all, the Deathstation.
:-) )

OK, on the Deathstation perhaps. I think you have now touched the
unimplemented part of the world, no?
;-)

There is nothing in the C language that prohibits free() from changing
the way the machine interprets the bits. If certain pointer bit
patterns were like "int" before (always valid), and are like "float" now
(have trap representations), the simple act of testing the value of "p"
can fault.

I would buy this if we have had a construct like:

p = free(p);

... but this is not the case.

*This* is how p can become invalid after free(): not by having its
stored bit pattern change, but rather by having the *meaning* of those
bits change.

Ehrmm, p is still declared as a pointer and not a float. I assume my
compiler to still evaluate it as a pointer. How then can the meaning of
the (unchanged) bitpattern have changed (in an implementation other than
the imaginary evil Deathstation?)

The meaning can change because the implementation may implement a
mapping between the logical address (the value stored in p) and the
physical hardware address (the physical location of the data). This
mapping is not required to be constant, and will typically change
frequently (think about paging).
free(p) can mark the logical address as being unused, without changing
the value of p.


mvh // Jens M Andreasen

Bart v Ingen Schenau
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Douglas A. Gwyn]

Brian Inglis wrote:

The Deathstation could change the tag bits defining the type from
pointer to float in it's type TLB. AFAIK there's nothing in the spec
to prohibit free() from being implemented as an intrinsic which sets p
to NULL or BAD bitpattern.

I believe that is contrary to the general C object model,
which for objects not flagged as "volatile" does not allow
*content* (bit pattern) to be changed by external forces.
However, the *value* can change. I liked the credit card
analogy.
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[CBFalconer]

Chris Torek wrote:

..... snip ...
This issue is quite tricky, but it may become clear if you can
answer me this question: what is the value represented by the
following bit pattern?

11000011 11110101 01001000 01000000
(in hex: 0xc3 0xf5 0x48 0x40)

jmp 0x48f5; nop; (* for an 8080/z80 *) :-)

--
"If you want to post a followup via groups.google.com, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Richard Bos]

Jens M Andreasen <[e-mail]> wrote:

On Fri, 21 Jan 2005 03:59:55 +0000, Chris Torek wrote:

This issue is quite tricky, but it may become clear if you can answer me
this question: what is the value represented by the following bit pattern?

11000011 11110101 01001000 01000000
(in hex: 0xc3 0xf5 0x48 0x40)

I will give you two possible answers (though there are more):

3.14 (a float)
1078523331 (an int)

and will I tell you that at least one of them is correct (and the machine
in question has a Pentium III CPU).
Depends on what register your bitpattern resides in. Given the option
between float and int, my guess is one of the SSE registers?

And if it's in memory?

What if I tell you, instead, that the machine is a Deathstation 9900,
and the bits represent a pointer? What is the value of that pointer?
How will you interpret them?

I wouldn't ...

You are now saying that:

free(p)
if(p)
...

... *could* result in a cast to float whith a possible raise of an
exception (NaN or whatever)

No, he's saying that it could result in a pointer which, since the
free(), refers to an invalid page frame, being loaded in an address
register, and the processor causing a trap because of a memory
violation.
This is longwinded.

It's deadly serious. It could even be the way forward in the battle
against wild pointer errors - I would be all for it.

*This* is how p can become invalid after free(): not by having its
stored bit pattern change, but rather by having the *meaning* of those
bits change.

Ehrmm, p is still declared as a pointer and not a float. I assume my
compiler to still evaluate it as a pointer. How then can the meaning of
the (unchanged) bitpattern have changed (in an implementation other than
the imaginary evil Deathstation?)

Suppose your phone number is 098-12345. Suppose you move to a different
city, and your phone is disconnected. I dial your old number. Not a
single digit in that number has changed, yet I now get an "invalid
number" tone, rather than your voice.

Richard
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Richard Bos]

Jens M Andreasen <[e-mail]> wrote:

On Fri, 21 Jan 2005 04:02:14 +0000, Richard Bos wrote:

If, OTOH, the segment that p pointed to has just been deallocated, it
could result in

LD ADDR1, null (ok, ADDR1 contains a null pointer) LD ADDR2, (p) (not
ok. The processor detects an attempt to load a
pointer into ADDR2 which is not valid, and causes a trap. The OS
reacts to this by terminating your program with extreme prejudice.)
This is perhaps the best description of the situation so far

I would have thought that it is the following MOVE that raises the
exception. Afterall we haven't done anything nasty (yet.)

Yes, you have. You've threatened to access memory you ought to keep your
grubby paws out of, since you've given it back to the OS.
Realworld examples, please?

You miss the point. What is there to guarantee that the _next_ service
pack, bugfix, or kernel upgrade of your favourite OS doesn't implement
this feature, in the name of better safety?

Richard
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Jarno A Wuolijoki]

On Mon, 24 Jan 2005, Keith Thompson wrote:

or even if every *possible* implementation does the same thing,
the behavior is still undefined as far as the standard is concerned.

So this 'thing' is simultaneously both defined and undefined?

...ah, you meant real-world-possible..
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Brian Inglis]

On Sat, 22 Jan 2005 02:47:00 -0000 in comp.lang.c.moderated, Jens M
Andreasen <[e-mail]> wrote:

On Fri, 21 Jan 2005 03:59:55 +0000, Chris Torek wrote:

But what if free() changed something in the hardware so that the very
same -- unchanged -- bits in p, that *used* to mean "the contents of the
memory bank located in the south wing of the building", now mean "please
trigger the nuclear warhead"? (This is, after all, the Deathstation.
:-) )

OK, on the Deathstation perhaps. I think you have now touched the
unimplemented part of the world, no?
;-)

Ehrmm, p is still declared as a pointer and not a float. I assume my
compiler to still evaluate it as a pointer. How then can the meaning of
the (unchanged) bitpattern have changed (in an implementation other than
the imaginary evil Deathstation?)

On at least one real life Deathstation, even the value returned by
malloc() could be considered "indeterminate", if virtual memory is
overcommitted: you will segfault and can be terminated.

--
Thanks. Take care, Brian Inglis Calgary, Alberta, Canada

Br**********@CSi.com (Brian[dot]Inglis{at}SystematicSW[dot]ab[dot]ca)
fake address use address above to reply
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Brian Inglis]

On Mon, 24 Jan 2005 21:57:47 -0000 in comp.lang.c.moderated, Bart van
Ingen Schenau <Ba********************@ict.nl> wrote:

On Sat, 22 Jan 2005 02:47:00 -0000, Jens M Andreasen <[e-mail]>
wrote:

On Fri, 21 Jan 2005 03:59:55 +0000, Chris Torek wrote:

*This* is how p can become invalid after free(): not by having its
stored bit pattern change, but rather by having the *meaning* of those
bits change.

Ehrmm, p is still declared as a pointer and not a float. I assume my
compiler to still evaluate it as a pointer. How then can the meaning of
the (unchanged) bitpattern have changed (in an implementation other than
the imaginary evil Deathstation?)

The meaning can change because the implementation may implement a
mapping between the logical address (the value stored in p) and the
physical hardware address (the physical location of the data). This
mapping is not required to be constant, and will typically change
frequently (think about paging).
free(p) can mark the logical address as being unused, without changing
the value of p.

Some debugging implementations setup each object in its own page(s),
so that if any access is made outside allocated storage, or after
free() is called, the reference is trapped.

--
Thanks. Take care, Brian Inglis Calgary, Alberta, Canada

Br**********@CSi.com (Brian[dot]Inglis{at}SystematicSW[dot]ab[dot]ca)
fake address use address above to reply
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Brian Inglis]

On Mon, 24 Jan 2005 21:57:18 -0000 in comp.lang.c.moderated, "Douglas
A. Gwyn" <DA****@null.net> wrote:

Jens M Andreasen wrote:

I would have thought that it is the following MOVE that raises the
exception. Afterall we haven't done anything nasty (yet.)

Yes you have: you tried to load an unmapped value
into an address register. A trap is highly appropriate.

AFAIR this would happen in 286 protected mode with invalid segment
selectors.

--
Thanks. Take care, Brian Inglis Calgary, Alberta, Canada

Br**********@CSi.com (Brian[dot]Inglis{at}SystematicSW[dot]ab[dot]ca)
fake address use address above to reply
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Jens M Andreasen]

On Mon, 24 Jan 2005 21:57:20 +0000, Gordon Burditt wrote:

Ehrmm, p is still declared as a pointer and not a float. I assume my
compiler to still evaluate it as a pointer. How then can the meaning of
the (unchanged) bitpattern have changed (in an implementation other than
the imaginary evil Deathstation?)
How can the usability of a credit card change when it is reported stolen?
The number on the card is unchanged.

And as long as you just keep the credit-card in your pocket, nothing bad
happens. It is when you start using the teller-machine and actually move
some values that the police will trap you. No?
Memory maps can change during the life of a process. Segments can
become valid or invalid (and having this happen during calls to malloc()
or free() is not unreasonable). Loading a "large" pointer with an
invalid segment into an i386 segment register will cause a trap. No
dereference is required (you won't get a chance to try). A NULL pointer
is treated as a special case here: you can load the pointer, but not
dereference.
I tried hard (on a pII, Chris Torek mentioned pIII) to do something wild
enough to bailout. Didn't happen. Maybe I wasn't trying hard enough?

So if you can show a shortish scary example, it would be most welcome.
When you say "large" pointer, is that a reference to the
"small/large/huge" memorymodels? Deliberately mixing the different memory
models indeed have implications.

I suppose it is not that important anymore so, never mind ...


Gordon L. Burditt

--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Douglas A. Gwyn]

Brian Inglis wrote:

On at least one real life Deathstation, even the value returned by
malloc() could be considered "indeterminate", if virtual memory is
overcommitted: you will segfault and can be terminated.

Yeah, I was appalled when I heard that Linux had done that.
That is *not* a conforming C implementation. If malloc
returns a non-null pointer, then the storage shall have
been *allocated*, i.e. it exists, has addresses, etc.
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Keith Thompson]

Brian Inglis <Br**********@SystematicSW.Invalid> writes:
[...]

On at least one real life Deathstation, even the value returned by
malloc() could be considered "indeterminate", if virtual memory is
overcommitted: you will segfault and can be terminated.

I don't think so. Though it's maximally perverse, the Deathstation's
C implementation is conforming.

There are real-world (not-quite-)C implementations that behave as
described, but the Deathstation would never do such a thing.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Keith Thompson]

Brian Inglis <Br**********@SystematicSW.Invalid> writes:
[...]

Some debugging implementations setup each object in its own page(s),
so that if any access is made outside allocated storage, or after
free() is called, the reference is trapped.

Does that catch invalid references to p, or only references to *p?

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Richard Bos]

Brian Inglis <Br**********@SystematicSW.Invalid> wrote:

On Sat, 22 Jan 2005 02:47:00 -0000 in comp.lang.c.moderated, Jens M
Andreasen <[e-mail]> wrote:

Ehrmm, p is still declared as a pointer and not a float. I assume my
compiler to still evaluate it as a pointer. How then can the meaning of
the (unchanged) bitpattern have changed (in an implementation other than
the imaginary evil Deathstation?)

On at least one real life Deathstation, even the value returned by
malloc() could be considered "indeterminate", if virtual memory is
overcommitted: you will segfault and can be terminated.

But AFAIK that is _not_ conforming. If you get a pointer from malloc(),
you own that memory. If you pass a pointer to free(), you do not own
that memory any longer, and this is what allows the segfault.

Richard
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Francis Glassborow]

In message <cl****************@plethora.net>, Douglas A. Gwyn
<DA****@null.net> writes

I believe that is contrary to the general C object model,
which for objects not flagged as "volatile" does not allow
*content* (bit pattern) to be changed by external forces.
However, the *value* can change. I liked the credit card
analogy.

I think you are mistaken, at least the above does not seem to match the
decision made in Sydney wrt indeterminate values. IIRC it was the almost
unanimous opinion of those present at that meeting that a change to a
bit pattern that did not change the value (for example, normalising a
pointer representation on a machine using multiple representations of
addresses) was allowed at almost any time. It was also the opinion of
those present that an indeterminate value can be represented by any and
all possible bit patterns.
--
Francis Glassborow ACCU
Author of 'You Can Do It!' see http://www.spellen.org/youcandoit
For project ideas and contributions: http://www.spellen.org/youcandoit/projects
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Richard Bos]

Jarno A Wuolijoki <jw******@cs.Helsinki.FI> wrote:

On Mon, 24 Jan 2005, Keith Thompson wrote:

or even if every *possible* implementation does the same thing,
the behavior is still undefined as far as the standard is concerned.

So this 'thing' is simultaneously both defined and undefined?

..ah, you meant real-world-possible..

Even when a certain case of UB is now real-world impossible (and in the
case under discussion, it is not even that), computer science, for both
hardware and software, advances so rapidly that there's no guarantee
that it won't be possible next year, and state of the art in five.

Richard
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Gordon Burditt]

>> How can the usability of a credit card change when it is reported stolen?

The number on the card is unchanged.

And as long as you just keep the credit-card in your pocket, nothing bad
happens. It is when you start using the teller-machine and actually move
some values that the police will trap you. No?

Unless you get caught for some other reason, and the police inventory your
pockets when you are put in jail.

Memory maps can change during the life of a process. Segments can
become valid or invalid (and having this happen during calls to malloc()
or free() is not unreasonable). Loading a "large" pointer with an
invalid segment into an i386 segment register will cause a trap. No
dereference is required (you won't get a chance to try). A NULL pointer
is treated as a special case here: you can load the pointer, but not
dereference.

I tried hard (on a pII, Chris Torek mentioned pIII) to do something wild
enough to bailout. Didn't happen. Maybe I wasn't trying hard enough?

Are you using the large memory model, or it's 32-bit equivalent
(sometimes called "intergalactic model"?) The time is fast arriving
when software won't be able to tolerate the restrictive limit of a
tiny 4GB address space of a user program (for that matter, some
programs would really like to have single arrays larger than 4GB).
Then, they'll have to start using 48-bit pointers on the i386
architecture (or abandon i386 entirely, which won't happen easily),
and THAT's when you'll run into the problem that loading an invalid
address causes a trap.

Some GUI programs may have trouble fitting all the fonts into 4GB,
and I expect that an animated screen image (HDTV resolution, 64-bit
color, etc.) could get very large.
When you say "large" pointer, is that a reference to the
"small/large/huge" memorymodels? Deliberately mixing the different memory >models indeed have implications.

Nobody said anything about "mixing" models. Any model in which a
pointer contains a protected-mode selector piece (16 bits) in a
segment register will be affected.
I suppose it is not that important anymore so, never mind ...

It will become very important in the future, when 4GB starts being
a tight constraint on a program. Whether or not it is still called
"large model" (like the DOS 16-bit segment/16-bit offset) or something
else is a different question.

Gordon L. Burditt
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Jens M Andreasen]

On Thu, 27 Jan 2005 05:50:22 +0000, Brian Inglis wrote:

On Mon, 24 Jan 2005 21:57:18 -0000 in comp.lang.c.moderated, "Douglas A.
Gwyn" <DA****@null.net> wrote:

Jens M Andreasen wrote:

I would have thought that it is the following MOVE that raises the
exception. Afterall we haven't done anything nasty (yet.)
Yes you have: you tried to load an unmapped value into an address
register. A trap is highly appropriate.

AFAIR this would happen in 286 protected mode with invalid segment
selectors.

Yes it would have perhaps had happened with a noncorforming malloc, and at
the time you are referring to, there was no other standard except for a
general 'look & feel'!

/j --
Thanks. Take care, Brian Inglis Calgary, Alberta, Canada

Br**********@CSi.com (Brian[dot]Inglis{at}SystematicSW[dot]ab[dot]ca)
fake address use address above to reply

--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Jens M Andreasen]

On Thu, 27 Jan 2005 05:50:06 +0000, Richard Bos wrote:

Jens M Andreasen <[e-mail]> wrote:

This is longwinded.

It's deadly serious. It could even be the way forward in the battle
against wild pointer errors - I would be all for it.

Now this is a sensible reply!
mvh // Jens M Andreasen
PS: The pointer in the example is not in the wild ... yet :P
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Nils Weller]

In article <cl****************@plethora.net>, Douglas A. Gwyn wrote:

Brian Inglis wrote:

On at least one real life Deathstation, even the value returned by
malloc() could be considered "indeterminate", if virtual memory is
overcommitted: you will segfault and can be terminated.
Yeah, I was appalled when I heard that Linux had done that.

Then you may also be appalled to hear that it's not just Linux that
supports memory overcommitment; HP-UX, Tru64 UNIX, AIX, IRIX and FreeBSD
do so too, though I think Solaris and UnixWare do not.

While the usefulness of this policy is debatable for malloc() and
friends (indeed, some systems do not support lazy allocation through
the interface(s) underlying malloc()), it makes a lot of sense for
copy-on-write mappings obtained via fork() or mmap().

Consider processes occupying a large amount of virtual memory. Those
processes would not be able to create a new process and execute another
application via fork() + exec*() if the system eagerly allocates memory.
Consider shared libraries, the data segments of which need to be mapped
copy-on-write as well. Not every process linking a large shared library,
such as libc on Unix, will want to use all of it. Consider sparse arrays
and regular files mapped with mmap() and MAP_PRIVATE ...

Even if there is no way to convince you that lazy allocation does make
sense, you have to understand that it is a feature, not a bug. Linux and
the other systems I have mentioned above make it *optional*. You can
choose to use safe eager allocation instead if you so desire, using
/proc/sys/vm/overcommit_memory or sysctl vm.overcommit_memory on Linux.
The extent to which the system shall overcommit can be configured too,
using sysctl vm.overcommit_ratio.
That is *not* a conforming C implementation. If malloc
returns a non-null pointer, then the storage shall have
been *allocated*, i.e. it exists, has addresses, etc.

Then you will also have to complain about every other system application
that may disrupt the execution of a conforming C program, such as the
Unix kill command and debuggers (using ptrace() and/or procfs to alter
the state of the program.)

As long as these things are useful, people will continue to prefer their
availability in favor of strict standards compliance.

--
My real email address is ``nils<at>gnulinux<dot>nl''
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Jens M Andreasen]

On Thu, 27 Jan 2005 05:50:08 +0000, Richard Bos wrote:

Jens M Andreasen <[e-mail]> wrote:

Realworld examples, please?

You miss the point ...

No I did not! You have missed my point (?)
Give me one intstance where 'p == NULL' will fail (on that line) and I
will shut up! This is after p=malloc(somesize),free(p) ..
mvh // Jens M Andreasen
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Jens M Andreasen]

On Fri, 28 Jan 2005 03:00:16 +0000, Douglas A. Gwyn wrote:

Brian Inglis wrote:

On at least one real life Deathstation, even the value returned by
malloc() could be considered "indeterminate", if virtual memory is
overcommitted: you will segfault and can be terminated.

Yeah, I was appalled when I heard that Linux had done that. That is *not*
a conforming C implementation. If malloc returns a non-null pointer, then
the storage shall have been *allocated*, i.e. it exists, has addresses,
etc.

Nitpicking off-topic:

It is the C-lib that would have have done the overcommitting. The
Linux kernel itself is pretty braindead (hopefully!) and will just do
weird things when you run out of swap.

/j
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Douglas A. Gwyn]

Francis Glassborow wrote:

... IIRC it was the almost
unanimous opinion of those present at that meeting that a change to a
bit pattern that did not change the value (for example, normalising a
pointer representation on a machine using multiple representations of
addresses) was allowed at almost any time.

The problem with that is that previously we all seemed
to agree that a value could be safely copied via the
equivalent of memcpy in the same way. But if the
representation can change while its bytes are being
copied, that would no longer hold.
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Jens M Andreasen]

On Fri, 28 Jan 2005 03:01:07 +0000, Gordon Burditt wrote:

[ja:]

And as long as you just keep the credit-card in your pocket, nothing bad
happens. It is when you start using the teller-machine and actually move
some values that the police will trap you. No?
Unless you get caught for some other reason, and the police inventory your
pockets when you are put in jail.

Hey, wait a second ... The tellermachine/MMU anology wasn't that bad!
Extendinding it to mean any bitpattern I may or may not have floating
around in memory leads us astray ...

I tried hard (on a pII, Chris mentioned pIII) to do something wild
enough to bailout. Didn't happen. Maybe I wasn't trying hard enough?

Are you using the large memory model, or it's 32-bit equivalent
(sometimes called "intergalactic model"?)

That would be the 'intergalactic' model :) Although I still have 5 inch
install disks with a 'no nonsense' license, I have no machine with an OS
where it can legally (nor practically) be installed.
Nobody said anything about "mixing" models. Any model in which a
pointer contains a protected-mode selector piece (16 bits) in a segment
register will be affected.

OK! I suppose you still keep a DOS partition around for reference? If so,
please compile and run: p = malloc(somesize),p?free(p):, p? ...

We are now back in a timespace when even the proposed standard wasn't
more than a mere wink in the eye(s) of the founding fathers, but:

If memory serves me well, I would say that for a given project of some
size: As long as malloc did not fail, p would be within limits.

The key here is that we are not casting any random value into a pointer.
The implications for a task-switching OS would otherwise be tremendous:

p = malloc(somesize)
q(p) // q will free p but implementation cannot see it
/**/ // taskswitch here, flush, reload and fail?
.....
Mmmm ... I can see a flaw in the taskswitch argument above, running in
ring zero as a kernel might very well be different from running in
userspace. And we are discussing specifics of a 386 class processor? This
might be a good entrypoint for shutting me up!

/j
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Brian Inglis]

On Fri, 28 Jan 2005 03:00:16 -0000 in comp.lang.c.moderated, "Douglas
A. Gwyn" <DA****@null.net> wrote:

Brian Inglis wrote:

On at least one real life Deathstation, even the value returned by
malloc() could be considered "indeterminate", if virtual memory is
overcommitted: you will segfault and can be terminated.
Yeah, I was appalled when I heard that Linux had done that.

ISTR it being IBM AIX.
That is *not* a conforming C implementation. If malloc
returns a non-null pointer, then the storage shall have
been *allocated*, i.e. it exists, has addresses, etc.

--
Thanks. Take care, Brian Inglis Calgary, Alberta, Canada

Br**********@CSi.com (Brian[dot]Inglis{at}SystematicSW[dot]ab[dot]ca)
fake address use address above to reply
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Jarno A Wuolijoki]

On Fri, 28 Jan 2005, Richard Bos wrote:

Jarno A Wuolijoki <jw******@cs.Helsinki.FI> wrote:

On Mon, 24 Jan 2005, Keith Thompson wrote:

or even if every *possible* implementation does the same thing,
the behavior is still undefined as far as the standard is concerned.

So this 'thing' is simultaneously both defined and undefined?
..ah, you meant real-world-possible..

Even when a certain case of UB is now real-world impossible (and in the
case under discussion, it is not even that), computer science, for both
hardware and software, advances so rapidly that there's no guarantee
that it won't be possible next year, and state of the art in five.

Read it again and you'll find that I'm very much not talking about
the real world there;)

With a certain mindset 'every possible implementation does X' can be
read as 'it can be proven from [a list of c&v's] that a compiler
that performs some other action but X is not an implementation' which
is the same as 'the standard requires the compiler to do X' which is
the same as 'X is the defined behaviour here' which contradicts
'X is UB'.

Of course if you introduce some offtopic issues like physics of the
world surrounding us into the definition of 'possible', this logic
breaks down.
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Dag-Erling Smørgrav]

"Douglas A. Gwyn" <DA****@null.net> writes:

Brian Inglis wrote:

On at least one real life Deathstation, even the value returned by
malloc() could be considered "indeterminate", if virtual memory is
overcommitted: you will segfault and can be terminated.

Yeah, I was appalled when I heard that Linux had done that.
That is *not* a conforming C implementation. If malloc
returns a non-null pointer, then the storage shall have
been *allocated*, i.e. it exists, has addresses, etc.

Pretty much every major operating system today, including the one
you're currently using, overcommits memory.

DES
--
Dag-Erling Smørgrav - de*@des.no
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Lawrence Kirby]

On Fri, 28 Jan 2005 03:00:41 +0000, Francis Glassborow wrote:

In message <cl****************@plethora.net>, Douglas A. Gwyn
<DA****@null.net> writes

I believe that is contrary to the general C object model,
which for objects not flagged as "volatile" does not allow
*content* (bit pattern) to be changed by external forces.
However, the *value* can change. I liked the credit card
analogy.

I think you are mistaken, at least the above does not seem to match the
decision made in Sydney wrt indeterminate values. IIRC it was the almost
unanimous opinion of those present at that meeting that a change to a
bit pattern that did not change the value (for example, normalising a
pointer representation on a machine using multiple representations of
addresses) was allowed at almost any time.

But consider that I can view any (addressable) object as an array of
unsigned char. If the value (effectively the representation) of a
non-volatile object changes under that view there needs to be a good
reason for it.
It was also the opinion of
those present that an indeterminate value can be represented by any and
all possible bit patterns.

But whether the value of an object is indeterminate or not depends on the
type of lvalue used to access it. If I access an indeterminate pointer
with a pointer lvalue then, fair enough, that could change it to anything.
If I only access its bytes with unsigned char lvalues the implementation
has no business doing pointer related things behind the scenes.

Lawrence
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Lawrence Kirby]

On Fri, 28 Jan 2005 03:00:16 +0000, Douglas A. Gwyn wrote:

Brian Inglis wrote:

On at least one real life Deathstation, even the value returned by
malloc() could be considered "indeterminate", if virtual memory is
overcommitted: you will segfault and can be terminated.
Yeah, I was appalled when I heard that Linux had done that.
That is *not* a conforming C implementation.

I don't see a problem with this.
If malloc
returns a non-null pointer, then the storage shall have
been *allocated*, i.e. it exists, has addresses, etc.

How will a strictly conforming program tell the difference?

Consider that the *only* program that a conforming C implementation must
translate and execute successfully is the one specified in 5.2.4.1, and
AFAIK this program isn't required to call malloc(). For any other strictly
conforming program all we know is that to the extent that the
implementation does execute it successfully the output generated must be
consistent with the abstract machine as specified by 5.1.2 and its
subclauses.

The execution of any program except the designated one may fail at any
time for any reason as long as the failure mode is clean in the sense that
it doesn't produce wrong output. There's nothing wrong with incomplete
output as long as the program didn't terminate normally, and didn't
otherwise violate 5.1.2.3.

There's no way a strictly conforming program can tell the difference (i.e.
engineer output based on this) between, say, malloc() overcommitting at
the point of allocation, the system subsequently taking memory away from
the program, a different failure such as a "stack" allocation failure,
kill -9, and so on.

7.20.3.3p3 says:

"The malloc function returns either a null pointer or a pointer to the
allocated space".

This is a description of the abstract machine (so says 5.1.2.3p1). A
conforming implementation need only follow the semantics of the abstract
machine as far as it is required to by the relevant parts of the standard,
notably in sections 4 and 5.

Lawrence
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Francis Glassborow]

In message <cl****************@plethora.net>, Douglas A. Gwyn
<DA****@null.net> writes

Francis Glassborow wrote:

... IIRC it was the almost
unanimous opinion of those present at that meeting that a change to a
bit pattern that did not change the value (for example, normalising a
pointer representation on a machine using multiple representations of
addresses) was allowed at almost any time.

The problem with that is that previously we all seemed
to agree that a value could be safely copied via the
equivalent of memcpy in the same way. But if the
representation can change while its bytes are being
copied, that would no longer hold.

I do not see any subsequent comments on the Sydney decision and I note
that neither you nor Larry were at the Redmond meeting. If either of you
believe the decision was wrong then I think you need to raise it fairly
quickly with WG14.

The specific point you raise was not raised at the Sydney meeting and
certainly should be considered.

--
Francis Glassborow ACCU
Author of 'You Can Do It!' see http://www.spellen.org/youcandoit
For project ideas and contributions: http://www.spellen.org/youcandoit/projects
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[CBFalconer]

Nils Weller wrote:

Douglas A. Gwyn wrote:
..... snip ...
Even if there is no way to convince you that lazy allocation does
make sense, you have to understand that it is a feature, not a bug.
Linux and the other systems I have mentioned above make it
*optional*. You can choose to use safe eager allocation instead if
you so desire, using /proc/sys/vm/overcommit_memory or sysctl
vm.overcommit_memory on Linux. The extent to which the system
shall overcommit can be configured too, using sysctl
vm.overcommit_ratio.

That is *not* a conforming C implementation. If malloc
returns a non-null pointer, then the storage shall have
been *allocated*, i.e. it exists, has addresses, etc.

Then you will also have to complain about every other system
application that may disrupt the execution of a conforming C
program, such as the Unix kill command and debuggers (using
ptrace() and/or procfs to alter the state of the program.)

As long as these things are useful, people will continue to prefer
their availability in favor of strict standards compliance.

There is no conformance problem if the system, on finding that the
lazy allocated memory is not available, simply pauses the program
for i/o until that memory chunk can be physically allocated. Then
the only thing affected is the run timing.

--
"If you want to post a followup via groups.google.com, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Richard Bos]

Jens M Andreasen <[e-mail]> wrote:

On Fri, 28 Jan 2005 03:01:07 +0000, Gordon Burditt wrote:

Nobody said anything about "mixing" models. Any model in which a
pointer contains a protected-mode selector piece (16 bits) in a segment
register will be affected.
OK! I suppose you still keep a DOS partition around for reference?

Yes.
If so, please compile and run: p = malloc(somesize),p?free(p):, p? ...
Not on your nelly. First, it's unsyntactical; second, several important
parts are missing; and third, an experiment on one implementation is
irrelevant, really. MS-DOS C compilers often did things that tried to go
beyond what the OS itself strictly allowed.
We are now back in a timespace when even the proposed standard wasn't
more than a mere wink in the eye(s) of the founding fathers, but:
Eh? 1995 was earlier than 1989?
If memory serves me well, I would say that for a given project of some
size: As long as malloc did not fail, p would be within limits.

The key here is that we are not casting any random value into a pointer.
You're not casting _anything_, random or otherwise.
The implications for a task-switching OS would otherwise be tremendous:

p = malloc(somesize)
q(p) // q will free p but implementation cannot see it
/**/ // taskswitch here, flush, reload and fail?
I don't see where you get that idea. There are many ways around this, of
which the most simple one in pure ISO C is that the value of p may now
be a trap value when seen as a pointer, but not as an array of unsigned
chars (since unsigned char does not have any trap values).
This might be a good entrypoint for shutting me up!

You reading the Standard would be a better solution to that problem.

Richard
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Dag-Erling Smørgrav]

Jens M Andreasen <[e-mail]> writes:

It is the C-lib that would have have done the overcommitting. The
Linux kernel itself is pretty braindead (hopefully!) and will just do
weird things when you run out of swap.

Wrong. The C library uses underlying kernel facilities (mmap() /
brk() / sbrk()) to allocate memory. The fact that those underlying
kernel facilities only allocate address space, not physical memory
(the physical memory is allocated only when the aforementioned address
space is accessed), is a feature of the kernel, not of the C library.

Moving away from strict C and into the POSIX realm, the C library or
the application can work around memory overcommit by touching every
allocated page (forcing physical memory or swap space to be allocated)
with a SIGSEGV handler installed to catch unsatisfied page faults, or
by using a large non-sparse file as a backing store.

DES
--
Dag-Erling Smørgrav - de*@des.no
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Michael Wojcik]

In article <cl****************@plethora.net>, Jens M Andreasen <[e-mail]> writes:

Give me one intstance where 'p == NULL' will fail (on that line) and I
will shut up! This is after p=malloc(somesize),free(p) ..

Perhaps there isn't one. I've just tried ILE C on the AS/400, which
was a promising candidate, and it let the test through, in this simple
case.

It might fail in a more complex one, where the freed pointer wasn't
in scope; for example, if I freed a pointer and then passed it to a
function in another t.u. which tested it for null. That's because on
the AS/400 pointers aren't simple addresses. They're 128-bit objects
(for a 64-bit address space) that have to be "materialized" before
they can be used. The AS/400 doesn't use a separate virtual address
space for each process ("job" in OS/400); it uses a single virtual
address space for the entire system, and uses strong pointer typing
and hardware address validation to enforce storage access rules.

Here's what the output of the %p format specifier looks like in ILE
C:

intp is SPP:0000 :1aefQPADEV0001MWW 006945 :174:1:24d4
funcp is PRP:0:24d4

The first is an int pointer, the second a function pointer.

The "materialize pointer" instruction does some sophisticated
checking. For example, the following code will produce a run-time
violation and abort the program (or, if the program is compiled for
debugging, suspend it), sending an operator message to the message
queue of the user who submitted it:

void *voidp;
int (*funcp)(void) = main;
voidp = (void *)funcp;

The AS/400 will not let you convert a function pointer to a void
pointer, even with a cast. It catches that at runtime. That's no
odder than trapping a reference to a free'd pointer; it'd be trivial
for the materialization of the latter, even just to read its value
(and not dereference it), to check to see that it was still valid.

It doesn't require any change to the pointer representation, either.
A sequence of 128 bits may be a valid pointer at one point in a
program's lifetime, and not at another, and the implementation could
detect that if it wanted to. In this case, it appears that the IBM
compiler team decided not to trap this error; but the technology is
there for them to have done so.

And they might, in another revision of the compiler. And that's why
it's stupid to assume that, just because no one can point to such an
implementation today, such an implementation will never exist.

--
Michael Wojcik mi************@microfocus.com

They had forgathered enough of course in all the various times; they had
again and again, since that first night at the theatre, been face to face
over their question; but they had never been so alone together as they were
actually alone - their talk hadn't yet been so supremely for themselves.
-- Henry James

[Gordon Burditt]

>There is no conformance problem if the system, on finding that the

lazy allocated memory is not available, simply pauses the program
for i/o until that memory chunk can be physically allocated. Then
the only thing affected is the run timing.

Unless the system deadlocks. It will if you have enough processes
trying to grow that also hold lots of memory, and more processes
waiting on those that hold more memory, and few processes that are
going to release sufficient memory that aren't waiting on the first
two sets.

Then again, the possibility of power failures or repossession of the
computer system also makes the implementation non-conforming.

Gordon L. Burditt
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Douglas A. Gwyn]

Dag-Erling Smørgrav wrote:

Moving away from strict C and into the POSIX realm, the C library or
the application can work around memory overcommit by touching every
allocated page (forcing physical memory or swap space to be allocated)
with a SIGSEGV handler installed to catch unsatisfied page faults, or
by using a large non-sparse file as a backing store.

When the SIGSEGV occurs all that can reliably be
determined is that malloc should have returned null.
That is malloc's job in the first place; fix it.

The idea that every reliable app will have to take
extraordinary measures to handle this situation
indicates what a horrible design decision it was.
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Douglas A. Gwyn]

Gordon Burditt wrote:

Then again, the possibility of power failures or repossession of the
computer system also makes the implementation non-conforming.

No. Those lie outside the scope of the C standard.
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Dag-Erling Smørgrav]

"Douglas A. Gwyn" <DA****@null.net> writes:

[memory overcommit considered harmful]

Look, I can grok that ivory towers look really neat, but don't they
get awfully cold in the winter? Wouldn't you rather be living in the
real world?

DES
--
Dag-Erling Smørgrav - de*@des.no
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.

[Douglas A. Gwyn]

Dag-Erling Smørgrav wrote:

"Douglas A. Gwyn" <DA****@null.net> writes:

[memory overcommit considered harmful]

Look, I can grok that ivory towers look really neat, but don't they
get awfully cold in the winter? Wouldn't you rather be living in the
real world?

In the real world, it is not considered acceptable for
correct programs to be crashed due to such a poor design.
Also, in the world I come from, programmers *learn* how
to implement sparse arrays so that they don't *have* to
wreck the whole system.
--
comp.lang.c.moderated - moderation address: cl**@plethora.net -- you must
have an appropriate newsgroups line in your header for your mail to be seen,
or the newsgroup name in square brackets in the subject line. Sorry.