sizeof([ALLOCATED MEMORY])

[ballpointpenthief]

If I have malloc()'ed a pointer and want to read from it as if it were
an array, I need to know that I won't be reading past the last index.

If this is a pointer to a pointer, a common technique seems to be
setting a NULL pointer to the end of the list, and here we know that
the allocated memory has been exhausted. All good.

When this is a pointer to another type, say int, I could have a
variable that records how much memory is being allocated and use that
to track the size of the 'array'.
Alternatively, we could set the end of the 'array' to some kind of
error-code, such as 99 or MAX_INT.
I don't like either of these techniques.

So, what is a good way to stop a loop reading or writing past the
memory allocated to a pointer?
Or if possible, what is a good way of determining the size of memory
allocated to a pointer?

Cheers,
Matt

[Robert Latest]

On 2006-05-11, Howard Hinnant <ho************ @gmail.com> wrote:

The proposal is an expanded interface giving the programmer more
information about the runtime state of his program at little or no cost.
The information does not need to be generated, it already exists.
Correct. The information, to remind myself and the thread in
general, is the usable size of the memory block *really* obtained
in a malloc() call.
It's
just that there is currently no interface with which to obtain this
information.
Correct.
It has been shown that this extra information can have a
positive impact on program performance (not standard library efficiency)
if utilized. Programs that do not need or want this extra information
are free to ignore it at no cost.
Correct.
If you mean that an efficient library interface is beyond the scope of
the C language definition, I think you do the creators of this
definition a disservice.
Probably.
Personally I'm impressed that so many aspects
of C have withstood the test of time over the past 30 years.
So am I.
Given all
of the advances in software development in this time frame, it could not
have done so well without a fundamentally sound and useful interface.
True. Although parts of the interface are a little braindead.
I believe only a small part of this interface needs a minor tweak to
keep up with changing times.

I would like to see it as a commonly available extension. There
is no need to put it into the language itself.

There's a funny conundrum: Let's say that the allocated size of a
memory were available to the programmer just by calling a
function on the pointer to it -- like you said, an easily
implemented feature because the information is there. This would
immediately render the interface to each and every function that
requires a "buffer size" argument to accompany a pointer to
buffer space (such as fgets()) clumsy and stupid. Like in
fgets(buffer, memsize(buffer) , stream)

I would be interested to see if there is actually a sound
*technical* reason why the "allocated-size" information cannot be
made available through the C interface, such as that there could
be implementations on which this information is *not* available.

robert

[Keith Thompson]

Robert Latest <bo*******@yaho o.com> writes:

On 2006-05-11, Howard Hinnant <ho************ @gmail.com> wrote:

The proposal is an expanded interface giving the programmer more
information about the runtime state of his program at little or no cost.
The information does not need to be generated, it already exists.
Correct. The information, to remind myself and the thread in
general, is the usable size of the memory block *really* obtained
in a malloc() call.

You're assuming the information is there. I can imagine that it might
be either nonexistent or meaningless in some implementations . (I
don't know enough about actual implementations to know how common this
is.)

[...]
There's a funny conundrum: Let's say that the allocated size of a
memory were available to the programmer just by calling a
function on the pointer to it -- like you said, an easily
implemented feature because the information is there. This would
immediately render the interface to each and every function that
requires a "buffer size" argument to accompany a pointer to
buffer space (such as fgets()) clumsy and stupid. Like in
fgets(buffer, memsize(buffer) , stream)

Consider:
char buffer[80];
fgets(buffer, sizeof buffer, stream);
Since buffer isn't allocated by malloc(), there's no way fgets() can
determine how much space is allocated to it.

Presumably the memsize() function would invoke undefined behavior if
invoked with a pointer that doesn't point to space allocated by
malloc().

--
Keith Thompson (The_Other_Keit h) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

[CBFalconer]

Robert Latest wrote:

.... snip ...
I would be interested to see if there is actually a sound
*technical* reason why the "allocated-size" information cannot be
made available through the C interface, such as that there could
be implementations on which this information is *not* available.

Yes, there is. Any such function would obviously be system
specific, which is not a problem. However it could only be called
on pointers that had been allocated via malloc in the first place.
This does not apply to most pointers.

Another reason is that a system call will obviously never be as
efficient as a single data access, which is what is required for
simply remembering.

It is conceded that we already have the limitation to malloced
pointers for calls to free and realloc. This is one of the great
insecurities of the C pointer system. It is also the sort of
limitation that, in practice, makes runtime checking impossible.

There is also no reason why the information should be available.
Consider a system in which all memory is 'use once and discard'.
Not many such exist, outside the DeathStar series.

--
"If you want to post a followup via groups.google.c om, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson
More details at: <http://cfaj.freeshell. org/google/>
Also see <http://www.safalra.com/special/googlegroupsrep ly/>

[Joe Wright]

Robert Latest wrote:

On 2006-05-11, Howard Hinnant <ho************ @gmail.com> wrote:

The proposal is an expanded interface giving the programmer more
information about the runtime state of his program at little or no cost.
The information does not need to be generated, it already exists.

Correct. The information, to remind myself and the thread in
general, is the usable size of the memory block *really* obtained
in a malloc() call.

It's
just that there is currently no interface with which to obtain this
information.

Correct.

It has been shown that this extra information can have a
positive impact on program performance (not standard library efficiency)
if utilized. Programs that do not need or want this extra information
are free to ignore it at no cost.

Correct.

If you mean that an efficient library interface is beyond the scope of
the C language definition, I think you do the creators of this
definition a disservice.

Probably.

Personally I'm impressed that so many aspects
of C have withstood the test of time over the past 30 years.

So am I.

Given all
of the advances in software development in this time frame, it could not
have done so well without a fundamentally sound and useful interface.

True. Although parts of the interface are a little braindead.

I believe only a small part of this interface needs a minor tweak to
keep up with changing times.

I would like to see it as a commonly available extension. There
is no need to put it into the language itself.

There's a funny conundrum: Let's say that the allocated size of a
memory were available to the programmer just by calling a
function on the pointer to it -- like you said, an easily
implemented feature because the information is there. This would
immediately render the interface to each and every function that
requires a "buffer size" argument to accompany a pointer to
buffer space (such as fgets()) clumsy and stupid. Like in
fgets(buffer, memsize(buffer) , stream)

I would be interested to see if there is actually a sound
*technical* reason why the "allocated-size" information cannot be
made available through the C interface, such as that there could
be implementations on which this information is *not* available.

robert

A couple of years ago I created just such a system. It involves spoofing
such that *alloc and free call a wrapper which saves the requested size
and the returned pointer in a node in a linked list. I implemented a new
function, 'size_t size(void *);' which trips through the list, finds the
pointer and returns the size. The free() function is also wrapped such
that if the pointer is not found, it is a NOP such that attempts to
'free' wild or indeterminate pointers are blocked.

--
Joe Wright
"Everything should be made as simple as possible, but not simpler."
--- Albert Einstein ---

[Gordon Burditt]

>There's a funny conundrum: Let's say that the allocated size of a

memory were available to the programmer just by calling a
function on the pointer to it -- like you said, an easily
implemented feature because the information is there. This would
The information is there *FOR MALLOC'D MEMORY* and for the *beginning*
of a malloc'd buffer.
immediately render the interface to each and every function that
requires a "buffer size" argument to accompany a pointer to
buffer space (such as fgets()) clumsy and stupid. Like in
fgets(buffer , memsize(buffer) , stream)
There is no rule that the pointer passed to fgets() must be obtained
from malloc(), or that you only pass a pointer to the beginning
of such an area to fgets().
I would be interested to see if there is actually a sound
*technical* reason why the "allocated-size" information cannot be
made available through the C interface, such as that there could
be implementations on which this information is *not* available.

I think trying to debug a program that uses the "allocated-size"
information incorrectly would be a problem, especially if it
malfunctions only on systems where "what-you-requested-is-what-you-get"
but the system you're testing on doesn't do that.

Note that there are systems where you cannot (at least not cheaply)
validate a pointer that supposedly came from malloc(), and you
cannot find the beginning of a block given a pointer into it. And
you can't find the size of a buffer not allocated by malloc on
almost every system.

Gordon L. Burditt

[Keith Thompson]

Joe Wright <jo********@com cast.net> writes:
[...]

A couple of years ago I created just such a system. It involves
spoofing such that *alloc and free call a wrapper which saves the
requested size and the returned pointer in a node in a linked list. I
implemented a new function, 'size_t size(void *);' which trips through
the list, finds the pointer and returns the size. The free() function
is also wrapped such that if the pointer is not found, it is a NOP
such that attempts to 'free' wild or indeterminate pointers are
blocked.

Sounds interesting. It would have been even more interesting if your
free() wrapper had indicated an error if the pointer is not found.
This could be used to discover bugs rather than just masking them.
(If your program free()s an invalid pointer, ignoring the problem
isn't likely to fix the underlying problem.)

--
Keith Thompson (The_Other_Keit h) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

[Joe Wright]

Keith Thompson wrote:

Joe Wright <jo********@com cast.net> writes:
[...]

A couple of years ago I created just such a system. It involves
spoofing such that *alloc and free call a wrapper which saves the
requested size and the returned pointer in a node in a linked list. I
implemented a new function, 'size_t size(void *);' which trips through
the list, finds the pointer and returns the size. The free() function
is also wrapped such that if the pointer is not found, it is a NOP
such that attempts to 'free' wild or indeterminate pointers are
blocked.

Sounds interesting. It would have been even more interesting if your
free() wrapper had indicated an error if the pointer is not found.
This could be used to discover bugs rather than just masking them.
(If your program free()s an invalid pointer, ignoring the problem
isn't likely to fix the underlying problem.)

I did think about that but I am not trying to 'fix' the problem, but
rather to make it go away. You can pass any pointer you like to my
free() and if the pointer is valid, the memory is freed. If it is not,
nothing happens.

--
Joe Wright
"Everything should be made as simple as possible, but not simpler."
--- Albert Einstein ---

[Howard Hinnant]

In article <ln************ @nuthaus.mib.or g>,
Keith Thompson <ks***@mib.or g> wrote:

Robert Latest <bo*******@yaho o.com> writes:

On 2006-05-11, Howard Hinnant <ho************ @gmail.com> wrote:

The proposal is an expanded interface giving the programmer more
information about the runtime state of his program at little or no cost.
The information does not need to be generated, it already exists.

Correct. The information, to remind myself and the thread in
general, is the usable size of the memory block *really* obtained
in a malloc() call.

You're assuming the information is there. I can imagine that it might
be either nonexistent or meaningless in some implementations . (I
don't know enough about actual implementations to know how common this
is.)

I know. I've written commercial malloc systems for both desktop and
embedded (even bareboard) systems. If you're going to implement the C
realloc interface, you have to know the size of a pointer passed to you.
Furthermore, in such systems if the concept of adjacent blocks exists,
you can answer what the size of those blocks are, and whether or not
they are allocated. If the concept of adjacent blocks does not exist in
the allocator, then the answer to the question: Is there free memory
adjacent to this block of memory? Is: No.

It is pretty simple and cheap to answer these questions. It is
expensive not to be able to ask these questions.

-Howard

[S.Tobias]

Robert Latest <bo*******@yaho o.com> wrote:

On 2006-05-11, Ion Gaztañaga <ig********@gma il.com> wrote:

Specially for C, since it's a language concerned with
speed and resource usage.

No it isn't: (n869.txt is a late draft of C99)

$ for f in {speed,efficien cy,performance, resource}; do grep $f n869.txt; done
exceptions. The programmer can achieve the efficiency of

....by using "dedicated (and probably platform-specific) libraries"?
[checking...] No.
# exceptions. The programmer can achieve the efficiency of
# translation-time evaluation through static
# initialization, such as
# const static double one_third = 1.0/3.0;
(It's part of a footnote.)

From the same text file:
# This International Standard specifies the form and
# establishes the interpretation of programs expressed in the
# programming language C. Its purpose is to promote
# portability, reliability, maintainability , and efficient
# execution of C language programs on a variety of computing
# systems.

--
Stan Tobias
mailx `echo si***@FamOuS.Be dBuG.pAlS.INVALID | sed s/[[:upper:]]//g`

[CBFalconer]

Howard Hinnant wrote:

Keith Thompson <ks***@mib.or g> wrote:

.... snip ...

You're assuming the information is there. I can imagine that it
might be either nonexistent or meaningless in some implementations .
(I don't know enough about actual implementations to know how
common this is.)

I know. I've written commercial malloc systems for both desktop and
embedded (even bareboard) systems. If you're going to implement the C
realloc interface, you have to know the size of a pointer passed to you.

Not necessarily. Consider the hypothetical "one time use"
allocater. realloc need only allocate a new chunk of the
appropriate size and copy the old over. It doesn't need to know
the size of the old, even for the copying if it can detect 'the
end' by some other means, analogous to encountering EOF.

The only known system that does this is the DeathStar, and then
only when it will expose coding failures.

--
"If you want to post a followup via groups.google.c om, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson
More details at: <http://cfaj.freeshell. org/google/>
Also see <http://www.safalra.com/special/googlegroupsrep ly/>