473,890 Members | 1,359 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

sizeof([ALLOCATED MEMORY])

If I have malloc()'ed a pointer and want to read from it as if it were
an array, I need to know that I won't be reading past the last index.

If this is a pointer to a pointer, a common technique seems to be
setting a NULL pointer to the end of the list, and here we know that
the allocated memory has been exhausted. All good.

When this is a pointer to another type, say int, I could have a
variable that records how much memory is being allocated and use that
to track the size of the 'array'.
Alternatively, we could set the end of the 'array' to some kind of
error-code, such as 99 or MAX_INT.
I don't like either of these techniques.

So, what is a good way to stop a loop reading or writing past the
memory allocated to a pointer?
Or if possible, what is a good way of determining the size of memory
allocated to a pointer?

Cheers,
Matt

May 3 '06
74 4728
On 2006-05-11, Howard Hinnant <ho************ @gmail.com> wrote:
The proposal is an expanded interface giving the programmer more
information about the runtime state of his program at little or no cost.
The information does not need to be generated, it already exists.
Correct. The information, to remind myself and the thread in
general, is the usable size of the memory block *really* obtained
in a malloc() call.
It's
just that there is currently no interface with which to obtain this
information.
Correct.
It has been shown that this extra information can have a
positive impact on program performance (not standard library efficiency)
if utilized. Programs that do not need or want this extra information
are free to ignore it at no cost.
Correct.
If you mean that an efficient library interface is beyond the scope of
the C language definition, I think you do the creators of this
definition a disservice.
Probably.
Personally I'm impressed that so many aspects
of C have withstood the test of time over the past 30 years.
So am I.
Given all
of the advances in software development in this time frame, it could not
have done so well without a fundamentally sound and useful interface.
True. Although parts of the interface are a little braindead.
I believe only a small part of this interface needs a minor tweak to
keep up with changing times.


I would like to see it as a commonly available extension. There
is no need to put it into the language itself.

There's a funny conundrum: Let's say that the allocated size of a
memory were available to the programmer just by calling a
function on the pointer to it -- like you said, an easily
implemented feature because the information is there. This would
immediately render the interface to each and every function that
requires a "buffer size" argument to accompany a pointer to
buffer space (such as fgets()) clumsy and stupid. Like in
fgets(buffer, memsize(buffer) , stream)

I would be interested to see if there is actually a sound
*technical* reason why the "allocated-size" information cannot be
made available through the C interface, such as that there could
be implementations on which this information is *not* available.

robert
May 14 '06 #41
Robert Latest <bo*******@yaho o.com> writes:
On 2006-05-11, Howard Hinnant <ho************ @gmail.com> wrote:
The proposal is an expanded interface giving the programmer more
information about the runtime state of his program at little or no cost.
The information does not need to be generated, it already exists.
Correct. The information, to remind myself and the thread in
general, is the usable size of the memory block *really* obtained
in a malloc() call.


You're assuming the information is there. I can imagine that it might
be either nonexistent or meaningless in some implementations . (I
don't know enough about actual implementations to know how common this
is.)

[...]
There's a funny conundrum: Let's say that the allocated size of a
memory were available to the programmer just by calling a
function on the pointer to it -- like you said, an easily
implemented feature because the information is there. This would
immediately render the interface to each and every function that
requires a "buffer size" argument to accompany a pointer to
buffer space (such as fgets()) clumsy and stupid. Like in
fgets(buffer, memsize(buffer) , stream)


Consider:
char buffer[80];
fgets(buffer, sizeof buffer, stream);
Since buffer isn't allocated by malloc(), there's no way fgets() can
determine how much space is allocated to it.

Presumably the memsize() function would invoke undefined behavior if
invoked with a pointer that doesn't point to space allocated by
malloc().

--
Keith Thompson (The_Other_Keit h) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
May 14 '06 #42
Robert Latest wrote:
.... snip ...
I would be interested to see if there is actually a sound
*technical* reason why the "allocated-size" information cannot be
made available through the C interface, such as that there could
be implementations on which this information is *not* available.


Yes, there is. Any such function would obviously be system
specific, which is not a problem. However it could only be called
on pointers that had been allocated via malloc in the first place.
This does not apply to most pointers.

Another reason is that a system call will obviously never be as
efficient as a single data access, which is what is required for
simply remembering.

It is conceded that we already have the limitation to malloced
pointers for calls to free and realloc. This is one of the great
insecurities of the C pointer system. It is also the sort of
limitation that, in practice, makes runtime checking impossible.

There is also no reason why the information should be available.
Consider a system in which all memory is 'use once and discard'.
Not many such exist, outside the DeathStar series.

--
"If you want to post a followup via groups.google.c om, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson
More details at: <http://cfaj.freeshell. org/google/>
Also see <http://www.safalra.com/special/googlegroupsrep ly/>
May 14 '06 #43
Robert Latest wrote:
On 2006-05-11, Howard Hinnant <ho************ @gmail.com> wrote:
The proposal is an expanded interface giving the programmer more
information about the runtime state of his program at little or no cost.
The information does not need to be generated, it already exists.


Correct. The information, to remind myself and the thread in
general, is the usable size of the memory block *really* obtained
in a malloc() call.
It's
just that there is currently no interface with which to obtain this
information.


Correct.
It has been shown that this extra information can have a
positive impact on program performance (not standard library efficiency)
if utilized. Programs that do not need or want this extra information
are free to ignore it at no cost.


Correct.
If you mean that an efficient library interface is beyond the scope of
the C language definition, I think you do the creators of this
definition a disservice.


Probably.
Personally I'm impressed that so many aspects
of C have withstood the test of time over the past 30 years.


So am I.
Given all
of the advances in software development in this time frame, it could not
have done so well without a fundamentally sound and useful interface.


True. Although parts of the interface are a little braindead.
I believe only a small part of this interface needs a minor tweak to
keep up with changing times.


I would like to see it as a commonly available extension. There
is no need to put it into the language itself.

There's a funny conundrum: Let's say that the allocated size of a
memory were available to the programmer just by calling a
function on the pointer to it -- like you said, an easily
implemented feature because the information is there. This would
immediately render the interface to each and every function that
requires a "buffer size" argument to accompany a pointer to
buffer space (such as fgets()) clumsy and stupid. Like in
fgets(buffer, memsize(buffer) , stream)

I would be interested to see if there is actually a sound
*technical* reason why the "allocated-size" information cannot be
made available through the C interface, such as that there could
be implementations on which this information is *not* available.

robert


A couple of years ago I created just such a system. It involves spoofing
such that *alloc and free call a wrapper which saves the requested size
and the returned pointer in a node in a linked list. I implemented a new
function, 'size_t size(void *);' which trips through the list, finds the
pointer and returns the size. The free() function is also wrapped such
that if the pointer is not found, it is a NOP such that attempts to
'free' wild or indeterminate pointers are blocked.

--
Joe Wright
"Everything should be made as simple as possible, but not simpler."
--- Albert Einstein ---
May 14 '06 #44
>There's a funny conundrum: Let's say that the allocated size of a
memory were available to the programmer just by calling a
function on the pointer to it -- like you said, an easily
implemented feature because the information is there. This would
The information is there *FOR MALLOC'D MEMORY* and for the *beginning*
of a malloc'd buffer.
immediately render the interface to each and every function that
requires a "buffer size" argument to accompany a pointer to
buffer space (such as fgets()) clumsy and stupid. Like in
fgets(buffer , memsize(buffer) , stream)
There is no rule that the pointer passed to fgets() must be obtained
from malloc(), or that you only pass a pointer to the beginning
of such an area to fgets().
I would be interested to see if there is actually a sound
*technical* reason why the "allocated-size" information cannot be
made available through the C interface, such as that there could
be implementations on which this information is *not* available.


I think trying to debug a program that uses the "allocated-size"
information incorrectly would be a problem, especially if it
malfunctions only on systems where "what-you-requested-is-what-you-get"
but the system you're testing on doesn't do that.

Note that there are systems where you cannot (at least not cheaply)
validate a pointer that supposedly came from malloc(), and you
cannot find the beginning of a block given a pointer into it. And
you can't find the size of a buffer not allocated by malloc on
almost every system.

Gordon L. Burditt
May 14 '06 #45
Joe Wright <jo********@com cast.net> writes:
[...]
A couple of years ago I created just such a system. It involves
spoofing such that *alloc and free call a wrapper which saves the
requested size and the returned pointer in a node in a linked list. I
implemented a new function, 'size_t size(void *);' which trips through
the list, finds the pointer and returns the size. The free() function
is also wrapped such that if the pointer is not found, it is a NOP
such that attempts to 'free' wild or indeterminate pointers are
blocked.


Sounds interesting. It would have been even more interesting if your
free() wrapper had indicated an error if the pointer is not found.
This could be used to discover bugs rather than just masking them.
(If your program free()s an invalid pointer, ignoring the problem
isn't likely to fix the underlying problem.)

--
Keith Thompson (The_Other_Keit h) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
May 14 '06 #46
Keith Thompson wrote:
Joe Wright <jo********@com cast.net> writes:
[...]
A couple of years ago I created just such a system. It involves
spoofing such that *alloc and free call a wrapper which saves the
requested size and the returned pointer in a node in a linked list. I
implemented a new function, 'size_t size(void *);' which trips through
the list, finds the pointer and returns the size. The free() function
is also wrapped such that if the pointer is not found, it is a NOP
such that attempts to 'free' wild or indeterminate pointers are
blocked.


Sounds interesting. It would have been even more interesting if your
free() wrapper had indicated an error if the pointer is not found.
This could be used to discover bugs rather than just masking them.
(If your program free()s an invalid pointer, ignoring the problem
isn't likely to fix the underlying problem.)

I did think about that but I am not trying to 'fix' the problem, but
rather to make it go away. You can pass any pointer you like to my
free() and if the pointer is valid, the memory is freed. If it is not,
nothing happens.

--
Joe Wright
"Everything should be made as simple as possible, but not simpler."
--- Albert Einstein ---
May 14 '06 #47
In article <ln************ @nuthaus.mib.or g>,
Keith Thompson <ks***@mib.or g> wrote:
Robert Latest <bo*******@yaho o.com> writes:
On 2006-05-11, Howard Hinnant <ho************ @gmail.com> wrote:
The proposal is an expanded interface giving the programmer more
information about the runtime state of his program at little or no cost.
The information does not need to be generated, it already exists.


Correct. The information, to remind myself and the thread in
general, is the usable size of the memory block *really* obtained
in a malloc() call.


You're assuming the information is there. I can imagine that it might
be either nonexistent or meaningless in some implementations . (I
don't know enough about actual implementations to know how common this
is.)


I know. I've written commercial malloc systems for both desktop and
embedded (even bareboard) systems. If you're going to implement the C
realloc interface, you have to know the size of a pointer passed to you.
Furthermore, in such systems if the concept of adjacent blocks exists,
you can answer what the size of those blocks are, and whether or not
they are allocated. If the concept of adjacent blocks does not exist in
the allocator, then the answer to the question: Is there free memory
adjacent to this block of memory? Is: No.

It is pretty simple and cheap to answer these questions. It is
expensive not to be able to ask these questions.

-Howard
May 14 '06 #48
Robert Latest <bo*******@yaho o.com> wrote:
On 2006-05-11, Ion Gaztañaga <ig********@gma il.com> wrote:
Specially for C, since it's a language concerned with
speed and resource usage.


No it isn't: (n869.txt is a late draft of C99)

$ for f in {speed,efficien cy,performance, resource}; do grep $f n869.txt; done
exceptions. The programmer can achieve the efficiency of


....by using "dedicated (and probably platform-specific) libraries"?
[checking...] No.
# exceptions. The programmer can achieve the efficiency of
# translation-time evaluation through static
# initialization, such as
# const static double one_third = 1.0/3.0;
(It's part of a footnote.)

From the same text file:
# This International Standard specifies the form and
# establishes the interpretation of programs expressed in the
# programming language C. Its purpose is to promote
# portability, reliability, maintainability , and efficient
# execution of C language programs on a variety of computing
# systems.

--
Stan Tobias
mailx `echo si***@FamOuS.Be dBuG.pAlS.INVALID | sed s/[[:upper:]]//g`
May 15 '06 #49
Howard Hinnant wrote:
Keith Thompson <ks***@mib.or g> wrote:

.... snip ...

You're assuming the information is there. I can imagine that it
might be either nonexistent or meaningless in some implementations .
(I don't know enough about actual implementations to know how
common this is.)


I know. I've written commercial malloc systems for both desktop and
embedded (even bareboard) systems. If you're going to implement the C
realloc interface, you have to know the size of a pointer passed to you.


Not necessarily. Consider the hypothetical "one time use"
allocater. realloc need only allocate a new chunk of the
appropriate size and copy the old over. It doesn't need to know
the size of the old, even for the copying if it can detect 'the
end' by some other means, analogous to encountering EOF.

The only known system that does this is the DeathStar, and then
only when it will expose coding failures.

--
"If you want to post a followup via groups.google.c om, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson
More details at: <http://cfaj.freeshell. org/google/>
Also see <http://www.safalra.com/special/googlegroupsrep ly/>
May 15 '06 #50

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
2052
by: Andreas Suurkuusk | last post by:
Hi, I just noticed your post in the "C# memory problem: no end for our problem?" thread. In the post you implied that I do not how the garbage collector works and that I mislead people. Since the thread is over a month old, I decided to start a new one with my response. Please see my comments inline.
4
13021
by: Frank Esser | last post by:
I am using SQL 8 Personal edition with sp2 applied. I set the max server memory to 32MB and leave the min server memory at 0. When my application starts hitting the database hard the memory usage reported through task manager peaks between 41-42MB. I've stopped and restarted the MSSQLserver service and checked that the running values are what I set them to be. Does anybody have any ideas as to why the sqlservr.exe would be utilizing more...
0
1055
by: Bill Burwell | last post by:
Which memory properties, or what combinations of memory properties, provide useful information about a program's memory usage when that program has just started leaking memory? While I have a VB bias, it seems to me the answer to this question should be generic - that is language independent.
4
2597
by: Franklin Lee | last post by:
Hi All, I use new to allocate some memory,even I doesn't use delete to release them. When my Application exit, OS will release them. Am I right? If I'm right, how about Thread especally on Solaries OS? This means that I use new to allocate memory in one Thread and doesn't use delete to release them.
9
2362
by: Mike P | last post by:
I know everything about reference counting and making sure you don't have large objects lying around. I have also profiled my app with multiple tools. I know about the fact GC collects memory but not necessary give it back to the OS. It seems that .NET win app will only return memory to the OS when the OS is asking for it. But!!! When the OS is asking for it is usually too late, tons of swapping and slow performance.
22
3496
by: xixi | last post by:
hi, we are using db2 udb v8.1 for windows, i have changed the buffer pool size to accommadate better performance, say size 200000, if i have multiple connection to the same database from application server, will each connection take the memory 800M (200000 x 4k = 800 M), so the memory took will be 800M times number of connections, or the total memory get from bufferpool will be 800M?
14
20797
by: Alessandro Monopoli | last post by:
Hi all, I'm searching a PORTABLE way to get the available and total physical memory. Something like "getTotalMemory" and it returns the memory installed on my PC in bytes, and "getAvailableMemory" and it returns the available memory in bytes. Do you know is there's a C function, a c++ Object or anything else that compiles in Linux and Windows to get these data?
5
24916
by: kumarmdb2 | last post by:
Hi guys, For last few days we are getting out of private memory error. We have a development environment. We tried to figure out the problem but we believe that it might be related to the OS (I am new to Windows so not sure). We are currently bouncing the instance to overcome this error. This generally happen at the end of business day only (So maybe memory might be getting used up?). We have already increased the statement heap & ...
1
2057
by: Jean-Paul Calderone | last post by:
On Tue, 22 Apr 2008 14:54:37 -0700 (PDT), yzghan@gmail.com wrote: The test doesn't demonstrate any leaks. It does demonstrate that memory usage can remain at or near peak memory usage even after the objects for which that memory was allocated are no longer live in the process. This is only a leak if peak memory goes up again each time you create any new objects. Try repeated allocations of a large dictionary and observe how memory...
5
505
by: cham | last post by:
Hi, I am working on c++ in a linux system ( Fedora core 4 ), kernel version - 2.6.11-1.1369_FC4 gcc version - 4.0.0 20050519 ( Red Hat 4.0.0-8 ) In my code i am creating a vector to store pointers of type structure "SAMPLE_TABLE_STRUCT" ( size of this structure is 36 bytes ). I create an instance of structure "SAMPLE_TABLE_STRUCT" using operator "new" and push back into the vector,this is done inside a for loop for
0
9980
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9826
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
11236
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10830
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
8018
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
7172
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5855
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
6061
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
2
4276
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.