473,836 Members | 2,210 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

memory allocation wrapper

Hello all

I've written a wrapper for malloc and friends. Its
available from

http://www.lelanthran.com/downloads/os_mem/index.php

The reason for doing writing this so that newbies
can finally get answers to the following
newbie questions:
1. How do I know how much memory a pointer points to?
2. How can I tell if a pointer is valid?

I know most of the regs don't need this, but be
a good samaritan and point out the bugs :-)

Comments and criticisms(sp?) welcome; note that it aims
for c99 compliance only.

goose,
Aug 18 '06 #1
10 2075
"goose" <lk***@webmail. co.zawrote in message
Hello all

I've written a wrapper for malloc and friends. Its
available from

http://www.lelanthran.com/downloads/os_mem/index.php

The reason for doing writing this so that newbies
can finally get answers to the following
newbie questions:
1. How do I know how much memory a pointer points to?
2. How can I tell if a pointer is valid?

I know most of the regs don't need this, but be
a good samaritan and point out the bugs :-)

Comments and criticisms(sp?) welcome; note that it aims
for c99 compliance only.

goose,
There's a much easier way of achieveing what you want to do.
Simply put a structure at the start of the allocated memory, add your
control information, and return the pointer immediately after.
Then when the user calls the free, subtract the size of the control
structure, and free. Similarly for the access functions.

There is a niggly snag. malloc() must return memory aligned for any purpose.
In practise you can get "good enough" portability by padding your structure
out to an even number of doubles. In standards land, you risk the horrors of
UB, and there is no way of achieving a fully compliant program in ANSI C.

--
www.personal.leeds.ac.uk/~bgy1mm
freeware games to download.
Aug 19 '06 #2
Malcolm wrote:
"goose" <lk***@webmail. co.zawrote in message
>>Hello all

I've written a wrapper for malloc and friends. Its
available from

http://www.lelanthran.com/downloads/os_mem/index.php

The reason for doing writing this so that newbies
can finally get answers to the following
newbie questions:
1. How do I know how much memory a pointer points to?
2. How can I tell if a pointer is valid?

I know most of the regs don't need this, but be
a good samaritan and point out the bugs :-)

Comments and criticisms(sp?) welcome; note that it aims
for c99 compliance only.

goose,

There's a much easier way of achieveing what you want to do.
Simply put a structure at the start of the allocated memory, add your
control information, and return the pointer immediately after.
Then when the user calls the free, subtract the size of the control
structure, and free. Similarly for the access functions.
I did initially have it this way but I wasn't able to
figure out how determine if a pointer is valid. The other
things I wanted was to be able to tell at any given time
how much total memory the program is using and to allow
the program to free all allocated memory.

The last (and most important) reason I did it this way
is because I'm designing my own language[1] and implementing
it in C and would like to have my language automatically GC.

Within My Own Toy Language if I only use os_mem_* functions
I can implement the gc much easier.
[1] Every programmer goes through the phase of thinking
"surely there must be a better way" and then goes off
designing their own language with all the nifty features
they'd like to have. It's probably a rite of passage, or
something :-)

goose,
Aug 19 '06 #3

goose wrote:
Hello all
Good start. You might consider adding:

(1)A sentinel field, say a char[4] == 'HEAP' before and after each
block, so you can detect block write overruns.

(2) Have your free() zero out the block data, to ensure the program
won't continue running okay even if it accesses the data after the
pointer is freed.

(3) make your free() a macro, so you can append a "ptr = NULL"
statement sot he user can't use a dangling pointer.

Aug 19 '06 #4
goose wrote:
I've written a wrapper for malloc and friends. Its
available from

http://www.lelanthran.com/downloads/os_mem/index.php

The reason for doing writing this so that newbies
can finally get answers to the following
newbie questions:
1. How do I know how much memory a pointer points to?
2. How can I tell if a pointer is valid?

I know most of the regs don't need this, but be
a good samaritan and point out the bugs :-)
It look fairly sound, but I did not look that deeply at it. I just
can't get out of my head the serious performance problems! find_ptr is
O(#allocations) , which is just brutal!
Comments and criticisms(sp?) welcome; note that it aims
for c99 compliance only.
Ok, since you are going for "C99" then why don't you use stdint.h? (In
fact, why don't you use pstdint.h, which you can find here:
http://www.pobox.com/~qed/pstdint.h ?)

With (p)stdint.h you can cast pointers to uintptr_t. This means you
can put a metric on pointers and thus sort and hash them. Without
this, I think this would just be too costly to be used by anyone.

--
Paul Hsieh
http://www.pobox.com/~qed/
http://bstring.sf.net/

Aug 19 '06 #5
"Ancient_Hacker " <gr**@comcast.n etwrites:
(2) Have your free() zero out the block data, to ensure the program
won't continue running okay even if it accesses the data after the
pointer is freed.
Clearing data to some value other than zero is probably better.
Zero is often a valid value for data and so it doesn't cause as
many problems as might be wanted.
(3) make your free() a macro, so you can append a "ptr = NULL"
statement sot he user can't use a dangling pointer.
The argument to free() is not necessarily an lvalue.
--
int main(void){char p[]="ABCDEFGHIJKLM NOPQRSTUVWXYZab cdefghijklmnopq rstuvwxyz.\
\n",*q="kl BIcNBFr.NKEzjwC IxNJC";int i=sizeof p/2;char *strchr();int putchar(\
);while(*q){i+= strchr(p,*q++)-p;if(i>=(int)si zeof p)i-=sizeof p-1;putchar(p[i]\
);}return 0;}
Aug 19 '06 #6
Ancient_Hacker wrote:
goose wrote:
>Hello all
Good start. You might consider adding:

(1)A sentinel field, say a char[4] == 'HEAP' before and after each
block, so you can detect block write overruns.

(2) Have your free() zero out the block data, to ensure the program
won't continue running okay even if it accesses the data after the
pointer is freed.
Some other value is probably better than zero; something more
recognizable like 0xDEADBEEF
(3) make your free() a macro, so you can append a "ptr = NULL"
statement sot he user can't use a dangling pointer.
Not a good idea; this will just hide errors.
--
Clark S. Cox III
cl*******@gmail .com
Aug 19 '06 #7

"goose" <lk***@webmail. co.zawrote in message
news:ec******** **@ctb-nnrp2.saix.net. ..
Malcolm wrote:
>"goose" <lk***@webmail. co.zawrote in message
>>>Hello all

I've written a wrapper for malloc and friends. Its
available from

http://www.lelanthran.com/downloads/os_mem/index.php

The reason for doing writing this so that newbies
can finally get answers to the following
newbie questions:
1. How do I know how much memory a pointer points to?
2. How can I tell if a pointer is valid?

I know most of the regs don't need this, but be
a good samaritan and point out the bugs :-)

Comments and criticisms(sp?) welcome; note that it aims
for c99 compliance only.

goose,

There's a much easier way of achieveing what you want to do.
Simply put a structure at the start of the allocated memory, add your
control information, and return the pointer immediately after.
Then when the user calls the free, subtract the size of the control
structure, and free. Similarly for the access functions.

I did initially have it this way but I wasn't able to
figure out how determine if a pointer is valid. The other
things I wanted was to be able to tell at any given time
how much total memory the program is using and to allow
the program to free all allocated memory.
Have a member called void *me which points to the structure itself. Then on
free check that this member is intact.
On a 4GB system you have only a 1 in 4 billion chance of being wrong. Shred
the pointer on free to avoid being freed twice.
--
www.personal.leeds.ac.uk/~bgy1mm
freeware games to download.
Aug 19 '06 #8
Clark S. Cox III wrote:
Ancient_Hacker wrote:
>goose wrote:
>>Hello all
Good start. You might consider adding:

(1)A sentinel field, say a char[4] == 'HEAP' before and after each
block, so you can detect block write overruns.

(2) Have your free() zero out the block data, to ensure the program
won't continue running okay even if it accesses the data after the
pointer is freed.

Some other value is probably better than zero; something more
recognizable like 0xDEADBEEF
>(3) make your free() a macro, so you can append a "ptr = NULL"
statement sot he user can't use a dangling pointer.

Not a good idea; this will just hide errors.
On which topic, on attempting to reallocate something not allocated
through your code you should at least print an error to stderr or even
abort the program. The same applies to freeing of course.

Your (goose) check for sorted pointers is not valid. You can only use
the relational operators (e.g. <) on pointers to the same object or one
past the object, not on pointers to different objects. Personally I
would just scrap the bit about unsorted pointers since I can't see the
value of it.

Other than that and the points others have made it looks like a good start.
--
Flash Gordon
Still sigless on this computer.
Aug 19 '06 #9
goose wrote:
Hello all
<snipped original announcement>

Hello all

I've summarised the replies here so that I could
reply to all without scattering the replies through
the thread.
---------------------------------------------------------------
Malcom wrote:
>>>>>There's a much easier way of achieveing what you want to do.
>Simply put a structure at the start of the allocated memory, add your
>control information, and return the pointer immediately after.
>Then when the user calls the free, subtract the size of the control
>structur e, and free. Similarly for the access functions.
>
I replied:
>>>
I did initially have it this way but I wasn't able to
figure out how determine if a pointer is valid. The other
things I wanted was to be able to tell at any given time
how much total memory the program is using and to allow
the program to free all allocated memory.
Malcolm replied:
>Have a member called void *me which points to the structure itself.
Then on
>free check that this member is intact.
On a 4GB system you have only a 1 in 4 billion chance of being wrong.
Shred
>the pointer on free to avoid being freed twice.
Certainly the odds are low, but I feel a little
uncomfortable knowing that every 4 billion pointers[1]
the valid() function is certain to succeed even
when the pointer is invalid.

[1]I'm not all that sure that that really is the
probability anyway. A more practical probability
is sure to be less than that due to me using
up addresses in os_mem itself.

---------------------------------------------------------------
Ancient Hacker wrote:
>Good start. You might consider adding:

(1)A sentinel field, say a char[4] == 'HEAP' before and after each
block, so you can detect block write overruns.
A sentinel is a good idea; I might add that in
(although its bound to never be 100% reliable).

The other two points sufficiently covered by
Ben Pfaff downthread.

---------------------------------------------------------------
Flash Gordon wrote:
>Clark S. Cox III wrote:
>Ancient_Hack er wrote:
>>goose wrote:

Hello all

Good start. You might consider adding:

(1)A sentinel field, say a char[4] == 'HEAP' before and after each
block, so you can detect block write overruns.

(2) Have your free() zero out the block data, to ensure the program
won't continue running okay even if it accesses the data after the
pointer is freed.


Some other value is probably better than zero; something more
recognizable like 0xDEADBEEF
>>(3) make your free() a macro, so you can append a "ptr = NULL"
statement sot he user can't use a dangling pointer.


Not a good idea; this will just hide errors.
[I've reformatted this to shorter line lengths]
>
On which topic, on attempting to reallocate something not
allocated through your code you should at least print an
error to stderr or even abort the program. The same applies
to freeing of course.
I agree, an attempt to realloc invalid pointer should be
handled more seriously (as it means the caller seriously
mixed up his pointers and there is probably more wrong
than simply reallocing a buffer not allocated by os_mem).

I think perhaps a callback in the init() function, or
raising a signal should be sufficient. Not really a good
idea to abort the program without letting the caller
clean up (close files, notify user, etc). Printing to
stderr might confuse the user without helping the
caller too much.

Obviously I need to put more thought into this.

>
Your (goose) check for sorted pointers is not valid. You can
only use the relational operators (e.g. <) on pointers to the
same object >or one past the object, not on pointers to
different objects. Personally I would just scrap the bit
about unsorted pointers since I can't see the value of it.
Well, neither could I, which is why it is in (I wanted the
list of pointers to be sorted so that find_ptr has reasonable
running time) but unused. I suppose I really should remove it,
but I'll need to make changes so that find_ptr has better
running time.
>
Other than that and the points others have made it looks like a good
start.
Thanks :-)
---------------------------------------------------------------
Paul Hsieh wrote:
>goose wrote:
>I've written a wrapper for malloc and friends. Its
available from

http://www.lelanthran.com/downloads/os_mem/index.php

The reason for doing writing this so that newbies
can finally get answers to the following
newbie questions:
1. How do I know how much memory a pointer points to?
2. How can I tell if a pointer is valid?

I know most of the regs don't need this, but be
a good samaritan and point out the bugs :-)


It look fairly sound, but I did not look that deeply at it. I just
can't get out of my head the serious performance problems! find_ptr is
O(#allocations ), which is just brutal!
Yes, I knew that when I wrote it :-(. I was banking on
the caller not using memory routines in any time-critical
part of the program as malloc and friends generally are
quite time-intensive anyway. The reason it got written that
way was because I figured I'd profile and optimise
later when I used this in any real project.
>
>Comments and criticisms(sp?) welcome; note that it aims
for c99 compliance only.


Ok, since you are going for "C99" then why don't you use stdint.h? (In
fact, why don't you use pstdint.h, which you can find here:
http://www.pobox.com/~qed/pstdint.h ?)
I'm not familiar with pstdint.h.
>With (p)stdint.h you can cast pointers to uintptr_t. This means you
can put a metric on pointers and thus sort and hash them.
I'll give it a quick look this coming week. Thanks for the
response though.
>Without
this, I think this would just be too costly to be used by anyone.
Well, a quick way to speed up the implementation of find_ptr
would be to search from the beginning and the end of the array
at the same time (in the same loop); this will effectively
halve the running time of find_ptr. Makes it O(n/2) instead
of O(n). I'll profile before and after making this change
(and add a file detailing the profiling in with the rest
of the project). If it is still unacceptable, then I'll
investigate the hash solution.

---------------------------------------------------------------
Thanks to all for your comments; I'll do the best I can
to follow them and where I don't I've explained above.
If i've not replied to your post in this post, that means
I agree with you :-).

I'll update os_mem, repost to my website and let you
all know when it's ready for criticisms again (probably
only next weekend).

Later
goose,
ps. I've also decided to change the licence from
GPL to BSD; this allows the user to freely incorporate
os_mem into closed code while keeping their code
closed.
Aug 20 '06 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
8215
by: chris | last post by:
Hi all, I need to know, what is the difference between dynamic memory allocation, and stack allocation ? 1. If I have a class named DestinationAddress, when should I use dynamic memory allocation to create object of that class ? 2. If it says "dynamic memory allocation", is it mean the following code : DestinationAddress* dest = new DestinationAddress(); // code 1
66
3655
by: Johan Tibell | last post by:
I've written a piece of code that uses sockets a lot (I know that sockets aren't portable C, this is not a question about sockets per se). Much of my code ended up looking like this: if (function(socket, args) == -1) { perror("function"); exit(EXIT_FAILURE); } I feel that the ifs destroy the readability of my code. Would it be
24
19106
by: Ken | last post by:
In C programming, I want to know in what situations we should use static memory allocation instead of dynamic memory allocation. My understanding is that static memory allocation like using array is faster than malloc, but dynamic memory allocation is more flexible. Please comment... thanks.
1
7981
by: Peterwkc | last post by:
Hello all expert, i have two program which make me desperate bu after i have noticed the forum, my future is become brightness back. By the way, my problem is like this i the first program was compiled and run without any erros but the second program has a run time error when the function return from allocate and the ptr become NULL. How to fixed this? Second Program: /* Best Method to allocate memory for 2D Array because it's ...
3
3000
by: ranjeetasharma81 | last post by:
Hi all, I have a big C-cod, in which there are lots of dynamic memory allocation used. I want to replace dynamic memroy allocation by static arrays. The following are the problems that i am facing: 1- From structure and dynamic memory allocation point of view, the code is very complicated. The code has various “nested structures” with a number of levels. The size of memory allocated for pointer to structure or its pointer...
1
1670
by: David d'Angers | last post by:
hi group: i've been struggling with this simple problem with dismay i need all the file names in a directory since the number of files in a directory is unknown, a friend here in the group hinted me to use dynamic memory allocation: namely functions like malloc etc in the spirit of code re-use
14
3842
by: vivek | last post by:
i have some doubts on dynamic memory allocation and stacks and heaps where is the dynamic memory allocation used? in function calls there are some counters like "i" in the below function. Is this stored in stack. If yes whether it will be deleted on exiting from the function. is dynamic memory allocation needed for this purpose
10
4434
by: swornavidhya.mahadevan | last post by:
Which allocation (Static / Dynamic) is suitable for the situation when we are trying to allocate for a overloaded memory when the memory is full and no space to allocate. What will happen if both the allocation is impossible. Sworna vidhya
2
2294
by: Andy Baker | last post by:
I have recently written a .NET wrapper for a C++ DLL file and it seems to be working well, but I do have some concerns about memory. From my limited C programming experience (college 15+ years ago), I seem to remember that when using structures etc in C, you had to allocate memory to them then deallocate them when you had finished using them. My C# code creates a variable of type structure, passes it to the C++ DLL and then returns...
0
9658
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10824
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10533
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10579
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
10244
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
7775
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5813
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4443
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
3103
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.