467,915 Members | 1,558 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 467,915 developers. It's quick & easy.

memory pool library ?

Any one know of an open source memory pool library?. I can't seem to
find any implemented in C (many available in C++ e.g. Boost). Google is
not turning up anything useful ...
Jul 5 '07 #1
  • viewed: 6450
Share:
11 Replies
Grey Alien <gr**@andromeda.comwrites:
Any one know of an open source memory pool library?. I can't seem to
find any implemented in C (many available in C++ e.g. Boost). Google
is not turning up anything useful ...
It very much depends on what kind of pooling you're after; the
term is ambiguous. Apache has one implementation of memory
pools, GNU libc through its obstacks has another.
--
"Welcome to the wonderful world of undefined behavior, where the demons
are nasal and the DeathStation users are nervous." --Daniel Fox
Jul 5 '07 #2


Ben Pfaff wrote:
Grey Alien <gr**@andromeda.comwrites:

>>Any one know of an open source memory pool library?. I can't seem to
find any implemented in C (many available in C++ e.g. Boost). Google
is not turning up anything useful ...


It very much depends on what kind of pooling you're after; the
term is ambiguous. Apache has one implementation of memory
pools, GNU libc through its obstacks has another.
Anything simple really. I am currently alloc/dealloc small bits of
memory (a struct) several times during a programs lifecycle and I want
to alloc memory at prog start (say 5MB) so I don't have the overhead of
malloc/free (not to mention fragmentation). I can then request/release
chunks from the pool as and when necessary, during the prog lifecycle.
Jul 5 '07 #3
Grey Alien wrote:
... I am currently alloc/dealloc small bits of
memory (a struct) several times during a programs lifecycle and I want
to alloc memory at prog start (say 5MB) so I don't have the overhead of
malloc/free (not to mention fragmentation). I can then request/release
chunks from the pool as and when necessary, during the prog lifecycle.
The simplest system is to work with fixed-size memory blocks. Break the
large block into several blocks of the desired size. When a block is
deallocated, link it into a list of free blocks, which is used to
allocate. No searching, no merging. You can write that in about 30
minutes -- some here, much less!

--
Thad
Jul 6 '07 #4
Grey Alien wrote:
Ben Pfaff wrote:
Grey Alien <gr**@andromeda.comwrites:
>>Any one know of an open source memory pool library?. I can't
seem to find any implemented in C (many available in C++ e.g.
Boost). Google is not turning up anything useful ...

It very much depends on what kind of pooling you're after; the
term is ambiguous. Apache has one implementation of memory
pools, GNU libc through its obstacks has another.

Anything simple really. I am currently alloc/dealloc small bits
of memory (a struct) several times during a programs lifecycle
and I want to alloc memory at prog start (say 5MB) so I don't
have the overhead of malloc/free (not to mention fragmentation).
I can then request/release chunks from the pool as and when
necessary, during the prog lifecycle.
Take a look at the debug mechanism in my nmalloc package (for
DJGPP, but fairly portable), which is arranged to work with an
initially assigned block. See:

<http://cbfalconer.home.att.net/download/>

--
<http://www.cs.auckland.ac.nz/~pgut001/pubs/vista_cost.txt>
<http://www.securityfocus.com/columnists/423>
<http://www.aaxnet.com/editor/edit043.html>
cbfalconer at maineline dot net

--
Posted via a free Usenet account from http://www.teranews.com

Jul 6 '07 #5
On Jul 6, 3:53 am, Grey Alien <g...@andromeda.comwrote:
Any one know of an open source memory pool library?. I can't seem to
find any implemented in C (many available in C++ e.g. Boost). Google is
not turning up anything useful ...
http://cprops.sourceforge.net/
-Raxit
Jul 6 '07 #6
On Jul 6, 3:53 am, Grey Alien <g...@andromeda.comwrote:
Any one know of an open source memory pool library?. I can't seem to
find any implemented in C (many available in C++ e.g. Boost). Google is
not turning up anything useful ...
<possibly off-topic/implementation specific>

I just replied pointing to cprops, But anyone here to justify Explicit
use of Memory pool ? Don't implementaions are quiet smart enough to
decide ?

</possibly off-topic/implementation specific>

-Raxit

Jul 6 '07 #7
On Fri, 06 Jul 2007 00:39:12 -0400, CBFalconer <cb********@yahoo.com>
wrote:
>Grey Alien wrote:
>Ben Pfaff wrote:
Grey Alien <gr**@andromeda.comwrites:

Any one know of an open source memory pool library?. I can't
seem to find any implemented in C (many available in C++ e.g.
Boost). Google is not turning up anything useful ...

It very much depends on what kind of pooling you're after; the
term is ambiguous. Apache has one implementation of memory
pools, GNU libc through its obstacks has another.

Anything simple really. I am currently alloc/dealloc small bits
of memory (a struct) several times during a programs lifecycle
and I want to alloc memory at prog start (say 5MB) so I don't
have the overhead of malloc/free (not to mention fragmentation).
I can then request/release chunks from the pool as and when
necessary, during the prog lifecycle.

Take a look at the debug mechanism in my nmalloc package (for
DJGPP, but fairly portable), which is arranged to work with an
initially assigned block. See:
PC-lint for C/C++ (NT) Vers. 8.00u, Copyright Gimpel Software
1985-2006

--- Module: nmalloc.c (C)
_
#include <unistd.h/* sbrk, write */
nmalloc.c(78) : Error 322: Unable to open include file 'unistd.h'

<unistd.his not a standard C header. You could at least enclose its
include in some kind of preprocessor define.

--
jay

http://www.microsoft.com/windows/pro...a/default.mspx
Jul 6 '07 #8
jaysome wrote:
CBFalconer <cb********@yahoo.comwrote:
.... snip ...
>>
Take a look at the debug mechanism in my nmalloc package (for
DJGPP, but fairly portable), which is arranged to work with an
^^^^^^^^^^^^^^^^^^^
>initially assigned block. See:

PC-lint for C/C++ (NT) Vers. 8.00u, Copyright Gimpel Software
1985-2006

--- Module: nmalloc.c (C)
_
#include <unistd.h/* sbrk, write */
nmalloc.c(78) : Error 322: Unable to open include file 'unistd.h'

<unistd.his not a standard C header. You could at least enclose
its include in some kind of preprocessor define.
Note the underlined words in my original message. If I remember
aright, that include supplies write() and sbrk(). The debuggery
won't work from pre-initialization on without it.

--
<http://www.cs.auckland.ac.nz/~pgut001/pubs/vista_cost.txt>
<http://www.securityfocus.com/columnists/423>
<http://www.aaxnet.com/editor/edit043.html>
cbfalconer at maineline dot net

--
Posted via a free Usenet account from http://www.teranews.com

Jul 6 '07 #9


Thad Smith wrote:
Grey Alien wrote:
>... I am currently alloc/dealloc small bits of memory (a struct)
several times during a programs lifecycle and I want to alloc memory
at prog start (say 5MB) so I don't have the overhead of malloc/free
(not to mention fragmentation). I can then request/release chunks from
the pool as and when necessary, during the prog lifecycle.


The simplest system is to work with fixed-size memory blocks. Break the
large block into several blocks of the desired size. When a block is
deallocated, link it into a list of free blocks, which is used to
allocate. No searching, no merging. You can write that in about 30
minutes -- some here, much less!

struct mempool_t
{
unsigned char * pool_ ;
unsigned int block_size ;

/* variable that keeps track of whether an element is free or not */
unsigned char * available ;
};
/**************/
/* Public API */
/**************/

/* Creates the pool, N elements of size item_size.
Returns 0 on success, 1 otherwise */

int mempool_create(struct mempool_t *pool,
const unsigned int num_elements,
const unsigned int item_size);
/* Requests mem chunk from pool, 0 ptr if no more mem
or rqst could not be fulfilled */

unsigned char * mempool_malloc(struct mempool_t *pool,
const unsigned int num_elements) ;
/* Requests zero-inited mem chunk from pool,
0 ptr if no more mem or rqst could not be fulfilled */

unsigned char * mempool_calloc(struct mempool_t *pool,
const unsigned int num_elements) ;
/* Requests mem chunk realloc from pool,
0 ptr if no more mem or rqst could not be fulfilled */

unsigned char * mempool_realloc(struct mempool_t *pool,
const unsigned int num_elements) ;
/* Requests mempool resize (expand only) 0 if succesful, 1 otherwise */
int mempool_resize(struct mempool_t **pool,
const unsigned int num_elements) ;
/* Local (static) functions */
int mempool_defrag(struct mempool_t * pool);
Jul 6 '07 #10
On Jul 5, 4:30 pm, Grey Alien <g...@andromeda.comwrote:
Ben Pfaff wrote:
Grey Alien <g...@andromeda.comwrites:
>Any one know of an open source memory pool library?. I can't seem to
find any implemented in C (many available in C++ e.g. Boost). Google
is not turning up anything useful ...
It very much depends on what kind of pooling you're after; the
term is ambiguous. Apache has one implementation of memory
pools, GNU libc through its obstacks has another.

Anything simple really. I am currently alloc/dealloc small bits of
memory (a struct) several times during a programs lifecycle and I want
to alloc memory at prog start (say 5MB) so I don't have the overhead of
malloc/free (not to mention fragmentation). I can then request/release
chunks from the pool as and when necessary, during the prog lifecycle.
Are you sure there's any advantage to doing this? Have you measured
the overhead? What fragmentation problems are you seeing?

Jul 6 '07 #11
Grey Alien wrote:
Thad Smith wrote:
>Grey Alien wrote:
>>... I am currently alloc/dealloc small bits of memory (a struct)
several times during a programs lifecycle and I want to alloc memory
at prog start (say 5MB) so I don't have the overhead of malloc/free
(not to mention fragmentation). I can then request/release chunks
from the pool as and when necessary, during the prog lifecycle.

The simplest system is to work with fixed-size memory blocks. Break
the large block into several blocks of the desired size. When a block
is deallocated, link it into a list of free blocks, which is used to
allocate. No searching, no merging. You can write that in about 30
minutes -- some here, much less!

This is *EXACTLY* what I want to do - everything I have come accross so
far (thanks to those who responded with links), is overly complicated or
overkill for what I want to do.
...
This is what I've come up with so far. Comments/feedback welcome.
struct mempool_t
{

unsigned char * pool_ ;
unsigned int block_size ;

/* variable that keeps track of whether an element is free or not */
unsigned char * available ;
};
The only thing that you need is the lead of a free list:
void ** freelist;

You can wrap that in a struct if you want. You might add a block
counter if you want to be able to quickly tell how many blocks are free,
but this wasn't part of your requirements.

/* Public API */
/* Creates the pool, N elements of size item_size. Return 0 on succes, 1
otherwise */
int mempool_create(struct mempool_t *pool, const unsigned int
num_elements, const unsigned int item_size);
That looks good, although I would make the second and third parameters
of type size_t. There is no point in declaring the parameter const.
/* requests mem chunk from pool, 0 ptr if no more mem or rqst could not
be fulfilled */
unsigned char * mempool_malloc(struct mempool_t *pool, const unsigned
int num_elements) ;
Whoa. With the method I proposed, you would return a single buffer for
each call. The second parameter isn't needed. Also, I recommend
returning a void*, rather than unsigned char* to make it more generic.
/* requests zero-inited mem chunk from pool, 0 ptr if no more mem or
rqst could not be fulfilled */
unsigned char * mempool_calloc(struct mempool_t *pool, const unsigned
int num_elements) ;
The same comments for mempool_malloc() apply to mempool_calloc().
/* requests mem chunk realloc from pool, 0 ptr if no more mem or rqst
could not be fulfilled */
unsigned char * mempool_realloc(struct mempool_t *pool, const unsigned
int num_elements) ;
Stop. My proposal was for arrays of fixed size. A realloc function is
not appropriate.
/* requests mempool resize (expand only) 0 if succesful, 1 otherwise */
int mempool_resize(struct mempool_t **pool, const unsigned int
num_elements) ;
OK. This does make sense, although since it only expands I would
probably call it mempool_expand, where the parameter is the number of
additional blocks. This would also give a reason to save the original
size in the struct, which I missed earlier.
/* Local (static) functions */
int mempool_defrag(struct mempool_t * pool);
No. There is no point to defragmenting the pool because the only
benefit is to coalesce small portions to allow larger chunks to be
allocated. Since all buffers are the same size, it doesn't apply.

--
Thad
Jul 8 '07 #12

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

17 posts views Thread by ~Gee | last post: by
22 posts views Thread by xixi | last post: by
5 posts views Thread by bull | last post: by
10 posts views Thread by Markus.Elfring | last post: by
1 post views Thread by GaŽl | last post: by
11 posts views Thread by Grey Alien | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.