472,968 Members | 1,749 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,968 software developers and data experts.

memory pool library ?

Any one know of an open source memory pool library?. I can't seem to
find any implemented in C (many available in C++ e.g. Boost). Google is
not turning up anything useful ...
Jul 5 '07 #1
11 7131
Grey Alien <gr**@andromeda.comwrites:
Any one know of an open source memory pool library?. I can't seem to
find any implemented in C (many available in C++ e.g. Boost). Google
is not turning up anything useful ...
It very much depends on what kind of pooling you're after; the
term is ambiguous. Apache has one implementation of memory
pools, GNU libc through its obstacks has another.
--
"Welcome to the wonderful world of undefined behavior, where the demons
are nasal and the DeathStation users are nervous." --Daniel Fox
Jul 5 '07 #2


Ben Pfaff wrote:
Grey Alien <gr**@andromeda.comwrites:

>>Any one know of an open source memory pool library?. I can't seem to
find any implemented in C (many available in C++ e.g. Boost). Google
is not turning up anything useful ...


It very much depends on what kind of pooling you're after; the
term is ambiguous. Apache has one implementation of memory
pools, GNU libc through its obstacks has another.
Anything simple really. I am currently alloc/dealloc small bits of
memory (a struct) several times during a programs lifecycle and I want
to alloc memory at prog start (say 5MB) so I don't have the overhead of
malloc/free (not to mention fragmentation). I can then request/release
chunks from the pool as and when necessary, during the prog lifecycle.
Jul 5 '07 #3
Grey Alien wrote:
... I am currently alloc/dealloc small bits of
memory (a struct) several times during a programs lifecycle and I want
to alloc memory at prog start (say 5MB) so I don't have the overhead of
malloc/free (not to mention fragmentation). I can then request/release
chunks from the pool as and when necessary, during the prog lifecycle.
The simplest system is to work with fixed-size memory blocks. Break the
large block into several blocks of the desired size. When a block is
deallocated, link it into a list of free blocks, which is used to
allocate. No searching, no merging. You can write that in about 30
minutes -- some here, much less!

--
Thad
Jul 6 '07 #4
Grey Alien wrote:
Ben Pfaff wrote:
Grey Alien <gr**@andromeda.comwrites:
>>Any one know of an open source memory pool library?. I can't
seem to find any implemented in C (many available in C++ e.g.
Boost). Google is not turning up anything useful ...

It very much depends on what kind of pooling you're after; the
term is ambiguous. Apache has one implementation of memory
pools, GNU libc through its obstacks has another.

Anything simple really. I am currently alloc/dealloc small bits
of memory (a struct) several times during a programs lifecycle
and I want to alloc memory at prog start (say 5MB) so I don't
have the overhead of malloc/free (not to mention fragmentation).
I can then request/release chunks from the pool as and when
necessary, during the prog lifecycle.
Take a look at the debug mechanism in my nmalloc package (for
DJGPP, but fairly portable), which is arranged to work with an
initially assigned block. See:

<http://cbfalconer.home.att.net/download/>

--
<http://www.cs.auckland.ac.nz/~pgut001/pubs/vista_cost.txt>
<http://www.securityfocus.com/columnists/423>
<http://www.aaxnet.com/editor/edit043.html>
cbfalconer at maineline dot net

--
Posted via a free Usenet account from http://www.teranews.com

Jul 6 '07 #5
On Jul 6, 3:53 am, Grey Alien <g...@andromeda.comwrote:
Any one know of an open source memory pool library?. I can't seem to
find any implemented in C (many available in C++ e.g. Boost). Google is
not turning up anything useful ...
http://cprops.sourceforge.net/
-Raxit
Jul 6 '07 #6
On Jul 6, 3:53 am, Grey Alien <g...@andromeda.comwrote:
Any one know of an open source memory pool library?. I can't seem to
find any implemented in C (many available in C++ e.g. Boost). Google is
not turning up anything useful ...
<possibly off-topic/implementation specific>

I just replied pointing to cprops, But anyone here to justify Explicit
use of Memory pool ? Don't implementaions are quiet smart enough to
decide ?

</possibly off-topic/implementation specific>

-Raxit

Jul 6 '07 #7
On Fri, 06 Jul 2007 00:39:12 -0400, CBFalconer <cb********@yahoo.com>
wrote:
>Grey Alien wrote:
>Ben Pfaff wrote:
Grey Alien <gr**@andromeda.comwrites:

Any one know of an open source memory pool library?. I can't
seem to find any implemented in C (many available in C++ e.g.
Boost). Google is not turning up anything useful ...

It very much depends on what kind of pooling you're after; the
term is ambiguous. Apache has one implementation of memory
pools, GNU libc through its obstacks has another.

Anything simple really. I am currently alloc/dealloc small bits
of memory (a struct) several times during a programs lifecycle
and I want to alloc memory at prog start (say 5MB) so I don't
have the overhead of malloc/free (not to mention fragmentation).
I can then request/release chunks from the pool as and when
necessary, during the prog lifecycle.

Take a look at the debug mechanism in my nmalloc package (for
DJGPP, but fairly portable), which is arranged to work with an
initially assigned block. See:
PC-lint for C/C++ (NT) Vers. 8.00u, Copyright Gimpel Software
1985-2006

--- Module: nmalloc.c (C)
_
#include <unistd.h/* sbrk, write */
nmalloc.c(78) : Error 322: Unable to open include file 'unistd.h'

<unistd.his not a standard C header. You could at least enclose its
include in some kind of preprocessor define.

--
jay

http://www.microsoft.com/windows/pro...a/default.mspx
Jul 6 '07 #8
jaysome wrote:
CBFalconer <cb********@yahoo.comwrote:
.... snip ...
>>
Take a look at the debug mechanism in my nmalloc package (for
DJGPP, but fairly portable), which is arranged to work with an
^^^^^^^^^^^^^^^^^^^
>initially assigned block. See:

PC-lint for C/C++ (NT) Vers. 8.00u, Copyright Gimpel Software
1985-2006

--- Module: nmalloc.c (C)
_
#include <unistd.h/* sbrk, write */
nmalloc.c(78) : Error 322: Unable to open include file 'unistd.h'

<unistd.his not a standard C header. You could at least enclose
its include in some kind of preprocessor define.
Note the underlined words in my original message. If I remember
aright, that include supplies write() and sbrk(). The debuggery
won't work from pre-initialization on without it.

--
<http://www.cs.auckland.ac.nz/~pgut001/pubs/vista_cost.txt>
<http://www.securityfocus.com/columnists/423>
<http://www.aaxnet.com/editor/edit043.html>
cbfalconer at maineline dot net

--
Posted via a free Usenet account from http://www.teranews.com

Jul 6 '07 #9


Thad Smith wrote:
Grey Alien wrote:
>... I am currently alloc/dealloc small bits of memory (a struct)
several times during a programs lifecycle and I want to alloc memory
at prog start (say 5MB) so I don't have the overhead of malloc/free
(not to mention fragmentation). I can then request/release chunks from
the pool as and when necessary, during the prog lifecycle.


The simplest system is to work with fixed-size memory blocks. Break the
large block into several blocks of the desired size. When a block is
deallocated, link it into a list of free blocks, which is used to
allocate. No searching, no merging. You can write that in about 30
minutes -- some here, much less!

struct mempool_t
{
unsigned char * pool_ ;
unsigned int block_size ;

/* variable that keeps track of whether an element is free or not */
unsigned char * available ;
};
/**************/
/* Public API */
/**************/

/* Creates the pool, N elements of size item_size.
Returns 0 on success, 1 otherwise */

int mempool_create(struct mempool_t *pool,
const unsigned int num_elements,
const unsigned int item_size);
/* Requests mem chunk from pool, 0 ptr if no more mem
or rqst could not be fulfilled */

unsigned char * mempool_malloc(struct mempool_t *pool,
const unsigned int num_elements) ;
/* Requests zero-inited mem chunk from pool,
0 ptr if no more mem or rqst could not be fulfilled */

unsigned char * mempool_calloc(struct mempool_t *pool,
const unsigned int num_elements) ;
/* Requests mem chunk realloc from pool,
0 ptr if no more mem or rqst could not be fulfilled */

unsigned char * mempool_realloc(struct mempool_t *pool,
const unsigned int num_elements) ;
/* Requests mempool resize (expand only) 0 if succesful, 1 otherwise */
int mempool_resize(struct mempool_t **pool,
const unsigned int num_elements) ;
/* Local (static) functions */
int mempool_defrag(struct mempool_t * pool);
Jul 6 '07 #10
On Jul 5, 4:30 pm, Grey Alien <g...@andromeda.comwrote:
Ben Pfaff wrote:
Grey Alien <g...@andromeda.comwrites:
>Any one know of an open source memory pool library?. I can't seem to
find any implemented in C (many available in C++ e.g. Boost). Google
is not turning up anything useful ...
It very much depends on what kind of pooling you're after; the
term is ambiguous. Apache has one implementation of memory
pools, GNU libc through its obstacks has another.

Anything simple really. I am currently alloc/dealloc small bits of
memory (a struct) several times during a programs lifecycle and I want
to alloc memory at prog start (say 5MB) so I don't have the overhead of
malloc/free (not to mention fragmentation). I can then request/release
chunks from the pool as and when necessary, during the prog lifecycle.
Are you sure there's any advantage to doing this? Have you measured
the overhead? What fragmentation problems are you seeing?

Jul 6 '07 #11
Grey Alien wrote:
Thad Smith wrote:
>Grey Alien wrote:
>>... I am currently alloc/dealloc small bits of memory (a struct)
several times during a programs lifecycle and I want to alloc memory
at prog start (say 5MB) so I don't have the overhead of malloc/free
(not to mention fragmentation). I can then request/release chunks
from the pool as and when necessary, during the prog lifecycle.

The simplest system is to work with fixed-size memory blocks. Break
the large block into several blocks of the desired size. When a block
is deallocated, link it into a list of free blocks, which is used to
allocate. No searching, no merging. You can write that in about 30
minutes -- some here, much less!

This is *EXACTLY* what I want to do - everything I have come accross so
far (thanks to those who responded with links), is overly complicated or
overkill for what I want to do.
...
This is what I've come up with so far. Comments/feedback welcome.
struct mempool_t
{

unsigned char * pool_ ;
unsigned int block_size ;

/* variable that keeps track of whether an element is free or not */
unsigned char * available ;
};
The only thing that you need is the lead of a free list:
void ** freelist;

You can wrap that in a struct if you want. You might add a block
counter if you want to be able to quickly tell how many blocks are free,
but this wasn't part of your requirements.

/* Public API */
/* Creates the pool, N elements of size item_size. Return 0 on succes, 1
otherwise */
int mempool_create(struct mempool_t *pool, const unsigned int
num_elements, const unsigned int item_size);
That looks good, although I would make the second and third parameters
of type size_t. There is no point in declaring the parameter const.
/* requests mem chunk from pool, 0 ptr if no more mem or rqst could not
be fulfilled */
unsigned char * mempool_malloc(struct mempool_t *pool, const unsigned
int num_elements) ;
Whoa. With the method I proposed, you would return a single buffer for
each call. The second parameter isn't needed. Also, I recommend
returning a void*, rather than unsigned char* to make it more generic.
/* requests zero-inited mem chunk from pool, 0 ptr if no more mem or
rqst could not be fulfilled */
unsigned char * mempool_calloc(struct mempool_t *pool, const unsigned
int num_elements) ;
The same comments for mempool_malloc() apply to mempool_calloc().
/* requests mem chunk realloc from pool, 0 ptr if no more mem or rqst
could not be fulfilled */
unsigned char * mempool_realloc(struct mempool_t *pool, const unsigned
int num_elements) ;
Stop. My proposal was for arrays of fixed size. A realloc function is
not appropriate.
/* requests mempool resize (expand only) 0 if succesful, 1 otherwise */
int mempool_resize(struct mempool_t **pool, const unsigned int
num_elements) ;
OK. This does make sense, although since it only expands I would
probably call it mempool_expand, where the parameter is the number of
additional blocks. This would also give a reason to save the original
size in the struct, which I missed earlier.
/* Local (static) functions */
int mempool_defrag(struct mempool_t * pool);
No. There is no point to defragmenting the pool because the only
benefit is to coalesce small portions to allow larger chunks to be
allocated. Since all buffers are the same size, it doesn't apply.

--
Thad
Jul 8 '07 #12

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

17
by: ~Gee | last post by:
Hi Folks! Please see the program below: 1 #include<iostream> 2 #include<list> 3 #include <unistd.h> 4 using namespace std; 5 int main() 6 { 7 {
22
by: xixi | last post by:
hi, we are using db2 udb v8.1 for windows, i have changed the buffer pool size to accommadate better performance, say size 200000, if i have multiple connection to the same database from...
5
by: bull | last post by:
hi could any one explain with example the following in a better way to understand 1. what is stack in memory, how the programs are stored in stack , what is the use of stack 2. What is heap in...
10
by: Markus.Elfring | last post by:
Some APIs/SDKs are available to modify settings like "readable", "writeable" or "executeable". Examples: - http://msdn.microsoft.com/library/en-us/memory/base/memory_protection_constants.asp -...
1
by: Gaël | last post by:
Hi everybody! I have a really big problem with ASP.NET application. I noticed that the w3wp.exe memory size, increase with the time and the use of my website. When it raise a certain value, w3wp...
6
by: Stan | last post by:
There was a number of postings about aspnet_wp worker process taking too much memory and eventually choking the webserver. One issue is still not clear to me - how can I narrow it down to an...
7
by: toton | last post by:
Hi, I have a STL vector of of characters and the character class has a Boost array of points. The things are vector<Characterchars; and class Character{ private: array<Point,Npoints; }; Now...
11
by: Grey Alien | last post by:
I am looking to write a very simple memory pool library to store only one data type at a time - i.e. to provide a contiguous block of memory to be alloc'd free'd by the calling program. I am I...
6
by: CANCER.0707 | last post by:
The problem statement is as follows Create a library that creates pools of memory of different sizes.For e.g. one pool of 32 byte size, another pool of 64 byte size and so on.Create an array of...
0
by: lllomh | last post by:
Define the method first this.state = { buttonBackgroundColor: 'green', isBlinking: false, // A new status is added to identify whether the button is blinking or not } autoStart=()=>{
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 4 Oct 2023 starting at 18:00 UK time (6PM UTC+1) and finishing at about 19:15 (7.15PM) The start time is equivalent to 19:00 (7PM) in Central...
0
tracyyun
by: tracyyun | last post by:
Hello everyone, I have a question and would like some advice on network connectivity. I have one computer connected to my router via WiFi, but I have two other computers that I want to be able to...
2
by: giovanniandrean | last post by:
The energy model is structured as follows and uses excel sheets to give input data: 1-Utility.py contains all the functions needed to calculate the variables and other minor things (mentions...
4
NeoPa
by: NeoPa | last post by:
Hello everyone. I find myself stuck trying to find the VBA way to get Access to create a PDF of the currently-selected (and open) object (Form or Report). I know it can be done by selecting :...
1
by: Teri B | last post by:
Hi, I have created a sub-form Roles. In my course form the user selects the roles assigned to the course. 0ne-to-many. One course many roles. Then I created a report based on the Course form and...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 1 Nov 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM) Please note that the UK and Europe revert to winter time on...
0
NeoPa
by: NeoPa | last post by:
Introduction For this article I'll be focusing on the Report (clsReport) class. This simply handles making the calling Form invisible until all of the Reports opened by it have been closed, when it...
3
by: GKJR | last post by:
Does anyone have a recommendation to build a standalone application to replace an Access database? I have my bookkeeping software I developed in Access that I would like to make available to other...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.