Virtual memory allocator recommendations ?

[David Given]

I have a situation where I need to be able to allocate chunks on
unmapped virtual memory.

Because the memory I'm allocating isn't mapped, I can't use a normal
memory allocator, as most of them want to store their metadata about the
free list in the unused holes in the heap. Instead, I need a memory
allocator that stores its metadata out-of-line (perversely enough, a
simple malloc() heap would be fine for that).

Does anyone know of such a beast?

This sort of thing would be useful for allocating any memory-like
resource, such as space in a file, or blocks of time in a day, or that
sort of thing. So I'm sure they're out there; I just can't find any...

--
┌─── ｄｇï¼*ｃï½� ｗｌï½�ｒｋ ．ｃï½�ï½� ───── http://www.cowlark.com ─────
│ "...it's not that well-designed GUI's are rare, it's just that the
│ three-armed users GUI's are designed for are rare." --- Mike Uhl on
│ a.f.c

[Ian Collins]

David Given wrote:

I have a situation where I need to be able to allocate chunks on
unmapped virtual memory.

Because the memory I'm allocating isn't mapped, I can't use a normal
memory allocator, as most of them want to store their metadata about the
free list in the unused holes in the heap. Instead, I need a memory
allocator that stores its metadata out-of-line (perversely enough, a
simple malloc() heap would be fine for that).

Does anyone know of such a beast?

It's not hard to do with system specific APIs, but I don't think there's
a what in standard C. So you'd be better off asking in a platform
specific group, this level of access tends to be very platform specific.

--
Ian Collins.

[Ian Collins]

Ian Collins wrote:

David Given wrote:

>I have a situation where I need to be able to allocate chunks on
unmapped virtual memory.

Because the memory I'm allocating isn't mapped, I can't use a normal
memory allocator, as most of them want to store their metadata about the
free list in the unused holes in the heap. Instead, I need a memory
allocator that stores its metadata out-of-line (perversely enough, a
simple malloc() heap would be fine for that).

Does anyone know of such a beast?

It's not hard to do with system specific APIs, but I don't think there's
a what in standard C. So you'd be better off asking in a platform
specific group, this level of access tends to be very platform specific.

Oops, make that "way in standard C".

--
Ian Collins.

[Keith Thompson]

Ian Collins <ia******@hotma il.comwrites:

David Given wrote:

>I have a situation where I need to be able to allocate chunks on
unmapped virtual memory.

Because the memory I'm allocating isn't mapped, I can't use a normal
memory allocator, as most of them want to store their metadata about the
free list in the unused holes in the heap. Instead, I need a memory
allocator that stores its metadata out-of-line (perversely enough, a
simple malloc() heap would be fine for that).

Does anyone know of such a beast?

It's not hard to do with system specific APIs, but I don't think there's
a what in standard C. So you'd be better off asking in a platform
specific group, this level of access tends to be very platform specific.

Actually, what he's trying to do doesn't sound very system specific to
me. As he wrote in the original article:

This sort of thing would be useful for allocating any memory-like
resource, such as space in a file, or blocks of time in a day, or
that sort of thing. So I'm sure they're out there; I just can't
find any...

So what he's looking for is a way to allocate chunks out of some range
of <whatever(memor y addresses, file offsets, times of day) without
storing metadata alongside the allocated chunks. He could just as
well be allocating ranges of numbers.

I think the question is more about algorithms and data structures than
about any particular system or even any particular language, so
comp.programmin g is probably the best place to ask.

--
Keith Thompson (The_Other_Keit h) ks***@mib.org <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

[Richard Harter]

On Tue, 07 Oct 2008 23:51:42 +0100, David Given <[e-mail] >
wrote:

>I have a situation where I need to be able to allocate chunks on
unmapped virtual memory.

Because the memory I'm allocating isn't mapped, I can't use a normal
memory allocator, as most of them want to store their metadata about the
free list in the unused holes in the heap. Instead, I need a memory
allocator that stores its metadata out-of-line (perversely enough, a
simple malloc() heap would be fine for that).

Does anyone know of such a beast?

This sort of thing would be useful for allocating any memory-like
resource, such as space in a file, or blocks of time in a day, or that
sort of thing. So I'm sure they're out there; I just can't find any...

You might be interested in the getspace allocator that I wrote.
The source code and documentation are at
http://home.tiac.net/~cri_a/source_code/getspace/

The main reason I thought you might be interested is that all
metadata is stored separately from from the allocated space.
However is built on top of malloc/free and is portable. If this
works for you, you are welcome to go ahead and use it.
Richard Harter, cr*@tiac.net
http://home.tiac.net/~cri, http://www.varinoma.com
Save the Earth now!!
It's the only planet with chocolate.

[David Given]

Richard Harter wrote:
[...]

You might be interested in the getspace allocator that I wrote.
The source code and documentation are at
http://home.tiac.net/~cri_a/source_code/getspace/

Thanks, but being built on top of malloc it doesn't actually allocate
its own chunks, so it doesn't seem to be suitable for what I want, alas.

Any suggestions for other places to look? comp.lang.progr amming?

--
┌─── ｄｇï¼*ｃï½� ｗｌï½�ｒｋ ．ｃï½�ï½� ───── http://www.cowlark.com ─────
│ "...it's not that well-designed GUI's are rare, it's just that the
│ three-armed users GUI's are designed for are rare." --- Mike Uhl on
│ a.f.c

[Ian Collins]

David Given wrote:

Richard Harter wrote:
[...]

>You might be interested in the getspace allocator that I wrote.
The source code and documentation are at
http://home.tiac.net/~cri_a/source_code/getspace/

Thanks, but being built on top of malloc it doesn't actually allocate
its own chunks, so it doesn't seem to be suitable for what I want, alas.

Any suggestions for other places to look? comp.lang.progr amming?

You didn't state your platform, but a group for that platform would be a
good place to ask.

--
Ian Collins

[Keith Thompson]

David Given <[e-mail] writes:

Richard Harter wrote:
[...]

>You might be interested in the getspace allocator that I wrote.
The source code and documentation are at
http://home.tiac.net/~cri_a/source_code/getspace/

Thanks, but being built on top of malloc it doesn't actually allocate
its own chunks, so it doesn't seem to be suitable for what I want, alas.

Any suggestions for other places to look? comp.lang.progr amming?

There is no comp.lang.progr amming. I already suggested
comp.programmin g.

--
Keith Thompson (The_Other_Keit h) ks***@mib.org <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

[Richard Harter]

On Fri, 10 Oct 2008 00:35:17 +0100, David Given <[e-mail] >
wrote:

>Richard Harter wrote:
[...]

>You might be interested in the getspace allocator that I wrote.
The source code and documentation are at
http://home.tiac.net/~cri_a/source_code/getspace/

Thanks, but being built on top of malloc it doesn't actually allocate
its own chunks, so it doesn't seem to be suitable for what I want, alas.

Any suggestions for other places to look? comp.lang.progr amming?

As others have said, you need to look at groups specific to your
platform. There is no way to portably write a storage allocator
in standard C only - to get chunks of memory you have to have
access to the OS memory management facilities.
Richard Harter, cr*@tiac.net
http://home.tiac.net/~cri, http://www.varinoma.com
Save the Earth now!!
It's the only planet with chocolate.

[David Given]

Richard Harter wrote:
[...]

As others have said, you need to look at groups specific to your
platform. There is no way to portably write a storage allocator
in standard C only - to get chunks of memory you have to have
access to the OS memory management facilities.

I think I haven't made myself clear.

I don't want anything platform specific. What I want is an allocator for
an *abstract linear resource*, which could be virtual memory, or time,
or space, or anything like that --- I want to be able to allocate chunks
from a contiguous linear range of numbers. Not only can this be done in
portable C, I *want* it in portable C, because I have to run it on at
least two highly dissimilar platforms.

i.e. an interface something like this:

typedef unsigned int domain_t;

void init_allocator( domain_t start, domain_t length);
domain_t allocrange(doma in_t length);
void freerange(domai n_t start);
domain_t resizerange(dom ain_t start, domain_t newlength);

domain_t is utterly abstract and does not need to represent any physical
resource. I only mentioned virtual memory because that's the specific
problem I need to solve, but I'm actually looking for a general solution.

As I said, this is a very similar problem to the one that's solved by
malloc(), but most malloc() implementations make assumptions about what
they're allocating that allows them to take shortcuts, which don't apply
in my case. Which means I can't use them.

--
David Given
[e-mail]