473,881 Members | 1,670 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Virtual memory allocator recommendations ?

I have a situation where I need to be able to allocate chunks on
unmapped virtual memory.

Because the memory I'm allocating isn't mapped, I can't use a normal
memory allocator, as most of them want to store their metadata about the
free list in the unused holes in the heap. Instead, I need a memory
allocator that stores its metadata out-of-line (perversely enough, a
simple malloc() heap would be fine for that).

Does anyone know of such a beast?

This sort of thing would be useful for allocating any memory-like
resource, such as space in a file, or blocks of time in a day, or that
sort of thing. So I'm sure they're out there; I just can't find any...

--
┌─── dg*co wlark .com ───── http://www.cowlark.com ─────
│ "...it's not that well-designed GUI's are rare, it's just that the
│ three-armed users GUI's are designed for are rare." --- Mike Uhl on
│ a.f.c
Oct 7 '08 #1
12 3012
David Given wrote:
I have a situation where I need to be able to allocate chunks on
unmapped virtual memory.

Because the memory I'm allocating isn't mapped, I can't use a normal
memory allocator, as most of them want to store their metadata about the
free list in the unused holes in the heap. Instead, I need a memory
allocator that stores its metadata out-of-line (perversely enough, a
simple malloc() heap would be fine for that).

Does anyone know of such a beast?
It's not hard to do with system specific APIs, but I don't think there's
a what in standard C. So you'd be better off asking in a platform
specific group, this level of access tends to be very platform specific.

--
Ian Collins.
Oct 7 '08 #2
Ian Collins wrote:
David Given wrote:
>I have a situation where I need to be able to allocate chunks on
unmapped virtual memory.

Because the memory I'm allocating isn't mapped, I can't use a normal
memory allocator, as most of them want to store their metadata about the
free list in the unused holes in the heap. Instead, I need a memory
allocator that stores its metadata out-of-line (perversely enough, a
simple malloc() heap would be fine for that).

Does anyone know of such a beast?
It's not hard to do with system specific APIs, but I don't think there's
a what in standard C. So you'd be better off asking in a platform
specific group, this level of access tends to be very platform specific.
Oops, make that "way in standard C".

--
Ian Collins.
Oct 7 '08 #3
Ian Collins <ia******@hotma il.comwrites:
David Given wrote:
>I have a situation where I need to be able to allocate chunks on
unmapped virtual memory.

Because the memory I'm allocating isn't mapped, I can't use a normal
memory allocator, as most of them want to store their metadata about the
free list in the unused holes in the heap. Instead, I need a memory
allocator that stores its metadata out-of-line (perversely enough, a
simple malloc() heap would be fine for that).

Does anyone know of such a beast?
It's not hard to do with system specific APIs, but I don't think there's
a what in standard C. So you'd be better off asking in a platform
specific group, this level of access tends to be very platform specific.
Actually, what he's trying to do doesn't sound very system specific to
me. As he wrote in the original article:

This sort of thing would be useful for allocating any memory-like
resource, such as space in a file, or blocks of time in a day, or
that sort of thing. So I'm sure they're out there; I just can't
find any...

So what he's looking for is a way to allocate chunks out of some range
of <whatever(memor y addresses, file offsets, times of day) without
storing metadata alongside the allocated chunks. He could just as
well be allocating ranges of numbers.

I think the question is more about algorithms and data structures than
about any particular system or even any particular language, so
comp.programmin g is probably the best place to ask.

--
Keith Thompson (The_Other_Keit h) ks***@mib.org <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
Oct 8 '08 #4
On Tue, 07 Oct 2008 23:51:42 +0100, David Given <dg@cowlark.com >
wrote:
>I have a situation where I need to be able to allocate chunks on
unmapped virtual memory.

Because the memory I'm allocating isn't mapped, I can't use a normal
memory allocator, as most of them want to store their metadata about the
free list in the unused holes in the heap. Instead, I need a memory
allocator that stores its metadata out-of-line (perversely enough, a
simple malloc() heap would be fine for that).

Does anyone know of such a beast?

This sort of thing would be useful for allocating any memory-like
resource, such as space in a file, or blocks of time in a day, or that
sort of thing. So I'm sure they're out there; I just can't find any...
You might be interested in the getspace allocator that I wrote.
The source code and documentation are at
http://home.tiac.net/~cri_a/source_code/getspace/

The main reason I thought you might be interested is that all
metadata is stored separately from from the allocated space.
However is built on top of malloc/free and is portable. If this
works for you, you are welcome to go ahead and use it.
Richard Harter, cr*@tiac.net
http://home.tiac.net/~cri, http://www.varinoma.com
Save the Earth now!!
It's the only planet with chocolate.
Oct 8 '08 #5
Richard Harter wrote:
[...]
You might be interested in the getspace allocator that I wrote.
The source code and documentation are at
http://home.tiac.net/~cri_a/source_code/getspace/
Thanks, but being built on top of malloc it doesn't actually allocate
its own chunks, so it doesn't seem to be suitable for what I want, alas.

Any suggestions for other places to look? comp.lang.progr amming?

--
┌─── dg*co wlark .com ───── http://www.cowlark.com ─────
│ "...it's not that well-designed GUI's are rare, it's just that the
│ three-armed users GUI's are designed for are rare." --- Mike Uhl on
│ a.f.c
Oct 9 '08 #6
David Given wrote:
Richard Harter wrote:
[...]
>You might be interested in the getspace allocator that I wrote.
The source code and documentation are at
http://home.tiac.net/~cri_a/source_code/getspace/

Thanks, but being built on top of malloc it doesn't actually allocate
its own chunks, so it doesn't seem to be suitable for what I want, alas.

Any suggestions for other places to look? comp.lang.progr amming?
You didn't state your platform, but a group for that platform would be a
good place to ask.

--
Ian Collins
Oct 9 '08 #7
David Given <dg@cowlark.com writes:
Richard Harter wrote:
[...]
>You might be interested in the getspace allocator that I wrote.
The source code and documentation are at
http://home.tiac.net/~cri_a/source_code/getspace/

Thanks, but being built on top of malloc it doesn't actually allocate
its own chunks, so it doesn't seem to be suitable for what I want, alas.

Any suggestions for other places to look? comp.lang.progr amming?
There is no comp.lang.progr amming. I already suggested
comp.programmin g.

--
Keith Thompson (The_Other_Keit h) ks***@mib.org <http://www.ghoti.net/~kst>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
Oct 10 '08 #8
On Fri, 10 Oct 2008 00:35:17 +0100, David Given <dg@cowlark.com >
wrote:
>Richard Harter wrote:
[...]
>You might be interested in the getspace allocator that I wrote.
The source code and documentation are at
http://home.tiac.net/~cri_a/source_code/getspace/

Thanks, but being built on top of malloc it doesn't actually allocate
its own chunks, so it doesn't seem to be suitable for what I want, alas.

Any suggestions for other places to look? comp.lang.progr amming?
As others have said, you need to look at groups specific to your
platform. There is no way to portably write a storage allocator
in standard C only - to get chunks of memory you have to have
access to the OS memory management facilities.
Richard Harter, cr*@tiac.net
http://home.tiac.net/~cri, http://www.varinoma.com
Save the Earth now!!
It's the only planet with chocolate.
Oct 10 '08 #9
Richard Harter wrote:
[...]
As others have said, you need to look at groups specific to your
platform. There is no way to portably write a storage allocator
in standard C only - to get chunks of memory you have to have
access to the OS memory management facilities.
I think I haven't made myself clear.

I don't want anything platform specific. What I want is an allocator for
an *abstract linear resource*, which could be virtual memory, or time,
or space, or anything like that --- I want to be able to allocate chunks
from a contiguous linear range of numbers. Not only can this be done in
portable C, I *want* it in portable C, because I have to run it on at
least two highly dissimilar platforms.

i.e. an interface something like this:

typedef unsigned int domain_t;

void init_allocator( domain_t start, domain_t length);
domain_t allocrange(doma in_t length);
void freerange(domai n_t start);
domain_t resizerange(dom ain_t start, domain_t newlength);

domain_t is utterly abstract and does not need to represent any physical
resource. I only mentioned virtual memory because that's the specific
problem I need to solve, but I'm actually looking for a general solution.

As I said, this is a very similar problem to the one that's solved by
malloc(), but most malloc() implementations make assumptions about what
they're allocating that allows them to take shortcuts, which don't apply
in my case. Which means I can't use them.

--
David Given
dg@cowlark.com
Oct 10 '08 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
2109
by: Scott Brady Drummonds | last post by:
Hi, everyone, A coworker and I have been pondering a memory allocation problem that we're having with a very large process. Our joint research has led us to the conclusion that we may have to replace the STL allocator. But, before I even attempt something like this, I wanted to ask some questions to this group: 1) We read online that STL caches allocated memory so that containers and their contents that are freed at one point in...
1
2334
by: Spur | last post by:
Hi all, I implemented a memory allocation/deallocation class that logs all new/delete calls (overloaded) and remembers for each allocated block where it was allocated from (using a macro that passes __FILE__ and __LINE__). On destruction, it reports all undeleted blocks (memory leaks). The class is implemented in a separate .h/.cpp pair, and should be linked to
17
3876
by: ~Gee | last post by:
Hi Folks! Please see the program below: 1 #include<iostream> 2 #include<list> 3 #include <unistd.h> 4 using namespace std; 5 int main() 6 { 7 {
5
2483
by: Kai-Uwe Bux | last post by:
Hi, I decided to abandon the direct use of raw pointers in my programs and intend to replace them with templates like: dyn_array_of < typename T, typename Allocator > pointer_to < typename T, typename Allocator > polymorphic_ptr < typename T, typename Allocator >
3
3944
by: Chris | last post by:
I am having a very strange problem involving virtual functions in template classes. First of all, here is an extremely simplified structure of the two classes I am having problems with. template<class Type> class base { public: base& operator/=(const base&); Type *image;
8
4000
by: barcaroller | last post by:
I have a pointer to a memory block. Is there a way I can map a vector<Tto this memory block? I would like to take advantage of the powerful vector<T> member functions. Please note that I cannot copy the data into the vector because the memory block is used/read by other objects.
3
2084
by: yasmin | last post by:
I am dealing with internal memory fragmentation issues in my program (C++ program on freeBSD). It involves a large number of fixed-size structures. For example for my records which are typically fixed at 100 bytes, the default allocator allocates 128 bytes (closest power of two) wasting 28 bytes per records and this adds up to a significant amount over several thousand records. Before jumping in and writing my own custom allocator which...
3
2834
by: vrsathyan | last post by:
Hi.., While executing the following code in purifier.., std::vector<int> vecX; vecX.clear(); int iCount = 0; { int iVal;
2
4083
by: hari83 | last post by:
Hi, While compiling my source through make file, I recieve the below error: __RTTI__1nJTProperty4nJRWCString___ ../../../commonServices/lib/libCommonServices.a(ConfigFacility.o) void RW_PMapAssoc<std::map<RWCString*,SQLDataSource*,rw_deref_compare<std::less<RWCString>,RWCString>,std::allocator<SQLDataSource*> >,RWTPtrMap<RWCString,SQLDataSource,std::less<RWCString>,std::allocator<SQLDataSource*>...
0
9928
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, well explore What is ONU, What Is Router, ONU & Routers main usage, and What is the difference between ONU and Router. Lets take a closer look ! Part I. Meaning of...
0
10718
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10816
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
10401
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
7953
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupr who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
7110
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5781
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5977
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
2
4196
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.