Hi people,
I am writing a library that will implement a binary file format.
Well... I'm thinking, as a design, it would be nice to have my library
able to promise to not use more than a certain amount of RAM. I could
have a "SetCacheSize(l ong NewSize);" function. So as part of that
promise, when required memory exceeds that limit, the extra data is
saved to disk and flushed from RAM.
Now to keep that promise, it would be nice to have some control over
the memory management. For example, with malloc, you can't really be
sure of how much RAM is allocated. I'd like to just allocate a block,
say 16MB at first, and then suballocate out of there. I'd also like to
know if I can't allocate from that 16MB (or whatever size it is), so I
can start freeing cached data.
I heard that C++ allocators might be a solution. I've never used
allocators before, but they look promising. For me to use single
allocators, I'd like these qualities.
1) Portable across the main desktop platforms (linux, win32, Mac)
2) Quite fast. Single-threaded is fine if it helps increase speed.
3) Pooled allocation. I pre-define a maximum size that it can allocate
out of, and it allocates out of there. It could pre-allocate the entire
amount, or just allocate in blocks that will never become more than
this limit.
4) Lets me know if we have enough space to allocate another object, or
it's design is simple enough that I can figure out for my self if we
have enough space within the pool. (If not, I'll just flush many
objects to the disk and free up some RAM :) )
5) Doesn't suck
Any ideas anyone? Like I said, I don't know anything about allocators.
would gcc's "__pool_all oc" do this for me?
5  4172  co**********@ho tmail.com wrote:
Hi people,
I am writing a library that will implement a binary file format.
Well... I'm thinking, as a design, it would be nice to have my library
able to promise to not use more than a certain amount of RAM. I could
have a "SetCacheSize(l ong NewSize);" function. So as part of that
promise, when required memory exceeds that limit, the extra data is
saved to disk and flushed from RAM.
Now to keep that promise, it would be nice to have some control over
the memory management. For example, with malloc, you can't really be
sure of how much RAM is allocated. I'd like to just allocate a block,
say 16MB at first, and then suballocate out of there. I'd also like to
know if I can't allocate from that 16MB (or whatever size it is), so I
can start freeing cached data.
I heard that C++ allocators might be a solution. I've never used
allocators before, but they look promising. For me to use single
allocators, I'd like these qualities.
1) Portable across the main desktop platforms (linux, win32, Mac)
2) Quite fast. Single-threaded is fine if it helps increase speed.
3) Pooled allocation. I pre-define a maximum size that it can allocate
out of, and it allocates out of there. It could pre-allocate the entire
amount, or just allocate in blocks that will never become more than
this limit.
4) Lets me know if we have enough space to allocate another object, or
it's design is simple enough that I can figure out for my self if we
have enough space within the pool. (If not, I'll just flush many
objects to the disk and free up some RAM :) )
5) Doesn't suck
Any ideas anyone? Like I said, I don't know anything about allocators.
would gcc's "__pool_all oc" do this for me?
I don't have a lot to add here, but I'll mention that "allocators " are
specifically objects used by standard library containers for dynamic
memory allocation. This is generally only a subset of the total memory
allocations of a program. For more general control you'll probably also
need to at least redefine operator new to use a custom memory allocator.
And this does nothing for memory usage of "stack" objects.
I don't have a lot to add here, but I'll mention that "allocators " are
specifically objects used by standard library containers for dynamic
memory allocation. This is generally only a subset of the total memory
allocations of a program. For more general control you'll probably also
need to at least redefine operator new to use a custom memory allocator.
And this does nothing for memory usage of "stack" objects.
Thanks Mark.
I won't be using operator new at all, though, or virtual functions
even. Just allocatoring RAM and then redefining it to the correct class. co**********@ho tmail.com wrote:
Hi people,
I am writing a library that will implement a binary file format.
Well... I'm thinking, as a design, it would be nice to have my library
able to promise to not use more than a certain amount of RAM. I could
have a "SetCacheSize(l ong NewSize);" function. So as part of that
promise, when required memory exceeds that limit, the extra data is
saved to disk and flushed from RAM.
[allocator stuff snipped]
Any ideas anyone? Like I said, I don't know anything about allocators.
IMO this seems overengineered in the extreme. Your code can use a fixed
buffer allocated according to the NewSize parameter via "buffer = new
unsigned char[NewSize]" or whatever and just use that, why do you want
to do anything more complicated than that ?
Like other posters have intimated, allocators are fiddly beasts that
have their uses, but are you sure you're not someone who's just found
out about these neat things called "hammers" and now thinks that your
library looks like a nail ?
Any ideas anyone? Like I said, I don't know anything about allocators.
IMO this seems overengineered in the extreme. Your code can use a fixed
buffer allocated according to the NewSize parameter via "buffer = new
unsigned char[NewSize]" or whatever and just use that, why do you want
to do anything more complicated than that ?
I guess because allocating out of my own buffer, means I must now do my
own memory management. So, let's say that my structs take up 48 bytes.
So now I must allocate 48 bytes
out of that big block. Then I must manage the free blocks. I'm now
doing my own memory management.
I've tried this sort of thing before it's a total pain.
Although I was doing variable size allocation. Perhaps the trauma of
trying to get variable size allocation before (writing my own malloc
almost) put me off fixed size allocators? Are you saying that making
your own fixed size code to allocate is easy?
Like other posters have intimated, allocators are fiddly beasts that
have their uses, but are you sure you're not someone who's just found
out about these neat things called "hammers" and now thinks that your
library looks like a nail ?
I'm still trying to understand the hammer! This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics | |
by: Fraser Ross |
last post by:
If anyone has the book Effective STL do you agree that the word value should
be type on P51, 2nd paragraph from the bottom, in the sentence starting 'In
both cases'. allocator's size_type is required to be size_t so why say it
is typically a typedef for size_t?
Fraser.
|
by: Orjan Westin |
last post by:
Hi,
I have an interesting (read frustrating) problem. I'm writing a
generic container class, which holds data as well as links to other
instances of itself, like this:
template<class T>
class node
{
public:
|
by: John Harrison |
last post by:
If you specify an allocator in an STL container is it a requirement that
the allocator allocates object of the right type, or can you assume that
the container will rebind the allocator to the correct type?
For example I tried the following code which uses a 'wrong' allocator and
was slightly surrised to find it compiles on the three compilers I tried
it on
#include <vector>
#include <memory>
|
by: Mark P |
last post by:
Hi,
I'm looking for some info on the default STL allocator, std::alloc. In
particular, I'm wondering if it is optimized to handle many allocations
of small objects. I'm thinking along the lines of a pool-based
approach, for example.
Any references would also be appreciated.
Thanks,
|
by: Joshua Kolden |
last post by:
STL allocators are templates so that when you write one you are obliged
to make it work with any type. However, the Intel IPP library that we
use has memory aligned allocators for each of 15 different types. For
example an 8 bit unsigned array is allocated with ippsMalloc_8u(size).
So I want to create memory aligned allocators for use with the STL (in
particular the vector container) that is fast (due to alignment), and
pointer...
| | |
by: **Developer** |
last post by:
If I need to allocate memory, maybe because I'm going to:
Marshal.StructureToPtr
Does it matter which of the following I use?
Marshal.AllocCoTaskMem
Marshal.AllocHGlobal
I know the arguments are different which would make one more convient in a
given situation, but other that that does it make any signifficant
difference.
|
by: Protoman |
last post by:
Why won't the compiler let me write this:
template <class T>
class Allocator
{
public:
Allocator(){}
T* allocate(){static long long i=0; T* j=new(pool) T; i++; return
&*j;}
private:
|
by: ddh |
last post by:
Hi,
I have a map type such as std::map<int, int>, which I used to manage
many values in a fixed order. But I have thousands of values to be
inserted into the map, and in some case, the action of inserting and
re-building is very frequent, that say, many times per second.
I've made some tests to insert 1500 values into a map, it cost about
30ms in my athlon XP1800+. I think it shouldn't be so expensive. When I
do an inserting action, I...
|
by: xushiwei |
last post by:
Most of the C++ programmers do not benefit from "Garbage Collection"
technique (GC). They are sick of deleting objects but have to do this.
There are some C/C++ memory GC implementations, but they are complex
and are not widely used.
I am going to introduce a new memory management technique named "GC
Allocator". "GC Allocator" isn't an implementation, but a concept.
Now, we have two "GC Allocator" implementations, named "AutoFreeAlloc"...
|
by: * Tong * |
last post by:
Hi,
I'm following the example in "C++ Templates: The Complete Guide",
section 5.4 Template Template Parameters.
It's basics/stack8.hpp example has a statement of "#include <allocator>",
but I got an error for it.
$ cat -n allocator.cc
1 #include <allocator>
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
| | |
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
| | |
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
| | |
