Hello,
I've read in the FAQ that modern versions of malloc/free can cache
memory, i.e. free does not return memory to the OS so that it may be
re-used by the next malloc call. I was thinking about writing some kind
of high level memory manger for a scientific project, but then I may
just be better off with malloc/free ... .
Here's the background: I need to allocate lots of vectros, matrices and
structures iteratively, specificly - I loop over a list of many data
sets and compare each of the sets with a list of other dataset. The
dataset are spectra with 2 double vectors of datapoints that were
recorded from a mass spectrometer, the spectra are of different size
(30 to 7000 data points in each vector).
For each comparsion I need to allocate a matrix, some vectors and
structures. So I was thinking to implement some kind of thread safe
(I'm planing to run the comparisons in parallel) memory manager that
will keep the matrices, vectors and structures so that they can be
re-used for the next round of comparsion (the vectors and matrices may
need to be re-sized, or I just increase but never decrese them). The
content of the memory can get lost after each round, it's just
important that memory allocation is fast (I've several mio.
comparsions).
Anyway, I'm not sure whether such a memory manager makes much sense if
malloc/free do caching, or maybe there are already efficient caching
systems available. What's your recommendation or expereince with large
numbers of malloc/free calls?
I don't have much experience in C, so even trivial answers may help ;-)
..
(I'm using gcc 2.96 under linux and the standard c-compiler on sun.9
thanks a lot for your help
+kind regards,
Arne
3  1548  ar*********@gma il.com wrote:
Hello,
I've read in the FAQ that modern versions of malloc/free can cache
memory, i.e. free does not return memory to the OS so that it may be
re-used by the next malloc call. I was thinking about writing some kind
of high level memory manger for a scientific project, but then I may
just be better off with malloc/free ... .
[...]
I don't have much experience in C, so even trivial answers may help ;-)
"You'd just be better off with malloc/free."
Don't optimize prematurely: you are likely to wind up chasing
imaginary problems and ignoring the real ones. This isn't a slam
at your inexperience; even expert programmers have been found to
be startlingly unable to predict which parts of their programs
will be the performance bottlenecks.
Begin by writing a straightforward program using straightforward
techniques. Work with the program until you have confidence that it
operates correctly. Then *if* the performance is inadequate and *if*
profiling and other measurement techniques show that a lot of time is
wasted in malloc/free -- then and only then should you start thinking
about customized wrappers and replacements.
(I'm using gcc 2.96 under linux and the standard c-compiler on sun.9
Aside: If performance is important to you, why are you using such
antique compilers? gcc is up to 4-dot-something, and the Sun Studio
compiler and tool set is free for the downloading. Compiler version
number does not always correlate positively with program speed, but
it's eccentric to insist on versions that were already old when
Clinton was elected.
--
Eric Sosman es*****@acm-dot-org.invalid
On 2006-08-12 14:36:10 -0400, ar*********@gma il.com said:
Hello,
I've read in the FAQ that modern versions of malloc/free can cache
memory, i.e. free does not return memory to the OS so that it may be
re-used by the next malloc call. I was thinking about writing some kind
of high level memory manger for a scientific project, but then I may
just be better off with malloc/free ... .
Odds are malloc/free will do just fine for you; it is almost never a
good idea to second-guess your compiler/OS/libraries unless you
actually observe something amiss.
--
Clark S. Cox, III cl*******@gmail .com
Eric Sosman wrote:
ar*********@gma il.com wrote:
>Hello,
I've read in the FAQ that modern versions of malloc/free can cache
memory, i.e. free does not return memory to the OS so that it may be
re-used by the next malloc call. I was thinking about writing some kind
of high level memory manger for a scientific project, but then I may
just be better off with malloc/free ... .
[...]
I don't have much experience in C, so even trivial answers may help ;-)
"You'd just be better off with malloc/free."
Don't optimize prematurely: you are likely to wind up chasing
imaginary problems and ignoring the real ones. This isn't a slam
at your inexperience; even expert programmers have been found to
be startlingly unable to predict which parts of their programs
will be the performance bottlenecks.
<snip>
Even if you find a necessity of custom memory manager after heeding to
Eric's very sound advice, do not rush to writing your own, unless you
KNOW your memory usage patterns that only you can take advantage of.
There are a few known approaches worth googling; customalloc is one of them.
- Ark This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics | |
by: benevilent |
last post by:
Hey,
I'm trying to debug the memory allocation in an embedded use of the
Python interpreter.
The longer I leave my program running the more memory it consumes. The
total number of objects in the system is not increasing (many are being
allocated and deallocated).
Using mtrace I have established that the only memory which is not being
|
by: sbayeta |
last post by:
Hi,
I'd like to know who is responsible of memory recycling and
defragmentation in a C/C++ program, assuming all the memory
allocation/deallocation is done using malloc/free or new/delete.
Thanks
|
by: jimjim |
last post by:
Hello,
This is a simple question for you all, I guess .
int main(){
double *g= new double;
*g = 9;
delete g;
cout<< sizeof(g)<<" "<<sizeof(double)<<" "<<sizeof(*g)<<" "<<*g<<" "<<endl;
*g = 111;
cout<< sizeof(g)<<" "<<sizeof(double)<<" "<<sizeof(*g)<<" "<<*g<<" "<<endl;
|
by: RoSsIaCrIiLoIA |
last post by:
why not to build
a malloc_m() and a free_m() that *check* (if memory_debug=1) if
1) there are some errors in bounds of *all* allocated arrays
from them (and trace-print the path of code that make
the error and exit) just when malloc_m and free_m start.
(I use a 1100 static array for write the path (like a queue)
of called function, operator, etc and write using '+' where is
find the memory error)
2) if the pointer to free is alredy...
|
by: uday.das |
last post by:
hi ,
I am not sure but I think it might be to avoid heap fragmentation due
to several malloc calls. Are there any strong reason ? What are the
other things that should take care when we implement a memory manager ,
like i) garabage collector kind of thing , ii) Safe Free of pointer in
wrapper of Free functions etc .
Thanks
Uday
| | |
by: smnoff |
last post by:
I have searched the internet for malloc and dynamic malloc; however, I still
don't know or readily see what is general way to allocate memory to char *
variable that I want to assign the substring that I found inside of a
string.
Any ideas?
|
by: Dave Rahardja |
last post by:
I need to design a container that must hold a set of references to a variety
of object types, managed by different memory managers. At some time, the
container must destroy each object. Currently I have the following design in
mind:
class Destructible // interface
{
public:
virtual void destroy() = 0;
|
by: jeungster |
last post by:
Hello,
I'm trying to track down a memory issue with a C++ application that
I'm working on:
In a nutshell, the resident memory usage of my program continues to
grow as the program runs. It starts off at a nice 4% of memory, then
slowly grows up to 50% and beyond. This translates to around 2 gigs
of physical memory, and that's really way more memory than this
program should be taking up.
|
by: jacob navia |
last post by:
Suppose that you have a module that always allocates
memory without ever releasing it because the guy that
wrote it was lazy, as lazy as me.
Now, you want to reuse it in a loop. What do you do?
Contrary to some people that will start crying to that
&@@""#~ programmer that wrote this sh!!!! you keep
your cool and you do the following:
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
| | |
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
| | |
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
| |
