My pet project is a command-line utility (preprocessor), so it runs and
terminates. It uses lots of memory allocations; most of them are quite
small. What to do with the allocated objects when they are no longer
used? I consider the following "pure" strategies:
- Meticulously free anything ever malloc'ed. This involves
inefficiencies of calling free, but worse yet, in certain cases object
duplication appears necessary to ensure uniqueness of custodial pointers.
- Forget about no-longer-used objects and let them rot in the heap. This
is the fastest and most compact but may lead to out-of-memory failures
on a really complex processing job where method 1 would succeed. It
would sound like negligence then, even though I've never observed this
happen in practice.
- Use a different allocator endowed with garbage collector. I am not
concerned about non-deterministic performance since for my utility what
counts is the total execution time. However, I am a little scared by
disclaimers that a GC may be tricked into reclaiming memory which is
still in use. [Does anyone know of a bullet-proof GC?]
I'd like to ask for advice on which strategy (or a mix of them) is
considered the best for this sort of tasks?
--
Thank you,
Ark
6  1673 
Ark Khasin wrote:
My pet project is a command-line utility (preprocessor), so it runs and
terminates. It uses lots of memory allocations; most of them are quite
small. What to do with the allocated objects when they are no longer
used? I consider the following "pure" strategies:
- Meticulously free anything ever malloc'ed. This involves
inefficiencies of calling free, but worse yet, in certain cases object
duplication appears necessary to ensure uniqueness of custodial pointers.
Well, I would rather expect malloc() to be the heavy-weight call here,
not free(). This method, is the traditional one, which works in all cases.
- Forget about no-longer-used objects and let them rot in the heap. This
is the fastest and most compact but may lead to out-of-memory failures
on a really complex processing job where method 1 would succeed. It
Right, if being lazy, bad things can happen. If doing a lot of malloc's,
I wouldn't recommend being lazy. :)
would sound like negligence then, even though I've never observed this
happen in practice.
- Use a different allocator endowed with garbage collector. I am not
concerned about non-deterministic performance since for my utility what
counts is the total execution time. However, I am a little scared by
disclaimers that a GC may be tricked into reclaiming memory which is
still in use. [Does anyone know of a bullet-proof GC?]
There are issues with e.g. Boehm GC, but such cases appears to be rather
constructed. For a non-safety related command line utility, I don't see
the problem with using Boehm GC.
--
Tor <bw****@wvtqvm. vw | tr i-za-h a-z>
Tor Rustad wrote:
Ark Khasin wrote:
>- Use a different allocator endowed with garbage collector. I am not
concerned about non-deterministic performance since for my utility
what counts is the total execution time. However, I am a little scared
by disclaimers that a GC may be tricked into reclaiming memory which
is still in use. [Does anyone know of a bullet-proof GC?]
There are issues with e.g. Boehm GC, but such cases appears to be rather
constructed. For a non-safety related command line utility, I don't see
the problem with using Boehm GC.
The trouble is, the current version of the utility (Unimal) is used in
the build process of a safety-critical product, so IMO it would not be
wise to take chances. Currently, it uses a mix of thoughtful cleanup and
deliberate leaks but I just thought there is a better way...
--
Ark
Ark Khasin wrote:
Tor Rustad wrote:
>Ark Khasin wrote:
>>- Use a different allocator endowed with garbage collector. I am not
concerned about non-deterministic performance since for my utility
what counts is the total execution time. However, I am a little
scared by disclaimers that a GC may be tricked into reclaiming memory
which is still in use. [Does anyone know of a bullet-proof GC?]
There are issues with e.g. Boehm GC, but such cases appears to be
rather constructed. For a non-safety related command line utility, I
don't see the problem with using Boehm GC.
The trouble is, the current version of the utility (Unimal) is used in
the build process of a safety-critical product, so IMO it would not be
wise to take chances. Currently, it uses a mix of thoughtful cleanup and
deliberate leaks but I just thought there is a better way...
If building safety-critical program, I want tools which do their job
well. During the build process, correctness of the output is critical,
not really avoiding a failure to build. Extra memory can be added, so
can swap space.
The Boehm GC does indeed do some very low-level stuff, and using it will
increase the intrinsic complexity of your tool, which may introduce some
nasty bugs. An alternative, is to first use the GC as a *leak detector*,
and also pull in a test-case where you compare the output, when GC is
used and when it isn't. Those outputs should always be identical.
If being in your shoes, I guess my choice would be going for explicit
free() calls, but you are in a better position to judge this. Using GC
as a leak-detector in debug builds for a while, will give you even
better know how.
--
Tor <bw****@wvtqvm. vw | tr i-za-h a-z>
Tor Rustad wrote:
Ark Khasin wrote:
>Tor Rustad wrote:
>>Ark Khasin wrote:
- Use a different allocator endowed with garbage collector. I am not
concerned about non-deterministic performance since for my utility
what counts is the total execution time. However, I am a little
scared by disclaimers that a GC may be tricked into reclaiming
memory which is still in use. [Does anyone know of a bullet-proof GC?]
There are issues with e.g. Boehm GC, but such cases appears to be
rather constructed. For a non-safety related command line utility, I
don't see the problem with using Boehm GC.
The trouble is, the current version of the utility (Unimal) is used in
the build process of a safety-critical product, so IMO it would not be
wise to take chances. Currently, it uses a mix of thoughtful cleanup
and deliberate leaks but I just thought there is a better way...
If building safety-critical program, I want tools which do their job
well. During the build process, correctness of the output is critical,
not really avoiding a failure to build.
Thank you indeed for this observation. It put me in the justifiably
conservative mindset.
>
The Boehm GC does indeed do some very low-level stuff, and using it will
increase the intrinsic complexity of your tool, which may introduce some
nasty bugs. An alternative, is to first use the GC as a *leak detector*,
and also pull in a test-case where you compare the output, when GC is
used and when it isn't. Those outputs should always be identical.
This of course can uncover an error but cannot prove its absence. As a
leak detector... I'll have to study this further because e.g. some leaks
may remain intentional.
>
If being in your shoes, I guess my choice would be going for explicit
free() calls, but you are in a better position to judge this. Using GC
as a leak-detector in debug builds for a while, will give you even
better know how.
Thank you
--
Ark
Ark Khasin wrote:
>
My pet project is a command-line utility (preprocessor), so it
runs and terminates. It uses lots of memory allocations; most of
them are quite small. What to do with the allocated objects when
they are no longer used? I consider the following "pure"
strategies:
- Meticulously free anything ever malloc'ed. This involves
inefficiencies of calling free, but worse yet, in certain cases
object duplication appears necessary to ensure uniqueness of
custodial pointers.
- Forget about no-longer-used objects and let them rot in the heap.
This is the fastest and most compact but may lead to out-of-
memory failures on a really complex processing job where method
1 would succeed. It would sound like negligence then, even
though I've never observed this happen in practice.
- Use a different allocator endowed with garbage collector. I am
not concerned about non-deterministic performance since for my
utility what counts is the total execution time. However, I am
a little scared by disclaimers that a GC may be tricked into
reclaiming memory which is still in use. [Does anyone know of
a bullet-proof GC?]
I'd like to ask for advice on which strategy (or a mix of them)
is considered the best for this sort of tasks?
You omitted 'Use a malloc package that avoid the long sequences on
free". I developed this because of the problem, and all operations
are O(1). It is semi-portable, requiring being able to treat the
memory as a big char array (or arrays), and needs gcc, because of
gcc style variadic macros. It also depends on the presence of the
sbrk() system call. It is available at:
<http://cbfalconer.home .att.net/download/nmalloc.zip>
--
Chuck F (cbfalconer at maineline dot net)
<http://cbfalconer.home .att.net>
Try the download section.
--
Posted via a free Usenet account from http://www.teranews.com dj******@csclub .uwaterloo.ca.i nvalid wrote:
Ark Khasin <ak*****@macroe xpressions.comw rote:
>My pet project is a command-line utility (preprocessor), so it
runs and terminates. It uses lots of memory allocations; most of
them are quite small. What to do with the allocated objects when
they are no longer used? I consider the following "pure"
strategies:
- Meticulously free anything ever malloc'ed. This involves
inefficiencies of calling free, but worse yet, in certain
cases object duplication appears necessary to ensure
uniqueness of custodial pointers.
Don't worry about the inefficiencies of calling free; if they're
noticeable at all, the inefficiencies of leaking a bunch of memory
will be even worse.
To the contrary, the usual malloc package has O(n) performance for
free, where n is the number of allocated blocks. The makes the
operation of freeing a large segment of those blocks O(n * n).
Elsethread I have given references to my malloc package, which is
O(1) and makes the large segment of blocks free operation O(n).
When you have n in the 10,000 to 20,000 range you can easily see
the difference between O(n) and O(n * n) free() operation. My
hashlib testing package has special provisions for this.
--
Chuck F (cbfalconer at maineline dot net)
<http://cbfalconer.home .att.net>
Try the download section.
--
Posted via a free Usenet account from http://www.teranews.com This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics | |
by: Scott Abel |
last post by:
For immediate release:
The Rockley Group Content Management Workshop Series
Coming to Atlanta, Seattle, Vancouver, Chicago, Washington, DC,
Toronto, and Research Triangle Park
Learn more: http://www.rockley.com/workshops.htm
The Rockley Group Content Management Workshop Series is designed to
|
by: Tron Thomas |
last post by:
Given the following information about memory management in C++:
-----
The c-runtime dynamic memory manager (and most other commercial memory
managers) has issues with fragmentation similar to a hard drive file
system.
Over time, the more often use call new/delete or alloc/free, there
will be
gaps and fragments in the heap. This can lead to inefficient use of
available memory, as well as cache-hit inefficiencies.
|
by: Ben Jeurissen |
last post by:
Hello,
I have to deal with the following issue in C++:
Two threads are started from the main thread, both capturing images from
a different firewire camera. Both threads take shots of 460800 bytes at
a rate of 30 frames per second. This works fine, without frame loss, and
I can display the two framestreams on screen.
Now I would like to write these frames to my disk too (2 10000rpm in
|
by: ~Gee |
last post by:
Hi Folks!
Please see the program below:
1 #include<iostream>
2 #include<list>
3 #include <unistd.h>
4 using namespace std;
5 int main()
6 {
7 {
|
by: Jeff Greenberg |
last post by:
Not an experienced c++ programmer here and I've gotten myself a bit
stuck. I'm trying to implement a class lib and I've run into a sticky
problem that I can't solve. I'd appreciate any help that I can get!
Consider 3 classes in the following heirarchy:
base
/ \
deriv1 deriv2
\
| | |
by: Antar |
last post by:
Hi,
I'm kind of a newbie on DB management but I have to deal with a huge DB
used for real time operations. I got a temporal table where current
data is stored to work with frecuently, and then a table for each past
month (historic tables, that is). The issue is that each of these
tables are 1 GB. When the user wants to display data for a past month,
the SQL Server process inmediatly jumps from 50 MB to > 1 GB on memory
(I guess it loads...
|
by: ira2402 |
last post by:
Hi All,
We are developing sw for a small embedded OS and we have limited
memory.
We are looking for algorithms, links, and articles about this.
The goal is efficient utilization of small amount of memory - means -
allocation for fixed length blocks / variable length blocks.
Thanks.
|
by: weidongtom |
last post by:
Hi,
I've written the code that follows, and I use the function add_word(),
it seems to work fine
*before* increase_arrays() is called that uses realloc() to allocate
more memory to words. But *after* calling increase_arrays(), I
received segmentation fault. I tried to step it through gdb, and I
found out that after calling increase_arrays(), words's original
value is modified, and if I tried to access it, I get <address 0x11
out of...
|
by: jacob navia |
last post by:
Suppose that you have a module that always allocates
memory without ever releasing it because the guy that
wrote it was lazy, as lazy as me.
Now, you want to reuse it in a loop. What do you do?
Contrary to some people that will start crying to that
&@@""#~ programmer that wrote this sh!!!! you keep
your cool and you do the following:
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
| | |
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
| | |
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
| |
