473,586 Members | 2,702 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

A solution for the allocation failures problem

1:
It is not possible to check EVERY malloc result within complex software.

2:
The reasonable solution (use a garbage collector) is not possible for
whatever reasons.

3:
A solution like the one proposed by Mr McLean (aborting) is not
possible for software quality reasons. The program must decide
if it is possible to just abort() or not.

Solution:

1) At program start, allocate a big buffer that is not used
elsewhere in the program. This big buffer will be freed when
a memory exhaustion situation arises, to give enough memory
to the error reporting routines to close files, or otherwise
do housekeeping chores.

2) xmalloc()

static int (*mallocfailedH andler)(int);
void *xmalloc(size_t nbytes)
{
restart:
void *r = malloc(nbytes);
if (r)
return r;
// Memory exhaustion situation.
// Release some memory to the malloc/free system.
if (BigUnusedBuffe r)
free(BigUnusedB uffer);
BigUnusedBuffer = NULL;
if (mallocfailedHa ndler == NULL) {
// The handler has not been set. This means
// this application does not care about this
// situation. We exit.
fprintf(stderr,
"Allocation failure of %u bytes\n",
nbytes);
fprintf(stderr, "Program exit\n");
exit(EXIT_FAILU RE);
}
// The malloc handler has been set. Call it.
if (mallocfailedHa ndler(nbytes)) {
goto restart;
}
// The handler failed to solve the problem.
// Exit without any messages.
exit(EXIT_FAILU RE);
}

4:
Using the above solution the application can abort if needed, or
make a long jump to a recovery point, where the program can continue.

The recovery handler is supposed to free memory, and reallocate the
BigUnusedBuffer , that has been set to NULL;
--
jacob navia
jacob at jacob point remcomp point fr
logiciels/informatique
http://www.cs.virginia.edu/~lcc-win32
Jan 29 '08
158 6007
Richard Bos wrote:
Functions
which succeed should not (IIRC are not allowed to) set errno.
Generally, any function may set errno to a nonzero value, even
if it succeeded.

However, no C library function may set errno to zero.

-- Ralf
Jan 31 '08 #31
On Tue, 29 Jan 2008 13:52:21 -0800, ediebur wrote:
jacob navia wrote:
>1:
It is not possible to check EVERY malloc result within complex
software.

Pardon me.I haven't programmed for a few years but I was involved in
some fairly complex software development. Why is it not possible to
check every malloc result?
Dunno; I've been cranking code for nigh on 30 years and I don't
understand why. Others - CBF, for example - appear to be equally baffled.

Jan 31 '08 #32
"Bartc" <bc@freeuk.comw rites:
"CBFalconer " <cb********@yah oo.comwrote in message
news:47******** *******@yahoo.c om...
>Stan Milam wrote:
>>jacob navia wrote:

1: It is not possible to check EVERY malloc result within
complex software.

I disagree. It is possible. You just have to decide you want to
sweat the details bad enough and do the hard work.

What hard work? For example:

if (!(p = malloc(N * sizeof *p))) fixitup(&p, N * sizeof *p);
if (!p) exit(EXIT_FAILU RE);

Hmm, perhaps not quite as eloquent as:

p=heap(N);

Then you can get on with the next thing. Of course this is a made-up
language, but this form clearly has some attraction and I think trying to
achieve something similar in C was the aim of this and the xmalloc thread.
Changing the syntax for memory allocation doesn't magically solve
anything.
(I keep seeing this form, usually as 'new', in other languages; what do
/they/ do when out-of-memory?)
It varies from language to language. Typically either the allocator
returns a null pointer on failure (exactly what C's malloc() does), or
it throws/raises an exception (if the language supports exceptions).

It's *always* possible for an attempted memory allocation to fail.
The possible ways to handle this possibility are:

1. Ignore it and keep going (dangerous).
2. Use an allocator that immediately aborts the program on failure.
3. Check for failure on each allocation. On failure:
3a. Immediately abort the program (equivalent to 2).
3b. Clean up and abort the program.
3c. Clean up and continue processing (this can be difficult, but
it's important if you want the program to be robust).
4. If the language supports exceptions (C doesn't), catch the error at
whatever level is appropriate, not necessarily immediately after
the attempted allocation. Perform appropriate cleanup and recovery
in the exception handler.

--
Keith Thompson (The_Other_Keit h) <ks***@mib.or g>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
Jan 31 '08 #33

"Keith Thompson" <ks***@mib.orgw rote in message
1. Ignore it and keep going (dangerous).
2. Use an allocator that immediately aborts the program on failure.
3. Check for failure on each allocation. On failure:
3a. Immediately abort the program (equivalent to 2).
3b. Clean up and abort the program.
3c. Clean up and continue processing (this can be difficult, but
it's important if you want the program to be robust).
4. If the language supports exceptions (C doesn't), catch the error at
whatever level is appropriate, not necessarily immediately after
the attempted allocation. Perform appropriate cleanup and recovery
in the exception handler.
5. Demand user intervention to supply the required memory.

--
Free games and programming goodies.
http://www.personal.leeds.ac.uk/~bgy1mm

Jan 31 '08 #34
jacob navia wrote:
[...]
Mr Bjarnason is found of saying how clever he is. This is of course
not verifiable here. He can claim he never makes allocation mistakes
but this sounds very hollow to me. Like other claims done by the
"regulars" here, they sound pathetic, like the one of the famous
Dan Pop that claimed that he never had a crash of a program written by
him.

Boasting here is very easy.
This, from the person who launched the thread with an
admission that his own programming skills were such that
he found it "impossible " to check for malloc() failure ...

--
Er*********@sun .com
Jan 31 '08 #35

"Eric Sosman" <Er*********@su n.comwrote in message
jacob navia wrote:
>>
Boasting here is very easy.

This, from the person who launched the thread with an
admission that his own programming skills were such that
he found it "impossible " to check for malloc() failure ...
He means impossible to write and test custom allocation failure hnadling
code for each call.
Any idiot can put malloc() in a wrapper or inside an if statement. It's what
does in the body of that if statement that can a cause problems. Not for
every large program, in my opinion, but for some programs.

--
Free games and programming goodies.
http://www.personal.leeds.ac.uk/~bgy1mm

Jan 31 '08 #36
Malcolm McLean wrote:
>
"Eric Sosman" <Er*********@su n.comwrote in message
>jacob navia wrote:
>>>
Boasting here is very easy.

This, from the person who launched the thread with an
admission that his own programming skills were such that
he found it "impossible " to check for malloc() failure ...
He means impossible to write and test custom allocation failure hnadling
code for each call.
Any idiot can put malloc() in a wrapper or inside an if statement. It's
what does in the body of that if statement that can a cause problems.
Not for every large program, in my opinion, but for some programs.
"There is no man blinder as the one that doesn't want to see"
--
jacob navia
jacob at jacob point remcomp point fr
logiciels/informatique
http://www.cs.virginia.edu/~lcc-win32
Jan 31 '08 #37
In article <87************ @kvetch.smov.or g>,
Keith Thompson <ks***@mib.orgw rote:
>It's *always* possible for an attempted memory allocation to fail.
(Or any other resource allocation)
>The possible ways to handle this possibility are:

1. Ignore it and keep going (dangerous).
2. Use an allocator that immediately aborts the program on failure.
3. Check for failure on each allocation. On failure:
3a. Immediately abort the program (equivalent to 2).
3b. Clean up and abort the program.
3c. Clean up and continue processing (this can be difficult, but
it's important if you want the program to be robust).
3d. Clean up and abort the current operation, and report failure to
the caller.
Let the caller choose how to handle it, which might be any of
the ways listed here. (Note that 1 may be less dangerous at
a higher level, depending on the operation that failed.)
>4. If the language supports exceptions (C doesn't), catch the error at
whatever level is appropriate, not necessarily immediately after
the attempted allocation. Perform appropriate cleanup and recovery
in the exception handler.
(Approximately equivalent to 3d.)
dave

--
Dave Vandervies dj3vande at eskimo dot com
Yeah. They're exactly the same product, just a different colour of plastic
in the case. In this case, the Microtech was much better, because it had a
lower $/suck ratio. --Graham Reed in the scary devil monastery
Jan 31 '08 #38
On Jan 31, 9:24 am, Keith Thompson <ks...@mib.orgw rote:
"Bartc" <b...@freeuk.co mwrites:
"CBFalconer " <cbfalco...@yah oo.comwrote in message
news:47******** *******@yahoo.c om...
Stan Milam wrote:
jacob navia wrote:
>>1: It is not possible to check EVERY malloc result within
complex software.
>I disagree. It is possible. You just have to decide you want to
sweat the details bad enough and do the hard work.
What hard work? For example:
if (!(p = malloc(N * sizeof *p))) fixitup(&p, N * sizeof *p);
if (!p) exit(EXIT_FAILU RE);
Hmm, perhaps not quite as eloquent as:
p=heap(N);
Then you can get on with the next thing. Of course this is a made-up
language, but this form clearly has some attraction and I think trying to
achieve something similar in C was the aim of this and the xmalloc thread.

Changing the syntax for memory allocation doesn't magically solve
anything.
(I keep seeing this form, usually as 'new', in other languages; what do
/they/ do when out-of-memory?)

It varies from language to language. Typically either the allocator
returns a null pointer on failure (exactly what C's malloc() does), or
it throws/raises an exception (if the language supports exceptions).

It's *always* possible for an attempted memory allocation to fail.
The possible ways to handle this possibility are:

1. Ignore it and keep going (dangerous).
This is not a solution.
2. Use an allocator that immediately aborts the program on failure.
You only do this because you don't know how to solve the problem
without all that memory, or else, you *can't* solve the problem
without all that memory (rare.)
3. Check for failure on each allocation. On failure:
3a. Immediately abort the program (equivalent to 2).
3b. Clean up and abort the program.
3c. Clean up and continue processing (this can be difficult, but
it's important if you want the program to be robust).
3a = 3b = 2. Aborting doesn't require that you clean up first -- any
real OS(TM) will clean you up upon aborting anyways.

Your 3c only makes sense if you are talking about your whole program,
not just the code fragments where the error occurs (because that
typically will make no sense.) But you are inherently omitting how
you *achieve* it. In straight C you do it by observing the following
in your primitive calls:

3c1) Undo all operations in your scope and return from the current
function with an error code (my typical approach is NULL for pointers
and negative numbers for status codes).

which is applied recursively throughout your "primitives " call chain.
And in your policy/management calls:

3c2) If a strategy fails, go to the next strategy until all are
exhausted, then if everything fails log or return an error code that
amounts to "INCOMPLETE DUE TO LACK OR RESOURCES".

Of course this means you have to understand the distinction between
primitive calls and policy/management calls in your program.
Primitives make and handle data structures, whereas policy performs
actions on these data structures.
4. If the language supports exceptions (C doesn't), catch the error at
whatever level is appropriate, not necessarily immediately after
the attempted allocation. Perform appropriate cleanup and recovery
in the exception handler.
This is just a fancy way of implementing 3c with special language
features.

Going back to "at those lines of code" programming solutions I would
add:

5. Defer the error (but retain this information) until it can be
handled in an aggregated way.

6. Provide a stand-in for typical functionality. (I.e., return some
static memory declaration.) This can only be done in rare instances
where contention is a less serious issue than running out of memory,
or if you can guard the memory location with a semaphore, for example.

--
Paul Hsieh
http://www.pobox.com/~qed/
http://bstring.sf.net/
Jan 31 '08 #39
Paul Hsieh wrote, On 31/01/08 19:40:
On Jan 31, 9:24 am, Keith Thompson <ks...@mib.orgw rote:
>"Bartc" <b...@freeuk.co mwrites:
>>"CBFalconer " <cbfalco...@yah oo.comwrote in message
news:47****** *********@yahoo .com...
Stan Milam wrote:
jacob navia wrote:
>1: It is not possible to check EVERY malloc result within
> complex software.
I disagree. It is possible. You just have to decide you want to
sweat the details bad enough and do the hard work.
What hard work? For example:
if (!(p = malloc(N * sizeof *p))) fixitup(&p, N * sizeof *p);
if (!p) exit(EXIT_FAILU RE);
Hmm, perhaps not quite as eloquent as:
p=heap(N);
Then you can get on with the next thing. Of course this is a made-up
language, but this form clearly has some attraction and I think trying to
achieve something similar in C was the aim of this and the xmalloc thread.
Changing the syntax for memory allocation doesn't magically solve
anything.
>>(I keep seeing this form, usually as 'new', in other languages; what do
/they/ do when out-of-memory?)
It varies from language to language. Typically either the allocator
returns a null pointer on failure (exactly what C's malloc() does), or
it throws/raises an exception (if the language supports exceptions).

It's *always* possible for an attempted memory allocation to fail.
The possible ways to handle this possibility are:

1. Ignore it and keep going (dangerous).

This is not a solution.
Agreed. However, it is what some people seem to do.
>2. Use an allocator that immediately aborts the program on failure.

You only do this because you don't know how to solve the problem
without all that memory, or else, you *can't* solve the problem
without all that memory (rare.)
Agreed.
>3. Check for failure on each allocation. On failure:
3a. Immediately abort the program (equivalent to 2).
3b. Clean up and abort the program.
3c. Clean up and continue processing (this can be difficult, but
it's important if you want the program to be robust).

3a = 3b = 2. Aborting doesn't require that you clean up first -- any
real OS(TM) will clean you up upon aborting anyways.
<snip>

Cleaning up is not always simply a matter of freeing memory and closing
files. In the clean-up code we have for one application I work on it
also involves...
Sending a message to the client to say it is aborting
Deleting temporary files that where deliberately *not* opened using
tmpfile (there is reason for this)
Logging that it is crashing
Sending an email saying that it is crashing I've tested this and it
can manage it some of the time
--
Flash Gordon
Jan 31 '08 #40

This thread has been closed and replies have been disabled. Please start a new discussion.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.