By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,702 Members | 1,101 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,702 IT Pros & Developers. It's quick & easy.

A solution for the allocation failures problem

P: n/a
1:
It is not possible to check EVERY malloc result within complex software.

2:
The reasonable solution (use a garbage collector) is not possible for
whatever reasons.

3:
A solution like the one proposed by Mr McLean (aborting) is not
possible for software quality reasons. The program must decide
if it is possible to just abort() or not.

Solution:

1) At program start, allocate a big buffer that is not used
elsewhere in the program. This big buffer will be freed when
a memory exhaustion situation arises, to give enough memory
to the error reporting routines to close files, or otherwise
do housekeeping chores.

2) xmalloc()

static int (*mallocfailedHandler)(int);
void *xmalloc(size_t nbytes)
{
restart:
void *r = malloc(nbytes);
if (r)
return r;
// Memory exhaustion situation.
// Release some memory to the malloc/free system.
if (BigUnusedBuffer)
free(BigUnusedBuffer);
BigUnusedBuffer = NULL;
if (mallocfailedHandler == NULL) {
// The handler has not been set. This means
// this application does not care about this
// situation. We exit.
fprintf(stderr,
"Allocation failure of %u bytes\n",
nbytes);
fprintf(stderr,"Program exit\n");
exit(EXIT_FAILURE);
}
// The malloc handler has been set. Call it.
if (mallocfailedHandler(nbytes)) {
goto restart;
}
// The handler failed to solve the problem.
// Exit without any messages.
exit(EXIT_FAILURE);
}

4:
Using the above solution the application can abort if needed, or
make a long jump to a recovery point, where the program can continue.

The recovery handler is supposed to free memory, and reallocate the
BigUnusedBuffer, that has been set to NULL;
--
jacob navia
jacob at jacob point remcomp point fr
logiciels/informatique
http://www.cs.virginia.edu/~lcc-win32
Jan 29 '08 #1
Share this Question
Share on Google+
158 Replies


P: n/a
CJ
On 29 Jan 2008 at 11:47, jacob navia wrote:
1:
It is not possible to check EVERY malloc result within complex software.

2:
The reasonable solution (use a garbage collector) is not possible for
whatever reasons.

3:
A solution like the one proposed by Mr McLean (aborting) is not
possible for software quality reasons. The program must decide
if it is possible to just abort() or not.

Solution:

1) At program start, allocate a big buffer that is not used
elsewhere in the program. This big buffer will be freed when
a memory exhaustion situation arises, to give enough memory
to the error reporting routines to close files, or otherwise
do housekeeping chores.

2) xmalloc()
Why xmalloc? There's already a mechanism for dealing with such problems,
viz signals.

#include <signal.h>

void alloc_failure_handle(int x)
{
fputs("Dis baby gon crash and burn!\n", stderr);
exit(EXIT_FAILURE);
/* or insert your free-buffer-and-clean-up stuff if you prefer */
}

main()
{
signal(SIGSEGV, alloc_failure_handle);
/* ... */
}

Jan 29 '08 #2

P: n/a

"jacob navia" <ja***@nospam.comwrote in message
news:fn**********@aioe.org...
1:
It is not possible to check EVERY malloc result within complex software.
<snip>
>
4:
Using the above solution the application can abort if needed, or
make a long jump to a recovery point, where the program can continue.

The recovery handler is supposed to free memory, and reallocate the
BigUnusedBuffer, that has been set to NULL;
5:
add exception handling (ie, as a library feature, or, possibly as a compiler
extension).

this way, we can throw, and see if the app has any nifty ideas...
I propose implementing something combining both signal handling and
unwinding features.

sza=(1<<31)-1;
while(1)
{
i=exBegin();
if(!i)
{
//try...
pa=malloc_throw(sza);
}else if(i==EX_NO_MEM)
{
//catch...
sza=(sza/3)*2;
continue;
}else
{
//cleanup...
exRelay(i); //relay to next handler
}
break;
}
or something...
(ok, nevermind that this idea is stupid, as directly handling the NULL
pointer is simpler).

sza=(1<<31)-1;
pa=malloc(sza);
while(!pa)
{
sza=(sza/3)*2;
pa=malloc(sza);
}

>
--
jacob navia
jacob at jacob point remcomp point fr
logiciels/informatique
http://www.cs.virginia.edu/~lcc-win32
Jan 29 '08 #3

P: n/a
In article <sl*******************@nospam.invalid>,
CJ <cj@nospam.invalidwrote:
>Why xmalloc? There's already a mechanism for dealing with such problems,
viz signals.
signal(SIGSEGV, alloc_failure_handle);
An ignored malloc() failure is not guaranteed to lead to a SIGSEGV.
It may just result in memory corruption.

-- Richard

--
:wq
Jan 29 '08 #4

P: n/a
CJ wrote:
On 29 Jan 2008 at 11:47, jacob navia wrote:
>1:
It is not possible to check EVERY malloc result within complex
software.

2:
The reasonable solution (use a garbage collector) is not possible for
whatever reasons.

3:
A solution like the one proposed by Mr McLean (aborting) is not
possible for software quality reasons. The program must decide
if it is possible to just abort() or not.

Solution:

1) At program start, allocate a big buffer that is not used
elsewhere in the program. This big buffer will be freed when
a memory exhaustion situation arises, to give enough memory
to the error reporting routines to close files, or otherwise
do housekeeping chores.

2) xmalloc()

Why xmalloc? There's already a mechanism for dealing with such
problems, viz signals.

#include <signal.h>

void alloc_failure_handle(int x)
{
fputs("Dis baby gon crash and burn!\n", stderr);
The Standard library functions are not guaranteed to be reentrant and
you cannot safely call them from a signal handler.
exit(EXIT_FAILURE);
/* or insert your free-buffer-and-clean-up stuff if you prefer */
}

main()
{
signal(SIGSEGV, alloc_failure_handle);
/* ... */
}
The state of a program after it has handled a SIGSEGV is undefined as
per the Standard.

IMHO, it's far better to handle out-of-memory conditions within normal
code rather than invoking all the complexity and fragility of signals.

Jan 29 '08 #5

P: n/a
jacob navia <ja***@nospam.comwrote:
1:
It is not possible to check EVERY malloc result within complex software.
Bollocks from the start. Well done.

Richard
Jan 29 '08 #6

P: n/a

"jacob navia" <ja***@nospam.comwrote in message
3:
A solution like the one proposed by Mr McLean (aborting) is not
possible for software quality reasons. The program must decide
if it is possible to just abort() or not.
xmalloc() calls a caller-defined emergency function. It can do several
things, one of which is to simply exit. Don't confuse the requirement to
return the memory requested or (inherently) abort with the requirement to
abort.

--
Free games and programming goodies.
http://www.personal.leeds.ac.uk/~bgy1mm

Jan 29 '08 #7

P: n/a
jacob navia wrote:
>A solution for the allocation failures problem
You may have noticed this is somewhat controversial here. Doubt you will get
much agreement. It's one of those things where everyone has their own ideas.

--
Bart
Jan 29 '08 #8

P: n/a
jacob navia wrote:
1:
It is not possible to check EVERY malloc result within complex software.
Why not? It's no harder than checking every fopen() for
success -- or is that another impossibility?
2:
The reasonable solution (use a garbage collector) is not possible for
whatever reasons.
Garbage collection (when it works) eases the memory management
problem by relieving the programmer of the need to call free().
But collecting all the garbage does not imply that every malloc()
will succeed! In fact, a collector that cannot relocate the non-
garbage is likely to increase fragmentation and produce allocation
failures when a free()-as-soon-as-possible strategy would not.
3:
A solution like the one proposed by Mr McLean (aborting) is not
possible for software quality reasons. The program must decide
if it is possible to just abort() or not.

Solution:

1) At program start, allocate a big buffer that is not used
elsewhere in the program. This big buffer will be freed when
a memory exhaustion situation arises, to give enough memory
to the error reporting routines to close files, or otherwise
do housekeeping chores.
This is a reasonable thing to try, and has been tried
often. The hard part is choosing a value for "big:" too little
and there's not enough for the cleanup activity, too much and
you provoke allocation failures that don't need to happen.
4:
Using the above solution the application can abort if needed, or
make a long jump to a recovery point, where the program can continue.
The problems of longjmp() are well understood. It's possible
to use it effectively, but the programmers who write functions
"between" the setjmp() and the longjmp() must be constantly aware
that they might not get a chance to clean up:

char *this = dup(getenv("THIS"));
char *that = dup(getenv("THAT"));
printf ("THIS = %s, THAT = %s\n", this, that);
free (this);
free (that);

See the memory leak? If the second dup() eventually calls
longjmp() and returns to an ancestor of this code, the
memory allocated to `this' is never freed.

It's possible to work around this problem: various people
have put together macro packages that imitate a try/finally
discipline, for example. But the language itself gives little
help, and the compilers won't warn if somebody forgets (for
example, when a function that originally didn't need cleanup
acquires such a need during maintenance). Error recovery
based on longjmp() is do-able, but difficult.

--
Er*********@sun.com
Jan 29 '08 #9

P: n/a
jacob navia <ja***@nospam.comwrites:
1:
It is not possible to check EVERY malloc result within complex software.
[...]

It's not only possible, it's required.

--
Keith Thompson (The_Other_Keith) <ks***@mib.org>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
Jan 29 '08 #10

P: n/a

"Richard Bos" <rl*@hoekstra-uitgeverij.nlschreef in bericht
news:47*****************@news.xs4all.nl...
jacob navia <ja***@nospam.comwrote:
>1:
It is not possible to check EVERY malloc result within complex software.

Bollocks from the start. Well done.

Richard
Think he meant it's not possible to HANDLE every malloc failure. Indeed its
pretty easy putting an if in there, but not so easy to actually gracefully
handle it.

Jan 29 '08 #11

P: n/a
"Serve Laurijssen" <ni@hao.comwrites:
"Richard Bos" <rl*@hoekstra-uitgeverij.nlschreef in bericht
news:47*****************@news.xs4all.nl...
>jacob navia <ja***@nospam.comwrote:
>>1:
It is not possible to check EVERY malloc result within complex software.

Bollocks from the start. Well done.

Think he meant it's not possible to HANDLE every malloc
failure. Indeed its pretty easy putting an if in there, but not so
easy to actually gracefully handle it.
Yeah, programming is hard.

--
Keith Thompson (The_Other_Keith) <ks***@mib.org>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
Jan 29 '08 #12

P: n/a
Serve Laurijssen <ni@hao.comwrote:
"Richard Bos" <rl*@hoekstra-uitgeverij.nlschreef in bericht
news:47*****************@news.xs4all.nl...
jacob navia <ja***@nospam.comwrote:
1:
It is not possible to check EVERY malloc result within complex software.
Bollocks from the start. Well done.

Richard
Think he meant it's not possible to HANDLE every malloc failure. Indeed its
pretty easy putting an if in there, but not so easy to actually gracefully
handle it.
If you write software not intending to recover from a malloc failure, then,
yes, upon inspection your code will prove to be very difficult to modify to
handle such recovery.

But, that's beside the point. If at outset you write your code intending to
handle recovery, its not difficult at all. I don't remember (and granted I
can't remember what I wrote when I first began programming in C) ever being
in a situation where I found it difficult to recover or unwind from a path
because of a failed malloc call. Of course, I have developed a very
structured, almost rote method for writing software which suits me. But I
did so by necessity, because from very early on I never accepted the premise
that memory failure could or should be ignored.

I originally learned to program with Perl, then JavaScript (Navigator 3.0).
Subsequently, to me memory management was a _feature_. I found it much more
satisfying to have the capacity (if not perfectly realized or utilized) to
write more resilient programs.

When I don't wish to exercise that feature, and if C is otherwise not
particularly suited to a task, I use another language.

Aside from glib, I can't off-hand think of any widely used Free Software C
library which doesn't check and propogate memory allocation failures.

Jan 29 '08 #13

P: n/a
On Tue, 29 Jan 2008 12:47:27 +0100, jacob navia <ja***@nospam.com>
wrote in comp.lang.c:
1:
It is not possible to check EVERY malloc result within complex software.
Yes, it is. Poor programmers might not wish to do so, and might
produce arguments defending their choice, but that does not change the
fact that it is possible.

--
Jack Klein
Home: http://JK-Technology.Com
FAQs for
comp.lang.c http://c-faq.com/
comp.lang.c++ http://www.parashift.com/c++-faq-lite/
alt.comp.lang.learn.c-c++
http://www.club.cc.cmu.edu/~ajo/docs/FAQ-acllc.html
Jan 29 '08 #14

P: n/a


jacob navia wrote:
1:
It is not possible to check EVERY malloc result within complex software.
Pardon me.I haven't programmed for a few years but I was involved in
some fairly complex software development. Why is it not possible to
check every malloc result?
Jan 29 '08 #15

P: n/a
On Tue, 29 Jan 2008 12:47:27 +0100,
jacob navia <ja***@nospam.comwrote:
1:
It is not possible to check EVERY malloc result within complex software.
Why not?

Martien
--
|
Martien Verbruggen | System Administration is a dirty job, but
| someone said I had to do it.
|
Jan 29 '08 #16

P: n/a
jacob navia wrote:
1:
It is not possible to check EVERY malloc result within complex software.
I disagree. It is possible. You just have to decide you want to sweat
the details bad enough and do the hard work.

Regards,
Stan Milam
--
Life simultaneously, and paradoxically, makes indurate and pliable the soul.
Jan 30 '08 #17

P: n/a
ed*****@rcn.com wrote:
jacob navia wrote:
>1:
It is not possible to check EVERY malloc result within complex
software.

Pardon me.I haven't programmed for a few years but I was involved
in some fairly complex software development. Why is it not
possible to check every malloc result?
Don't believe everything you find written here.

--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]: <http://cbfalconer.home.att.net>
Try the download section.
--
Posted via a free Usenet account from http://www.teranews.com

Jan 30 '08 #18

P: n/a
On Tue, 29 Jan 2008 12:47:27 +0100, jacob navia wrote:
1:
It is not possible to check EVERY malloc result within complex software.
I already asked why once, but I'll ask again - WHY is it not possible?

It's actually very simple. When I write code which calls malloc - that
is, *immediately after writing the call to malloc* - the very next thing
I do is write the code which checks whether it failed.

ptr = malloc(x);
if ( ! ptr ) { handle }

It's a simple habit. If you develop the habit, it's not merely possible,
but trivial.

Nor is it limited to malloc:

MyWindow *win = createWindow();
if ( ! win ) { handle }

Do I know whether createWindow calls malloc? No. Nor do I care. What I
care about is that it can fail, and on failure, returns NULL. Nor is
this limited to allocations:

int sock = socket(...);
if ( sock == -1 ) { handle }

c = fgetc(fp);
if ( c == EOF ) { handle }

Why is it more difficult to check for allocation failures than to check
that you got a legitimate (non-EOF) character from a file, or a
legitimate socket handle, or any other case where the value returned may
be an indicator of a failure state?

Jan 30 '08 #19

P: n/a
Kelsey Bjarnason wrote:
On Tue, 29 Jan 2008 12:47:27 +0100, jacob navia wrote:
>1:
It is not possible to check EVERY malloc result within complex software.

I already asked why once, but I'll ask again - WHY is it not possible?

It's actually very simple. When I write code which calls malloc - that
is, *immediately after writing the call to malloc* - the very next thing
I do is write the code which checks whether it failed.
That's my usual practice, too, but sometimes I'll make
multiple malloc() calls and check all the results at once:

p = malloc(N * sizeof *p);
q = malloc(N * M * sizeof *q);
if (p == NULL || q == NULL) {
free (p);
free (q);
return SORRY_DAVE;
}

The disadvantage of this approach is that if the first
malloc() fails *and* if malloc() sets errno, any information
it may have placed in errno has probably been lost by the
time the results are tested. In my limited experience, few
malloc() implementations set errno to describe the failure
(usually if I call perror() after a failed malloc() I get
"No error"), so the disadvantage isn't crippling.

--
Er*********@sun.com
Jan 30 '08 #20

P: n/a
William Ahern wrote:
>
Serve Laurijssen <ni@hao.comwrote:
Think he meant it's not possible to HANDLE every malloc failure. Indeed its
pretty easy putting an if in there, but not so easy to actually gracefully
handle it.

But, that's beside the point. If at outset you write your code intending to
handle recovery, its not difficult at all.
Sometimes yes and sometimes no, depending on the work being done.
Allocation failure in one application I developed required
(automagical) powering up another network node to run another
instance of the application, and moving a portion of the
unexpectedly large body of information to the new node without
significantly impacting throughput. The recovery process was
complicated by the need to implement the reverse when loading
shrank.

/Sometimes/ it's not difficult. :-)

--
Morris Dovey
DeSoto Solar
DeSoto, Iowa USA
http://www.iedu.com/DeSoto
Jan 30 '08 #21

P: n/a
Kelsey Bjarnason wrote:
>
[snips]

On Tue, 29 Jan 2008 10:00:39 -0800, William Ahern wrote:
But, that's beside the point. If at outset you write your code intending
to handle recovery, its not difficult at all. I don't remember (and
granted I can't remember what I wrote when I first began programming in
C) ever being in a situation where I found it difficult to recover or
unwind from a path because of a failed malloc call. Of course, I have
developed a very structured, almost rote method for writing software
which suits me. But I did so by necessity, because from very early on I
never accepted the premise that memory failure could or should be
ignored.

I think that's the key to it. I keep hearing how it's so hard to check
such things, the code is messy, yadda yadda yadda, yet it's not. Why,
then, do some of us do this and find it relatively straightforward, while
some don't?
It's also possible that a failed malloc() is not fatal.

I have a function which builds an index of records in a database.
The sort routine allocates memory in chunks. As long as the first
malloc() succeeds, it is irrelevent whether later mallocs() succeed
or not. As it wants more memory, it malloc()s another chunk for
its buffers, until either it has all the memory it needs to hold
all of the sort keys, hits a configurable maximum number of chunks,
or malloc() fails. At that point, it simply uses whatever memory
it allocated. A failed malloc() simply means "don't allocate any
more chunks", and is handled no differently than "I have already
allocated the maximum number of chunks I was told to allow".

[...]

--
+-------------------------+--------------------+-----------------------+
| Kenneth J. Brody | www.hvcomputer.com | #include |
| kenbrody/at\spamcop.net | www.fptech.com | <std_disclaimer.h|
+-------------------------+--------------------+-----------------------+
Don't e-mail me at: <mailto:Th*************@gmail.com>
Jan 30 '08 #22

P: n/a
jacob navia <ja***@nospam.comwrites:
[...]
lcc-win tries to improve C. Most of the people here just try to
keep the language as it is without even getting rid of warts
like asctime() or similar.
You seem to be under the impression that the people who post here have
the ability to fix or get rid of asctime(). We don't.

But you know that.

--
Keith Thompson (The_Other_Keith) <ks***@mib.org>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
Jan 30 '08 #23

P: n/a
On Wed, 30 Jan 2008 10:19:21 -0500, Kenneth Brody wrote:
Kelsey Bjarnason wrote:
>>
[snips]

On Tue, 29 Jan 2008 10:00:39 -0800, William Ahern wrote:
But, that's beside the point. If at outset you write your code
intending to handle recovery, its not difficult at all. I don't
remember (and granted I can't remember what I wrote when I first
began programming in C) ever being in a situation where I found it
difficult to recover or unwind from a path because of a failed malloc
call. Of course, I have developed a very structured, almost rote
method for writing software which suits me. But I did so by
necessity, because from very early on I never accepted the premise
that memory failure could or should be ignored.

I think that's the key to it. I keep hearing how it's so hard to check
such things, the code is messy, yadda yadda yadda, yet it's not. Why,
then, do some of us do this and find it relatively straightforward,
while some don't?

It's also possible that a failed malloc() is not fatal.
Kinda the point to the whole discussion. The tools - xmalloc et al -
treat failure as fatal, when it often isn't, and even if it is, *I* - the
developer - want to decide what to do about it.
point, it simply uses whatever memory it allocated. A failed malloc()
simply means "don't allocate any more chunks", and is handled no
differently than "I have already allocated the maximum number of chunks
I was told to allow".
Bingo. It is *not* a situation where the only possible response is to
scream and die.

Jan 30 '08 #24

P: n/a
Stan Milam wrote:
jacob navia wrote:
>1: It is not possible to check EVERY malloc result within
complex software.

I disagree. It is possible. You just have to decide you want to
sweat the details bad enough and do the hard work.
What hard work? For example:

if (!(p = malloc(N * sizeof *p))) fixitup(&p, N * sizeof *p);
if (!p) exit(EXIT_FAILURE);

can handle and recover from all recoverable malloc errors. Takes 2
whole lines. You can customize the fixitup operation as needed.
If you actually want the silly xmalloc action just replace
fixitup() with exit(). That is only 1 line.

--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]: <http://cbfalconer.home.att.net>
Try the download section.

--
Posted via a free Usenet account from http://www.teranews.com

Jan 31 '08 #25

P: n/a
Kenneth Brody wrote:
>
.... snip ...
>
It's also possible that a failed malloc() is not fatal.

I have a function which builds an index of records in a database.
The sort routine allocates memory in chunks. As long as the first
malloc() succeeds, it is irrelevent whether later mallocs() succeed
or not. As it wants more memory, it malloc()s another chunk for
its buffers, until either it has all the memory it needs to hold
all of the sort keys, hits a configurable maximum number of chunks,
or malloc() fails. At that point, it simply uses whatever memory
it allocated. A failed malloc() simply means "don't allocate any
more chunks", and is handled no differently than "I have already
allocated the maximum number of chunks I was told to allow".
This is similar to the action of my hashlib package [1]. It an
attempt to store a new item fails because of malloc failure, the
package cleans up (after that particular store attempt) and returns
failure (and an available error code) to the application. The
application would normally tell the user, but might simply end a
phase of the processing.

[1] <http://cbfalconer.home.att.net/download/>

--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]: <http://cbfalconer.home.att.net>
Try the download section.

--
Posted via a free Usenet account from http://www.teranews.com

Jan 31 '08 #26

P: n/a
jacob navia wrote:
>
.... snip ...
>
These reasons are the ones that prompted me to add try/except to
the lcc-win C compiler. This allows for reasonable error handling.

All those people telling us otherwise are just telling us PLAIN
NONSENSE.

lcc-win tries to improve C. Most of the people here just try to
keep the language as it is without even getting rid of warts
like asctime() or similar.
No, most people here try to alter the language in the appropriate
newsgroup, known as "comp.std.c". They realize that comp.lang.c
discusses the use of the actual language, as specified in the
various ISO standards and K&R, and thus do not waste time and
bandwidth with nonsensical off-topic posts here.

--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]: <http://cbfalconer.home.att.net>
Try the download section.

--
Posted via a free Usenet account from http://www.teranews.com

Jan 31 '08 #27

P: n/a
On Jan 30, 7:54 pm, CBFalconer <cbfalco...@yahoo.comwrote:
Stan Milam wrote:
jacob navia wrote:
1: It is not possible to check EVERY malloc result within
complex software.
I disagree. It is possible. You just have to decide you want to
sweat the details bad enough and do the hard work.

What hard work? For example:

if (!(p = malloc(N * sizeof *p))) fixitup(&p, N * sizeof *p);
if (!p) exit(EXIT_FAILURE);

can handle and recover from all recoverable malloc errors. Takes 2
whole lines. You can customize the fixitup operation as needed.
If you actually want the silly xmalloc action just replace
fixitup() with exit(). That is only 1 line.
Your solution is over-complicated. Here's a nicer one:

int main (void)
{
return 0;
}

You can customize main() as needed.
Jan 31 '08 #28

P: n/a

"CBFalconer" <cb********@yahoo.comwrote in message
news:47***************@yahoo.com...
Stan Milam wrote:
>jacob navia wrote:
>>1: It is not possible to check EVERY malloc result within
complex software.

I disagree. It is possible. You just have to decide you want to
sweat the details bad enough and do the hard work.

What hard work? For example:

if (!(p = malloc(N * sizeof *p))) fixitup(&p, N * sizeof *p);
if (!p) exit(EXIT_FAILURE);
Hmm, perhaps not quite as eloquent as:

p=heap(N);

Then you can get on with the next thing. Of course this is a made-up
language, but this form clearly has some attraction and I think trying to
achieve something similar in C was the aim of this and the xmalloc thread.

(I keep seeing this form, usually as 'new', in other languages; what do
/they/ do when out-of-memory?)

--
Bart

Jan 31 '08 #29

P: n/a
Eric Sosman <Er*********@sun.comwrote:
That's my usual practice, too, but sometimes I'll make
multiple malloc() calls and check all the results at once:

p = malloc(N * sizeof *p);
q = malloc(N * M * sizeof *q);
if (p == NULL || q == NULL) {
free (p);
free (q);
return SORRY_DAVE;
}

The disadvantage of this approach is that if the first
malloc() fails *and* if malloc() sets errno, any information
it may have placed in errno has probably been lost by the
time the results are tested.
Only if another error occurred in the mean time. Functions which succeed
should not (IIRC are not allowed to) set errno.

Richard
Jan 31 '08 #30

P: n/a
Richard Bos wrote:
Functions
which succeed should not (IIRC are not allowed to) set errno.
Generally, any function may set errno to a nonzero value, even
if it succeeded.

However, no C library function may set errno to zero.

-- Ralf
Jan 31 '08 #31

P: n/a
On Tue, 29 Jan 2008 13:52:21 -0800, ediebur wrote:
jacob navia wrote:
>1:
It is not possible to check EVERY malloc result within complex
software.

Pardon me.I haven't programmed for a few years but I was involved in
some fairly complex software development. Why is it not possible to
check every malloc result?
Dunno; I've been cranking code for nigh on 30 years and I don't
understand why. Others - CBF, for example - appear to be equally baffled.

Jan 31 '08 #32

P: n/a
"Bartc" <bc@freeuk.comwrites:
"CBFalconer" <cb********@yahoo.comwrote in message
news:47***************@yahoo.com...
>Stan Milam wrote:
>>jacob navia wrote:

1: It is not possible to check EVERY malloc result within
complex software.

I disagree. It is possible. You just have to decide you want to
sweat the details bad enough and do the hard work.

What hard work? For example:

if (!(p = malloc(N * sizeof *p))) fixitup(&p, N * sizeof *p);
if (!p) exit(EXIT_FAILURE);

Hmm, perhaps not quite as eloquent as:

p=heap(N);

Then you can get on with the next thing. Of course this is a made-up
language, but this form clearly has some attraction and I think trying to
achieve something similar in C was the aim of this and the xmalloc thread.
Changing the syntax for memory allocation doesn't magically solve
anything.
(I keep seeing this form, usually as 'new', in other languages; what do
/they/ do when out-of-memory?)
It varies from language to language. Typically either the allocator
returns a null pointer on failure (exactly what C's malloc() does), or
it throws/raises an exception (if the language supports exceptions).

It's *always* possible for an attempted memory allocation to fail.
The possible ways to handle this possibility are:

1. Ignore it and keep going (dangerous).
2. Use an allocator that immediately aborts the program on failure.
3. Check for failure on each allocation. On failure:
3a. Immediately abort the program (equivalent to 2).
3b. Clean up and abort the program.
3c. Clean up and continue processing (this can be difficult, but
it's important if you want the program to be robust).
4. If the language supports exceptions (C doesn't), catch the error at
whatever level is appropriate, not necessarily immediately after
the attempted allocation. Perform appropriate cleanup and recovery
in the exception handler.

--
Keith Thompson (The_Other_Keith) <ks***@mib.org>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
Jan 31 '08 #33

P: n/a

"Keith Thompson" <ks***@mib.orgwrote in message
1. Ignore it and keep going (dangerous).
2. Use an allocator that immediately aborts the program on failure.
3. Check for failure on each allocation. On failure:
3a. Immediately abort the program (equivalent to 2).
3b. Clean up and abort the program.
3c. Clean up and continue processing (this can be difficult, but
it's important if you want the program to be robust).
4. If the language supports exceptions (C doesn't), catch the error at
whatever level is appropriate, not necessarily immediately after
the attempted allocation. Perform appropriate cleanup and recovery
in the exception handler.
5. Demand user intervention to supply the required memory.

--
Free games and programming goodies.
http://www.personal.leeds.ac.uk/~bgy1mm

Jan 31 '08 #34

P: n/a
jacob navia wrote:
[...]
Mr Bjarnason is found of saying how clever he is. This is of course
not verifiable here. He can claim he never makes allocation mistakes
but this sounds very hollow to me. Like other claims done by the
"regulars" here, they sound pathetic, like the one of the famous
Dan Pop that claimed that he never had a crash of a program written by
him.

Boasting here is very easy.
This, from the person who launched the thread with an
admission that his own programming skills were such that
he found it "impossible" to check for malloc() failure ...

--
Er*********@sun.com
Jan 31 '08 #35

P: n/a

"Eric Sosman" <Er*********@sun.comwrote in message
jacob navia wrote:
>>
Boasting here is very easy.

This, from the person who launched the thread with an
admission that his own programming skills were such that
he found it "impossible" to check for malloc() failure ...
He means impossible to write and test custom allocation failure hnadling
code for each call.
Any idiot can put malloc() in a wrapper or inside an if statement. It's what
does in the body of that if statement that can a cause problems. Not for
every large program, in my opinion, but for some programs.

--
Free games and programming goodies.
http://www.personal.leeds.ac.uk/~bgy1mm

Jan 31 '08 #36

P: n/a
Malcolm McLean wrote:
>
"Eric Sosman" <Er*********@sun.comwrote in message
>jacob navia wrote:
>>>
Boasting here is very easy.

This, from the person who launched the thread with an
admission that his own programming skills were such that
he found it "impossible" to check for malloc() failure ...
He means impossible to write and test custom allocation failure hnadling
code for each call.
Any idiot can put malloc() in a wrapper or inside an if statement. It's
what does in the body of that if statement that can a cause problems.
Not for every large program, in my opinion, but for some programs.
"There is no man blinder as the one that doesn't want to see"
--
jacob navia
jacob at jacob point remcomp point fr
logiciels/informatique
http://www.cs.virginia.edu/~lcc-win32
Jan 31 '08 #37

P: n/a
In article <87************@kvetch.smov.org>,
Keith Thompson <ks***@mib.orgwrote:
>It's *always* possible for an attempted memory allocation to fail.
(Or any other resource allocation)
>The possible ways to handle this possibility are:

1. Ignore it and keep going (dangerous).
2. Use an allocator that immediately aborts the program on failure.
3. Check for failure on each allocation. On failure:
3a. Immediately abort the program (equivalent to 2).
3b. Clean up and abort the program.
3c. Clean up and continue processing (this can be difficult, but
it's important if you want the program to be robust).
3d. Clean up and abort the current operation, and report failure to
the caller.
Let the caller choose how to handle it, which might be any of
the ways listed here. (Note that 1 may be less dangerous at
a higher level, depending on the operation that failed.)
>4. If the language supports exceptions (C doesn't), catch the error at
whatever level is appropriate, not necessarily immediately after
the attempted allocation. Perform appropriate cleanup and recovery
in the exception handler.
(Approximately equivalent to 3d.)
dave

--
Dave Vandervies dj3vande at eskimo dot com
Yeah. They're exactly the same product, just a different colour of plastic
in the case. In this case, the Microtech was much better, because it had a
lower $/suck ratio. --Graham Reed in the scary devil monastery
Jan 31 '08 #38

P: n/a
On Jan 31, 9:24 am, Keith Thompson <ks...@mib.orgwrote:
"Bartc" <b...@freeuk.comwrites:
"CBFalconer" <cbfalco...@yahoo.comwrote in message
news:47***************@yahoo.com...
Stan Milam wrote:
jacob navia wrote:
>>1: It is not possible to check EVERY malloc result within
complex software.
>I disagree. It is possible. You just have to decide you want to
sweat the details bad enough and do the hard work.
What hard work? For example:
if (!(p = malloc(N * sizeof *p))) fixitup(&p, N * sizeof *p);
if (!p) exit(EXIT_FAILURE);
Hmm, perhaps not quite as eloquent as:
p=heap(N);
Then you can get on with the next thing. Of course this is a made-up
language, but this form clearly has some attraction and I think trying to
achieve something similar in C was the aim of this and the xmalloc thread.

Changing the syntax for memory allocation doesn't magically solve
anything.
(I keep seeing this form, usually as 'new', in other languages; what do
/they/ do when out-of-memory?)

It varies from language to language. Typically either the allocator
returns a null pointer on failure (exactly what C's malloc() does), or
it throws/raises an exception (if the language supports exceptions).

It's *always* possible for an attempted memory allocation to fail.
The possible ways to handle this possibility are:

1. Ignore it and keep going (dangerous).
This is not a solution.
2. Use an allocator that immediately aborts the program on failure.
You only do this because you don't know how to solve the problem
without all that memory, or else, you *can't* solve the problem
without all that memory (rare.)
3. Check for failure on each allocation. On failure:
3a. Immediately abort the program (equivalent to 2).
3b. Clean up and abort the program.
3c. Clean up and continue processing (this can be difficult, but
it's important if you want the program to be robust).
3a = 3b = 2. Aborting doesn't require that you clean up first -- any
real OS(TM) will clean you up upon aborting anyways.

Your 3c only makes sense if you are talking about your whole program,
not just the code fragments where the error occurs (because that
typically will make no sense.) But you are inherently omitting how
you *achieve* it. In straight C you do it by observing the following
in your primitive calls:

3c1) Undo all operations in your scope and return from the current
function with an error code (my typical approach is NULL for pointers
and negative numbers for status codes).

which is applied recursively throughout your "primitives" call chain.
And in your policy/management calls:

3c2) If a strategy fails, go to the next strategy until all are
exhausted, then if everything fails log or return an error code that
amounts to "INCOMPLETE DUE TO LACK OR RESOURCES".

Of course this means you have to understand the distinction between
primitive calls and policy/management calls in your program.
Primitives make and handle data structures, whereas policy performs
actions on these data structures.
4. If the language supports exceptions (C doesn't), catch the error at
whatever level is appropriate, not necessarily immediately after
the attempted allocation. Perform appropriate cleanup and recovery
in the exception handler.
This is just a fancy way of implementing 3c with special language
features.

Going back to "at those lines of code" programming solutions I would
add:

5. Defer the error (but retain this information) until it can be
handled in an aggregated way.

6. Provide a stand-in for typical functionality. (I.e., return some
static memory declaration.) This can only be done in rare instances
where contention is a less serious issue than running out of memory,
or if you can guard the memory location with a semaphore, for example.

--
Paul Hsieh
http://www.pobox.com/~qed/
http://bstring.sf.net/
Jan 31 '08 #39

P: n/a
Paul Hsieh wrote, On 31/01/08 19:40:
On Jan 31, 9:24 am, Keith Thompson <ks...@mib.orgwrote:
>"Bartc" <b...@freeuk.comwrites:
>>"CBFalconer" <cbfalco...@yahoo.comwrote in message
news:47***************@yahoo.com...
Stan Milam wrote:
jacob navia wrote:
>1: It is not possible to check EVERY malloc result within
> complex software.
I disagree. It is possible. You just have to decide you want to
sweat the details bad enough and do the hard work.
What hard work? For example:
if (!(p = malloc(N * sizeof *p))) fixitup(&p, N * sizeof *p);
if (!p) exit(EXIT_FAILURE);
Hmm, perhaps not quite as eloquent as:
p=heap(N);
Then you can get on with the next thing. Of course this is a made-up
language, but this form clearly has some attraction and I think trying to
achieve something similar in C was the aim of this and the xmalloc thread.
Changing the syntax for memory allocation doesn't magically solve
anything.
>>(I keep seeing this form, usually as 'new', in other languages; what do
/they/ do when out-of-memory?)
It varies from language to language. Typically either the allocator
returns a null pointer on failure (exactly what C's malloc() does), or
it throws/raises an exception (if the language supports exceptions).

It's *always* possible for an attempted memory allocation to fail.
The possible ways to handle this possibility are:

1. Ignore it and keep going (dangerous).

This is not a solution.
Agreed. However, it is what some people seem to do.
>2. Use an allocator that immediately aborts the program on failure.

You only do this because you don't know how to solve the problem
without all that memory, or else, you *can't* solve the problem
without all that memory (rare.)
Agreed.
>3. Check for failure on each allocation. On failure:
3a. Immediately abort the program (equivalent to 2).
3b. Clean up and abort the program.
3c. Clean up and continue processing (this can be difficult, but
it's important if you want the program to be robust).

3a = 3b = 2. Aborting doesn't require that you clean up first -- any
real OS(TM) will clean you up upon aborting anyways.
<snip>

Cleaning up is not always simply a matter of freeing memory and closing
files. In the clean-up code we have for one application I work on it
also involves...
Sending a message to the client to say it is aborting
Deleting temporary files that where deliberately *not* opened using
tmpfile (there is reason for this)
Logging that it is crashing
Sending an email saying that it is crashing I've tested this and it
can manage it some of the time
--
Flash Gordon
Jan 31 '08 #40

P: n/a
Paul Hsieh <we******@gmail.comwrites:
On Jan 31, 9:24 am, Keith Thompson <ks...@mib.orgwrote:
[...]
>It's *always* possible for an attempted memory allocation to fail.
The possible ways to handle this possibility are:

1. Ignore it and keep going (dangerous).

This is not a solution.
I didn't mean to imply that it is.
>2. Use an allocator that immediately aborts the program on failure.

You only do this because you don't know how to solve the problem
without all that memory, or else, you *can't* solve the problem
without all that memory (rare.)
Agreed.
>3. Check for failure on each allocation. On failure:
3a. Immediately abort the program (equivalent to 2).
3b. Clean up and abort the program.
3c. Clean up and continue processing (this can be difficult, but
it's important if you want the program to be robust).

3a = 3b = 2. Aborting doesn't require that you clean up first -- any
real OS(TM) will clean you up upon aborting anyways.
No, 3b is not equivalent to 2. Cleaning up can include
application-specific stuff that the OS can't be expected to handle.
For example, a text editor might save a copy of its buffer to disk
before aborting, as opposed to having the memory allocator
preemptively abort the program without giving the application to save
anything.
Your 3c only makes sense if you are talking about your whole program,
not just the code fragments where the error occurs (because that
typically will make no sense.) But you are inherently omitting how
you *achieve* it.
Yes, I am.

[...]
>4. If the language supports exceptions (C doesn't), catch the error at
whatever level is appropriate, not necessarily immediately after
the attempted allocation. Perform appropriate cleanup and recovery
in the exception handler.

This is just a fancy way of implementing 3c with special language
features.
Yes, but it can be a lot easier for the programmer (at the expense of
extra work for the implementation).

[snip]

--
Keith Thompson (The_Other_Keith) <ks***@mib.org>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
Jan 31 '08 #41

P: n/a
"Paul Hsieh" <we******@gmail.comwrote in message news
On Jan 31, 9:24 am, Keith Thompson <ks...@mib.orgwrote:
>2. Use an allocator that immediately aborts the program on failure.

You only do this because you don't know how to solve the problem
without all that memory, or else, you *can't* solve the problem
without all that memory (rare.)
Rarely it makes sense to have adjustable buffer sizes. But only rarely. A
theoretical result is that you _can_ perform any computation with just a
disk file and a finite list of states, but generally if a small amount of
memory is not made available, you might as well forget about the operation.

You've instantly doubled you development costs by insisting on this.

--
Free games and programming goodies.
http://www.personal.leeds.ac.uk/~bgy1mm

Jan 31 '08 #42

P: n/a
[snips]

On Thu, 31 Jan 2008 17:21:04 +0000, Ed Jensen wrote:
>In _C_, as a developer, I adopt habits which I find work. Yes, the
code to propagate errors can make the code longer, but guess what?
I'll take longer over random crashing any day, thanks.
Straw man. I never advocated not checking for allocation failures and
allowing programs to crash.
Oh, so then we're agreed: the code to do this in C is exactly as easy to
read as the code to do this in C, thus making all this nonsense about it
being difficult a complete load of twaddle.
This is (as noted) not any harder to write (or read) than code without
-error checking once the habit has been developed
Gosh, "not any harder to write (or read)". That's quite a claim!
Given that you apparently have trouble reading English, it's not
surprising you'd have trouble reading something such as C. I trimmed the
quoted part so that you can ponder how "once the habit has been
developed", being dropped from your rejoinder, makes your response look
somewhat foolish.
I
think I proved otherwise, but if you feel memory management in C is "not
any harder to write (or read)", you go on believing it, Kelsey.
Perhaps, if you learn to read, we can continue this. Until then, with
both English and C apparently beyond your ken, there's little point.

Feb 1 '08 #43

P: n/a
[snips]

On Thu, 31 Jan 2008 18:27:11 +0100, jacob navia wrote:
Mr Bjarnason is found of saying how clever he is.
I'm sure you'll post links to _any_ post in which I claim to be clever,
to back that up, right?

Whoops, no, you won't, because you're lying. Why, I'm not sure, but you
are.
verifiable here. He can claim he never makes allocation mistakes but
this sounds very hollow to me.
Where did I say that?

No, seriously, where did I say I never make allocation mistakes? That
would include, among other things, allocating more memory than is
required, less than is optimal, a whole range of cases.

I'm sure you'll post a link to any post where I said I never make any
allocation mistakes, to back up your claim, right?

Whoops, no, you won't, because you're lying. Why, I'm not sure, but you
are.
Feb 1 '08 #44

P: n/a
Malcolm McLean wrote:
"Keith Thompson" <ks***@mib.orgwrote in message
>1. Ignore it and keep going (dangerous).
2. Use an allocator that immediately aborts the program on failure.
3. Check for failure on each allocation. On failure:
3a. Immediately abort the program (equivalent to 2).
3b. Clean up and abort the program.
3c. Clean up and continue processing (this can be difficult, but
it's important if you want the program to be robust).
4. If the language supports exceptions (C doesn't), catch the
error at whatever level is appropriate, not necessarily
immediately after the attempted allocation. Perform
appropriate cleanup and recovery in the exception handler.
5. Demand user intervention to supply the required memory.
Due to my natural intransigence, when machines 'demand' I tend to
ignore them. This has been known to lead to program failure. :-)

--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]: <http://cbfalconer.home.att.net>
Try the download section.

--
Posted via a free Usenet account from http://www.teranews.com

Feb 1 '08 #45

P: n/a
Malcolm McLean wrote:
"Kelsey Bjarnason" <kb********@gmail.comwrote in message
>>
.... snip ...
>>
That -3, since it isn't trapped, now merrily attempts to allocate
some 18446744073709551613 bytes, with who knows what consequences.

The consquence can only be to return zero for an insufficient
memory condition.

An application that asks for a negative amount of memory is bugged.
There's no ideal way to treat a bugged application.
You seem to have a reading impediment. It asked for:
18446744073709551613
repeat 18446744073709551613 bytes.

To me, 18446744073709551613 is not a negative number. If the
malloc system has such a memory block available, it should mark it
as used and return a pointer to it.

--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]: <http://cbfalconer.home.att.net>
Try the download section.

--
Posted via a free Usenet account from http://www.teranews.com

Feb 1 '08 #46

P: n/a
jacob navia wrote:
>
.... snip ...
>
Boasting here is very easy.
Nah. There are two things I never do. Boast, and make mistakes.

--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]: <http://cbfalconer.home.att.net>
Try the download section.

--
Posted via a free Usenet account from http://www.teranews.com

Feb 1 '08 #47

P: n/a
jacob navia wrote:
>
.... snip ...
>
C can use garbage collection and exceptions. For an implementation
of those features in C see the lcc-win compiler system
No, C can't. C can use an extension library implementing such a
system for some subset of the memory allocation problems. Please
try to keep that distinction in mind. The C standard already
specifies various things about the malloc/calloc/realloc/free
subsystem.

--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]: <http://cbfalconer.home.att.net>
Try the download section.

--
Posted via a free Usenet account from http://www.teranews.com

Feb 1 '08 #48

P: n/a
jacob navia wrote:
Mr Bjarnason is found of saying how clever he is. This is of course
not verifiable here. He can claim he never makes allocation mistakes
but this sounds very hollow to me. Like other claims done by the
"regulars" here, they sound pathetic, like the one of the famous
Dan Pop that claimed that he never had a crash of a program written by
him.

Boasting here is very easy.
Sometimes it's good to be an "irregular" so as to be (or at least
feel) exempt from this kind of envy. <g>

Hmm - I'm not terribly clever, so if you consider yourself
particularly clever you're welcome to feel superior. I don't let
my lack of cleverness keep me from boasting a bit from time to
time, but just so that you don't feel threatened I'll offer the
following credentials right up front:

~: ls -al --full-time core
-rw------- 1 mrd 262144 Tue Jan 29 14:17:30 2008
core

Jacob, you can cast yourself in the role of pariah if you choose
- but so doing won't gain you much traction. No offense intended,
it just hasn't ever worked in anyone's favor here...

I don't know Kelsey, but can offer that Dan Pop liked to play the
role of an irascible old curmudgeon. He plays that role well, but
you're less clever than you imagine if you confuse the actor with
the role.

--
Morris Dovey
DeSoto Solar
DeSoto, Iowa USA
http://www.iedu.com/DeSoto
Feb 1 '08 #49

P: n/a
ed*****@rcn.com wrote:
>
jacob navia wrote:
1:
It is not possible to check
EVERY malloc result within complex software.

Pardon me.I haven't programmed for a few years but I was involved in
some fairly complex software development. Why is it not possible to
check every malloc result?
10 Because it's impossible to write "if"
as many times as "malloc".

9

--
pete
Feb 1 '08 #50

158 Replies

This discussion thread is closed

Replies have been disabled for this discussion.