Recovering gracefully from a realloc() failure

[Philip Pemberton]

Hi,
I'm currently playing around with a little spam filter for my Linksys NSLU2
(I would have used Spamassassin but that's slow even on my 2GHz Athlon64 - on
a 266MHz XScale it's likely to be intolerably slow). As part of that, I'm
keeping track of the DNSBL blocklists that are being queried using a
dynamically-allocated array.

When a new DNSBL zone is added to the zone array, I use realloc() to
increase the amount of memory available for the array. The newly-allocated
block then gets filled in with the zone's details, and a null block (one where
the "zone" pointer is set to NULL) is stored to act as a terminator.

At the moment, I'm using code like this:
if ((zones = realloc(zones, (zonecount + 2) * sizeof(ZONE))) == NULL) {
// something broke, bail out
exit(1);
}

But this has the side effect of clobbering the "zones" pointer if realloc
fails, making anything other than a bail-out impossible (because the zone
table has been mangled). What I was thinking of doing was something like this:
if ((ztemp = realloc(zones, (zonecount + 2) * sizeof(ZONE))) == NULL) {
// something broke, tell the caller and let them deal with it
return ERROR_MEMALLOC;
} else {
// it worked, update the pointer
zones = ztemp;
}

What I was wondering was how realloc() actually behaved. Can the old value
of "zones" be considered to be valid if realloc() fails (i.e. returns NULL)?
In other words, can I continue on using the old value of "zones" (i.e. before
the reassignment) if realloc() fails, and just post an error to syslog along
the lines of "couldn't add zone table entry, out of memory"?

That said, "out of memory" is a pretty rare condition these days, what with
virtual memory and such, but I'd like to handle this situation as gracefully
as possible. If that means bailing out then fine, but if I can make my code
recover from the error, then I'd like to add the code to make that happen.

I've checked the Linux manpages on this, and K&R, but haven't found a
conclusive answer.

Thanks.
--
Phil. | Kitsune: Acorn RiscPC SA202 64M+6G ViewFinder
ph*****@despammed.com (valid!)| Cheetah: Athlon64 3200+ A8VDeluxeV2 512M+100G
http://www.philpem.me.uk/ | Tiger: Toshiba SatPro4600 Celeron700 256M+40G
No software patents! <http://www.eff.org/> / <http://www.ffii.org/>

[Al Balmer]

On Thu, 20 Apr 2006 19:13:41 +0100, Philip Pemberton
<ph*****@despammed.com> wrote:

Hi,
I'm currently playing around with a little spam filter for my Linksys NSLU2
(I would have used Spamassassin but that's slow even on my 2GHz Athlon64 - on
a 266MHz XScale it's likely to be intolerably slow). As part of that, I'm
keeping track of the DNSBL blocklists that are being queried using a
dynamically-allocated array.

When a new DNSBL zone is added to the zone array, I use realloc() to
increase the amount of memory available for the array. The newly-allocated
block then gets filled in with the zone's details, and a null block (one where
the "zone" pointer is set to NULL) is stored to act as a terminator.

At the moment, I'm using code like this:
if ((zones = realloc(zones, (zonecount + 2) * sizeof(ZONE))) == NULL) {
// something broke, bail out
exit(1);
}

But this has the side effect of clobbering the "zones" pointer if realloc
fails, making anything other than a bail-out impossible (because the zone
table has been mangled). What I was thinking of doing was something like this:
if ((ztemp = realloc(zones, (zonecount + 2) * sizeof(ZONE))) == NULL) {
// something broke, tell the caller and let them deal with it
return ERROR_MEMALLOC;
} else {
// it worked, update the pointer
zones = ztemp;
}

What I was wondering was how realloc() actually behaved. Can the old value
of "zones" be considered to be valid if realloc() fails (i.e. returns NULL)?
From the standard:
"If memory for the new
object cannot be allocated, the old object is not deallocated and its
value is unchanged."
In other words, can I continue on using the old value of "zones" (i.e. before
the reassignment) if realloc() fails, and just post an error to syslog along
the lines of "couldn't add zone table entry, out of memory"?

That said, "out of memory" is a pretty rare condition these days, what with
virtual memory and such, but I'd like to handle this situation as gracefully
as possible. If that means bailing out then fine, but if I can make my code
recover from the error, then I'd like to add the code to make that happen.

I've checked the Linux manpages on this, and K&R, but haven't found a
conclusive answer.
Thanks.

--
Al Balmer
Sun City, AZ

[pragma]

According to the malloc functions manual, one can find (on the
RETURN VALUE section) the final phrase on the last paragraph: If
realloc() fails the original block is left untouched - it is not freed
or moved. So, if (zonecount + 2) > 0 is _always_ TRUE, your memory
block still there.

You can try to use a custom realloc() implementation, that wraps
errors and prints out some debugging info, and even use it as a macro
(i.e. for less function call overhead):

#include <stdlib.h>
#include <stdio.h>

/* __FILE__ and __LINE__ are usefull prepocessor macros */
#define XREALLOC(ptr, size) xrealloc(ptr, size, __FILE__, __LINE__)

void *
xrealloc (void *ptr, size_t size, char *fname, int lnum)
{
register void *value = realloc (ptr, size);

if (value == NULL)
{
fprintf (stderr, "%s:%d - realloc error\n", fname, lnum);
return NULL; //here the caller can deal with the error, but it's
recommended to exit ()
/* exit (EXIT_FAILURE); */
}
return value;
}

You can use the macro this way:

ztemp = (ZONE *) XREALLOC (zones, (zonecount + 2) * sizeof (ZONE));

and wrap (ztemp == NULL) to free memory and try again, et cetera.
If your application is linear (i.e. make the job in a serial way),
thre's no sense to wait some memory to apear, and exit() is the better
choice, but if we are expecting some memory to be realeased on the next
instants (wonder a multi-threaded application allocating/freeing memory
asychronously), make a try counter, sleep for a while, and try until
the counter reaches a limit (i.e. wait 100 us, and try 5 counts before
exit ()).

----mauricio

[Philip Pemberton]

Al Balmer wrote:

From the standard:
"If memory for the new
object cannot be allocated, the old object is not deallocated and its
value is unchanged."

Thanks for that - guess I'll have to try and get a copy of the C standard at
some point. K&R is great for most things, but I'm surprised it doesn't cover
realloc (or if it does it's certainly not listed in the index).

I gather that by "the standard" you mean "ISO/IEC 9899:1999: Programming
Language C"?

Thanks.
--
Phil. | Kitsune: Acorn RiscPC SA202 64M+6G ViewFinder
ph*****@despammed.com (valid!)| Cheetah: Athlon64 3200+ A8VDeluxeV2 512M+100G
http://www.philpem.me.uk/ | Tiger: Toshiba SatPro4600 Celeron700 256M+40G
No software patents! <http://www.eff.org/> / <http://www.ffii.org/>

[CBFalconer]

Philip Pemberton wrote:

.... snip ...
But this has the side effect of clobbering the "zones" pointer if
realloc fails, making anything other than a bail-out impossible
(because the zone table has been mangled). What I was thinking of
doing was something like this:

if ((ztemp = realloc(zones, (zonecount + 2) * sizeof(ZONE))) == NULL) {
// something broke, tell the caller and let them deal with it
return ERROR_MEMALLOC;
} else {
// it worked, update the pointer
zones = ztemp;
}

What I was wondering was how realloc() actually behaved. Can the
old value of "zones" be considered to be valid if realloc() fails
(i.e. returns NULL)? In other words, can I continue on using the
old value of "zones" (i.e. before the reassignment) if realloc()
fails, and just post an error to syslog along the lines of
"couldn't add zone table entry, out of memory"?

That is the correct way to handle it. On failure realloc will not
disturb the old pointer or the material to which it points.

--
I hear the voices. G W Bush, 2006-04-18

[Keith Thompson]

Philip Pemberton <ph*****@despammed.com> writes:

Al Balmer wrote:

From the standard:
"If memory for the new
object cannot be allocated, the old object is not deallocated and its
value is unchanged."

Thanks for that - guess I'll have to try and get a copy of the C
standard at some point. K&R is great for most things, but I'm
surprised it doesn't cover realloc (or if it does it's certainly not
listed in the index).

I gather that by "the standard" you mean "ISO/IEC 9899:1999:
Programming Language C"?

That's the current official standard, yes. Most implementations don't
yet fully support it; the older 1990 standard is almost universally
supported (often with extensions).

A draft of the standard, including the C99 standard with later
modifications ("Technical Corrigenda") merged in and marked with
change bars, is freely available as n1124.pdf. I have a copy of the
actual standard (for which I paid $18 to ANSI), but these days I
almost always use n1124 instead.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

[Flash Gordon]

Philip Pemberton wrote:

Al Balmer wrote:

From the standard:
"If memory for the new
object cannot be allocated, the old object is not deallocated and its
value is unchanged."
Thanks for that - guess I'll have to try and get a copy of the C
standard at some point. K&R is great for most things, but I'm surprised
it doesn't cover realloc (or if it does it's certainly not listed in the
index).

<fx: wanders off to grab copy of K&R2>

It's in the index of the second edition. If you have the first edition
then I suggest you go out and buy the "modern" version that was
published in 1989 since it tells you about all sorts of cool "new"
things, such as function prototypes and void.
I gather that by "the standard" you mean "ISO/IEC 9899:1999: Programming
Language C"?

I'm sure he does. I suggest you look here
http://clc-wiki.net/wiki/c_standard
--
Flash Gordon, living in interesting times.
Web site - http://home.flash-gordon.me.uk/
comp.lang.c posting guidelines and intro:
http://clc-wiki.net/wiki/Intro_to_clc

Inviato da X-Privat.Org - Registrazione gratuita http://www.x-privat.org/join.php

[Al Balmer]

On Thu, 20 Apr 2006 20:30:46 +0100, Philip Pemberton
<ph*****@despammed.com> wrote:

Al Balmer wrote:

From the standard:
"If memory for the new
object cannot be allocated, the old object is not deallocated and its
value is unchanged."
Thanks for that - guess I'll have to try and get a copy of the C standard at
some point. K&R is great for most things, but I'm surprised it doesn't cover
realloc (or if it does it's certainly not listed in the index).

It's listed in the second edition - page 252. A short paragraph, in
the usual K&R style of no unnecessary words :-)
I gather that by "the standard" you mean "ISO/IEC 9899:1999: Programming
Language C"?

Yes. It's available as a PDF from ANSI for $18.00. That's what I use.

--
Al Balmer
Sun City, AZ

[stathis gotsis]

"pragma" <mp****@if.ufrgs.br> wrote in message
news:11*********************@v46g2000cwv.googlegro ups.com...

According to the malloc functions manual, one can find (on the
RETURN VALUE section) the final phrase on the last paragraph: If
realloc() fails the original block is left untouched - it is not freed
or moved. So, if (zonecount + 2) > 0 is _always_ TRUE, your memory
block still there.

You can try to use a custom realloc() implementation, that wraps
errors and prints out some debugging info, and even use it as a macro
(i.e. for less function call overhead):

#include <stdlib.h>
#include <stdio.h>

/* __FILE__ and __LINE__ are usefull prepocessor macros */
#define XREALLOC(ptr, size) xrealloc(ptr, size, __FILE__, __LINE__)

void *
xrealloc (void *ptr, size_t size, char *fname, int lnum)
{
register void *value = realloc (ptr, size);

if (value == NULL)
{
fprintf (stderr, "%s:%d - realloc error\n", fname, lnum);
return NULL; //here the caller can deal with the error, but it's
recommended to exit ()
/* exit (EXIT_FAILURE); */
}
return value;
}

You can use the macro this way:

ztemp = (ZONE *) XREALLOC (zones, (zonecount + 2) * sizeof (ZONE));
And why would you need to cast XREALLOC() here?
and wrap (ztemp == NULL) to free memory and try again, et cetera.

[Barry Schwarz]

On Thu, 20 Apr 2006 20:30:46 +0100, Philip Pemberton
<ph*****@despammed.com> wrote:

Al Balmer wrote:

From the standard:
"If memory for the new
object cannot be allocated, the old object is not deallocated and its
value is unchanged."

Thanks for that - guess I'll have to try and get a copy of the C standard at
some point. K&R is great for most things, but I'm surprised it doesn't cover
realloc (or if it does it's certainly not listed in the index).

It's in the index on page 269 in my copy. The index refers to page
252. The wording is a tad different but it matches the intent that Al
quoted.
Remove del for email

[Skarmander]

Philip Pemberton wrote:
<snip>

That said, "out of memory" is a pretty rare condition these days, what
with virtual memory and such,
This is OT, but virtual memory is neither inexhaustible nor necessarily
unbounded. My machine in particular has a fixed amount of virtual memory,
and it does get exhausted (mostly by rude applications who *could* do with
less memory if they hadn't been written with the assumption that large
amounts of memory will magically appear and disappear). I still prefer
applications crashing left and right to losing my system through thrashing.
but I'd like to handle this situation as gracefully as possible.

Good for you. Usually a best effort suffices. Whatever you do is better than
assuming it never fails, even if it's just displaying "out of memory" rather
than "segmentation fault".

S.