By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,905 Members | 900 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,905 IT Pros & Developers. It's quick & easy.

Checking return values for errors, a matter of style?

P: n/a
I've written a piece of code that uses sockets a lot (I know that
sockets aren't portable C, this is not a question about sockets per
se). Much of my code ended up looking like this:

Expand|Select|Wrap|Line Numbers
  1. if (function(socket, args) == -1) {
  2. perror("function");
  3. exit(EXIT_FAILURE);
  4. }
I feel that the ifs destroy the readability of my code. Would it be
better to declare an int variable (say succ) and use the following
structure?

Expand|Select|Wrap|Line Numbers
  1. int succ;
  2.  
  3. succ = function(socket, args);
  4. if (succ == -1) {
  5. perror("function");
  6. exit(EXIT_FAILURE);
  7. }
What's considered "best practice" (feel free to substitute with: "what
do most good programmers use")?
Jul 31 '06 #1
Share this Question
Share on Google+
66 Replies


P: n/a
Johan Tibell said:
I've written a piece of code that uses sockets a lot (I know that
sockets aren't portable C, this is not a question about sockets per
se). Much of my code ended up looking like this:

if (function(socket, args) == -1) {
perror("function");
exit(EXIT_FAILURE);
}

I feel that the ifs destroy the readability of my code. Would it be
better to declare an int variable (say succ) and use the following
structure?

int succ;

succ = function(socket, args);
if (succ == -1) {
perror("function");
exit(EXIT_FAILURE);
}

What's considered "best practice" (feel free to substitute with: "what
do most good programmers use")?
My normal practice is to use an int to catch the return value so that I can
inspect it. My library routines - at least, the ones that don't return
pointers - return int, with one particular value indicating success, and
any value other than that indicating the reason for the error. There's no
point in my wondering /why/ a function failed if I can't be bothered to
store the error code returned by that function!

But there are times when I use the other way - classic example would be an
fgets loop:

while(fgets(buffer, sizeof buffer, stdin) != NULL)
{
dosomethingwith(buffer);
}

--
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at above domain (but drop the www, obviously)
Jul 31 '06 #2

P: n/a
Johan Tibell posted:
I've written a piece of code that uses sockets a lot (I know that
sockets aren't portable C, this is not a question about sockets per
se). Much of my code ended up looking like this:

if (function(socket, args) == -1) {
perror("function");
exit(EXIT_FAILURE);
}

I feel that the ifs destroy the readability of my code. Would it be
better to declare an int variable (say succ) and use the following
structure?

int succ;

succ = function(socket, args);
if (succ == -1) {
perror("function");
exit(EXIT_FAILURE);
}

What's considered "best practice" (feel free to substitute with: "what
do most good programmers use")?

Have you considered something like:

void Assure(int const i)
{
if(-1 == i)
{
perror("function");
exit(EXIT_FAILURE);
}
}

And then putting something like the following in your code:

Assure( function(socket,args) );

--

Frederick Gotham
Jul 31 '06 #3

P: n/a
Johan Tibell a écrit :
I've written a piece of code that uses sockets a lot (I know that
sockets aren't portable C, this is not a question about sockets per
se). Much of my code ended up looking like this:

if (function(socket, args) == -1) {
perror("function");
exit(EXIT_FAILURE);
}

I feel that the ifs destroy the readability of my code. Would it be
better to declare an int variable (say succ) and use the following
structure?

int succ;

succ = function(socket, args);
if (succ == -1) {
perror("function");
exit(EXIT_FAILURE);
}

What's considered "best practice" (feel free to substitute with: "what
do most good programmers use")?
Using the second form allows you to easily see the return value in
the debugger. To see the error code using the first form you would
have to follow the machine in machine code and read the return
register... not so easy in many debuggers.

Jul 31 '06 #4

P: n/a


Johan Tibell wrote On 07/31/06 17:20,:
I've written a piece of code that uses sockets a lot (I know that
sockets aren't portable C, this is not a question about sockets per
se). Much of my code ended up looking like this:

if (function(socket, args) == -1) {
perror("function");
exit(EXIT_FAILURE);
}

I feel that the ifs destroy the readability of my code. Would it be
better to declare an int variable (say succ) and use the following
structure?

int succ;

succ = function(socket, args);
if (succ == -1) {
perror("function");
exit(EXIT_FAILURE);
}

What's considered "best practice" (feel free to substitute with: "what
do most good programmers use")?
I don't see much difference between these two styles.
They take about the same amount of verbiage, and are equally
readable. I think I'd prefer the second style (slightly) if
the returned value has a significance beyond just success vs.
failure or if the function call is long and involved; I'd favor
the first style if the returned value is strictly a yes/no
status and the function call is fairly brief. It's not worth
making a fetish of, though.

If my program had a lot of these and I got tired of
typing the boilerplate over and over (probably misspelling
EXIT_FALIURE a few times), I'd write myself a tiny wrapper
along the lines of

static void crash(const char *message) {
if (message != NULL)
perror (message);
exit (EXIT_FAILURE);
}

Then the "down in the trenches" code becomes

if (function(socket, args) == -1)
crash ("function");

succ = function(socket, args);
if (succ == -1)
crash ("function");

Let's see: Expending five lines on the wrapper function saves
me two lines each time I use it, so I'm ahead of the game as
soon as I've checked for my third error ;-) More importantly,
the smaller "footprint" of the error-handling code lets the
reader scan it with perhaps a little more ease.

--
Er*********@sun.com

Jul 31 '06 #5

P: n/a
jacob navia <ja***@jacob.remcomp.frwrites:
Johan Tibell a écrit :
>I've written a piece of code that uses sockets a lot (I know that
sockets aren't portable C, this is not a question about sockets per
se). Much of my code ended up looking like this:
if (function(socket, args) == -1) {
perror("function");
exit(EXIT_FAILURE);
}
I feel that the ifs destroy the readability of my code. Would it be
better to declare an int variable (say succ) and use the following
structure?
int succ;
succ = function(socket, args);
if (succ == -1) {
perror("function");
exit(EXIT_FAILURE);
}
What's considered "best practice" (feel free to substitute with:
"what
do most good programmers use")?

Using the second form allows you to easily see the return value in
the debugger. To see the error code using the first form you would
have to follow the machine in machine code and read the return
register... not so easy in many debuggers.
That could be a valid approach, depending on your development style
and environment.

Personally, I don't use debuggers very often, so it usually wouldn't
occur to me to distort my code to make it easier to use in a debugger.
But if you find it useful, go for it.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Jul 31 '06 #6

P: n/a
"Johan Tibell" <jo**********@gmail.comwrites:
I've written a piece of code that uses sockets a lot (I know that
sockets aren't portable C, this is not a question about sockets per
se). Much of my code ended up looking like this:

if (function(socket, args) == -1) {
perror("function");
exit(EXIT_FAILURE);
}

I feel that the ifs destroy the readability of my code. Would it be
better to declare an int variable (say succ) and use the following
structure?

int succ;

succ = function(socket, args);
if (succ == -1) {
perror("function");
exit(EXIT_FAILURE);
}

What's considered "best practice" (feel free to substitute with: "what
do most good programmers use")?
I find both forms about equally readable.

If you're only going to refer to the function's result once, as you do
here, there's no real need to store the value in a variable. If
you're going to refer to it more than once, a variable is helpful,
perhaps essential. For example:

int result;

result = function(socket, args);
if (result != 0) {
fprintf(stderr,
"function returned %d, errno = %d\n",
result,
errno);
exit(EXIT_FAILURE);
}

But if the function returns only a simple success/failure code, and
the detailed information is elsewhere, this probably isn't necessary.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Jul 31 '06 #7

P: n/a
Keith Thompson a écrit :
Personally, I don't use debuggers very often, so it usually wouldn't
occur to me to distort my code to make it easier to use in a debugger.
Ahhh... You do not use debuggers very often?

Mmmm... Well, they are great tools. I use them very often,
actually I spend most of the time either in the editor
or in the debugger.

The only time when I did not use a debugger was when I was writing
the debugger for lcc-win32. My debugger wasn't then able
to debug itself so I had to develop it without any help, what made
things considerably more difficult...

But if you find it useful, go for it.
Yes, I use them very often.

Jul 31 '06 #8

P: n/a
On 2006-07-31, Johan Tibell <jo**********@gmail.comwrote:
I've written a piece of code that uses sockets a lot (I know that
sockets aren't portable C, this is not a question about sockets per
se). Much of my code ended up looking like this:

if (function(socket, args) == -1) {
perror("function");
exit(EXIT_FAILURE);
}

I feel that the ifs destroy the readability of my code. Would it be
better to declare an int variable (say succ) and use the following
structure?

int succ;

succ = function(socket, args);
if (succ == -1) {
perror("function");
exit(EXIT_FAILURE);
}

What's considered "best practice" (feel free to substitute with: "what
do most good programmers use")?
It really depends on what your target is. In most of my code, I'm
thinking of removing all the checks for stuff like malloc(10) because
there is only a 0.02% chance that I'll be porting my code to a
non-desktop machine where memory is scarce.

OTOH, if you work for a company that deals in all sorts of systems, it
may be worthwhile to have a meaningful error message printed in the case
of problems.

It's not so much a matter of style as it is a matter of practibility:
your code must be robust, but it should also be easy to read. Nowadays
programs targeted at home computers or servers can assume that you'll
have a 99.99% success rate on a functioning system when allocating
memory < 1Kb.

--
Andrew Poelstra <website down>
To reach my email, use <email also down>
New server ETA: 5 minutes ago.
Aug 1 '06 #9

P: n/a
Andrew Poelstra <fa**********@wp.netwrites:
On 2006-07-31, Johan Tibell <jo**********@gmail.comwrote:
[...]
>What's considered "best practice" (feel free to substitute with: "what
do most good programmers use")?

It really depends on what your target is. In most of my code, I'm
thinking of removing all the checks for stuff like malloc(10) because
there is only a 0.02% chance that I'll be porting my code to a
non-desktop machine where memory is scarce.

OTOH, if you work for a company that deals in all sorts of systems, it
may be worthwhile to have a meaningful error message printed in the case
of problems.

It's not so much a matter of style as it is a matter of practibility:
your code must be robust, but it should also be easy to read. Nowadays
programs targeted at home computers or servers can assume that you'll
have a 99.99% success rate on a functioning system when allocating
memory < 1Kb.
Why would you want to settle for 99.99% when you can get 100%?

It takes only one failure to ruin your whole day.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Aug 1 '06 #10

P: n/a
On 2006-08-01, Keith Thompson <ks***@mib.orgwrote:
Andrew Poelstra <fa**********@wp.netwrites:
>On 2006-07-31, Johan Tibell <jo**********@gmail.comwrote:
[...]
>>What's considered "best practice" (feel free to substitute with: "what
do most good programmers use")?

It really depends on what your target is. In most of my code, I'm
thinking of removing all the checks for stuff like malloc(10) because
there is only a 0.02% chance that I'll be porting my code to a
non-desktop machine where memory is scarce.

OTOH, if you work for a company that deals in all sorts of systems, it
may be worthwhile to have a meaningful error message printed in the case
of problems.

It's not so much a matter of style as it is a matter of practibility:
your code must be robust, but it should also be easy to read. Nowadays
programs targeted at home computers or servers can assume that you'll
have a 99.99% success rate on a functioning system when allocating
memory < 1Kb.

Why would you want to settle for 99.99% when you can get 100%?

It takes only one failure to ruin your whole day.
Because that tiny percentage is the difference between
p = malloc (sizeof *p * q);

and
p = malloc (sizeof *p * q);
if (rv = !!p)
{
/* Rest of function here. */
}

Those ifs nest up and it becomes a pain to manage them. I prefer not to
have multiple returns because they annoy me; if I want to know what a
function returns, I like to skip to the end. If I see a "return 0" I
know that I either have a foolproof function, or I have to go and find
other return values elsewhere in the code. Better to have a
return rv; /* rv is 0 on success, or -1 on error (bad memory or file) */

Another idea would be to create a varadic function that checks multiple
memory allocations:
if (chk_mem (p, q, r, s, t) && chk_file (inf, outf))
{
/* Rest of function is happy. */
}

Hmm. I'm not sure whether to be sad that my readability argument is
falling apart, or happy that I'm learning better code design.
Also, I have an diagnostic library that will send errors to any stream
specified. So, if a user has a problem with a 12-byte allocation and the
rest of the system is for some reason working fine, I can tell him to
run myprog --debug and send me the output of myprog.log.

There's an extra step in there where he's emailing me a log, but
otherwise the communication would be the same; I'd have to tell
him to increase his swap space or to run fewer programs. (I'm
assuming here that a bad allocation will give me a nice GPF or
segfault; this isn't always true and is one of the most dangerous
assumptions I tend to make.)

--
Andrew Poelstra <website down>
To reach my email, use <email also down>
New server status: I've figured out how the case opens.
Aug 1 '06 #11

P: n/a
Andrew Poelstra <fa**********@wp.netwrites:
On 2006-08-01, Keith Thompson <ks***@mib.orgwrote:
>[...]
Why would you want to settle for 99.99% when you can get 100%?

It takes only one failure to ruin your whole day.

Because that tiny percentage is the difference between
p = malloc (sizeof *p * q);

and
p = malloc (sizeof *p * q);
if (rv = !!p)
{
/* Rest of function here. */
}

Those ifs nest up and it becomes a pain to manage them. I prefer not to
have multiple returns because they annoy me; if I want to know what a
function returns, I like to skip to the end. If I see a "return 0" I
know that I either have a foolproof function, or I have to go and find
other return values elsewhere in the code. Better to have a
return rv; /* rv is 0 on success, or -1 on error (bad memory or file) */
[...]

If you just abort the program on a malloc() failure, there's no reason
for the level of nesting to get out of hand.

p = malloc(sizeof *p * q);
if (p == NULL) {
fprintf(stderr, "malloc failed\n");
exit(EXIT_FAILURE);
}
/* Rest of function here */

It's not a *great* way to handle the error, but it's better than
ignoring it.

Or, as I suggested, write a wrapper:

void *my_malloc(size_t size)
{
void *result = malloc(size);
if (result == NULL) {
fprintf(stderr, "malloc failed\n");
exit(EXIT_FAILURE);
}
else {
return result;
}
}

....

p = my_malloc(sizeof *p * q);

The alternative is to ignore allocation failures and risk undefined
behavior.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Aug 1 '06 #12

P: n/a

Andrew Poelstra wrote:
On 2006-08-01, Keith Thompson <ks***@mib.orgwrote:
Andrew Poelstra <fa**********@wp.netwrites:
It really depends on what your target is. In most of my code, I'm
thinking of removing all the checks for stuff like malloc(10) because
there is only a 0.02% chance that I'll be porting my code to a
non-desktop machine where memory is scarce.
Why would you want to settle for 99.99% when you can get 100%?
>
Also, I have an diagnostic library that will send errors to any stream
specified. So, if a user has a problem with a 12-byte allocation and the
rest of the system is for some reason working fine, I can tell him to
run myprog --debug and send me the output of myprog.log.
This won't work at all. If I run your program and get a segfault
because of a memory error, when I run your diagnostic there's
a very good chance that the problem won't re-occur. It will
end up being an error that happens .01% of the time, but
never when I'm running the diagnostic tool! Pretty quickly,
I'll just decide that your code is unstable and I'll stop using
it. And if I ever look through the source and see unchecked
mallocs, I'll stop using it immediately! Putting a wrapper
around malloc ( "xmalloc" and "Malloc" are the two common
names I've seen, "my_malloc" has been mentioned in this
thread) that aborts on error is a much better approach, if
for no other reason than keeping your users happy.
At run time,I'd much rather see:
"Out of memory", than "Segmentation fault".

Aug 1 '06 #13

P: n/a

Frederick Gotham wrote:
Johan Tibell posted, w.r.t. checking return values:
What's considered "best practice" (feel free to substitute with: "what
do most good programmers use")?

Have you considered something like:

void Assure(int const i)
{
if(-1 == i)
{
perror("function");
exit(EXIT_FAILURE);
}
}
That's nice. How about:

#define Assure(x) if (!(x)) {fprintf(stderr, "In %s:%s " #x "
failed\n",\
__FILE__, __func__); exit(EXIT_FAILURE);}
>
And then putting something like the following in your code:

Assure( function(socket,args) );
Which allows slightly more flexibility:
Assure (function(a,b,c) == expected_value);

Aug 1 '06 #14

P: n/a
Andrew Poelstra said:

<snip>
It's not so much a matter of style as it is a matter of practibility:
your code must be robust, but it should also be easy to read.
Your code is not going to be robust if it doesn't check whether a request
for an external resource was successful. I agree it should be easy to read,
but that doesn't mean leaving the code out!
Nowadays
programs targeted at home computers or servers can assume that you'll
have a 99.99% success rate on a functioning system when allocating
memory < 1Kb.
Programmers who make such an assumption should not be writing for the home
market or the server market. They should be writing in crayon on droolproof
paper.

--
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at above domain (but drop the www, obviously)
Aug 1 '06 #15

P: n/a
Johan Tibell wrote:
I've written a piece of code that uses sockets a lot (I know that
[...]
What's considered "best practice" (feel free to substitute with: "what
do most good programmers use")?

Following style makes it clear from reading point of view,

/********/
#define RET_CODE_SUCCESS 1
#define RET_CODE_FAILURE -1

....

if (RET_CODE_FAILURE == function(socket, args);) {
perror("function");
exit(EXIT_FAILURE);
}
/********/

As far as storing the return value in a variable is concerned, if you
are not going to use it, there is no need to store it. If you want to
check for the returned value during debugging, you can step-in to that
function.

Aug 1 '06 #16

P: n/a

Andrew Poelstra wrote:
On 2006-08-01, Keith Thompson <ks***@mib.orgwrote:
Andrew Poelstra <fa**********@wp.netwrites:
On 2006-07-31, Johan Tibell <jo**********@gmail.comwrote:
[...]
>
Because that tiny percentage is the difference between
p = malloc (sizeof *p * q);

and
p = malloc (sizeof *p * q);
if (rv = !!p)
{
/* Rest of function here. */
}

Those ifs nest up and it becomes a pain to manage them.
Your customer pulling his hair on a segmentation fault while doing an
intense calculation or running a major server is even a bigger pain,
for him and for your repute.

Another idea would be to create a varadic function that checks multiple
memory allocations:
if (chk_mem (p, q, r, s, t) && chk_file (inf, outf))
{
/* Rest of function is happy. */
}
What if you want to do some thing if memory allocation fails at one
point and some thing else if it fails at some other point?

Hmm. I'm not sure whether to be sad that my readability argument is
falling apart, or happy that I'm learning better code design.
Better code design and readability, IMHO, complement each other.

There's an extra step in there where he's emailing me a log, but
otherwise the communication would be the same; I'd have to tell
him to increase his swap space or to run fewer programs.
One of the worst pieces of advice I have ever heard. Do you work for,
ahem, M$?

Aug 1 '06 #17

P: n/a
On 2006-08-01, Richard Heathfield <in*****@invalid.invalidwrote:
Andrew Poelstra said:
>It's not so much a matter of style as it is a matter of practibility:
your code must be robust, but it should also be easy to read.

Your code is not going to be robust if it doesn't check whether a request
for an external resource was successful. I agree it should be easy to read,
but that doesn't mean leaving the code out!
What exactly /would/ be the way to do such a thing? I ask you because
you don't like multiple returns or the break statement, both of which
would be a typical response.
>Nowadays
programs targeted at home computers or servers can assume that you'll
have a 99.99% success rate on a functioning system when allocating
memory < 1Kb.

Programmers who make such an assumption should not be writing for the home
market or the server market. They should be writing in crayon on droolproof
paper.
Being as every other post was pretty much exactly as insulting as this,
I'd say that I wasn't not wrong on any minor point! I'm glad that I haven't
had the chance to make these foolhardy changes to my actual code yet.

I've written a new interface to my error library so that it will be able
to handle memory failures gracefully, log to a runtime-determined file,
check for bad files or memory, and ensure that a proper message reaches
the user if it can't go on.

--
Andrew Poelstra <website down>
To reach my email, use <email also down>
New server time-to-be-up: Today
Aug 1 '06 #18

P: n/a
On 2006-08-01, Andrew Poelstra <fa**********@wp.netwrote:
I wasn't not wrong on any minor point!
Seems it's not just the 2AM folks getting their "not"s mixed up! I meant
"I wasn't wrong on some minor point!"

--
Andrew Poelstra <website down>
To reach my email, use <email also down>
New server ETA: 42
Aug 1 '06 #19

P: n/a
On Tue, 01 Aug 2006 00:19:00 GMT, Andrew Poelstra
<fa**********@wp.netwrote:
>It's not so much a matter of style as it is a matter of practibility:
your code must be robust, but it should also be easy to read. Nowadays
programs targeted at home computers or servers can assume that you'll
have a 99.99% success rate on a functioning system when allocating
memory < 1Kb.
You've forgotten the rule of computing probability - if there's one
chance in a million, it happens every second.

You must have written some of the code for Windows.

--
Al Balmer
Sun City, AZ
Aug 1 '06 #20

P: n/a
Johan Tibell (in 11*********************@s13g2000cwa.googlegroups.c om)
said:

| I've written a piece of code that uses sockets a lot (I know that
| sockets aren't portable C, this is not a question about sockets per
| se). Much of my code ended up looking like this:

<"succeed or die" examples snipped>

| What's considered "best practice" (feel free to substitute with:
| "what do most good programmers use")?

IMO, "best practice" is to detect all detectable errors and recover
from all recoverable errors - and to provide a clear explanation
(however terse) of non-recoverable errors.

I have difficulty imagining that any "good programmer" would ignore
errors and/or fail to provide recovery from recoverable errors in
production software.

--
Morris Dovey
DeSoto Solar
DeSoto, Iowa USA
http://www.iedu.com/DeSoto
Aug 1 '06 #21

P: n/a
Andrew Poelstra said:
On 2006-08-01, Richard Heathfield <in*****@invalid.invalidwrote:
>Andrew Poelstra said:
>>It's not so much a matter of style as it is a matter of practibility:
your code must be robust, but it should also be easy to read.

Your code is not going to be robust if it doesn't check whether a request
for an external resource was successful. I agree it should be easy to
read, but that doesn't mean leaving the code out!

What exactly /would/ be the way to do such a thing? I ask you because
you don't like multiple returns or the break statement, both of which
would be a typical response.
Well, the check is done in the obvious way:

p = malloc(n * sizeof *p);
if(p == NULL)
{

The response to a failure depends on the situation. I've covered this in
some detail in my one-and-only contribution to "the literature", so I'll
just bullet-point some possible responses here:

* abort the program. The "student solution" - suitable only for high school
students and, perhaps, example programs (with a big red warning flag).
* break down the memory requirement into two or more sub-blocks.
* use less memory!
* point to a fixed-length buffer instead (and remember not to free it!)
* allocate an emergency reserve at the beginning of the program
* use virtual memory on another machine networked to this one (this
is a lot of work, but it may be worth it on super-huge projects)
>>Nowadays
programs targeted at home computers or servers can assume that you'll
have a 99.99% success rate on a functioning system when allocating
memory < 1Kb.

Programmers who make such an assumption should not be writing for the
home market or the server market. They should be writing in crayon on
droolproof paper.

Being as every other post was pretty much exactly as insulting as this,
Well, I wasn't really trying to insult you. I was just trying to communicate
my concern at the prevalence of this really bad practice of not checking
that an external resource acquisition attempt succeeded. You wouldn't omit
a check on fopen or socket or connect, so why on malloc?
I'd say that I wasn't not wrong on any minor point! I'm glad that I
haven't had the chance to make these foolhardy changes to my actual code
yet.
So am I. :-)

--
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at above domain (but drop the www, obviously)
Aug 1 '06 #22

P: n/a
Morris Dovey wrote:
IMO, "best practice" is to detect all detectable errors and recover
from all recoverable errors - and to provide a clear explanation
(however terse) of non-recoverable errors.

I have difficulty imagining that any "good programmer" would ignore
errors and/or fail to provide recovery from recoverable errors in
production software.
The topic of this post is not whether to check errors or not. Read the
original message.

Aug 1 '06 #23

P: n/a
In article <ln************@nuthaus.mib.org>,
Keith Thompson <ks***@mib.orgwrote:
>Nowadays
programs targeted at home computers or servers can assume that you'll
have a 99.99% success rate on a functioning system when allocating
memory < 1Kb.
>Why would you want to settle for 99.99% when you can get 100%?
Of course, as noted in previous threads in this newsgroup, checking
malloc() results will only increase your success rate from 99.99% to
99.991% on the many modern operating systems which overcommit memory.

On the other hand, if someone uses your code in a situation where it
Really Matters that it fail cleanly, they will presumably take steps
to ensure that malloc() doesn't overcommit.

-- Richard
Aug 1 '06 #24

P: n/a
jacob navia wrote:
Keith Thompson a écrit :
>Personally, I don't use debuggers very often, so it usually wouldn't
occur to me to distort my code to make it easier to use in a debugger.

Ahhh... You do not use debuggers very often?

Mmmm... Well, they are great tools. I use them very often,
actually I spend most of the time either in the editor
or in the debugger.
That's a common but by no means universal approach to coding. Personally, I
find that coding as if my system had no debuggers results in code for which
I don't need a debugger, and this is why I don't use debuggers often.

I think that this style of coding is overall more speedy than the one where
you do assume debuggers are readily available for assistance, but I have no
hard data on it, and doubtlessly it will vary by individual.
The only time when I did not use a debugger was when I was writing
the debugger for lcc-win32. My debugger wasn't then able
to debug itself so I had to develop it without any help, what made
things considerably more difficult...
Well, you could have used a different debugger... Or, like compilers
bootstrapping themselves, you could have used a rudimentary but easily
eye-proofed version of your debugger to develop the rest with.

S.
Aug 1 '06 #25

P: n/a
Skarmander wrote:
>The only time when I did not use a debugger was when I was writing
the debugger for lcc-win32. My debugger wasn't then able
to debug itself so I had to develop it without any help, what made
things considerably more difficult...
Well, you could have used a different debugger... Or, like compilers
bootstrapping themselves, you could have used a rudimentary but easily
eye-proofed version of your debugger to develop the rest with.
To use a different debugger I would have to be sure that the
compiler generated the right debug information, that the linker
linked (and compressed it) OK.

That was the last and most difficult task I did for my system,
and it really was a lot of pain because at the start I had only
printf as a tool. Any bug in the compiler, assembler or linker
in the handling of the debug information would appear there!

After I got the debugger running more or less I started using
the Microsoft debugger that to my amazement did manage to
read through the sometimes buggy debug information without
crashing!

THEN it was much easier and I could start caring about user interface,
etc.

Now my debugger is able to debug itself, and I could bootstrap
the 64 bit version with much less effort.

but we are steering off topic and I fear the regulars coming :-)
Aug 1 '06 #26

P: n/a
On 2006-08-01, Al Balmer <al******@att.netwrote:
On Tue, 01 Aug 2006 00:19:00 GMT, Andrew Poelstra
<fa**********@wp.netwrote:
>>It's not so much a matter of style as it is a matter of practibility:
your code must be robust, but it should also be easy to read. Nowadays
programs targeted at home computers or servers can assume that you'll
have a 99.99% success rate on a functioning system when allocating
memory < 1Kb.

You've forgotten the rule of computing probability - if there's one
chance in a million, it happens every second.

You must have written some of the code for Windows.
I made a typo in a malloc call and the thing BSOD'd. Haven't done any
real programming in Windows since.

It's nice not working for an corporation that imposes systems on you.
I'm pretty sure that I should enjoy this while I can.

(As to the original post, I've mentioned that I am improving my
diagnostic/error handling library to handle malloc() failures
predictably.)

--
Andrew Poelstra <website down>
To reach my email, use <email also down>
New server ETA: 42
Aug 1 '06 #27

P: n/a
jacob navia wrote:
but we are steering off topic and I fear the regulars coming :-)
They're always there, and always watching us; hence, no need to be paranoid.

S.
Aug 1 '06 #28

P: n/a
jacob navia wrote:
Keith Thompson a écrit :
>Personally, I don't use debuggers very often, so it usually wouldn't
occur to me to distort my code to make it easier to use in a debugger.


Ahhh... You do not use debuggers very often?

Mmmm... Well, they are great tools. I use them very often,
actually I spend most of the time either in the editor
or in the debugger.
Try a different style of development. If you get used to working test
first, you will find yourself using the debugger less and less. You
write a failing test, then write the code to pass it. If everything
passes, move on. If something breaks, back out your change a try
something else.

--
Ian Collins.
Aug 1 '06 #29

P: n/a
Richard Heathfield wrote:
Andrew Poelstra said:
>On 2006-08-01, Richard Heathfield <in*****@invalid.invalidwrote:
>>Andrew Poelstra said:
It's not so much a matter of style as it is a matter of practibility:
your code must be robust, but it should also be easy to read.
Your code is not going to be robust if it doesn't check whether a request
for an external resource was successful. I agree it should be easy to
read, but that doesn't mean leaving the code out!
What exactly /would/ be the way to do such a thing? I ask you because
you don't like multiple returns or the break statement, both of which
would be a typical response.

Well, the check is done in the obvious way:

p = malloc(n * sizeof *p);
if(p == NULL)
{

The response to a failure depends on the situation. I've covered this in
some detail in my one-and-only contribution to "the literature",
I take it you're referring to "C Unleashed"; I haven't read it and am not
currently in a position to read it, so I hope you'll excuse me if I clumsily
raise points you discuss in depth in the book.
so I'll just bullet-point some possible responses here:

* abort the program. The "student solution" - suitable only for high school
students and, perhaps, example programs (with a big red warning flag).
This is not doing justice to the significant amount of work required to make
a program robust in the face of memory exhaustion. Quite bluntly: it's not
always worth it in terms of development time versus worst possible outcome
and likelihood of that outcome, even for programs used outside high school.

I'm not saying the tradeoffs involved are always correctly assessed (it's
probably a given that the cost of failure is usually underestimated), but I
do believe they exist.

I presume that instead of "aborting the program" we may read "exiting the
program immediately but as cleanly as possible", by the way, with the latter
being just that bit more desirable.
* break down the memory requirement into two or more sub-blocks.
Applicable only if the failure is a result of trying to allocate more memory
than you really need in one transaction, which is either a flaw or an
inappropriate optimization (or both, depending on your point of view).
* use less memory!
Will solve the problem, in the sense that "don't do that then" will cure any
pain you may experience while moving your arm. The bit we're interested in
is when you've decided that you absolutely have to move your arm.
* point to a fixed-length buffer instead (and remember not to free it!)
Thread unsafe (unavoidably breaches modularity by aliasing a global, if you
like it more general), increased potential for buffer overflow, requires
checking for an exceptional construct in the corresponding free() wrapper
(I'm assuming we'd wrap this).

I don't see how this would ever be preferential to your next solution:
* allocate an emergency reserve at the beginning of the program
How big of an emergency reserve, though? To make this work, your biggest
allocation should never exceed the emergency reserve (this could be tedious
to enforce, but is doable) and it will only allow you to complete whatever
you're doing right now.

Of all the solutions suggested, though, I'd say this one has the most
potential for practical success, *if* we signal the OOM condition *in
addition to* returning emergency reserve storage. At the end of an
individual transaction, we can detect that the next one will fail if no more
memory has become available, so we can take any measures we can take
(including failing gracefully) with the program in a well-known state (we
can do the same for a fail-fast allocation like malloc() offers, but it
requires more effort, and may involve rewriting things we can't rewrite).
* use virtual memory on another machine networked to this one (this
is a lot of work, but it may be worth it on super-huge projects)
This is not really an answer to "how do I deal with memory allocation
failure in a program", but to "how do I make sure my program doesn't
encounter memory allocation failure". For such projects as you mention
you'll probably want a custom memory allocation library to begin with (which
may or may not involve the standard malloc() at all, depending on the
portability/performance tradeoffs involved).
>>>Nowadays
programs targeted at home computers or servers can assume that you'll
have a 99.99% success rate on a functioning system when allocating
memory < 1Kb.
Programmers who make such an assumption should not be writing for the
home market or the server market. They should be writing in crayon on
droolproof paper.
Being as every other post was pretty much exactly as insulting as this,

Well, I wasn't really trying to insult you. I was just trying to communicate
my concern at the prevalence of this really bad practice of not checking
that an external resource acquisition attempt succeeded. You wouldn't omit
a check on fopen or socket or connect, so why on malloc?
Playing devil's advocate for a moment: because the failures you cite are
both more common and easier to recover from, if they are recoverable at all,
so checking for them is more worthwhile.

This doesn't actually justify omitting a *check* on failure, of course,
since even dying with an error in a well-defined way is better than invoking
undefined behavior. The cost of checking is so insignificant that, if you
really need to get your savings there, you're probably doing memory
allocation wrong to begin with.

S.
Aug 1 '06 #30

P: n/a
Skarmander said:
Richard Heathfield wrote:
<snip>
>>
The response to a failure depends on the situation. I've covered this in
some detail in my one-and-only contribution to "the literature",

I take it you're referring to "C Unleashed"; I haven't read it and am not
currently in a position to read it, so I hope you'll excuse me if I
clumsily raise points you discuss in depth in the book.
>so I'll just bullet-point some possible responses here:

* abort the program. The "student solution" - suitable only for high
school
students and, perhaps, example programs (with a big red warning flag).

This is not doing justice to the significant amount of work required to
make a program robust in the face of memory exhaustion. Quite bluntly:
it's not always worth it in terms of development time versus worst
possible outcome and likelihood of that outcome, even for programs used
outside high school.
Perhaps I should have made it clearer that I'm referring to programs that
are intended to be used over and over and over by lotsa lotsa people. If
it's not worth the programmer's time to write the code robustly, it's not
worth my time to use his program. Of course, it may be worth /his/ time to
use his own program.
I'm not saying the tradeoffs involved are always correctly assessed (it's
probably a given that the cost of failure is usually underestimated), but
I do believe they exist.
Yes, the cost of failure can be a lost customer, repeated N times.
I presume that instead of "aborting the program" we may read "exiting the
program immediately but as cleanly as possible", by the way, with the
latter being just that bit more desirable.
You may indeed. I was using "abort" in a more general sense, not the
technical C sense (std lib function).
>* break down the memory requirement into two or more sub-blocks.

Applicable only if the failure is a result of trying to allocate more
memory than you really need in one transaction, which is either a flaw or
an inappropriate optimization (or both, depending on your point of view).
No, you've misunderstood. I'm thinking about situations where you would
ideally like a contiguous block of N bytes, but would be able to manage
with B blocks of (N + B - 1) / B bytes (or, more generally, a bunch of
blocks that between them total N bytes). The allocator might find that
easier to manage.
>* use less memory!

Will solve the problem, in the sense that "don't do that then" will cure
any pain you may experience while moving your arm. The bit we're
interested in is when you've decided that you absolutely have to move your
arm.
Consider reading a complete line into memory. It's good practice to extend
the buffer by a multiple (typically 1.5 or 2) whenever you run out of RAM.
So - let's say you've got a buffer of 8192 bytes, and you've filled it but
you still haven't finished reading the line. So you try to realloc to
16384, but realloc says no. Well, nothing in the rules says you're
necessarily going to need all that extra RAM, so it might be worth trying
to realloc to 8192 + 4096 = 12288 or something like that instead.
>* point to a fixed-length buffer instead (and remember not to free it!)

Thread unsafe (unavoidably breaches modularity by aliasing a global, if
you like it more general), increased potential for buffer overflow,
requires checking for an exceptional construct in the corresponding free()
wrapper (I'm assuming we'd wrap this).
Who says it has to be global? It would never have occurred to me to use a
file scope object for such a purpose.

I don't see how this would ever be preferential to your next solution:
>* allocate an emergency reserve at the beginning of the program

How big of an emergency reserve, though? To make this work, your biggest
allocation should never exceed the emergency reserve (this could be
tedious to enforce, but is doable) and it will only allow you to complete
whatever you're doing right now.
Yes, but that might be enough to get the user's data to a point where it can
be saved, and reloaded later in a more memory-rich environment.

<snip>
>* use virtual memory on another machine networked to this one (this
is a lot of work, but it may be worth it on super-huge projects)
This is not really an answer to "how do I deal with memory allocation
failure in a program", but to "how do I make sure my program doesn't
encounter memory allocation failure".
Well, that would be nice, but I presume that most programmers would rather
have their RAM nice and local, where they can get at it quickly and easily.
It's a port in a storm, not a general purpose allocation strategy.

<snip>

--
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at above domain (but drop the www, obviously)
Aug 1 '06 #31

P: n/a
"Johan Tibell" <jo**********@gmail.comwrites:
Morris Dovey wrote:
>IMO, "best practice" is to detect all detectable errors and recover
from all recoverable errors - and to provide a clear explanation
(however terse) of non-recoverable errors.

I have difficulty imagining that any "good programmer" would ignore
errors and/or fail to provide recovery from recoverable errors in
production software.

The topic of this post is not whether to check errors or not.
It is now.
Read the original message.
We've moved on.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Aug 1 '06 #32

P: n/a
Richard Heathfield <in*****@invalid.invalidwrites:
[...]
The response to a failure depends on the situation. I've covered this in
some detail in my one-and-only contribution to "the literature", so I'll
just bullet-point some possible responses here:

* abort the program. The "student solution" - suitable only for high school
students and, perhaps, example programs (with a big red warning flag).
* break down the memory requirement into two or more sub-blocks.
* use less memory!
* point to a fixed-length buffer instead (and remember not to free it!)
* allocate an emergency reserve at the beginning of the program
* use virtual memory on another machine networked to this one (this
is a lot of work, but it may be worth it on super-huge projects)
[...]

Out of curiosity, how often do real-world programs really do something
fancy in response to a malloc() failure?

The simplest solution, as you say, is to immediately abort the program
(which is far better than ignoring the error). The next simplest
solution is to do some cleanup (print a coherent error message, flush
buffers, close files, release resources, log the error, etc.) and
*then* abort the program.

I've seen suggestions that, if a malloc() call fails, the program can
fall back to an alternative algorithm that uses less memory. How
realistic is this? If there's an algorithm that uses less memory, why
not use it in the first place? (The obvious answer: because it's
slower.) Do programmers really go to the effort of implementing two
separate algorithms, one of which will be used only on a memory
failure (and will therefore not be tested as thoroughly as the primary
algorithm)?

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Aug 1 '06 #33

P: n/a
Keith Thompson said:

<snip>
>
Out of curiosity, how often do real-world programs really do something
fancy in response to a malloc() failure?
Six times, for sufficiently variable values of six.
The simplest solution, as you say, is to immediately abort the program
(which is far better than ignoring the error). The next simplest
solution is to do some cleanup (print a coherent error message, flush
buffers, close files, release resources, log the error, etc.) and
*then* abort the program.
In other words, get out gracefully, preferably without losing any user data.
But that may involve completing the current task for which you wanted
memory in the first place.
I've seen suggestions that, if a malloc() call fails, the program can
fall back to an alternative algorithm that uses less memory. How
realistic is this?
I've had to do it myself in "real" code.
If there's an algorithm that uses less memory, why
not use it in the first place?
The answer is embarrassingly obvious.
(The obvious answer: because it's slower.)
Quite so.
Do programmers really go to the effort of implementing two
separate algorithms, one of which will be used only on a memory
failure
Yes, sometimes, if the situation warrants it. Often, it won't, but often !=
always. Normally, the goal is to fail gracefully without losing user data.
But some programs Must Not Fail - e.g. safety-critical stuff.
(and will therefore not be tested as thoroughly as the primary
algorithm)?
It needs to be tested thoroughly. The primary algorithm will be "tested"
more thoroughly in the sense that it's used more often and so there's a
greater likelihood that bugs will be exposed, but that's true of a lot of
code - for example, an OS's "copy a file" operation is tested more
thoroughly than its "format a disk drive" operation, simply by virtue of
the fact that users are more likely to /use/ the file copy, and every use
is a test. That doesn't mean the format code didn't get an appropriate
level of testing.

--
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at above domain (but drop the www, obviously)
Aug 1 '06 #34

P: n/a
Richard Heathfield wrote:
Skarmander said:
>Richard Heathfield wrote:
<snip>
>>The response to a failure depends on the situation. I've covered this in
some detail in my one-and-only contribution to "the literature",
I take it you're referring to "C Unleashed"; I haven't read it and am not
currently in a position to read it, so I hope you'll excuse me if I
clumsily raise points you discuss in depth in the book.
>>so I'll just bullet-point some possible responses here:

* abort the program. The "student solution" - suitable only for high
school
students and, perhaps, example programs (with a big red warning flag).
This is not doing justice to the significant amount of work required to
make a program robust in the face of memory exhaustion. Quite bluntly:
it's not always worth it in terms of development time versus worst
possible outcome and likelihood of that outcome, even for programs used
outside high school.

Perhaps I should have made it clearer that I'm referring to programs that
are intended to be used over and over and over by lotsa lotsa people. If
it's not worth the programmer's time to write the code robustly, it's not
worth my time to use his program. Of course, it may be worth /his/ time to
use his own program.
You're using "robust" as if it's a yes or no property. You're free to storm
off in a huff shouting "well that's not robust then!" if someone presents
you with a program that can deal with just about anything except memory
exhaustion when it finally comes round the bend, but that doesn't mean
everyone would or should.
>I'm not saying the tradeoffs involved are always correctly assessed (it's
probably a given that the cost of failure is usually underestimated), but
I do believe they exist.

Yes, the cost of failure can be a lost customer, repeated N times.
Don't sell it short. It could be a DEAD CUSTOMER, repeated N times.

Or it could be a single entry in a log somewhere.
>>* break down the memory requirement into two or more sub-blocks.
Applicable only if the failure is a result of trying to allocate more
memory than you really need in one transaction, which is either a flaw or
an inappropriate optimization (or both, depending on your point of view).

No, you've misunderstood. I'm thinking about situations where you would
ideally like a contiguous block of N bytes, but would be able to manage
with B blocks of (N + B - 1) / B bytes (or, more generally, a bunch of
blocks that between them total N bytes). The allocator might find that
easier to manage.
I see what you're getting at; the allocator may be unable to satisfy your
request due to fragmentation and the like. This doesn't work very well as a
strategy to recover, however; you're looking at writing a program that
allocates either a contiguous region of bytes or (if that should happen not
to work) finds some way to deal with a bunch of regions.

If you need to be able to handle the latter, though, you'll write code that
deals with a bunch of regions in the first place, with the amount of regions
possibly equal to 1. (You may choose to split these cases, but it's unlikely
to become faster or more maintainable.)

Of course, then you need to settle on some sort of minimum region size
you're willing to accept, somewhere between the absolute lower bound of the
minimum allocator overhead and the minimum meaningful region for your
application.

All this will not make the program more *robust*, however, it'll simply
allow it to make better use of existing resources. ("Simply" should
nevertheless not be taken lightly.) Robustness is given by how well the
system behaves when the resources run dry, not how well it squeezes water
from the remaining stones when there's a drought in sight.
>>* use less memory!
Will solve the problem, in the sense that "don't do that then" will cure
any pain you may experience while moving your arm. The bit we're
interested in is when you've decided that you absolutely have to move your
arm.

Consider reading a complete line into memory. It's good practice to extend
the buffer by a multiple (typically 1.5 or 2) whenever you run out of RAM.
So - let's say you've got a buffer of 8192 bytes, and you've filled it but
you still haven't finished reading the line. So you try to realloc to
16384, but realloc says no. Well, nothing in the rules says you're
necessarily going to need all that extra RAM, so it might be worth trying
to realloc to 8192 + 4096 = 12288 or something like that instead.
Thanks, that explanation was a bit more meaningful than "use less memory". :-)

This is a good amendment to a standard exponential allocation strategy.
>>* point to a fixed-length buffer instead (and remember not to free it!)
Thread unsafe (unavoidably breaches modularity by aliasing a global, if
you like it more general), increased potential for buffer overflow,
requires checking for an exceptional construct in the corresponding free()
wrapper (I'm assuming we'd wrap this).

Who says it has to be global? It would never have occurred to me to use a
file scope object for such a purpose.
So N fixed-length buffers, then? Obviously N can't be dynamic, or you're
back to square one.
>I don't see how this would ever be preferential to your next solution:
>>* allocate an emergency reserve at the beginning of the program
How big of an emergency reserve, though? To make this work, your biggest
allocation should never exceed the emergency reserve (this could be
tedious to enforce, but is doable) and it will only allow you to complete
whatever you're doing right now.

Yes, but that might be enough to get the user's data to a point where it can
be saved, and reloaded later in a more memory-rich environment.
Yes, as I said, of all your solutions, this one comes closest to actually
making things more robust in the face of imminent failure. The other items,
while helpful, delay the inevitable. Valuable as that is, I'm more
interested in what to do when the inevitable comes around, as it inevitably
will.
<snip>
>>* use virtual memory on another machine networked to this one (this
is a lot of work, but it may be worth it on super-huge projects)
This is not really an answer to "how do I deal with memory allocation
failure in a program", but to "how do I make sure my program doesn't
encounter memory allocation failure".

Well, that would be nice, but I presume that most programmers would rather
have their RAM nice and local, where they can get at it quickly and easily.
It's a port in a storm, not a general purpose allocation strategy.
A strange sort of port, when you need to first build a ship, then tow it
across the sea to its destination.

It's a "you might consider it" thing that seems very problem-specific to me;
"link computers together so you have more resources" is certainly an
approach, but not one I'd expect to see as a recommendation for making
programs more robust. That's not to say the suggestion itself isn't
valuable, of course.

S.
Aug 1 '06 #35

P: n/a
Richard Heathfield <in*****@invalid.invalidwrites:
Keith Thompson said:
[...]
>Do programmers really go to the effort of implementing two
separate algorithms, one of which will be used only on a memory
failure

Yes, sometimes, if the situation warrants it. Often, it won't, but often !=
always. Normally, the goal is to fail gracefully without losing user data.
But some programs Must Not Fail - e.g. safety-critical stuff.
In that context, I would think it would *usually* make more sense to
use just the slower and more robust algorithm in the first place.

Possibly the faster and more memory-intensive method would be
necessary to meet deadlines, and if you run out of memory you can fall
back to the slower method and continue running in a degraded mode.
But then again, safety-critical real-time code tends not to use
dynamic memory allocation at all.

I think that, 99% of the time, the best response to an allocation
failure is to clean up and abort. (The tendency to omit the "clean
up" portion is regrettable; the tendency to omit the "abort" portion,
i.e., to ignore the error, is even more so.) That other 1% requires
the other 99% of the effort.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Aug 1 '06 #36

P: n/a
Keith Thompson <ks***@mib.orgwrites:
Do programmers really go to the effort of implementing two
separate algorithms, one of which will be used only on a memory
failure (and will therefore not be tested as thoroughly as the
primary algorithm)?
Only occasionally, in my experience. Sometimes I do this if I
don't want to introduce an error path in a stack of function
calls that doesn't have one and which shouldn't have one.

One example is a merge sort (that requires O(n) extra storage)
that falls back to another sort algorithm if storage is not
available.
--
"...deficient support can be a virtue.
It keeps the amateurs off."
--Bjarne Stroustrup
Aug 1 '06 #37

P: n/a
On Tue, 01 Aug 2006 22:27:24 GMT, Keith Thompson <ks***@mib.org>
wrote:
>Richard Heathfield <in*****@invalid.invalidwrites:
>Keith Thompson said:
[...]
>>Do programmers really go to the effort of implementing two
separate algorithms, one of which will be used only on a memory
failure

Yes, sometimes, if the situation warrants it. Often, it won't, but often !=
always. Normally, the goal is to fail gracefully without losing user data.
But some programs Must Not Fail - e.g. safety-critical stuff.

In that context, I would think it would *usually* make more sense to
use just the slower and more robust algorithm in the first place.

Possibly the faster and more memory-intensive method would be
necessary to meet deadlines, and if you run out of memory you can fall
back to the slower method and continue running in a degraded mode.
But then again, safety-critical real-time code tends not to use
dynamic memory allocation at all.

I think that, 99% of the time, the best response to an allocation
failure is to clean up and abort. (The tendency to omit the "clean
up" portion is regrettable; the tendency to omit the "abort" portion,
i.e., to ignore the error, is even more so.) That other 1% requires
the other 99% of the effort.
I've worked on lots of safety-critical (and money-critical) systems,
and I'd emphasize the word "system." The program is only a part of the
system, albeit a very important part. Sometimes the best thing the
program can do is get out of the way and let the failsafes take over.
Sometimes, in fact, the computer itself gets out of the way, for
example if it detects an unrecoverable memory fault and can no longer
depend on it's own calculations.

This is a big subject, and 99% off-topic <G>.

--
Al Balmer
Sun City, AZ
Aug 1 '06 #38

P: n/a
Skarmander wrote:
jacob navia wrote:
>Keith Thompson a écrit :
>>Personally, I don't use debuggers very often, so it usually wouldn't
occur to me to distort my code to make it easier to use in a debugger.


Ahhh... You do not use debuggers very often?

Mmmm... Well, they are great tools. I use them very often,
actually I spend most of the time either in the editor
or in the debugger.
That's a common but by no means universal approach to coding.
Personally, I find that coding as if my system had no debuggers results
in code for which I don't need a debugger, and this is why I don't use
debuggers often.

I think that this style of coding is overall more speedy than the one
where you do assume debuggers are readily available for assistance, but
I have no hard data on it, and doubtlessly it will vary by individual.
>The only time when I did not use a debugger was when I was writing
the debugger for lcc-win32. My debugger wasn't then able
to debug itself so I had to develop it without any help, what made
things considerably more difficult...
Well, you could have used a different debugger... Or, like compilers
bootstrapping themselves, you could have used a rudimentary but easily
eye-proofed version of your debugger to develop the rest with.

S.
The only time I used a debugger in 35 years of
programming, was when an assembler decoding
subroutine failed to cross a 64K barrier properly.
(I did not write that one.).
To me, it smells like "let the system catch my errors"
while you should try to avoid them in the first place.
Sloppy typing can cause errors which are not
found by your compiler/debugger.
Aug 1 '06 #39

P: n/a
Skarmander said:
Richard Heathfield wrote:
<snip>
>>
Perhaps I should have made it clearer that I'm referring to programs that
are intended to be used over and over and over by lotsa lotsa people. If
it's not worth the programmer's time to write the code robustly, it's not
worth my time to use his program. Of course, it may be worth /his/ time
to use his own program.
You're using "robust" as if it's a yes or no property. You're free to
storm off in a huff shouting "well that's not robust then!" if someone
presents you with a program that can deal with just about anything except
memory exhaustion when it finally comes round the bend, but that doesn't
mean everyone would or should.
<grinNo, there's no harumphing going on over here. But basic resource
acquisition checking is as fundamental to robustness as steering so as not
to bump the kerb is to driving.
>>I'm not saying the tradeoffs involved are always correctly assessed
(it's probably a given that the cost of failure is usually
underestimated), but I do believe they exist.

Yes, the cost of failure can be a lost customer, repeated N times.
Don't sell it short. It could be a DEAD CUSTOMER, repeated N times.
Absolutely. And a dead customer is a lost customer, right?
Or it could be a single entry in a log somewhere.
Or not even that. We're back to undefined behaviour.

>>>* break down the memory requirement into two or more sub-blocks.
Applicable only if the failure is a result of trying to allocate more
memory than you really need in one transaction, which is either a flaw
or an inappropriate optimization (or both, depending on your point of
view).

No, you've misunderstood. I'm thinking about situations where you would
ideally like a contiguous block of N bytes, but would be able to manage
with B blocks of (N + B - 1) / B bytes (or, more generally, a bunch of
blocks that between them total N bytes). The allocator might find that
easier to manage.
I see what you're getting at; the allocator may be unable to satisfy your
request due to fragmentation and the like. This doesn't work very well as
a strategy to recover, however; you're looking at writing a program that
allocates either a contiguous region of bytes or (if that should happen
not to work) finds some way to deal with a bunch of regions.
Done it. It wasn't pretty, but it is certainly possible sometimes. Not all
the time, I grant you. (If there were one-size-fits-all, we'd all know
about it and probably many of us would be using it.)

<snip>
All this will not make the program more *robust*, however, it'll simply
allow it to make better use of existing resources. ("Simply" should
nevertheless not be taken lightly.) Robustness is given by how well the
system behaves when the resources run dry, not how well it squeezes water
from the remaining stones when there's a drought in sight.
If the objective is to gather enough moisture to survive until the rescue
helicopter arrives - that is, if the objective is to complete the immediate
task so that a consistent set of user data can be saved before you bomb out
- then it's a sensible approach.

<snip>
>Consider reading a complete line into memory. It's good practice to
extend the buffer by a multiple (typically 1.5 or 2) whenever you run out
of RAM. So - let's say you've got a buffer of 8192 bytes, and you've
filled it but you still haven't finished reading the line. So you try to
realloc to 16384, but realloc says no. Well, nothing in the rules says
you're necessarily going to need all that extra RAM, so it might be worth
trying to realloc to 8192 + 4096 = 12288 or something like that instead.
Thanks, that explanation was a bit more meaningful than "use less memory".
:-)
I was trying not to rewrite the book, okay? :-)
>>>* point to a fixed-length buffer instead (and remember not to free it!)
Thread unsafe (unavoidably breaches modularity by aliasing a global, if
you like it more general), increased potential for buffer overflow,
requires checking for an exceptional construct in the corresponding
free() wrapper (I'm assuming we'd wrap this).

Who says it has to be global? It would never have occurred to me to use a
file scope object for such a purpose.
So N fixed-length buffers, then? Obviously N can't be dynamic, or you're
back to square one.
I was assuming just the one, actually, local to the function where the
storage is needed.

<snip>

--
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at above domain (but drop the www, obviously)
Aug 2 '06 #40

P: n/a

Keith Thompson wrote:
"Johan Tibell" <jo**********@gmail.comwrites:
Morris Dovey wrote:
IMO, "best practice" is to detect all detectable errors and recover
from all recoverable errors - and to provide a clear explanation
(however terse) of non-recoverable errors.

I have difficulty imagining that any "good programmer" would ignore
errors and/or fail to provide recovery from recoverable errors in
production software.
The topic of this post is not whether to check errors or not.

It is now.
Then quoting my question in your post as if your responding to some
claim I made makes no sense at all.

Aug 2 '06 #41

P: n/a
Sjouke Burry a écrit :
>
The only time I used a debugger in 35 years of
programming, was when an assembler decoding
subroutine failed to cross a 64K barrier properly.
(I did not write that one.).
To me, it smells like "let the system catch my errors"
while you should try to avoid them in the first place.
Sloppy typing can cause errors which are not
found by your compiler/debugger.
Well, is a question of taste. I am not a wizard/genius
programmer and I think I will never find all possible
situations that my software will find. Specially when
interacting with other party software that is underspecified
(like in the case of writing a linker, or an assembler)
there is no other way to know "what is that routine returning"
actually.

And yes, debuggers speed up the development, specially if
used within a good IDE. "Go to definition" "Show usage of"
and many other tools that modern IDEs offer are now a
second nature to me.

Yes, sometimes you do not have a debugger, and then
my first task is usually to write one. Last embedded system
I did was a DSP with 16 bit CPU and 80K RAM. The first thing
I wrote for it was a debugger that used the serial port.

You write all programs correctly in 35 years... never need
a debugger... OK. That's YOUR way of doing things.

In 35 years of coding I never wrote a significant system without having
to go in the debugger to develop it.

Obviously "hello world" type of programs run the first time. But
serious systems?

jacob
Aug 2 '06 #42

P: n/a
Keith Thompson <ks***@mib.orgwrote:
Richard Heathfield <in*****@invalid.invalidwrites:
[...]
The response to a failure depends on the situation. I've covered this in
some detail in my one-and-only contribution to "the literature", so I'll
just bullet-point some possible responses here:

* abort the program. The "student solution" - suitable only for high school
students and, perhaps, example programs (with a big red warning flag).
* break down the memory requirement into two or more sub-blocks.
* use less memory!
* point to a fixed-length buffer instead (and remember not to free it!)
* allocate an emergency reserve at the beginning of the program
* use virtual memory on another machine networked to this one (this
is a lot of work, but it may be worth it on super-huge projects)
[...]

Out of curiosity, how often do real-world programs really do something
fancy in response to a malloc() failure?
Depends on the situation. For example, if the program is a word
processor, then I would hope that the response to a malloc() failure
that occurs when trying to paste a humungous graphic would be to put up
a message saying "sorry, no memory to paste this picture" and keep the
rest of the document in a workable, savable condition, and not to throw
up your hands and die, trashing the still usable existing document.
I've seen suggestions that, if a malloc() call fails, the program can
fall back to an alternative algorithm that uses less memory. How
realistic is this? If there's an algorithm that uses less memory, why
not use it in the first place? (The obvious answer: because it's
slower.) Do programmers really go to the effort of implementing two
separate algorithms, one of which will be used only on a memory
failure (and will therefore not be tested as thoroughly as the primary
algorithm)?
I would expect that for, say, sorting code this is not all that unusual.
Ditto for, say, handling massive queries in database servers.

Richard
Aug 2 '06 #43

P: n/a
rl*@hoekstra-uitgeverij.nl (Richard Bos) writes:
Keith Thompson <ks***@mib.orgwrote:
[...]
>Out of curiosity, how often do real-world programs really do something
fancy in response to a malloc() failure?

Depends on the situation. For example, if the program is a word
processor, then I would hope that the response to a malloc() failure
that occurs when trying to paste a humungous graphic would be to put up
a message saying "sorry, no memory to paste this picture" and keep the
rest of the document in a workable, savable condition, and not to throw
up your hands and die, trashing the still usable existing document.
Thank you, that's an excellent example.

In general, a batch computational program will perform a series of
operations, and if any of them fails it's likely (but by no means
certain) that the best you can do is throw out the whole thing. An
interactive program, on the other hand, performs a series of tasks
that don't necessarily depend on each other, so it makes more sense to
abort one of them and continue with the others.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Aug 2 '06 #44

P: n/a
Richard Heathfield wrote:
Skarmander said:
>Richard Heathfield wrote:
<snip>
>>Perhaps I should have made it clearer that I'm referring to programs that
are intended to be used over and over and over by lotsa lotsa people. If
it's not worth the programmer's time to write the code robustly, it's not
worth my time to use his program. Of course, it may be worth /his/ time
to use his own program.
You're using "robust" as if it's a yes or no property. You're free to
storm off in a huff shouting "well that's not robust then!" if someone
presents you with a program that can deal with just about anything except
memory exhaustion when it finally comes round the bend, but that doesn't
mean everyone would or should.

<grinNo, there's no harumphing going on over here. But basic resource
acquisition checking is as fundamental to robustness as steering so as not
to bump the kerb is to driving.
Checking, yes. Failure strategies are another matter.

A program that merrily malloc()s along and does not check the return value
is brittle, no argument there. A program that checks and decides to quit at
that point need not be -- but see below.
>>>I'm not saying the tradeoffs involved are always correctly assessed
(it's probably a given that the cost of failure is usually
underestimated), but I do believe they exist.
Yes, the cost of failure can be a lost customer, repeated N times.
Don't sell it short. It could be a DEAD CUSTOMER, repeated N times.

Absolutely. And a dead customer is a lost customer, right?
Worse, a dead customer is a lost customer whose relatives will probably
claim. But let us not "bicker and argue over who killed who"...
>Or it could be a single entry in a log somewhere.

Or not even that. We're back to undefined behaviour.
That's not what I meant. A program that deals with memory exhaustion by
exiting cleanly may actually offer enough guarantees to the customer, as far
as the customer is concerned. If that's the case, we needn't worry about
dead customers (as the customer has done this for us), and we'd probably
waste time doing so.
>All this will not make the program more *robust*, however, it'll simply
allow it to make better use of existing resources. ("Simply" should
nevertheless not be taken lightly.) Robustness is given by how well the
system behaves when the resources run dry, not how well it squeezes water
from the remaining stones when there's a drought in sight.

If the objective is to gather enough moisture to survive until the rescue
helicopter arrives - that is, if the objective is to complete the immediate
task so that a consistent set of user data can be saved before you bomb out
- then it's a sensible approach.
Yes, this (make sure you're in a consistent state before bailing out) is the
sense in which robustness applies to memory exhaustion.

Your initial post was confusing because you made it look as if exiting with
an error was unacceptable, as the "high school solution", while what you
meant (I take it) was that a program should respond by working towards a
consistent state, and then (as it eventually will have to) bail out. ("Wait
for more memory to become available" may be an approach for a very limited
set of circumstances, but it's not recommendable in general at all, as it'll
probably lead to deadlocks.)

If the high school student has looked at the program carefully and decided
that, at the point of memory exhaustion, the program state was consistent
enough (and this is given by the specifications, explicit or not), then
simply exiting is altogether acceptable. My argument was that many "high
school exits" are acceptable in just that way. Not all, and exiting as a
blanket approach to everything isn't acceptable, but in many cases that lone
exit() is just what is called for.

Your other recommendations were useful strategies to trade some (often
negligible) performance for more social memory requirements. I see how in
many cases these might be considered as making things more robust in a given
environment, however, so it's mostly arguing semantics.
>>>>* point to a fixed-length buffer instead (and remember not to free it!)
Thread unsafe (unavoidably breaches modularity by aliasing a global, if
you like it more general), increased potential for buffer overflow,
requires checking for an exceptional construct in the corresponding
free() wrapper (I'm assuming we'd wrap this).
Who says it has to be global? It would never have occurred to me to use a
file scope object for such a purpose.
So N fixed-length buffers, then? Obviously N can't be dynamic, or you're
back to square one.

I was assuming just the one, actually, local to the function where the
storage is needed.
My bad, I was thinking in terms of a global approach to memory allocation
(wrappers around malloc() and the like).

S.
Aug 2 '06 #45

P: n/a

jacob navia wrote:
Johan Tibell a écrit :
I've written a piece of code that uses sockets a lot (I know that
sockets aren't portable C, this is not a question about sockets per
se). Much of my code ended up looking like this:

if (function(socket, args) == -1) {
perror("function");
exit(EXIT_FAILURE);
}

I feel that the ifs destroy the readability of my code. Would it be
better to declare an int variable (say succ) and use the following
structure?

int succ;

succ = function(socket, args);
if (succ == -1) {
perror("function");
exit(EXIT_FAILURE);
}

What's considered "best practice" (feel free to substitute with: "what
do most good programmers use")?

Using the second form allows you to easily see the return value in
the debugger.
Unless the enthusiastic compiler optimised the variable
away (its not really needed in the above example, unless
it gets tested again).

goose,

Aug 2 '06 #46

P: n/a

Johan Tibell wrote:
I've written a piece of code that uses sockets a lot (I know that
sockets aren't portable C, this is not a question about sockets per
se). Much of my code ended up looking like this:

if (function(socket, args) == -1) {
perror("function");
exit(EXIT_FAILURE);
}

I feel that the ifs destroy the readability of my code. Would it be
better to declare an int variable (say succ) and use the following
structure?

int succ;

succ = function(socket, args);
if (succ == -1) {
perror("function");
exit(EXIT_FAILURE);
}

What's considered "best practice" (feel free to substitute with: "what
do most good programmers use")?
You're in luck :-), here is a slightly modified (incomplete) function
that I wrote (during the course of play:) in the last 30 minutes.

The macros ERROR and DIAGNOSTIC are (currently) identical
and merely print the message to screen (with filename, line
number and function name):

----------------------
#define TEST_INPUT ("test.in")
#define TOK_FOPEN (1)
#define TOK_FERROR (2)
#define TOK_INIT (3)
bool test_token (void)
{
FILE *in = fopen (TEST_INPUT, "r");
jmp_buf handler;
int e, c, counter;

/* All errors caught and handled here */
if ((e = setjmp (handler))!=0) {
switch (e) {
case TOK_FERROR:
ERROR ("read error on '%s', read %i bytes\n",
TEST_INPUT, counter);
break;

case TOK_INIT:
ERROR ("unable to initialise '%s' for reading\n",
TEST_INPUT);
break;

case TOK_FOPEN:
ERROR ("unable to open file '%s' for reading\n",
TEST_INPUT);
break;

default:
ERROR ("unknown error\n");
break;
}
if (in) {
fclose (in); in = NULL;
}
return false;
}
/* Meat of function */
if (!in) longjmp (handler, TOK_FOPEN);

DIAGNOSTIC ("translation of '%s' started\n", TEST_INPUT);

c = fgetc (in);
counter = 0;

if (!token_init ()) longjmp (handler, TOK_INIT);

while (c!=EOF) {
counter++;
if (feed_char (c)) {
/* new token awaits us */
}
c = fgetc (in);
}
if (ferror (in)) {
longjmp (handler, TOK_FERROR);
}
printf ("\n");
return true;
}
-----------------------
hth,
goose,

Aug 2 '06 #47

P: n/a

Andrew Poelstra wrote:

<snipped>
programs targeted at home computers or servers can assume that you'll
have a 99.99% success rate on a functioning system when allocating
memory < 1Kb.
Statistically, a 99.99% success rate means that your
program will *certainly* fail in the future.

goose,
Smile, its a joke :-)

Aug 2 '06 #48

P: n/a

Andrew Poelstra wrote:

<snipped>
Because that tiny percentage is the difference between
p = malloc (sizeof *p * q);

and
p = malloc (sizeof *p * q);
if (rv = !!p)
{
/* Rest of function here. */
}

Those ifs nest up and it becomes a pain to manage them.
Then don't; when your number[1] is up and the malloc(10)
call fails, rather exit immediately than have the program
behave unpredictably.

#define FMALLOC(ptr,size) (ptr=malloc (size) ? ptr : exit (-1))
....
char *p; FMALLOC(ptr, sizeof *p * q)
....
Tracking UB is a bloody nightmare for the maintainer!!!
Being unable to reproduce the bug is morale-killer.

[1] When your 0.02% or whatever finally comes up.

goose,

Aug 2 '06 #49

P: n/a

Andrew Poelstra wrote:
On 2006-08-01, Richard Heathfield <in*****@invalid.invalidwrote:
Andrew Poelstra said:
It's not so much a matter of style as it is a matter of practibility:
your code must be robust, but it should also be easy to read.
Your code is not going to be robust if it doesn't check whether a request
for an external resource was successful. I agree it should be easy to read,
but that doesn't mean leaving the code out!

What exactly /would/ be the way to do such a thing? I ask you because
you don't like multiple returns or the break statement, both of which
would be a typical response.
See my setjmp/longjmp "solution" above; yes its dirty but it
removes the error recovery code from the logic so that the logic
at least can look clean. It also lets you go mad with error recovery
without you having your logic all messed up.
>
Nowadays
programs targeted at home computers or servers can assume that you'll
have a 99.99% success rate on a functioning system when allocating
memory < 1Kb.
Programmers who make such an assumption should not be writing for the home
market or the server market. They should be writing in crayon on droolproof
paper.

Being as every other post was pretty much exactly as insulting as this,
have you met Dan Pop yet ? :-)
I'd say that I wasn't not wrong on any minor point! I'm glad that I haven't
had the chance to make these foolhardy changes to my actual code yet.
s/not//
This /point/ applies to any resource, not just memory.
<otThats what I battle to get into most programmers
skulls: GC doesn't really help as memory is just another
resource and should be treated as such i.e. the language
(and/or compiler) may let you forget all about managing
memory, but you'll still have to do it resource management
anyway. So don't get too smug about your GC language.
All that will happen is that you'll lose the force-of-habit
action that comes with using malloc and forget to free some
*other* resource.
>
I've written a new interface to my error library so that it will be able
to handle memory failures gracefully, log to a runtime-determined file,
check for bad files or memory, and ensure that a proper message reaches
the user if it can't go on.
Good for you

goose,

Aug 2 '06 #50

66 Replies

This discussion thread is closed

Replies have been disabled for this discussion.