bizarre malloc problem

[Snis Pilbor]

First, let me announce that this is very possibly off-topic because
malloc is a specific third party accessory to c, etc. I spent about an
hour trying to find a more appropriate newsgroup and failed. If anyone
could point one out, I would be much obliged. I'm using MSVC .NET, but
the only .NET specific newsgroups I could find were vbasic newsgroups
and you'll agree a C malloc question is less off-topic here than there,
at least.

I seem to be suffering the exact opposite of fragmentation: I have no
problem whatsoever allocation a large array of structures, but if I try
to allocate room for just 1 single structure, the program immediately
throws an exception. Debugging indicates the exception is indeed
thrown by malloc itself and not by some later line of code.

I temporarily solved the problem as follows:
original code:

void myfunction( void )
{
struct mystructure *m;

m = (struct mystructure *) malloc( sizeof(struct mystructure) );

...
}

band-aid solution code:

#define MAX_MYSTRUCTURES 1000
void myfunction( void )
{
static int x = 0;
static struct mystructure *buf = NULL;
struct mystructure *m;

if ( !buf )
buf = (struct mystructure *) malloc( MAX_MYSTRUCTURES *
sizeof(struct mystructure) );

m = &buf[x++];

...
}

Amazingly the original code throws an exception when it reaches the
malloc line, while the band-aid fix code, which allocates MUCH MORE
memory, does not.
(the structure in question is pretty small-- 4 pointers, 4 ints and 3
chars)

If anyone could go above and beyond and offer me some insight even
though this isn't pure unblemished by-the-books C, I would be much
indebted to you.

Snis

[Gordon Burditt]

>First, let me announce that this is very possibly off-topic because

malloc is a specific third party accessory to c, etc. I spent about an
malloc() is an integral part of the C library and is very on-topic.
hour trying to find a more appropriate newsgroup and failed. If anyone
could point one out, I would be much obliged. I'm using MSVC .NET, but
the only .NET specific newsgroups I could find were vbasic newsgroups
and you'll agree a C malloc question is less off-topic here than there,
at least. I seem to be suffering the exact opposite of fragmentation: I have no
problem whatsoever allocation a large array of structures, but if I try
to allocate room for just 1 single structure, the program immediately
throws an exception. Debugging indicates the exception is indeed
thrown by malloc itself and not by some later line of code.
When malloc() throws an exception, it is usually because something
stomped the malloc() arena *SOMEWHERE* before that malloc() call.
And you really have no idea where the problem is.
I temporarily solved the problem as follows:
original code:

void myfunction( void )
{
struct mystructure *m;

m = (struct mystructure *) malloc( sizeof(struct mystructure) );

...
}

band-aid solution code:

#define MAX_MYSTRUCTURES 1000
void myfunction( void )
{
static int x = 0;
static struct mystructure *buf = NULL;
struct mystructure *m;

if ( !buf )
buf = (struct mystructure *) malloc( MAX_MYSTRUCTURES *
sizeof(struct mystructure) );

m = &buf[x++];

...
}

Amazingly the original code throws an exception when it reaches the
malloc line, while the band-aid fix code, which allocates MUCH MORE
memory, does not.
"reaches the malloc line" - you mean RUNS OUT OF MEMORY?
How do you know this, since your code doesn't check for malloc()
returning NULL?
(the structure in question is pretty small-- 4 pointers, 4 ints and 3
chars)
Show me the code that writes on the structure you allocate.
Are you sure you don't have subscripts that go out of range?
Strings that are too long?
If anyone could go above and beyond and offer me some insight even
though this isn't pure unblemished by-the-books C, I would be much
indebted to you.

Show us the code. ALL of it.

Gordon L. Burditt

[Default User]

Snis Pilbor wrote:

I seem to be suffering the exact opposite of fragmentation: I have no
problem whatsoever allocation a large array of structures, but if I try
to allocate room for just 1 single structure, the program immediately
throws an exception. Debugging indicates the exception is indeed
thrown by malloc itself and not by some later line of code.

Learn this mantra, "Minimal, complete, compilable program that
demonstrates the problem."
Only that way can we tell what's going on. There is little or no chance
the implementation is broken in such a fundamental way. So you are
doing something wrong. As you don't know what that is, you aren't
qualified (and I mean that in a non-mean way) to decide what code we
should look at.

Try again, striving for the goal above. You may find the problem on
your own when constructing that.

Brian

[Flash Gordon]

Snis Pilbor wrote:

First, let me announce that this is very possibly off-topic because
malloc is a specific third party accessory to c, etc.
malloc is not a 3rd party accessory, it is part of the standard C
library and so on topic.
I spent about an
hour trying to find a more appropriate newsgroup and failed. If anyone
could point one out, I would be much obliged. I'm using MSVC .NET, but
the only .NET specific newsgroups I could find were vbasic newsgroups
and you'll agree a C malloc question is less off-topic here than there,
at least.

I seem to be suffering the exact opposite of fragmentation: I have no
problem whatsoever allocation a large array of structures, but if I try
to allocate room for just 1 single structure, the program immediately
throws an exception. Debugging indicates the exception is indeed
thrown by malloc itself and not by some later line of code.
This normally means you have corrupted the the structures used to manage
the heap (on implementations with a heap). This is generally the result
of either running off the end of a buffer or freeing a pointer twice.
I temporarily solved the problem as follows:
original code:

void myfunction( void )
{
struct mystructure *m;

m = (struct mystructure *) malloc( sizeof(struct mystructure) );
You don't need the cast. If the compiler complains when you remove the
cast then either you have failed to include stdlib.h or you are
compiling as C++ instead of C. A simpler, less error prone option would be:

struct mystructure *m = malloc( sizeof *m );

However, this is not the cause of your problem in this case.
...
}

band-aid solution code:

#define MAX_MYSTRUCTURES 1000
void myfunction( void )
{
static int x = 0;
static struct mystructure *buf = NULL;
struct mystructure *m;

if ( !buf )
buf = (struct mystructure *) malloc( MAX_MYSTRUCTURES *
sizeof(struct mystructure) );

m = &buf[x++];

...
}

Amazingly the original code throws an exception when it reaches the
malloc line, while the band-aid fix code, which allocates MUCH MORE
memory, does not.
(the structure in question is pretty small-- 4 pointers, 4 ints and 3
chars)

If anyone could go above and beyond and offer me some insight even
though this isn't pure unblemished by-the-books C, I would be much
indebted to you.

Check the rest of your code for buffer overruns. That band aid will fail
the moment you have an important demo.
--
Flash Gordon
Living in interesting times.
Although my email address says spam, it is real and I read it.

[Snis Pilbor]

Flash Gordon wrote:

Snis Pilbor wrote:

(snip)

I seem to be suffering the exact opposite of fragmentation: I have no
problem whatsoever allocation a large array of structures, but if I try
to allocate room for just 1 single structure, the program immediately
throws an exception. Debugging indicates the exception is indeed
thrown by malloc itself and not by some later line of code.

This normally means you have corrupted the the structures used to manage
the heap (on implementations with a heap). This is generally the result
of either running off the end of a buffer or freeing a pointer twice.

(snip)

Ahh, thank you so very kindly for this insight. Knowing this, I
shifted my attention to a different part of my code and believe I was
able to find the culprit.

Old bad code:

char *str_alloc( char *string )
{
char *x = (char *) malloc( strlen(string) * sizeof(char));
strcpy(x,string);
return x;
}

Fixed code:

char *str_alloc( char *string )
{
char *x = (char *) malloc( (strlen(string)+1) * sizeof(char) );
strcpy(x,string);
return x;
}

Wow, I have learned a lot from this. One of these days I am going to
have to teach myself the intricate details of how malloc works. The
naive expectation would be to assume the strcpy in the unfixed code
above would have immediately excepted. But that would be the clumsy,
slow, inefficient Java way to do it. Thanks again for the help,
everyone who chipped in. Oh, and sorry about not posting the entire
code in my OP-- the entire code is over 2000 lines so I didn't imagine
that would have been appropriate.

Snis

[Clark S. Cox III]

On 2005-06-28 21:18:09 -0400, "Snis Pilbor" <sn********@yahoo.com> said:

Flash Gordon wrote:

Snis Pilbor wrote: (snip)

I seem to be suffering the exact opposite of fragmentation: I have no
problem whatsoever allocation a large array of structures, but if I try
to allocate room for just 1 single structure, the program immediately
throws an exception. Debugging indicates the exception is indeed
thrown by malloc itself and not by some later line of code.

This normally means you have corrupted the the structures used to manage
the heap (on implementations with a heap). This is generally the result
of either running off the end of a buffer or freeing a pointer twice.

(snip)

Ahh, thank you so very kindly for this insight. Knowing this, I
shifted my attention to a different part of my code and believe I was
able to find the culprit.

Old bad code:

char *str_alloc( char *string )
{
char *x = (char *) malloc( strlen(string) * sizeof(char));
strcpy(x,string);
return x;
}

Fixed code:

char *str_alloc( char *string )
{
char *x = (char *) malloc( (strlen(string)+1) * sizeof(char) );
strcpy(x,string);
return x;
}

1. Cast is unnecessary, and could hide potential bugs
2. "* sizeof(char)" is redundant, sizeof(char) is always 1
3. No check for malloc failure

Re-fixed code:

char *str_alloc( char *string )
{
char *x = malloc(strlen(string)+1);

if(x)
{
strcpy(x,string);
}

return x;
}
Wow, I have learned a lot from this. One of these days I am going to have to teach myself the intricate details of how malloc works. The
naive expectation would be to assume the strcpy in the unfixed code
above would have immediately excepted. But that would be the clumsy,
slow, inefficient Java way to do it. Thanks again for the help,
everyone who chipped in. Oh, and sorry about not posting the entire
code in my OP-- the entire code is over 2000 lines so I didn't imagine
that would have been appropriate.

That's usually what happens :). When you try to cut your code down to
a post-able sample that demonstrates the problem, you end up finding
the problem yourself.

--
Clark S. Cox, III
cl*******@gmail.com

[CBFalconer]

Snis Pilbor wrote:

First, let me announce that this is very possibly off-topic because
malloc is a specific third party accessory to c, etc. I spent about .... snip ...
I temporarily solved the problem as follows:
original code:

void myfunction( void )
{
struct mystructure *m;

m = (struct mystructure *) malloc( sizeof(struct mystructure) );
...
}

That code is casting the result from malloc, which it should not
do. The cast is very likely hiding the serious error of failing to
#include <stdlib.h>. malloc is very much on topic here, because
it, and its actions, are specified by the C standard.

The statement should read:

m = malloc(sizeof *m);

which should not have any problems if the type of m is properly
defined, and stdlib.h has been included.
--
"A man who is right every time is not likely to do very much."
-- Francis Crick, co-discover of DNA
"There is nothing more amazing than stupidity in action."
-- Thomas Matthews

[CBFalconer]

Snis Pilbor wrote:

.... snip ...
Fixed code:

char *str_alloc( char *string )
{
char *x = (char *) malloc( (strlen(string)+1) * sizeof(char) );
strcpy(x,string);
return x;
}

Wow, I have learned a lot from this. One of these days I am going
to have to teach myself the intricate details of how malloc works.

.... snip ...

Not enough. You are still casting malloc, which is NEVER needed
and only serves to prevent the compiler diagnosing errors.
sizeof(char) is 1 by definition. Thus your malloc call should be:

char *x;

if (x = malloc(1 + strlen(string))) strcpy(x, string);

since it is fairly hard to copy into non-assigned storage. You
should always test the success of malloc (and realloc).

--
"A man who is right every time is not likely to do very much."
-- Francis Crick, co-discover of DNA
"There is nothing more amazing than stupidity in action."
-- Thomas Matthews

[Lawrence Kirby]

On Tue, 28 Jun 2005 21:33:54 -0400, Clark S. Cox III wrote:

....

Re-fixed code:

char *str_alloc( char *string )
Better still make that

char *str_alloc( const char *string )

because str_alloc() doesn't write to the original string.
{
char *x = malloc(strlen(string)+1);

if(x)
{
strcpy(x,string);
}

return x;
}
Wow, I have learned a lot from this. One of these days I am going to

have to teach myself the intricate details of how malloc works.
There's no need to learn ho malloc() works internally, that will vary from
implementation to implementation. You just need to know how to use it
properly.
The
naive expectation would be to assume the strcpy in the unfixed code
above would have immediately excepted. But that would be the clumsy,
slow, inefficient Java way to do it. Thanks again for the help,
everyone who chipped in. Oh, and sorry about not posting the entire
code in my OP-- the entire code is over 2000 lines so I didn't imagine
that would have been appropriate.

Yes, C doesn't inherently have bounds checking. This isn't a malloc()
rwlated probblem specifically, you could just as easily have written past
the end of a declared array.

Lawrence