realloc() hang

[Jonathan Shan]

Hello all,

I am trying to run a program which has dynamic array of type struct.
The program works until the line which uses realloc function to
allocate more memory.

I have tried to reproduce this in a simpler code, but in the simpler
code the program works fine.

Is there any reason realloc would just hang without producing any error
message?

Jonathan Shan

[Eric Sosman]

Jonathan Shan wrote On 07/21/06 15:42,:

Hello all,

I am trying to run a program which has dynamic array of type struct.
The program works until the line which uses realloc function to
allocate more memory.

I have tried to reproduce this in a simpler code, but in the simpler
code the program works fine.

Is there any reason realloc would just hang without producing any error
message?

No "good" reason, but plenty of "bad" ones. Most
likely, the program has made the same kind of error that
can sometimes cause the memory-management functions to
crash: called free() or realloc() on memory that wasn't
dynamically allocated or that has already been released,
stomped on the memory-manager's data structures with a
wild pointer or crazy array index, or something of that
sort.

Since it sounds as though realloc() may be in an
infinite loop, I'd be particularly suspicious of double-
freeing: releasing the same memory pointer twice. In some
implementations this may well introduce a loop into the
manager's internally-maintained lists of available space;
if realloc() is hunting through a list that looks like

+-- A -- B --+
| |
+------ C <------+

.... you can see that it might take an appreciable length
of time before it gets to the end ...

--
Er*********@sun.com

[Jonathan Shan]

free() function is not used in the program. The memory is allocated
using malloc(). Then this pointer is passed around (by reference)
through several functions until it reaches a function called insert.
Inside insert function is a realloc() to increase the length of the
array. I have specifically checked for the address of pointer that is
being passed into realloc. It is "correct", as in it matches the
original address when declared initially.

The strangest part is that inside each function nothing is happening to
the original array. It is just simply being passed around. I have
created simpler code which works correctly. The difference between the
working and non-working code is just some functions, declarations, ...
which don't touch the array.

Backtrace using gdb is:

#0 0xb7eb0189 in pthread_setcanceltype () from /lib/tls/libc.so.6
#1 0xb7e3bb81 in malloc_get_state () from /lib/tls/libc.so.6
#2 0xb7e3a33c in realloc () from /lib/tls/libc.so.6
#3 0x08048f85 in insert_to_bucket (rvalid=0x8085068, s1=0xb7b00690
"172.16.0.240", buf=0xbffcb9b4 "SNMPv2-MIB::sysDescr.0 = No Such Object
available on this agent at this OID")
at testfile.c:235
#4 0x08048b70 in print_result (status=0, sp=0xb7b004e8,
pdu=0xb7b112a8, qvalid=0xbffcbf44) at testfile.c:105
#5 0x08048ef3 in synchronous (pvalid=0xbffcbf44) at testfile.c:209
#6 0x0804900a in main (argc=1, argv=0xbffcbfc4) at testfile.c:261

Jonathan Shan

Eric Sosman wrote:

Jonathan Shan wrote On 07/21/06 15:42,:

Hello all,

I am trying to run a program which has dynamic array of type struct.
The program works until the line which uses realloc function to
allocate more memory.

I have tried to reproduce this in a simpler code, but in the simpler
code the program works fine.

Is there any reason realloc would just hang without producing any error
message?

No "good" reason, but plenty of "bad" ones. Most
likely, the program has made the same kind of error that
can sometimes cause the memory-management functions to
crash: called free() or realloc() on memory that wasn't
dynamically allocated or that has already been released,
stomped on the memory-manager's data structures with a
wild pointer or crazy array index, or something of that
sort.

Since it sounds as though realloc() may be in an
infinite loop, I'd be particularly suspicious of double-
freeing: releasing the same memory pointer twice. In some
implementations this may well introduce a loop into the
manager's internally-maintained lists of available space;
if realloc() is hunting through a list that looks like

+-- A -- B --+
| |
+------ C <------+

... you can see that it might take an appreciable length
of time before it gets to the end ...

--
Er*********@sun.com

[Eric Sosman]

Because it makes a perfectly sensible sequence of remarks
hard to follow, that's why.

Why do people dislike it so much?

Top-posting: Writing a response before rather than after
the text being responded to.

Please name a bad habit that Usenet readers dislike.

(I've rearranged your top-posting for better readability;
please don't make it necessary again.)

Jonathan Shan wrote On 07/21/06 16:24,:

Eric Sosman wrote:

>>Jonathan Shan wrote On 07/21/06 15:42,:

>>[...]
Is there any reason realloc would just hang without producing any error
message?

No "good" reason, but plenty of "bad" ones. [...]
Since it sounds as though realloc() may be in an
infinite loop, I'd be particularly suspicious of double-
freeing: releasing the same memory pointer twice. [...]

free() function is not used in the program.

If realloc() decides to "move" a piece of allocated
memory, it allocates a new piece and "frees" the old one.
So even if you never call the free() function, you may be
"free"ing memory anyhow.

The memory is allocated
using malloc(). Then this pointer is passed around (by reference)
through several functions until it reaches a function called insert.
Inside insert function is a realloc() to increase the length of the
array. I have specifically checked for the address of pointer that is
being passed into realloc. It is "correct", as in it matches the
original address when declared initially.

That in itself is somewhat suspicious. If you allocate
a piece of memory and then expand it a few times with realloc(),
there is no reason to believe the memory will still be in the
same place: realloc() may have allocated a new memory area,
copied the old contents into it, and freed the original. But
if you are just handing the same, unaltered pointer around,
that pointer is no longer valid. If you hand that same pointer
to realloc() a second time, after the memory it formerly pointed
to has been freed, that is just as bad as calling free() twice
on the same memory area.

The strangest part is that inside each function nothing is happening to
the original array. It is just simply being passed around. I have
created simpler code which works correctly. The difference between the
working and non-working code is just some functions, declarations, ...
which don't touch the array.

I'm not saying that your problem *is* a double-free, just
that the symptom is suggestive of that cause. It could perfectly
well be a "wild pointer" or "buffer overrun" issue; such errors
have a nasty habit of changing their behavior when you make
"unrelated" changes to the program -- add a printf() and a bug
suddenly appears in a completely different part of the program,
add another printf() to help track it down and the bug vanishes
again ... Errors of this kind are sometimes called "Heisenbugs"
because the observer affects the observed. And because of their
changeable symptoms, they can be bloody murder to track down ...

I'd suggest that you do a really thorough check for double-
freeing (the symptom is highly suggestive), but if that's not
the problem you'll have to work even harder. Posting some code
snippets (complete, compilable, and concise) might draw some
helpful ideas, too. Good luck!

--
Er*********@sun.com

[Gordon Burditt]

>I am trying to run a program which has dynamic array of type struct.

>The program works until the line which uses realloc function to
allocate more memory.

I have tried to reproduce this in a simpler code, but in the simpler
code the program works fine.

If realloc() misbehaves, chances are you passed it a bad pointer
(already freed, not returned from malloc() or realloc()) or wrote
out of range *ANYWHERE IN PREVIOUSLY EXECUTED CODE*. Either one
invokes the wrath of undefined behavior.

>Is there any reason realloc would just hang without producing any error
message?

realloc() is not supposed to "produce an error message". If it
fails, it returns NULL. However, if you have invoked the wrath of
undefined behavior by scribbling on memory (the bug could be
*ANYWHERE* you have previously executed), it might produce messages
like "smegmentation violation -- core dumped".

Gordon L. Burditt

[Richard Tobin]

In article <1153520931.365052@news1nwk>,
Eric Sosman <Er*********@sun.comwrote:

>Because it makes a perfectly sensible sequence of remarks
hard to follow, that's why.

And yet, it didn't make it hard to follow at all.

-- Richard

[Jonathan Shan]

Gordon Burditt wrote:

If realloc() misbehaves, chances are you passed it a bad pointer
(already freed, not returned from malloc() or realloc()) or wrote
out of range *ANYWHERE IN PREVIOUSLY EXECUTED CODE*. Either one
invokes the wrath of undefined behavior.

Indeed, this is source of the problem. What I was trying to do was
create a string from known strings and integers which ranged from 0 to
255. Sprinkled about were numerous mallocs involving both struct and
char*. The fix was to remove all the pointers, and create fixed-size
arrays.

Jonathan Shan

[Default User]

Jonathan Shan wrote:

Gordon Burditt wrote:

If realloc() misbehaves, chances are you passed it a bad pointer
(already freed, not returned from malloc() or realloc()) or wrote
out of range *ANYWHERE IN PREVIOUSLY EXECUTED CODE*. Either one
invokes the wrath of undefined behavior.

Indeed, this is source of the problem. What I was trying to do was
create a string from known strings and integers which ranged from 0 to
255. Sprinkled about were numerous mallocs involving both struct and
char*. The fix was to remove all the pointers, and create fixed-size
arrays.

That's a shotgun fix. You got it to work without really understanding
why it was failing before. The better solution would have been track
down whatever was really causing the problem and fix that, rather than
just blindly changing code until it works.

Somewhere you had a buffer overrun or a double free or something else
that was corrupting dynamic storage.

Brian