How does STL report out of memory errors?

[pit3k]

My program tries to call the vector::push_back(...) member function in a
loop couple dozen milion times, which is what I expect.

After about 20 millions successfull push_back() calls the program
abnormally aborts with a core dump.
Investigation under debugger showed that this happens because the
internal vector's pointer was beeing decreased from about 0x40000000 to
0x00000009 and next push_back() call caused the core dump.

How am I supposed to detect such a situation in my program?
Can I made STL to throw an out of memory exception maybe?

--
pit3k

[Ron Natalie]

pit3k wrote:

How am I supposed to detect such a situation in my program?
Can I made STL to throw an out of memory exception maybe?

C++ reports failure to dynamically allocate by throwing a bad_alloc
exception. Of course dealing with that is rather involved.

[Victor Bazarov]

pit3k wrote:

My program tries to call the vector::push_back(...) member function in a
loop couple dozen milion times, which is what I expect.

After about 20 millions successfull push_back() calls the program
abnormally aborts with a core dump.
Investigation under debugger showed that this happens because the
internal vector's pointer was beeing decreased from about 0x40000000 to
0x00000009 and next push_back() call caused the core dump.

How am I supposed to detect such a situation in my program?
Can I made STL to throw an out of memory exception maybe?

You don't have to make it. It should throw 'std::bad_alloc' if it cannot
allocate.

V

[mlimber]

pit3k wrote:

My program tries to call the vector::push_back(...) member function in a
loop couple dozen milion times, which is what I expect.

After about 20 millions successfull push_back() calls the program
abnormally aborts with a core dump.
Investigation under debugger showed that this happens because the
internal vector's pointer was beeing decreased from about 0x40000000 to
0x00000009 and next push_back() call caused the core dump.
This was probably because it had to resize the vector to add the new
element, and since it couldn't just tack the space onto the end of your
existing vector because it ran out of room, it tried to relocate the
entire thing. But something went wrong. It should throw an exception
when such an error occurs.

1. What STL implementation are you using? Is it a checked
implementation in which you can enable debugging features?

2. Could you use vector::reserve to preallocate space and avoid
implicit growing?

3. Are you sure your iterators (or pointers) to the vector are not
being invalidated and then used?

4. Are you sure you're catching any exception that might be thrown?

5. An uncaught exception sometimes appears like this in Unix:
run_my_memory_hog Abort

Cheers! --M

[pit3k]

mlimber napisal(a):

pit3k wrote:

My program tries to call the vector::push_back(...) member function in a
loop couple dozen milion times, which is what I expect.
After about 20 millions successfull push_back() calls the program
abnormally aborts with a core dump.

First, I meant std::list, not std::vector. It shouldn't change much
though.
1. What STL implementation are you using? Is it a checked
implementation in which you can enable debugging features?
I have no idea. How can I chek that?
2. Could you use vector::reserve to preallocate space and avoid
implicit growing?
I'm not using vector but list, sorry for mistake in OP.
3. Are you sure your iterators (or pointers) to the vector are not
being invalidated and then used?
I'm positive.
4. Are you sure you're catching any exception that might be thrown?
Yes.
5. An uncaught exception sometimes appears like this in Unix:

Not applies.

I've also noticed, that just before my program core dumps it's using
just about 700MB of memory, while the machine I'm running it on has 4GB
RAM.
//
// Below is what my code looks like:
//

struct CallDataRec
{
int xxx;
char yyy[10];
CallDataRec() { xxx = 0; yyy[0] = '\0'; }
};
void read_all()
{
std::list<CallDataRec> buffer;
CallDataRec cdr;

try
{
while (!stream.eof())
{
stream >> cdr;
buffer.push_back(cdr); // It core dump INSIDE the push_back()
// at about 20 millionth iteration
}
}
catch (...)
{
cerr << "Unknown exception" << endl;
}
}

--
pit3k

[Old Wolf]

pit3k wrote:

mlimber napisal(a):

pit3k wrote:

My program tries to call the vector::push_back(...) member
function in a loop couple dozen milion times, which is what I
expect. After about 20 millions successfull push_back() calls
the program abnormally aborts with a core dump.

First, I meant std::list, not std::vector. It shouldn't change much
though.

vector stores all its members as a single contiguous array,
which gets reallocated when it fills up.
list does a new allocation for each new member.
This is a big difference.
struct CallDataRec
{
int xxx;
char yyy[10];
CallDataRec() { xxx = 0; yyy[0] = '\0'; }
};
void read_all()
{
std::list<CallDataRec> buffer;
CallDataRec cdr;

try
{
while (!stream.eof())
This is an error and could lead to a core dump. Please read the
FAQ on how to use eof() correctly.
{
stream >> cdr;
This fails to compile, as there is no operator>> defined for
writing to a CallDataRec.
buffer.push_back(cdr); // It core dump INSIDE the push_back()
// at about 20 millionth iteration
Probably there was heap corruption in the 'stream >> cdr' operation.
}
}
catch (...)
{
cerr << "Unknown exception" << endl;
}
Try catching a std::exception &, as some compilers (eg. Borland) have
been known to have problems with catch(...) .
}

Please post a COMPLETE program that compiles correctly, and
demonstrates the problem. Then, someone here will be able
to give you an exact answer as to what is going on.

Hint: the I/O loop should look like:

while (stream >> cdr)
buffer.push_back(cdr);

[pit3k]

Old Wolf napisal(a):

pit3k wrote:

mlimber napisal(a):

pit3k wrote:
My program tries to call the vector::push_back(...) member
function in a loop couple dozen milion times, which is what I
expect. After about 20 millions successfull push_back() calls
the program abnormally aborts with a core dump.
First, I meant std::list, not std::vector. It shouldn't change much
though.

[SNIP]
Please post a COMPLETE program that compiles correctly, and
demonstrates the problem. Then, someone here will be able
to give you an exact answer as to what is going on.

OK, I've just used GNU g++ compiler instead of HP aCC.

The results:
g++ version throws std::bad_alloc exception, when it's memory usage (as
shown by top) reaches 953MB.
aCC version core dumps with "Bus error" when memory usage reaches
870MB.

Does that mean that HP compiler / STL implementation is broken or is
the core dump behaviour also valid?
Results were obtained using the following test program:

#include <list>
#include <iostream>

int main()
{
std::list<int> buffer;

try
{
for(;;)
buffer.push_back(0);
}
catch (std::bad_alloc &e)
{
std::cerr << "Memory allocation error" << std::endl;
}
catch (...)
{
std::cerr << "Unknown exception" << std::endl;
}

return 0;
}