By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
446,413 Members | 999 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 446,413 IT Pros & Developers. It's quick & easy.

Problems allocating memory

P: n/a
Hi everyone.
I've made a class Matrix, which contains a pointer to the data and some
methods which all return a copy of the matrix modified in some way. The
program works quite well for small matrices, but as matrix dimension
grows up it crashes in deterministic locations depending on data,
machine and OS.
In particular, this happens when I test it with 3000x3000 matrices
(which take up more than 30MB) under three different machines/OSes. The
crash happens when much memory is still available (for example, under
cygwin, more than 1GB of virtual memory available), and I cannot think
there is no contiguous space for allocating the matrix.
I always use references and never pointers to matrices, so Matrix
objects are not dynamically allocated (the data, however, is allocated
by the constructor). Could this be a problem with stack/heap limitations?
Can someone help me with some idea on how to further investigate the
problem?

Thank you,
Ignazio
Mar 9 '07 #1
Share this Question
Share on Google+
20 Replies


P: n/a
"Neclepsio" <to*******************@gmail.comwrote in message
news:45********@x-privat.org...
: Hi everyone.
: I've made a class Matrix, which contains a pointer to the data and
some
: methods which all return a copy of the matrix modified in some way.
The
: program works quite well for small matrices, but as matrix dimension
: grows up it crashes in deterministic locations depending on data,
: machine and OS.
: In particular, this happens when I test it with 3000x3000 matrices
: (which take up more than 30MB) under three different machines/OSes.
The
: crash happens when much memory is still available (for example, under
: cygwin, more than 1GB of virtual memory available), and I cannot think
: there is no contiguous space for allocating the matrix.
: I always use references and never pointers to matrices, so Matrix
: objects are not dynamically allocated (the data, however, is allocated
: by the constructor). Could this be a problem with stack/heap
limitations?
: Can someone help me with some idea on how to further investigate the
: problem?

The first thing to check is that you have provided correct
implementations of the destructor, the copy-constructor, and the
copy-assignment operators.

But to make your life easier in the first place, you should consider
using std::vector instead of a raw pointer to store the matrix data.

hth -Ivan
--
http://ivan.vecerina.com/contact/?subject=NG_POST <- email contact form
Brainbench MVP for C++ <http://www.brainbench.com

Mar 9 '07 #2

P: n/a
Thank you for replying.

Ivan Vecerina wrote:
The first thing to check is that you have provided correct
implementations of the destructor, the copy-constructor, and the
copy-assignment operators.
Everything is ok. The second?
But to make your life easier in the first place, you should consider
using std::vector instead of a raw pointer to store the matrix data.
I needed raw pointers to use the BLAS library.

Thank you,
Ignazio
Mar 9 '07 #3

P: n/a
Neclepsio wrote:
Hi everyone.
I've made a class Matrix, which contains a pointer to the data and some
methods which all return a copy of the matrix modified in some way. The
program works quite well for small matrices, but as matrix dimension
grows up it crashes in deterministic locations depending on data,
machine and OS.
In particular, this happens when I test it with 3000x3000 matrices
(which take up more than 30MB) under three different machines/OSes. The
crash happens when much memory is still available (for example, under
cygwin, more than 1GB of virtual memory available), and I cannot think
there is no contiguous space for allocating the matrix.
I always use references and never pointers to matrices, so Matrix
objects are not dynamically allocated (the data, however, is allocated
by the constructor). Could this be a problem with stack/heap limitations?
The fact that it crashes under three different platforms suggests not.
Can someone help me with some idea on how to further investigate the
problem?
If you can reproduce the problem in a small program then post the code
here. Plently will be willing to take a look.
>
Thank you,
Ignazio
john
Mar 9 '07 #4

P: n/a
Neclepsio wrote:
Hi everyone.
I've made a class Matrix, which contains a pointer to the data and some
methods which all return a copy of the matrix modified in some way. The
program works quite well for small matrices, but as matrix dimension
grows up it crashes in deterministic locations depending on data,
machine and OS.
In particular, this happens when I test it with 3000x3000 matrices
(which take up more than 30MB) under three different machines/OSes. The
crash happens when much memory is still available (for example, under
cygwin, more than 1GB of virtual memory available), and I cannot think
there is no contiguous space for allocating the matrix.
I always use references and never pointers to matrices, so Matrix
objects are not dynamically allocated (the data, however, is allocated
by the constructor). Could this be a problem with stack/heap limitations?
There is no "heap limitation", unless you impose it yourself
(hint: try "ulimit -a"). The stack is definitely limited, in
the order of a few MB usually (try "ulimit -s") -- but you only
hit this limitation if you allocate too many/much _local_ variables.
Can someone help me with some idea on how to further investigate the
problem?
If this is an x86 platform, I highly recommend valgrind,
this a utility that will point out many bugs to you, and does
not require a recompile. Other than that you may try
electric fence ("efence"), though this requires a recompile
and is not as useful as valgrind.

And the obvious way out -- recompile your code with debug
information on (usually the -g option), run it under a
debugger (like gdb), let it crash, then backtrace and
check where the problem occurs. Put tons of debug printouts
there and hopefully all will become clear.

HTH,
- J.
Mar 9 '07 #5

P: n/a
>
>But to make your life easier in the first place, you should consider
using std::vector instead of a raw pointer to store the matrix data.


I needed raw pointers to use the BLAS library.
std::vector does not prevent you from getting a raw pointer to the data.

john
Mar 9 '07 #6

P: n/a
Jacek Dziedzic wrote:
The stack is definitely limited, in
the order of a few MB usually (try "ulimit -s") -- but you only
hit this limitation if you allocate too many/much _local_ variables.
Yes, I know. In my despair, I was wondering if something like:

Matrix A(3000, 3000);
Matrix B(3000, 3000);
Matrix C(3000, 3000);

or

Matrix B = A.Transpose() * B.SubMatrix(Vector::All, Vector::All);
//(each method returns a new Matrix object)

Would take up much stack space... Anyway, since I get a bad_alloc in
Matrix constructor (which dynamically allocates the space), I don't
think it's the point.
If this is an x86 platform, I highly recommend valgrind,
this a utility that will point out many bugs to you, and does
not require a recompile. Other than that you may try
electric fence ("efence"), though this requires a recompile
and is not as useful as valgrind.
Thank you, I'm downloading it.
And the obvious way out -- recompile your code with debug
information on (usually the -g option), run it under a
debugger (like gdb), let it crash, then backtrace and
check where the problem occurs. Put tons of debug printouts
there and hopefully all will become clear.
Yes, I've done... On the cygwin machine (the most accessable for me) it
crashes on the line:

Matrix temp = P.Transpose();

which just allocates and fills a new Matrix with the same dimensions as
caller (just swapped).

Thank you,
Ignazio

Mar 9 '07 #7

P: n/a
John Harrison wrote:
If you can reproduce the problem in a small program then post the code
here. Plently will be willing to take a look.
I'll try to write it as small as I can.

Thank you,
Ignazio
Mar 9 '07 #8

P: n/a
John Harrison wrote:
std::vector does not prevent you from getting a raw pointer to the data.
So, if new can't allocate the memory, why should std::vector be able to?

Thank you,
Ignazio
Mar 9 '07 #9

P: n/a
* Neclepsio:
John Harrison wrote:
>std::vector does not prevent you from getting a raw pointer to the data.

So, if new can't allocate the memory, why should std::vector be able to?
The std::vector code is more likely to be correct.

--
A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
Mar 9 '07 #10

P: n/a
* Alf P. Steinbach:
The std::vector code is more likely to be correct.
Care to elaborate on this hollow claim?

stl::vector is an array with all kinds of nasty strings attached - much
like the rest of this troublesome library. It's an array with an identity
crisis, it's an array with an attitude, but in the end it is still an array.
If you can't new that array yourself, then STL surely can't either. STL *will*
end up doing the new itself.

There's an enormous amount of uncalled for STL zealotry going on, with people
"suggesting" to use STL without even the *slightest* bit of validation to
their claims. The STL is *not* The One Answer. In fact, often, it's exactly
the WRONG answer.

--
Martijn van Buul - pi**@dohd.org
Mar 9 '07 #11

P: n/a
* Martijn van Buul:
* Alf P. Steinbach:
>The std::vector code is more likely to be correct.

Care to elaborate
no.

--
A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
Mar 9 '07 #12

P: n/a
Neclepsio wrote:
John Harrison wrote:
>std::vector does not prevent you from getting a raw pointer to the data.


So, if new can't allocate the memory, why should std::vector be able to?

Thank you,
Ignazio
You are changing the subject, you said that you couldn't use std::vector
because you needed raw pointers. I pointed out that this wasn't true. I
didn't say std::vector would cure your problem.

john
Mar 9 '07 #13

P: n/a
John Harrison wrote:
You are changing the subject, you said that you couldn't use std::vector
because you needed raw pointers. I pointed out that this wasn't true. I
didn't say std::vector would cure your problem.
Ah, ok, I misunderstood your point. Thank you.
Mar 9 '07 #14

P: n/a
Alf P. Steinbach wrote:
The std::vector code is more likely to be correct.
This would not solve my problem: understand where I'm wrong.

Ignazio
Mar 9 '07 #15

P: n/a
* Neclepsio:
Alf P. Steinbach wrote:
>The std::vector code is more likely to be correct.

This would not solve my problem: understand where I'm wrong.
See Ivan Vecerina's reply earlier in the thread.

--
A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
Mar 9 '07 #16

P: n/a
Neclepsio wrote:
Alf P. Steinbach wrote:
>The std::vector code is more likely to be correct.

This would not solve my problem: understand where I'm wrong.
As long as we don't see the code that is giving you trouble, we won't be
able to help you solve _that_ problem. See the FAQ [5.8] for helpful hints
on how to post about code that does not work.
Best

Kai-Uwe Bux
Mar 9 '07 #17

P: n/a
Neclepsio wrote:
Hi everyone.
I've made a class Matrix, which contains a pointer to the data and some
methods which all return a copy of the matrix modified in some way. The
program works quite well for small matrices, but as matrix dimension
grows up it crashes in deterministic locations depending on data,
machine and OS.
In particular, this happens when I test it with 3000x3000 matrices
(which take up more than 30MB) under three different machines/OSes. The
crash happens when much memory is still available (for example, under
cygwin, more than 1GB of virtual memory available), and I cannot think
there is no contiguous space for allocating the matrix.
I always use references and never pointers to matrices, so Matrix
objects are not dynamically allocated (the data, however, is allocated
by the constructor). Could this be a problem with stack/heap limitations?
Can someone help me with some idea on how to further investigate the
problem?

Thank you,
Ignazio
Have you got exception handling so failed allocation will cause a throw?
Is the allocation and deallocation kept within each class? (reset
pointer to NULL and throw if access tried, and have a test)
Never keep more than one permanent pointer to the memory, (get the
pointer only within the function requiring it)
Try adding index checking via class operator[] overloading, and throw
exceptions if outside the range)
Always pass the classes that make up the matrix by reference, avoids a
copy constructor and subsequent destructor which may free memory
inappropriately
Preferably use an STL class like vector which handles memory well
already, (why reinvent the wheel)
Good luck.

JB

Mar 9 '07 #18

P: n/a
"Martijn van Buul" <pi**@dohd.orgwrote in message
news:sl******************@mud.stack.nl...
:* Alf P. Steinbach:
: The std::vector code is more likely to be correct.
:
: Care to elaborate on this hollow claim?
:
: stl::vector is an array with all kinds of nasty strings attached -
much
: like the rest of this troublesome library. It's an array with an
identity
: crisis, it's an array with an attitude, but in the end it is still an
array.
: If you can't new that array yourself, then STL surely can't either.
: STL *will* end up doing the new itself.
:
: There's an enormous amount of uncalled for STL zealotry going on, with
people
: "suggesting" to use STL without even the *slightest* bit of validation
to
: their claims. The STL is *not* The One Answer. In fact, often, it's
exactly
: the WRONG answer.

One should start by using well-designed and well-tested libraries
before trying to reinvent the wheel. Especially when one is unable
to correctly manages memory.

This is just sound advice to first "get things working".
Where do you see zealotrly in there?

Until proven wrong ("show us the code..."), incorrect memory management
in the OP's code remains the most likely hypothesis.

Ivan
--
http://ivan.vecerina.com/contact/?subject=NG_POST <- email contact form

Mar 10 '07 #19

P: n/a
"Neclepsio" <to*******************@gmail.comwrote in message
news:45**********@x-privat.org...
: Thank you for replying.
:
: Ivan Vecerina wrote:
: The first thing to check is that you have provided correct
: implementations of the destructor, the copy-constructor, and the
: copy-assignment operators.
:
: Everything is ok.
How do we know?
[ your final comment does nothing to make me trust a bare claim ]

: The second?
The second is: another source of Undefined Behavior.
Third: you are actually allocating more memory than is available.

: But to make your life easier in the first place, you should consider
: using std::vector instead of a raw pointer to store the matrix data.
:
: I needed raw pointers to use the BLAS library.

Sure: &vec.front() or &vec[i]
And of course, there are C++ equivalents to BLAS, as others pointed out.

So there are several options to solve the problem.
But if you want a diagnosis, please post the source code.
Kind regards,
Ivan
--
http://ivan.vecerina.com/contact/?subject=NG_POST <- email contact form
Brainbench MVP for C++ <http://www.brainbench.com

Mar 10 '07 #20

P: n/a
Neclepsio wrote:
SNIP
I've found the problem. While in one case the memory was effectively too
little, so I got the bad_alloc exception, in the other cases it was a
mistake I'd made: I throwed a char* exception (the only left in my code,
survived to grep thanks to a parenthesis), and users simply reported a
crash I associated with what I saw on my computer.
I'm sorry to have disturbed you.

Thank you all
Ignazio

Mar 10 '07 #21

This discussion thread is closed

Replies have been disabled for this discussion.