stack vs. heap. a loaded question.

[Dan Elliott]

Hello all,

I am writing a program which needs to run as quickly as possible, but holds
a lot of data in memory (around 1GB for a usual run). Is this too much
memory to even consider putting most/all of it on the stack? Does it
matter?

Any considerations I might take into account when deciding how to allocate
the many data structures? By the way, the system will consist of a handful
of very large data structures (primarily matrices and vectors).

Historically, I have always used the heap. This was more a result of my C
background than anything else. However, I am now writing a system which
will use only C++ and OO practices. Therefore, I feel I have a new choice
to make. Amy assistance to make this design as efficient as possible would
be greatly appreciated.

Thank you,

dan elliott

[Daniel L Elliott]

Thank you for the education everyone! These excellent posts will help me
create the best-possible design.

- dan elliott

On Wed, 17 Nov 2004 19:57:17 -0800, E. Robert Tisdale wrote:

Prash wrote:

Stack is there for automatic variables and its size is limited.
So, I suggest that you use heap for your data storage.

Actually, automatic storage [the program stack]
and free storage [the heap] share the same virtual memory.
The typical program stack grows up from the bottom of virtual memory
and the heap grows downward to the top of the stack.

The stack size may be limited but can be easily increased
using shell commands or compiler options.
For example, on a Linux workstation:

> limit stacksize

stacksize 10240 kbytes

> limit stacksize unlimited
> limit stacksize

stacksize unlimited

If [you] have performace to consider, then see to it that some time
object creation and deletion might take and you should minimize on that;
like, instead of allocation 10 small memory chunks,
you can create a larger chunk of memory and then divide it by yourself.

Also, I'd suggest you not to use variable sized arrays
as they are implemented in libraries using lists or something
and access to them is definetly slower
than the fixed sized conventional arrays.

No!

C99 style variable size arrays

http://gcc.gnu.org/ml/gcc/2004-05/msg00746.html

are allocated from free storage [the stack]
and have performance characteristics similar to "convention al arrays".

Also, maybe you can have a look at the GNU Scientific library
as an option for components. www.gnu.org/software/gsl/

You may as well check out
the Vector, Signal and Image Processing Library

http://www.vsipl.org/

or, better yet,
the High Performance Embedded Computing Software Initiative

http://www.hpec-si.org/

which actually proposes a C++ language binding.

[Daniel L Elliott]

Ah yes! Very helpful. Thank you.

- dan elliott

On Fri, 19 Nov 2004 16:23:33 +0100, Jacek Dziedzic wrote:

Daniel L Elliott wrote:

In what way is the "stack" limited in size
that the "heap" is not? I sorry if this is something I should have
learned as an undergrad.

As far as I understand it, upon the start of your program a fixed
amount of memory is denoted as 'stack' and the rest of the memory
is denoted as heap. In that way, stack is usually limited to a
certain amount -- often only 1MB in order not to waste too much
memory that could be used as heap (because you can't shrink the
stack if it is not used).

That way you get a limited amount of stack storage. This can
bite you in a cruel manner if you don't remember about it.
Under Linux, when the stack is overfilled you usually only
get "Aborted" and your program dies instantly.

The size of the stack is usually controlled via a compiler
switch.

You might try to run these two programs:

// program 1
// This tries to have a 50MB structure on stack
// Will most likely crash
void foo() {
char bigarray[50000000];
bigarray[0]=0; // use bigarray to make sure the
// compiler does not optimize it away
}

int main() {
foo();
}

// program 2
// This tries to allocate a 50MB structure on the heap
// Will most likely succeed, if you have 50MB spare memory
#include<iostre am>
void foo() {
try{
char* bigarray = new char[50000000];
}
catch(std::bad_ alloc) {
std::cerr << "Shit, not enough memory" << std::endl;
}
// use bigarray here...
}

int main() {
foo();
}

HTH,
- J.