basic memory question

[solartimba]

I am a stats man that uses C++ to analyze stock data, but I have run
into a problem regarding memory usage.

First, though, I need to give a little background info. I want to
write a program that will open up 132 files containing daily stock
market data. I estimate that each file contains around 91,000 bytes.
Multiplied by 132, this equals 12 million bytes. Given that I have
512 megs of memory, I thought that I would have no problem having all
this data in memory at the same time. But just to be sure before I
wrote all the code, I did a simple test of creating a very large array
to find out what size would cause the system to crash. The test
program's code is below:
int main()
{
float array[259100]={4};
array[258999]=2222;
cout << "\nARRAY 1 " << array[258999];

return 0;

} //end main()

This program crashes if the array size is increased beyond 259,000.
Why? The array only takes up 1.04 million bytes.

So, my question is: what do I need to do to have all that data in
memory at the same time?

Thanks

[Ivan Vecerina]

"solartimba" <ka*****@hotmail.com> wrote in message
news:1a**************************@posting.google.c om...

I am a stats man that uses C++ to analyze stock data, but I have run
into a problem regarding memory usage. .... But just to be sure before I
wrote all the code, I did a simple test of creating a very large array
to find out what size would cause the system to crash. The test
program's code is below: .... int main()
{
float array[259100]={4};
array[258999]=2222;
cout << "\nARRAY 1 " << array[258999];

return 0;

} //end main()
This function allocates the array as a local variable
of a fixed size (which typically means it is stored on
the stack if you know waht this is).
Local variables typically cannot be as large as
arrays allocated dynamically (on the heap) using
'new' or a standard container.
This program crashes if the array size is increased beyond 259,000.
Why? The array only takes up 1.04 million bytes.

So, my question is: what do I need to do to have all that data in
memory at the same time?

Use a container, such as std::vector:

#include <vector>
#include <iostream>

int main()
{
std::vector<float> array;
array.resize(1000000); // will just work
array[258999] = 2222;
std::cout << "\nARRAY 1 " << array[258999];
return 0;
}

If you find this code hard to understand,
I would recommend the book "Accelerated C++".
It should help for the type of thing you
are doing...
I hope this helps,
Ivan
--
http://ivan.vecerina.com
http://www.brainbench.com <> Brainbench MVP for C++

[Unforgiven]

solartimba wrote:

I am a stats man that uses C++ to analyze stock data, but I have run
into a problem regarding memory usage.

First, though, I need to give a little background info. I want to
write a program that will open up 132 files containing daily stock
market data. I estimate that each file contains around 91,000 bytes.
Multiplied by 132, this equals 12 million bytes. Given that I have
512 megs of memory, I thought that I would have no problem having all
this data in memory at the same time. But just to be sure before I
wrote all the code, I did a simple test of creating a very large array
to find out what size would cause the system to crash. The test
program's code is below:
int main()
{
float array[259100]={4};
array[258999]=2222;

You are allocating arrays on the stack. Stack space for a program is
(usually) limited by the compiler. You must either increase the stack size
(look at your compiler's documentation for that) or allocate the array on
the heap. The best and safest way to do that is to use std::vector.

--
Unforgiven

A: Top Posting!
Q: What is the most annoying thing on Usenet?

[klaas]

solartimba wrote:

I am a stats man that uses C++ to analyze stock data, but I have run
into a problem regarding memory usage.

First, though, I need to give a little background info. I want to
write a program that will open up 132 files containing daily stock
market data. I estimate that each file contains around 91,000 bytes.
Multiplied by 132, this equals 12 million bytes. Given that I have
512 megs of memory, I thought that I would have no problem having all
this data in memory at the same time. But just to be sure before I
wrote all the code, I did a simple test of creating a very large array
to find out what size would cause the system to crash. The test
program's code is below:

well probably ypour floating point number does not fit in a byte right?
probaly it fits in a word of four bytes. If you then multiply by four
you'll get the right amount right?

[Jerry Coffin]

In article <1a**************************@posting.google.com >,
ka*****@hotmail.com says...

I am a stats man that uses C++ to analyze stock data, but I have run
into a problem regarding memory usage.

First, though, I need to give a little background info. I want to
write a program that will open up 132 files containing daily stock
market data. I estimate that each file contains around 91,000 bytes.
Multiplied by 132, this equals 12 million bytes. Given that I have
512 megs of memory, I thought that I would have no problem having all
this data in memory at the same time. But just to be sure before I
wrote all the code, I did a simple test of creating a very large array
to find out what size would cause the system to crash.

First of all, you almost certainly want to use memory mapping to handle
this. It's theoretically not portable, but nearly every reasonably
modern OS supports it.

The problem you're running into is that many systems limit the stack
size to (considerably) less than all of memory. There are a couple of
ways of handling this: you can allocate the space statically or on the
heap, for a couple of the most obvious methods -- The easiest way is
probably to use std::vector instead of an array, in which case the space
will be allocated on the heap automatically.

--
Later,
Jerry.

The universe is a figment of its own imagination.