size of array XXX is too large

[ultr]

I need a large 3D array of structures:

struct s {
char a[40];
int b;
};
s s_array[10000][10000][100];

s_array is declared as global.

Unfortunatelly, when compiling, I get an error: "size of array
's_array' is too large"
(using GCC, but I think it doesn't matter)

But I have read, that global variables can be very large. Is it true?
And how can I make it in other way?

Thank you for answers.

[Alf P. Steinbach]

* ultr:

I need a large 3D array of structures:

struct s {
char a[40];
int b;
};
s s_array[10000][10000][100];

s_array is declared as global.

Unfortunatelly, when compiling, I get an error: "size of array
's_array' is too large"
(using GCC, but I think it doesn't matter)
Try to compute the size of your array assuming s is 42 bytes.

Hint: when you multiply by 10, just add a zero at the end of the number.

But I have read, that global variables can be very large. Is it true?
Yes.

And how can I make it in other way?

Smaller, or, wait a few years.
--
A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

[mlimber]

ultr wrote:

I need a large 3D array of structures:

struct s {
char a[40];
int b;
};
s s_array[10000][10000][100];

s_array is declared as global.

Unfortunatelly, when compiling, I get an error: "size of array
's_array' is too large"
(using GCC, but I think it doesn't matter)

But I have read, that global variables can be very large. Is it true?
And how can I make it in other way?

You can allocate static storage for variables, and it can generally be
larger since it's not allocated <ot>on the stack</ot> like automatic
variables. You could also use dynamic memory allocation (new/delete or
malloc/free) to achieve the same goal.

However, your array *is* very large, and on a typical 32-bit system it
would be (10^10)*(40+4) = 440000000000 = 0x66720B3000 which is well
beyond 32-bit pointer ranges. (Are you writing exclusively for 64-bit
machines or better?)

You could solve this problem in any number of ways. One is to access
the array in chunks so you only use part of that memory at any one
time. Another would be to break up your array behind some abstraction.
Etc. Etc. Give us a better idea of what you're doing, and perhaps we
can make some better suggestions.

Cheers! --M

[Mark P]

ultr wrote:

I need a large 3D array of structures:

struct s {
char a[40];
int b;
};
s s_array[10000][10000][100];

s_array is declared as global.

Unfortunatelly, when compiling, I get an error: "size of array
's_array' is too large"
(using GCC, but I think it doesn't matter)

But I have read, that global variables can be very large. Is it true?
And how can I make it in other way?

Thank you for answers.

Think about how much space you're asking for. sizeof(s) is at least 40
(more actually, but I like round numbers). s_array has 10000 * 10000 *
100 = 10^10 elements. So you're asking for at least 400GB of memory. I
think you need more RAM or a better algorithm.

[Rolf Magnus]

ultr wrote:

I need a large 3D array of structures:

struct s {
char a[40];
int b;
};
s s_array[10000][10000][100];

s_array is declared as global.

Unfortunatelly, when compiling, I get an error: "size of array
's_array' is too large"
(using GCC, but I think it doesn't matter)
Do you actually have enough memory? Do you have a 64 bit platform. 32 bit
ones usually can only handle memory blocks of up to 4 GB size, which is by
far not enough for your array.
But I have read, that global variables can be very large. Is it true?
And how can I make it in other way?

Instead of holding hundreds of gigabytes in memory, you should have them on
disk and only load the parts currently needed. You probably need some
clever caching.

[JE]

ultr wrote:

I need a large 3D array of structures:

struct s {
char a[40];
int b;
};
s s_array[10000][10000][100];

<snip>

If your data is sparse, you could use a map instead of an array.

Best regards, JE

[Howard]

Instead of holding hundreds of gigabytes in memory, you should have them
on
disk and only load the parts currently needed. You probably need some
clever caching.

I don't think most of us even _have_ over 400gig of hard drive space.
Drives that big are pretty new, aren't they? (I'd suggest he find another
solution entirely.)

-Howard

[Ian Collins]

Howard wrote:

Instead of holding hundreds of gigabytes in memory, you should have them
on
disk and only load the parts currently needed. You probably need some
clever caching.

I don't think most of us even _have_ over 400gig of hard drive space.
Drives that big are pretty new, aren't they? (I'd suggest he find another
solution entirely.)

You never know, he might be running on a SunFire server with 570Gb of RAM...

--
Ian Collins.