473,324 Members | 2,548 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,324 software developers and data experts.

correcting code for larger 2D arrays help

Hello all, I'm a physicist whose rewriting a numerical simulation,
previously written in IDL, in C with the goal reducing runtime. As you
may imagine, my C programming skills are quite poor but I am learning.
My problem is this. I had sucessfully written a C code that worked
( with considerable speedup over the IDL version ) using an array size
of 100x100. However, I am working towards making the array size closer
to 300x300. Since I had defined 2D arrays in my code as

double array[nCols][nRows];

upon increasing nRows above approximately 250 with nCols = 100 the
program was segfaulting. After some reading I came to the conclusion
that I was creating arrays that were too big for the stack and should
instead be allocated memory on the heap? It's quite possible that i'm
very wrong here so please point out my mistakes. To allocate space on
the heap for the arrays, and for conveniance in array indexing, I
chose to replace the static array creation lines with a malloc call
like (found in the cfaq)

double **array = malloc ( nCols * sizeof ( *array ) );
array[0] = malloc ( nCols * nRows * sizeof ( **array ) );
for ( ii = 1; ii < nCols; ++ii )
array[ii] = array[0] + ii * nRows;

such that indexing the array may be done according to

for ( ii = 0; ii < nCols; ++ii )
{
for ( jj = 0; jj < nRows; ++jj)
{
array[ii][jj] = 0.0;
}
}

Now this also appeared to work. Upon altering the code to use the
malloc statement instead of the static [][] approach I tested the
outputs of the simultaion. They were fine up to a point where I had
converted most of the static array creation lines to the malloc method
above. However, trying to convert more caused a problem. Irrelevant of
which additional array I tried to allocate space for on the heap, the
simulation results would become rubbish. It does not appear to be a
problem with the simulation itself, since the problem occurs with
small array sizes as well which work fine using a static array
declaration, but rather with my poor attempt at memory management in
C. So my questions are:

1. Are arrays of size 300x300 large enough to overwhelm the space
available on the stack (if relevant my machine is an Intel Core 2 Duo
running linux)?

2. What is the standard approach to creating large 2D arrays, i.e., is
the approach I have chosen appropriate?

3. Is it possible I have allocated too many 2D arrays (perhaps 30 in
all) using the above malloc approach and that is the cause of my
simulation woes?

Any help would be great as my brain is slowly melting away.
Dav.

May 30 '07 #1
2 1979
Dr Dav said:

<snip>
>
1. Are arrays of size 300x300 large enough to overwhelm the space
available on the stack (if relevant my machine is an Intel Core 2 Duo
running linux)?
Typical double on a modern desktop system, is 8 bytes. 8 * 300 * 300 is
a mere 720,000 bytes. Nowadays, that's peanuts for *dynamic*
allocation, but could easily cause problems with static allocation,
yes.
2. What is the standard approach to creating large 2D arrays, i.e., is
the approach I have chosen appropriate?
If you got it from the FAQ, it should be fine (although I must admit
it's not how I'd have done it myself). I note, however, that your
example code doesn't ascertain whether the memory allocation succeeded.
Allocation requests can and *do* fail, so you should check to see
whether malloc gave you a null pointer rather than hope it didn't.
3. Is it possible I have allocated too many 2D arrays (perhaps 30 in
all) using the above malloc approach and that is the cause of my
simulation woes?
You may get more joy out of allocating smaller blocks, and allocating
more of them to compensate. Consider:

struct array_double_2d
{
size_t y;
size_t x;
double **data;
};

int ok = 1;
array_double_2d arr = { 300, 300 };
arr.data = malloc(arr.y * sizeof *arr.data);
if(arr.data != NULL)
{
size_t i = 0;
while(ok && i < arr.y)
{
arr.data[i] = malloc(arr.x * sizeof *arr.data[i]);
if(arr.data[i] == NULL)
{
ok = 0;
}
}
}
else
{
ok = 0;
}

This allocates 300 blocks of (in your case) 2400 bytes, rather than one
great big block of 720000 bytes. *One* 720,000 byte block is no big
deal, but lots of them could become a problem. By splitting your
allocation request down into lots of much smaller requests, you make it
easier for the memory manager to satisfy those requests.

--
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at the above domain, - www.
May 30 '07 #2
Richard Heathfield <rj*@see.sig.invalidwrites:
Dr Dav said:

<snip>
>>
1. Are arrays of size 300x300 large enough to overwhelm the space
available on the stack (if relevant my machine is an Intel Core 2 Duo
running linux)?

Typical double on a modern desktop system, is 8 bytes. 8 * 300 * 300 is
a mere 720,000 bytes. Nowadays, that's peanuts for *dynamic*
allocation, but could easily cause problems with static allocation,
yes.
[big snip]

There are three storage durations, automatic, static, and allocated.

Automatic storage duration refers to objects declared locally within a
function or block (sometimes referred to as "stack"). Static storage
duration refers to objects that exist throughout the lifetime of the
program; they're declared with the keyword "static" or outside any
function. Allocated storage duration refers to objects allocated by
calls to malloc (or calloc, or realloc) (sometimes referred to as
"heap"),

An implementation is likely to place different limits on these three
kinds of storage duration; automatic duration often has the lowest
limit. As long as you can deal with the differing semantics,
switching from automatic to static storage duration *might* solve your
problem.

Some systems also provide ways to change memory allocation limits,
either system-wide or for a single process. <OT>On Unix-like systems,
see "limit" or "ulimit">,<OT>

Also, if the bounds of your arrays are constant and you choose to use
dynamic allocation (malloc), that can simplify your code. Many
examples of dynamic allocation of two-dimensional arrays are designed
to allow for both dimensions being determined at execution time. If
you know in advance that you want your arrays to be exactly 300 x 300,
you can use a single allocation. For example:

#include <stdio.h>
#include <stdlib.h>

#define MATRIX_SIZE 300
typedef double matrix[MATRIX_SIZE][MATRIX_SIZE];

int main(void)
{
matrix *m;
int i, j;
m = malloc(sizeof *m);
if (m) {
printf("Allocated %lu bytes\n", (unsigned long)sizeof *m);
}
else {
fprintf(stderr, "Allocation failed\n");
exit(EXIT_FAILURE);
}
for (i = 0; i < MATRIX_SIZE; i ++) {
for (j = 0; j < MATRIX_SIZE; j ++) {
(*m)[i][j] = i + j;
}
}
printf("(*m)[123][234] = %g\n", (*m)[123][234]);
return 0;
}

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
May 30 '07 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

53
by: Cardman | last post by:
Greetings, I am trying to solve a problem that has been inflicting my self created Order Forms for a long time, where the problem is that as I cannot reproduce this error myself, then it is...
6
by: gsb | last post by:
Don't know if this is the right place to post this JavaScript issue. If not, could someone point me in the right direction please. I am trying to make a "cross browser compliant" floating...
3
by: Sourin | last post by:
Hi all, I am trying to write code for my experiments. My work involves huge datasets, and as such my code needs to be memory efficient. I did some hand calculations regarding the amount of...
26
by: Martin Jørgensen | last post by:
Hi, I don't understand these errors I get: g++ Persort.cpp Persort.cpp: In function 'int main()': Persort.cpp:43: error: name lookup of 'j' changed for new ISO 'for' scoping Persort.cpp:37:...
1
by: vv1 | last post by:
Write a C program for reading in a message string (with no blanks) and decoding the message. Store the decoded message in another string called outString. After decoding is complete, print...
0
by: vp1 | last post by:
Write a C program for reading in a message string (with no blanks) and decoding the message. Store the decoded message in another string called outString. After decoding is complete, ...
10
by: Frank | last post by:
I've done this a few times. In a solution I have a project, Say P1, and need another project that will contain much code that is similar to that of P1. I hope no one gets hung up on why I...
1
by: srinivasarv | last post by:
Dear Friends, kindly help me in correcting the following code This code is for printing the report from the combo box after selection which connects to different reports as selected. I have...
7
by: ashjas | last post by:
Hello, I have run this logic on turboc++ 3.0 and it is working fine on it but its not running on msvs2008 c++. i am not able to assign the value like this *temp=*main,where main and temp are char...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.