473,320 Members | 1,707 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Storing large number of values in 2D array

I want to store the (x,y) coordinates of about 10,000 points in a 2D
array.How can I store them and retrieve them later?

Oct 20 '06 #1
12 3951

rajus wrote:
I want to store the (x,y) coordinates of about 10,000 points in a 2D
array.How can I store them and retrieve them later?
I'm afraid that your question doesn't relate specifically to the use of
the C langage, and is off topic in this forum. You might try one of the
comp.programming groups where they discuss algorithms and data
structures.

But, before you go, why don't you think about a sparse matrix
implementation?
Something like
struct { long x; long y; } coord[10000];

HTH
--
Lew Pitcher

Oct 20 '06 #2
rajus wrote:
I want to store the (x,y) coordinates of about 10,000 points in a 2D
array.How can I store them and retrieve them later?
You have 10 000 x 2 points i.e. 20 000 points to store.
To minimize space you could use 12 bits (coordinates up
to 4096x4096) and store 5 points per 64 bits, making it
around 32 000 bytes for the 2D array.

Using only 10 bits (1024x1024 coordinates) you would store
6 points per 64 bits unit, around 26 666 bytes for the array.

You would need to develop a small package that reads any sequence of
10 or 12 bits in the array, making a function like

uint32 GetPointsCoords(int x,int y)
{
// Here you calculate the offset in the array
// of your point, and return a 32 bit result
// containing in the upper 16 bits the y coordinate
// and in the lower 16 bits the x coordinate
// for instance.
}

You can save yourself development time of course if you
just make

unsigned short array[10000][2];

This would take 40 000 bytes, an increase of 14 000
but no development effort, you just index the array:

x = array[2357][0];
y = array[2357][1];

With short in 16 bits, you would get enough coordinate space to go
to 65535 x 65535 screens. The largest screens today go to
2500 x 2000 so your software will last (for a while...)

jacob
Oct 20 '06 #3
On Fri, 20 Oct 2006 20:46:28 +0200, jacob navia
<ja***@jacob.remcomp.frwrote:
>rajus wrote:
>I want to store the (x,y) coordinates of about 10,000 points in a 2D
array.How can I store them and retrieve them later?
You have 10 000 x 2 points i.e. 20 000 points to store.
To minimize space you could use 12 bits (coordinates up
to 4096x4096) and store 5 points per 64 bits, making it
around 32 000 bytes for the 2D array.
Of course, the extra code required to do this would probably make up
for the saved space. I like Lew's suggestion, using whatever size
integer is appropriate.

--
Al Balmer
Sun City, AZ
Oct 20 '06 #4
rajus wrote:
I want to store the (x,y) coordinates of about 10,000 points in a 2D
array.How can I store them and retrieve them later?
#include <stdlib.h>
#include <stdio.h>

typedef struct point {
double x;
double y;
} point, *pointAddr;

static size_t dim = 10000u;

int main(void)
{
size_t i;

/* The use of calloc to zero array of floats is not portable. */
/* Allocate array with malloc() */
pointAddr array = malloc(dim * sizeof( point) );
/* Check for success */
if (array == NULL) {
puts("FATAL ERROR: Out of memory.");
exit(EXIT_FAILURE);
}
/* Initialize array to a known state. */
for (i = 0; i < dim; i++) {
array[i].x = 0;
array[i].y = 0;
}

/* do whatever you like with the array here... */

return 0;
}

Oct 20 '06 #5

Lew Pitcher wrote:
rajus wrote:
I want to store the (x,y) coordinates of about 10,000 points in a 2D
array.How can I store them and retrieve them later?

I'm afraid that your question doesn't relate specifically to the use of
the C langage, and is off topic in this forum. You might try one of the
comp.programming groups where they discuss algorithms and data
structures.
If this isn't a C question, then I don't know what one is.
But, before you go, why don't you think about a sparse matrix
implementation?
Something like
struct { long x; long y; } coord[10000];
With 10,000 points, a sparse array implementation is a total waste of
time and energy.

Even with doubles that's {typically} 16*10,000 = 160,000 bytes.

Considering 1 GB of PC3200 DDR2 ECC ram at $112:
http://www.ramseeker.com/pc/index.php
that's about 1/2 cent worth of memory. Hardly worth worrying about
sparse implementations.

IMO-YMMV.
HTH
--
Lew Pitcher
Oct 20 '06 #6

rajus wrote:
I want to store the (x,y) coordinates of about 10,000 points in a 2D
array.How can I store them and retrieve them later?
Something a bit ambiguous about your post is:
"How can I store them and retrieve them later?"

You can obviously poke them into an array and pull them out when you
need them. The code I posted earlier is an example of how you might
prepare an array of 2d points for use.

But you might be talking about permanent storage. In such a case, the
C answer is to use fwrite() to save them to disk and fread() to read
them back into memory. This answer is not portable across different
systems because different systems have different binary formats. In
other words, if you write out binary data using fwrite() then you
should only read the data back into memory using fread() on machines
with similar architecture. So if you wrote the data out on a Windows
machine, it probably won't read in correctly on a SPARC machine.

Also, if you plan to have many sets of data (or need the binary
compatibility referred to above), you might want to store them in a
database.

We do not have enough information for a very clear answer if you want
to know about long term storage and retrieval.

Oct 20 '06 #7
dc*****@connx.com wrote:
[snip]
With 10,000 points, a sparse array implementation is a total waste of
time and energy.

Even with doubles that's {typically} 16*10,000 = 160,000 bytes.

Considering 1 GB of PC3200 DDR2 ECC ram at $112:
http://www.ramseeker.com/pc/index.php
that's about 1/2 cent worth of memory. Hardly worth worrying about
sparse implementations.
Time to temper my remark. I keep forgetting that much of (most?) C
programming is actually embedded work. On an embedded system, you
might have very limited resources. In such a circumstance, a skiplist
might be a good way to store a sparse vector.

There are advanced codes available for sparse matrices and vectors, but
generally they are designed for huge data sets.

Oct 20 '06 #8

dc*****@connx.com wrote:
Lew Pitcher wrote:
rajus wrote:
I want to store the (x,y) coordinates of about 10,000 points in a 2D
array.How can I store them and retrieve them later?
I'm afraid that your question doesn't relate specifically to the use of
the C langage, and is off topic in this forum. You might try one of the
comp.programming groups where they discuss algorithms and data
structures.

If this isn't a C question, then I don't know what one is.
You got that one right!

- William Hughes

Oct 20 '06 #9
dc*****@connx.com writes:
Lew Pitcher wrote:
>rajus wrote:
I want to store the (x,y) coordinates of about 10,000 points in a 2D
array.How can I store them and retrieve them later?

I'm afraid that your question doesn't relate specifically to the use of
the C langage, and is off topic in this forum. You might try one of the
comp.programming groups where they discuss algorithms and data
structures.

If this isn't a C question, then I don't know what one is.
>But, before you go, why don't you think about a sparse matrix
implementation?
Something like
struct { long x; long y; } coord[10000];

With 10,000 points, a sparse array implementation is a total waste of
time and energy.

Even with doubles that's {typically} 16*10,000 = 160,000 bytes.
Suppose each (x,y) coordinate consists of a pair of integers, each of
which can be in the range 0..999999. If the OP wants to store
information about each point in the corresponding element of a 2D
array, the array would have to have one trillion (10**12) elements.
Some sort of sparse array representation is just about mandatory.

I think the OP wants to store information about a point, and use the
coordinates of the point to retrieve it later. I might consider using
a hash table, hashing the (x,y) coordinate pair to obtain the
retrieval key.

But we need more information from the OP. How are the (x,y)
coordinates represented (int, double, whatever)? What are the
possible ranges? What information to you need for each point? How
important is fast access to each point?

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Oct 20 '06 #10
rajus wrote:
I want to store the (x,y) coordinates of about 10,000 points in a 2D
array.How can I store them and retrieve them later?
Well, declare a 2D array of points. Store points in the array. Retireve them
later from the array. What exactly is the problem that makes you ask such a
trivial question?

--
Best regards,
Andrey Tarasevich
Oct 20 '06 #11
Al Balmer wrote:
On Fri, 20 Oct 2006 20:46:28 +0200, jacob navia
<ja***@jacob.remcomp.frwrote:

>>rajus wrote:
>>>I want to store the (x,y) coordinates of about 10,000 points in a 2D
array.How can I store them and retrieve them later?

You have 10 000 x 2 points i.e. 20 000 points to store.
To minimize space you could use 12 bits (coordinates up
to 4096x4096) and store 5 points per 64 bits, making it
around 32 000 bytes for the 2D array.


Of course, the extra code required to do this would probably make up
for the saved space. I like Lew's suggestion, using whatever size
integer is appropriate.
I do not think that a simple code for accessing a 2D 12 bit
integer array would make 14 000 bytes of code in C.
At most it would take 150-200 bytes of code, maybe more in RISC
processors, but never 12 000 bytes.

But that was a memory efficient solution, good for embedded
systems where memory considerations are important.

I proposed also a 16 bit solution, not knowing precisely the context.

What is important to emphasize is that integers of bit-size other
than 8-16-32 are possible, and can be a space saving in many situations.

I have seen proposals to add to C
int:12 a;

meaning a is a 12 bit integer. This, with support for array notation of
those integers would be interesting in systems where memory is important

jacob
Oct 20 '06 #12
On Fri, 20 Oct 2006 14:04:48 -0700, Andrey Tarasevich
<an**************@hotmail.comwrote:
>rajus wrote:
>I want to store the (x,y) coordinates of about 10,000 points in a 2D
array.How can I store them and retrieve them later?

Well, declare a 2D array of points. Store points in the array. Retireve them
later from the array. What exactly is the problem that makes you ask such a
trivial question?
Trivial questions are not unwelcome here.

However, your question, "What exactly is the problem", is certainly
pertinent here, and the answer to that might make the original
question less trivial :-)

--
Al Balmer
Sun City, AZ
Oct 20 '06 #13

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Jim | last post by:
I have a very large array of floating point numbers which I am trying to store in an image type field. I am using the dataset class to directly enter the data but am having trouble trying to get...
6
by: gv | last post by:
Hi all, I need to store values in a client application then later when done send them to SQL 2000. I know there is several ways to do this but, looking for the fastest , and most effient way...
7
by: Arnold | last post by:
I need to read a binary file and store it into a buffer in memory (system has large amount of RAM, 2GB+) then pass it to a function. The function accepts input as 32 bit unsigned longs (DWORD). I...
5
by: netcoder77 | last post by:
Has anyone tried this in VB .NET or via VBScript? Can it be done? How do we handle retrieving a binary data format (the photo) using ADSI or VB .NET? All my searching on the net yielded no useful...
2
by: gh | last post by:
Hi, I have a string variable which contains n number of comma delimited elements and I would like to store each element into an array but I could not figure how to do it. for example,...
20
by: Martin Jørgensen | last post by:
Hi, I'm reading a number of double values from a file. It's a 2D-array: 1 2 3 4 5 6 7 ------------- 1 3.2 2 0 2.1 3 9.3 4
2
by: assgar | last post by:
Hi Developemnt on win2003 server. Final server will be linux Apache,Mysql and PHP is being used. I use 2 scripts(form and process). The form displays multiple dynamic rows with chechboxs,...
7
by: pereges | last post by:
I've to store an array of structures: typedef struct { double origin; double e_field_at_origin_real, e_field_at_origin_imag; double direction; double pathlength; int depth; }ray;
6
by: Carl Banks | last post by:
I was wondering if anyone had any advice on this. This is not to study graph theory; I'm using the graph to represent a problem domain. The graphs could be arbitrarily large, and could easily...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.