Hello,
Say I have 1000 text files and each is a list of 32768 integers.
I have written a C program to read this data into a large matrix. I am
using fopen in combination with fscanf to read the data in. However, it
takes about 20 seconds to complete and I wonder if there is a faster way.
For example, I found that I could use 'fread' to read the data into a
string that looks like this:
91\n212\n34\n40\n25\n100\n300\n ... \0
and it is nearly instantaneous. However, is there a quick way to
convert this string into an array of doubles?
Thanks.
-Matt
Here is my existing code. Sorry if it is ugly. It is the first C code
I've written in a long time:
#include <stdio.h>
#include <stdlib.h>
#define nrows 32768L
#define nfiles 1000L
int main(void)
{
double *data;
unsigned long filenum, pos;
char filename[20];
FILE *fp;
int i;
/* Create output matrix */
data = (double *) malloc((size_t)((nrows*nfiles)*sizeof(double)));
for(filenum=1; filenum<=nfiles; ++filenum) {
// Determine current file name
sprintf(filename, "data%lu.dat", filenum);
// Open the file
fp = fopen (filename,"r");
pos = nrows*(filenum-1L);
for (i=0; i<nrows; ++i)
fscanf(fp, "%lf", data+i+pos);
fclose(fp);
}
// De-allocate memory
free((char*) (data));
return 0;
}