fgets and problems reading into array

[Eigenvector]

I've been dinking around with this code for a while now and I can't seem to
figure out where the problem lies. The original problem was that I for some
reason or another couldn't allocate few 80,000,000 element arrays. Normally
that shouldn't be a problem but it was. So I tried to do some calloc calls
and everything went south from there.

Essentially I'm trying to read in a big file, stick each line into an array,
then from then on do stuff to it. The do stuff part should work without a
hitch, if I can just get that read in part to work.

Here's the code.
#include <stdio.h>
#include <string.h>

int main()
{
int i=0;
int j=0;

int eof_marker=0;

char *array[1000000];
char *out_array[1000000];

char input_line[80];

FILE *fp;
FILE *fp_out;
/*************** *************** *************** ****/

fp=fopen("/var/tmp/cpu.out", "r");
fp_out=fopen("/var/tmp/cpu.output2", "w");

for (i=0; i<1000000; i++)
array[i]=calloc(80, sizeof(char));

while(fgets(inp ut_line, sizeof(input_li ne), fp) != NULL)
{
strcat(array[i], input_line); /* At this line the code core dumps */
i++;
}
eof_marker=i;
}

Please forgive M$'s abominable text parsing, my neat indenting doesn't seem
to have survived intact.

I don't think I understand the logic behind malloc and calloc. I've read
the FAQ and a few past threads from this group already - which is where my
syntax comes from mostly. It just doesn't seem to be clicking.

Rob

[Artie Gold]

Eigenvector wrote:

I've been dinking around with this code for a while now and I can't seem to
figure out where the problem lies. The original problem was that I for some
reason or another couldn't allocate few 80,000,000 element arrays. Normally
that shouldn't be a problem but it was. So I tried to do some calloc calls
and everything went south from there.

Essentially I'm trying to read in a big file, stick each line into an array,
then from then on do stuff to it. The do stuff part should work without a
hitch, if I can just get that read in part to work.

Here's the code.
#include <stdio.h>
#include <string.h>

int main()
{
int i=0;
int j=0;

int eof_marker=0;

char *array[1000000];
char *out_array[1000000];

char input_line[80];

FILE *fp;
FILE *fp_out;
/*************** *************** *************** ****/

fp=fopen("/var/tmp/cpu.out", "r");
fp_out=fopen("/var/tmp/cpu.output2", "w");

for (i=0; i<1000000; i++)
array[i]=calloc(80, sizeof(char)); ^^^^^^^^^^^^
sizeof(char) is 1 by definition.

....but back to your problem. What is the value of `i' here?

while(fgets(inp ut_line, sizeof(input_li ne), fp) != NULL)
{
strcat(array[i], input_line); /* At this line the code core dumps */
BOOM!
i++;
}
eof_marker=i;
}

Please forgive M$'s abominable text parsing, my neat indenting doesn't seem
to have survived intact.

I don't think I understand the logic behind malloc and calloc. I've read
the FAQ and a few past threads from this group already - which is where my
syntax comes from mostly. It just doesn't seem to be clicking.

HTH,
--ag

--
Artie Gold -- Austin, Texas

[Kelsey Bjarnason]

[snips]

On Fri, 25 Jul 2003 18:25:30 -0700, Eigenvector wrote:

for (i=0; i<1000000; i++)
array[i]=calloc(80, sizeof(char));
Three items on one line...

1) why not allocate the length of the line actually read in? That way
short lines don't waste space, long lines don't overflow.

2) No check for calloc failure.

3) sizeof(char)? Just use 1.
while(fgets(inp ut_line, sizeof(input_li ne), fp) != NULL)
{
strcat(array[i], input_line); /* At this line the code core dumps

*/

Assuming calloc properly zero-inits the buffer, that should be okay...
except you don't even know if the allocation worked or not.

'Course, the fact that i currently points past the _last_ element of the
array might be an issue...

[Eigenvector]

"Kelsey Bjarnason" <ke*****@xxnosp amyy.lightspeed .bc.ca> wrote in message
news:pa******** *************** *****@xxnospamy y.lightspeed.bc .ca...

[snips]

On Fri, 25 Jul 2003 18:25:30 -0700, Eigenvector wrote:

for (i=0; i<1000000; i++)
array[i]=calloc(80, sizeof(char));

Three items on one line...

1) why not allocate the length of the line actually read in? That way
short lines don't waste space, long lines don't overflow.

2) No check for calloc failure.

3) sizeof(char)? Just use 1.

while(fgets(inp ut_line, sizeof(input_li ne), fp) != NULL)
{
strcat(array[i], input_line); /* At this line the code core dumps

*/

Assuming calloc properly zero-inits the buffer, that should be okay...
except you don't even know if the allocation worked or not.

'Course, the fact that i currently points past the _last_ element of the
array might be an issue...

Oh, crap!!!!!!! I didn't even think of that! I forgot to reset the value
for i

Please tell me that the value for 'i' was in fact the reason why my code
bombed.

You'll have to forgive some of the generic formatting and other strange
anomolies in my code - some of it was straight plagarism just to figure out
how the syntax for calloc worked. It'll get cleaned out when I actually get
the code working.

[Burne C]

"Eigenvecto r" <co*******@hotm ail.com> wrote in message news:EA******** ********@news.u swest.net...

I've been dinking around with this code for a while now and I can't seem to
figure out where the problem lies. The original problem was that I for some
reason or another couldn't allocate few 80,000,000 element arrays. Normally
that shouldn't be a problem but it was. So I tried to do some calloc calls
and everything went south from there.

Essentially I'm trying to read in a big file, stick each line into an array,
then from then on do stuff to it. The do stuff part should work without a
hitch, if I can just get that read in part to work.

Here's the code.
#include <stdio.h>
#include <string.h>

int main()
{
int i=0;
int j=0;

int eof_marker=0;

char *array[1000000];
char *out_array[1000000];

char input_line[80];

FILE *fp;
FILE *fp_out;
/*************** *************** *************** ****/

fp=fopen("/var/tmp/cpu.out", "r");
fp_out=fopen("/var/tmp/cpu.output2", "w");

for (i=0; i<1000000; i++)
array[i]=calloc(80, sizeof(char));

while(fgets(inp ut_line, sizeof(input_li ne), fp) != NULL)
{
strcat(array[i], input_line); /* At this line the code core dumps */
i++;
}
I think "i" is the main problem, you didn't reset i = 0 before the while loop. Also, why do you use
strcat instead of strcpy ? ( I know that it is not a problem, because calloc fills the array block
with all zero. )

BTW, you should add code to trap the failure of fopen and calloc.

eof_marker=i;
}

Please forgive M$'s abominable text parsing, my neat indenting doesn't seem
to have survived intact.

I don't think I understand the logic behind malloc and calloc. I've read
the FAQ and a few past threads from this group already - which is where my
syntax comes from mostly. It just doesn't seem to be clicking.

Rob

--
BC

[Randy Howard]

In article <EA************ ****@news.uswes t.net>, co*******@hotma il.com
says...

char *array[1000000];
char *out_array[1000000];

You've already received several good responses to the program operation,
but you should be aware that there is no guarantee that you can declare
an array this large and have it work (portably). A fair number of
compilers will bomb out on arrays that large. I believe the limit for
the size of an array declared this way is 64KB in C99, and half that for
C89, but I can't recall the "chapter and verse" on that as I can't
remember the last time I even attempted to declare an array this large in
this way.

clc Faq entries 16.3 and 19.23 cover this, perhaps a bit obliquely.
(http://www.eskimo.com/~scs/C-faq/faq.html)

[Eigenvector]

"Randy Howard" <ra**********@F OOmegapathdslBA R.net> wrote in message
news:MP******** *************** *@news.megapath dsl.net...

In article <EA************ ****@news.uswes t.net>, co*******@hotma il.com
says...

char *array[1000000];
char *out_array[1000000];
You've already received several good responses to the program operation,
but you should be aware that there is no guarantee that you can declare
an array this large and have it work (portably). A fair number of
compilers will bomb out on arrays that large. I believe the limit for
the size of an array declared this way is 64KB in C99, and half that for
C89, but I can't recall the "chapter and verse" on that as I can't
remember the last time I even attempted to declare an array this large in
this way.

Well that's kind of lame. I understand that not every computer has loads of
memory, but putting an actual limit or even recognizing a limit seems
artificial and unnecessary. I guess programmers need to make their code as
efficient as possible at all times, but I intensely dislike those types of
standards.

64Kb is pretty damn small, maybe not for a PC, but certainly my 16 GB
internal memory UNIX server shouldn't have this limit imposed on it. But
like you said, its not guarenteed, and there are other more efficient ways
of doing the job.
clc Faq entries 16.3 and 19.23 cover this, perhaps a bit obliquely.
(http://www.eskimo.com/~scs/C-faq/faq.html)

[Richard Heathfield]

Eigenvector wrote:

<snip>

64Kb is pretty damn small, maybe not for a PC, but certainly my 16 GB
internal memory UNIX server shouldn't have this limit imposed on it.

The limit is not imposed on your implementation. It is imposed on programs
which the programmer requires to be portable to any conforming C compiler.

<snip>

--
Richard Heathfield : bi****@eton.pow ernet.co.uk
"Usenet is a strange place." - Dennis M Ritchie, 29 July 1999.
C FAQ: http://www.eskimo.com/~scs/C-faq/top.html
K&R answers, C books, etc: http://users.powernet.co.uk/eton

[Kevin Easton]

Eigenvector <co*******@hotm ail.com> wrote:

"Randy Howard" <ra**********@F OOmegapathdslBA R.net> wrote in message
news:MP******** *************** *@news.megapath dsl.net...

In article <EA************ ****@news.uswes t.net>, co*******@hotma il.com
says...

> char *array[1000000];
> char *out_array[1000000];

You've already received several good responses to the program operation,
but you should be aware that there is no guarantee that you can declare
an array this large and have it work (portably). A fair number of
compilers will bomb out on arrays that large. I believe the limit for
the size of an array declared this way is 64KB in C99, and half that for
C89, but I can't recall the "chapter and verse" on that as I can't
remember the last time I even attempted to declare an array this large in
this way.

Well that's kind of lame. I understand that not every computer has loads of
memory, but putting an actual limit or even recognizing a limit seems
artificial and unnecessary. I guess programmers need to make their code as
efficient as possible at all times, but I intensely dislike those types of
standards.

Don't think of it as telling you your array won't work if it's bigger
than 64kB - instead think of it as promising that you that your array
*will* work if it's smaller than 64kB.

- Kevin.

[Kelsey Bjarnason]

[snips]

On Sun, 27 Jul 2003 05:50:49 -0700, Eigenvector wrote:

Well that's kind of lame. I understand that not every computer has loads of
memory, but putting an actual limit or even recognizing a limit seems
artificial and unnecessary. I guess programmers need to make their code as
efficient as possible at all times, but I intensely dislike those types of
standards.

64Kb is pretty damn small, maybe not for a PC, but certainly my 16 GB
internal memory UNIX server shouldn't have this limit imposed on it. But
like you said, its not guarenteed, and there are other more efficient ways
of doing the job.

It's not a limit; you can (perhaps) create objects terabytes in size.
What it is is a statement that you cannot create objects larger than X and
expect them to actually work across all conforming implementations .