reading multiple files

[leorulez]

Is there any way to read multiple files (more than 1000 files) and then
write into one single output file using C? Right now in my program, I
have a loop which asks for the filename and writes into the output file
but this is tedious. Imagine typing 1000 filenames...is there a
efficient way to do this??

Thanks

[Roberto Waltman]

On 24 May 2006 08:33:51 -0700, le******@gmail. com wrote:

Is there any way to read multiple files (more than 1000 files) and then
write into one single output file using C? Right now in my program, I
have a loop which asks for the filename and writes into the output file
but this is tedious. Imagine typing 1000 filenames...is there a
efficient way to do this??

Thanks

<OFF-TOPIC>
You are asking, "how?"
The first question that comes to my mind is "why?"
This is trivial to do with the built in facilities of most command
shells. What problem are you trying to solve? Why do you want to do
it "using C"?
</OFF-TOPIC>

[leorulez]

I was thinking of using C because I can do better in C compared to
other languages. Could you please brief about the built in facilities
of the command shells to do this task?

Roberto Waltman wrote:

On 24 May 2006 08:33:51 -0700, le******@gmail. com wrote:

Is there any way to read multiple files (more than 1000 files) and then
write into one single output file using C? Right now in my program, I
have a loop which asks for the filename and writes into the output file
but this is tedious. Imagine typing 1000 filenames...is there a
efficient way to do this??

Thanks

<OFF-TOPIC>
You are asking, "how?"
The first question that comes to my mind is "why?"
This is trivial to do with the built in facilities of most command
shells. What problem are you trying to solve? Why do you want to do
it "using C"?
</OFF-TOPIC>

[SM Ryan]

le******@gmail. com wrote:
# Is there any way to read multiple files (more than 1000 files) and then
# write into one single output file using C? Right now in my program, I
# have a loop which asks for the filename and writes into the output file
# but this is tedious. Imagine typing 1000 filenames...is there a
# efficient way to do this??

The file name is fopen is an (char*) expression. It can be a
string constant or anything else that is (char*). For example
to open the file with names like fwxyzDDD,
int i; for (i=0; i<1000; i++) {
static char F[] = "fwxyz%03d" ;
char f[sizeof F+3];
sprintf(f,F,i);
FILE *fn = fopen(f,"r"); if (!fn) {perror(fn); continue;}
...
fclose(fn);
}
Or if you have list of file names file (for example the output of
the unix find command), you can fgets the file names, and open
the file name you fgets.

--
SM Ryan http://www.rawbw.com/~wyrmwif/
Death is the worry of the living. The dead, like myself,
only worry about decay and necrophiliacs.

[Roberto Waltman]

le******@gmail. com wrote:

Could you please brief about the built in facilities
of the command shells to do this task?
Roberto Waltman wrote:

le******@gmail. com wrote:

>Is there any way to read multiple files (more than 1000 files) and then
>write into one single output file using C?

<OFF-TOPIC>

This is trivial to do with the built in facilities of most command

shells. What problem are you trying to solve? Why do you want to do
it "using C"?

<OFF-TOPIC>
Let's assume you want to concatenate all "*.log" files in a directory
into a single file:
In a Windows 2000 command window:
copy *.log all_together.no w
In Linux/Unix/???BSD under a bash shell:
cat *.log >all_together.n ow
It is very easy to extend this to copy only files whose names match a
pattern, or where modified between specific dates, or in several
directories, etc.
But these issues have nothing to do with the C language, please post
again in a newsgroup dedicated to the computer platform you are using.
</OFF-TOPIC>

[leorulez]

SM Ryan wrote:

le******@gmail. com wrote:
# Is there any way to read multiple files (more than 1000 files) and then
# write into one single output file using C? Right now in my program, I
# have a loop which asks for the filename and writes into the output file
# but this is tedious. Imagine typing 1000 filenames...is there a
# efficient way to do this??

The file name is fopen is an (char*) expression. It can be a
string constant or anything else that is (char*). For example
to open the file with names like fwxyzDDD,
int i; for (i=0; i<1000; i++) {
static char F[] = "fwxyz%03d" ;
char f[sizeof F+3];
sprintf(f,F,i);
FILE *fn = fopen(f,"r"); if (!fn) {perror(fn); continue;}
...
fclose(fn);
}
Or if you have list of file names file (for example the output of
the unix find command), you can fgets the file names, and open
the file name you fgets.

--
SM Ryan http://www.rawbw.com/~wyrmwif/
Death is the worry of the living. The dead, like myself,
only worry about decay and necrophiliacs.

Lets consider I have a list of all the file names in a file called
list.txt. I wrote the following code

fp2=fopen("list .txt","r");
while ( fgets(line, sizeof(line), fp2) != NULL)
{
printf("%s", line);
strcpy(name,lin e);
printf("%s", name);
printf("%d\n",s trlen(name));
fp = fopen(name, "r" );
fp1 = fopen("out.txt" ,"a");
if (fp==NULL)
{
printf("error in opening\n");
exit(1);
}
.....
.....
.....

The file pointer returns null resulting in "error in opening". when I
checked the string length of the filename its always 2 characters more
then the original length. Could you please tell me how I can use the
variable "name" (or "line") in fopen.

[Richard Heathfield]

Before I tackle your question, could I just say that I'm rather concerned
that the reaction to your perfectly legitimate question was to say: (a) you
should use your shell to do this, and (b) shells are off-topic, clear off.
If that isn't hostility, I don't know what is.

Folks, we have no evidence that this guy even /has/ a shell that can
concatenate many files into one large one. The C Standard imposes no such
requirement on implementations . Your assumption that he has a shell is
completely off-topic, and those who made it should be ashamed of
themselves.

It's the worst kind of "topic-cop" behaviour - a bit like sneaking a bag of
class A drugs into a guy's pocket and then busting him for possession.

The question is reasonable and, in my opinion, topical.

Okay - the priority is to get the filenames into some kind of
computer-readable list, and le******@gmail. com appears to have achieved
that. So now he just has a tiny problem getting in his way. The fix,
le******@gmail. com, is easy, you'll be glad to hear. See below.

le******@gmail. com said:

<snip>

Lets consider I have a list of all the file names in a file called
list.txt. I wrote the following code
Excuse me a minute:

char *end = NULL;

Thanks. (All will become clear shortly!)

fp2=fopen("list .txt","r");
The fopen call might fail, in which case fp2 will be NULL. You should check
for this.
while ( fgets(line, sizeof(line), fp2) != NULL)
Make sure line is at least FILENAME_MAX + 1 bytes in size.
{
printf("%s", line);
strcpy(name,lin e);

Yes, you've got the filename. Unfortunately, you've also got a newline
character. One easy way to get rid of it is:

end = strchr(name, '\n'); /* find the newline */
if(end != NULL)
{
*end = '\0'; /* chop it off! */
}
else
{
you might want to wonder about why it didn't find a newline; basically, it
means that the filename was so long it couldn't all fit in the buffer, in
which case the rest of the filename will be handed to you on the next
fgets call. Messy. Better to make sure your buffer's big enough to start
with (see above).
}

I hope that helps.

--
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at above domain (but drop the www, obviously)

[Eric Sosman]

le******@gmail. com wrote On 05/24/06 12:29,:

Lets consider I have a list of all the file names in a file called
list.txt. I wrote the following code

fp2=fopen("list .txt","r");
while ( fgets(line, sizeof(line), fp2) != NULL)
{
printf("%s", line);
strcpy(name,lin e);
printf("%s", name);
printf("%d\n",s trlen(name));
fp = fopen(name, "r" );
fp1 = fopen("out.txt" ,"a");
if (fp==NULL)
{
printf("error in opening\n");
exit(1);
}
....
....
....

The file pointer returns null resulting in "error in opening". when I
checked the string length of the filename its always 2 characters more
then the original length. Could you please tell me how I can use the
variable "name" (or "line") in fopen.

When fgets() reads a line, it stores the entire line
including the '\n' at the end. Since the name of the
file you are trying to open is probably "foo.txt" and not
"foo.txt\n" , you need to search for and remove that '\n'
character. Something like this will do it:

char *p;
...
p = strchr(line, '\n');
if (p != NULL)
*p = '\0';

This much would explain one extra character per line,
but you've reported seeing two. It may be that you've mis-
counted, and that simply removing the '\n' will solve your
problem. A more troublesome possibility is that "list.txt"
may have originated on a system that uses a different line-
ending convention from yours, and that it wasn't properly
translated to your system's conventions when it was moved
there. For example, Windows systems terminate lines with
the two-character pair '\r','\n', while POSIX systems use
'\n' by itself. If the '\r','\n' pairs weren't translated
to single '\n' characters, then the lines you read from the
file will look like "foo.txt\r\ n"; removing the '\n' will
leave you with "foo.txt\r" , and fopen() will probably still
fail.

If this is the case, the best cure is to clean up the
procedure that brought you "list.txt" in the first place, so
the line endings will be translated properly. If that's not
possible, you can add a little more code to remove '\r' (if
it's present), just as the code above removes '\n'. This
isn't perfectly bullet-proof, though, because some systems
use even stranger line-ending conventions than '\r','\n' --
for example, a system that used '\n','\r' would produce a
file that if read on a POSIX system would look like

first line\n
\rsecond line\n
\rthird line\n
...

It's for reasons like this that I recommend revisiting your
file-transfer process instead of trying to outguess the
remote system's line-ending conventions.

--
Er*********@sun .com

[SM Ryan]

le******@gmail. com wrote:
#
# SM Ryan wrote:
# > le******@gmail. com wrote:
# > # Is there any way to read multiple files (more than 1000 files) and then
# > # write into one single output file using C? Right now in my program, I
# > # have a loop which asks for the filename and writes into the output file
# > # but this is tedious. Imagine typing 1000 filenames...is there a
# > # efficient way to do this??
# >
# > The file name is fopen is an (char*) expression. It can be a
# > string constant or anything else that is (char*). For example
# > to open the file with names like fwxyzDDD,
# > int i; for (i=0; i<1000; i++) {
# > static char F[] = "fwxyz%03d" ;
# > char f[sizeof F+3];
# > sprintf(f,F,i);
# > FILE *fn = fopen(f,"r"); if (!fn) {perror(fn); continue;}
# > ...
# > fclose(fn);
# > }
# > Or if you have list of file names file (for example the output of
# > the unix find command), you can fgets the file names, and open
# > the file name you fgets.
# >
# > --
# > SM Ryan http://www.rawbw.com/~wyrmwif/
# > Death is the worry of the living. The dead, like myself,
# > only worry about decay and necrophiliacs.
#
#
#
# Lets consider I have a list of all the file names in a file called
# list.txt. I wrote the following code
#
# fp2=fopen("list .txt","r");
# while ( fgets(line, sizeof(line), fp2) != NULL)
# {
# printf("%s", line);
# strcpy(name,lin e);
# printf("%s", name);
# printf("%d\n",s trlen(name));

Print it out it in the most irritatingly verbose manner possible, like
{char *q=name; for (; *q; q++) printf("<%02X>% c",*q,' '<=*q && *q<127?*q:'.'); }
printf("\n");
and then make sure it's not sneaking in any extra non-printing characters,
like the \n fgets leaves at the end of the buffer.

Voluminous output no longer kills trees. If you can't understand what's
happenning print everything so you know what's on whether than guessing.

# fp = fopen(name, "r" );
# fp1 = fopen("out.txt" ,"a");
# if (fp==NULL)
# {
# printf("error in opening\n");
# exit(1);
# }

Most implementations set errno on fopen failure, so you can do
perror(name)
and get both the name you think you're using and the exact error.

--
SM Ryan http://www.rawbw.com/~wyrmwif/
Where do you get those wonderful toys?

[leorulez]

Eric Sosman wrote:

le******@gmail. com wrote On 05/24/06 12:29,:

Lets consider I have a list of all the file names in a file called
list.txt. I wrote the following code

fp2=fopen("list .txt","r");
while ( fgets(line, sizeof(line), fp2) != NULL)
{
printf("%s", line);
strcpy(name,lin e);
printf("%s", name);
printf("%d\n",s trlen(name));
fp = fopen(name, "r" );
fp1 = fopen("out.txt" ,"a");
if (fp==NULL)
{
printf("error in opening\n");
exit(1);
}
....
....
....

The file pointer returns null resulting in "error in opening". when I
checked the string length of the filename its always 2 characters more
then the original length. Could you please tell me how I can use the
variable "name" (or "line") in fopen.

When fgets() reads a line, it stores the entire line
including the '\n' at the end. Since the name of the
file you are trying to open is probably "foo.txt" and not
"foo.txt\n" , you need to search for and remove that '\n'
character. Something like this will do it:

char *p;
...
p = strchr(line, '\n');
if (p != NULL)
*p = '\0';

This much would explain one extra character per line,
but you've reported seeing two. It may be that you've mis-
counted, and that simply removing the '\n' will solve your
problem. A more troublesome possibility is that "list.txt"
may have originated on a system that uses a different line-
ending convention from yours, and that it wasn't properly
translated to your system's conventions when it was moved
there. For example, Windows systems terminate lines with
the two-character pair '\r','\n', while POSIX systems use
'\n' by itself. If the '\r','\n' pairs weren't translated
to single '\n' characters, then the lines you read from the
file will look like "foo.txt\r\ n"; removing the '\n' will
leave you with "foo.txt\r" , and fopen() will probably still
fail.

If this is the case, the best cure is to clean up the
procedure that brought you "list.txt" in the first place, so
the line endings will be translated properly. If that's not
possible, you can add a little more code to remove '\r' (if
it's present), just as the code above removes '\n'. This
isn't perfectly bullet-proof, though, because some systems
use even stranger line-ending conventions than '\r','\n' --
for example, a system that used '\n','\r' would produce a
file that if read on a POSIX system would look like

first line\n
\rsecond line\n
\rthird line\n
...

It's for reasons like this that I recommend revisiting your
file-transfer process instead of trying to outguess the
remote system's line-ending conventions.

--
Er*********@sun .com

Thanks a lot for the help. It worked. I did

p = strchr(line, '\n');
if (p != NULL)
*p = '\0';
q = strchr(line, '\r');
if (q != NULL)
*q = '\0';

This eliminated both "\n" and "\r". Thanks again.