By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
446,236 Members | 2,051 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 446,236 IT Pros & Developers. It's quick & easy.

tailing a log file using c ...

P: n/a
Hi all,

problem - need to read a log (text) file looking for certain entries,
this file will still be written to by another 3rd party process, and
hence constantly require monitoring (dare I say tailing ?)

Is it possible / practical / efficient to 'tail' a log file using c.
Questions are :

1. If I was reading file and I reached EOF, would I see new file
writes - or would I have to re-open file to see new additions to it ?

2. Would I use unix 'tail' command fed into a c program ?

3. Any advice appreciated ?

Cheers

Tony
Nov 13 '05 #1
Share this Question
Share on Google+
10 Replies


P: n/a

"Tony Stock" <ts****@hotmail.com> schrieb im Newsbeitrag
news:7a**********************@posting.google.com.. .
1. If I was reading file and I reached EOF, would I see new file
writes - or would I have to re-open file to see new additions to it ?
Depends on the operating system. On many UNIXes, and on Windows, yes, you
will see the new writes.

You have to open the file in a shared mode using unbuffered I/O (open(),
read(), close()).

Copy the data from read() to an internal buffer and print it when it's ready
(complete line).

Keep the file open and attempt to read() beyond the EOF. If an error is
returned, no new data is available, otherwise read as much as you can
process.

2. Would I use unix 'tail' command fed into a c program ?


You could also do that, calling system() to issue a tail command and
retrieving its output. But the other solution is simpler. :-)


Nov 13 '05 #2

P: n/a
"Ekkehard Morgenstern" <ek******************@onlinehome.de> wrote:
"Tony Stock" <ts****@hotmail.com> schrieb:
1. If I was reading file and I reached EOF, would I see new
file writes - or would I have to re-open file to see new
additions to it ?

It's not guaranteed in standard C. When you open the file it may be
completely locked down so no other program could append log entries
to it.
Depends on the operating system. On many UNIXes, and on Windows,
yes, you will see the new writes.

You have to open the file in a shared mode using unbuffered I/O
(open(), read(), close()).


Unbuffered I/O is not part of Standard C, and is off-topic on comp.lang.c.
The open, read, close functions can be discussed on comp.unix.programmer,
even if you are writing code for Windows, since they are POSIX functions.

--
Simon.
Nov 13 '05 #3

P: n/a
In article <3f***********************@news.optusnet.com.au> ,
Simon Biber <ne**@ralminNOSPAM.cc> wrote:

Unbuffered I/O is not part of Standard C, and is off-topic on comp.lang.c.
The open, read, close functions can be discussed on comp.unix.programmer,
even if you are writing code for Windows, since they are POSIX functions.


I don't think that true. The standard defines setvbuf() to set/change
the buffering mode of a stream.

#include <stdio.h>
int setvbuf(FILE *stream, char *buf, int mode , size_t size);

--
Rouben Rostamian
Nov 13 '05 #4

P: n/a
ts****@hotmail.com (Tony Stock) wrote:
Hi all,

problem - need to read a log (text) file looking for certain entries,
this file will still be written to by another 3rd party process, and
hence constantly require monitoring (dare I say tailing ?)

Is it possible / practical / efficient to 'tail' a log file using c.
Yes.
Questions are :

1. If I was reading file and I reached EOF, would I see new file
writes - or would I have to re-open file to see new additions to it ?
If you clear the eof condition and re-read the file, you'll see
new data.
2. Would I use unix 'tail' command fed into a c program ?
If you had asked this on comp.unix.programmer the answer would
be to read the man page for popen(), but...
3. Any advice appreciated ?


#include <stdio.h>
#include <stdlib.h>
#include <ctype.h>

int main(int argc, char *argv[])
{
char buf[BUFSIZ];
fpos_t pos;
FILE *fp;

if (argc != 2) {
exit(EXIT_FAILURE);
}

if (NULL == (fp = fopen(argv[1], "r"))) {
fprintf(stderr,"Cannot find file: %s\n", argv[1]);
exit(EXIT_FAILURE);
}

if ( 0 != fseek(fp, 0, SEEK_END)) {
fprintf(stderr,"Cannot read file: %s\n", argv[1]);
exit(EXIT_FAILURE);
}

if (-1 == fgetpos(fp, &pos)) {
fprintf(stderr,"Cannot read file: %s\n", argv[1]);
exit(EXIT_FAILURE);
}

for (;;) {
fgets(buf, BUFSIZ, fp);
if (ferror(fp)) {
fprintf(stderr,"Error reading file: %s\n", argv[1]);
exit(EXIT_FAILURE);
}
if (feof(fp)) {
fsetpos(fp, &pos);
clearerr(fp);
#if 0
/* non-ansi code! */
sleep(1); /* give up the cpu to other processes */
#endif
continue;
}
fgetpos(fp, &pos);
printf("%s", buf);
fflush(stdout);
}

return EXIT_SUCCESS;
}
--
Floyd L. Davidson <http://web.newsguy.com/floyd_davidson>
Ukpeagvik (Barrow, Alaska) fl***@barrow.com
Nov 13 '05 #5

P: n/a
Tony Stock wrote:
Hi all,

problem - need to read a log (text) file looking for certain entries,
this file will still be written to by another 3rd party process, and
hence constantly require monitoring (dare I say tailing ?)

Is it possible / practical / efficient to 'tail' a log file using c.

Questions are :

1. If I was reading file and I reached EOF, would I see new file
writes - or would I have to re-open file to see new additions to it ?

2. Would I use unix 'tail' command fed into a c program ?


Off-topic here, but 'tail -f logfile | your_program' would be the
simplest, and most flexible, solution. Or use the syslog facility.
Nov 13 '05 #6

P: n/a
Thanks for the responses - food for thought.

It will be on a unix platform (unknown at present though)

Just a little confused, one poster said that it needs to be UNBUFFERED
io (i.e. open / read / close), but the sample code posted uses a
normal fileopen (fopen) ????
Thanks

Tony
Nov 13 '05 #7

P: n/a
ts****@hotmail.com (Tony Stock) wrote:
Thanks for the responses - food for thought.

It will be on a unix platform (unknown at present though)

Just a little confused, one poster said that it needs to be UNBUFFERED
io (i.e. open / read / close), but the sample code posted uses a
normal fileopen (fopen) ????


Data will be delivered in chunks, and the specifics depend on
whether you use setvbuf (or not) to change the buffering.

A file (as opposed to a terminal) accessed by fopen() will default
to block buffering; hence, you won't see any changes until the
total equals the block size (probably BUFSIZ, defined in stdio.h,
and likely to be 4k or 8k.).

If your logs are line oriented (almost certainly true) you'll
probably want to use setvbuf to change access to line buffered, in
which case each newline will cause data to be available. Otherwise
a single line log entry might not be available until other log
entries occur in sufficient numbers to fill the buffer when block
buffering is used.

The other alternative is to turn off buffering (or use non-portable
low level unbuffered i/o functions), in which case the size of the
chunk will be 1 character, and you're program can happily thrash
away at each and every byte...

--
Floyd L. Davidson <http://web.newsguy.com/floyd_davidson>
Ukpeagvik (Barrow, Alaska) fl***@barrow.com
Nov 13 '05 #8

P: n/a
In <bp**********@pc18.math.umbc.edu> ro****@pc18.math.umbc.edu (Rouben Rostamian) writes:
In article <3f***********************@news.optusnet.com.au> ,
Simon Biber <ne**@ralminNOSPAM.cc> wrote:

Unbuffered I/O is not part of Standard C, and is off-topic on comp.lang.c.
The open, read, close functions can be discussed on comp.unix.programmer,
even if you are writing code for Windows, since they are POSIX functions.


I don't think that true. The standard defines setvbuf() to set/change
the buffering mode of a stream.

#include <stdio.h>
int setvbuf(FILE *stream, char *buf, int mode , size_t size);


This only suppresses the buffering performed by the standard C library.
Sometimes, you may need to suppress the buffering performed by the OS,
too, and setvbuf is very unlikely to help (although, in theory, it could).

But when people talk about unbuffered I/O (or raw I/O or low level I/O)
they usually mean I/O performed using the OS primitives, rather than the
(relatively high level) <stdio.h> stuff.

Dan
--
Dan Pop
DESY Zeuthen, RZ group
Email: Da*****@ifh.de
Nov 13 '05 #9

P: n/a
Floyd Davidson <fl***@barrow.com> wrote:
ts****@hotmail.com (Tony Stock) wrote:
Thanks for the responses - food for thought.

It will be on a unix platform (unknown at present though)

Just a little confused, one poster said that it needs to be UNBUFFERED
io (i.e. open / read / close), but the sample code posted uses a
normal fileopen (fopen) ????


Data will be delivered in chunks, and the specifics depend on
whether you use setvbuf (or not) to change the buffering.

A file (as opposed to a terminal) accessed by fopen() will default
to block buffering; hence, you won't see any changes until the
total equals the block size (probably BUFSIZ, defined in stdio.h,
and likely to be 4k or 8k.).

If your logs are line oriented (almost certainly true) you'll
probably want to use setvbuf to change access to line buffered, in
which case each newline will cause data to be available. Otherwise
a single line log entry might not be available until other log
entries occur in sufficient numbers to fill the buffer when block
buffering is used.

The other alternative is to turn off buffering (or use non-portable
low level unbuffered i/o functions), in which case the size of the
chunk will be 1 character, and you're program can happily thrash
away at each and every byte...


Well, all of that sounds good... but it doesn't do that.

The code that I posted uses fgets() for input, and acts as described
for line buffering because of that. By changing the input to fread()
behavior as described for unbuffered input can be obtained.

Each read either reaches the end of a line or the end of the
buffer (which ever is important for the mode used) _or_ it hits
an end of file condition. Hence the new data is always available.

--
Floyd L. Davidson <http://web.newsguy.com/floyd_davidson>
Ukpeagvik (Barrow, Alaska) fl***@barrow.com
Nov 13 '05 #10

P: n/a
Tony Stock wrote:

Thanks for the responses - food for thought.

It will be on a unix platform (unknown at present though)

Just a little confused, one poster said that it needs to be UNBUFFERED
io (i.e. open / read / close), but the sample code posted uses a
normal fileopen (fopen) ????
This is OT on c.l.c. but FYI here is an extract from the info file
for GNU tail:
`-f'
`--follow[=HOW]'
Loop forever trying to read more characters at the end of the file,
presumably because the file is growing. This option is ignored
when reading from a pipe. If more than one file is given, `tail'
prints a header whenever it gets output from a different file, to
indicate which file that output is from.

There are two ways to specify how you'd like to track files with
this option, but that difference is noticeable only when a
followed file is removed or renamed. If you'd like to continue to
track the end of a growing file even after it has been unlinked,
use `--follow=descriptor'. This is the default behavior, but it
is not useful if you're tracking a log file that may be rotated
(removed or renamed, then reopened). In that case, use
`--follow=name' to track the named file by reopening it
periodically to see if it has been removed and recreated by some
other program.


--
Chuck F (cb********@yahoo.com) (cb********@worldnet.att.net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net> USE worldnet address!
Nov 13 '05 #11

This discussion thread is closed

Replies have been disabled for this discussion.