473,320 Members | 2,146 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

tailing a log file using c ...

Hi all,

problem - need to read a log (text) file looking for certain entries,
this file will still be written to by another 3rd party process, and
hence constantly require monitoring (dare I say tailing ?)

Is it possible / practical / efficient to 'tail' a log file using c.
Questions are :

1. If I was reading file and I reached EOF, would I see new file
writes - or would I have to re-open file to see new additions to it ?

2. Would I use unix 'tail' command fed into a c program ?

3. Any advice appreciated ?

Cheers

Tony
Nov 13 '05 #1
10 8967

"Tony Stock" <ts****@hotmail.com> schrieb im Newsbeitrag
news:7a**********************@posting.google.com.. .
1. If I was reading file and I reached EOF, would I see new file
writes - or would I have to re-open file to see new additions to it ?
Depends on the operating system. On many UNIXes, and on Windows, yes, you
will see the new writes.

You have to open the file in a shared mode using unbuffered I/O (open(),
read(), close()).

Copy the data from read() to an internal buffer and print it when it's ready
(complete line).

Keep the file open and attempt to read() beyond the EOF. If an error is
returned, no new data is available, otherwise read as much as you can
process.

2. Would I use unix 'tail' command fed into a c program ?


You could also do that, calling system() to issue a tail command and
retrieving its output. But the other solution is simpler. :-)


Nov 13 '05 #2
"Ekkehard Morgenstern" <ek******************@onlinehome.de> wrote:
"Tony Stock" <ts****@hotmail.com> schrieb:
1. If I was reading file and I reached EOF, would I see new
file writes - or would I have to re-open file to see new
additions to it ?

It's not guaranteed in standard C. When you open the file it may be
completely locked down so no other program could append log entries
to it.
Depends on the operating system. On many UNIXes, and on Windows,
yes, you will see the new writes.

You have to open the file in a shared mode using unbuffered I/O
(open(), read(), close()).


Unbuffered I/O is not part of Standard C, and is off-topic on comp.lang.c.
The open, read, close functions can be discussed on comp.unix.programmer,
even if you are writing code for Windows, since they are POSIX functions.

--
Simon.
Nov 13 '05 #3
In article <3f***********************@news.optusnet.com.au> ,
Simon Biber <ne**@ralminNOSPAM.cc> wrote:

Unbuffered I/O is not part of Standard C, and is off-topic on comp.lang.c.
The open, read, close functions can be discussed on comp.unix.programmer,
even if you are writing code for Windows, since they are POSIX functions.


I don't think that true. The standard defines setvbuf() to set/change
the buffering mode of a stream.

#include <stdio.h>
int setvbuf(FILE *stream, char *buf, int mode , size_t size);

--
Rouben Rostamian
Nov 13 '05 #4
ts****@hotmail.com (Tony Stock) wrote:
Hi all,

problem - need to read a log (text) file looking for certain entries,
this file will still be written to by another 3rd party process, and
hence constantly require monitoring (dare I say tailing ?)

Is it possible / practical / efficient to 'tail' a log file using c.
Yes.
Questions are :

1. If I was reading file and I reached EOF, would I see new file
writes - or would I have to re-open file to see new additions to it ?
If you clear the eof condition and re-read the file, you'll see
new data.
2. Would I use unix 'tail' command fed into a c program ?
If you had asked this on comp.unix.programmer the answer would
be to read the man page for popen(), but...
3. Any advice appreciated ?


#include <stdio.h>
#include <stdlib.h>
#include <ctype.h>

int main(int argc, char *argv[])
{
char buf[BUFSIZ];
fpos_t pos;
FILE *fp;

if (argc != 2) {
exit(EXIT_FAILURE);
}

if (NULL == (fp = fopen(argv[1], "r"))) {
fprintf(stderr,"Cannot find file: %s\n", argv[1]);
exit(EXIT_FAILURE);
}

if ( 0 != fseek(fp, 0, SEEK_END)) {
fprintf(stderr,"Cannot read file: %s\n", argv[1]);
exit(EXIT_FAILURE);
}

if (-1 == fgetpos(fp, &pos)) {
fprintf(stderr,"Cannot read file: %s\n", argv[1]);
exit(EXIT_FAILURE);
}

for (;;) {
fgets(buf, BUFSIZ, fp);
if (ferror(fp)) {
fprintf(stderr,"Error reading file: %s\n", argv[1]);
exit(EXIT_FAILURE);
}
if (feof(fp)) {
fsetpos(fp, &pos);
clearerr(fp);
#if 0
/* non-ansi code! */
sleep(1); /* give up the cpu to other processes */
#endif
continue;
}
fgetpos(fp, &pos);
printf("%s", buf);
fflush(stdout);
}

return EXIT_SUCCESS;
}
--
Floyd L. Davidson <http://web.newsguy.com/floyd_davidson>
Ukpeagvik (Barrow, Alaska) fl***@barrow.com
Nov 13 '05 #5
Tony Stock wrote:
Hi all,

problem - need to read a log (text) file looking for certain entries,
this file will still be written to by another 3rd party process, and
hence constantly require monitoring (dare I say tailing ?)

Is it possible / practical / efficient to 'tail' a log file using c.

Questions are :

1. If I was reading file and I reached EOF, would I see new file
writes - or would I have to re-open file to see new additions to it ?

2. Would I use unix 'tail' command fed into a c program ?


Off-topic here, but 'tail -f logfile | your_program' would be the
simplest, and most flexible, solution. Or use the syslog facility.
Nov 13 '05 #6
Thanks for the responses - food for thought.

It will be on a unix platform (unknown at present though)

Just a little confused, one poster said that it needs to be UNBUFFERED
io (i.e. open / read / close), but the sample code posted uses a
normal fileopen (fopen) ????
Thanks

Tony
Nov 13 '05 #7
ts****@hotmail.com (Tony Stock) wrote:
Thanks for the responses - food for thought.

It will be on a unix platform (unknown at present though)

Just a little confused, one poster said that it needs to be UNBUFFERED
io (i.e. open / read / close), but the sample code posted uses a
normal fileopen (fopen) ????


Data will be delivered in chunks, and the specifics depend on
whether you use setvbuf (or not) to change the buffering.

A file (as opposed to a terminal) accessed by fopen() will default
to block buffering; hence, you won't see any changes until the
total equals the block size (probably BUFSIZ, defined in stdio.h,
and likely to be 4k or 8k.).

If your logs are line oriented (almost certainly true) you'll
probably want to use setvbuf to change access to line buffered, in
which case each newline will cause data to be available. Otherwise
a single line log entry might not be available until other log
entries occur in sufficient numbers to fill the buffer when block
buffering is used.

The other alternative is to turn off buffering (or use non-portable
low level unbuffered i/o functions), in which case the size of the
chunk will be 1 character, and you're program can happily thrash
away at each and every byte...

--
Floyd L. Davidson <http://web.newsguy.com/floyd_davidson>
Ukpeagvik (Barrow, Alaska) fl***@barrow.com
Nov 13 '05 #8
In <bp**********@pc18.math.umbc.edu> ro****@pc18.math.umbc.edu (Rouben Rostamian) writes:
In article <3f***********************@news.optusnet.com.au> ,
Simon Biber <ne**@ralminNOSPAM.cc> wrote:

Unbuffered I/O is not part of Standard C, and is off-topic on comp.lang.c.
The open, read, close functions can be discussed on comp.unix.programmer,
even if you are writing code for Windows, since they are POSIX functions.


I don't think that true. The standard defines setvbuf() to set/change
the buffering mode of a stream.

#include <stdio.h>
int setvbuf(FILE *stream, char *buf, int mode , size_t size);


This only suppresses the buffering performed by the standard C library.
Sometimes, you may need to suppress the buffering performed by the OS,
too, and setvbuf is very unlikely to help (although, in theory, it could).

But when people talk about unbuffered I/O (or raw I/O or low level I/O)
they usually mean I/O performed using the OS primitives, rather than the
(relatively high level) <stdio.h> stuff.

Dan
--
Dan Pop
DESY Zeuthen, RZ group
Email: Da*****@ifh.de
Nov 13 '05 #9
Floyd Davidson <fl***@barrow.com> wrote:
ts****@hotmail.com (Tony Stock) wrote:
Thanks for the responses - food for thought.

It will be on a unix platform (unknown at present though)

Just a little confused, one poster said that it needs to be UNBUFFERED
io (i.e. open / read / close), but the sample code posted uses a
normal fileopen (fopen) ????


Data will be delivered in chunks, and the specifics depend on
whether you use setvbuf (or not) to change the buffering.

A file (as opposed to a terminal) accessed by fopen() will default
to block buffering; hence, you won't see any changes until the
total equals the block size (probably BUFSIZ, defined in stdio.h,
and likely to be 4k or 8k.).

If your logs are line oriented (almost certainly true) you'll
probably want to use setvbuf to change access to line buffered, in
which case each newline will cause data to be available. Otherwise
a single line log entry might not be available until other log
entries occur in sufficient numbers to fill the buffer when block
buffering is used.

The other alternative is to turn off buffering (or use non-portable
low level unbuffered i/o functions), in which case the size of the
chunk will be 1 character, and you're program can happily thrash
away at each and every byte...


Well, all of that sounds good... but it doesn't do that.

The code that I posted uses fgets() for input, and acts as described
for line buffering because of that. By changing the input to fread()
behavior as described for unbuffered input can be obtained.

Each read either reaches the end of a line or the end of the
buffer (which ever is important for the mode used) _or_ it hits
an end of file condition. Hence the new data is always available.

--
Floyd L. Davidson <http://web.newsguy.com/floyd_davidson>
Ukpeagvik (Barrow, Alaska) fl***@barrow.com
Nov 13 '05 #10
Tony Stock wrote:

Thanks for the responses - food for thought.

It will be on a unix platform (unknown at present though)

Just a little confused, one poster said that it needs to be UNBUFFERED
io (i.e. open / read / close), but the sample code posted uses a
normal fileopen (fopen) ????
This is OT on c.l.c. but FYI here is an extract from the info file
for GNU tail:
`-f'
`--follow[=HOW]'
Loop forever trying to read more characters at the end of the file,
presumably because the file is growing. This option is ignored
when reading from a pipe. If more than one file is given, `tail'
prints a header whenever it gets output from a different file, to
indicate which file that output is from.

There are two ways to specify how you'd like to track files with
this option, but that difference is noticeable only when a
followed file is removed or renamed. If you'd like to continue to
track the end of a growing file even after it has been unlinked,
use `--follow=descriptor'. This is the default behavior, but it
is not useful if you're tracking a log file that may be rotated
(removed or renamed, then reopened). In that case, use
`--follow=name' to track the named file by reopening it
periodically to see if it has been removed and recreated by some
other program.


--
Chuck F (cb********@yahoo.com) (cb********@worldnet.att.net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net> USE worldnet address!
Nov 13 '05 #11

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

8
by: Eric Lilja | last post by:
Hello, I had what I thought was normal text-file and I needed to locate a string matching a certain pattern in that file and, if found, replace that string. I thought this would be simple but I had...
6
by: jalkadir | last post by:
I am trying to find a way to remove the leading and tailing blanks from a string. What I have come up with is not doing the work as I expected; because if the value I pass to the function is "...
3
by: StGo | last post by:
How can i read/write file's custom attributs(like subject,author...) in C#??? Thanks :))
9
by: ALI-R | last post by:
Hi,, I have two questions : 1) Is it mandatory that config file of a desktop application must be App.config 2) Is it possible to update config file in your code?? thanks for your help. ALI
13
by: Sky Sigal | last post by:
I have created an IHttpHandler that waits for uploads as attachments for a webmail interface, and saves it to a directory that is defined in config.xml. My question is the following: assuming...
4
by: psbasha | last post by:
Hi, I have a string str1 = 'Pnt ', I would like to remove the spaces from the tailing end of a string. I/P : str1 = 'Pnt ' O/P : 'Pnt'
3
by: JDeats | last post by:
I have some .NET 1.1 code that utilizes this technique for encrypting and decrypting a file. http://support.microsoft.com/kb/307010 In .NET 2.0 this approach is not fully supported (a .NET 2.0...
0
by: Evan Klitzke | last post by:
Everyone, I'm interested in writing a python program that reads from a log file and then executes actions based on the lines. I effectively want to write a loop that does something like this: ...
1
by: Shaw | last post by:
I have a number, for example 17.7 and I want to format to a string as "17.70", with the tailing zero. Using Math.Round( dValue, 2 ) - produces "17.7". Any other functions which will do what I...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.