How to "tail" a file using file objects ?

[Bruce Ward]

There's something I obviously don't understand about Python file objects.
(I'm an experienced programmer, but a python noob)

In the following test code, I open a text file, and seek to EOF.
Then, once/second, I try to read any new lines.
However, the file object sees no increase in file length even when 1000 new 100-char. lines are added(!)
(Same result in python v2.5.2 on Ubuntu 8.04 & v2.6.5 on Scientific Linux 5.4)

What perfectly obvious thing am I missing here?

#!/usr/bin/python2.6
 
import os
import stat
import time
 
filename = "testfile.txt"
f = open (filename, "r")
f.seek(0, 2)
 
while True:
    line = f.readline()
    size1 = os.stat(filename)[stat.ST_SIZE]
    size2 = os.fstat(f.fileno())[stat.ST_SIZE]
    print "size1 =", size1, "; size2 =", size2, "; line = \"" + line + "\""
    time.sleep(1)
 

---- output ---- (1000 new lines pasted into file after 2nd line)
:
size1 = 10 ; size2 = 10 ; line = ""
size1 = 10 ; size2 = 10 ; line = ""
size1 = 100010 ; size2 = 10 ; line = ""
size1 = 100010 ; size2 = 10 ; line = ""
:

[Bruce Ward]

Nuts. I always post questions too fast.
This is not a python problem; I get the same results when I recode in C, using file descriptors, so that I'm reasonably low-level.
Apparently, I just never understood Linux file I/O... (for 20 years)...
Sorry to waste everyone's time.

[dwblas]

Generally speaking, when you open a file, the significant stats are read/calculated and stored in variables. So EOF is stored in a variable and not changed. In Python, os.stat(fname) returns a tuple. The 6th element is the file's size. You will probably have to poll the file periodically and check for a change in the size.

[Markus]

Adding to dwblas' post, you could use something like pyinotify to listen for events on files, and then perform actions accordingly.

[Bruce Ward]

dwblas, Markus:

Thanks, guys! Between you, you have opened both my eyes!

dwblas: You have confirmed what I was beginning to suspect (but had remained ignorant of all these years).

Markus: I had not even suspected the existence of inotify/pynotify, even though inotify has apparently been in the Linux kernel for some time.

I am a very happy geek right now.
Again, thanks!