wait until change

[Tom]

Hi,

I have the following problem. I analyse data from a file. When I am done
analysing all the data that was in that file, I want to let my program
wait until this specific file has changed (which indicates that new data
has been added to the file from a third party program that I don't
control). If that file has changed, I want my program to continue.

What would be the appropriate command to check if the file changed? And
how can I implement "wait until" into my program? I googled a lot to
find any hints, but couldn't find anything helpful. But I found
something about a python that was stabbed in NY in 2000. :-) Thought
that was funny. :-)

I would deeply appreciate, if you could maybe include some little
fragments of code to show me how to use your suggestions. I am
relatively new to python and I often have problems with Pythons Library
Reference, because it doesn't show examples and is very technically.
That makes it difficult for me to really understand that. It also would
be great, if you could tell me which modules I have to include. I hope
all that is not asked too much. I assume that both problems can be
solved with a few easy lines of code. I just don't know them. :-)

Thanks a lot for your help and suggestions.
Regards Tom

[Peter Hansen]

Tom wrote:

I have the following problem. I analyse data from a file. When I am done
analysing all the data that was in that file, I want to let my program
wait until this specific file has changed (which indicates that new data
has been added to the file from a third party program that I don't
control). If that file has changed, I want my program to continue.

Would it be sufficient to have your program periodically wake up and
check the timestamp on the file? To do that, a combination of time.sleep()
and os.stat() would be sufficient, and quite cross-platform.

-Peter

[Lane LiaBraaten]

On Friday 17 October 2003 12:35 pm, Tom wrote:

Hi,

I have the following problem. I analyse data from a file. When I am done
analysing all the data that was in that file, I want to let my program
wait until this specific file has changed (which indicates that new data
has been added to the file from a third party program that I don't
control). If that file has changed, I want my program to continue.

What would be the appropriate command to check if the file changed? And
how can I implement "wait until" into my program? I googled a lot to
find any hints, but couldn't find anything helpful. But I found
something about a python that was stabbed in NY in 2000. :-) Thought
that was funny. :-)
I would use os.stat() which returns all sorts of info about the file including
modification time.

More info at: http://web.pydoc.org/1.5.2/os.html

time=os.stat(fi le)[8]

# analyze data

while 1:
if os.stat(file)[8]>time:
#analyze data again

I would deeply appreciate, if you could maybe include some little
fragments of code to show me how to use your suggestions. I am
relatively new to python and I often have problems with Pythons Library
Reference, because it doesn't show examples and is very technically.
That makes it difficult for me to really understand that. It also would
be great, if you could tell me which modules I have to include. I hope
all that is not asked too much. I assume that both problems can be
solved with a few easy lines of code. I just don't know them. :-)

Thanks a lot for your help and suggestions.
Regards Tom

I'm sure there are a number of ways to do this.

Good Luck,
LGL

[Tom]

Lane LiaBraaten wrote:

I would use os.stat() which returns all sorts of info about the file including
modification time.

More info at: http://web.pydoc.org/1.5.2/os.html

time=os.stat(f ile)[8]

# analyze data

while 1:
if os.stat(file)[8]>time:
#analyze data again

Hi Lane,

thanks for your answer it works great and gives me a lot of new ideas
that I can work with.
That "while 1" command looks interessting. I never used while that way.
The Library Reference doesn't explain it. But am I right if I assume
that this is a loop that continues forever? Despite using break of
course. I ask, because your suggestion works, but unfortunately uses
100% of my CPU power. :-( Well, it works, but is there maybe another way
to do the same without using so much of my resources? :-) That would be
great.

Thank's again.
Tom

[Peter Hansen]

Tom wrote:

thanks for your answer it works great and gives me a lot of new ideas
that I can work with.
That "while 1" command looks interessting. I never used while that way.
The Library Reference doesn't explain it. But am I right if I assume
that this is a loop that continues forever? Despite using break of
course. I ask, because your suggestion works, but unfortunately uses
100% of my CPU power. :-( Well, it works, but is there maybe another way
to do the same without using so much of my resources? :-) That would be
great.

Use time.sleep().

-Peter

[Tom]

Peter Hansen wrote:

Use time.sleep().

-Peter

If I use sleep, I have to know how long the process should sleep, but
sometimes it should sleep for seconds and sometimes for minutes. So it
would be good if I can put it to sleep and somehow wake it up if the
file changed.

Tom

[Tim Hochberg]

Tom wrote:

Peter Hansen wrote:

Use time.sleep().

-Peter

If I use sleep, I have to know how long the process should sleep, but
sometimes it should sleep for seconds and sometimes for minutes. So it
would be good if I can put it to sleep and somehow wake it up if the
file changed.

If you use something like::

ctime = os.stat(file).s t_ctime
while True:
if os.stat(file).s t_ctime > ctime:
break
time.sleep(1.0)
# Do stuff...

The load on your CPU will be negligible and your program will wake up at
most 1 second after the modification

-tim

[Peter Hansen]

Tom wrote:

Peter Hansen wrote:

Use time.sleep().

-Peter

If I use sleep, I have to know how long the process should sleep, but
sometimes it should sleep for seconds and sometimes for minutes. So it
would be good if I can put it to sleep and somehow wake it up if the
file changed.

See Tim's excellent reply.

Also, note that there are non-cross-platform solutions that basically
notify your program "as soon as" (possibly without any latency guarantees)
a change has happened. They are harder to use and quite likely unnecessary
in your case, and the time.sleep() approach Tim points out is simpler
and therefore the best thing to start with.

You do have to confirm of course that this other application won't
be writing to the file so often that you might run into trouble.

Does this other program write updates infrequently, and does your
Python analysis script run quickly enough, that if your code wakes
up within a few seconds of each update it will complete its work
soon enough to avoid problems?

-Peter

[Werner Schiendl]

Hi,

What's the problem?

Just wait as long as you can accept to be "late" when your file changes.

So if you can accept e. g. 5 seconds delay, you'll change Lane's example to:

time=os.stat(fi le)[8]

# analyze data

while 1:
if os.stat(file)[8]>time:
#analyze data again
else:
time.sleep(5)
This will drastically reduce your CPU load ;-)

hth

Werner

Tom wrote:

Peter Hansen wrote:

Use time.sleep().

-Peter

If I use sleep, I have to know how long the process should sleep, but
sometimes it should sleep for seconds and sometimes for minutes. So it
would be good if I can put it to sleep and somehow wake it up if the
file changed.

Tom

[Dave Benjamin]

In article <3F************ ***@engcorp.com >, Peter Hansen wrote:

Also, note that there are non-cross-platform solutions that basically
notify your program "as soon as" (possibly without any latency guarantees)
a change has happened. They are harder to use and quite likely unnecessary
in your case, and the time.sleep() approach Tim points out is simpler
and therefore the best thing to start with.

Here's one that uses PythonWin and WMI. I wrote this awhile ago because I
was experimenting with writing a templating system that would process input
files "on the fly" whenever they changed. It seems to drive my CPU usage up,
though - not sure why, because I'm pretty sure that events.NextEven t()
blocks until an event comes. Anyway, I don't know if this is useful to
anyone, but here it is... (apologies for the wide margins):

import time, win32com; time.sleep(1)
from win32com.client import GetObject

interval = 1
filenames = [r'c:\test.txt', r'c:\output.txt ']

class FileChangeEvent :
def __init__(self, filename, new_size, old_size):
self.filename = filename
self.new_size = new_size
self.old_size = old_size

def on_change(event ):
print event.__dict__

wmi = GetObject(r'win mgmts:{imperson ationLevel=impe rsonate}!\\.\ro ot\cimv2')
filespec = ' OR '.join(["TargetInstance .Name='%s'" % filename.replac e('\\', '\\\\') for filename in filenames])
query = "SELECT * FROM __InstanceModif icationEvent WITHIN %d WHERE TargetInstance ISA 'CIM_DataFile' AND (%s)" % (interval, filespec)
events = wmi.ExecNotific ationQuery(quer y)

while True:
event = events.NextEven t()
on_change(FileC hangeEvent(str( event.TargetIns tance.Name),
int(event.Targe tInstance.FileS ize),
int(event.Previ ousInstance.Fil eSize)))

--
..:[ dave benjamin (ramenboy) -:- www.ramenfest.com -:- www.3dex.com ]:.
: d r i n k i n g l i f e o u t o f t h e c o n t a i n e r :