469,338 Members | 8,644 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,338 developers. It's quick & easy.

waiting for file lock?

hi
i need to go into a directory to grab some files and do some
processing.
The thing is, i need to wait till the process that generates the files
in that directory to finish
before i can grab the files. eg if file A is being generated and has
not finished, my python script will not go into the directory.
how can i check that file A has actually finished? thanks

Apr 11 '06 #1
3 2526
In article <11********************@j33g2000cwa.googlegroups.c om>,
s9************@yahoo.com wrote:
i need to go into a directory to grab some files and do some
processing.
The thing is, i need to wait till the process that generates the files
in that directory to finish
before i can grab the files. eg if file A is being generated and has
not finished, my python script will not go into the directory.
how can i check that file A has actually finished?


I wrote a similar system that watches for new files arriving in an
"uploads" directory, whether copied there via FTP or using a GUI desktop
script. My heuristic was to only process files whose last-modified
date/time was at least 5 minutes in the past. My assumption was that it
was unlikely that 5 minutes would go by between more information being
added to a file.
Apr 11 '06 #2
On Tue, 11 Apr 2006 19:07:19 +1200, rumours say that Lawrence D'Oliveiro
<ld*@geek-central.gen.new_zealand> might have written:
In article <11********************@j33g2000cwa.googlegroups.c om>,
s9************@yahoo.com wrote:
i need to go into a directory to grab some files and do some
processing.
The thing is, i need to wait till the process that generates the files
in that directory to finish
before i can grab the files. eg if file A is being generated and has
not finished, my python script will not go into the directory.
how can i check that file A has actually finished?
I wrote a similar system that watches for new files arriving in an
"uploads" directory, whether copied there via FTP or using a GUI desktop
script. My heuristic was to only process files whose last-modified
date/time was at least 5 minutes in the past. My assumption was that it
was unlikely that 5 minutes would go by between more information being
added to a file.


This works (unless there are long network timeouts, when downloading...),
but another idea is to wait for the existence of a zero-byte sentinel file
that is created last, after the transfer (or process, in general) has ended.

This method is the one that has worked best for me.
--
TZOTZIOY, I speak England very best.
"Dear Paul,
please stop spamming us."
The Corinthians
Apr 11 '06 #3
s9************@yahoo.com wrote:
hi
i need to go into a directory to grab some files and do some
processing.
The thing is, i need to wait till the process that generates the files
in that directory to finish
before i can grab the files. eg if file A is being generated and has
not finished, my python script will not go into the directory.
how can i check that file A has actually finished? thanks

I don't know if you have control of the program that is writing the
file. If you do, change it so that it writes the files with some
temporary name and renames them when it is done. Since rename is
basically atomic, the file won't show up as a candidate to be
processed until it is actually completed.

Something like:

Process A -> writes files into directory with .tmp extensions.
Process B -> scans directory but never processes anything with
.tmp extensions
Process A -> renames file.tmp to file when it is finished so
that process B picks it up on its next scan.
-Larry Bates
Apr 11 '06 #4

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

6 posts views Thread by Pekka Niiranen | last post: by
44 posts views Thread by Charles Law | last post: by
5 posts views Thread by bughunter | last post: by
5 posts views Thread by Ulrich Wisser | last post: by
1 post views Thread by Michael Fuhr | last post: by
1 post views Thread by CARIGAR | last post: by
reply views Thread by suresh191 | last post: by
1 post views Thread by Marylou17 | last post: by
1 post views Thread by Marylou17 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.