472,331 Members | 1,701 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,331 software developers and data experts.

How to find out if another process is using a file

I'm writing a program which reads a series of data files as they are dumped
into a directory by another process. At the moment, it gets sporadic bugs
when it tries to read files which are only partially written.

I'm looking for a function which will tell me if a file is opened in
write-mode by another process - if it is, my program will ignore it for now
and come back to it later. This needs to work on linux and windows. Mac
OS would be a bonus too. An os-independent solution would be good, but I
could write os-specific options and have it pick the appropriate one.

Is there any way of doing this? I've drawn a blank with google so far.

A nasty hack would be to use the file modification time, and wait until
that's a few seconds in the past, but is there a nice solution?
--
I'm at CAMbridge, not SPAMbridge
Jan 18 '07 #1
8 4847
js
How about using lock?
Let writing process locks the files before writing, and unlock after
the job's done.

I think it'd work file in most environment.

On 1/19/07, Tom Wright <te***@spam.ac.ukwrote:
I'm writing a program which reads a series of data files as they are dumped
into a directory by another process. At the moment, it gets sporadic bugs
when it tries to read files which are only partially written.

I'm looking for a function which will tell me if a file is opened in
write-mode by another process - if it is, my program will ignore it for now
and come back to it later. This needs to work on linux and windows. Mac
OS would be a bonus too. An os-independent solution would be good, but I
could write os-specific options and have it pick the appropriate one.

Is there any way of doing this? I've drawn a blank with google so far.

A nasty hack would be to use the file modification time, and wait until
that's a few seconds in the past, but is there a nice solution?
--
I'm at CAMbridge, not SPAMbridge
--
http://mail.python.org/mailman/listinfo/python-list
Jan 18 '07 #2
"Tom Wright" <te***@spam.ac.ukescribiˇ en el mensaje
news:eo**********@gemini.csx.cam.ac.uk...
I'm writing a program which reads a series of data files as they are
dumped
into a directory by another process. At the moment, it gets sporadic bugs
when it tries to read files which are only partially written.

I'm looking for a function which will tell me if a file is opened in
write-mode by another process - if it is, my program will ignore it for
now
and come back to it later. This needs to work on linux and windows. Mac
OS would be a bonus too. An os-independent solution would be good, but I
could write os-specific options and have it pick the appropriate one.
Use os.open with the O_EXCL flag; will fail if the other process has the
file still open (and will fail if another process is reading the file, too,
not just if someone is writing).

--
Gabriel Genellina
Jan 18 '07 #3
In article <ma***************************************@python. org>,
"Gabriel Genellina" <ga******@yahoo.com.arwrote:
"Tom Wright" <te***@spam.ac.ukescribiˇ en el mensaje
news:eo**********@gemini.csx.cam.ac.uk...
I'm writing a program which reads a series of data files as they are
dumped
into a directory by another process. At the moment, it gets sporadic bugs
when it tries to read files which are only partially written.

I'm looking for a function which will tell me if a file is opened in
write-mode by another process - if it is, my program will ignore it for
now
and come back to it later. This needs to work on linux and windows. Mac
OS would be a bonus too. An os-independent solution would be good, but I
could write os-specific options and have it pick the appropriate one.

Use os.open with the O_EXCL flag; will fail if the other process has the
file still open (and will fail if another process is reading the file, too,
not just if someone is writing).
O_EXCL fails if the file exists at all - whether closed or open.

Donn Cave, do**@u.washington.edu
Jan 18 '07 #4

In article <do************************@gnus01.u.washington.ed u>,
Donn Cave <do**@u.washington.eduwrites:
|In article <ma***************************************@python. org>,
| "Gabriel Genellina" <ga******@yahoo.com.arwrote:
| "Tom Wright" <te***@spam.ac.ukescribiˇ en el mensaje
| news:eo**********@gemini.csx.cam.ac.uk...
|
| I'm writing a program which reads a series of data files as they are
| dumped
| into a directory by another process. At the moment, it gets sporadic bugs
| when it tries to read files which are only partially written.
| >
| I'm looking for a function which will tell me if a file is opened in
| write-mode by another process - if it is, my program will ignore it for
| now
| and come back to it later. This needs to work on linux and windows. Mac
| OS would be a bonus too. An os-independent solution would be good, but I
| could write os-specific options and have it pick the appropriate one.
|
| Use os.open with the O_EXCL flag; will fail if the other process has the
| file still open (and will fail if another process is reading the file, too,
| not just if someone is writing).
|>
|O_EXCL fails if the file exists at all - whether closed or open.

Yes. In theory. In practice, it usually works on normal files, provided
that all opens are local. Under some circumstances, it will even work
for NFS mounted files, as far as I recall.

MVS (now zOS) and other mainframe systems had what the poster wants, but
modern systems don't. I shall not recommend that the poster asks IBM
for a copy of zOS, for reasons that will be well-known to anyone who used
MVS (which are NOT the ones claimed by the Unix brigade, which are mostly
bogus) :-)

Under Linux, you can do something with fuser, and I am pretty certain that
modern Macintoshes (i.e. BSD) will have an equivalent. It can't be made
reliable (unlike under MVS), but might reduce the number of problems.
Regards,
Nick Maclaren.
Jan 18 '07 #5
In article <eo**********@gemini.csx.cam.ac.uk>,
nm**@cus.cam.ac.uk (Nick Maclaren) wrote:
In article <do************************@gnus01.u.washington.ed u>,
Donn Cave <do**@u.washington.eduwrites:
|In article <ma***************************************@python. org>,
| "Gabriel Genellina" <ga******@yahoo.com.arwrote:
| "Tom Wright" <te***@spam.ac.ukescribiˇ en el mensaje
| news:eo**********@gemini.csx.cam.ac.uk...
| Use os.open with the O_EXCL flag; will fail if the other process has the
| file still open (and will fail if another process is reading the file,
| too,
| not just if someone is writing).
|>
|O_EXCL fails if the file exists at all - whether closed or open.

Yes. In theory. In practice, it usually works on normal files, provided
that all opens are local. Under some circumstances, it will even work
for NFS mounted files, as far as I recall.
Mm, by "fail", I meant

An attempt to open with O_EXCL set will "fail" if the file exists at all,
i.e., the file will not be opened, a negative value will be returned,
and errno will be set to EEXIST.

What I neglected to mention is that this effect obtains when O_EXCL
is used in combination with O_CREAT. Without O_CREAT, O_EXCL doesn't
mean anything and is ignored.

If there is any significant difference between theory and practice
in this matter, it's news to me.

Donn
Jan 18 '07 #6

In article <do************************@gnus01.u.washington.ed u>,
Donn Cave <do**@u.washington.eduwrites:
| |>
| |O_EXCL fails if the file exists at all - whether closed or open.
|
| Yes. In theory. In practice, it usually works on normal files, provided
| that all opens are local. Under some circumstances, it will even work
| for NFS mounted files, as far as I recall.
|>
|Mm, by "fail", I meant
|>
|An attempt to open with O_EXCL set will "fail" if the file exists at all,
|i.e., the file will not be opened, a negative value will be returned,
|and errno will be set to EEXIST.
|>
|What I neglected to mention is that this effect obtains when O_EXCL
|is used in combination with O_CREAT. Without O_CREAT, O_EXCL doesn't
|mean anything and is ignored.
|>
|If there is any significant difference between theory and practice
|in this matter, it's news to me.

Actually, it is undefined behaviour in POSIX, and I have used a Unix
where O_EXCL on its own failed if the file existed (of course, it also
failed if it DIDN'T exist, because of the lack of O_CREAT). I can't
remember which - perhaps AIX or OSF/1 (not DEC).

But I was referring to a different problem. Some remote and parallel
filing systems handle such things very badly, and it is often possible
to create a file even if it exists and O_CREAT|O_EXCL is set. A similar
remark applies to 'special' files, even on fairly mainstream, local
filing systems.
Regards,
Nick Maclaren.
Jan 18 '07 #7
js wrote:
How about using lock?
Let writing process locks the files before writing, and unlock after
the job's done.
Is locking mandatory or co-operative? I don't have any control over the
process which is doing the writing, so if it's co-operative it's no good to
me.

If it's mandatory, then I can try to acquire a lock on the file - if this
fails or blocks, then the other process must have it open. Will this work?
Jan 19 '07 #8
Tom Wright wrote:
js wrote:
>How about using lock?
Let writing process locks the files before writing, and unlock after
the job's done.

Is locking mandatory or co-operative? I don't have any control over the
process which is doing the writing, so if it's co-operative it's no good
to me.

If it's mandatory, then I can try to acquire a lock on the file - if this
fails or blocks, then the other process must have it open. Will this
work?
AFAIK it's cooperative.

Diez
Jan 19 '07 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Peter ┼strand | last post by:
There's a new PEP available: PEP 324: popen5 - New POSIX process module A copy is included below. Comments are appreciated. ---- PEP:...
1
by: Dan Jones | last post by:
I'm writing a script to process a directory tree of images.  In each directory, I need to process each image and generate an HTML file listing...
2
by: Sri | last post by:
I am writing an asp.net applicaition using VB coding. In a function, I am opening an excel file with the following code, Dim objExcel As Object...
2
by: John Regan | last post by:
Hello All I am trying to find the owner of a file or folder on our network (Windows 2000 Server) using VB.Net and/or API. so I can search for...
5
by: su | last post by:
to find which process dumped core at the promt we give $ file core.28424 core.28424: ELF 32-bit LSB core file of 'soffice.bin' (signal 11),...
9
by: Carl | last post by:
Hi Is it possible to find out if another process is using a paticular file (xml)? There does not seem to be anything useful in the File och...
3
by: moongirl | last post by:
Part of an application I am building includes a button which the user can click on to view a PDF file in Acrobat Reader, using the following code:...
3
by: dittytwo | last post by:
Hi all I have been looking around the web and can't seem to find a solution the solution that i have found and manipulated seems to bring...
0
by: tammygombez | last post by:
Hey everyone! I've been researching gaming laptops lately, and I must say, they can get pretty expensive. However, I've come across some great...
0
better678
by: better678 | last post by:
Question: Discuss your understanding of the Java platform. Is the statement "Java is interpreted" correct? Answer: Java is an object-oriented...
0
by: teenabhardwaj | last post by:
How would one discover a valid source for learning news, comfort, and help for engineering designs? Covering through piles of books takes a lot of...
0
by: CD Tom | last post by:
This happens in runtime 2013 and 2016. When a report is run and then closed a toolbar shows up and the only way to get it to go away is to right...
0
jalbright99669
by: jalbright99669 | last post by:
Am having a bit of a time with URL Rewrite. I need to incorporate http to https redirect with a reverse proxy. I have the URL Rewrite rules made...
0
by: antdb | last post by:
Ⅰ. Advantage of AntDB: hyper-convergence + streaming processing engine In the overall architecture, a new "hyper-convergence" concept was...
0
by: Matthew3360 | last post by:
Hi there. I have been struggling to find out how to use a variable as my location in my header redirect function. Here is my code. ...
2
by: Matthew3360 | last post by:
Hi, I have a python app that i want to be able to get variables from a php page on my webserver. My python app is on my computer. How would I make it...
0
by: AndyPSV | last post by:
HOW CAN I CREATE AN AI with an .executable file that would suck all files in the folder and on my computerHOW CAN I CREATE AN AI with an .executable...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.