469,292 Members | 1,500 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,292 developers. It's quick & easy.

How to find out if another process is using a file

I'm writing a program which reads a series of data files as they are dumped
into a directory by another process. At the moment, it gets sporadic bugs
when it tries to read files which are only partially written.

I'm looking for a function which will tell me if a file is opened in
write-mode by another process - if it is, my program will ignore it for now
and come back to it later. This needs to work on linux and windows. Mac
OS would be a bonus too. An os-independent solution would be good, but I
could write os-specific options and have it pick the appropriate one.

Is there any way of doing this? I've drawn a blank with google so far.

A nasty hack would be to use the file modification time, and wait until
that's a few seconds in the past, but is there a nice solution?
--
I'm at CAMbridge, not SPAMbridge
Jan 18 '07 #1
8 4673
js
How about using lock?
Let writing process locks the files before writing, and unlock after
the job's done.

I think it'd work file in most environment.

On 1/19/07, Tom Wright <te***@spam.ac.ukwrote:
I'm writing a program which reads a series of data files as they are dumped
into a directory by another process. At the moment, it gets sporadic bugs
when it tries to read files which are only partially written.

I'm looking for a function which will tell me if a file is opened in
write-mode by another process - if it is, my program will ignore it for now
and come back to it later. This needs to work on linux and windows. Mac
OS would be a bonus too. An os-independent solution would be good, but I
could write os-specific options and have it pick the appropriate one.

Is there any way of doing this? I've drawn a blank with google so far.

A nasty hack would be to use the file modification time, and wait until
that's a few seconds in the past, but is there a nice solution?
--
I'm at CAMbridge, not SPAMbridge
--
http://mail.python.org/mailman/listinfo/python-list
Jan 18 '07 #2
"Tom Wright" <te***@spam.ac.ukescribiˇ en el mensaje
news:eo**********@gemini.csx.cam.ac.uk...
I'm writing a program which reads a series of data files as they are
dumped
into a directory by another process. At the moment, it gets sporadic bugs
when it tries to read files which are only partially written.

I'm looking for a function which will tell me if a file is opened in
write-mode by another process - if it is, my program will ignore it for
now
and come back to it later. This needs to work on linux and windows. Mac
OS would be a bonus too. An os-independent solution would be good, but I
could write os-specific options and have it pick the appropriate one.
Use os.open with the O_EXCL flag; will fail if the other process has the
file still open (and will fail if another process is reading the file, too,
not just if someone is writing).

--
Gabriel Genellina
Jan 18 '07 #3
In article <ma***************************************@python. org>,
"Gabriel Genellina" <ga******@yahoo.com.arwrote:
"Tom Wright" <te***@spam.ac.ukescribiˇ en el mensaje
news:eo**********@gemini.csx.cam.ac.uk...
I'm writing a program which reads a series of data files as they are
dumped
into a directory by another process. At the moment, it gets sporadic bugs
when it tries to read files which are only partially written.

I'm looking for a function which will tell me if a file is opened in
write-mode by another process - if it is, my program will ignore it for
now
and come back to it later. This needs to work on linux and windows. Mac
OS would be a bonus too. An os-independent solution would be good, but I
could write os-specific options and have it pick the appropriate one.

Use os.open with the O_EXCL flag; will fail if the other process has the
file still open (and will fail if another process is reading the file, too,
not just if someone is writing).
O_EXCL fails if the file exists at all - whether closed or open.

Donn Cave, do**@u.washington.edu
Jan 18 '07 #4

In article <do************************@gnus01.u.washington.ed u>,
Donn Cave <do**@u.washington.eduwrites:
|In article <ma***************************************@python. org>,
| "Gabriel Genellina" <ga******@yahoo.com.arwrote:
| "Tom Wright" <te***@spam.ac.ukescribiˇ en el mensaje
| news:eo**********@gemini.csx.cam.ac.uk...
|
| I'm writing a program which reads a series of data files as they are
| dumped
| into a directory by another process. At the moment, it gets sporadic bugs
| when it tries to read files which are only partially written.
| >
| I'm looking for a function which will tell me if a file is opened in
| write-mode by another process - if it is, my program will ignore it for
| now
| and come back to it later. This needs to work on linux and windows. Mac
| OS would be a bonus too. An os-independent solution would be good, but I
| could write os-specific options and have it pick the appropriate one.
|
| Use os.open with the O_EXCL flag; will fail if the other process has the
| file still open (and will fail if another process is reading the file, too,
| not just if someone is writing).
|>
|O_EXCL fails if the file exists at all - whether closed or open.

Yes. In theory. In practice, it usually works on normal files, provided
that all opens are local. Under some circumstances, it will even work
for NFS mounted files, as far as I recall.

MVS (now zOS) and other mainframe systems had what the poster wants, but
modern systems don't. I shall not recommend that the poster asks IBM
for a copy of zOS, for reasons that will be well-known to anyone who used
MVS (which are NOT the ones claimed by the Unix brigade, which are mostly
bogus) :-)

Under Linux, you can do something with fuser, and I am pretty certain that
modern Macintoshes (i.e. BSD) will have an equivalent. It can't be made
reliable (unlike under MVS), but might reduce the number of problems.
Regards,
Nick Maclaren.
Jan 18 '07 #5
In article <eo**********@gemini.csx.cam.ac.uk>,
nm**@cus.cam.ac.uk (Nick Maclaren) wrote:
In article <do************************@gnus01.u.washington.ed u>,
Donn Cave <do**@u.washington.eduwrites:
|In article <ma***************************************@python. org>,
| "Gabriel Genellina" <ga******@yahoo.com.arwrote:
| "Tom Wright" <te***@spam.ac.ukescribiˇ en el mensaje
| news:eo**********@gemini.csx.cam.ac.uk...
| Use os.open with the O_EXCL flag; will fail if the other process has the
| file still open (and will fail if another process is reading the file,
| too,
| not just if someone is writing).
|>
|O_EXCL fails if the file exists at all - whether closed or open.

Yes. In theory. In practice, it usually works on normal files, provided
that all opens are local. Under some circumstances, it will even work
for NFS mounted files, as far as I recall.
Mm, by "fail", I meant

An attempt to open with O_EXCL set will "fail" if the file exists at all,
i.e., the file will not be opened, a negative value will be returned,
and errno will be set to EEXIST.

What I neglected to mention is that this effect obtains when O_EXCL
is used in combination with O_CREAT. Without O_CREAT, O_EXCL doesn't
mean anything and is ignored.

If there is any significant difference between theory and practice
in this matter, it's news to me.

Donn
Jan 18 '07 #6

In article <do************************@gnus01.u.washington.ed u>,
Donn Cave <do**@u.washington.eduwrites:
| |>
| |O_EXCL fails if the file exists at all - whether closed or open.
|
| Yes. In theory. In practice, it usually works on normal files, provided
| that all opens are local. Under some circumstances, it will even work
| for NFS mounted files, as far as I recall.
|>
|Mm, by "fail", I meant
|>
|An attempt to open with O_EXCL set will "fail" if the file exists at all,
|i.e., the file will not be opened, a negative value will be returned,
|and errno will be set to EEXIST.
|>
|What I neglected to mention is that this effect obtains when O_EXCL
|is used in combination with O_CREAT. Without O_CREAT, O_EXCL doesn't
|mean anything and is ignored.
|>
|If there is any significant difference between theory and practice
|in this matter, it's news to me.

Actually, it is undefined behaviour in POSIX, and I have used a Unix
where O_EXCL on its own failed if the file existed (of course, it also
failed if it DIDN'T exist, because of the lack of O_CREAT). I can't
remember which - perhaps AIX or OSF/1 (not DEC).

But I was referring to a different problem. Some remote and parallel
filing systems handle such things very badly, and it is often possible
to create a file even if it exists and O_CREAT|O_EXCL is set. A similar
remark applies to 'special' files, even on fairly mainstream, local
filing systems.
Regards,
Nick Maclaren.
Jan 18 '07 #7
js wrote:
How about using lock?
Let writing process locks the files before writing, and unlock after
the job's done.
Is locking mandatory or co-operative? I don't have any control over the
process which is doing the writing, so if it's co-operative it's no good to
me.

If it's mandatory, then I can try to acquire a lock on the file - if this
fails or blocks, then the other process must have it open. Will this work?
Jan 19 '07 #8
Tom Wright wrote:
js wrote:
>How about using lock?
Let writing process locks the files before writing, and unlock after
the job's done.

Is locking mandatory or co-operative? I don't have any control over the
process which is doing the writing, so if it's co-operative it's no good
to me.

If it's mandatory, then I can try to acquire a lock on the file - if this
fails or blocks, then the other process must have it open. Will this
work?
AFAIK it's cooperative.

Diez
Jan 19 '07 #9

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

1 post views Thread by Peter ┼strand | last post: by
1 post views Thread by Dan Jones | last post: by
3 posts views Thread by moongirl | last post: by
reply views Thread by suresh191 | last post: by
reply views Thread by harlem98 | last post: by
1 post views Thread by Geralt96 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.