473,700 Members | 2,350 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

How to find out if another process is using a file

I'm writing a program which reads a series of data files as they are dumped
into a directory by another process. At the moment, it gets sporadic bugs
when it tries to read files which are only partially written.

I'm looking for a function which will tell me if a file is opened in
write-mode by another process - if it is, my program will ignore it for now
and come back to it later. This needs to work on linux and windows. Mac
OS would be a bonus too. An os-independent solution would be good, but I
could write os-specific options and have it pick the appropriate one.

Is there any way of doing this? I've drawn a blank with google so far.

A nasty hack would be to use the file modification time, and wait until
that's a few seconds in the past, but is there a nice solution?
--
I'm at CAMbridge, not SPAMbridge
Jan 18 '07 #1
8 4998
js
How about using lock?
Let writing process locks the files before writing, and unlock after
the job's done.

I think it'd work file in most environment.

On 1/19/07, Tom Wright <te***@spam.ac. ukwrote:
I'm writing a program which reads a series of data files as they are dumped
into a directory by another process. At the moment, it gets sporadic bugs
when it tries to read files which are only partially written.

I'm looking for a function which will tell me if a file is opened in
write-mode by another process - if it is, my program will ignore it for now
and come back to it later. This needs to work on linux and windows. Mac
OS would be a bonus too. An os-independent solution would be good, but I
could write os-specific options and have it pick the appropriate one.

Is there any way of doing this? I've drawn a blank with google so far.

A nasty hack would be to use the file modification time, and wait until
that's a few seconds in the past, but is there a nice solution?
--
I'm at CAMbridge, not SPAMbridge
--
http://mail.python.org/mailman/listinfo/python-list
Jan 18 '07 #2
"Tom Wright" <te***@spam.ac. ukescribiů en el mensaje
news:eo******** **@gemini.csx.c am.ac.uk...
I'm writing a program which reads a series of data files as they are
dumped
into a directory by another process. At the moment, it gets sporadic bugs
when it tries to read files which are only partially written.

I'm looking for a function which will tell me if a file is opened in
write-mode by another process - if it is, my program will ignore it for
now
and come back to it later. This needs to work on linux and windows. Mac
OS would be a bonus too. An os-independent solution would be good, but I
could write os-specific options and have it pick the appropriate one.
Use os.open with the O_EXCL flag; will fail if the other process has the
file still open (and will fail if another process is reading the file, too,
not just if someone is writing).

--
Gabriel Genellina
Jan 18 '07 #3
In article <ma************ *************** ************@py thon.org>,
"Gabriel Genellina" <ga******@yahoo .com.arwrote:
"Tom Wright" <te***@spam.ac. ukescribiů en el mensaje
news:eo******** **@gemini.csx.c am.ac.uk...
I'm writing a program which reads a series of data files as they are
dumped
into a directory by another process. At the moment, it gets sporadic bugs
when it tries to read files which are only partially written.

I'm looking for a function which will tell me if a file is opened in
write-mode by another process - if it is, my program will ignore it for
now
and come back to it later. This needs to work on linux and windows. Mac
OS would be a bonus too. An os-independent solution would be good, but I
could write os-specific options and have it pick the appropriate one.

Use os.open with the O_EXCL flag; will fail if the other process has the
file still open (and will fail if another process is reading the file, too,
not just if someone is writing).
O_EXCL fails if the file exists at all - whether closed or open.

Donn Cave, do**@u.washingt on.edu
Jan 18 '07 #4

In article <do************ ************@gn us01.u.washingt on.edu>,
Donn Cave <do**@u.washing ton.eduwrites:
|In article <ma************ *************** ************@py thon.org>,
| "Gabriel Genellina" <ga******@yahoo .com.arwrote:
| "Tom Wright" <te***@spam.ac. ukescribiů en el mensaje
| news:eo******** **@gemini.csx.c am.ac.uk...
|
| I'm writing a program which reads a series of data files as they are
| dumped
| into a directory by another process. At the moment, it gets sporadic bugs
| when it tries to read files which are only partially written.
| >
| I'm looking for a function which will tell me if a file is opened in
| write-mode by another process - if it is, my program will ignore it for
| now
| and come back to it later. This needs to work on linux and windows. Mac
| OS would be a bonus too. An os-independent solution would be good, but I
| could write os-specific options and have it pick the appropriate one.
|
| Use os.open with the O_EXCL flag; will fail if the other process has the
| file still open (and will fail if another process is reading the file, too,
| not just if someone is writing).
|>
|O_EXCL fails if the file exists at all - whether closed or open.

Yes. In theory. In practice, it usually works on normal files, provided
that all opens are local. Under some circumstances, it will even work
for NFS mounted files, as far as I recall.

MVS (now zOS) and other mainframe systems had what the poster wants, but
modern systems don't. I shall not recommend that the poster asks IBM
for a copy of zOS, for reasons that will be well-known to anyone who used
MVS (which are NOT the ones claimed by the Unix brigade, which are mostly
bogus) :-)

Under Linux, you can do something with fuser, and I am pretty certain that
modern Macintoshes (i.e. BSD) will have an equivalent. It can't be made
reliable (unlike under MVS), but might reduce the number of problems.
Regards,
Nick Maclaren.
Jan 18 '07 #5
In article <eo**********@g emini.csx.cam.a c.uk>,
nm**@cus.cam.ac .uk (Nick Maclaren) wrote:
In article <do************ ************@gn us01.u.washingt on.edu>,
Donn Cave <do**@u.washing ton.eduwrites:
|In article <ma************ *************** ************@py thon.org>,
| "Gabriel Genellina" <ga******@yahoo .com.arwrote:
| "Tom Wright" <te***@spam.ac. ukescribiů en el mensaje
| news:eo******** **@gemini.csx.c am.ac.uk...
| Use os.open with the O_EXCL flag; will fail if the other process has the
| file still open (and will fail if another process is reading the file,
| too,
| not just if someone is writing).
|>
|O_EXCL fails if the file exists at all - whether closed or open.

Yes. In theory. In practice, it usually works on normal files, provided
that all opens are local. Under some circumstances, it will even work
for NFS mounted files, as far as I recall.
Mm, by "fail", I meant

An attempt to open with O_EXCL set will "fail" if the file exists at all,
i.e., the file will not be opened, a negative value will be returned,
and errno will be set to EEXIST.

What I neglected to mention is that this effect obtains when O_EXCL
is used in combination with O_CREAT. Without O_CREAT, O_EXCL doesn't
mean anything and is ignored.

If there is any significant difference between theory and practice
in this matter, it's news to me.

Donn
Jan 18 '07 #6

In article <do************ ************@gn us01.u.washingt on.edu>,
Donn Cave <do**@u.washing ton.eduwrites:
| |>
| |O_EXCL fails if the file exists at all - whether closed or open.
|
| Yes. In theory. In practice, it usually works on normal files, provided
| that all opens are local. Under some circumstances, it will even work
| for NFS mounted files, as far as I recall.
|>
|Mm, by "fail", I meant
|>
|An attempt to open with O_EXCL set will "fail" if the file exists at all,
|i.e., the file will not be opened, a negative value will be returned,
|and errno will be set to EEXIST.
|>
|What I neglected to mention is that this effect obtains when O_EXCL
|is used in combination with O_CREAT. Without O_CREAT, O_EXCL doesn't
|mean anything and is ignored.
|>
|If there is any significant difference between theory and practice
|in this matter, it's news to me.

Actually, it is undefined behaviour in POSIX, and I have used a Unix
where O_EXCL on its own failed if the file existed (of course, it also
failed if it DIDN'T exist, because of the lack of O_CREAT). I can't
remember which - perhaps AIX or OSF/1 (not DEC).

But I was referring to a different problem. Some remote and parallel
filing systems handle such things very badly, and it is often possible
to create a file even if it exists and O_CREAT|O_EXCL is set. A similar
remark applies to 'special' files, even on fairly mainstream, local
filing systems.
Regards,
Nick Maclaren.
Jan 18 '07 #7
js wrote:
How about using lock?
Let writing process locks the files before writing, and unlock after
the job's done.
Is locking mandatory or co-operative? I don't have any control over the
process which is doing the writing, so if it's co-operative it's no good to
me.

If it's mandatory, then I can try to acquire a lock on the file - if this
fails or blocks, then the other process must have it open. Will this work?
Jan 19 '07 #8
Tom Wright wrote:
js wrote:
>How about using lock?
Let writing process locks the files before writing, and unlock after
the job's done.

Is locking mandatory or co-operative? I don't have any control over the
process which is doing the writing, so if it's co-operative it's no good
to me.

If it's mandatory, then I can try to acquire a lock on the file - if this
fails or blocks, then the other process must have it open. Will this
work?
AFAIK it's cooperative.

Diez
Jan 19 '07 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
2598
by: Peter Ňstrand | last post by:
There's a new PEP available: PEP 324: popen5 - New POSIX process module A copy is included below. Comments are appreciated. ---- PEP: 324 Title: popen5 - New POSIX process module
1
7589
by: Dan Jones | last post by:
I'm writing a script to process a directory tree of images.  In each directory, I need to process each image and generate an HTML file listing all of the images and links to the subdirectories. Just about every source I can find on the 'net for processing subdirectories points you at Find::Find.  However, I'm trying to do something like this: enter directory open INDEX, ".\index.html" print INDEX HTMLheader
2
10694
by: Sri | last post by:
I am writing an asp.net applicaition using VB coding. In a function, I am opening an excel file with the following code, Dim objExcel As Object Dim objWorkBook As Object objExcel = CreateObject("Excel.Application") objWorkBook = objExcel.Workbooks.Open(targetfilename) * targetfilename is a variable that stores the excel file name and path. In the error handler,
2
5042
by: John Regan | last post by:
Hello All I am trying to find the owner of a file or folder on our network (Windows 2000 Server) using VB.Net and/or API. so I can search for Folders that don't follow our company's specified folder structure and naming conventions and then send a Net send message to those users telling them to rectify. The information I want to get is when you select the file/folder and then: Properties -> Security Tab -> Advanced Button -> Owner Tab ->...
5
4058
by: su | last post by:
to find which process dumped core at the promt we give $ file core.28424 core.28424: ELF 32-bit LSB core file of 'soffice.bin' (signal 11), Intel 80386, version 1 (SYSV), from 'soffice.bin' from this command we know 'soffice.bin' process dumped core. Now can i do the same using python i.e. finding which process dumped core? if so how can i do it?
9
5405
by: Carl | last post by:
Hi Is it possible to find out if another process is using a paticular file (xml)? There does not seem to be anything useful in the File och Filestream class for this purpose. regards Carl
3
5263
by: moongirl | last post by:
Part of an application I am building includes a button which the user can click on to view a PDF file in Acrobat Reader, using the following code: System.Diagnostics.Process.Start(@"C:\Program Files \Adobe\Reader 8.0\Reader\AcroRd32.exe", @PDFfilePath); However the path to acrobat reader is the path on my PC - what if the path is different on the user's PC? How can I change the code to find
3
5921
by: dittytwo | last post by:
Hi all I have been looking around the web and can't seem to find a solution the solution that i have found and manipulated seems to bring back the whole list of currently running process's which hogs a lot of CPU time in its self.. basic idea of my program is i have multiple threads doing things and i have a thread that gathers data within this thread i can currently get all sorts of information using ...
0
8725
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, weíll explore What is ONU, What Is Router, ONU & Routerís main usage, and What is the difference between ONU and Router. Letís take a closer look ! Part I. Meaning of...
0
9214
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9074
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
8924
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
6560
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Duprť who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
4656
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
3088
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
2392
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2027
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.