is file open in system ? - other than lsof

[bvidinli]

is there a way to find out if file open in system ? -
please write if you know a way other than lsof. because lsof if slow for me.
i need a faster way.
i deal with thousands of files... so, i need a faster / python way for this.
thanks.
--
Ý.Bahattin Vidinli
Elk-Elektronik Müh.
-------------------
iletisim bilgileri (Tercih sirasina gore):
skype: bvidinli (sesli gorusme icin, www.skype.com)
msn: bv******@iyibirisi.com
yahoo: bvidinli

+90.532.7990607
+90.505.5667711

[A.T.Hofkamp]

On 2008-04-16, bvidinli <bv******@gmail.comwrote:

is there a way to find out if file open in system ? -
please write if you know a way other than lsof. because lsof if slow for me.
i need a faster way.
i deal with thousands of files... so, i need a faster / python way for this.
thanks.

This is not a Python question but an OS question.
(Python is not going to deliver what the OS doesn't provide).

Please first find an alternative way at OS level (ie ask this question at an
appropiate OS news group). Once you have found that, you can think about Python
support for that alternative.
Sincerely,
Albert

[Chris McAloney]

On 16-Apr-08, at 9:20 AM, A.T.Hofkamp wrote:

On 2008-04-16, bvidinli <bv******@gmail.comwrote:

>is there a way to find out if file open in system ? -
please write if you know a way other than lsof. because lsof if
slow for me.
i need a faster way.
i deal with thousands of files... so, i need a faster / python way
for this.
thanks.

This is not a Python question but an OS question.
(Python is not going to deliver what the OS doesn't provide).

Please first find an alternative way at OS level (ie ask this
question at an
appropiate OS news group). Once you have found that, you can think
about Python
support for that alternative.

I agree with Albert that this is very operating-system specific.
Since you mentioned 'lsof', I'll assume that you are at least using a
Unix variant, meaning that the fcntl module will be available to you,
so you can check if the file is already locked.

Beyond that, I think more information on your application would be
necessary before we could give you a solid answer. Do you only need
to know if the file is open, or do you want only the files that are
open for writing? If you only care about the files that are open for
writing, then checking for a write-lock with fcntl will probably do
the trick. Are you planning to check all of the "thousands of files"
individually to determine if they're open? If so, I think it's
unlikely that doing this from Python will actually be faster than a
single 'lsof' call.

If you're on Linux, you might also want to have a look at the /proc
directory tree ("man proc"), as this is where lsof gets its
information from on Linux machines.

Chris

[Thomas Guettler]

bvidinli schrieb:

is there a way to find out if file open in system ? -
please write if you know a way other than lsof. because lsof if slow for me.
i need a faster way.
i deal with thousands of files... so, i need a faster / python way for this.
thanks.

On Linux there are symlinks from /proc/PID/fd to the open
files. You could use this:

#!/usr/bin/env python
import os
pids=os.listdir('/proc')
for pid in sorted(pids):
try:
int(pid)
except ValueError:
continue
fd_dir=os.path.join('/proc', pid, 'fd')
for file in os.listdir(fd_dir):
try:
link=os.readlink(os.path.join(fd_dir, file))
except OSError:
continue
print pid, link
--
Thomas Guettler, http://www.thomas-guettler.de/
E-Mail: guettli (*) thomas-guettler + de

[Nick Craig-Wood]

Thomas Guettler <[e-mail]:

bvidinli schrieb:

is there a way to find out if file open in system ? -
please write if you know a way other than lsof. because lsof if slow for me.
i need a faster way.
i deal with thousands of files... so, i need a faster / python way for this.
thanks.

On Linux there are symlinks from /proc/PID/fd to the open
files. You could use this:

#!/usr/bin/env python
import os
pids=os.listdir('/proc')
for pid in sorted(pids):
try:
int(pid)
except ValueError:
continue
fd_dir=os.path.join('/proc', pid, 'fd')
for file in os.listdir(fd_dir):
try:
link=os.readlink(os.path.join(fd_dir, file))
except OSError:
continue
print pid, link

Unfortunately I think that is pretty much exactly what lsof does so is
unlikely to be any faster!

However if you have 1000s of files to check you only need to do the
above scan once and build a dict with all open files in whereas lsof
will do it once per call so that may be a useful speedup.

Eg

------------------------------------------------------------
import os
pids=os.listdir('/proc')
open_files = {}
for pid in sorted(pids):
try:
int(pid)
except ValueError:
continue
fd_dir=os.path.join('/proc', pid, 'fd')
try:
fds = os.listdir(fd_dir)
except OSError:
continue
for file in fds:
try:
link=os.readlink(os.path.join(fd_dir, file))
except OSError:
continue
if not os.path.exists(link):
continue
open_files.setdefault(link, []).append(pid)

for link in sorted(open_files.keys()):
print "%s : %s" % (link, ", ".join(map(str, open_files[link])))
------------------------------------------------------------
You might want to consider http://pyinotify.sourceforge.net/ depending
on exactly what you are doing...

--
Nick Craig-Wood <ni**@craig-wood.com-- http://www.craig-wood.com/nick

[A.T.Hofkamp]

On 2008-04-17, bvidinli <bv******@gmail.comwrote:

is there another way, any python command sequence that i can check if
a file is open at the time of "before i process file"

i am not interested in " the file may be written after i access it.."
the important point is " the time at i first access it."

my routine is something like:
for i in listoffiles:
checkfileopen(i)
processfile()

This code does not give you what you are after; in between 'checkfileopen()' and
'processfile()' somebody may open the file and mess with it.

I quite doubt that you can get what you want, at OS level.

Even if you get exclusive open(), you are not safe imho.
After you opened the file for access (and before you perform the read() call),
another process may also open it, and alter the data while you are reading.
The same may happen between 2 read() calls.

Note that a read() at OS level is not the same as a read() at Python (or C
fread()) level. Python pretends to read an entire file in one call, but the OS
is using disk-blocks of the underlying file system as unit of read/write.
In other words, even if nobody messes with the file at the moment you open it,
you don't necessarily get an uncorrupted file afaik.

Sincerely,
Albert