I'm having a number of problems with the fcntl module. First off, my
system info:
Mac OS X
Darwin igskcicglthearn.cr.usgs.gov 8.10.1 Darwin Kernel Version
8.10.1: Wed May 23 16:33:00 PDT 2007; root:xnu-792.22.5~1/RELEASE_I386
i386 i386
Python 2.5.1 (built from source)
OK, the weirdness:
First of all, if I try this:
file = open("counter.txt","w+")
fcntl.flock(file.fileno(), fcntl.LOCK_NB)
I get this:
---------------------------------------------------------------------------
<type 'exceptions.IOError' Traceback (most recent call
last)
/Users/mhearne/src/python/<ipython consolein <module>()
<type 'exceptions.IOError'>: [Errno 9] Bad file descriptor
However, if I try this:
fcntl.flock(file.fileno(), fcntl.LOCK_EX)
I get no errors.
Proceeding forward with the locked file, let's say I do the above in
Python interactive Process A. Then in python interactive Process B, I
repeat the "open" function on the same file with the same
permissions. Then, in each process, I write some text to the file
using the write() method. After closing the file in both processes,
the only text I see in the file is from Process B!
According to my Python Cookbook:
"Exclusive lock: This denies all _other_ processes both read and write
access to the file."
I seem to be experiencing the reverse of that description. Is this my
lack of understanding, or have I discovered a bug?
Thanks,
Mike 9 5354
On Aug 30, 4:19 pm, "mhearne808[insert-at-sign-here]gmail[insert-dot-
here]com" <mhearne...@gmail.comwrote:
I'm having a number of problems with the fcntl module. First off, my
system info:
Mac OS X
Darwin igskcicglthearn.cr.usgs.gov 8.10.1 Darwin Kernel Version
8.10.1: Wed May 23 16:33:00 PDT 2007; root:xnu-792.22.5~1/RELEASE_I386
i386 i386
Python 2.5.1 (built from source)
OK, the weirdness:
First of all, if I try this:
file = open("counter.txt","w+")
fcntl.flock(file.fileno(), fcntl.LOCK_NB)
I get this:
---------------------------------------------------------------------------
<type 'exceptions.IOError' Traceback (most recent call
last)
/Users/mhearne/src/python/<ipython consolein <module>()
<type 'exceptions.IOError'>: [Errno 9] Bad file descriptor
However, if I try this:
fcntl.flock(file.fileno(), fcntl.LOCK_EX)
I get no errors.
Proceeding forward with the locked file, let's say I do the above in
Python interactive Process A. Then in python interactive Process B, I
repeat the "open" function on the same file with the same
permissions. Then, in each process, I write some text to the file
using the write() method. After closing the file in both processes,
the only text I see in the file is from Process B!
According to my Python Cookbook:
"Exclusive lock: This denies all _other_ processes both read and write
access to the file."
I seem to be experiencing the reverse of that description. Is this my
lack of understanding, or have I discovered a bug?
Thanks,
Mike
I've been doing some experiments, and here are some specific examples
to try. I'll designate the two interactive python processes as PA and
PB. Both processes were started in the same directory. Here goes:
PA: import fcntl
PA: f = open("foo.txt","w+")
PA: fcntl.flock(f.fileno(),fcntl.LOCK_EX)
PA: f.write("text1\n")
PB: f = open("foo.txt","w+")
PB: f.write("text2\n")
PA: f.close()
PB: f.close()
contents of foo.txt are:
text2
Second experiment:
PA: f = open("foo.txt","w+")
PA: fcntl.flock(f.fileno(),fcntl.LOCK_EX)
PB: f = open("foo.txt","w+")
PA: f.write("text1\n")
PB: f.write("text2\n")
PA: f.write("text3\n")
PB: f.close()
PA: f.write("text4\n")
PA: f.close()
contents of foo.txt are:
text1
text3
text4
Third experiment:
PA: f = open("foo.txt","w+")
PA: fcntl.flock(f.fileno(),fcntl.LOCK_EX)
PA: f.write("text1\n")
PB: f = open("foo.txt","w+")
PB: f.write("text2\n")
PB: f.close()
PA: f.close()
contents of foo.txt are:
text1
Fourth experiment:
PA: f = open("foo.txt","w+")
PA: f.write("text1\n")
PB: f = open("foo.txt","w+")
PB: f.write("text2\n")
PB: f.close()
PA: f.close()
contents of foo.txt are:
text1
>From these last two experiments I can only conclude that file locking
isn't doing a durned thing.
What's going on?
--Mike
On 8/30/07, mhearne808 wrote:
I'm having a number of problems with the fcntl module.
Read this first: http://linux.die.net/man/2/flock
First of all, if I try this:
file = open("counter.txt","w+")
fcntl.flock(file.fileno(), fcntl.LOCK_NB)
I get this:
---------------------------------------------------------------------------
<type 'exceptions.IOError' Traceback (most recent call
last)
/Users/mhearne/src/python/<ipython consolein <module>()
<type 'exceptions.IOError'>: [Errno 9] Bad file descriptor
That should be:
>>fcntl.flock(f.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
Proceeding forward with the locked file, let's say I do the above in
Python interactive Process A. Then in python interactive Process B, I
repeat the "open" function on the same file with the same
permissions. Then, in each process, I write some text to the file
using the write() method. After closing the file in both processes,
the only text I see in the file is from Process B!
This is due to two issues: caching and file position. When you open
the file in both processes as 'w+', they are both positioned at the
*current* EOF, but from that point on the offset is not externally
influenced. The correct sequence of events should be:
- open file in mode w+
- obtain exclusive lock
- f.seek(0, 2) # (to end of file)
- write to file
- f.flush() # or f.close()
- release lock
Is this my lack of understanding, or have I discovered a bug?
If you find yourself asking this question, it's too often the former :)
-Miles
Sorry, that last quote-only reply was accidental. :)
On 8/30/07, mhearne808 wrote:
I've been doing some experiments, and here are some specific examples
to try.
[snipped examples]
From these last two experiments I can only conclude that file locking
isn't doing a durned thing.
What's going on?
File locking isn't doing a durned thing in those cases because you're
only obtaining the lock from a single process.
According to my Python Cookbook:
"Exclusive lock: This denies all _other_ processes both read and write
access to the file."
This is only for mandatory locking; POSIX flock is advisory locking,
which states: "Only one process may hold an exclusive lock for a given
file at a given time." Advisory locks don't have any effect on
processes that don't use locks. Mandatory locks are kernel enforced,
but non-POSIX and not available in Mac OS X.
-Miles
On Aug 31, 12:23 am, Miles <semantic...@gmail.comwrote:
Sorry, that last quote-only reply was accidental. :)
On 8/30/07, mhearne808 wrote:
I've been doing some experiments, and here are some specific examples
to try.
[snipped examples]
From these last two experiments I can only conclude that file locking
isn't doing a durned thing.
What's going on?
File locking isn't doing a durned thing in those cases because you're
only obtaining the lock from a single process.
According to my Python Cookbook:
"Exclusive lock: This denies all _other_ processes both read and write
access to the file."
This is only for mandatory locking; POSIX flock is advisory locking,
which states: "Only one process may hold an exclusive lock for a given
file at a given time." Advisory locks don't have any effect on
processes that don't use locks. Mandatory locks are kernel enforced,
but non-POSIX and not available in Mac OS X.
-Miles
I think I'm still confused. Maybe I should explain the behavior that
I want, and then figure out if it is possible.
I have a script that will be run from a cron job once a minute. One
of the things this script will do is open a file to stash some
temporary results. I expect that this script will always finish its
work in less than 15 seconds, but I didn't want to depend on that.
Thus I started to look into file locking, which I had hoped I could
use in the following fashion:
Process A opens file foo
Process A locks file foo
Process A takes more than a minute to do its work
Process B wakes up
Process B determines that file foo is locked
Process B quits in disgust
Process A finishes its work
Since I couldn't figure out file locking, I decided to just have
Process A create a "pid" file in the directory - analogous to the
"Occupied" sign on an airplane bathroom. This works, but it seems a
little hacky.
--Mike
On 8/31/07, mhearne808 wrote:
I have a script that will be run from a cron job once a minute. One
of the things this script will do is open a file to stash some
temporary results. I expect that this script will always finish its
work in less than 15 seconds, but I didn't want to depend on that.
Thus I started to look into file locking, which I had hoped I could
use in the following fashion:
Process A opens file foo
Process A locks file foo
Process A takes more than a minute to do its work
Process B wakes up
Process B determines that file foo is locked
Process B quits in disgust
Process A finishes its work
That would look like (untested):
import fcntl, sys
f = open('foo', 'w+')
try:
fcntl.flock(f.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
except IOError, e:
if e.args[0] == 35:
sys.exit(1)
else:
raise
f.seek(0, 2) # seek to end
# do your thing with the file
f.flush()
fcntl.flock(f.fileno(), fcntl.LOCK_UN)
f.close()
-Miles
"mhearne808[insert-at-sign-here]gmail[insert-dot-here]com" <mh********@gmail.comwrites:
I think I'm still confused.
What Miles tried to tell you is that you should call fcnt.flock from
both PA and PB. In the example you posted, you failed to call it from
PB. No lock call, so no locking happened.
I have a script that will be run from a cron job once a minute. One
of the things this script will do is open a file to stash some
temporary results. I expect that this script will always finish its
work in less than 15 seconds, but I didn't want to depend on that.
Thus I started to look into file locking, which I had hoped I could
use in the following fashion:
Process A opens file foo
Process A locks file foo
Process A takes more than a minute to do its work
Process B wakes up
Process B determines that file foo is locked
Process B quits in disgust
Process A finishes its work
File locking supports that scenario, as you suspected. You need to
use flock with LOCK_EX|LOCK_NB. If the call succeeds, you got the
lock. If you get an exception whose errno is EWOULDBLOCK, you quit in
disgust.
On 8/31/07, mhearne808 wrote:
Looking at my flock(3) man page, I'm guessing that "35" is the error
code for EWOULDBLOCK. Which system header file am I supposed to look
in to figure that magic number out?
I got the error number by looking at the IOError exception raised when
playing with the interactive interpreter, but I really should have
written:
from errno import EWOULDBLOCK
....
if e.args[0] == EWOULDBLOCK:
....
- Miles
In article <11**********************@o80g2000hse.googlegroups .com>,
"mhearne808[insert-at-sign-here]gmail[insert-dot-here]com"
<mh********@gmail.comwrote:
Looking at my flock(3) man page, I'm guessing that "35" is the error
code for EWOULDBLOCK. Which system header file am I supposed to look
in to figure that magic number out?
On a MacOS system, you can find them in /usr/include/sys/errno.h
On a Linux system, try /usr/include/asm-generic/errno.h
However, if you're writing in Python, you will probably have an easier
time using the "errno" module, e.g.,
] import errno
] errno.errorcode[35]
'EDEADLOCK'
Note that some codes have multiple names (e.g., EAGAIN and EWOULDBLOCK)
so that this lookup may not return exactly the name you're expecting.
Cheers,
-M
--
Michael J. Fromberger | Lecturer, Dept. of Computer Science http://www.dartmouth.edu/~sting/ | Dartmouth College, Hanover, NH, USA
In message <ma**************************************@python.o rg>, Miles
wrote:
except IOError, e:
if e.args[0] == 35:
Why not
except IOError, (ErrNo, Msg) :
if ErrNo == errno.EAGAIN : This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Meyer, Tony |
last post by:
(I did try to google for an answer to this, but couldn't find anything,
although plenty of instances of the warning).
I don't understand this warning:
>>> import fcntl
C:\Program...
|
by: Pierre Rouleau |
last post by:
Hi all!
I am using Python 2.3.1 on Win32 (NT, 2000).
Whenever a file imports the standard tempfile module, Python 2.3.1
issues the following warning:
C:\Python23\lib\fcntl.py:7:...
|
by: Ryan Grow |
last post by:
Hi,
I'm trying to use fcntl to set an existing file
descriptor to be nonblocking. This contrived example
exhibits the behavior of python that is preventing me
from doing this:
import os,...
|
by: Chris Green |
last post by:
Hey folks,
Is there anyway for a signal handler in python to get the information
from a 3 argument signal handler rather than just the signal number
and stack frame?
I've got an application...
|
by: Omid Fatemi |
last post by:
I followed discussion in the group about this problem:
ImportError: No module named fcntl
I found out there are two modules:
FCNTL
fcntl
the first one is obsolete and shouldn't be used.
But...
| |
by: thakadu |
last post by:
The following code works as expected when run in the main body of a
python script (ver 2.3.5) on OpenBSD v3.8. but when it is in the body
of a function definition it does not work. It does not...
|
by: marcello |
last post by:
Hello
I need to do this:
1 opening a file for writing/appending
2 to lock the file as for writing (i mean: the program
that lock can keep writing, all others programs can't )
3 wtite and...
|
by: Mitko Haralanov |
last post by:
I am trying to use the advisory locking with fcntl over NFS (thus, me
choosing fcntl instead of flock and friends).
I have the following code:
lockdata = struct.pack ("hhllhh", fcntl.F_RDLCK, 0,...
|
by: xucs007 |
last post by:
I ran following 2 programs (lock1, lock2) at almost same time,
to write either "123456", or "222" to file "aaa" at the same time.
But I often just got "222456" in "aaa" .
Is this a bug of python...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
| |
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated ...
| |
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |