473,224 Members | 1,477 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,224 software developers and data experts.

flock seems very unsafe, python fcntl bug?

I ran following 2 programs (lock1, lock2) at almost same time,
to write either "123456", or "222" to file "aaa" at the same time.
But I often just got "222456" in "aaa" .

Is this a bug of python fcntl module ? See 2 programs I ran:

#!/usr/bin/env python
import fcntl, time
file = open('aaa', "w")
fcntl.flock(file, fcntl.LOCK_EX)
file.write('123456')
time.sleep(10)
file.close()
#!/usr/bin/env python
import fcntl, time
file = open('aaa', "w")
fcntl.flock(file, fcntl.LOCK_EX)
file.write('222')
time.sleep(10)
file.close()
Jun 27 '08 #1
2 3619
Your problem is that open(...,'w') is not locked.

Use something like:
lockf = open('aaa', 'a')
fnctl.flock(lockf,fnctl.LOCK_EX)
file = open('aaa', 'w')
file.write('asdf')
file.close()
lockf.close()

xu*****@gmail.com wrote:
I ran following 2 programs (lock1, lock2) at almost same time,
to write either "123456", or "222" to file "aaa" at the same time.
But I often just got "222456" in "aaa" .

Is this a bug of python fcntl module ? See 2 programs I ran:

#!/usr/bin/env python
import fcntl, time
file = open('aaa', "w")
fcntl.flock(file, fcntl.LOCK_EX)
file.write('123456')
time.sleep(10)
file.close()
#!/usr/bin/env python
import fcntl, time
file = open('aaa', "w")
fcntl.flock(file, fcntl.LOCK_EX)
file.write('222')
time.sleep(10)
file.close()
Jun 27 '08 #2
Jens Henrik Leonhard Jensen <jh**@statsbiblioteket.dkwrote:
Your problem is that open(...,'w') is not locked.

Use something like:
lockf = open('aaa', 'a')
fnctl.flock(lockf,fnctl.LOCK_EX)
file = open('aaa', 'w')
file.write('asdf')
file.close()
lockf.close()
I've not seen that trick before - it is a good one. It wouldn't work
for mandatory locking though...

The basic problem is getting the file open for read/write without
destroying the contents before you have the lock. None of the
arguments to open() do the right thing....

Using an append open is a nice solution to that, apart from the fact
that you can only append data onto the file hence the other open.

Here are some other possible solutions and a test harness! The low
level open is probably the most unixy solution. If you could specify
os.O_RDWR|os.O_CREAT to the python open() function then it would be
the best, but none of the documented open strings do that

$ strace python -c 'open("aaa", "r+")' 2>&1 | grep 'open("aaa"'
open("aaa", O_RDWR|O_LARGEFILE) = 3
$ strace python -c 'open("aaa", "w+")' 2>&1 | grep 'open("aaa"'
open("aaa", O_RDWR|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
$ strace python -c 'open("aaa", "wr+")' 2>&1 | grep 'open("aaa"'
open("aaa", O_RDWR|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
$ strace python -c 'open("aaa", "a+")' 2>&1 | grep 'open("aaa"'
open("aaa", O_RDWR|O_CREAT|O_APPEND|O_LARGEFILE, 0666) = 3
------------------------------------------------------------
"""
Testing writing stuff to a file with locking
"""

import os
import sys
import fcntl

def method_0(data):
"""
Use an normal open

This doesn't work because it truncates the data before it has the
lock
"""
fd = open('aaa', 'w')
fcntl.flock(fd,fcntl.LOCK_EX)
fd.write(data)
fd.close()

def method_1(data):
"""
Use an additional append open to lock
"""
lock_fd = open('aaa', 'a')
fcntl.flock(lock_fd,fcntl.LOCK_EX)
fd = open('aaa', 'w')
fd.write(data)
fd.close()
lock_fd.close()

def method_2(data):
"""
Use a low level open
"""
fd = os.fdopen(os.open('aaa', os.O_CREAT|os.O_RDWR), "r+")
fcntl.flock(fd, fcntl.LOCK_EX)
fd.truncate()
fd.write(data)
fd.close()

def method_3(data):
"""
Using high level functions only

Possibly racy on open when file doesn't exist
"""
if not os.path.exists('aaa'):
fd = open('aaa', 'w+')
else:
fd = open('aaa', 'r+')
fcntl.flock(fd,fcntl.LOCK_EX)
fd.truncate()
fd.write(data)
fd.close()

def check():
fd = open('aaa', 'r')
fcntl.flock(fd,fcntl.LOCK_EX)
data = fd.read()
fd.close()
if data not in ("sausage", "potato"):
raise AssertionError("Wrong data %r" % data)

if os.path.exists("aaa"):
os.unlink("aaa")

if len(sys.argv) < 2:
print "Syntax: %s <method number 0..3>" % sys.argv[0]
raise SystemExit(1)
method = globals()["method_"+sys.argv[1]]
if os.fork():
while 1:
method("sausage")
check()
else:
while 1:
method("potato")
check()
------------------------------------------------------------
--
Nick Craig-Wood <ni**@craig-wood.com-- http://www.craig-wood.com/nick
Jun 27 '08 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Rex Karz | last post by:
Newbie here. I interpret the fine print at http://us2.php.net/manual/en/function.flock.php to mean that flock() does not work where the file being locked is on an NFS filesystem. The...
6
by: Pierre Rouleau | last post by:
Hi all! I am using Python 2.3.1 on Win32 (NT, 2000). Whenever a file imports the standard tempfile module, Python 2.3.1 issues the following warning: C:\Python23\lib\fcntl.py:7:...
0
by: Ryan Grow | last post by:
Hi, I'm trying to use fcntl to set an existing file descriptor to be nonblocking. This contrived example exhibits the behavior of python that is preventing me from doing this: import os,...
24
by: Matt Feinstein | last post by:
Hi all-- I'm new to Python, and was somewhat taken aback to discover that the core language lacks some basic numerical types (e.g., single-precision float, short integers). I realize that there...
2
by: Raghul | last post by:
Is it possible to monitor a folder in the python?My question is if I put any file in it that particular folder my script should monitor the folder and read the file name.If so what function can I...
3
by: thakadu | last post by:
The following code works as expected when run in the main body of a python script (ver 2.3.5) on OpenBSD v3.8. but when it is in the body of a function definition it does not work. It does not...
1
by: luca | last post by:
Hi, I was trying to make to work directly l i2c with python with the calls ioctl. But I have of the problems and I do not succeed to go ahead. this and l error 129# python pcf8591_ioctl.py...
4
by: Time Waster | last post by:
Is this a stupid use of flock: FILE *fp=fopen(SOME_FILE_CONSTANT,"r+"); flock(fileno(fp),LOCK_EX); something important here, including reads and a write to fp flock(fileno(fp),LOCK_UN);...
1
by: Seb | last post by:
I'm trying to implement a file server using the code below. However the locking doesn't work. I can delete while put'ing a file. Anyone got an idea about why? best regards, seb
1
isladogs
by: isladogs | last post by:
The next online meeting of the Access Europe User Group will be on Wednesday 6 Dec 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, Mike...
0
by: VivesProcSPL | last post by:
Obviously, one of the original purposes of SQL is to make data query processing easy. The language uses many English-like terms and syntax in an effort to make it easy to learn, particularly for...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
2
by: jimatqsi | last post by:
The boss wants the word "CONFIDENTIAL" overlaying certain reports. He wants it large, slanted across the page, on every page, very light gray, outlined letters, not block letters. I thought Word Art...
0
by: fareedcanada | last post by:
Hello I am trying to split number on their count. suppose i have 121314151617 (12cnt) then number should be split like 12,13,14,15,16,17 and if 11314151617 (11cnt) then should be split like...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.