avoiding file corruption

[Amir Michail]

Hi,

Trying to open a file for writing that is already open for writing
should result in an exception.

It's all too easy to accidentally open a shelve for writing twice and
this can lead to hard to track down database corruption errors.

Amir

[Paolo Pantaleo]

27 Aug 2006 00:44:33 -0700, Amir Michail <am******@gmail .com>:

Hi,

Trying to open a file for writing that is already open for writing
should result in an exception.

It's all too easy to accidentally open a shelve for writing twice and
this can lead to hard to track down database corruption errors.

Amir

--
http://mail.python.org/mailman/listinfo/python-list

Even if it could be strange, the OS usually allow you to open a file
twice, that's up to the programmer to ensure the consistency of the
operations.

PAolo

--
if you have a minute to spend please visit my photogrphy site:
http://mypic.co.nr

[Amir Michail]

Paolo Pantaleo wrote:

27 Aug 2006 00:44:33 -0700, Amir Michail <am******@gmail .com>:

Hi,

Trying to open a file for writing that is already open for writing
should result in an exception.

It's all too easy to accidentally open a shelve for writing twice and
this can lead to hard to track down database corruption errors.

Amir

--
http://mail.python.org/mailman/listinfo/python-list

Even if it could be strange, the OS usually allow you to open a file
twice, that's up to the programmer to ensure the consistency of the
operations.

PAolo

But if this is usually a serious bug, shouldn't an exception be raised?

Amir

>

--
if you have a minute to spend please visit my photogrphy site:
http://mypic.co.nr

[Diez B. Roggisch]

Amir Michail schrieb:

Paolo Pantaleo wrote:

>27 Aug 2006 00:44:33 -0700, Amir Michail <am******@gmail .com>:

>>Hi,

Trying to open a file for writing that is already open for writing
should result in an exception.

It's all too easy to accidentally open a shelve for writing twice and
this can lead to hard to track down database corruption errors.

Amir

--
http://mail.python.org/mailman/listinfo/python-list

Even if it could be strange, the OS usually allow you to open a file
twice, that's up to the programmer to ensure the consistency of the
operations.

PAolo

But if this is usually a serious bug, shouldn't an exception be raised?

executing "rm -rf /" via subprocess is usually also a bad idea. So? No
language can prevent you from doing such mistake. And there is no way to
know if a file is opened twice - it might that you open the same file
twice via e.g. a network share. No way to know that it is the same file.

Diez

[Paddy]

Amir Michail wrote:

Hi,

Trying to open a file for writing that is already open for writing
should result in an exception.

It's all too easy to accidentally open a shelve for writing twice and
this can lead to hard to track down database corruption errors.

Amir

I've never done this in anger so feel free to mock (a little :-).

I'd have a fixed field at the beginning of the field that can hold the
hostname process number, and access time of a writing process, togeher
with a sentinal value that means "no process has access to the file".

A program would:
1. wait a random time.
2. open for update the file
3. read the locking data
4. If it is already being used by another process then goto 1.
5. write the process's locking data and time into the lock field.
6 Modify the files other fields.
7 write the sentinal value to the locking field.
8. Close and flush the file to disk.

I have left what to do if a process has locked the file for too long as
a simple exercise for you ;-).

- Paddy.

[Amir Michail]

Diez B. Roggisch wrote:

Amir Michail schrieb:

Paolo Pantaleo wrote:

27 Aug 2006 00:44:33 -0700, Amir Michail <am******@gmail .com>:
Hi,

Trying to open a file for writing that is already open for writing
should result in an exception.

It's all too easy to accidentally open a shelve for writing twice and
this can lead to hard to track down database corruption errors.

Amir

--
http://mail.python.org/mailman/listinfo/python-list

Even if it could be strange, the OS usually allow you to open a file
twice, that's up to the programmer to ensure the consistency of the
operations.

PAolo

But if this is usually a serious bug, shouldn't an exception be raised?

executing "rm -rf /" via subprocess is usually also a bad idea. So? No
language can prevent you from doing such mistake. And there is no way to
know if a file is opened twice - it might that you open the same file
twice via e.g. a network share. No way to know that it is the same file.

Diez

The scenario I have in mind is something like this:

def f():
db=shelve.open( 'test.db', 'c')
# do some stuff with db
g()
db.close()

def g():
db=shelve.open( 'test.db', 'c')
# do some stuff with db
db.close()

I think it would be easy for python to check for this problem in
scenarios like this.

Amir

[Diez B. Roggisch]

Amir Michail schrieb:

Diez B. Roggisch wrote:

>Amir Michail schrieb:

>>Paolo Pantaleo wrote:
27 Aug 2006 00:44:33 -0700, Amir Michail <am******@gmail .com>:
Hi,
>
Trying to open a file for writing that is already open for writing
should result in an exception.
>
It's all too easy to accidentally open a shelve for writing twice and
this can lead to hard to track down database corruption errors.
>
Amir
>
--
http://mail.python.org/mailman/listinfo/python-list
>
Even if it could be strange, the OS usually allow you to open a file
twice, that's up to the programmer to ensure the consistency of the
operations .

PAolo

But if this is usually a serious bug, shouldn't an exception be raised?

executing "rm -rf /" via subprocess is usually also a bad idea. So? No
language can prevent you from doing such mistake. And there is no way to
know if a file is opened twice - it might that you open the same file
twice via e.g. a network share. No way to know that it is the same file.

Diez

The scenario I have in mind is something like this:

def f():
db=shelve.open( 'test.db', 'c')
# do some stuff with db
g()
db.close()

def g():
db=shelve.open( 'test.db', 'c')
# do some stuff with db
db.close()

I think it would be easy for python to check for this problem in
scenarios like this.

You are requesting a general solution for a very particular problem. As
I pointed out, that solution is unlikely to work reliably - if not
infeasible at all.

If you really have problems as the above, use a custom wrapper for
shelve that prevents _you_ from making that mistake.

Diez

[Bryan Olson]

Amir Michail wrote:

Trying to open a file for writing that is already open for writing
should result in an exception.

It's all too easy to accidentally open a shelve for writing twice and
this can lead to hard to track down database corruption errors.

The right solution is file locking. Unfortunately, the Python
tandard distribution doesn't have a portable file lock, but you
can do it on Unix and Win NT or better. See:

http://mail.python.org/pipermail/pyt...ry/002957.html

and/or

http://aspn.activestate.com/ASPN/Coo...n/Recipe/65203.
--
--Bryan

[Grant Edwards]

On 2006-08-27, Amir Michail <am******@gmail .comwrote:

Trying to open a file for writing that is already open for writing
should result in an exception.

MS Windows seems to do something similar, and it pisses me off
no end. Trying to open a file and read it while somebody else
has it open for writing causes an exception. If I want to open
a file and read it while it's being writtent to, that's my
business.

Likewise, if I want to have a file open for writing twice,
that's my business as well. I certainly don't want to be
hobbled to prevent me from wandering off in the wrong direction.

It's all too easy to accidentally open a shelve for writing
twice and this can lead to hard to track down database
corruption errors.

It's all to easy to delete the wrong element from a list. It's
all to easy to re-bind the wrong object to a name. Should
lists be immutable and names be permanently bound?

--
Grant Edwards grante Yow! I'm in a twist
at contest!! I'm in a
visi.com bathtub! It's on Mars!! I'm
in tip-top condition!

[Amir Michail]

Grant Edwards wrote:

On 2006-08-27, Amir Michail <am******@gmail .comwrote:

Trying to open a file for writing that is already open for writing
should result in an exception.

MS Windows seems to do something similar, and it pisses me off
no end. Trying to open a file and read it while somebody else
has it open for writing causes an exception. If I want to open
a file and read it while it's being writtent to, that's my
business.

Likewise, if I want to have a file open for writing twice,
that's my business as well. I certainly don't want to be
hobbled to prevent me from wandering off in the wrong direction.

It's all too easy to accidentally open a shelve for writing
twice and this can lead to hard to track down database
corruption errors.

It's all to easy to delete the wrong element from a list. It's
all to easy to re-bind the wrong object to a name. Should
lists be immutable and names be permanently bound?

How often do you need to open a file multiple times for writing?

As a high-level language, Python should prevent people from corrupting
data as much as possible.

Amir

--
Grant Edwards grante Yow! I'm in a twist
at contest!! I'm in a
visi.com bathtub! It's on Mars!! I'm
in tip-top condition!