shelve keeping old (unused) data around?

[AK]

Hello,

I'm using shelve module in python 2.3 and I found that it does a very
odd thing, at least very unexpected to me. It seems that the data of a
replaced or deleted key stays in filename.dat file.

Example:

d = [1]*1000
from shelve import *
s = open('test')
s['test'] = d
s.close()
test.dat is 4100 bytes
d = 1
s = open('test')
s['test'] = d
s.close()

test.dat is still 4100 bytes

If I do del s['test'], it stays the same.

[Skip Montanaro]

nospam> I'm using shelve module in python 2.3 and I found that it does a
nospam> very odd thing, at least very unexpected to me. It seems that
nospam> the data of a replaced or deleted key stays in filename.dat
nospam> file.

The bits which actually get written to the disk depend on the underlying
library writing those bits. The shelve module has no say in the matter.
All it does to create a file is ask anydbm to open it, then pickle the
values so they can be written out as strings. Again, shelve doesn't do that
directly. It asks the object created by anydbm to do that. You didn't say
(perhaps you don't know) what the underlying library is on your system. It
seems likely that the library deletes the key, but leaves the data dangling
(perhaps also marked free so it can reuse it at a later time).

Skip

[AK]

In article <ma**********************************@python.org >, Skip
Montanaro wrote:

nospam> I'm using shelve module in python 2.3 and I found that it does a
nospam> very odd thing, at least very unexpected to me. It seems that
nospam> the data of a replaced or deleted key stays in filename.dat
nospam> file.

The bits which actually get written to the disk depend on the underlying
library writing those bits. The shelve module has no say in the matter.
All it does to create a file is ask anydbm to open it, then pickle the
values so they can be written out as strings. Again, shelve doesn't do that
directly. It asks the object created by anydbm to do that. You didn't say
(perhaps you don't know) what the underlying library is on your system. It
seems likely that the library deletes the key, but leaves the data dangling
(perhaps also marked free so it can reuse it at a later time).

Skip
Okay, as long as it's not a bug.. For the record, it seems like it's
bsddb on this side. Thanks, Skip!

-AK