By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
445,870 Members | 1,212 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 445,870 IT Pros & Developers. It's quick & easy.

pickling subclasses of dict/list

P: n/a
I'm having trouble pickling subclasses of dict when they contain cycles.
In particular:
import pickle
class D(dict): pass
d = D()
d[1] = d # add a cycle.
print d {1: {...}} pickle.dump(d, open('d.pickle', 'w')) Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/tmp/python-rSB6hN", line 6, in ?
pickle.dump(d, open('d.pickle', 'w'))
File "/usr/lib/python2.3/pickle.py", line 1382, in dump
Pickler(file, protocol, bin).dump(obj)
[...]
File "/usr/lib/python2.3/pickle.py", line 414, in save_reduce
save(func)
RuntimeError: maximum recursion depth exceeded

Note that pickling dict's that contain cycles works just fine, as does
pickling classes that contain cycles in their instance variables. E.g.:
d = {}
d[1] = d
print d
pickle.dump(d, open('d.pickle', 'w'))
print pickle.load(open('d.pickle'))

{1: {...}}

I tried several things with __getstate__, __reduce__, etc., but couldn't
get this to work. Is there some magic that I'm missing? What's the
best way to get around this? (And should I file this as a bug in
pickle? Or am I just not seeing the right way to do it?)

-Edward

Jul 18 '05 #1
Share this Question
Share on Google+
1 Reply


P: n/a
On Sat, 03 Jul 2004 14:37:28 -0400, Edward Loper
<ed*****@gradient.cis.upenn.edu> wrote:
I'm having trouble pickling subclasses of dict when they contain cycles.

Note that pickling dict's that contain cycles works just fine, as does
pickling classes that contain cycles in their instance variables. E.g.:
>>> d = {}
>>> d[1] = d
>>> print d
>>> pickle.dump(d, open('d.pickle', 'w'))
>>> print pickle.load(open('d.pickle')) {1: {...}}

I tried several things with __getstate__, __reduce__, etc., but couldn't
get this to work. Is there some magic that I'm missing? What's the
best way to get around this? (And should I file this as a bug in
pickle? Or am I just not seeing the right way to do it?)


I think the answer to this problem is to use the extended __reduce__ API
documented here:

http://www.python.org/peps/pep-0307.html

Your subclass's __reduce__ can return a dictitems (or listitems) iterator, the
items of which will get pickled after the subclass instance is memoized, thus
avoiding the infinite recursion. The following works for your simple example:
class D(dict): .... def __reduce__(self):
.... return (D, (), None, None, self.iteritems())
.... d = D()
d[1] = d
pickle.dumps(d) 'c__main__\nD\np0\n(tRp1\nI1\ng1\ns.' s = _
dd = pickle.loads(s)
dd.items()

[(1, {1: {...}})]

---
Greg Chapman

Jul 18 '05 #2

This discussion thread is closed

Replies have been disabled for this discussion.