395,972 Members | 4,639 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 395,972 IT Pros & Developers. It's quick & easy.

pickle an instance of a custom class

PBlitz
P: 3
I want to save the content of a custom made OrderedDict. (I got the OrderedDict from http://aspn.activestate.com/ASPN/Python)
The pickle modul seemed to be the solution for the job.
At first I tried to pickle the "nacked" OrderedDict derived from dict, but the pickle routine saves only a empty dict. A secont try to "replace" the __dict__ with the __getstate__ and __setstate__ methods also faild.

How can I save the content (the items of the derived dict and the _keys list) of the OrderedDict via the pickle modul?

here is the code:

Expand|Select|Wrap|Line Numbers
  1. class OrderedDict(dict):
  2.     """This dictionary class extends UserDict to record the order in which items are 
  3.     added. Calling keys(), values(), items(), etc. will return results in this order."""
  4.  
  5.     def __init__(self, d = None):
  6.         self._keys = []
  7.         if d == None: d = {}
  8.         self.update(d)
  9.  
  10.     def __setitem__(self, key, item):
  11.         dict.__setitem__(self,key, item)
  12.         if key not in self._keys: self._keys.append(key)
  13.  
  14.     def items(self):
  15.         return zip(self._keys, self.values())
  16.  
  17.     def keys(self):
  18.         return self._keys[:]
  19.  
  20.     def update(self, d):
  21.         dict.update(self, d)
  22.         for key in d.keys():
  23.             if key not in self._keys: self._keys.append(key)
  24.  
  25.     def __getstate__(self):
  26.         """Return state values to be pickled."""
  27.         return (dict(self.items()), self._keys)
  28.  
  29.     def __setstate__(self, state):
  30.         """Restore state from the unpickled state values."""
  31.         self.update(state[0])
  32.         self._keys = state[1]
  33.  
  34. if __name__ == "__main__":
  35.     import pickle
  36.     o = OrderedDict()
  37.     o['a'] = [0,1]
  38.     o['b'] = [2,3]
  39.     p = pickle.dumps(o)
  40.     u = pickle.loads(p)
  41.     print o # output should be {'b': [2, 3], 'a': [0, 1]}
  42.     print u # output should be {'b': [2, 3], 'a': [0, 1]}
  43.  
  44. # Output:
  45. #{'b': [2, 3], 'a': [0, 1]}
  46. #{}                                      
  47.  
Mar 10 '08 #1
Share this Question
Share on Google+
4 Replies


bvdet
Expert Mod 2.5K+
P: 2,765
I want to save the content of a custom made OrderedDict. (I got the OrderedDict from http://aspn.activestate.com/ASPN/Python)
The pickle modul seemed to be the solution for the job.
At first I tried to pickle the "nacked" OrderedDict derived from dict, but the pickle routine saves only a empty dict. A secont try to "replace" the __dict__ with the __getstate__ and __setstate__ methods also faild.

How can I save the content (the items of the derived dict and the _keys list) of the OrderedDict via the pickle modul?
The dictionary elements will display in the correct order if you implement a __repr__() method. You will also need to override the dict.values() method so the values will be paired with the correct keys. Object pickling and unpickling should work without __getstate__() and __setstate__() methods.
Expand|Select|Wrap|Line Numbers
  1. class OrderedDict(dict):
  2.  
  3.     def __init__(self, d = None):
  4.         self._keys = []
  5.         if d == None: d = {}
  6.         self.update(d)
  7.  
  8.     def __setitem__(self, key, item):
  9.         dict.__setitem__(self,key, item)
  10.         if key not in self._keys: self._keys.append(key)
  11.  
  12.     def items(self):
  13.         return zip(self._keys, self.values())
  14.  
  15.     def keys(self):
  16.         return self._keys[:]
  17.  
  18.     def itervalues(self):
  19.         for k in self._keys:
  20.             yield self[k]
  21.  
  22.     def values(self):
  23.         return list(self.itervalues())    
  24.  
  25.     def update(self, d):
  26.         dict.update(self, d)
  27.         for key in d.keys():
  28.             if key not in self._keys: self._keys.append(key)
  29.  
  30.     def __repr__(self):
  31.         return '{%s}' % ', '.join([': '.join([repr(k),str(v)]) \
  32.                                    for k,v in zip(self.keys(), self.values())])
  33.  
  34. if __name__ == "__main__":
  35.     dd = OrderedDict()
  36.     dd['z'] = [0,1]
  37.     dd['b'] = [2,3]
  38.     dd['x'] = [21,22]
  39.     dd['y'] = [45,46]
  40.     dd['a'] = [9,10]
  41.     p = pickle.dumps(dd)
  42.     u = pickle.loads(p)
  43.     print dd
  44.     print u
Output:

>>> {'z': [0, 1], 'b': [2, 3], 'x': [21, 22], 'y': [45, 46], 'a': [9, 10]}
{'z': [0, 1], 'b': [2, 3], 'x': [21, 22], 'y': [45, 46], 'a': [9, 10]}
>>>
Mar 11 '08 #2

PBlitz
P: 3
Thank you for the help. Unfortunately my output with your code is:
{'z': [0, 1], 'b': [2, 3], 'x': [21, 22], 'y': [45, 46], 'a': [9, 10]}
{}

the pickle p contains:
c__main__
OrderedDict
p0
(tp1
Rp2
.

This means the pickle dumps no dict values nor _keys list values.
Maybe the reason is my configuration. I use jython 2.2.1 on a windows machine.
Is there a implementation difference between python and jython?
Mar 11 '08 #3

bvdet
Expert Mod 2.5K+
P: 2,765
Thank you for the help. Unfortunately my output with your code is:
{'z': [0, 1], 'b': [2, 3], 'x': [21, 22], 'y': [45, 46], 'a': [9, 10]}
{}

the pickle p contains:
c__main__
OrderedDict
p0
(tp1
Rp2
.

This means the pickle dumps no dict values nor _keys list values.
Maybe the reason is my configuration. I use jython 2.2.1 on a windows machine.
Is there a implementation difference between python and jython?
I am using Python 2.3 on Windows XP. I don't know the differences. I have pickled class instances successfully many times. Your pickle module may be broken.
Mar 11 '08 #4

PBlitz
P: 3
Here a more precise description of the issue:
Pickling of a class derived from dict creates an empty pickle in jython
2.2.1. In contrast Python 2.5.2 delivers the expected result.
So I assume a bug in the jython pickle module. Does anyone know if a developer release of jython already solves the problem? Or how to patch the module?

# code describing the issue:

Expand|Select|Wrap|Line Numbers
  1. class MyDict(dict):
  2.     def __init__(self, d = None):
  3.         if d == None: d = {}
  4.         dict.__init__(self, d)
  5.  
  6. if __name__ == "__main__":
  7.     import pickle
  8.     md = MyDict({'b': [2, 3], 'z': [0, 1]})
  9.     p = pickle.dumps(md)
  10.     rd = pickle.loads(p)
  11.     print "-> pickle p:\n", p
  12.     print "-> MyDict md:\n", md
  13.     print "-> reconstructed MyDict rd:\n", rd
  14.  
# jython 2.2.1 output with empty pickle p:

-> pickle p:
c__main__
MyDict
p0
(tp1
Rp2
.
-> MyDict md:
{'b': [2, 3], 'z': [0, 1]}
-> reconstructed MyDict rd:
{}

# in contrast python 2.5.2 output with filled pickle p:

-> pickle p:
ccopy_reg
_reconstructor
p0
(c__main__
MyDict
p1
c__builtin__
dict
p2
(dp3
S'b'
p4
(lp5
I2
aI3
asS'z'
p6
(lp7
I0
aI1
astp8
Rp9
.
-> MyDict md:
{'b': [2, 3], 'z': [0, 1]}
-> reconstructed MyDict rd:
{'b': [2, 3], 'z': [0, 1]}
Mar 12 '08 #5

Post your reply

Sign in to post your reply or Sign up for a free account.