By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,235 Members | 1,011 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,235 IT Pros & Developers. It's quick & easy.

Problem: Pickle and collections.defaultdict with default_factory set do not work

P: 2
Hello all!

I have the following weird problem and since I am new to Python I somehow cannot figure out an elegant solution. The problem reduces to the following question:

How to pickle a collections.defaultdict object that has set the default_factory property?

For Example (from the IDLE console):

Expand|Select|Wrap|Line Numbers
  1. >>> words = collections.defaultdict(lambda: 1)
  2. >>> f = file("temp","w")
  3. >>> pickle.dump(words,f)
  4.  
  5. Traceback (most recent call last):
  6.   File "<pyshell#77>", line 1, in <module>
  7.     pickle.dump(words,f)
  8.   File "C:\Python25\lib\pickle.py", line 1362, in dump
  9.     Pickler(file, protocol).dump(obj)
  10.   File "C:\Python25\lib\pickle.py", line 224, in dump
  11.     self.save(obj)
  12.   File "C:\Python25\lib\pickle.py", line 331, in save
  13.     self.save_reduce(obj=obj, *rv)
  14.   File "C:\Python25\lib\pickle.py", line 401, in save_reduce
  15.     save(args)
  16.   File "C:\Python25\lib\pickle.py", line 286, in save
  17.     f(self, obj) # Call unbound method with explicit self
  18.   File "C:\Python25\lib\pickle.py", line 562, in save_tuple
  19.     save(element)
  20.   File "C:\Python25\lib\pickle.py", line 286, in save
  21.     f(self, obj) # Call unbound method with explicit self
  22.   File "C:\Python25\lib\pickle.py", line 748, in save_global
  23.     (obj, module, name))
  24. PicklingError: Can't pickle <function <lambda> at 0x00C030B0>: it's not found as __main__.<lambda>
  25.  
  26. If I repeat the same with words = collections.defaultdict() everything goes fine..
  27.  
  28. In my real code I have a class that has an object property which is a defaultdict with the same default_factory set. I want to dump an object from this class with pickle and I get the following error: 
  29. Traceback (most recent call last):
  30.   File "BayesianFilter.py", line 125, in <module>
  31.     pickle.dump(bf,file(sys.argv[1]+'.bf','w'))
  32.   File "C:\Python25\lib\copy_reg.py", line 69, in _reduce_ex
  33.     raise TypeError, "can't pickle %s objects" % base.__name__
  34. TypeError: can't pickle function objects
If I do not set a default_factory in the constructor of the defaultdict object everything goes ok.. which makes me think that the problem is exactly this.

I know that Pickle cannot dump objects that have function properties. I did not find a way to unset the default_factory property from the defaultdict object before dumping though.. Neither I found a way to set it again after a defaultdict object was created once..

Does anyone have a clue how to resolve this issue elegantly?

Thanks for the help
Mar 21 '08 #1
Share this Question
Share on Google+
3 Replies


Subsciber123
P: 87
I think I understand what the issue is, but how to solve it I don't know. Why do you need to pickle the object? I have never found a use for that module myself, so I would be interested to know what it is being used for in your case. There is probably a more elegant way to save information from the program than using the pickle module.
Mar 21 '08 #2

P: 2
I think I understand what the issue is, but how to solve it I don't know. Why do you need to pickle the object? I have never found a use for that module myself, so I would be interested to know what it is being used for in your case. There is probably a more elegant way to save information from the program than using the pickle module.
Basically, I need some sort of persistence layer for my program. I.e. to store objects and load them from files afterwards. I have read about the pickle module and it seems a good solution to me. If anyone knows a better way to do this, I would like to know.
Mar 24 '08 #3

P: n/a
solution:

Expand|Select|Wrap|Line Numbers
  1. def give_me_1(): return 1
  2. words = collections.defaultdict(give_me_1)
  3.  
  4. # the rest was good
  5. f = file("temp","w")
  6. pickle.dump(words,f)
Sep 29 '10 #4

Post your reply

Sign in to post your reply or Sign up for a free account.