472,127 Members | 1,950 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,127 software developers and data experts.

Memory error while saving dictionary using pickle

I am trying to save a dictionary of size 65000X50 to a local file and
I get the memory error problem.

How do I go about resolving this? Is there way to partition the pickle
object and combine later if this is a problem due to limited resources
(memory) on the machine (it is 32 bit machine Win XP, with 4GB RAM).

Please advice.

Thank you,
Nagu

Here is the detail description of the error:

Traceback (most recent call last):
File "<pyshell#12>", line 1, in <module>
s = pickle.dumps(itemsim)
File "C:\Python25\lib\pickle.py", line 1366, in dumps
Pickler(file, protocol).dump(obj)
File "C:\Python25\lib\pickle.py", line 224, in dump
self.save(obj)
File "C:\Python25\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python25\lib\pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python25\lib\pickle.py", line 663, in _batch_setitems
save(v)
File "C:\Python25\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python25\lib\pickle.py", line 600, in save_list
self._batch_appends(iter(obj))
File "C:\Python25\lib\pickle.py", line 615, in _batch_appends
save(x)
File "C:\Python25\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python25\lib\pickle.py", line 562, in save_tuple
save(element)
File "C:\Python25\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python25\lib\pickle.py", line 477, in save_float
self.write(FLOAT + repr(obj) + '\n')
MemoryError: out of memory
Jul 7 '08 #1
0 1468

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

4 posts views Thread by Darren Dale | last post: by
6 posts views Thread by Byron | last post: by
10 posts views Thread by Luis P. Mendes | last post: by
4 posts views Thread by Jacob H | last post: by
reply views Thread by Enrique Palomo Jiménez | last post: by
3 posts views Thread by Catherine Moroney | last post: by
reply views Thread by leo001 | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.