473,320 Members | 1,950 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Troubleshooting pickle errors

Hi

I am grafing pickling functionality on a big dictionary of object
instances and I am having numerous problems. I think understand the
notion of what can/can't be pickled and I know how to use __getstate__
/ __setstate__ to weed out offending references. But I am finding it
very hard to identify _which_ reference is causing exceptions like:

File "C:\UserApps\Python23\Lib\copy_reg.py", line 69, in _reduce_ex
raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle function objects

PrettyPrinting the exception gets me:

{'args': ("can't pickle function objects",)}.

The exception includes no references to the guilty party! How do I
know _which_ object caused the issue? Though my current objects (i.e.
the stuff I am trying to pickle) are new, its instances hold
references to a lot of instances of existing utility classes of mine
that are database aware. I would not like to modify those utility
classes any more than I _have_ to since I use them elsewhere. I have
already stripped out database Connection references, as well as File
references. Those were obvious.

How do I identify the remaining issues? If I work with a smaller data
set, I often miss problems entirely, so reducing the pickling scope is
not an easy option to implement. The best I have managed so far is to
open the pickle file and look near the end, where the exception
presumably occurred. But I would expect there to be a more obvious
way to troubleshoot this stuff.

Cheers

JLuc
Jul 18 '05 #1
0 1245

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Guenter Walser | last post by:
Hello, When using the codeline: pickle.dump(self, file, pickle.HIGHEST_PROTOCOL) my program runs perfectly fine: --------------------------- Testprotocol -----------------------------
2
by: GMane Python | last post by:
Hey all. I have a ( list | tuple | dictionary ) with several graphics in it. Trying to cPickle it to a file, I get errors. For instance, UnpickleableError: Cannon pickle <type 'ImagingCore'>...
4
by: temposs | last post by:
I'm trying to pickle a class, and while I get no errors or anything, almost none of the class instance gets pickled, and I don't know why...Here's the pickled output: (i__main__ TrainingMatrix...
0
by: Mike P. | last post by:
Hi all, I'm working on a simulation (can be considered a game) in Python where I want to be able to dump the simulation state to a file and be able to load it up later. I have used the standard...
0
by: greg.landrum | last post by:
Hi, I have a fairly sizable body of python code that I'm in the process of modernizing. One of the modernization steps is converting to use new-style classes, and this is leading to problems...
16
by: John Salerno | last post by:
Here's what I have: import pickle data = open(r'C:\pickle_data.txt', 'w') image = open(r'C:\peakhell.jpg') pickle.Pickler(data) data.dump(image) data.close() image.close()
10
by: crystalattice | last post by:
I'm creating an RPG for experience and practice. I've finished a character creation module and I'm trying to figure out how to get the file I/O to work. I've read through the python newsgroup...
9
by: tonyr1988 | last post by:
I'm a complete python n00b writing my first program (or attempting to, anyway). I'm trying to make the transition from Java, so if you could help me, it would be greatly appreciated. Here's the...
1
by: IceMan85 | last post by:
Hi to all, I have spent the whole morning trying, with no success to pickle an object that I have created. The error that I get is : Can't pickle 'SRE_Match' object: <_sre.SRE_Match object at...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.