Pickling and inheritance are making me hurt

[Kirk Strauser]

I have a module that defines a Search class and a SearchResult class. I use
these classes by writing other modules that subclass both of them as needed
to interface with particular search engines.

My problem is that Search defines a method (called automatically by __del__)
to save its results between invocations:

def _saveresults(self):
self._oldresults = self._results
file = open(self._storefile(), 'w')
pickle.dump(self._oldresults, file)
file.close()

The problem I'm having is the the pickle.dump call is failing whenever the
objects in "self.data" are instances of derivatives of SearchResult rather
than instances of SearchResult itself (which is pretty much always the
case):

Exception pickle.PicklingError: <pickle.PicklingError instance at
0xb7f7ad6c> in <bound method Search.__del__ of <__main__.Search object at
0xb7ec954c>> ignored
Now, if I overload _saveresults inside a subclass of Search, then it works.
It seems like the problem is that _saveresults is only looking inside the
same namespace as Search (where it's originally defined), even if it's
actually been inherited by another class in a different module. Is there a
way around this?

--
Kirk Strauser

[John J. Lee]

Kirk Strauser <ki**@strauser.com> writes:

I have a module that defines a Search class and a SearchResult class. I use
these classes by writing other modules that subclass both of them as needed
to interface with particular search engines.

My problem is that Search defines a method (called automatically by __del__) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Don't Do That. __del__ methods are bad. They mess up cyclic garbage
collection, IIRC. ISTR other problems with them, too...
[...] Exception pickle.PicklingError: <pickle.PicklingError instance at
0xb7f7ad6c> in <bound method Search.__del__ of <__main__.Search object at
0xb7ec954c>> ignored

[...]

....yeah, there's one: thanks for reminding me ;-)

You may want to read up on __getstate__ and __setstate__, BTW.
John

[Tim Peters]

[Kirk Strauser]

I have a module that defines a Search class and a SearchResult class.
Try posting a minimal self-contained code sample that fails.
I use these classes by writing other modules that subclass both of them as
needed to interface with particular search engines.

My problem is that Search defines a method (called automatically by __del__)
to save its results between invocations:

def _saveresults(self):
self._oldresults = self._results
file = open(self._storefile(), 'w')
pickle.dump(self._oldresults, file)
file.close()

The problem I'm having is the the pickle.dump call is failing whenever the
objects in "self.data" are instances of derivatives of SearchResult rather
than instances of SearchResult itself (which is pretty much always the
case):

Exception pickle.PicklingError: <pickle.PicklingError instance at
0xb7f7ad6c> in <bound method Search.__del__ of <__main__.Search object at
0xb7ec954c>> ignored

Now, if I overload _saveresults inside a subclass of Search, then it works.
It seems like the problem is that _saveresults is only looking inside the
same namespace as Search (where it's originally defined), even if it's
actually been inherited by another class in a different module. Is there a
way around this?

Not enough relevant information to say. __del__ looks irrelevant
here, apart from that you happen to be pickling while __del__ is
executing. Try pickling an instance of a subclass of Search directly
to see what happens. There are many reasons for why PicklingError may
get raised.

It's certainly true that any function in Python (method or not)
executes with the globals of the module in which it's defined. While
you mentioned that as a possible issue, it seemed to come out of the
blue (didn't seem to follow from anything said before it).