473,379 Members | 1,123 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,379 software developers and data experts.

Pickling and unpickling inherited attributes

I have a serious problem and I hope there is some solution. It is
easier to illustrate with a simple code:
class Parent(object): __slots__=['A', 'B']
def __init__(self, a, b):
self.A=a; self.B=b
def __getstate__(self):
return self.A, self.B
def __setstate__(self, tup):
self.A, self.B=tup

class Child(Parent): __slots__=['C',]
def __init__(self, c):
self.C=c
def __getstate__(self):
return self.C,
def __setstate__(self, tup):
self.C, =tup

obj=Child(1)
obj.A=2
obj.B=3
obj.A 2 obj.B 3 obj.C 1 objct.Z=4
Traceback (most recent call last):
File "<pyshell#60>", line 1, in -toplevel-
objct.Z=4
AttributeError: 'Child' object has no attribute 'Z'

So far so good.. Object obj inherited attributes (A and B) from the
parent class and refuses to take any new ones. But look what happens
when I try to pickle and unpickle it:
import cPickle
File=open('test', 'w')
cPickle.dump(obj, File)
File.close()

file1=open('test', 'r')
objct=cPickle.load(file1)
file1.close()
objct.A
Traceback (most recent call last):
File "<pyshell#55>", line 1, in -toplevel-
objct.A
AttributeError: A objct.C

1

Its own attribute (C) value is restored but the value of an inherited
attribute (A) is not. I tried pickling protocol 2, and module pickle
instead of cPickle, all with the same result.

What can be done?! Or maybe nothing can be done?

I would greatly appreciate an advice. A lot of code is written, but now
we have a need for pickling and unpickling objects and discovered this
problem.

Oct 30 '05 #1
9 3558
Sorry, I copied and pasted a wrong piece from shell at one part of the
code:

objct.Z=4 was in fact obj.Z=4 and it did refuse to accept Z (because it
is not in __slots__).

But the question remains: why the value of attribute A is not preserved
during pickling and unpickling and what can be done about it, if
anything?

Oct 30 '05 #2
The reason the state of obj.A and obj.B aren't preserved is because
your __getstate__ and __setstate__ don't preserve them - they only save
obj.C, and you don't make a call to the parent class's respective
methods. Here's what I mean:
import pickle
class Child(Parent):
__slots__=['C',]
def __init__(self, c):
self.C=c
def __getstate__(self):
return Parent.__getstate__(self) + (self.C)
def __setstate__(self, tup):
self.C = tup.pop()
Parent.__setstate__(self, tup)
obj = Child('foo')
obj.A = 'bar'
obj.B = 'baz'
objct = pickle.loads(pickle.dumps(obj))
objct.A 'bar' objct.B 'baz' objct.C

'foo'

Oct 31 '05 #3
Of course, in __setstate__, there should be a tup = list(tup) as well -
sheer forgetfulness and a confusing name in one.

Oct 31 '05 #4
Thanks so much! It makes perfect sense and I am sure it will work now.

Oct 31 '05 #5
Alex <Ou****@gmail.com> wrote:
I have a serious problem and I hope there is some solution. It is
easier to illustrate with a simple code:
class Parent(object): __slots__=['A', 'B']
def __init__(self, a, b):
self.A=a; self.B=b
def __getstate__(self):
return self.A, self.B
def __setstate__(self, tup):
self.A, self.B=tup

class Child(Parent):

__slots__=['C',]
def __init__(self, c):
self.C=c
def __getstate__(self):
return self.C,
def __setstate__(self, tup):
self.C, =tup


Child.__getstate__ and __setstate__ need to cooperate with those of
Parent, not just override them as yours do. __getstate__ is easy:

in Child:

def __getstate__(self):
return super(Child, self).__getstate__ + (self.C,)

i.e., each class will successively append its parts of state to the
resulting overall tuple. It's harder to do for __setstate__ without
building into it a strong dependency on how many items of the tuple the
Parent is taking for its own use; you need to establish some protocol of
your own for that purpose, such as:

in Parent:

def __setstate__(self, tup):
self.A, self.B = tup[:2]
self._tup = tup[2:]

in Child:

def __setstate__(self, tup):
super(Child, self).__setstate__(tup)
self.C, = self._tup[:1]
self._tup = self._tup[1:]

(the last statement is needed in case Child was further subclassed by a
Grandchild class following the same set-state protocol).

Working with lists rather than tuples might in fact be simpler. But
simplest might be if __setstate__ was allowed to return something
different than None, so it could return "whatever's left of the tuple"
rather than relying on that nasty self._tup thingy (which is going to be
left there, empty, in the end -- though, each class could conditionally
delete it if it was empty). However, while I believe it is not
currently enforced that __setstate__ might return a non-None value to be
ignored, I'm not sure it's _allowed_, either, so I wouldn't try (lest
this application should break in strange ways in future Python
versions).
Alex

Oct 31 '05 #6
Alex wrote:
I have a serious problem and I hope there is some solution. It is
easier to illustrate with a simple code:

class Parent(object):
__slots__=['A', 'B']
def __init__(self, a, b):
self.A=a; self.B=b
def __getstate__(self):
return self.A, self.B
def __setstate__(self, tup):
self.A, self.B=tup
class Child(Parent):
__slots__=['C',]
def __init__(self, c):
self.C=c
def __getstate__(self):
return self.C,
def __setstate__(self, tup):
self.C, =tup
obj=Child(1)
obj.A=2
obj.B=3
obj.A
2
obj.B
3
obj.C
1
objct.Z=4

Traceback (most recent call last):
File "<pyshell#60>", line 1, in -toplevel-
objct.Z=4
AttributeError: 'Child' object has no attribute 'Z'

So far so good.. Object obj inherited attributes (A and B) from the
parent class and refuses to take any new ones. But look what happens
when I try to pickle and unpickle it:

import cPickle
File=open('test', 'w')
cPickle.dump(obj, File)
File.close()

file1=open('test', 'r')
objct=cPickle.load(file1)
file1.close()
objct.A

Traceback (most recent call last):
File "<pyshell#55>", line 1, in -toplevel-
objct.A
AttributeError: A
objct.C


1

Its own attribute (C) value is restored but the value of an inherited
attribute (A) is not. I tried pickling protocol 2, and module pickle
instead of cPickle, all with the same result.

What can be done?! Or maybe nothing can be done?

I would greatly appreciate an advice. A lot of code is written, but now
we have a need for pickling and unpickling objects and discovered this
problem.

You have explicitly told the objects' class definition to only store the
C attribute as a part of the object state.

If you change the definition of child to:

class Child(Parent):

__slots__=['C',]
def __init__(self, c):
self.C=c
def __getstate__(self):
return self.A, self.B, self.C,
def __setstate__(self, tup):
self.A, self.B, self.C, =tup

everything works as you expect. But I presume you want the subclass to
require no knowledge of the superclass state?

In that case you might consider rewriting Child as

class Child(Parent):

__slots__ = ['C'] # only tuples need trailing comma
def __init__(self, c):
self.C = c
def __getstate__(self):
return Parent.__getstate__(self) + (self.C, )
def __setstate__(self, t):
self.C = t[-1]
Parent.__setstate__(self, t[:-1])

This would work adequately. May I ask your use case for __slots__? I
presume there is a specific reason why you can't just code the classes as

class Parent(object):
def __init__(self, a, b):
self.A = a
self.B = b

class Child(Parent):
def __init__(self, c):
self.C = c

since these definitions will pickle and unpickle perfectly.

A couple of other notes:

First, it's always safest to open pickle files in binary mode, as this
will eliminate the chance of platform differences making a pickle
written on one architecture or platform being unreadable by another.

Secondly, it's usual to have the subclass explicitly call the superclass
to initialize that portion of the instance. So it would be more
typically Pythonic to write

class Child(Parent):
def __init__(self, c):
Parent.__init__(self, a, b)
self.C = c

This allows you to pass the a and b values into the creator call in the
following way:

obj = Child(2, 3, 1)

rather than explicitly setting attributes of the Child instance in line.
If your inheritance patterns are complex you may find it useful to
investigate the super() function.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006 www.python.org/pycon/

Oct 31 '05 #7
Thanks to both Alex Martelli and Steve Holden.. We need __slots__ for
other reasons, too long to explain, but basically to prevent assignment
of extra attributes.

I ended up changing child classes this way:

def __getstate__(self):
return(Parent.__getstate__(self), self.C)
def __setstate__(self, data):
Parent.__setstate__(self, data[0])
self.C=data[1:]

This seems to work fine.

Nov 1 '05 #8
Alex wrote:
Thanks to both Alex Martelli and Steve Holden.. We need __slots__ for
other reasons, too long to explain, but basically to prevent assignment
of extra attributes.

I ended up changing child classes this way:

def __getstate__(self):
return(Parent.__getstate__(self), self.C)
def __setstate__(self, data):
Parent.__setstate__(self, data[0])
self.C=data[1:]

This seems to work fine.

OK, but do be aware that slots was intended solely as a
memory-conservation measure where large numbers of objects are being
created. You are therefore subverting its true intent by using it to
limit the assignment of extra attributes (and there has been much recent
discussion on this list, though not in this thread, about what to do
instead.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006 www.python.org/pycon/

Nov 1 '05 #9

OK, but do be aware that slots was intended solely as a
memory-conservation measure where large numbers of objects are being
created. You are therefore subverting its true intent by using it to
limit the assignment of extra attributes (and there has been much recent
discussion on this list, though not in this thread, about what to do
instead.

regards
Steve
--

Oh, that too... We have tens of thousands of these objects in the
memory at the same moment, so memory conservation is another reason for
slots.

Nov 2 '05 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Bob | last post by:
I've read over section 3.14.5.2 of the doc about a zillion times, and it still makes absolutely no sense to me. Can someone please explain it? What I'd like to do is, basically, have the object...
1
by: Thomas Guettler | last post by:
Hi! If I pickle a class which inherits from dict, I don't get the same data again, if I unpickle it. def test_pickle(): home="..." server=WorkflowServer(home) server.save()
1
by: Thomas Guettler | last post by:
Hi! After unpickling the objects are not the same any more. Is this a bug or feature? import pickle class MyDictContainer(dict): def __init__(self): dict.__init__(self)
1
by: Marc | last post by:
Hi all, After some research I've decided that my previous question (Confusing problem between Tkinter.Intvar...) was headed in the wrong direction. Partly because I think I have a greater...
2
by: max(01)* | last post by:
i understand than most objects are pickable, but i cannot get a function pickled, or a class for that matter: $ python Python 2.3.3 (#2, Jan 13 2004, 00:47:05) on linux2 Type "help",...
1
by: fedor | last post by:
Hi all, happy new year, I was trying to pickle a instance of a subclass of a tuple when I ran into a problem. Pickling doesn't work with HIGHEST_PROTOCOL. How should I rewrite my class so I can...
1
by: Erik Max Francis | last post by:
I've come across a limitation in unpickling certain types of complex data structures which involve instances that override __hash__, and was wondering if it was known (basic searches didn't seem to...
0
by: Shahin Saadati | last post by:
Hi, The following sample code is to pickle and unpickle an object. It works fine with CPython, but the unpickling fails in Jython and I receive an error stating that "A" is unsafe to unpickle...
3
by: sushant.sirsikar | last post by:
Hi, I am new in Python World.I want to know what is mean by ``pickling'' and ``unpickling'' ? And how can we used it?Please Give Me some links of Picking Examples. Thanks Sushant
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.