By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
444,001 Members | 1,176 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 444,001 IT Pros & Developers. It's quick & easy.

bug in copy.deepcopy or in getattr or in my understanding?

P: n/a
Dear experts,

I got some unexpected behavior in getattr and copy.deepcopy (see
transcript below). I'm not sure if this is actually a bug in
copy.deepcopy or if I'm doing something too magical with getattr.
Comments would be appreciated.

Thanks,
-Emin

######### Transcript follows ##################
Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>># THE FOLLOWING BREAKS
class a:
.... def foo(self):
.... print 'hi'
....
>>class b(a):
.... def __init__(self):
.... self.y = getattr(self,'foo')
....
>>c = b()
import copy
copy.deepcopy(c)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "c:\Python25\lib\copy.py", line 162, in deepcopy
y = copier(x, memo)
File "c:\Python25\lib\copy.py", line 291, in _deepcopy_inst
state = deepcopy(state, memo)
File "c:\Python25\lib\copy.py", line 162, in deepcopy
y = copier(x, memo)
File "c:\Python25\lib\copy.py", line 254, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "c:\Python25\lib\copy.py", line 189, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "c:\Python25\lib\copy.py", line 322, in _reconstruct
y = callable(*args)
File "c:\Python25\lib\copy_reg.py", line 92, in __newobj__
return cls.__new__(cls, *args)
TypeError: instancemethod expected at least 2 arguments, got 0
>># THE FOLLOWING WORKS
class b(a):
.... def __init__(self):
.... self.x = self.__class__.__bases__[0].__dict__['foo']
....
>>c=b()
copy.deepcopy(c)
<__main__.b instance at 0x00EADE18>

Jan 4 '07 #1
Share this Question
Share on Google+
4 Replies


P: n/a
At Thursday 4/1/2007 17:26, Emin wrote:
>I got some unexpected behavior in getattr and copy.deepcopy (see
transcript below). I'm not sure if this is actually a bug in
copy.deepcopy or if I'm doing something too magical with getattr.
Comments would be appreciated.
Both examples are different. #1 stores a *bound* method into an
instance attribute. Bound methods carry a reference to "self", this
creates a cyclic reference that may cause problems to the garbage
collector (and confuses deepcopy, apparently).
#2 uses and *unbound* method and it's the usual way.
>class a:
... def foo(self):
... print 'hi'
...
>class b(a): #1
... def __init__(self):
... self.y = getattr(self,'foo')
>class b(a): #2
... def __init__(self):
... self.x = self.__class__.__bases__[0].__dict__['foo']
For #2 you can simply say:

class b(a):
x = a.foo

If you have to choose at runtime (a simplified version of your own code):

class b(a):
def __init__(self):
name = select_method_to_use(..., default="foo")
self.x = getattr(a, name)

You *know* your bases because you wrote them in the class statement
(or use super() instead of playing with __bases__); and getattr works
fine here so you don't need to mess with the __dict__ details.

(Note that #1 was *almost* right, you had to replace self by the base class)
--
Gabriel Genellina
Softlab SRL


__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas

Jan 5 '07 #2

P: n/a
Dear Gabriel,

Thank you for your reply. As you guessed, I want to be able to select
the method at runtime as in your final example, but when I tried your
suggestion I got the same error (see below). I think the problem is
that getattr is donig something different than in my example where I
explicitly get it from the dict (see the very end of the transcript
below):
--------------------------- Transcript Follows
----------------------------------------------------------

Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>># The following shows that using getattr to grab a method is
# incompatible with copy.deepcopy
class a:
.... def foo(self): pass
....
>>class b(a):
.... def __init__(self):
.... self.x = getattr(a,'foo')
....
>>import copy
c=b()
copy.deepcopy(c)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "c:\Python25\lib\copy.py", line 162, in deepcopy
y = copier(x, memo)
File "c:\Python25\lib\copy.py", line 291, in _deepcopy_inst
state = deepcopy(state, memo)
File "c:\Python25\lib\copy.py", line 162, in deepcopy
y = copier(x, memo)
File "c:\Python25\lib\copy.py", line 254, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "c:\Python25\lib\copy.py", line 189, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "c:\Python25\lib\copy.py", line 322, in _reconstruct
y = callable(*args)
File "c:\Python25\lib\copy_reg.py", line 92, in __newobj__
return cls.__new__(cls, *args)
TypeError: instancemethod expected at least 2 arguments, got 0
>>>
# The following shows that getattr is doing something different
# than looking in the __dict__ of base classes
b.__bases__[0].__dict__['foo']
<function foo at 0x009F0CF0>
>>getattr(a,'foo')
<unbound method a.foo>
>>>

On Jan 4, 10:08 pm, Gabriel Genellina <gagsl...@yahoo.com.arwrote:
At Thursday 4/1/2007 17:26, Emin wrote:
I got some unexpected behavior in getattr and copy.deepcopy (see
transcript below). I'm not sure if this is actually a bug in
copy.deepcopy or if I'm doing something too magical with getattr.
Comments would be appreciated.Both examples are different. #1 stores a *bound* method into an
instance attribute. Bound methods carry a reference to "self", this
creates a cyclic reference that may cause problems to the garbage
collector (and confuses deepcopy, apparently).
#2 uses and *unbound* method and it's the usual way.
>>class a:
... def foo(self):
... print 'hi'
...
>>class b(a): #1
... def __init__(self):
... self.y = getattr(self,'foo')
>>class b(a): #2
... def __init__(self):
... self.x = self.__class__.__bases__[0].__dict__['foo']For #2 you can simply say:

class b(a):
x = a.foo

If you have to choose at runtime (a simplified version of your own code):

class b(a):
def __init__(self):
name = select_method_to_use(..., default="foo")
self.x = getattr(a, name)

You *know* your bases because you wrote them in the class statement
(or use super() instead of playing with __bases__); and getattr works
fine here so you don't need to mess with the __dict__ details.

(Note that #1 was *almost* right, you had to replace self by the base class)

--
Gabriel Genellina
Softlab SRL

__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!http://www.yahoo.com.ar/respuestas
Jan 5 '07 #3

P: n/a
Emin wrote:
Thank you for your reply. As you guessed, I want to be able to select
the method at runtime as in your final example, but when I tried your
suggestion I got the same error (see below). I think the problem is
that getattr is donig something different than in my example where I
explicitly get it from the dict (see the very end of the transcript
below):
This has nothing to do with getattr(). You currently can deep-copy
functions, but neither bound nor unbound methods:
>>import copy
def check(obj):
.... try:
.... copy.deepcopy(obj)
.... except:
.... return "FAILED"
.... return "OK"
....
>>def function(*args): pass
....
>>class A:
.... method = function
....
>>check(function), check(A.method), check(A().method)
('OK', 'FAILED', 'FAILED')

Whether this a bug or a sensible limitation I don't know.

Peter
Jan 5 '07 #4

P: n/a
At Friday 5/1/2007 11:01, Emin wrote:
>Thank you for your reply. As you guessed, I want to be able to select
the method at runtime as in your final example, but when I tried your
suggestion I got the same error (see below). I think the problem is
that getattr is donig something different than in my example where I
explicitly get it from the dict (see the very end of the transcript
below):
I've modified it as follows:
- using new.instancemethod to create the dynamic method
- copy uses __getstate__ to determine what is needed to copy/save.
Assuming that self.x is derived from other attributes, it's not
actually needed in the saved state, and can be omited.
- __setstate__ does the inverse: after reconstructing the object
state, sets self.x to the right value

=== cut ===
import new

class A:
def foo(self, arg):
print "in foo, self=%r arg=%r" % (self, arg)

class B(A):
def __init__(self):
self.update_dynamic_methods()

def update_dynamic_methods(self):
"Should assign the dynamic methods based on other attributes"
self.x = new.instancemethod(getattr(A,"foo"),self,B)

def __getstate__(self):
state = self.__dict__.copy()
if 'x' in state: del state['x']
return state

def __setstate__(self, state):
self.__dict__.update(state)
self.update_dynamic_methods()

b=B()
b.something = 'something'
b.foo(123)
b.x(123)

import copy
b2 = copy.deepcopy(b)
assert b2.something=='something'
b2.foo(123)
b2.x(123)
=== cut ===

--
Gabriel Genellina
Softlab SRL


__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas

Jan 5 '07 #5

This discussion thread is closed

Replies have been disabled for this discussion.