473,378 Members | 1,138 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,378 software developers and data experts.

bug in copy.deepcopy or in getattr or in my understanding?

Dear experts,

I got some unexpected behavior in getattr and copy.deepcopy (see
transcript below). I'm not sure if this is actually a bug in
copy.deepcopy or if I'm doing something too magical with getattr.
Comments would be appreciated.

Thanks,
-Emin

######### Transcript follows ##################
Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>># THE FOLLOWING BREAKS
class a:
.... def foo(self):
.... print 'hi'
....
>>class b(a):
.... def __init__(self):
.... self.y = getattr(self,'foo')
....
>>c = b()
import copy
copy.deepcopy(c)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "c:\Python25\lib\copy.py", line 162, in deepcopy
y = copier(x, memo)
File "c:\Python25\lib\copy.py", line 291, in _deepcopy_inst
state = deepcopy(state, memo)
File "c:\Python25\lib\copy.py", line 162, in deepcopy
y = copier(x, memo)
File "c:\Python25\lib\copy.py", line 254, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "c:\Python25\lib\copy.py", line 189, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "c:\Python25\lib\copy.py", line 322, in _reconstruct
y = callable(*args)
File "c:\Python25\lib\copy_reg.py", line 92, in __newobj__
return cls.__new__(cls, *args)
TypeError: instancemethod expected at least 2 arguments, got 0
>># THE FOLLOWING WORKS
class b(a):
.... def __init__(self):
.... self.x = self.__class__.__bases__[0].__dict__['foo']
....
>>c=b()
copy.deepcopy(c)
<__main__.b instance at 0x00EADE18>

Jan 4 '07 #1
4 3633
At Thursday 4/1/2007 17:26, Emin wrote:
>I got some unexpected behavior in getattr and copy.deepcopy (see
transcript below). I'm not sure if this is actually a bug in
copy.deepcopy or if I'm doing something too magical with getattr.
Comments would be appreciated.
Both examples are different. #1 stores a *bound* method into an
instance attribute. Bound methods carry a reference to "self", this
creates a cyclic reference that may cause problems to the garbage
collector (and confuses deepcopy, apparently).
#2 uses and *unbound* method and it's the usual way.
>class a:
... def foo(self):
... print 'hi'
...
>class b(a): #1
... def __init__(self):
... self.y = getattr(self,'foo')
>class b(a): #2
... def __init__(self):
... self.x = self.__class__.__bases__[0].__dict__['foo']
For #2 you can simply say:

class b(a):
x = a.foo

If you have to choose at runtime (a simplified version of your own code):

class b(a):
def __init__(self):
name = select_method_to_use(..., default="foo")
self.x = getattr(a, name)

You *know* your bases because you wrote them in the class statement
(or use super() instead of playing with __bases__); and getattr works
fine here so you don't need to mess with the __dict__ details.

(Note that #1 was *almost* right, you had to replace self by the base class)
--
Gabriel Genellina
Softlab SRL


__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas

Jan 5 '07 #2
Dear Gabriel,

Thank you for your reply. As you guessed, I want to be able to select
the method at runtime as in your final example, but when I tried your
suggestion I got the same error (see below). I think the problem is
that getattr is donig something different than in my example where I
explicitly get it from the dict (see the very end of the transcript
below):
--------------------------- Transcript Follows
----------------------------------------------------------

Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>># The following shows that using getattr to grab a method is
# incompatible with copy.deepcopy
class a:
.... def foo(self): pass
....
>>class b(a):
.... def __init__(self):
.... self.x = getattr(a,'foo')
....
>>import copy
c=b()
copy.deepcopy(c)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "c:\Python25\lib\copy.py", line 162, in deepcopy
y = copier(x, memo)
File "c:\Python25\lib\copy.py", line 291, in _deepcopy_inst
state = deepcopy(state, memo)
File "c:\Python25\lib\copy.py", line 162, in deepcopy
y = copier(x, memo)
File "c:\Python25\lib\copy.py", line 254, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "c:\Python25\lib\copy.py", line 189, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "c:\Python25\lib\copy.py", line 322, in _reconstruct
y = callable(*args)
File "c:\Python25\lib\copy_reg.py", line 92, in __newobj__
return cls.__new__(cls, *args)
TypeError: instancemethod expected at least 2 arguments, got 0
>>>
# The following shows that getattr is doing something different
# than looking in the __dict__ of base classes
b.__bases__[0].__dict__['foo']
<function foo at 0x009F0CF0>
>>getattr(a,'foo')
<unbound method a.foo>
>>>

On Jan 4, 10:08 pm, Gabriel Genellina <gagsl...@yahoo.com.arwrote:
At Thursday 4/1/2007 17:26, Emin wrote:
I got some unexpected behavior in getattr and copy.deepcopy (see
transcript below). I'm not sure if this is actually a bug in
copy.deepcopy or if I'm doing something too magical with getattr.
Comments would be appreciated.Both examples are different. #1 stores a *bound* method into an
instance attribute. Bound methods carry a reference to "self", this
creates a cyclic reference that may cause problems to the garbage
collector (and confuses deepcopy, apparently).
#2 uses and *unbound* method and it's the usual way.
>>class a:
... def foo(self):
... print 'hi'
...
>>class b(a): #1
... def __init__(self):
... self.y = getattr(self,'foo')
>>class b(a): #2
... def __init__(self):
... self.x = self.__class__.__bases__[0].__dict__['foo']For #2 you can simply say:

class b(a):
x = a.foo

If you have to choose at runtime (a simplified version of your own code):

class b(a):
def __init__(self):
name = select_method_to_use(..., default="foo")
self.x = getattr(a, name)

You *know* your bases because you wrote them in the class statement
(or use super() instead of playing with __bases__); and getattr works
fine here so you don't need to mess with the __dict__ details.

(Note that #1 was *almost* right, you had to replace self by the base class)

--
Gabriel Genellina
Softlab SRL

__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!http://www.yahoo.com.ar/respuestas
Jan 5 '07 #3
Emin wrote:
Thank you for your reply. As you guessed, I want to be able to select
the method at runtime as in your final example, but when I tried your
suggestion I got the same error (see below). I think the problem is
that getattr is donig something different than in my example where I
explicitly get it from the dict (see the very end of the transcript
below):
This has nothing to do with getattr(). You currently can deep-copy
functions, but neither bound nor unbound methods:
>>import copy
def check(obj):
.... try:
.... copy.deepcopy(obj)
.... except:
.... return "FAILED"
.... return "OK"
....
>>def function(*args): pass
....
>>class A:
.... method = function
....
>>check(function), check(A.method), check(A().method)
('OK', 'FAILED', 'FAILED')

Whether this a bug or a sensible limitation I don't know.

Peter
Jan 5 '07 #4
At Friday 5/1/2007 11:01, Emin wrote:
>Thank you for your reply. As you guessed, I want to be able to select
the method at runtime as in your final example, but when I tried your
suggestion I got the same error (see below). I think the problem is
that getattr is donig something different than in my example where I
explicitly get it from the dict (see the very end of the transcript
below):
I've modified it as follows:
- using new.instancemethod to create the dynamic method
- copy uses __getstate__ to determine what is needed to copy/save.
Assuming that self.x is derived from other attributes, it's not
actually needed in the saved state, and can be omited.
- __setstate__ does the inverse: after reconstructing the object
state, sets self.x to the right value

=== cut ===
import new

class A:
def foo(self, arg):
print "in foo, self=%r arg=%r" % (self, arg)

class B(A):
def __init__(self):
self.update_dynamic_methods()

def update_dynamic_methods(self):
"Should assign the dynamic methods based on other attributes"
self.x = new.instancemethod(getattr(A,"foo"),self,B)

def __getstate__(self):
state = self.__dict__.copy()
if 'x' in state: del state['x']
return state

def __setstate__(self, state):
self.__dict__.update(state)
self.update_dynamic_methods()

b=B()
b.something = 'something'
b.foo(123)
b.x(123)

import copy
b2 = copy.deepcopy(b)
assert b2.something=='something'
b2.foo(123)
b2.x(123)
=== cut ===

--
Gabriel Genellina
Softlab SRL


__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas

Jan 5 '07 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Andreas Kuntzagk | last post by:
Hi, There are three ways to (shallow)copy a list l I'm aware of: >>> l2=list(l) >>> l2=l >>> l2.copy.copy(l) Are there any differences? Are there more (reasonable) ways? I think the first...
2
by: Nick Jacobson | last post by:
This question is with regard to the * operator as used for sequence concatenation. There's the well-known gotcha: a = ] b = a*3 b = 4 print b
6
by: Alexander Zatvornitskiy | last post by:
Hello! I have trouble with copy/deepcopy. It seems, I just don't understand something. Please explain where things goes wrong and how to do it the right way. I have one class: class...
0
by: Joshua Ginsberg | last post by:
Howdy -- I have a class that has an attribute that is a dictionary that contains an object that has a kword argument that is a lambda. Confused yet? Simplified example: import copy class...
2
by: Alex | last post by:
Entering the following in the Python shell yields >>> help(dict.copy) Help on method_descriptor: copy(...) D.copy() -> a shallow copy of D >>>
4
by: KraftDiner | last post by:
I'm having trouble getting a copy of and object... (a deep copy) I'm writing a method that creates a mirror image of an object (on screen) In order to do this i need to get a copy of the object...
28
by: robert | last post by:
In very rare cases a program crashes (hard to reproduce) : * several threads work on an object tree with dict's etc. in it. Items are added, deleted, iteration over .keys() ... ). The threads are...
0
bartonc
by: bartonc | last post by:
I was playing around with the Simple Metaclassing thread and found something odd: >>> class aClass: ... classVar1 = 'hello' ... def __init__(self, arg1): ... self.instVar1 = arg1...
3
by: yoma | last post by:
python version 2.5 in module copy we all know that copy have two method: copy() and deepcopy(). and the explain is - A shallow copy constructs a new compound object and then (to the extent...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.