By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
428,785 Members | 2,074 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 428,785 IT Pros & Developers. It's quick & easy.

Dynamically adding and removing methods

P: n/a
Suppose I create a class with some methods:

py> class C:
.... def spam(self, x):
.... print "spam " * x
.... def ham(self, x):
.... print "ham * %s" % x
....
py> C().spam(3)
spam spam spam
C().ham(3) ham * 3

To dynamically remove the methods, delete them from the class like you
would any other attribute:

py> del C.ham
py> C().ham(3)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: C instance has no attribute 'ham'

That's fairly straight forward -- no major surprises there.

How does one dynamically add methods to an instance or class? You might
think you can do this:

py> def eggs(self, x):
.... print "eggs * %s" % x
....
py> inst = C()
py> inst.eggs = eggs
py> inst.eggs(3)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: eggs() takes exactly 2 arguments (1 given)

Even though we've added a function to the instance, it hasn't got all the
machinery to work as a proper method. A work-around is to explicitly pass
an instance:

py> inst.eggs(inst, 3)
eggs * 3

To create a proper method, we do this:

py> import new
py> inst.eggs = new.instancemethod(eggs, None, C)
py> inst.eggs(3)
eggs * 3

You can now delete the top-level function eggs, and the method bound to
inst will not be affected. Because eggs is bound to the instance, new
instances will not understand it:

py> C().eggs(3)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: C instance has no attribute 'eggs'

Or you could put the method in the class and have all instances recognise
it:

py> C.eggs = new.instancemethod(eggs, None, C)
py> C().eggs(3)
eggs * 3

Instead of passing None as the second argument to instancemethod, you can
pass an instance. If you do that, self will automatically be set to that
instance instead of the one doing the calling:

py> def truffles(self, x):
.... print self, x
....
py> C.truffles = new.instancemethod(truffles, inst, C)
py> inst
<__main__.C instance at 0xf6d2d6cc>
py> inst.truffles(3)
<__main__.C instance at 0xf6d2d6cc> 3

No surprises there. But look what happens when we use a new instance:

py> x = C()
py> x
<__main__.C instance at 0xf6d2ca6c> x.truffles(3)

<__main__.C instance at 0xf6d2d6cc> 3
Hope this is useful to some folks.
--
Steven.

Sep 24 '05 #1
Share this Question
Share on Google+
11 Replies


P: n/a
One caveat, as I recently discovered, to dynamically adding methods is
that it doesn't work for __foo__ methods. For example, you can't make
an object into an iterator by dynamically assigning bound methods to
obj.__iter__ and obj.next. Same thing with __getitem__, __setitem__,
etc; adding them directly to the instance doesn't get you a
subscriptable object. This is because the interpreter looks at the
methods defined by the class, rather than the instance's attributes.

Just a word of caution.

Collin Winter
Sep 24 '05 #2

P: n/a
Steven D'Aprano wrote:

Or you could put the method in the class and have all instances recognise
it:

py> C.eggs = new.instancemethod(eggs, None, C)
py> C().eggs(3)
eggs * 3


Why not just add it to the class directly? You just have to be sure
it's a class and not an instance of a class.

def beacon(self, x): .... print "beacon + %s" % x
.... C.beacon = beacon
dir(A) ['__doc__', '__module__', 'beacon', 'ham', 'spam'] A.beacon(3) beacon + 3 del beacon
dir(A) ['__doc__', '__module__', 'beacon', 'ham', 'spam'] A.beacon(3) beacon + 3 dir(C)

['__doc__', '__module__', 'beacon', 'ham', 'spam']
Cheers,
Ron

Sep 25 '05 #3

P: n/a
On Sun, 25 Sep 2005 14:52:56 +0000, Ron Adam wrote:
Steven D'Aprano wrote:

Or you could put the method in the class and have all instances recognise
it:

py> C.eggs = new.instancemethod(eggs, None, C)
py> C().eggs(3)
eggs * 3


Why not just add it to the class directly? You just have to be sure
it's a class and not an instance of a class.


Because I started off explicitly adding functions to instances directly,
and when I discovered that didn't work properly, I never even tried adding
it to the class until after I discovered that instancemethod() worked.

As far as I can see, Python's treatment of functions when you dynamically
add them to classes and instances is rather confused. See, for example:

py> class Klass:
.... pass
....
py> def eggs(self, x):
.... print "eggs * %s" % x
....
py> inst = Klass() # Create a class instance.
py> inst.eggs = eggs # Dynamically add a function/method.
py> inst.eggs(1)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: eggs() takes exactly 2 arguments (1 given)

From this, I can conclude that when you assign the function to the
instance attribute, it gets modified to take two arguments instead of one.
Test it by explicitly passing an instance:

py> inst.eggs(inst, 1)
eggs * 1

My hypothesis is confirmed.

Can we get the unmodified function back again?

py> neweggs = inst.eggs
py> neweggs(1)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: eggs() takes exactly 2 arguments (1 given)

Nope. That is a gotcha. Storing a function object as an attribute, then
retrieving it, doesn't give you back the original object again.

So while you can do this:

def printgraph(f): # print a graph of a function
parameters = get_params()
draw_graph(f, parameters)

you can't do this:

def printgraph(function): # print a graph of a function
parameters = get_params()
parameters.function = f # WARNING: f is modified here
draw_graph(parameters)

When storing the function object as an instance object, it is
half-converted to a method: even though eggs is modified to expect two
arguments, Python doesn't know enough to automatically pass the instance
object as the first argument like it does when you call a true instance
method.

Furthermore, the type of the attribute isn't changed:

py> type(eggs)
<type 'function'>
py> type(inst.eggs)
<type 'function'>

But if you assign a class attribute to a function, the type changes, and
Python knows to pass the instance object:

py> Klass.eggs = eggs
py> inst2 = Klass()
py> type(inst2.eggs)
<type 'instancemethod'>
py> inst2.eggs(1)
eggs * 1

The different behaviour between adding a function to a class and an
instance is an inconsistency. The class behaviour is useful, the instance
behaviour is broken.

>>> def beacon(self, x): ... print "beacon + %s" % x
...


Did you mean bacon? *wink*
>>> C.beacon = beacon
>>> dir(A)

['__doc__', '__module__', 'beacon', 'ham', 'spam']


Okay, you aren't showing all your code. What is A?
--
Steven.

Sep 25 '05 #4

P: n/a
Steven D'Aprano wrote:
py> class Klass:
... pass
...
py> def eggs(self, x):
... print "eggs * %s" % x
...
py> inst = Klass() # Create a class instance.
py> inst.eggs = eggs # Dynamically add a function/method.
py> inst.eggs(1)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: eggs() takes exactly 2 arguments (1 given)

From this, I can conclude that when you assign the function to the
instance attribute, it gets modified to take two arguments instead of one.
No. Look at your eggs function. It takes two arguments. So the
function is not modified at all. (Perhaps you expected it to be?)
Can we get the unmodified function back again?

py> neweggs = inst.eggs
py> neweggs(1)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: eggs() takes exactly 2 arguments (1 given)

Nope. That is a gotcha. Storing a function object as an attribute, then
retrieving it, doesn't give you back the original object again.
Again, look at your eggs function. It takes two arguments. So you got
exactly the same object back. Testing this:

py> class Klass:
.... pass
....
py> def eggs(self, x):
.... print "eggs * %s" % x
....
py> inst = Klass()
py> inst.eggs = eggs
py> neweggs = inst.eggs
py> eggs is neweggs
True

So you get back exactly what you previously assigned. Note that it's
actually with *classes*, not *instances* that you don't get back what
you set:

py> Klass.eggs = eggs
py> Klass.eggs
<unbound method Klass.eggs>
py> Klass.eggs is eggs
False
Furthermore, the type of the attribute isn't changed:

py> type(eggs)
<type 'function'>
py> type(inst.eggs)
<type 'function'>

But if you assign a class attribute to a function, the type changes, and
Python knows to pass the instance object:

py> Klass.eggs = eggs
py> inst2 = Klass()
py> type(inst2.eggs)
<type 'instancemethod'>
py> inst2.eggs(1)
eggs * 1

The different behaviour between adding a function to a class and an
instance is an inconsistency. The class behaviour is useful, the instance
behaviour is broken.


With classes, the descriptor machinery is invoked:

py> Klass.eggs
<unbound method Klass.eggs>
py> Klass.eggs.__get__(None, Klass)
<unbound method Klass.eggs>
py> Klass.eggs.__get__(Klass(), Klass)
<bound method Klass.eggs of <__main__.Klass instance at 0x01290BC0>>

Because instances do not invoke the descriptor machinery, you get a
different result:

py> inst.eggs
<function eggs at 0x0126EBB0>

However, you can manually invoke the descriptor machinery if that's what
you really want:

py> inst.eggs.__get__(None, Klass)
<unbound method Klass.eggs>
py> inst.eggs.__get__(inst, Klass)
<bound method Klass.eggs of <__main__.Klass instance at 0x012946E8>>
py> inst.eggs.__get__(inst, Klass)(1)
eggs * 1

Yes, the behavior of functions that are attributes of classes is
different from the behavior of functions that are attributes of
instances. But I'm not sure I'd say that it's broken. It's a direct
result of the fact that classes are the only things that implicitly
invoke the descriptor machinery.

Note that if instances invoked the descriptor machinery, setting a
function as an attribute of an instance would mean you'd always get
bound methods back. So code like the following would break:

py> class C(object):
.... pass
....
py> def f(x):
.... print 'f(%s)' % x
....
py> def g(obj):
.... obj.f('g')
....
py> c = C()
py> c.f = f
py> g(c)
f(g)

If instances invoked the descriptor machinery, "obj.f" would return a
bound method of the "c" instance, where "x" in the "f" function was
bound to the "c" object. Thus the call to "obj.f" would result in:

py> g(c)
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
File "<interactive input>", line 2, in g
TypeError: f() takes exactly 1 argument (2 given)

Not that I'm claiming I write code like this. ;) But I'd be hesitant
to call it broken.

STeVe
Sep 25 '05 #5

P: n/a
On Mon, 26 Sep 2005 04:37:17 +1000, Steven D'Aprano
<st***@REMOVETHIScyber.com.au> declaimed the following in
comp.lang.python:
As far as I can see, Python's treatment of functions when you dynamically
add them to classes and instances is rather confused. See, for example:

py> class Klass:
... pass
...
py> def eggs(self, x):
... print "eggs * %s" % x
...
py> inst = Klass() # Create a class instance.
py> inst.eggs = eggs # Dynamically add a function/method.
py> inst.eggs(1)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: eggs() takes exactly 2 arguments (1 given)

From this, I can conclude that when you assign the function to the
instance attribute, it gets modified to take two arguments instead of one.
Test it by explicitly passing an instance:
No, see below -- you DEFINED the function to take two arguments.
Binding it as a method did NOT create the magic to automatically pass
the instance as the first argument, so instead of "inst.eggs(1)" turning
into "eggs(inst, 1), you are only invoking "eggs(1)" which is an
argument short.
Can we get the unmodified function back again?

py> neweggs = inst.eggs
py> neweggs(1)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: eggs() takes exactly 2 arguments (1 given)

Nope. That is a gotcha. Storing a function object as an attribute, then
retrieving it, doesn't give you back the original object again.
Actually, in my trials, it did give you the original -- you never
tried invoking the original eggs with only one argument. Look at the
id() outputs:
def eggs(self, x): .... print "eggs * %s" % s
.... eggs(1) Traceback (most recent call last):
File "<interactive input>", line 1, in ?
TypeError: eggs() takes exactly 2 arguments (1 given) id(eggs) 15679408 class CLS: .... pass
.... chicken = CLS()
chicken.eggs = eggs
chicken.eggs(1) Traceback (most recent call last):
File "<interactive input>", line 1, in ?
TypeError: eggs() takes exactly 2 arguments (1 given) id(chicken.eggs) 15679408 scrambled = chicken.eggs
id(scrambled) 15679408

-- ================================================== ============ <
wl*****@ix.netcom.com | Wulfraed Dennis Lee Bieber KD6MOG <
wu******@dm.net | Bestiaria Support Staff <
================================================== ============ <
Home Page: <http://www.dm.net/~wulfraed/> <
Overflow Page: <http://wlfraed.home.netcom.com/> <

Sep 25 '05 #6

P: n/a
Steven D'Aprano wrote:
On Sun, 25 Sep 2005 14:52:56 +0000, Ron Adam wrote:

Steven D'Aprano wrote:
Or you could put the method in the class and have all instances recognise
it:

py> C.eggs = new.instancemethod(eggs, None, C)
py> C().eggs(3)
eggs * 3


Why not just add it to the class directly? You just have to be sure
it's a class and not an instance of a class.

Because I started off explicitly adding functions to instances directly,
and when I discovered that didn't work properly, I never even tried adding
it to the class until after I discovered that instancemethod() worked.

As far as I can see, Python's treatment of functions when you dynamically
add them to classes and instances is rather confused. See, for example:

py> class Klass:
... pass
...
py> def eggs(self, x):
... print "eggs * %s" % x
...
py> inst = Klass() # Create a class instance.
py> inst.eggs = eggs # Dynamically add a function/method.
py> inst.eggs(1)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: eggs() takes exactly 2 arguments (1 given)

From this, I can conclude that when you assign the function to the
instance attribute, it gets modified to take two arguments instead of one.
Test it by explicitly passing an instance:

py> inst.eggs(inst, 1)
eggs * 1

My hypothesis is confirmed.


You defined it to take two arguements.. (self, x). If it's found
directly in the object instead of indirectly in the objects parent
objects, it calls it just as you defined it.

>>> def beacon(self, x):

... print "beacon + %s" % x
...

Did you mean bacon? *wink*


Of course... remembering arbitrary word letter sequences is probably my
worst skill. ;-) That, and I think for some reason the name Francis
Beacon was in my mind at the time.
>>> C.beacon = beacon
>>> dir(A)

['__doc__', '__module__', 'beacon', 'ham', 'spam']

Okay, you aren't showing all your code. What is A?


'A' is an instace of 'C' which has 'ham' and 'spam' methods in it.

Define a funciton and add it directly to class 'C'.
def beacon(self, x): .... print "beacon + %s" % x
.... C.beacon = beacon

Show that it shows up in instance 'A' and can be used.
dir(A) ['__doc__', '__module__', 'beacon', 'ham', 'spam'] A.beacon(3) beacon + 3
Delete the function. And show it's usable as a method from instance 'A'.
del beacon
dir(A) ['__doc__', '__module__', 'beacon', 'ham', 'spam'] A.beacon(3) beacon + 3

Show it's still bound to class 'C' even thought the function was deleted.
dir(C)

['__doc__', '__module__', 'beacon', 'ham', 'spam']
Cheers,
Ron


Sep 27 '05 #7

P: n/a
On Tue, 27 Sep 2005 16:42:21 +0000, Ron Adam wrote:
>>> def beacon(self, x):
... print "beacon + %s" % x
...

Did you mean bacon? *wink*


Of course... remembering arbitrary word letter sequences is probably my
worst skill. ;-) That, and I think for some reason the name Francis
Beacon was in my mind at the time.


I think you mean Francis Bacon *wink*
Thanks to everybody for all the feedback, it was useful, but I was finding
that I couldn't keep it all straight in my head (as seen by the mistaken
conclusions I drew earlier). So I wrote a short script to systematically
test the various combinations of functions and methods. I'm satisfied with
the results. For the sake of completeness, here are my results:

In the script below, ham and spam are defined first as regular functions
outside of a class definition. ham is defined with a placeholder for
"self", spam is not. They are then assigned (in separate tests) to both an
instance and a class.

eggs is defined as a class method, and then assigned to an object outside
a class. Think of it as the control variable.

"Failure" means the call raises an exception, "Success" means it does not.

Here are the results:

####################

Comparing module function with instance method:
================================================== ===
Test: func= | ham | spam | eggs |
----------------------+---------+---------+---------+
func(arg) | Failure | Success | Failure |
inst.func(arg) | Failure | Success | Success |
inst.func(self, arg) | Success | Failure | Failure |
Comparing module function with class method:
================================================== ===
Test: func= | ham | spam | eggs |
----------------------+---------+---------+---------+
func(arg) | Failure | Success | Failure |
inst.func(arg) | Success | Failure | Success |
inst.func(self, arg) | Failure | Failure | Failure |
Comparing object types:
====================================
Where: | ham | spam | eggs |
--------------+------+------+------+
module level | f | f | m |
instance | f | f | m |
class | m | m | m |

Key: f = function; m = method; ? = other

####################

This clearly shows that assigning a function to a class attribute invokes
whatever machinery Python uses to turn that function into a true method,
but assigning it to an instance does not.
Here is the script for those interested:

####################

"""Testing of the dynamic addition of functions to classes and instances."""

TABLE = """
%s
================================================== ===
Test: func= | ham | spam | eggs |
----------------------+---------+---------+---------+
func(arg) | %s | %s | %s |
inst.func(arg) | %s | %s | %s |
inst.func(self, arg) | %s | %s | %s |
"""

TABLE2 = """
%s
====================================
Where: | ham | spam | eggs |
--------------+------+------+------+
module level | %s | %s | %s |
instance | %s | %s | %s |
class | %s | %s | %s |

Key: f = function; m = method; ? = other
"""

# Functions and methods to be tested:

def ham(self, x):
"""Function with placeholder self argument."""
return "ham " * x

def spam(x):
"""Function without placeholder self argument."""
return "spam " * x

class Klass:
def eggs(self, x):
"""Method defined in the class."""
return "%s eggs" % x

eggs = Klass.eggs

# Testing functions:

def get_type(obj):
s = type(obj).__name__
if s == "function":
return "f"
elif s == "instancemethod":
return "m"
else:
return "?"

def single_test(func, args):
"""Calls func(*args) and returns None if it succeeds or an exception if it fails."""
try:
func(*args)
return "Success"
except Exception, obj:
return "Failure"

def multiple_test(instance, label):
"""Runs multiple tests and returns a table of results."""
L = [label]
for f in (ham, spam, eggs, instance.ham, instance.spam, instance.eggs):
L.append(single_test(f, [1]))
for f in (instance.ham, instance.spam, instance.eggs):
L.append(single_test(f, [instance, 1]))
return TABLE % tuple(L)

def type_test(inst1, inst2):
L = ["Comparing object types:"]
for obj in (ham, spam, eggs, inst1.ham, inst1.spam, inst1.eggs, \
inst2.ham, inst2.spam, inst2.eggs):
L.append(get_type(obj))
return TABLE2 % tuple(L)

def main():
inst1 = Klass()
inst1.ham = ham
inst1.spam = spam
print multiple_test(inst1, "Comparing module function with instance method:")
inst2 = Klass()
Klass.ham = ham
Klass.spam = spam
print multiple_test(inst2, "Comparing module function with class method:")
print type_test(inst1, inst2)
if __name__ == "__main__":
main()

####################
I hope this is as interesting and useful for others as it was for me.
--
Steven.

Sep 28 '05 #8

P: n/a
Steven D'Aprano wrote:
On Tue, 27 Sep 2005 16:42:21 +0000, Ron Adam wrote:

>>>def beacon(self, x):

... print "beacon + %s" % x
...
Did you mean bacon? *wink*
Of course... remembering arbitrary word letter sequences is probably my
worst skill. ;-) That, and I think for some reason the name Francis
Beacon was in my mind at the time.

I think you mean Francis Bacon *wink*


Yes, I mean him, Beacon is his fathers sirname. I'm not sure if Francis
changed it or if his father did. (?)

This clearly shows that assigning a function to a class attribute invokes
whatever machinery Python uses to turn that function into a true method,
but assigning it to an instance does not.


Actually I think I'm getting more confused. At some point the function
is wrapped. Is it when it's assigned, referenced, or called?

Cheers,
Ron


Sep 28 '05 #9

P: n/a

"Ron Adam" <rr*@ronadam.com> wrote in message
news:sX********************@tornado.tampabay.rr.co m...
Actually I think I'm getting more confused. At some point the function
is wrapped. Is it when it's assigned, referenced, or called?


When it is referenced via the class.
If you lookup in class.__dict__, the function is still a function.
class C(object): .... def meth(self): pass
.... C.__dict__['meth'] <function meth at 0x0090B018> C.meth <unbound method C.meth> C().meth

<bound method C.meth of <__main__.C object at 0x008E4688>>

I am not sure, without looking, how much of this is language definition and
how much CPython implementation, but I think mostly the latter, as long as
the inheritance tree lookup behavior is as specified.

Terry J. Reedy

Sep 28 '05 #10

P: n/a
Terry Reedy wrote:
"Ron Adam" <rr*@ronadam.com> wrote in message
news:sX********************@tornado.tampabay.rr.co m...
Actually I think I'm getting more confused. At some point the function
is wrapped. Is it when it's assigned, referenced, or called?

When it is referenced via the class.
If you lookup in class.__dict__, the function is still a function.

class C(object): ... def meth(self): pass
...
C.__dict__['meth'] <function meth at 0x0090B018>
C.meth <unbound method C.meth>
C().meth

<bound method C.meth of <__main__.C object at 0x008E4688>>

I am not sure, without looking, how much of this is language definition and
how much CPython implementation, but I think mostly the latter


Well, being that the descriptor machinery is defined in the language
reference[1][2], I'd have to say it's entirely the former. The
descriptor machinery says basically that, for classes,
C.meth
should always be doing the equivalent of:
C.__dict__['meth'].__get__(None, C)
and for instances,
c.meth
should always be doing the equivalent of:
type(c).__dict__['meth'].__get__(c, type(c))

[1] http://docs.python.org/ref/descriptors.html
[2] http://docs.python.org/ref/descriptor-invocation.html

STeVe
Sep 28 '05 #11

P: n/a
Terry Reedy wrote:
"Ron Adam" <rr*@ronadam.com> wrote in message
news:sX********************@tornado.tampabay.rr.co m...
Actually I think I'm getting more confused. At some point the function
is wrapped. Is it when it's assigned, referenced, or called?

When it is referenced via the class.


Ok, that's what I suspected. Thanks for clarifying this.
If you lookup in class.__dict__, the function is still a function.
Now why did I not think of doing that? :-)

class C(object):
... def meth(self): pass
...
C.__dict__['meth']
<function meth at 0x0090B018>
C.meth
<unbound method C.meth>
C().meth
<bound method C.meth of <__main__.C object at 0x008E4688>>


Ok, I got it now. Given class 'C' below, i.m(1) does....
class C(object): .... def m(self, x):
.... return repr(x)
.... i = C()
boundmeth = i.__getattribute__('m')
boundmeth <bound method C.m of <__main__.C object at 0x009D1C70>> boundmeth(1) '1'
import dis
dis.dis(boundmeth) 3 0 LOAD_GLOBAL 0 (repr)
3 LOAD_FAST 1 (x)
6 CALL_FUNCTION 1
9 RETURN_VALUE dis.dis(C.m) 3 0 LOAD_GLOBAL 0 (repr)
3 LOAD_FAST 1 (x)
6 CALL_FUNCTION 1
9 RETURN_VALUE dis.dis(C.__dict__['m']) 3 0 LOAD_GLOBAL 0 (repr)
3 LOAD_FAST 1 (x)
6 CALL_FUNCTION 1
9 RETURN_VALUE

Hmm... that didn't tell me anything. (?)
boundmeth <bound method C.m of <__main__.C object at 0x009D1C70>> C.m <unbound method C.m> C.__dict__['m']

<function m at 0x009D6930>

Time to start digging around in the source code I guess. ;-)
I am not sure, without looking, how much of this is language definition and
how much CPython implementation, but I think mostly the latter, as long as
the inheritance tree lookup behavior is as specified.

Terry J. Reedy


Yes, it hard to tell sometimes where CPython ends and Python begins.

Cheers,
Ron

Sep 29 '05 #12

This discussion thread is closed

Replies have been disabled for this discussion.