MethodType/FunctionType and decorators

[Alex Popescu]

Hi all!

I am pretty new to Python, so please excuse me if I am missing
something. Lately, I've been playing with decorators and I am a bit
confused about some behavior. Here is the code that puzzles me:

in python shell:

def function():
pass

class A(object):
def method(self):
pass

from types import MethodType
from types import FunctionType

if type(function) is FunctionType:
print "this is what I expect"

if type(A.method) is MethodType:
print "this is what I expect"

so far so good... everything seems logical.

But if a decorator is declared things are becoming different:

def deco(function):
if type(function) is MethodType:
print "MethodType"
elif type(function) is FunctionType:
print "FunctionType"

@deco
def function2():
pass

# ==this prints out FunctionType (oke)

class A(object):
@deco
def method(self):
pass

# ==this prints out FunctionType (???)

Can somebody shed some light on why I am seeing this?

(Using Python 2.5.1 on Win XP).

TIA,

../alex
--
..w( the_mindstorm )p.

[Alex Popescu]

On Jul 5, 1:52 am, Alex Popescu <[e-mail]>
wrote:

Hi all!

I am pretty new to Python, so please excuse me if I am missing
something. Lately, I've been playing with decorators and I am a bit
confused about some behavior. Here is the code that puzzles me:

in python shell:

def function():
pass

class A(object):
def method(self):
pass

from types import MethodType
from types import FunctionType

if type(function) is FunctionType:
print "this is what I expect"

if type(A.method) is MethodType:
print "this is what I expect"

so far so good... everything seems logical.

But if a decorator is declared things are becoming different:

def deco(function):
if type(function) is MethodType:
print "MethodType"
elif type(function) is FunctionType:
print "FunctionType"

@deco
def function2():
pass

# ==this prints out FunctionType (oke)

class A(object):
@deco
def method(self):
pass

# ==this prints out FunctionType (???)

Can somebody shed some light on why I am seeing this?

(Using Python 2.5.1 on Win XP).

TIA,

./alex
--
.w( the_mindstorm )p.

First of all I should correct my example to use
isinstance(function_or_method, builtin_factory_function).

Secondly, I guess what is happening is the following:

- when testing from outside the class definition, the function is
already attached to the class instance and this is the reason why its
type is instancemethod
- for decorators, my test is executed before the class is defined and
so at that moment the function is still a function; it will become a
methodinstance only after the class definition is closed

Is this correct or am I just fabulating here?

TIA,

../alex
--
..w( the_mindstorm )p.

[Alex Martelli]

Alex Popescu <th***********************@gmail.comwrote:
...

- when testing from outside the class definition, the function is
already attached to the class instance and this is the reason why its
type is instancemethod

To be more precise: when you access any attribute of a class (or
instance thereof), Python checks if that attribute is a descriptor (i.e.
the attribute's type has a __get__ method -- there are finer
distinctions based on whether it also has a __set__ method, i.e. is or
isn't a "data descriptor", but they don't come into play here).

A function does have in its type a __get__ method, IOW every
(Python-coded) function is a descriptor (a non-data one, btw).

So, when you access C.f where C is a class and f's type is Python-coded
function, you get the results of calling __get__ on that f object
itself. Those results are... the envelope, please!... ta-da: a newly
minted method object (whose im_func is f).

- for decorators, my test is executed before the class is defined and
so at that moment the function is still a function; it will become a
methodinstance only after the class definition is closed

The function object will never become a method object; it will remain a
function object (and you can get at it with C.__dict__['f'] for
example); but if you access that name as an attribute of C (or of an
instance of C), e.g. as C.f or getattr(C, 'f'), f.__get__ will be called
(and return a freshly minted method object whose im_func is f).

Is this correct or am I just fabulating here?

You have roughly the right idea (as far as decorators are concerned),
just a slightly skewed vision of the rapport between function objects
and method objects, which I hoped I helped clarify.

Here are a few toy-level examples of what I've been saying, I hope they
can further help your understanding of function vs method issues:

>>def f(): pass

....

>>class C(object): pass

....

>>a=C.f

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: type object 'C' has no attribute 'f'

>>C.f = f
a=C.f
b=C.f
a == b

True

>>a is b

False

>>type(a), type(b)

(<type 'instancemethod'>, <type 'instancemethod'>)

>>a.im_func, b.im_func

(<function f at 0x66030>, <function f at 0x66030>)

>>a.im_func is f is b.im_func

True

>>c=f.__get__(C)
type(c)

<type 'instancemethod'>

>>c.im_func

<function f at 0x66030>

>>c.im_func is C.__dict__['f'] is f

True
Alex

[greg]

Alex Popescu wrote:

- for decorators, my test is executed before the class is defined and
so at that moment the function is still a function; it will become a
methodinstance only after the class definition is closed

Closer, but still not quite right. Even after the class
is defined, the function sitting inside it is still
just an ordinary function. You can see this by looking
at

A.__dict__['method']

An instancemethod is only created when you look the
method up in an instance, i.e.

a = A()
a.method

The instancemethod encapsulates a value for 'self' and
a reference to the underlying function. This is known
as a "bound method".

(You also get an instancemethod if you look the method
up in the class, i.e.

A.method

but in that case the instancemethod doesn't contain
a value for 'self', and is known as an "unbound
method".)

--
Greg

[Michele Simionato]

On Jul 5, 3:41 am, [e-mail] (Alex Martelli) wrote:

<snip>

Alex already explained everything beautifully. I will just add a link
to
the definite guide to descriptors: http://users.rcn.com/python/download/Descriptor.htm

Michele Simionato

(who spent lot of brain cycles studying descriptors *before* that
guide was written :-()

[Alex Popescu]

On Jul 5, 11:17 am, Michele Simionato <[e-mail]>
wrote:

On Jul 5, 3:41 am, [e-mail] (Alex Martelli) wrote:

<snip>

Alex already explained everything beautifully. I will just add a link
to
the definite guide to descriptors:http://users.rcn.com/python/download/Descriptor.htm

Michele Simionato

(who spent lot of brain cycles studying descriptors *before* that
guide was written :-()

Guys, I appreciate a lot your help and explanations. It looks like I
have to read/play a bit more as some of the terms/ideas are pretty new
to me (coming to Python with a 10 years Java bag, and only with a
small dynlang bag - Ruby, Perl).

Once again thanks,

../alex
--
..w( the_mindstorm )p.

[Steve Holden]

Alex Popescu wrote:

On Jul 5, 11:17 am, Michele Simionato <[e-mail]>
wrote:

>On Jul 5, 3:41 am, [e-mail] (Alex Martelli) wrote:

>><snip>

Alex already explained everything beautifully. I will just add a link
to
the definite guide to descriptors:http://users.rcn.com/python/download/Descriptor.htm

Michele Simionato

(who spent lot of brain cycles studying descriptors *before* that
guide was written :-()

Guys, I appreciate a lot your help and explanations. It looks like I
have to read/play a bit more as some of the terms/ideas are pretty new
to me (coming to Python with a 10 years Java bag, and only with a
small dynlang bag - Ruby, Perl).

Once again thanks,

Please also realise that for someone new to the language you have dived
right into the nitty gritty. This isn't necessarily a bad way to learn,
but it does mean that there's a lot of background to assimilate as you
go along.

Good luck with your studies.

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------

[Alex Popescu]

On Jul 5, 3:32 pm, Steve Holden <[e-mail]:

Alex Popescu wrote:

On Jul 5, 11:17 am, Michele Simionato <[e-mail]>
wrote:

On Jul 5, 3:41 am, [e-mail] (Alex Martelli) wrote:

><snip>
Alex already explained everything beautifully. I will just add a link
to
the definite guide to descriptors:http://users.rcn.com/python/download/Descriptor.htm

Michele Simionato

(who spent lot of brain cycles studying descriptors *before* that
guide was written :-()

Guys, I appreciate a lot your help and explanations. It looks like I
have to read/play a bit more as some of the terms/ideas are pretty new
to me (coming to Python with a 10 years Java bag, and only with a
small dynlang bag - Ruby, Perl).

Once again thanks,

Please also realise that for someone new to the language you have dived
right into the nitty gritty. This isn't necessarily a bad way to learn,
but it does mean that there's a lot of background to assimilate as you
go along.

Good luck with your studies.

I am starting to realize this on my own :-). The true story is that
while working on Groovy (I am a committer on this dynlang meant to run
on the Java VM: http://groovy.codehaus.org) and reading some Python
materials, my interest grew exponentially. And now I have decided to
see how other succesfull java project I have co-created (TestNG:
http://testng.org) would look like in Python (this giving me the
opportunity to explore Python in more depth).

Thanks for the encouragement.

../alex
--
..w( the_mindstorm )p.

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------

[Michele Simionato]

On Jul 5, 3:17 pm, Alex Popescu <[e-mail]>
wrote:

The true story is that
while working on Groovy (I am a committer on this dynlang meant to run
on the Java VM:http://groovy.codehaus.org) and reading some Python
materials, my interest grew exponentially. And now I have decided to
see how other succesfull java project I have co-created (TestNG:http://testng.org) would look like in Python (this giving me the
opportunity to explore Python in more depth).

If you are interested in testing, you should give a look at 1)
doctest; 2) py.test
(Python unittest framework should be old hat to you and not worth
looking at).

Michele Simionato

[Alex Popescu]

On Jul 5, 5:01 pm, Michele Simionato <[e-mail]>
wrote:

On Jul 5, 3:17 pm, Alex Popescu <[e-mail]>
wrote:

The true story is that
while working on Groovy (I am a committer on this dynlang meant to run
on the Java VM:http://groovy.codehaus.org) and reading some Python
materials, my interest grew exponentially. And now I have decided to
see how other succesfull java project I have co-created (TestNG:http://testng.org) would look like in Python (this giving me the
opportunity to explore Python in more depth).

If you are interested in testing, you should give a look at 1)
doctest; 2) py.test
(Python unittest framework should be old hat to you and not worth
looking at).

Michele Simionato

Thanks for the pointers Michele. I have heard about those and their
are on my todo list. However, the features of TestNG
are very advanced and I don't think they are available in other
frameworks (TestNG is not a unit testing framework,
but a full flavored testing framework that fits perfectly functional
testing, integration testing, and with some of the
very advanced features even performance and load testing).
But I will make sure I am taking a look at the ones you pointed me to
(and I think I have a longer list available around).

bests,

../alex
--
..w( the_mindstorm )p.

[Alex Martelli]

Alex Popescu <th***********************@gmail.comwrote:
...

frameworks (TestNG is not a unit testing framework,
but a full flavored testing framework that fits perfectly functional
testing, integration testing, and with some of the
very advanced features even performance and load testing).

Nice! Does it have any integration/interoperability with FIT/Fitnesse,
btw? For certain kinds of "functional testing" (where the specs are to
be mostly written by people with no programming skills but strong
business, accounting, &c, i.e., people _used_ to thinking in terms of
tables and spreadsheets) I find that approach very interesting...
Alex

[Alex Popescu]

On Jul 6, 6:19 am, [e-mail] (Alex Martelli) wrote:

Alex Popescu <[e-mail]:

...

frameworks (TestNG is not a unit testing framework,
but a full flavored testing framework that fits perfectly functional
testing, integration testing, and with some of the
very advanced features even performance and load testing).

Nice! Does it have any integration/interoperability with FIT/Fitnesse,
btw? For certain kinds of "functional testing" (where the specs are to
be mostly written by people with no programming skills but strong
business, accounting, &c, i.e., people _used_ to thinking in terms of
tables and spreadsheets) I find that approach very interesting...

Alex

That's an interesting idea! We never tried that before but I will
definitely look into it. Thanks for the suggestion Alex.

../alex
--
..w( the_mindstorm )p.