473,836 Members | 1,519 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Descriptor/Decorator challenge

I had an idea but no time to think it through.
Perhaps the under-under name mangling trick
can be replaced (in Py3.0) with a suitably designed decorator.
Your challenge is to write the decorator.
Any trick in the book (metaclasses, descriptors, etc) is fair game.
Raymond

-------- how we currently localize method access with name mangling
------
class A:
def __m(self):
print 'A.__m'
def am(self):
self.__m()

class B(A):
def __m(self):
print 'B.__m'
def bm(self):
self.__m()

m = B()
m.am() # prints 'A.__m'
m.bm() # prints 'B.__m'
-------- how I would like to localize method access with a decorator
------
class A:
@localmethod
def m(self):
print 'A.m'
def am(self):
self.m()

class B(A):
@localmethod
def m(self):
print 'B.m'
def bm(self):
self.m()

m = B()
m.am() # prints 'A.m'
m.bm() # prints 'B.m'

---------------------

P.S. Here's a link to the descriptor how-to:
http://users.rcn.com/python/download/Descriptor.htm

Mar 5 '07 #1
9 1441
On 5 Mar, 07:31, "Raymond Hettinger" <pyt...@rcn.com wrote:
I had an idea but no time to think it through.
Perhaps the under-under name mangling trick
can be replaced (in Py3.0) with a suitably designed decorator.
Your challenge is to write the decorator.
Any trick in the book (metaclasses, descriptors, etc) is fair game.
I had some time this lunchtimes and I needed to calm my nerves so I
took up your challenge :)
Here is my poor effort. I'm sure lots of things are wrong with it but
I'm not sure I'll look at it again.

from types import MethodType, FunctionType

# The suggested localmethod decorator
class localmethod(obj ect):
def __init__(self, f):
self.f = f
self.defclass = None
self.nextmethod = None
def __get__(self, obj, objtype=None):
callobj = obj or objtype
if callobj.callerc lass == self.defclass:
return MethodType(self .f, obj, objtype)
elif self.nextmethod :
return self.nextmethod .__get__(obj, objtype)
else:
raise AttributeError

class BoundMethod(obj ect):
def __init__(self, meth, callobj, callerclass):
self.meth = meth
self.callobj = callobj
self.callerclas s = callerclass
def __call__(self, *args, **kwargs):
callobj = self.callobj
try:
callobj.callerc lass = self.callerclas s
return self.meth(*args , **kwargs)
finally:
callobj.callerc lass = None

# A 'mormal' method decorator is needed as well
class method(object):
def __init__(self, f):
self.f = f
self.defclass = None
def __get__(self, obj, objtype=None):
callobj = obj or objtype
return BoundMethod(Met hodType(self.f, obj, objtype), callobj,
self.defclass)

class Type(type):
def __new__(self, name, bases, attrs):
for attr, val in attrs.items():
if type(val) == FunctionType:
attrs[attr] = method(val)
return type.__new__(se lf, name, bases, attrs)
def __init__(self, name, bases, attrs):
for attr, val in attrs.iteritems ():
if type(val) == localmethod:
val.defclass = self
for base in self.mro()[1:]:
if attr in base.__dict__:
nextmethod = base.__dict__[attr]
val.nextmethod = nextmethod
break
elif type(val) == method:
val.defclass = self
class Object(object):
__metaclass__ = Type
# Note: any class or object has to have a callerclass attribute
for this to work.
# That makes it thread - incompatible I guess.
callerclass = None

# Here is your example code

class A(Object):
@localmethod
def m(self):
print 'A.m'
def am(self):
self.m()

class B(A):
@localmethod
def m(self):
print 'B.m'
def bm(self):
self.m()

m = B()
m.am() # prints 'A.m'
m.bm() # prints 'B.m'

# Untested beyond this particular example!

--
Arnaud

Mar 5 '07 #2
On 5 Mar, 14:38, "Arnaud Delobelle" <arno...@google mail.comwrote:
On 5 Mar, 07:31, "Raymond Hettinger" <pyt...@rcn.com wrote:
I had an idea but no time to think it through.
Perhaps the under-under name mangling trick
can be replaced (in Py3.0) with a suitably designed decorator.
Your challenge is to write the decorator.
Any trick in the book (metaclasses, descriptors, etc) is fair game.

I had some time this lunchtimes and I needed to calm my nerves so I
took up your challenge :)
Here is my poor effort. I'm sure lots of things are wrong with it but
I'm not sure I'll look at it again.
Well in fact I couldn't help but try to improve it a bit. Objects now
don't need a callerclass attribute, instead all necessary info is
stored in a global __callerclass__ . Bits that didn't work now do.

So here is my final attempt, promised. The awkward bits are:
* how to find out where a method is called from
* how to resume method resolution once it has been established a
local method has to be bypassed, as I don't know how to interfere
directly with mro.

Feedback of any form is welcome (though I prefer when it's polite :)

--------------------
from types import MethodType, FunctionType

class IdDict(object):
def __init__(self):
self.objects = {}
def __getitem__(sel f, obj):
return self.objects.ge t(id(obj), None)
def __setitem__(sel f, obj, callerclass):
self.objects[id(obj)] = callerclass
def __delitem__(sel f, obj):
del self.objects[id(obj)]

# This stores the information about from what class an object is
calling a method
# It is decoupled from the object, better than previous version
# Maybe that makes it easier to use with threads?
__callerclass__ = IdDict()

# The purpose of this class is to update __callerclass__ just before
and after a method is called
class BoundMethod(obj ect):
def __init__(self, meth, callobj, callerclass):
self.values = meth, callobj, callerclass
def __call__(self, *args, **kwargs):
meth, callobj, callerclass = self.values
if callobj is None and args:
callobj = args[0]
try:
__callerclass__[callobj] = callerclass
return meth(*args, **kwargs)
finally:
del __callerclass__[callobj]

# A 'normal' method decorator is needed as well
class method(object):
def __init__(self, f):
self.f = f
def __get__(self, obj, objtype=None):
return BoundMethod(Met hodType(self.f, obj, objtype), obj,
self.defclass)

class LocalMethodErro r(AttributeErro r):
pass

# The suggested localmethod decorator
class localmethod(met hod):
def __get__(self, obj, objtype=None):
callobj = obj or objtype
defclass = self.defclass
if __callerclass__[callobj] is defclass:
return MethodType(self .f, obj, objtype)
else:
# The caller method is from a different class, so look for
the next candidate.
mro = iter((obj and type(obj) or objtype).mro())
for c in mro: # Skip all classes up to the localmethod's
class
if c == defclass: break
name = self.name
for base in mro:
if name in base.__dict__:
try:
return base.__dict__[name].__get__(obj,
objtype)
except LocalMethodErro r:
continue
raise LocalMethodErro r, "localmetho d '%s' is not accessible
outside object '%s'" % (self.name, self.defclass._ _name__)

class Type(type):
def __new__(self, name, bases, attrs):
# decorate all function attributes with 'method'
for attr, val in attrs.items():
if type(val) == FunctionType:
attrs[attr] = method(val)
return type.__new__(se lf, name, bases, attrs)
def __init__(self, name, bases, attrs):
for attr, val in attrs.iteritems ():
# Inform methods of what class they are created in
if isinstance(val, method):
val.defclass = self
# Inform localmethod of their name (in case they have to
be bypassed)
if isinstance(val, localmethod):
val.name = attr

class Object(object):
__metaclass__ = Type

# Here is your example code

class A(Object):
@localmethod
def m(self):
print 'A.m'
def am(self):
self.m()

class B(A):
@localmethod
def m(self):
print 'B.m'
def bm(self):
self.m()

m = B()
m.am() # prints 'A.m'
m.bm() # prints 'B.m'
# Added:
B.am(m) # prints 'A.m'
B.bm(m) # prints 'B.m'
m.m() # LocalMethodErro r (which descends from AttributeError)

# Untested beyond this particular example!
--------------------

--
Arnaud
Mar 5 '07 #3
On Mar 5, 2:31 am, "Raymond Hettinger" <pyt...@rcn.com wrote:
I had an idea but no time to think it through.
Perhaps the under-under name mangling trick
can be replaced (in Py3.0) with a suitably designed decorator.
Your challenge is to write the decorator.
Any trick in the book (metaclasses, descriptors, etc) is fair game.

Raymond

-------- how we currently localize method access with name mangling
------
class A:
def __m(self):
print 'A.__m'
def am(self):
self.__m()

class B(A):
def __m(self):
print 'B.__m'
def bm(self):
self.__m()

m = B()
m.am() # prints 'A.__m'
m.bm() # prints 'B.__m'

-------- how I would like to localize method access with a decorator
------
class A:
@localmethod
def m(self):
print 'A.m'
def am(self):
self.m()

class B(A):
@localmethod
def m(self):
print 'B.m'
def bm(self):
self.m()

m = B()
m.am() # prints 'A.m'
m.bm() # prints 'B.m'

---------------------

P.S. Here's a link to the descriptor how-to:
http://users.rcn.com/python/download/Descriptor.htm
What would the semantics be if m is decorated as local only in A or
only in B ?

George

Mar 6 '07 #4
On 5 Mar, 18:59, "Arnaud Delobelle" <arno...@google mail.comwrote:
[snip]
Well in fact I couldn't help but try to improve it a bit. Objects now
don't need a callerclass attribute, instead all necessary info is
stored in a global __callerclass__ . Bits that didn't work now do.
OK that wasn't really thought through. Because I changed the design
mid-way through writing it __callerclass__ wasn't doing the right
thing. I've sorted the issues I could see and made it (hopefully)
thread-safe. I'm not going to pollute this list again with my code so
I've put it at the following address:

http://marooned.org.uk/local.py

The problem is that all normal functions need to be decorated with
'@function' for it to work completely: if I understand correctly the
snippet below should raise an exception. It only does so if 'f' is
decorated with '@function' as below.

----------
@function
def f(x):
x.l()

class C(Object):
@localmethod
def l(self):
print "Shouldn't get here"
def v(self):
return f(self)

C().v() # Raises LocalMethod exception
----------

PS: in fact I'm not sure it's a good idea to decorate local methods:
what about local attributes which are not methods? They have to be
treated differently as only functions can be decorated.

What about functions / classes which are local to a module?

--
Arnaud
Mar 6 '07 #5
[George Sakkis]
What would the semantics be if m is decorated as local only in A or
only in B ?
The goal is to as closely as possible emulate the sematics of under-
under name mangling.
Raymond

Mar 6 '07 #6
On Tue, Mar 06, 2007 at 11:44:48AM -0800, Arnaud Delobelle wrote:
On 5 Mar, 18:59, "Arnaud Delobelle" <arno...@google mail.comwrote:
[snip]
Well in fact I couldn't help but try to improve it a bit. Objects now
don't need a callerclass attribute, instead all necessary info is
stored in a global __callerclass__ . Bits that didn't work now do.

OK that wasn't really thought through. Because I changed the design
mid-way through writing it __callerclass__ wasn't doing the right
thing. I've sorted the issues I could see and made it (hopefully)
thread-safe. I'm not going to pollute this list again with my code so
I've put it at the following address:

http://marooned.org.uk/local.py
I fooled around with this a bit and even when using different techniques
than Arnaud (namely stack inspection and/or class decorators) it ends up
looking the same. In order to pull this off you need to

1) mark the localmethods as special
(@localmethod works here)
2) mark all non-localmethods as not special
(metaclasses, class decorators, or module-level stack inspection)
3) keep track of the call stack
(can be implemented as part of #1 and #2 but adds overhead regardless)

Double underscore names take care of #1 at compile time and by definition
anything not name-manged falls into the non-special class of #2. After
that the special/non-special calls are given different names so the
native call semantics take care of the call stack for you.

With some bytecode manipulation it should be possible to fake the same
name mangling by using just @localmethod decorators and adding some
overhead to always checking the current class (eg A.m() for the function
am()) and falling back on doing the normal call of self.m(). This could
be made more exact by doing inspection after the fact with any of
metaclasses/class decorators/module inspection because then we could
inspect what is a @localmethod and what isn't all the way through the class
tree.

I could be wrong on the byte-hack part as I've only recently learned
to read the chicken bones that are byte codes (thanks to PyCon sprints
I got to spend time picking python-devs's brains over burritos). It
seem plausible if fragile.

-Jack
Mar 8 '07 #7
Raymond Hettinger wrote:
Any trick in the book (metaclasses, descriptors, etc) is fair game.
So you are asking for a serious hack, right?

As soon as I saw your challenge I thought "That's difficult. Very
difficult.
No way I can solve that with a simple descriptor/decorator. I need
more POWER.
Time to look at the byteplay module".

The byteplay module by Noam Raphael (http://byteplay.googlecode.com/
svn/trunk/byteplay.py) seems to exist just to make possible
spectacular hacks. So I took
your challenge as an opportunity to explore a bit its secrets. In
doing so,
I have also decided to break the rules and not to solve your problem,
but
a different one, which is the one I am more interested in ;)

Basically, I am happy with the double underscores, but I am unhappy
with
the fact that a method does not know the class where it is defined, so
that you have to repeat the name of the class in ``super``.

With some bytecode + metaclass hackery it is possible to make the
methods
smart enough to recognize the class where they are defined, so your
example could be solved as follows:

from currentclass import CurrentClassEna bled

class A(CurrentClassE nabled):
def m(self):
print 'A.m'
def am(self):
CurrentClass.m( self) # the same as A.m(self)

class B(A):
def m(self):
print 'B.m'
def bm(self):
CurrentClass.m( self) # the same as B.m(self)
def superm(self):
super(CurrentCl ass, self).m() # the same as super(B, self).m()

m = B()
m.am() # prints 'A.m'
m.bm() # prints 'B.m'
m.superm() # prints 'A.m'

As you see, as a byproduct, double underscores are not needed anymore,
since methods are called directly from the CurrentClass. The approach
also works for ordinary attributes which are not methods.

The code to enable recognition of CurrentClass is short enough to be
includede here, but I will qualify it as a five star-level hackery:

$ cat currentclass.py
# requires byteplay by Noam Raphael
# see http://byteplay.googlecode.com/svn/trunk/byteplay.py
from types import FunctionType
from byteplay import Code, LOAD_GLOBAL, STORE_FAST, LOAD_FAST

def addlocalvar(f, locname, globname):
if locname not in f.func_code.co_ names:
return f # do nothing
c = Code.from_code( f.func_code)
c.code[1:1] = [(LOAD_GLOBAL, globname), (STORE_FAST, locname)]
for i, (opcode, value) in enumerate(c.cod e[2:]):
if opcode == LOAD_GLOBAL and value == locname:
c.code[i+2] = (LOAD_FAST, locname)
f.func_code = c.to_code()
return f

class _CurrentClassEn abled(type):
def __new__(mcl, name, bases, dic):
for n, v in dic.iteritems() :
if isinstance(v, FunctionType):
dic[n] = addlocalvar(v, 'CurrentClass', name)
return super(_CurrentC lassEnabled, mcl).__new__(mc l, name,
bases, dic)

class CurrentClassEna bled:
__metaclass__ = _CurrentClassEn abled
Enjoy!

Michele Simionato

Mar 8 '07 #8
En Thu, 08 Mar 2007 05:37:39 -0300, Michele Simionato
<mi************ ***@gmail.comes cribió:
The code to enable recognition of CurrentClass is short enough to be
includede here, but I will qualify it as a five star-level hackery:
You forgot the standard disclaimer: "This is extremely dangerous stuff,
only highly trained professionals can do that! Kids, never try this at
home!"

--
Gabriel Genellina

Mar 8 '07 #9
On Mar 8, 10:20 am, "Gabriel Genellina" <gagsl-...@yahoo.com.a r>
wrote:
You forgot the standard disclaimer: "This is extremely dangerous stuff,
only highly trained professionals can do that! Kids, never try this at
home!"
;)

Yep, the only good use case for this kind of games is for prototyping
in current Python
features that are candidate for inclusion in future versions of
Python, but this was
what Raymond asked. However, there is also the fun of it ;)

Michele Simionato

Mar 8 '07 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

37
2617
by: Bengt Richter | last post by:
ISTM that @limited_expression_producing_function @another def func(): pass is syntactic sugar for creating a hidden list of functions. (Using '|' in place of '@' doesn't change the picture much (except for people whose tools depend on '@' ;-)). I.e., (not having the source or time to delve) the apparent semantics of the above is something roughly like
10
1580
by: sysfault | last post by:
Does anyone know of any good documentation on these topics, doesn't look like the official python tutorial covers them. Thanks in advance. -- A wise man knows he knows nothing.
6
1736
by: Daishi Harada | last post by:
Hi, I'm trying to find/write a memoizing decorator that works both for functions and methods. I've been looking at the following two samples: http://wiki.python.org/moin/PythonDecoratorLibrary#head-11870a08b0fa59a8622201abfac735ea47ffade5 http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/325205 AFAICT, both of these seem to be targetting bare
2
1547
by: WaterWalk | last post by:
Hello, I was recently learning python decorator and descriptor and emulated a @classmethod decorator: class EmuClassMethod(object): def __init__(self, f=None): self.f = f def __get__(self, obj, klass=None): if klass is None: klass = type(obj) def wrapped(*args): return self.f(klass, *args)
14
2150
by: Wildemar Wildenburger | last post by:
Hello folks :) This has got to be an FAQ, but somehow I can't seem to find an answer that I understand ... I thought: I'll just write a decorator that lets me react to method calls easily (the ever so popular observer-pattern). I've looked at some recepies, but I just don't get them (I'm feeling kinda dumb today, sorry). What I'd *like* to do is this: def observable(func):
4
2475
by: thomas.karolski | last post by:
Hi, I would like to create a Decorator metaclass, which automatically turns a class which inherits from the "Decorator" type into a decorator. A decorator in this case, is simply a class which has all of its decorator implementation inside a decorator() method. Every other attribute access is being proxied to decorator().getParent(). Here's my attempt: -------------------------------------------------------
0
928
by: Ben Finney | last post by:
Daniel <millerdev@gmail.comwrites: The fact that the 'property' function works as a decorator (in, as you point out, some cases only) is a happy accident. The 'property' function isn't *designed* to be used as a decorator, it just happens to work that way.
5
1137
by: Lee Harr | last post by:
I have a class with certain methods from which I want to select one at random, with weighting. The way I have done it is this .... import random def weight(value):
25
1464
by: Dmitry S. Makovey | last post by:
Hi, after hearing a lot about decorators and never actually using one I have decided to give it a try. My particular usecase is that I have class that acts as a proxy to other classes (i.e. passes messages along to those classes) however hand-coding this type of class is rather tedious, so I decided to use decorator for that. Can somebody tell me if what I'm doing is a potential shot-in-the-foot or am I on the right track? (Note, It's...
0
9671
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10854
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10558
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
9387
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
7794
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
6981
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5651
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
4459
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
3116
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.