473,387 Members | 1,517 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,387 software developers and data experts.

Metaclass to make all methods of a class thread-safe

Hi,
I've developed the Metaclass below, because I needed a way
to make a bunch of classes thread-safe.
I didn't want to change every method of the class by adding
lock.aqcuire()..lock.release() around the existing code.
So I made a metaclass that essentially replaces every method
of a class with a 'wrapper' method, that does the locking,
invocation, unlocking.

Is this the right approach? It seems to work fine. But I have
very little experience with metaclass programming, so I'd like
to hear some feedback.

Thanks !!

--Irmen de Jong.
# ---- source below ----

from types import FunctionType
from threading import RLock

class ThreadSafeMethodsMetaClass(type):
def __new__(meta, name, bases, dict):
meta.convert_methods(dict)
return super(ThreadSafeMethodsMetaClass, meta).__new__(meta, name, bases, dict)

def makeThreadsafeMethod(func):
def threadsafemethod(self, *args, **kwargs):
self._monitor_lockObj.acquire()
print ">>got lock"
try:
return func(self, *args, **kwargs)
finally:
self._monitor_lockObj.release()
print "<<released lock"
return threadsafemethod
makeThreadsafeMethod = staticmethod(makeThreadsafeMethod)

def convert_methods(cls, dict):
methods=[ v for k,v in dict.iteritems() if isinstance(v, FunctionType) ]
for m in methods:
dict[m.__name__]=cls.makeThreadsafeMethod(m)
dict["_monitor_lockObj"] = RLock()

convert_methods = classmethod(convert_methods)
class MyClass(object):
__metaclass__=ThreadSafeMethodsMetaClass

def __init__(self):
print "init!"
def method(self, a1, a2):
print a1,a2

m=MyClass()
m.method("irmen",42)
Jul 18 '05 #1
5 2236
Irmen de Jong <irmen@-nospam-remove-this-xs4all.nl> wrote in message news:<41***********************@news.xs4all.nl>...
Hi,
I've developed the Metaclass below, because I needed a way
to make a bunch of classes thread-safe.
I didn't want to change every method of the class by adding
lock.aqcuire()..lock.release() around the existing code.
So I made a metaclass that essentially replaces every method
of a class with a 'wrapper' method, that does the locking,
invocation, unlocking.

Is this the right approach? It seems to work fine. But I have
very little experience with metaclass programming, so I'd like
to hear some feedback.

Thanks !!

--Irmen de Jong.


Well, it looks okay, but consider the following:

1. you have a (metaclass) static method and a (metaclass) classmethod
which could be replaced by simple functions external to the metaclass;
this would make the code much easier to read and to understand; I don't
buy the argument that they should logically stay in the metaclass, it
is enough if they stay in the same module of the metaclass, not inside it;

2. the metaclass will automagically wrap even methods of subclasses, without
you knowing it; consider using a naming convention (es. only methods
starting with "t_" are magically wrapped); if still
you want the magic, consider defining a name convention such as methods
starting with a given prefix are NOT magically wrapped;

4. built-in methods and all the objects which are non instances of FunctionType
will be not wrapped; you may want this or not;

5. consider using decorators to wrap the methods you want to be
thread safe: they a more esplicit and easier to understand solution;
also in this way you will avoid the (possible) issue of metaclass
conflicts.

6. However, if you want to enhance a code which is already written
with a minimal change of the source code, the metaclass is the
simplest solution indeed.

Just my 0.02c,

Michele Simionato
Jul 18 '05 #2
Michele Simionato wrote:
Irmen de Jong <irmen@-nospam-remove-this-xs4all.nl> wrote in message news:<41***********************@news.xs4all.nl>... [...]
Is this the right approach? It seems to work fine. But I have
very little experience with metaclass programming, so I'd like
to hear some feedback.

Thanks !!

--Irmen de Jong.

Well, it looks okay, but consider the following:

1. you have a (metaclass) static method and a (metaclass) classmethod
which could be replaced by simple functions external to the metaclass;
this would make the code much easier to read and to understand; I don't
buy the argument that they should logically stay in the metaclass, it
is enough if they stay in the same module of the metaclass, not inside it;


Agreed, I was just modeling it after some example metaclass code that I found
on ASPN cookbook. The metaclass itself will only be a few lines then :-)
2. the metaclass will automagically wrap even methods of subclasses, without
you knowing it; consider using a naming convention (es. only methods
starting with "t_" are magically wrapped); if still
you want the magic, consider defining a name convention such as methods
starting with a given prefix are NOT magically wrapped;
This is good advice. I started by not wrapping methods with '__' prefix...
4. built-in methods and all the objects which are non instances of FunctionType
will be not wrapped; you may want this or not;
Built-in methods such as?
About the other objects: I only cared about wrapping class methods.
Shouldn't I have / should I use something else than FunctionType?

There is one thing though; methods that you're accessing trough the
class's __dict__ (which is what the meta class is doing, right?)
are of type <function>, rathar than <instancemethod> which I expected:
class A: .... def meth(self): pass
.... type(A.meth) <type 'instancemethod'> type(A.__dict__['meth']) <type 'function'>


Why is this?

5. consider using decorators to wrap the methods you want to be
thread safe: they a more esplicit and easier to understand solution;
also in this way you will avoid the (possible) issue of metaclass
conflicts.
No decorators, nooo sir, it must work on Python 2.3 too :)
6. However, if you want to enhance a code which is already written
with a minimal change of the source code, the metaclass is the
simplest solution indeed.


Thanks for your comments, Michele.

--Irmen
Jul 18 '05 #3
Irmen de Jong wrote:

There is one thing though; methods that you're accessing trough the
class's __dict__ (which is what the meta class is doing, right?)
are of type <function>, rathar than <instancemethod> which I expected:
class A: ... def meth(self): pass
... type(A.meth) <type 'instancemethod'> type(A.__dict__['meth']) <type 'function'>


Why is this?

I don't know for certain, but it occurs to me that the object returned
by A.meth will insert A in front of all other arguments to the method,
while the object returned by A.__dict__['meth'] does not do that. Logic
seems to suggest that the instancemethod type performs this
argument-list-mangling and then passes the results on to the
FunctionType object contained in __dict__ -- that is, instancemethod is
essentially a function wrapper that provides the standard method-call
translations for class instances.

Jeff Shannon
Technician/Programmer
Credit International

Jul 18 '05 #4
Irmen de Jong <irmen@-nospam-remove-this-xs4all.nl> wrote in message news:<41*********************@news.xs4all.nl>...
Built-in methods such as?
This is instructive:
class C(object): pass .... for attr in dir(C): print attr, type(getattr(C,attr)) ....
__class__ <type 'type'>
__delattr__ <type 'wrapper_descriptor'>
__dict__ <type 'dictproxy'>
__doc__ <type 'NoneType'>
__getattribute__ <type 'wrapper_descriptor'>
__hash__ <type 'wrapper_descriptor'>
__init__ <type 'wrapper_descriptor'>
__module__ <type 'str'>
__new__ <type 'builtin_function_or_method'>
__reduce__ <type 'method_descriptor'>
__reduce_ex__ <type 'method_descriptor'>
__repr__ <type 'wrapper_descriptor'>
__setattr__ <type 'wrapper_descriptor'>
__str__ <type 'wrapper_descriptor'>
__weakref__ <type 'getset_descriptor'>
For instance object.__new__ is a "builtin_function_or_method"
About the other objects: I only cared about wrapping class methods.
Shouldn't I have / should I use something else than FunctionType?
Yes, if you want to wrap staticmethods/classmethods and custom descriptors.
There is one thing though; methods that you're accessing trough the
class's __dict__ (which is what the meta class is doing, right?)
are of type <function>, rathar than <instancemethod> which I expected:
>>> class A: ... def meth(self): pass
... >>> type(A.meth) <type 'instancemethod'> >>> type(A.__dict__['meth']) <type 'function'> >>>


Why is this?


Google for Raymond Hettinger essay on descriptors and you will have
the answer. There is no real difference between functions and methods,
they are all descriptors and can be converted each other. For instance,
this converts a function to a bound method:
def f(self): pass .... f.__get__(C(),C) # magically converts the function to a bound method <bound method C.f of <__main__.C object at 0x403b7f8c>> f.__get__ <method-wrapper object at 0x403b7f4c>

You should really look at Raymond's essay to understand what is going on
under the cover when you write
C.m = f # magically convert the function to a bound method
C().m

<bound method C.f of <__main__.C object at 0x403b7f0c>>

Michele Simionato
Jul 18 '05 #5
Michele Simionato wrote:

[...interesting stuff about descriptors and method types...]

Thanks for this information, I have something to study
if I want to make my next metaclass :)

--Irmen
Jul 18 '05 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Jp Calderone | last post by:
Due to some bizarre constraints placed on me, I've written the following metaclass: import types def remove(t, o): return tuple() class BizarreMetaclass(type): def __new__(klass, name,...
3
by: Fernando Rodriguez | last post by:
Hi, I'm having trouble with a metaclass suposed to check the method signature of its classes. Here's the metaclass: class MetaChecker(type): def __new__(cls, name, bases, attribs): for...
4
by: Paul Morrow | last post by:
One of the beautiful things about Python is its clear, minimal syntax. So we must resist adding new syntax to the language, especially where there is a reasonable alternative. I believe that...
33
by: Jacek Generowicz | last post by:
I would like to write a metaclass which would allow me to overload names in the definition of its instances, like this class Foo(object): __metaclass__ = OverloadingClass att = 1 att = 3
2
by: zipher | last post by:
After searching through comp.lang.python and the web regarding metaclasses, I could not find an example for customing classes using metaclass parameters. I want to be able to create a class at...
39
by: Nicolas Fleury | last post by:
In the following example: class MyMetaclass(type): pass class MyBaseType(object): __metaclass__ = MyMetaclass class MyType(MyBaseType): x = 4 y = 5 z = 6 Is there any way to modify...
14
by: Pedro Werneck | last post by:
Hi I have a class A, with metaclass M_A, and class B, subclass of A, with metaclass M_B, subclass of M_A. A class C, subclass of B must have M_B or a subclass of it as metaclass, but what if...
9
by: Christian Eder | last post by:
Hi, I think I have discovered a problem in context of metaclasses and multiple inheritance in python 2.4, which I could finally reduce to a simple example: Look at following code: class...
4
by: Pedro Werneck | last post by:
Hi all I noticed something strange here while explaining decorators to someone. Not any real use code, but I think it's worth mentioning. When I access a class attribute, on a class with a...
1
by: Matthew Wilson | last post by:
I have been experimenting with metaclasses lately. It seems possible to define a metaclass by either subclassing type and then either redefining __init__ or __new__. Here's the signature for...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.