473,657 Members | 2,535 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Metaclass to make all methods of a class thread-safe

Hi,
I've developed the Metaclass below, because I needed a way
to make a bunch of classes thread-safe.
I didn't want to change every method of the class by adding
lock.aqcuire(). .lock.release() around the existing code.
So I made a metaclass that essentially replaces every method
of a class with a 'wrapper' method, that does the locking,
invocation, unlocking.

Is this the right approach? It seems to work fine. But I have
very little experience with metaclass programming, so I'd like
to hear some feedback.

Thanks !!

--Irmen de Jong.
# ---- source below ----

from types import FunctionType
from threading import RLock

class ThreadSafeMetho dsMetaClass(typ e):
def __new__(meta, name, bases, dict):
meta.convert_me thods(dict)
return super(ThreadSaf eMethodsMetaCla ss, meta).__new__(m eta, name, bases, dict)

def makeThreadsafeM ethod(func):
def threadsafemetho d(self, *args, **kwargs):
self._monitor_l ockObj.acquire( )
print ">>got lock"
try:
return func(self, *args, **kwargs)
finally:
self._monitor_l ockObj.release( )
print "<<released lock"
return threadsafemetho d
makeThreadsafeM ethod = staticmethod(ma keThreadsafeMet hod)

def convert_methods (cls, dict):
methods=[ v for k,v in dict.iteritems( ) if isinstance(v, FunctionType) ]
for m in methods:
dict[m.__name__]=cls.makeThread safeMethod(m)
dict["_monitor_lockO bj"] = RLock()

convert_methods = classmethod(con vert_methods)
class MyClass(object) :
__metaclass__=T hreadSafeMethod sMetaClass

def __init__(self):
print "init!"
def method(self, a1, a2):
print a1,a2

m=MyClass()
m.method("irmen ",42)
Jul 18 '05 #1
5 2259
Irmen de Jong <irmen@-nospam-remove-this-xs4all.nl> wrote in message news:<41******* *************** *@news.xs4all.n l>...
Hi,
I've developed the Metaclass below, because I needed a way
to make a bunch of classes thread-safe.
I didn't want to change every method of the class by adding
lock.aqcuire(). .lock.release() around the existing code.
So I made a metaclass that essentially replaces every method
of a class with a 'wrapper' method, that does the locking,
invocation, unlocking.

Is this the right approach? It seems to work fine. But I have
very little experience with metaclass programming, so I'd like
to hear some feedback.

Thanks !!

--Irmen de Jong.


Well, it looks okay, but consider the following:

1. you have a (metaclass) static method and a (metaclass) classmethod
which could be replaced by simple functions external to the metaclass;
this would make the code much easier to read and to understand; I don't
buy the argument that they should logically stay in the metaclass, it
is enough if they stay in the same module of the metaclass, not inside it;

2. the metaclass will automagically wrap even methods of subclasses, without
you knowing it; consider using a naming convention (es. only methods
starting with "t_" are magically wrapped); if still
you want the magic, consider defining a name convention such as methods
starting with a given prefix are NOT magically wrapped;

4. built-in methods and all the objects which are non instances of FunctionType
will be not wrapped; you may want this or not;

5. consider using decorators to wrap the methods you want to be
thread safe: they a more esplicit and easier to understand solution;
also in this way you will avoid the (possible) issue of metaclass
conflicts.

6. However, if you want to enhance a code which is already written
with a minimal change of the source code, the metaclass is the
simplest solution indeed.

Just my 0.02c,

Michele Simionato
Jul 18 '05 #2
Michele Simionato wrote:
Irmen de Jong <irmen@-nospam-remove-this-xs4all.nl> wrote in message news:<41******* *************** *@news.xs4all.n l>... [...]
Is this the right approach? It seems to work fine. But I have
very little experience with metaclass programming, so I'd like
to hear some feedback.

Thanks !!

--Irmen de Jong.

Well, it looks okay, but consider the following:

1. you have a (metaclass) static method and a (metaclass) classmethod
which could be replaced by simple functions external to the metaclass;
this would make the code much easier to read and to understand; I don't
buy the argument that they should logically stay in the metaclass, it
is enough if they stay in the same module of the metaclass, not inside it;


Agreed, I was just modeling it after some example metaclass code that I found
on ASPN cookbook. The metaclass itself will only be a few lines then :-)
2. the metaclass will automagically wrap even methods of subclasses, without
you knowing it; consider using a naming convention (es. only methods
starting with "t_" are magically wrapped); if still
you want the magic, consider defining a name convention such as methods
starting with a given prefix are NOT magically wrapped;
This is good advice. I started by not wrapping methods with '__' prefix...
4. built-in methods and all the objects which are non instances of FunctionType
will be not wrapped; you may want this or not;
Built-in methods such as?
About the other objects: I only cared about wrapping class methods.
Shouldn't I have / should I use something else than FunctionType?

There is one thing though; methods that you're accessing trough the
class's __dict__ (which is what the meta class is doing, right?)
are of type <function>, rathar than <instancemethod > which I expected:
class A: .... def meth(self): pass
.... type(A.meth) <type 'instancemethod '> type(A.__dict__['meth']) <type 'function'>


Why is this?

5. consider using decorators to wrap the methods you want to be
thread safe: they a more esplicit and easier to understand solution;
also in this way you will avoid the (possible) issue of metaclass
conflicts.
No decorators, nooo sir, it must work on Python 2.3 too :)
6. However, if you want to enhance a code which is already written
with a minimal change of the source code, the metaclass is the
simplest solution indeed.


Thanks for your comments, Michele.

--Irmen
Jul 18 '05 #3
Irmen de Jong wrote:

There is one thing though; methods that you're accessing trough the
class's __dict__ (which is what the meta class is doing, right?)
are of type <function>, rathar than <instancemethod > which I expected:
class A: ... def meth(self): pass
... type(A.meth) <type 'instancemethod '> type(A.__dict__['meth']) <type 'function'>


Why is this?

I don't know for certain, but it occurs to me that the object returned
by A.meth will insert A in front of all other arguments to the method,
while the object returned by A.__dict__['meth'] does not do that. Logic
seems to suggest that the instancemethod type performs this
argument-list-mangling and then passes the results on to the
FunctionType object contained in __dict__ -- that is, instancemethod is
essentially a function wrapper that provides the standard method-call
translations for class instances.

Jeff Shannon
Technician/Programmer
Credit International

Jul 18 '05 #4
Irmen de Jong <irmen@-nospam-remove-this-xs4all.nl> wrote in message news:<41******* **************@ news.xs4all.nl> ...
Built-in methods such as?
This is instructive:
class C(object): pass .... for attr in dir(C): print attr, type(getattr(C, attr)) ....
__class__ <type 'type'>
__delattr__ <type 'wrapper_descri ptor'>
__dict__ <type 'dictproxy'>
__doc__ <type 'NoneType'>
__getattribute_ _ <type 'wrapper_descri ptor'>
__hash__ <type 'wrapper_descri ptor'>
__init__ <type 'wrapper_descri ptor'>
__module__ <type 'str'>
__new__ <type 'builtin_functi on_or_method'>
__reduce__ <type 'method_descrip tor'>
__reduce_ex__ <type 'method_descrip tor'>
__repr__ <type 'wrapper_descri ptor'>
__setattr__ <type 'wrapper_descri ptor'>
__str__ <type 'wrapper_descri ptor'>
__weakref__ <type 'getset_descrip tor'>
For instance object.__new__ is a "builtin_functi on_or_method"
About the other objects: I only cared about wrapping class methods.
Shouldn't I have / should I use something else than FunctionType?
Yes, if you want to wrap staticmethods/classmethods and custom descriptors.
There is one thing though; methods that you're accessing trough the
class's __dict__ (which is what the meta class is doing, right?)
are of type <function>, rathar than <instancemethod > which I expected:
>>> class A: ... def meth(self): pass
... >>> type(A.meth) <type 'instancemethod '> >>> type(A.__dict__['meth']) <type 'function'> >>>


Why is this?


Google for Raymond Hettinger essay on descriptors and you will have
the answer. There is no real difference between functions and methods,
they are all descriptors and can be converted each other. For instance,
this converts a function to a bound method:
def f(self): pass .... f.__get__(C(),C ) # magically converts the function to a bound method <bound method C.f of <__main__.C object at 0x403b7f8c>> f.__get__ <method-wrapper object at 0x403b7f4c>

You should really look at Raymond's essay to understand what is going on
under the cover when you write
C.m = f # magically convert the function to a bound method
C().m

<bound method C.f of <__main__.C object at 0x403b7f0c>>

Michele Simionato
Jul 18 '05 #5
Michele Simionato wrote:

[...interesting stuff about descriptors and method types...]

Thanks for this information, I have something to study
if I want to make my next metaclass :)

--Irmen
Jul 18 '05 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
1814
by: Jp Calderone | last post by:
Due to some bizarre constraints placed on me, I've written the following metaclass: import types def remove(t, o): return tuple() class BizarreMetaclass(type): def __new__(klass, name, bases, attrs):
3
1685
by: Fernando Rodriguez | last post by:
Hi, I'm having trouble with a metaclass suposed to check the method signature of its classes. Here's the metaclass: class MetaChecker(type): def __new__(cls, name, bases, attribs): for name, value in attribs.iteritems():
4
1571
by: Paul Morrow | last post by:
One of the beautiful things about Python is its clear, minimal syntax. So we must resist adding new syntax to the language, especially where there is a reasonable alternative. I believe that Stefen Eischet's suggestion for automatically determining a method's type (class/instance/static) from the name of its first formal parameter is a reasonable alternative to any/all of the decorator syntax proposals. Just as the Python system...
33
2517
by: Jacek Generowicz | last post by:
I would like to write a metaclass which would allow me to overload names in the definition of its instances, like this class Foo(object): __metaclass__ = OverloadingClass att = 1 att = 3
2
1938
by: zipher | last post by:
After searching through comp.lang.python and the web regarding metaclasses, I could not find an example for customing classes using metaclass parameters. I want to be able to create a class at runtime by calling some function or 'meta-constructor' which returns a customized class and sets a class attribute according a given parameter. Ideally, I'd be able to do something like:
39
2914
by: Nicolas Fleury | last post by:
In the following example: class MyMetaclass(type): pass class MyBaseType(object): __metaclass__ = MyMetaclass class MyType(MyBaseType): x = 4 y = 5 z = 6 Is there any way to modify MyMetaclass to keep the order of x,y,z somewhere?
14
2026
by: Pedro Werneck | last post by:
Hi I have a class A, with metaclass M_A, and class B, subclass of A, with metaclass M_B, subclass of M_A. A class C, subclass of B must have M_B or a subclass of it as metaclass, but what if I need to 'disable' the code in M_B on C ? The correct way to do that seems to be with a M_C metaclass, subclass of M_B, implementing but not calling parent class methods, or calling 'type' methods.
9
1654
by: Christian Eder | last post by:
Hi, I think I have discovered a problem in context of metaclasses and multiple inheritance in python 2.4, which I could finally reduce to a simple example: Look at following code: class M_A (type) :
4
3306
by: Pedro Werneck | last post by:
Hi all I noticed something strange here while explaining decorators to someone. Not any real use code, but I think it's worth mentioning. When I access a class attribute, on a class with a custom metaclass with a __getattribute__ method, the method is used when acessing some attribute directly with the class object, but not when you do it from the instance.
1
3460
by: Matthew Wilson | last post by:
I have been experimenting with metaclasses lately. It seems possible to define a metaclass by either subclassing type and then either redefining __init__ or __new__. Here's the signature for __init__: def __init__(cls, name, bases, d): and here's __new__:
0
8397
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8310
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
8827
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
8732
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
8503
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
7333
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6167
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5632
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4315
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.