Creating (rather) generic plugin framework?

[Edvard Majakari]

Hi,

My idea is to create a system working as follows: each module knows
path to plugin directory, and that directory contains modules which
may add hooks to some points in the code.

Inspired by http://www.python.org/pycon/2005/pap...onHooking.html

I would create a class like this:

class Printer:

def printit(self, msg):
stuff = self.beforePrintHook(msg)
if stuff:
msg = stuff
print msg
self.afterPrintHook(msg)

def beforePrintHook(self, msg): pass
def afterPrintHook(self, msg): pass

Now, in the spirit of py.test, I'd like API to be practically no API at all :)
moreover, deploying a plugin must consist simply of adding appropriate file to
plugins directory, nothing more, and removing it would uninstall it. The
plugin should be able to somehow result in all future invocations to
Printer.printit() call hooks specified in the plugin. Now, the plugin module
for class above /might/ be along the following lines (I'm thinking of stuff
here, so I don't know yet what would be the most appropriate way):

### a very simple plugin which uppercases all data fed to it.

extensions = {'Printer': 'PrinterHook'}

class PrinterHook:

def beforePrintHook(self, msg):
return msg.upper()
def afterPrintHook(self, msg):
print "Called afterPrintHook with msg %s" % msg
Now, I have a very rude (I think) implementation which has two methods, first
the one that loads plugin modules:

def find_plugins():
mods = [mod for mod in os.listdir(PLUGIN_DIR) if mod.endswith('.py')]

# for each module in plugin dir, import the module and setup hooks. Hooks
# must have equal method names in plugin module as in the original class.
for mod in mods:
name = os.path.splitext(mod)[0]
fobj, fpath, desc = imp.find_module(os.path.join(PLUGIN_DIR, name))
module = imp.load_module(name, fobj, fpath, desc)
set_hooks(module)

....then the other that is responsible for setting up hooks

def set_hooks(mod):
# mod.extensions has "base" class names as keys, (hook, priority) as
# values
for k, hook in mod.extensions.items():
# get class object
hook_cls = mod.__dict__[hook]

try:
base = globals()[k]
except KeyError:
print "No such class to insert hooks to:", k
else:
for item in base.__dict__:
if item.endswith('Hook'):
# Override original (no-op) hook method
# uhh.. kludgety kludge
base.__dict__[item] = hook_cls.__dict__[item]

now, my plugindemo.py just does as follows:

find_plugins()
p = Printer()
p.printit('Hello, world!')

which prints

$ python2.4 plugindemo.py
HELLO, WORLD!
Called afterPrintHook with msg HELLO, WORLD!

But hey, this has many downsides. First off, mechanism doesn't support
arbitrary namespaces. Here, class identifier in the plugin must be as it is
seen from the module which calls the plugin (not a problem for me, but could
be more elegant; probably a mapping between logical class identifiers and
actual class names, hmm?). Second, if one wants to use many hooks (with
priority for conflicts), it is not possible now; set_hooks always overrides
potentially existing hooks. And probably many other problems that are not
obvious to me, but for the simple purpose I have in mind, it seems to work.

This is the first plugin system in Python I'm writing, so I can be a way off
the correct path..

[bruno at modulix]

Edvard Majakari wrote:

Hi,

My idea is to create a system working as follows: each module knows
path to plugin directory, and that directory contains modules which
may add hooks to some points in the code.

Inspired by http://www.python.org/pycon/2005/pap...onHooking.html

I would create a class like this:

class Printer:

def printit(self, msg):
stuff = self.beforePrintHook(msg)
if stuff:
msg = stuff
print msg
self.afterPrintHook(msg)

def beforePrintHook(self, msg): pass
def afterPrintHook(self, msg): pass

Now, in the spirit of py.test, I'd like API to be practically no API at all :)
moreover, deploying a plugin must consist simply of adding appropriate file to
plugins directory, nothing more, and removing it would uninstall it. The
plugin should be able to somehow result in all future invocations to
Printer.printit() call hooks specified in the plugin. Now, the plugin module
for class above /might/ be along the following lines (I'm thinking of stuff
here, so I don't know yet what would be the most appropriate way):

### a very simple plugin which uppercases all data fed to it.

extensions = {'Printer': 'PrinterHook'}

class PrinterHook:

def beforePrintHook(self, msg):
return msg.upper()
def afterPrintHook(self, msg):
print "Called afterPrintHook with msg %s" % msg
Now, I have a very rude (I think) implementation which has two methods, first
the one that loads plugin modules:

(snip code)

But hey, this has many downsides. First off, mechanism doesn't support
arbitrary namespaces. Here, class identifier in the plugin must be as it is
seen from the module which calls the plugin (not a problem for me, but could
be more elegant; probably a mapping between logical class identifiers and
actual class names, hmm?). Second, if one wants to use many hooks (with
priority for conflicts), it is not possible now; set_hooks always overrides
potentially existing hooks. And probably many other problems that are not
obvious to me, but for the simple purpose I have in mind, it seems to work.
Just a couple of ideas:
- using decorators for plugin hooks ? ie:

import hooks

class Whatever(anything):
@hooks.hook(for='Printer.beforePrintHook',priority =42)
def my_function_with_a_long_name(self, *args, **kw):
pass

The decorator would take care of "registering" the hook where relevant,
ie, storing it in a class attribute of the hooked class ?

which leads to:

- in the hooked class, use a dict class attribute for hooks:

from hooks import run_hooks

class Printer
# will be filled (and could even be created)
# by the @hook decorator
_hooks = {}

def print(self, msg):
# run_hooks will take care of selecting appropriate
# hooks (by looking up the class attribute _hooks)
# and running'em in order
msg = run_hooks(self, 'Printer.beforePrintHook', msg)
print msg
run_hooks(self, 'Printer.afterPrintHook', msg)

My 2 cents... I don't even know if this can be implemented (but I don't
see why it couldn't).
This is the first plugin system in Python I'm writing, so I can be a way off
the correct path..

<aol>

--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"

[bruno at modulix]

Edvard Majakari wrote:

Hi,

My idea is to create a system working as follows: each module knows
path to plugin directory, and that directory contains modules which
may add hooks to some points in the code.

Inspired by http://www.python.org/pycon/2005/pap...onHooking.html

I would create a class like this:

class Printer:

def printit(self, msg):
stuff = self.beforePrintHook(msg)
if stuff:
msg = stuff
print msg
self.afterPrintHook(msg)

def beforePrintHook(self, msg): pass
def afterPrintHook(self, msg): pass

Now, in the spirit of py.test, I'd like API to be practically no API at all :)
moreover, deploying a plugin must consist simply of adding appropriate file to
plugins directory, nothing more, and removing it would uninstall it. The
plugin should be able to somehow result in all future invocations to
Printer.printit() call hooks specified in the plugin. Now, the plugin module
for class above /might/ be along the following lines (I'm thinking of stuff
here, so I don't know yet what would be the most appropriate way):

### a very simple plugin which uppercases all data fed to it.

extensions = {'Printer': 'PrinterHook'}

class PrinterHook:

def beforePrintHook(self, msg):
return msg.upper()
def afterPrintHook(self, msg):
print "Called afterPrintHook with msg %s" % msg
Now, I have a very rude (I think) implementation which has two methods, first
the one that loads plugin modules:

(snip code)

But hey, this has many downsides. First off, mechanism doesn't support
arbitrary namespaces. Here, class identifier in the plugin must be as it is
seen from the module which calls the plugin (not a problem for me, but could
be more elegant; probably a mapping between logical class identifiers and
actual class names, hmm?). Second, if one wants to use many hooks (with
priority for conflicts), it is not possible now; set_hooks always overrides
potentially existing hooks. And probably many other problems that are not
obvious to me, but for the simple purpose I have in mind, it seems to work.
Just a couple of ideas:
- using decorators for plugin hooks ? ie:

import hooks

class Whatever(anything):
@hooks.hook(for='Printer.beforePrintHook',priority =42)
def my_function_with_a_long_name(self, *args, **kw):
pass

The decorator would take care of "registering" the hook where relevant,
ie, storing it in a class attribute of the hooked class ?

which leads to:

- in the hooked class, use a dict class attribute for hooks:

from hooks import run_hooks

class Printer
# will be filled (and could even be created)
# by the @hook decorator
_hooks = {}

def print(self, msg):
# run_hooks will take care of selecting appropriate
# hooks (by looking up the class attribute _hooks)
# and running'em in order
msg = run_hooks(self, 'Printer.beforePrintHook', msg)
print msg
run_hooks(self, 'Printer.afterPrintHook', msg)

My 2 cents... I don't even know if this can be implemented (but I don't
see why it couldn't).
This is the first plugin system in Python I'm writing, so I can be a way off
the correct path..

<aol>

--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"