Application Plugin Framework

[Ron]

Hello,

I'm attempting to develop a plugin framework for an application that I'm
working on. I wish to develop something in which all plugins exist in a
directory tree. The framework need only be given the root of the tree. The
framework then uses os.path.walk to search all for all files named
'plugin.pyc'. These are then loaded using imp.load_compiled(). They need
contain only one variable called 'plugin' which is a reference to an
instance of the plugin object. This is extrated from the loaded module
using getattr. After that, the plugins are easy to work with since they
implement a standard interface. Or so the theory goes. And this does work
fine provided the plugin is implemented entirely within that one file. If
there are support modules that the main plugin module imports, which exist
in the plugin's directory, then I get problems. The imports fail. I get
"ImportError: No module named <support module name>"

Here's PluginManager.py:
####################

import os
class PluginManager( object ):
def __init__( self, pluginDir ):
self._plugins = { }

os.path.walk( pluginDir, self._addPlugin, None )

def _addPlugin( self, arg, dirname, names ):
import imp

for filename in names:
fullFilename = os.path.join( dirname, filename )
if os.path.isfile( fullFilename ) and (filename == 'plugin.pyc'):
module = imp.load_compiled( 'plugin', fullFilename )
self._plugins[ plugin.name ] = getattr( module, 'plugin' )
return

PM = PluginManager( r'C:\Personal\SWDev\ModuleTest' ) # Root of the
plugin directory tree

print 'Plugin List: ', PM._plugins.keys()

for name,plugin in PM._plugins.iteritems():
plugin.doSomething( )

print 'done!'

#######################
My plugin.py file is in C:\Personal\SWDev\ModuleTest\Plugins\MyPlugin. It's
called plugin.pyc (which I compiled from the following source by importing
it into the interactive python shell.
#######################

import work
class MyPlugin( object ):
def __init__( self ):
self.name = 'MyPlugin'

def doSomething( self ):
work.foo( )
plugin = MyPlugin( )

#######################
Finally, work.py is in the same directory as plugin.py
#######################

def foo( ):
print 'foo called'

######################
Does anybody have any thoughts on how to get this, or something similar, to
work? I really just want to be able to load plugins into my app without
having to modify my app's source code, or modify some list of plugins. I
also would like to allow each plugin to have its own subdirectory so they
are easily installed or removed.

Thanks for your help.

[Harald Armin Massa]

Ron,

I'm attempting to develop a plugin framework for an application that I'm
working on. I wish to develop something in which all plugins exist in a
directory tree.

The PIL of the effbot is doing exactly this. (Python Image Library). I
know it, because I had to work around that dynamic for freezing it
with py2exe :)

www.effbot.org, search for PIL

There you can find how this problem was solved by the effbot. Guess
that should be good enough for mortals.

Harald

[Kay Schluehr]

Ron wrote:

Hello,

I'm attempting to develop a plugin framework for an application that I'm
working on. I wish to develop something in which all plugins exist in a
directory tree. The framework need only be given the root of the tree. The
framework then uses os.path.walk to search all for all files named
'plugin.pyc'. These are then loaded using imp.load_compiled(). They need
contain only one variable called 'plugin' which is a reference to an
instance of the plugin object. This is extrated from the loaded module
using getattr. After that, the plugins are easy to work with since they
implement a standard interface. Or so the theory goes. And this does work
fine provided the plugin is implemented entirely within that one file. If
there are support modules that the main plugin module imports, which exist
in the plugin's directory, then I get problems. The imports fail. I get
"ImportError: No module named <support module name>"

Here's PluginManager.py:
####################

import os
class PluginManager( object ):
def __init__( self, pluginDir ):
self._plugins = { }

os.path.walk( pluginDir, self._addPlugin, None )

def _addPlugin( self, arg, dirname, names ):
import imp

for filename in names:
fullFilename = os.path.join( dirname, filename )
if os.path.isfile( fullFilename ) and (filename == 'plugin.pyc'):
module = imp.load_compiled( 'plugin', fullFilename )
self._plugins[ plugin.name ] = getattr( module, 'plugin' )
return

PM = PluginManager( r'C:\Personal\SWDev\ModuleTest' ) # Root of the
plugin directory tree

print 'Plugin List: ', PM._plugins.keys()

for name,plugin in PM._plugins.iteritems():
plugin.doSomething( )

print 'done!'

#######################
My plugin.py file is in C:\Personal\SWDev\ModuleTest\Plugins\MyPlugin. It's
called plugin.pyc (which I compiled from the following source by importing
it into the interactive python shell.
#######################

import work
class MyPlugin( object ):
def __init__( self ):
self.name = 'MyPlugin'

def doSomething( self ):
work.foo( )
plugin = MyPlugin( )

#######################
Finally, work.py is in the same directory as plugin.py
#######################

def foo( ):
print 'foo called'

######################

Hi Ron,

the import directive does not lookup work.py in the same directory as
plugin.py. It searches through the directory of PluginManager.py and
otherwise searches through the directories of sys.path. The solution is
referencing work.py relative to the dir of the PluginManager.
MyFramework/
__init__.py
PluginManager.py
Plugins/
plugin.py
work.py
#######################
plugin.py
#######################
import MyFramework.Plugins.work as work

# do some stuff...

If import still causes trouble add the path of MyFramework to sys.path.
Regards,
Kay