468,768 Members | 1,415 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 468,768 developers. It's quick & easy.

Best practices for dynamically loading plugins at startup

Dear coders...

I'm working on an application that is supposed to support "plugins".
The idea is to use the plugins as packages like this:

Plugins/
__init__.py
Plugin1.py
Plugin2.py
Plugin3.py

When the application starts up I want to have these modules loaded
dynamically. Users can put their own plugin modules into the
Plugins/ directory and the application should know about it.

Since I don't know which plugins have been put into that directory
I cannot just "import Plugin1, Plugin2, Plugin3" in the "__init__.py".
So I need to find out the *.py there and load them during startup.
I could do that with a "walk" over that directory.

Each plugin is supposed to be a class derived from a general
"Plugin" superclass. I just don't know how to 'register' every
plugin. The main application needs to know which plugin classes
there are. On IRC I was recommended walking through all objects
and finding out if the class is a subclass of "Plugin". Another
person recommended using metaclasses to automatically register
the plugin in a global list.

Since I have only little real-life Python knowledge I wonder what the
best practice for this kind of problem is.

I looked at the "supybot" IRC bot to get an idea how plugins are handled
there. Unfortunately it was still a bit over my (python) head.

Regards
Christoph
--
~
~
~
".signature" [Modified] 3 lines --100%-- 3,41 All
Sep 25 '05 #1
5 2945
Christoph Haas napisał(a):
Since I don't know which plugins have been put into that directory
I cannot just "import Plugin1, Plugin2, Plugin3" in the "__init__.py".
So I need to find out the *.py there and load them during startup.
I could do that with a "walk" over that directory.


See entry for __import__ at
http://www.python.org/doc/2.3/lib/built-in-funcs.html

--
Jarek Zgoda
http://jpa.berlios.de/
Sep 25 '05 #2
Christoph Haas wrote:
Dear coders...

I'm working on an application that is supposed to support "plugins".
The idea is to use the plugins as packages like this:

Plugins/
__init__.py
Plugin1.py
Plugin2.py
Plugin3.py

When the application starts up I want to have these modules loaded
dynamically. Users can put their own plugin modules into the
Plugins/ directory and the application should know about it.

Since I don't know which plugins have been put into that directory
I cannot just "import Plugin1, Plugin2, Plugin3" in the "__init__.py".
So I need to find out the *.py there and load them during startup.
I could do that with a "walk" over that directory.

Each plugin is supposed to be a class derived from a general
"Plugin" superclass. I just don't know how to 'register' every
plugin. The main application needs to know which plugin classes
there are. On IRC I was recommended walking through all objects
and finding out if the class is a subclass of "Plugin". Another
person recommended using metaclasses to automatically register
the plugin in a global list.

Since I have only little real-life Python knowledge I wonder what the
best practice for this kind of problem is.

I looked at the "supybot" IRC bot to get an idea how plugins are handled
there. Unfortunately it was still a bit over my (python) head.


I recently came up against this exact problem. My preference is to have
the plugin writer call a method to register the plugins, as this allows
him the most control. Something along these lines:

class A_plugin(Plugin):
...

class Another_plugin(Plugin):
...

register( A_plugin )
register( Another_plugin, optional_custom_registration_parameters )
I also like the Mark Pilgrim approach of having the plugins follow a
strict naming convention, then searching for them with hasattr or by
grepping each plugin module's namespace. E.g., I have a build script
written in python that I use instead of the "make" utility; for each
target, I define a method called make_<target> in my plugin module, like
this:

function = "make_%s" % target
if hasattr(module, function):
getattr(module, function)()
else:
print >>sys.stderr, 'Unknown target "%s"' % target
Sep 26 '05 #3
I wrote this one:
--------------------------------------
def load_plugin(self, plugin, paths):
import imp
# check if we haven't loaded it already
try:
return sys.modules[plugin]
except KeyError:
pass
# ok, the load it
fp, filename, desc = imp.find_module(plugin, paths)
try:
mod = imp.load_module(plugin, fp, filename, desc)
finally:
if fp:
fp.close()
# instantiate and put into basket
clazz = mod.main(self.config)
if "input" in clazz.types:
self.inputs.append(clazz)
if "parser" in clazz.types:
self.parser.append(clazz)
if "output" in clazz.types:
self.outputs.append(clazz)
--------------------------------------
The imp module is the key:
http://docs.python.org/lib/module-imp.html

The parameters for the load module function, are found by look through
the directory. So my plugins had a convention:
They have a class called main, which is initialized with one argument,
the config object.

That is quite a simple plugin system. You can check out the whole thing
here:
https://developer.berlios.de/projects/feedisto/

Sep 26 '05 #4
On Sun, Sep 25, 2005 at 11:33:03PM -0400, Jeff Schwab wrote:
I recently came up against this exact problem. My preference is to have
the plugin writer call a method to register the plugins, as this allows
him the most control. Something along these lines:

class A_plugin(Plugin):
...

class Another_plugin(Plugin):
...

register( A_plugin )
register( Another_plugin, optional_custom_registration_parameters )


I like the idea. And "supybot" seems to do just that. What would the
"def register:" do? Maintain a global list of registered plugins?
I didn't like the idea of having global variables. Or would register
be a method of some "main" class? How would I access that?

Thanks to everybody else who posted ideas on my problem. I'm trying
all the proposals to get an idea of which approach works best.

Regards
Christoph
--
~
~
~
".signature" [Modified] 3 lines --100%-- 3,41 All
Sep 26 '05 #5
Christoph Haas wrote:
On Sun, Sep 25, 2005 at 11:33:03PM -0400, Jeff Schwab wrote:
I recently came up against this exact problem. My preference is to have
the plugin writer call a method to register the plugins, as this allows
him the most control. Something along these lines:

class A_plugin(Plugin):
...

class Another_plugin(Plugin):
...

register( A_plugin )
register( Another_plugin, optional_custom_registration_parameters )

I like the idea. And "supybot" seems to do just that. What would the
"def register:" do? Maintain a global list of registered plugins?
I didn't like the idea of having global variables. Or would register
be a method of some "main" class? How would I access that?


The registry clearly has to be shared between modules, so I think it's
best to make it a module-level variable. In the simple-minded code
below, I've stored it in the "plugins" module. A better choice might be
the module that defines your plugins' common base class, if they have one.
# BEGIN framework.py
if __name__ == "__main__":
import plugins
print plugins.plugin_registry
# END framework.py

# BEGIN plugins.py
# Define some plugins.

class A_plugin(object):
def __init__(self):
print "A_plugin()"

class Another_plugin(object):
def __init__(self):
print "Another_plugin()"

# Register the plugins.

import registry

plugin_registry = registry.Registry()
plugin_registry.register(A_plugin)
plugin_registry.register(Another_plugin)
# END plugins.py

# BEGIN registry.py
class Registry(object):
"Each instance may be used to store a set of objects."

def __init__(self):
"Initialize an empty registry."
self.registered_objects = set()

def register(self, o):
"Add an object to the registry."
self.registered_objects.add(o)

def __repr__(self):
return "Registry Contents:\n\t" + \
"\n\t".join(repr(o) for o in self.registered_objects)
# END registry.py
Sep 27 '05 #6

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

reply views Thread by David Levine | last post: by
10 posts views Thread by jojobar | last post: by
8 posts views Thread by Flavio | last post: by
reply views Thread by zhoujie | last post: by
reply views Thread by Marin | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.