Plugin system

[Reinhold Birkenfeld]

Hello,

another philosophical question:

How would you implement a plugin system (for a web-based application)?

Conditions are:
- One plugin can have one or many "parts" (or "subplugins ", or whatever)
- Each subplugin has a type
- The rest of the plugin interface is dependent on the type

I have one possible solution in mind, but I want to wait for your
opinion before revealing it.

Reinhold

--
[Windows ist wie] die Bahn: Man muss sich um nichts kuemmern, zahlt fuer
jede Kleinigkeit einen Aufpreis, der Service ist mies, Fremde koennen
jederzeit einsteigen, es ist unflexibel und zu allen anderen Verkehrs-
mitteln inkompatibel. -- Florian Diesch in dcoulm

[Irmen de Jong]

Reinhold Birkenfeld wrote:

How would you implement a plugin system (for a web-based application)?

This is so general a question that I cannot think of anything concrete to answer.
Please ask again, with more detail about what you want/need/have in mind.

--Irmen

[Reinhold Birkenfeld]

Irmen de Jong wrote:

Reinhold Birkenfeld wrote:

How would you implement a plugin system (for a web-based application)?

This is so general a question that I cannot think of anything concrete to answer.
Please ask again, with more detail about what you want/need/have in mind.

Okay, the question is too open: My focus is on how you would structure
the plugin code (directories, files) and how you would write code to
load these plugins.

Reinhold

--
[Windows ist wie] die Bahn: Man muss sich um nichts kuemmern, zahlt fuer
jede Kleinigkeit einen Aufpreis, der Service ist mies, Fremde koennen
jederzeit einsteigen, es ist unflexibel und zu allen anderen Verkehrs-
mitteln inkompatibel. -- Florian Diesch in dcoulm

[Peter Hansen]

Reinhold Birkenfeld wrote:

Irmen de Jong wrote:

Reinhold Birkenfeld wrote:

How would you implement a plugin system (for a web-based application)?

This is so general a question that I cannot think of anything concrete to answer.
Please ask again, with more detail about what you want/need/have in mind.

Okay, the question is too open: My focus is on how you would structure
the plugin code (directories, files) and how you would write code to
load these plugins.

And _this_ sounds like a combination of something that's more
a design issue (the structure) and a trivial implementation
detail (trivial in Python, anyway).

I'll try anyway, as a first test case: put all plugins in a
standard folder, with nothing else present in that folder.
They can be individual modules or packages as you will.
The code to load the plugin consists of an __import__...
and not much else.

Maybe this feedback will help you refine the question further...

-Peter

[Josiah Carlson]

Reinhold Birkenfeld <re************ ************@wo lke7.net> wrote:

Irmen de Jong wrote:

Reinhold Birkenfeld wrote:

How would you implement a plugin system (for a web-based application)?

This is so general a question that I cannot think of anything concrete to answer.
Please ask again, with more detail about what you want/need/have in mind.

Okay, the question is too open: My focus is on how you would structure
the plugin code (directories, files) and how you would write code to
load these plugins.

...Have a place for plugins
app/plugins

...Load the plugins on startup
import traceback
import os
for fn in os.listdir('plu gins'):
if fn[:1] != '_' and fn.split('.')[-1] in ('py', 'pyw'):
try:
globals()[fn] = \
__import__('plu gins.%s'%fn, globals(), locals(), [fn])
except:
traceback.print _exc()

Etcetera. May not be the best, but it will get the job done.

- Josiah

[Dan Perl]

"Peter Hansen" <pe***@engcorp. com> wrote in message
news:Rp******** ************@po wergate.ca...

Reinhold Birkenfeld wrote:

Irmen de Jong wrote:

Reinhold Birkenfeld wrote:

How would you implement a plugin system (for a web-based application)?

This is so general a question that I cannot think of anything concrete to
answer.
Please ask again, with more detail about what you want/need/have in mind.
Okay, the question is too open: My focus is on how you would structure
the plugin code (directories, files) and how you would write code to
load these plugins.

And _this_ sounds like a combination of something that's more
a design issue (the structure) and a trivial implementation
detail (trivial in Python, anyway).

I'll try anyway, as a first test case: put all plugins in a
standard folder, with nothing else present in that folder.
They can be individual modules or packages as you will.
The code to load the plugin consists of an __import__...
and not much else.

I have done something similar, with a few differences. I am using a
convention that all the plugins are implemented as a class with a specific
name ("Handler" in my case), each plugin in its own module (so each Handler
class is scoped by its module). I import all the modules in the directory
dedicated to plugins and I do the following checks: the module contains a
class Handler, the module is not the one implementing the base class itself
(bit of a hack, I admit), and the Handler class is a subclass of the base
class (use isinstance). The checks allow to have also other modules in the
plugins directory that do not actually implement a plugin (for instance, I
have a mixin class in that same directory).

Because you have both plugins and sub-plugins you may want to use some
checks like mine, so you have both plugins and sub-plugins in the same
directory, but you select only the plugin classes. That's if I understand
your requirements correctly.

Dan
Maybe this feedback will help you refine the question further...

-Peter

[Greg Copeland]

On Sat, 30 Oct 2004 07:58:27 -0700, Josiah Carlson wrote:

Reinhold Birkenfeld <re************ ************@wo lke7.net> wrote:

Irmen de Jong wrote:

> Reinhold Birkenfeld wrote:
>
>> How would you implement a plugin system (for a web-based application)?
>
> This is so general a question that I cannot think of anything concrete to answer.
> Please ask again, with more detail about what you want/need/have in mind.

Okay, the question is too open: My focus is on how you would structure
the plugin code (directories, files) and how you would write code to
load these plugins.

...Have a place for plugins
app/plugins

...Load the plugins on startup
import traceback
import os
for fn in os.listdir('plu gins'):
if fn[:1] != '_' and fn.split('.')[-1] in ('py', 'pyw'):
try:
globals()[fn] = \
__import__('plu gins.%s'%fn, globals(), locals(), [fn])
except:
traceback.print _exc()

Etcetera. May not be the best, but it will get the job done.

- Josiah

Just FYI, dynamically loaded plugins will not be caught by tools like
py2exe. Another mechanism, which may be more py2exe friendly, is to make
a tool which dynamically creates a python file, which lists all of the
imports. That way, you import the dynamically created file, but once it's
created, you immediately leverage the file for use with py2exe, which can
now follow all of the imports.

Of course, if you never plan on being a py2exe user, you can happily
ignore my ramblings. ;)

Cheers,

Greg

[Dan Perl]

"Dan Perl" <da*****@rogers .com> wrote in message
news:CJ******** ************@ro gers.com...

"Peter Hansen" <pe***@engcorp. com> wrote in message
news:Rp******** ************@po wergate.ca...

Reinhold Birkenfeld wrote:

Irmen de Jong wrote:
Reinhold Birkenfeld wrote:

>How would you implement a plugin system (for a web-based application)?

This is so general a question that I cannot think of anything concrete
to answer.
Please ask again, with more detail about what you want/need/have in
mind.

Okay, the question is too open: My focus is on how you would structure
the plugin code (directories, files) and how you would write code to
load these plugins.
And _this_ sounds like a combination of something that's more
a design issue (the structure) and a trivial implementation
detail (trivial in Python, anyway).

I'll try anyway, as a first test case: put all plugins in a
standard folder, with nothing else present in that folder.
They can be individual modules or packages as you will.
The code to load the plugin consists of an __import__...
and not much else.

I have done something similar, with a few differences. I am using a
convention that all the plugins are implemented as a class with a specific
name ("Handler" in my case), each plugin in its own module (so each
Handler class is scoped by its module). I import all the modules in the
directory dedicated to plugins and I do the following checks: the module
contains a class Handler, the module is not the one implementing the base
class itself (bit of a hack, I admit), and the Handler class is a subclass
of the base class (use isinstance). The checks allow to have also other
modules in the

I forgot what I'm doing in my own code. I looked at it again and found that
I'm not using isinstance(), instead I am doing a check for
"baseClass.Hand ler in hndlrCls.__base s__", which is much better, because it
doesn't need the creation of an instance. Here is the actual code (my
handlers are the equivalent of your plugins):
# Build a list of the modules that contain a valid Handler class.
hndlrList = []
for hndlrFile in glob.glob(
os.path.join(os .environ['zigzag_install _dir'], 'handlers',
'*.py')):
modName = os.path.split(o s.path.splitext (hndlrFile)[0])[1]
if modName == 'baseClass' or \
modName == 'handlerTemplat e':
continue
impMod = __import__('han dlers.'+modName ,
globals(),
locals(),
['Handler'])
if not vars(impMod).ha s_key('Handler' ):
continue
hndlrCls = vars(impMod)['Handler']
if not baseClass.Handl er in hndlrCls.__base s__:
continue
hndlrList.appen d(modName)

baseClass.Handl er is the base class for all my handlers.
plugins directory that do not actually implement a plugin (for instance, I
have a mixin class in that same directory).

Because you have both plugins and sub-plugins you may want to use some
checks like mine, so you have both plugins and sub-plugins in the same
directory, but you select only the plugin classes. That's if I understand
your requirements correctly.

Dan

Maybe this feedback will help you refine the question further...

-Peter