ne*******@gmail.com wrote:
I've seen various generator-based microthread implementations online,
but I've been wondering: has anyone used microthreads in this manner in
a game environment? Note, I am emphatically *not* referring to
Stackless, which I know has been used in a production game environment;
this post is referring strictly to the standard Python interpreter, (v.
2.4 or 2.5 as you wish).
I intend to try a little cooperative microthreading implementation in
my game server to avoid threading issues and possibly simplify the
programming model. This seems particularly attractive given the
enhancements Python 2.5 brings to generators.
If anyone else has tried this, what I would like to know is this:
Several. I'll largely talk about our experience with Kamaelia below, but
there's also LGT which is also pretty interesting.
LGT:
http://www.pygame.org/projects/9/20/
Kamaelia:
The reason I'll only talk about Kamaelia is because I'm the lead developer
on that project, and because I think it's a fun way to write this sort of
thing :-)
Naturally I'm biassed, bear that in mind :-)
* How many microthreads were you able to run concurrently with decent
performance?
Numbered in the thousands. It's been a while since we've done any benchmarks
however. (Most of our use of generators has revolved around network
systems, but we use Pygame fairly heavily)
* How much computation is feasible to carry out in a microthread?
(Certainly, this goes hand in hand with the previous question, of
course.)
As much as you like, however the more you do, the less microthreads you can
run (as you say, this goes hand in hand).
For short examples of what we're using generators for, take a peek here:
*
http://kamaelia.sourceforge.net/Cookbook.html
* What sort of scheduler did you use?
We use a simple round robin scheduler.
* Was it worth it? By "worth it," I mean in terms of performance and
ease of implementation.
Absolutely. There's a knock on of python style generators that people don't
tend to realise: it forces a your generators to be simple and focussed.
Given we augment generators with inboxes & outboxes for communications we
find we get higher levels of reuse of many components than we would
normally expect. We embed the generator in a class, giving far more
flexible communications that python 2.5's newly added facilities (this
approach works happily with python 2.2 onwards, meaning we've versions of
Kamaelia that even run on nokia phones.)
The only game we've written in Kamaelia at present is a simple bouncing cats
game (aimed at 2-4 year olds :-), which adds bouncing cats, and removes
them to a noise. However the only slow down I've really seen with that is
more due to the speed at which pygame & SDL can update the display AFAICT.
I'll no doubt have further questions later after I actually begin
implementing this stuff, but I suppose that will do for now.
Even if you're not interested in Kamaelia (then again you might - we find it
incredibly useful), you may find our tutorial useful - it shows you how to
go about building a simple scheduler and simple systems of communicating
generators in easy simple steps. (Tested on over a dozen novice python
programmers now).
*
http://kamaelia.sourceforge.net/MiniAxon/
Systems we're using this for:
* Collaborative whiteboarding (every client is also a server) including
audio
* Transcoding PVR for making a record of transmission of all BBC TV
channels, including collection of EPG data.
* Under development - video annotation/editing. (for allowing
collaborative discussion of how to post process video)
I suppose knocking up a simple game might be a nice thing to do as well :-)
For what it's worth, a focus in Kamaelia is on making it harder to do dumb
things when doing things in parallel - as you are doing when using threads
or generators. (That's the reason for the inbox/outbox metaphor - take it
out an inbox, it's yours to do what you want, put it in an outbox, you're
not allowed to do anything with it any more).
As a result generator based components and thread based components can be
mixed quite happily in Kamaelia, since you have explicit data handoff. This
might not seem immediately important to you, but as a system grows, you
need a way of handing blocking calls (since otherwise your generators and
scheduler stall), and putting those in a thread is a common idea. Doing
this safely is normally somewhat more complex than: (including imports)
import Axon
from Kamaelia.Chassis.Pipeline import pipeline
from Kamaelia.Util.Console import ConsoleEchoer
class ConsoleReader(Axon.ThreadedComponent.threadedcompo nent):
def main(self):
while 1:
data = raw_input(">>") # block waiting for user
self.send(data, "outbox") # send on to next thing
pipeline(
ConsoleReader(), # Threaded Component
ConsoleEchoer(), # Generator Component
).run()
Just as an indication of the sorts of things you can do with Kamaelia, we've
had some Google Summer of Code students working on a small selection of
Kamaelia related projects this summer. Perhaps the most relevant to you
might be the 3D work, which makes it relatively simple to create 3D objects
and then bounce them round the screen and interact with them.
Two of the students have blogs which follow their progress which you can
find here, which might also be useful when considering whether generators
are a good approach :-)
*
http://thfsoc.blogspot.com/
*
http://rjlsoc.blogspot.com/
Anyway, I've rambled on far too long, and now well off topic for the
questions you were asking, so I'll be quiet :-)
Have fun, and IMO, the generator approach (using Kamaelia, LGT, or home
rolled), is very much worth going down.
Best Regards,
Michael.
--
Michael Sparks, Kamaelia Project Lead
http://kamaelia.sourceforge.net/Home http://yeoldeclue.com/blog 
