Stackless python and microthreads

[Matt Leslie]

Hi,

I'm trying to use microthreads under stackless python, since they sound
like exactly what I am after, but I am having very little success.
I've got a fresh install of python 2.3.3 from python.org, then
downloaded the binary python2.3 release of stackless python from
www.stackless.com. This contained three files:

python23.dll, python23.lib, python23.exp

I searched for the original versions of these files, and copied:

stackless python23.dll over c:\winnt\system32\python23.dll
stackless python23.lib over C:\Program Files\python2.3\libs\python23.lib

Since I believe .exp files are associated with .lib files. I put the
..exp file in the same place as the lib. There was no existing file with
this name on my file system.

I then downloaded the microthreads package from
http://willware.net:8080/uthread.py

I then ran python, type 'import uthread' and immediately got the
following error.

---------------------------------------
Python 2.3.3 Stackless 3.0 040407 (#51, Apr 7 2004, 19:28:46) [MSC
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

import uthread.py

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "uthread.py", line 35, in ?
import continuation, sys, traceback, bisect, time, StringIO
ImportError: No module named continuation

Has anyone run into this before? Can someone tell me where to find this
continuation module?

Thanks,
Matt

[Mike C. Fletcher]

Continuations (which are the coolest things since programming was
invented) were forced out of Stackless Python around the end of version
2.0 for political reasons IIRC. Current Stackless doesn't have
continuation support at all, so any version of micro-threads written to
use continuations is just dead-on-arrival for the current version.
Stackless 1.0 and 2.0 were only for pre-2.3 releases of Python AFAIK.

I would imagine that *someone* has written a Tasklets-based
micro-threading implementation for the new Stackless (after all,
micro-threading is AFAIK *the* major use-case for Stackless (save for
those of us who like to play around with new modes of programming)) and
you'll just need to poke around for that rewrite.

In a former life I got paid to work on extending the continuation-based
micro-thread implementation... continuations are the most awesome toys
you can imagine... we will all mourn their passing in time...

Good luck,
Mike

Matt Leslie wrote:
....

I'm trying to use microthreads under stackless python, since they
sound like exactly what I am after,
....
python23.dll, python23.lib, python23.exp
....
I then downloaded the microthreads package from
http://willware.net:8080/uthread.py
....
ImportError: No module named continuation

....
________________________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://members.rogers.com/mcfletch/
blog: http://zope.vex.net/~mcfletch/plumbing/

[Peter Hansen]

Mike C. Fletcher wrote:

continuations are the most awesome toys
you can imagine... we will all mourn their passing in time...

Will that be _before_, or _after_ we understand what they are? ;-)

-Peter

[Donn Cave]

Quoth Peter Hansen <pe***@engcorp.com>:
| Mike C. Fletcher wrote:
|
| > continuations are the most awesome toys
| > you can imagine... we will all mourn their passing in time...
|
| Will that be _before_, or _after_ we understand what they are? ;-)

I gave them a try, and I mourn them with a touch of relief, I guess.
I'm the guy who won't bother to learn list comprehensions and generally
despises cool gimmicks, but continuations were more than just cool,
they were terribly powerful.

I hooked them into a graphics toolkit wrapper with an event dispatching
model where each window, network connection etc., ran in its own native
thread. No blocking, always ready to respond to a new message. When a
couple of these threads have to interact, they would queue messages back
and forth, and the programming paradigm can get awkward - a lot of state
can pile up as everyone's trying to keep track of their progress through
some kind of procedure.

Well, suppose you send your message and save the whole function state
somewhere, and return to the dispatcher. When the response comes back,
you pick up the function state and continue it, feeding the response in.

Now instead of a tedious kind of state machine, you're writing an plain,
ordinary function that sends messages to its peer and looks at the response,
as though that were all synchronous, and it's so much simpler. Yet the
execution underneath that isn't synchronous at all, because your computation
is suspended in between your send and the response. It really does return
every time it sends a message, it just starts up next time where it left
off.

Of course it's hairy, too. That's where the relief comes in. I'll never
be able to use continuations ... but then, I'll never have to deal with
anyone else's code that uses them.

Donn Cave, do**@drizzle.com

[Michele Simionato]

"Mike C. Fletcher" <mc******@rogers.com> wrote in message news:<ma**************************************@pyt hon.org>...

Continuations (which are the coolest things since programming was
invented) were forced out of Stackless Python around the end of version
2.0 for political reasons IIRC. Current Stackless doesn't have
continuation support at all, so any version of micro-threads written to
use continuations is just dead-on-arrival for the current version.
Stackless 1.0 and 2.0 were only for pre-2.3 releases of Python AFAIK.

In a former life I got paid to work on extending the continuation-based
micro-thread implementation... continuations are the most awesome toys
you can imagine... we will all mourn their passing in time...

Good luck,
Mike

Uhm ...

My experience with continuations (in Scheme) is limited, but I really
wonder if continuations have a place in a high level programming language.
I mean, continuations are basic building blocks: you can build on top
of them exception systems, generators, coroutines, microthreads, etc,
and it is an interesting learning experience to understand how this is
done.

However, to pass from continuations to something useful takes a
LONG way. Continuations are a too low level concept. A high level language
should already provide the right high level tools. I mean, the language
designer should implement generators, exception systems, microthreads, etc.
not the application programmer. When you have the high level concepts written
down by others, you don't need continuations.

So I guess now Stackless has microthreads built inside, and the application
programmer is not forced to write them on top of continations.

Speaking in general, generators and the current exception system are more
than enough for my typical needs in Python: which kind of applications do
you have in mind where you would like to have continuations? (a part from
microthreads and things like changing the language introducing new
control structures, etc, all stuff than should not be left to the
application programmer, IMHO).

Michele

[Michael Walter]

> My experience with continuations (in Scheme) is limited, but I really

wonder if continuations have a place in a high level programming language.
I mean, continuations are basic building blocks: you can build on top
of them exception systems, generators, coroutines, microthreads, etc,
and it is an interesting learning experience to understand how this is
done. Yeah exactly, they are one reason which allows Scheme to be really
high-level.

They're pretty useful for web applications, too -- the difference to
above-mentioned examples would be that IMHO there is no adequate
alternative to them in current Python.
However, to pass from continuations to something useful takes a
LONG way. Continuations are a too low level concept. That probably depends on the point of view. I don't think it matters
much whether a specific feature is built-in into the language as such or
part of the/a standard library.
A high level language should already provide the right high level tools. I mean, the language designer should implement generators, exception systems, microthreads, etc. not the application programmer. When you have the high level concepts written
down by others, you don't need continuations. Would you have an idea on how a higher-level replacement for
continuations in web application development? I'm excited (and _not_
being any sarcastic or stuff, honestely!).
Speaking in general, generators and the current exception system are more
than enough for my typical needs in Python: which kind of applications do
you have in mind where you would like to have continuations? (a part from
microthreads and things like changing the language introducing new
control structures, etc, all stuff than should not be left to the
application programmer, IMHO).

Web application development involving Continuations (such as in Seaside,
the PST web server etc.).

Cheers,
Michael

[Just]

In article <2g************@uni-berlin.de>,
Michael Walter <[e-mail]> wrote:

not the application programmer. When you have the high level concepts
written
down by others, you don't need continuations.

Would you have an idea on how a higher-level replacement for
continuations in web application development? I'm excited (and _not_
being any sarcastic or stuff, honestely!).

Co-routines.

Just

[Michael Walter]

Just wrote:

In article <2g************@uni-berlin.de>,
Michael Walter <[e-mail]> wrote:

not the application programmer. When you have the high level concepts
written
down by others, you don't need continuations.

Would you have an idea on how a higher-level replacement for
continuations in web application development? I'm excited (and _not_
being any sarcastic or stuff, honestely!).

Co-routines.

Would you care to elaborate? Can you store the state of a coroutine and
resume it multiple times?

Cheers,
Michael

[Just]

In article <2g************@uni-berlin.de>,
Michael Walter <[e-mail]> wrote:

Just wrote:

In article <2g************@uni-berlin.de>,
Michael Walter <[e-mail]> wrote:

Would you have an idea on how a higher-level replacement for
continuations in web application development? I'm excited (and _not_
being any sarcastic or stuff, honestely!).

Co-routines.

Would you care to elaborate? Can you store the state of a coroutine and
resume it multiple times?

You can resume a co-routine multiple times just like you can resume
generators mutliple times, but that's something completely different
from restarting a continuation multiple times.

When you mentioned continuations for web programming, I assumed you
meant their ability to turn flow control inside out (allowing to write
event-based networking code in a more natural way). Co-routines are a
much nicer abstraction for that than continuations.

Just

[Michael Walter]

Just wrote:

In article <2g************@uni-berlin.de>,
Michael Walter <[e-mail]> wrote:

Just wrote:

In article <2g************@uni-berlin.de>,
Michael Walter <[e-mail]> wrote:

Would you have an idea on how a higher-level replacement for
continuations in web application development? I'm excited (and _not_
being any sarcastic or stuff, honestely!).

Co-routines.
Would you care to elaborate? Can you store the state of a coroutine and
resume it multiple times?

You can resume a co-routine multiple times just like you can resume
generators mutliple times, but that's something completely different
from restarting a continuation multiple times.

Well, I think you would want to resume it multiple times based on a
given state (essentially like resuming a stored continuation).
When you mentioned continuations for web programming, I assumed you
meant their ability to turn flow control inside out (allowing to write
event-based networking code in a more natural way). Co-routines are a
much nicer abstraction for that than continuations.

Would you have some links/papers for me in regard to web programming and
coroutines? Some quick googling didn't bring anything up (but maybe I'm
just too tired).

What I meant was using continuations for "linear" code flow, such as
mentioned in
http://www.ai.mit.edu/~gregs/ll1-dis.../msg02456.html,
described in http://www.double.co.nz/scheme/modal-web-server.html and
applied in Seaside, Borges, PLT Web Server etc.

Cheers,
Michael

[Mike C. Fletcher]

Michele Simionato wrote:

"Mike C. Fletcher" <mc******@rogers.com> wrote in message news:<ma**************************************@pyt hon.org>...

Continuations (which are the coolest things since programming was
invented)

....
However, to pass from continuations to something useful takes a
LONG way. Continuations are a too low level concept. A high level language
should already provide the right high level tools. I mean, the language
designer should implement generators, exception systems, microthreads, etc.
not the application programmer. When you have the high level concepts written
down by others, you don't need continuations.

Except we did ;) . When we sat down to start working with
micro-threads, they were (for our purposes) extremely heavy beasts when
they were sleeping and scheduling themselves. So, having a simple
Python implementation built out of continuations, we were able to say
"hey, we can fix that cheaply" and dedicate a few weeks of work to the
project to improve the implementation.

Guido may be omniscient ;) , but I don't think he's achieved
omnipotence. He and the python-dev crowd can't, and *won't* implement
many things that people writing frameworks *need* (and really, in many
cases, they shouldn't be doing it anyway until the idea proves itself).
Having a building block such as continuations exposed allows framework
developers/meta-programmers the chance to experiment with new
flow-control structures and to do that experimentation *in a high-level
language* (does this sound familiar to all the PyPy peoples).
So I guess now Stackless has microthreads built inside, and the application
programmer is not forced to write them on top of continations.

Sure, but it was never the app programmer that did it anyway, so let's
leave them out of this. Using Python does not automatically mean you
are an applications programmer rather than a framework developer or
meta-programmer (btw, nothing wrong with being an apps programmer, it's
just a different type of work). Python isn't *just* a scripting
language either ;) .
Speaking in general, generators and the current exception system are more
than enough for my typical needs in Python: which kind of applications do
you have in mind where you would like to have continuations? (a part from
microthreads and things like changing the language introducing new
control structures, etc, all stuff than should not be left to the
application programmer, IMHO).

Why make that kind of experimentation difficult just to erect a barrier
between user and developer? Python isn't a place to keep every powerful
tool from the plebeian masses just because they might hurt themselves.
Heck, if we're going to do that, what the heck are we exposing
metaclasses for (and really, would the two of us be happy if they took
away our metaclasses)? There are framework developers/meta-programmers
using Python who just aren't interested in working in C code any more.
Giving them tools is *not* a bad thing.

Regarding applications; restartable exceptions, micro-threads,
long-running-calculation restarts, migrating running code across
machines, and creating new methods of parsing all seem like experiments
that would benefit from having continuations available to ease
implementation. Any of those is a thing that an app programmer might
want/need, and be willing to have a meta-programmer create for them...
why force that work into C code needlessly?

However, continuations are dead, and though I may mourn them, I'm not
interested in trying to resurrect them. At the moment my company's
current contracts would only be interested in are micro-threads; and
even there, we aren't really interested enough to switch from our
already-built frameworks; just too much inertia. Too bad, because a
micro-threaded networking environment is the closest thing I've ever
seen to the promised land for networking... garn I miss working on that.

Peace all,
Mike

________________________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://members.rogers.com/mcfletch/
blog: http://zope.vex.net/~mcfletch/plumbing/

[Michele Simionato]

"Mike C. Fletcher" <mc******@rogers.com> wrote in message news:<ma************************************@pytho n.org>...

Regarding applications; restartable exceptions, micro-threads,
long-running-calculation restarts, migrating running code across
machines, and creating new methods of parsing all seem like experiments
that would benefit from having continuations available to ease
implementation. Any of those is a thing that an app programmer might
want/need, and be willing to have a meta-programmer create for them...
why force that work into C code needlessly?

Ok, I see your point, so let me restate my doubts in this form:
are you sure that you need to go to such a low level as continuations?
Wouldn't generators/coroutines be enough to fullfill your needs?

---

On a different note, I could notice that giving so much power to
metaprogrammers may have unfortunate results. For instance, we already
have three different kind of interfaces in three different frameworks
(Zope, Twisted, Peak) whereas there should be only one obvious way.
So,
I mantain that certain things should be left to the language designer
(i.e. Guido) and not to the metaprogrammer.

Otherwise you ends up as in Scheme, where essentially everyone can be
the designer of his own little language, with the disadvantages that
you
can see. I think metaprogrammers should be more humble, and think ten
times before using metaprogramming features to change the language.
But sometime the temptation is irrestible ;)

Michele Simionato

[Andrew Bennetts]

On Fri, May 14, 2004 at 09:53:03PM -0700, Michele Simionato wrote:
[...]

On a different note, I could notice that giving so much power to
metaprogrammers may have unfortunate results. For instance, we already
have three different kind of interfaces in three different frameworks
(Zope, Twisted, Peak) whereas there should be only one obvious way.

FWIW, Twisted will be using Zope 3's interfaces framework soon, which will
leave just Zope and PEAK.

-Andrew.

[Michele Simionato]

Andrew Bennetts <an***************@puzzling.org> wrote in message news:<ma************************************@pytho n.org>...

FWIW, Twisted will be using Zope 3's interfaces framework soon

Aha! It seems clear that I have to find the time to start studying
Zope interfaces then ....

Michele

[Michele Simionato]

Michael Walter <[e-mail]> wrote in message news:<2g************@uni-berlin.de>...

What I meant was using continuations for "linear" code flow, such as
mentioned in
http://www.ai.mit.edu/~gregs/ll1-dis.../msg02456.html,
described in http://www.double.co.nz/scheme/modal-web-server.html and
applied in Seaside, Borges, PLT Web Server etc.

This is a most interesting reference, thanks! Incidentally, I am in the process
of learning Zope TAL/TALES/METAL, which I do not like that much; I would
rather prefer to generate HTML pages with Lisp macros and use real
s-expressions instead of XML/HTML, but I can't :-(
Lisp/Scheme is much more suitable for web applications (in principle)
than any other language I know, it is unfortunate that (in practice) it
is not that used ...
Michele Simionato

[Ville Vainio]

>>>>> "Michele" == Michele Simionato <mi***************@poste.it> writes:

Michele> of learning Zope TAL/TALES/METAL, which I do not like
Michele> that much; I would rather prefer to generate HTML pages
Michele> with Lisp macros and use real s-expressions instead of
Michele> XML/HTML, but I can't :-(

What would you do with macros? The resulting s-exprs would still need
to be evaluated to get the html, so the performace wouldn't be
better...

And you can use s-exprs in python too:

page = [
[ head ,
[ title, "my page"] ],
[ body,
[ heading, "welcome to my page"],
[ para , "paragraph1 blah blah"]
[ para , "paragraph2 more blah blah "]]
]

Further elaboration on the idea at

http://www.students.tut.fi/~vainio24/pysexpr/

--
Ville Vainio http://tinyurl.com/2prnb

[Michael Walter]

Ville Vainio wrote:

>>"Michele" == Michele Simionato <mi***************@poste.it> writes:

Michele> of learning Zope TAL/TALES/METAL, which I do not like
Michele> that much; I would rather prefer to generate HTML pages
Michele> with Lisp macros and use real s-expressions instead of
Michele> XML/HTML, but I can't :-(
What would you do with macros? [..]

Syntactic abstraction. I would love to see macros in Python (actually
working on some ideas for adding them to Python).

Cheers,
Michael

[Valentino Volonghi aka Dialtone]

mi***************@poste.it (Michele Simionato) writes:

This is a most interesting reference, thanks! Incidentally, I am in the process
of learning Zope TAL/TALES/METAL, which I do not like that much; I would
rather prefer to generate HTML pages with Lisp macros and use real
s-expressions instead of XML/HTML, but I can't :-(
Lisp/Scheme is much more suitable for web applications (in principle)
than any other language I know, it is unfortunate that (in practice) it
is not that used ...

Hi Michele :).

If all you need is s-expr you maybe can look at Nevow (pronounced
'nuevo'), which is the new web toolkit built on top of twisted.web.

It has a template syntax that is similar to ZPT.

And it has stan which is an s-expr-like syntax.

Here is a little helloworld example:

docFactory = rend.stan(
T.html[
T.head[
T.title['Hello'],
],
T.body[
T.p['Welcome to the wonderful world of Nevow!'],
],
]
)

More infos at www.nevow.com or in the nevow wiki:
http://divmod.org/users/wiki.twistd/....cgi/FrontPage

or in irc in #twisted.web

--
Valentino Volonghi aka Dialtone
Linux User #310274, Proud Gentoo User
Blog: http://vvolonghi.blogspot.com
Home Page: http://xoomer.virgilio.it/dialtone/

[Ville Vainio]

>>>>> "Valentino" == Valentino Volonghi aka Dialtone <di************************@virgilio.it> writes:

Valentino> And it has stan which is an s-expr-like syntax.

Valentino> Here is a little helloworld example:

Valentino> docFactory = rend.stan(
Valentino> T.html[
Valentino> T.head[
Valentino> T.title['Hello'],
Valentino> ],
Valentino> T.body[
Valentino> T.p['Welcome to the wonderful world of Nevow!'],
Valentino> ],
Valentino> ]
Valentino> )

Just a thought: it would be more s-expr like if it just had the
"function"ish part inside the list in the normal prefix notation:

[T.html,
[T.head,
[T.title, 'Hello']],
[T.body, [T.p, 'Welcome...]]]

In a way this seems slightly clearer (especially the , chars n/w ]
chars seem very suspicous to me.

--
Ville Vainio http://tinyurl.com/2prnb

[Valentino Volonghi aka Dialtone]

Ville Vainio <vi***@spammers.com> writes:

Just a thought: it would be more s-expr like if it just had the
"function"ish part inside the list in the normal prefix notation:

[T.html,
[T.head,
[T.title, 'Hello']],
[T.body, [T.p, 'Welcome...]]]

In a way this seems slightly clearer (especially the , chars n/w ]
chars seem very suspicous to me.

Sure, but I think this is less pythonic than the nevow version, and I
find that version also clearer, but I also think that this is a matter
of tastes :).


--
Valentino Volonghi aka Dialtone
Linux User #310274, Proud Gentoo User
Blog: http://vvolonghi.blogspot.com
Home Page: http://xoomer.virgilio.it/dialtone/

[Peter Hansen]

Valentino wrote:

Hi Michele :).

If all you need is s-expr you maybe can look at Nevow (pronounced
'nuevo'), which is the new web toolkit built on top of twisted.web.

Which raises the question, how do you pronounce "nuevo"?

(Perhaps noo-VOH, as in the French nouveau?)

-Peter

[Erik Max Francis]

Peter Hansen wrote:

Which raises the question, how do you pronounce "nuevo"?

(Perhaps noo-VOH, as in the French nouveau?)

Noo-WAY-voh. (More like NWAY-voh, actually.)

--
__ Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
/ \ San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
\__/ The quality, not the longevity, of one's life is what is
important. -- Dr. Martin Luther King, Jr.

[Michele Simionato]

Michael Walter <[e-mail]> wrote in message news:<2g************@uni-berlin.de>...

Ville Vainio wrote:

>>>"Michele" == Michele Simionato <mi***************@poste.it> writes:

Michele> of learning Zope TAL/TALES/METAL, which I do not like
Michele> that much; I would rather prefer to generate HTML pages
Michele> with Lisp macros and use real s-expressions instead of
Michele> XML/HTML, but I can't :-(
What would you do with macros? [..]

Syntactic abstraction. I would love to see macros in Python (actually
working on some ideas for adding them to Python).

NO!! I am really talking about prefix notation: I am pro macros when
you have prefix notations and s-expressions (i.e. HTML/XML), whereas
I am not advocating macros in Python, which has an infix notation.

There are macro systems based on pattern matching which can work with
infix notation (I hear Dylan has one), but I feel somewhat happier with
Lisp-style macros where everything is a list.
Michele Simionato

[Michele Simionato]

Valentino Volonghi aka Dialtone <di************************@virgilio.it> wrote in message news:<87************@vercingetorix.caesar.org>...

Hi Michele :).

If all you need is s-expr you maybe can look at Nevow (pronounced
'nuevo'), which is the new web toolkit built on top of twisted.web.

It has a template syntax that is similar to ZPT.

And it has stan which is an s-expr-like syntax.

Here is a little helloworld example:

docFactory = rend.stan(
T.html[
T.head[
T.title['Hello'],
],
T.body[
T.p['Welcome to the wonderful world of Nevow!'],
],
]
)

Interesting. But I have to check which facilities it provides to manipulate
s-expressions; if I was in a pervers mood, I could even generate nevow s-
expressions from Scheme ;)

Michele

[Donn Cave]

Quoth Peter Hansen <pe***@engcorp.com>:
| Valentino wrote:
....
| > If all you need is s-expr you maybe can look at Nevow (pronounced
| > 'nuevo'), which is the new web toolkit built on top of twisted.web.
|
| Which raises the question, how do you pronounce "nuevo"?

Good question. I thought I knew how to pronounce "nuevo", but if it
could be spelled "nevow" ... is that twisted, or what?

Donn

[Michael Walter]

Michele Simionato wrote:

Michael Walter <[e-mail]> wrote in message news:<2g************@uni-berlin.de>...

Ville Vainio wrote:

>>>>"Michele" == Michele Simionato <mi***************@poste.it> writes:

Michele> of learning Zope TAL/TALES/METAL, which I do not like
Michele> that much; I would rather prefer to generate HTML pages
Michele> with Lisp macros and use real s-expressions instead of
Michele> XML/HTML, but I can't :-(
What would you do with macros? [..]
Syntactic abstraction. I would love to see macros in Python (actually
working on some ideas for adding them to Python).

NO!! I am really talking about prefix notation: I am pro macros when
you have prefix notations and s-expressions (i.e. HTML/XML), whereas
I am not advocating macros in Python, which has an infix notation.

And my point was that I'm advocating macros in non-s-expression
languages, too, as I think they are equally valuable there.
There are macro systems based on pattern matching which can work with
infix notation (I hear Dylan has one), but I feel somewhat happier with
Lisp-style macros where everything is a list.

I'm pretty sure (and my experiments seem to indicate that, too) that you
can have both macros defined by pattern-matching and "raw" macros which
operate on the AST in Python, too.

Yes, Dylan has macros, too, despite its syntax.

Cheers,
Michael

[Christian Tismer]

Donn Cave wrote:

Quoth Peter Hansen <pe***@engcorp.com>:
| Mike C. Fletcher wrote:
|
| > continuations are the most awesome toys
| > you can imagine... we will all mourn their passing in time...
|
| Will that be _before_, or _after_ we understand what they are? ;-)

I gave them a try, and I mourn them with a touch of relief, I guess.
I'm the guy who won't bother to learn list comprehensions and generally
despises cool gimmicks, but continuations were more than just cool,
they were terribly powerful.
Yes. And terribly hard to maintain as well :-)
I hooked them into a graphics toolkit wrapper with an event dispatching
model where each window, network connection etc., ran in its own native
thread. No blocking, always ready to respond to a new message. When a
couple of these threads have to interact, they would queue messages back
and forth, and the programming paradigm can get awkward - a lot of state
can pile up as everyone's trying to keep track of their progress through
some kind of procedure.

Well, suppose you send your message and save the whole function state
somewhere, and return to the dispatcher. When the response comes back,
you pick up the function state and continue it, feeding the response in.
Ok, that's a nice pattern which I use as well, *with* current Stackless
and *without* continuations. Turning control flow inside out does not
depend upon continuations. You just need the ability to switch contexts,
which is in fact much less, but buys you a lot more.

A continuation is immutable. That means it can be activated multiple
times and it will not change state. How to explain this? Well, consider
a Python function, and you have several continuations captured from
that running function. Then you can jump at any continuation at any
time, as often as you like. This capability is incredible, but I never
saw an example of code that *really* had to have this feature.

Returning to some point in the running program and continuing there
is what most people need. This is less than a general continuation,
because the continuation is gone after resuming. You can call it
a one-shot continuation. On every context switch, the current
continuation is preserved, but it is never fired twice. This is very
much like threads.

Full continuations enforce that you have complete control over
the current program, especially you need to be able to track
all object references which must stay alive in a continuation.
This is not possible if your program state contains hidden
objects on the C stack, because we have no introspection feature
in the Python C implementation.
In other words: You cannot use continuations in the context of
C extensions.

Restricting ourselves to one-shot continuations, we don't need
to track any objects explicitly. This makes it possible to
deal with hidden objects just by saving and restoring parts of
the C stack.

And this makes tasklets more powerful that the old continuations:
it is possible to switch context all the time. The program
environment does not need to collaborate. Collaboration is more
efficient (i.e. trying to leave no state on the C stack), and
Stackless tries to do it almost everywhere. But that's not
a limitation. You can write ordinary C code and ask Stackless to
switch tasklets, and it will switch. I consider this more
valuable than continuations which pretend to be universal and
immportal, but only in a restricted context.
Now instead of a tedious kind of state machine, you're writing an plain,
ordinary function that sends messages to its peer and looks at the response,
as though that were all synchronous, and it's so much simpler. Yet the
execution underneath that isn't synchronous at all, because your computation
is suspended in between your send and the response. It really does return
every time it sends a message, it just starts up next time where it left
off.

Exactly what I'm doing with Stackless. I have my own implementation
of sockets and files, which pretend to have synchronized calls
only, but under the hood they are asynchronous, dispatching over
a poll/select loop and optimizing throughput. The code is much
shorter than asyncore, and reasonably more effective if your server
is computation-bound. Unfortunately I can't publish this module,
yet, it was contract work.

cheers - chris

--
Christian Tismer :^) <mailto:ti****@stackless.com>
Mission Impossible 5oftware : Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/
14109 Berlin : PGP key -> http://wwwkeys.pgp.net/
work [phone] home [phone] mobile [phone]
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/

[Matt Leslie]

Now instead of a tedious kind of state machine, you're writing an plain,
ordinary function that sends messages to its peer and looks at the
response,
as though that were all synchronous, and it's so much simpler. Yet the
execution underneath that isn't synchronous at all, because your
computation
is suspended in between your send and the response. It really does
return
every time it sends a message, it just starts up next time where it left
off.

Infact this is also similar to what I'm trying to do, except I'm doing
it back to front. I want to write a package to help simulate distributed
systems, making what would be simple synchronous method calls
asynchronous. I do this by intercepting method calls between threads,
and passing them on only after a simulated network delay - they are held
in an event queue and basically only executed when there is nothing
scheduled to happen 'before' it. It boils down to a similar structure, I
think.

Anyway, stackless would be good for this because I need lots of threads
so simulate large scale P2P systems. The thing is although there was a
fair amount of documentation on microthreads I can find nothing usefull
on tasklets! Can anyone point me at some? Please? The stackless mailing
list has turned up nothing so far...

Matt

[Christian Tismer]

Matt Leslie wrote:

....

Anyway, stackless would be good for this because I need lots of threads
so simulate large scale P2P systems.
This is true. For performance comparison reasons, and to make
Stackless "complete" in a way, I just developed real thread support
and measured context switch speed. Now it is not longer a claim,
but the truth (going to be Stackless 3.1):

Stackless soft-switched channels are 10 times faster than
using real threads.

Stackless tasklets are 10000 times smaller than threads on Windows.
The thing is although there was a
fair amount of documentation on microthreads I can find nothing usefull
on tasklets! Can anyone point me at some? Please? The stackless mailing
list has turned up nothing so far...

There isn't much, of course. There is the beginning of a glossary
on the website, there are lots of exmaples in preparation
after the last sprint, but actually there is still a lack
of documentation.

The C API is there, just look into stackless_api.h.

Then, for a description of individual functions,

import stackless
help(stackless)

gives you a lot of information.

Feel free to ask me for support, I will help as much as I can.

cheers - chris

--
Christian Tismer :^) <mailto:ti****@stackless.com>
Mission Impossible 5oftware : Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/
14109 Berlin : PGP key -> http://wwwkeys.pgp.net/
work [phone] home [phone] mobile [phone]
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/