A brief introduction for python-list: Unununium is an OS being developed
in Python wherever possible. This message is phrased for the Unununium
list, but as the questions are about hacking cPython, I have CCed
python-list as well. <http://unununium.org/>
As not everyone has heard yet, the current scheduler is insufficient for
our needs. The biggest restriction is that it allocates threads in pools
of 32, which becomes a problem when we need 8M * 32 stacks to accommodate
Python. Furthermore, statically allocating huge stacks is a problem
anyway, and we need to move to a multiple address space model.
To that end I've decided to get a new scheduler, and I want to do it in
Python. There are some things to consider:
Traditional multitasking is rather inefficient. Switching stacks and
blowing away the TLB cache on each process switch is a waste of time.
Two alternatives come to mind, Twisted's select() based reactor which is
a purely event based model, and stackless, which provides 'tasklets'.
The problem with Twisted is that it requires code to be rewritten in
terms of callbacks that happen in response to events. This works well
for applications designed that way, but we need a solution that works
for all applications, even those not in Python. Furthermore, scheduling
is coarse and purely cooperative, which is efficient for a single
application but does not scale to an entire system. However, it is
valuable to consider Twisted as a method for application-level
multitasking.
The problem with Stackless is that I can't find any information on it.
I've made inquiries to their mailing list about how tasklets are
scheduled and received no response. Perhaps someone knowledgeable has
subscribed since I last asked this question. If so, please share your
knowledge.
I'd like to use an existing solution, but that doesn't look like it will
happen. In that case I need to better understand Python's internals to
write a scheduler with it. I think if I were able to attach a python
routine to an interrupt handler, I could do with a minimal amount of
supporting C and assembly. However, I'm not sure how I can do this. The
problem is the infamous GIL; I don't know how I can acquire the lock
without multiple threads. Perhaps I don't need to acquire it if I'm
careful about what data I access, but I don't know enough about Python
to answer that. Further complicating the issue is that our build of
Python does not include thread support, because...well, there is none!
Please share your ideas.
