471,326 Members | 2,144 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,326 software developers and data experts.

Multiple python interpreters within the same process

How do I use multiple Python interpreters within the same process?

I know there's a function Py_NewInterpreter. However, how do I use functions
like Py_RunString etc. with it? They don't take any arguments that would
tell on which interpreter to run the string...?

Marcin
Jun 9 '07 #1
3 8516
On Jun 9, 5:00 pm, "Marcin Kalicinski" <kal...@poczta.onet.plwrote:
How do I use multiple Python interpreters within the same process?

I know there's a function Py_NewInterpreter. However, how do I use functions
like Py_RunString etc. with it? They don't take any arguments that would
tell on which interpreter to run the string...?

Marcin
You may want to look at the code module http://docs.python.org/lib/module-code.html

André

Jun 9 '07 #2
André wrote:
On Jun 9, 5:00 pm, "Marcin Kalicinski" <kal...@poczta.onet.plwrote:
>How do I use multiple Python interpreters within the same process?

I know there's a function Py_NewInterpreter. However, how do I use functions
like Py_RunString etc. with it? They don't take any arguments that would
tell on which interpreter to run the string...?

Marcin

You may want to look at the code module http://docs.python.org/lib/module-code.html
That's completely unrelated.

To answer Marcin's question, from what I understand, running multiple
Python interpreters is not supported. There are various issues with
object sharing and refcounts, etc., and are unlikely to be fixed soon if
ever.

- Josiah
Jun 9 '07 #3
On Jun 10, 9:07 am, Josiah Carlson <josiah.carl...@sbcglobal.net>
wrote:
André wrote:
On Jun 9, 5:00 pm, "Marcin Kalicinski" <kal...@poczta.onet.plwrote:
How do I use multiple Python interpreters within the same process?
I know there's a function Py_NewInterpreter. However, how do I use functions
like Py_RunString etc. with it? They don't take any arguments that would
tell on which interpreter to run the string...?
Marcin
You may want to look at the code modulehttp://docs.python.org/lib/module-code.html

That's completely unrelated.

To answer Marcin's question, from what I understand, running multiple
Python interpreters is not supported. There are various issues with
object sharing and refcounts, etc., and are unlikely to be fixed soon if
ever.
I really don't know why people keep propagating this myth that you
can't run multiple Python interpreters. :-(

The Python C API has supported multiple Python interpreter instances
within a single process for a long time. If you write your C program
correctly there isn't a problem. The only real limitation is that
different Python interpreter instances can't use different versions of
a C extension module as they when loaded are shared across all Python
interpreter instances.

C extension modules can also cause other problems as well, but this
isn't the fault of Python but of the module writers. Specifically, if
a C extension module is written to only use the simplified API for GIL
locking it can only be used with the first Python interpreter instance
created. If they use the wider GIL locking API properly then there is
no problem with using it in other Python interpreter instances.
Another issue although one which you aren't likely to come across
unless you are doing really tricky stuff, is where a C extension
module was written so as to assume that once it is loaded into a
Python interpreter instance, that that interpreter will not be
destroyed. From what I have seen Pyrex generated C extension modules
possibly have this latter problem and will cause a process to crash if
a Python interpreter instance is destroyed and then a new one created
in its place.

As proof that all this stuff can work, have a look at mod_python and
mod_wsgi. Both make use of multiple Python interpreter instances in a
single process. The mod_wsgi documentation even points out the above
problems with C extension modules in latter sections of:

http://code.google.com/p/modwsgi/wiki/ApplicationIssues

Graham

Jun 9 '07 #4

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

1 post views Thread by Maciej Sobczak | last post: by
2 posts views Thread by bmatt | last post: by
47 posts views Thread by Pierre Barbier de Reuille | last post: by
5 posts views Thread by Adam Atlas | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.