Embedding Python, threading and scalability

[Wenning Qiu]

I am researching issues related to emdedding Python in C++ for a
project.

My project will be running on an SMP box and requires scalability.
However, my test shows that Python threading has very poor performance
in terms of scaling. In fact it doesn't scale at all.

I wrote a simple test program to complete given number of iterations
of a simple loop. The total number of iterations can be divided evenly
among a number of threads. My test shows that as the number of threads
grows, the CPU usage grows and the response time gets longer. For
example, to complete the same amount of work, one thread takes 10
seconds, 2 threads take 20 seconds and 3 threads take 30 seconds.

The fundamental reason for lacking scalability is that Python uses a
global interpreter lock for thread safety. That global lock must be
held by a thread before it can safely access Python objects.

I thought I might be able to make embedded Python scalable by
embedding multiple interpreters and have them run independently in
different threads. However "Python/C API Reference Manual" chapter 8
says that "The global interpreter lock is also shared by all threads,
regardless of to which interpreter they belong". Therefore with
current implementation, even multiple interpreters do not provide
scalability.

Has anyone on this list run into the same problem that I have, or does
anyone know of any plan of totally insulating multiple embedded Python
interpreters?

Thanks,
Wenning Qiu

[Andrew Dalke]

Wenning Qiu:

I am researching issues related to emdedding Python in C++ for a
project. Has anyone on this list run into the same problem that I have, or does
anyone know of any plan of totally insulating multiple embedded Python
interpreters?

Ahh, the Global Interpreter Lock (GIL).

Years ago, Greg Stein had a version of Python 1.4 running with no GIL.

http://www.python.org/ftp/python/con...reading.README
Search for "free threading" to get more hits on this topic.

As I recalled, it slowed down the performance on
single-processor/single-threaded
machines, so the general take was to keep the GIL. In addition, see
http://groups.google.com/groups?selm...ython-list%40p
ython.org&oe=UTF-8&output=gplain
Tim Peters:
] The prospects for another version of that grow dimmer. Everyone (incl.
] Greg) has noticed that CPython internals, over time, increase their
reliance
] on the thread-safety guarantees of the global interpreter lock.

The only solutions I know of are explicit multi-process solutions:
- a generic system, like XML-RPC/SOAP/PVM/MPI/CORBA, on
which you build your own messaging system
- use systems like Pyro or Twisted, which understand Python objects
and implement 'transparent' proxying via network communications
- use POSH, which does the proxying through shared memory (but
this uses Intel-specific assembly)

Andrew
da***@dalkescientific.com

[Afanasiy]

On 8 Jul 2003 14:54:22 -0700, we*********@csgsystems.com (Wenning Qiu)
wrote:

I am researching issues related to emdedding Python in C++ for a
project.

My project will be running on an SMP box and requires scalability.
However, my test shows that Python threading has very poor performance
in terms of scaling. In fact it doesn't scale at all.

I wrote a simple test program to complete given number of iterations
of a simple loop. The total number of iterations can be divided evenly
among a number of threads. My test shows that as the number of threads
grows, the CPU usage grows and the response time gets longer. For
example, to complete the same amount of work, one thread takes 10
seconds, 2 threads take 20 seconds and 3 threads take 30 seconds.

The fundamental reason for lacking scalability is that Python uses a
global interpreter lock for thread safety. That global lock must be
held by a thread before it can safely access Python objects.

I asked once and was told it was best fixed by removing the documentation
which mentioned it. Others also stated it was unlikely to be fixed.

http://groups.google.com/groups?hl=e...2i%25404ax.com

However, I believe Lua since 4-work4, just before Lua 5, solved this.
Unfortunately Lua is not Python.

Another thing to consider if you care about SMP, is your C/C++ memory
management, assuming you aren't using something custom already, maybe a
shared heap. I have worked wonders with libhoard (and SmartHeap,
commercially). Some applications will run slower on SMP than if you
removed one of the processors.

www.hoard.org
www.microquill.com

mmm, graphs

-AB

[Aahz]

In article <ec*************************@posting.google.com> ,
Wenning Qiu <we*********@csgsystems.com> wrote:

My project will be running on an SMP box and requires scalability.
However, my test shows that Python threading has very poor performance
in terms of scaling. In fact it doesn't scale at all.
That's true for pure Python code.
The fundamental reason for lacking scalability is that Python uses a
global interpreter lock for thread safety. That global lock must be
held by a thread before it can safely access Python objects.
Correct. The problem is that the GIL makes Python more efficient in
many ways, because there's no need for fine-grained locking. You're
using Python inside-out for this purpose -- the way to scale Python in a
threaded environment is to call out to a C extension that releases the
GIL.
Has anyone on this list run into the same problem that I have, or does
anyone know of any plan of totally insulating multiple embedded Python
interpreters?

Sure! Use multiple processes.

Other people have mentioned Perl and Tcl in this thread. I wonder how
they deal with the problem of loading DLLs with static data.
--
Aahz (aa**@pythoncraft.com) <*> http://www.pythoncraft.com/

"Not everything in life has a clue in front of it...." --JMS

[Jeff Epler]

On Thu, Jul 10, 2003 at 03:54:14PM -0400, Aahz wrote:

Other people have mentioned Perl and Tcl in this thread. I wonder how
they deal with the problem of loading DLLs with static data.

As far as I know, tcl enforces a one interpreter to one thread requirement.
An extension should have only thread-local data, using a Tcl-supplied API.

Jeff

[Aahz]

In article <ma**********************************@python.org >,
Jeff Epler <je****@unpythonic.net> wrote:

On Thu, Jul 10, 2003 at 03:54:14PM -0400, Aahz wrote:

Other people have mentioned Perl and Tcl in this thread. I wonder how
they deal with the problem of loading DLLs with static data.

As far as I know, tcl enforces a one interpreter to one thread
requirement. An extension should have only thread-local data, using a
Tcl-supplied API.

What happens when Tcl wants to interact with some 3rd-party DLL that is
*not* thread-safe?
--
Aahz (aa**@pythoncraft.com) <*> http://www.pythoncraft.com/

"Not everything in life has a clue in front of it...." --JMS

[Jeff Epler]

On Thu, Jul 10, 2003 at 07:48:57PM -0400, Aahz wrote:

In article <ma**********************************@python.org >,
Jeff Epler <je****@unpythonic.net> wrote:

On Thu, Jul 10, 2003 at 03:54:14PM -0400, Aahz wrote:

Other people have mentioned Perl and Tcl in this thread. I wonder how
they deal with the problem of loading DLLs with static data.

As far as I know, tcl enforces a one interpreter to one thread
requirement. An extension should have only thread-local data, using a
Tcl-supplied API.

What happens when Tcl wants to interact with some 3rd-party DLL that is
*not* thread-safe?

I guess you'd have to do your own locking. Tcl has standard C APIs for
Conditions, Mutexes, and thread-specific data, see the Thread(3) manpage.
You'd have to surround all non-reentrant calls with Tcl_MutexLock(m)
.... Tcl_MutexUnlock(m). If two extensions wanted to use the same
non-thread-safe library, they'd have to cooperate in some way to use
the same 'm' to Tcl_Mutex*(). I don't know if there's a standard way to
do this, but I think that having the mutex defined in a shared lib they
both link might work.

Jeff

[Aahz]

In article <ma**********************************@python.org >,
Jeff Epler <je****@unpythonic.net> wrote:

On Thu, Jul 10, 2003 at 07:48:57PM -0400, Aahz wrote:

In article <ma**********************************@python.org >,
Jeff Epler <je****@unpythonic.net> wrote:

On Thu, Jul 10, 2003 at 03:54:14PM -0400, Aahz wrote:

Other people have mentioned Perl and Tcl in this thread. I wonder how
they deal with the problem of loading DLLs with static data.

As far as I know, tcl enforces a one interpreter to one thread
requirement. An extension should have only thread-local data, using a
Tcl-supplied API.

What happens when Tcl wants to interact with some 3rd-party DLL that is
*not* thread-safe?

I guess you'd have to do your own locking. Tcl has standard C APIs for
Conditions, Mutexes, and thread-specific data, see the Thread(3) manpage.
You'd have to surround all non-reentrant calls with Tcl_MutexLock(m)
... Tcl_MutexUnlock(m). If two extensions wanted to use the same
non-thread-safe library, they'd have to cooperate in some way to use
the same 'm' to Tcl_Mutex*(). I don't know if there's a standard way to
do this, but I think that having the mutex defined in a shared lib they
both link might work.

Yup. And that's exactly why there has been little movement to remove
the GIL from Python. One of Python's core strengths is the ease with
which random DLLs can be used from Python.
--
Aahz (aa**@pythoncraft.com) <*> http://www.pythoncraft.com/

"Not everything in life has a clue in front of it...." --JMS