multithreading concept

[S.Mohideen]

Hi Folks,

Python is praised about - me too. But at one instance it fails. It fails to
behave as a true multi-threaded application. That means utilizing all the
CPUs parallely in the SMP efficiently stays as a dream for a Python
Programmer.

Discussion threads say its due to GIL - global interpreter lock. But nobody
has mentioned any alternative to that apart from suggestions like "Code it
in C" and POSH (http://poshmodule.sf.net). Is there any other way we can
make Python programs really multithreaded in real sense.

Moin

[Paddy]

On Feb 7, 1:53 am, "S.Mohideen" <[e-mail]>
wrote:

Hi Folks,

Python is praised about - me too. But at one instance it fails. It fails to
behave as a true multi-threaded application. That means utilizing all the
CPUs parallely in the SMP efficiently stays as a dream for a Python
Programmer.

Discussion threads say its due to GIL - global interpreter lock. But nobody
has mentioned any alternative to that apart from suggestions like "Code it
in C" and POSH (http://poshmodule.sf.net). Is there any other way we can
make Python programs really multithreaded in real sense.

Moin

Actually their are a *lot* more suggestions & discussions to be found.
I myself move towards the "parallel processing is difficult. If you
think it's easy then your either lucky or theorising. Whilst it would
be nice to have threads==native threads for completeness sake, I'm
quit happy to run concurrent communicating processes, as on my
machines the OS helps me to see what's happening to the processes, and
stops processes trampling over shared data".

-Paddy.

[sturlamolden]

On Feb 7, 2:53 am, "S.Mohideen" <[e-mail]>
wrote:

Python is praised about - me too. But at one instance it fails. It fails to
behave as a true multi-threaded application. That means utilizing all the
CPUs parallely in the SMP efficiently stays as a dream for a Python
Programmer.

This has been discussed to death before. Win32 threads and pthreads
(which is what Python normally uses, depending on the platform) are
designed to stay idle most of the time. They are therefore not a tool
for utilizing the power of multiple CPUs, but rather make certain kind
of programming tasks easier to program (i.e. non-blocking I/O,
responsive UIs). The GIL is not a problem in this context. If threads
stay idle most of the time, the GIL does not harm.

If you want to utilize the computing power of multiple CPUs, you
should use multiple processes instead of threads. On Python this is
mandatory due to the GIL. In any other language it it highly
recommended. The de-factor standard for parallel multiprocessing (MPI)
uses multiple processes, even on SMPs. Anyone with serious intentions
of using multiple processors for parallel computing should use
multiple processes and fast IPC - not multiple threads, shared memory
and synchronization objects - even if the language is plain C. With
multiple threads, a lot of time is wasted doing context switches and
book keeping for the thread synchronization. In addition, obscure and
often very difficult to identify bugs are introduced.

There are a Python binding for MPI (mpi4py) and a similar pure Python
project (Parallel Python) that will take care of all these details for
you.

Discussion threads say its due to GIL - global interpreter lock. But nobody
has mentioned any alternative to that apart from suggestions like "Code it
in C" and POSH (http://poshmodule.sf.net). Is there any other way we can
make Python programs really multithreaded in real sense.

As mentioned, use MPI or Parallel Python. MPI is by far the more
mature, but Parallel Python could be easier for a pythoneer.
Multithreading has different use.

[Paul Boddie]

On 7 Feb, 02:53, "S.Mohideen" <[e-mail]>
wrote:

>
Python is praised about - me too. But at one instance it fails. It fails to
behave as a true multi-threaded application. That means utilizing all the
CPUs parallely in the SMP efficiently stays as a dream for a Python
Programmer.

Take a look at the Python Wiki for information on parallel processing
with Python:

http://wiki.python.org/moin/ParallelProcessing

Pure CPython code may not be able to use more than one CPU merely
through the use of threads (Jython and IronPython are different,
though), but using all the CPUs or cores in an SMP system is not
exactly a mere dream, as many of the projects listed on the above page
demonstrate.

Paul

[John Nagle]

sturlamolden wrote:

On Feb 7, 2:53 am, "S.Mohideen" <[e-mail]>
wrote:
This has been discussed to death before. Win32 threads and pthreads
(which is what Python normally uses, depending on the platform) are
designed to stay idle most of the time. They are therefore not a tool
for utilizing the power of multiple CPUs, but rather make certain kind
of programming tasks easier to program (i.e. non-blocking I/O,
responsive UIs).

Multithread compute-bound programs on multiple CPUs are
how you get heavy number-crunching work done on multiprocessors.
Of course, that's not something you use Python for, at least not
until it gets a real compiler.

It's also the direction games are going. The XBox 360 forced
game developers to go that way, since it's a 3-CPU shared memory
multiprocessor. That translates directly to multicore desktops
and laptops.

I went to a talk at Stanford last week by one of Intel's
CPU architects, and he said we're going have hundreds of
CPUs per chip reasonably soon. Python needs to get ready.

John Nagle

[Steve Holden]

John Nagle wrote:

sturlamolden wrote:

>On Feb 7, 2:53 am, "S.Mohideen" <[e-mail]>
wrote:
This has been discussed to death before. Win32 threads and pthreads
(which is what Python normally uses, depending on the platform) are
designed to stay idle most of the time. They are therefore not a tool
for utilizing the power of multiple CPUs, but rather make certain kind
of programming tasks easier to program (i.e. non-blocking I/O,
responsive UIs).

Multithread compute-bound programs on multiple CPUs are
how you get heavy number-crunching work done on multiprocessors.
Of course, that's not something you use Python for, at least not
until it gets a real compiler.

It's also the direction games are going. The XBox 360 forced
game developers to go that way, since it's a 3-CPU shared memory
multiprocessor. That translates directly to multicore desktops
and laptops.

I went to a talk at Stanford last week by one of Intel's
CPU architects, and he said we're going have hundreds of
CPUs per chip reasonably soon. Python needs to get ready.

Define "Python". Does "it" include you? What does it need to do to get
ready. How do you plan to help?

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Blog of Note: http://holdenweb.blogspot.com
See you at PyCon? http://us.pycon.org/TX2007

[sturlamolden]

On Feb 7, 6:17 pm, John Nagle <[e-mail]:

Multithread compute-bound programs on multiple CPUs are
how you get heavy number-crunching work done on multiprocessors.

In the scientific community, heavy CPU-bound tasks are either
parallelized using MPI and/or written in Fortran 90/95 and
parallelized using an expensive vectorizing compiler.

Of course, that's not something you use Python for, at least not
until it gets a real compiler.

That is also not correct:

1. Using Python does not change the complexity of the algorithm. Big-O
is still the same, and Big-O is still the main determinant of
performance.

2. I value my own time more than extra CPU cycles (and so does those
who pay my salary). If "Python is to slow", it is less expensive to
compensate by using more CPUs than using a less productive language
like Java or C++.

3. Only isolated bottlenecks really gain from being statically
compiled. These are usually very small parts of the program. They can
be identified with a profiler (intuition usually do not work very well
here) and rewritten in Pyrex, Fortran 95, C or assembly.

4. There is NumPy and SciPy, which can make Python fast enough for
most CPU-bound tasks. http://www.scipy.org/PerformancePython

5. "Premature optimization is the root of all evil in computer
science." (Donald Knuth)

6. Pyrex (the compiler you asked for) does actually exist.
C and Fortran compilers can produce efficient code because they know
the type of each variable. We have do a Python compiler that can do
the same thing. It is called 'Pyrex' and extends Python with static
types. Pyrex can therefore produce code that are just as efficient as
hand-tuned C (see the link above). One can take the bad-performing
Python code, add type declarations to the variables that Pyrex needs
to generate efficient code (but all variables need not be declared),
and leave the rest to the compiler. But this is only required for very
small portions of the code. Transforming time-critical Python code to
Pyrex is child's play. "First make it work, then make it fast."

At the University of Oslo, the HPC centre has been running Python
courses for its clients. Python does not perform any worse than C or
Fortran, one just has to know (1) how to use it, (2) when to use it,
and (3) when not to use it.

99% of benchmarks showing bad performance with Python is due to
programmers not understanding which operations are expensive in
interpreted languages, and trying to use Python as if it were C++. The
typical example would be code that use a loop instead of using the
built-in function 'map' or a vectorized array expression with NumPy.

It's also the direction games are going.

I believe that is due to ignorance. Threads are implemented to be in
an idle blocking state 99% of the time.

The XBox 360 forced
game developers to go that way, since it's a 3-CPU shared memory
multiprocessor. That translates directly to multicore desktops
and laptops.

MPI works on SMPs.

MPI does not use threads on SMPs because it performs worse than using
multiple processes.

[Sergei Organov]

"sturlamolden" <st**********@yahoo.nowrites:

On Feb 7, 6:17 pm, John Nagle <[e-mail]:

[...]

MPI does not use threads on SMPs because it performs worse than using
multiple processes.

I fail to see how threads in general could perform worse than
processes. I do understand that processes are inherently more
safe/secure, but when it comes to speed I really can't imagine why it
could happen that threads perform worse (poor threads implementation and
programming practices aside). Care to give some references?

-- Sergei.

[sturlamolden]

On Feb 7, 8:03 pm, Sergei Organov <[e-mail]:

I fail to see how threads in general could perform worse than
processes. I do understand that processes are inherently more
safe/secure, but when it comes to speed I really can't imagine why it
could happen that threads perform worse (poor threads implementation and
programming practices aside). Care to give some references?

I believe Nick Maclaren explained that to you (and me) on January 10
and 11 this group. As far as I have understood the issue, it has to do
with poor threads implementations. Look that up on Google groups and
re-read the discussion (or ask Nick Maclaren as he is far more
competent than me).

http://groups.google.com/group/comp....32083cdc8bc44b

[Carl J. Van Arsdall]

Paul Boddie wrote:

[snip]

Take a look at the Python Wiki for information on parallel processing
with Python:

http://wiki.python.org/moin/ParallelProcessing

What a great resource! That one is book marked for sure. I was
wondering if anyone here had any opinions on some of the technologies
listed in there. I've used a couple, but there are some that I've never
seen before. In particular, has anyone used rthread before? It looks
like something I may use (now orwhen it matures), are there opinions on it?

Under the cluster computing section, has anyone tried any of the other
technologies? I've only used Pyro and i love it, but I'd like opinions
and experiences with other technologies if anyone has anything to say.

-c
--

Carl J. Van Arsdall
cv*********@mvista.com
Build and Release
MontaVista Software

[S.Mohideen]

I would like to add my problem in this thread.
I have a network application in Python which sends and recv using a single
socket.
There is a dictionary on which I store/read data values. I want to seperate
the send and recv functionality on two different processes so that the
parallel execution becomes fast. Is there any way to do so, so that the
Dict's consitency is not lost(able to read & write) and also the performance
improves. I am looking upon the MPI4Py module to see if it does the job for
me. Any ideas would be appreciated.

----- Original Message -----
From: "Sergei Organov" <os*@javad.com>
To: <py*********@python.org>
Sent: Wednesday, February 07, 2007 1:03 PM
Subject: Re: multithreading concept

"sturlamolden" <st**********@yahoo.nowrites:

>On Feb 7, 6:17 pm, John Nagle <[e-mail]:

[...]

>MPI does not use threads on SMPs because it performs worse than using
multiple processes.

I fail to see how threads in general could perform worse than
processes. I do understand that processes are inherently more
safe/secure, but when it comes to speed I really can't imagine why it
could happen that threads perform worse (poor threads implementation and
programming practices aside). Care to give some references?

-- Sergei.

--
http://mail.python.org/mailman/listinfo/python-list

[Carl J. Van Arsdall]

S.Mohideen wrote:

I would like to add my problem in this thread.
I have a network application in Python which sends and recv using a single
socket.
There is a dictionary on which I store/read data values. I want to seperate
the send and recv functionality on two different processes so that the
parallel execution becomes fast. Is there any way to do so, so that the
Dict's consitency is not lost(able to read & write) and also the performance
improves. I am looking upon the MPI4Py module to see if it does the job for
me. Any ideas would be appreciated.

Well, from your description so far I think that MPI is going to be a bit
of overkill. I think you should consider threads or processors with
shared memory/objects (POSH). Then take a look at a producer/consumer
program to see how it works, that should get you to where you need to go.

HTH

-carl

--

Carl J. Van Arsdall
cv*********@mvista.com
Build and Release
MontaVista Software

[Bjoern Schliessmann]

S.Mohideen wrote:

There is a dictionary on which I store/read data values. I want to
seperate the send and recv functionality on two different
processes so that the parallel execution becomes fast.

What makes you think that'll be faster?

Remember:
- If you have one CPU, there is no parallelity at all.
- If you do have multiple CPUs but only one network device, there is
no parallel networking.

Regards,
Björn

--
BOFH excuse #188:

...disk or the processor is on fire.

[Carl J. Van Arsdall]

Bjoern Schliessmann wrote:

[snip]
What makes you think that'll be faster?

Remember:
- If you have one CPU, there is no parallelity at all.
- If you do have multiple CPUs but only one network device, there is
no parallel networking.

Not necessarily, if he's on a full duplex ethernet connection, then
there is some parallelity he can take advantage of. He has upstream and
downstream.

-c

--

Carl J. Van Arsdall
cv*********@mvista.com
Build and Release
MontaVista Software

[S.Mohideen]

I am sorry if I sound foolish.
Suppose I split my Net application code using parallel python into several
processes based upon the number of CPU available. That means a single socket
descriptor is distributed across all processes. Is parallelity can be
acheived using the processes send/recv on the single socket multiplexed
across all the processes.. I haven't tried it yet - would like to have any
past experience related to this.

----- Original Message -----
From: "Carl J. Van Arsdall" <cv*********@mvista.com>
To: <py*********@python.org>
Sent: Thursday, February 08, 2007 3:44 PM
Subject: Re: multithreading concept

Bjoern Schliessmann wrote:

>[snip]
What makes you think that'll be faster?

Remember:
- If you have one CPU, there is no parallelity at all.
- If you do have multiple CPUs but only one network device, there is
no parallel networking.

Not necessarily, if he's on a full duplex ethernet connection, then
there is some parallelity he can take advantage of. He has upstream and
downstream.

-c

--

Carl J. Van Arsdall
cv*********@mvista.com
Build and Release
MontaVista Software

--
http://mail.python.org/mailman/listinfo/python-list

[Paul Rubin]

"S.Mohideen" <mo**@blackhole.labs.rootshell.wswrites:

Suppose I split my Net application code using parallel python into
several processes based upon the number of CPU available. That means a
single socket descriptor is distributed across all processes. Is
parallelity can be acheived using the processes send/recv on the
single socket multiplexed across all the processes.. I haven't tried
it yet - would like to have any past experience related to this.

In Linux, you can open the socket before forking and then use it in
the child processes; there is also a way to pass open sockets from one
process to another, but the Python lib currently does not support that
feature. It's worth adding and there's an open RFE for it, but it
hasn't been important enough that anyone's bothered coding it so far.

[sturlamolden]

On Feb 9, 4:00 pm, "S.Mohideen" <[e-mail]>
wrote:

I am sorry if I sound foolish.
Suppose I split my Net application code using parallel python into several
processes based upon the number of CPU available. That means a single socket
descriptor is distributed across all processes. Is parallelity can be
acheived using the processes send/recv on the single socket multiplexed
across all the processes.. I haven't tried it yet - would like to have any
past experience related to this.

Is CPU or network the speed limiting factor in your application? There
are two kinds of problems: You have a 'CPU-bound problem' if you need
to worry about 'flops'. You have an 'I/O bound' problem if you worry
about 'bits per second'.

If your application is I/O bound, adding more CPUs to the task will
not help. The network connection does not become any faster just
because two CPUs share the few computations that need to be performed.
Python releases the GIL around all i/o operations in the standard
library, such as reading from a socket or writing to socket. If this
is what you need to 'parallelize', you can just use threads and ignore
the GIL. Python's threads can handle concurrent I/O perfectly well.
Remember that Google and YouTube use Python, and the GIL is not a show
stopper for them.

The GIL locks the process to one CPU. You need to get around this if
the power of one CPU or CPU core limits the speed of the application.
This can be the case in e.g. digital image processing, certain
computer games, and scientific programming. I have yet to see a CPU-
bound 'Net application', though.

If you are running Windows: take a look at the CPU usage in the task
manager. Does it say that one of the CPUs is running at full speed for
longer periods of time? If not, there is noting to gained from using
multiple CPUs.

[Bjoern Schliessmann]

Carl J. Van Arsdall wrote:

Not necessarily, if he's on a full duplex ethernet connection,
then there is some parallelity he can take advantage of. He has
upstream and downstream.

Partly agreed. There is one bus to the network device, and CPU
should be very much faster than the network device itself, so I
estimate there'll be no gain.

Regards,
Björn

--
BOFH excuse #353:

Second-system effect.

[Bryan Olson]

sturlamolden wrote:
[...]

If you want to utilize the computing power of multiple CPUs, you
should use multiple processes instead of threads. On Python this is
mandatory due to the GIL. In any other language it it highly
recommended. The de-factor standard for parallel multiprocessing (MPI)
uses multiple processes, even on SMPs.

That doesn't really work in Python. There have been projects to
allow Pythonic coordination of processes -- POSH had some good
ideas -- but none have reached fruition.

There's nothing like a close thing to a good defacto standard in
the area. Microsoft's Win32 threads can claim to get as close as
anything.
--
--Bryan

[Paul Boddie]

On 8 Mar, 10:48, Bryan Olson <[e-mail]:

>
That doesn't really work in Python. There have been projects to
allow Pythonic coordination of processes -- POSH had some good
ideas -- but none have reached fruition.

What makes all of the following not "Pythonic"...?

http://wiki.python.org/moin/ParallelProcessing

Things like the CSP paradigm have sort of made their way into the
Python language itself, via enhancements to the yield keyword, which
has the dubious distinction of being a keyword which appears to return
a value. I'm sure one could define "Pythonic" as being "you can write
code like you do now (but not like any of the ways encouraged by the
aforementioned solutions) and it just works over multiple processors/
cores", but that's a view which is somewhat detached from the
practicalities (and favoured practices) of concurrent programming,
especially given the few guarantees Python would be able to provide to
make such a thing work effectively.

Paul

[Paul Rubin]

"Paul Boddie" <pa**@boddie.org.ukwrites:

What makes all of the following not "Pythonic"...?
http://wiki.python.org/moin/ParallelProcessing

I'd say mainly that they don't allow sharing data between processes
except through expensive IPC mechanisms involving system calls.

I'm sure one could define "Pythonic" as being "you can write
code like you do now (but not like any of the ways encouraged by the
aforementioned solutions) and it just works over multiple processors/
cores", but that's a view which is somewhat detached from the
practicalities (and favoured practices) of concurrent programming,
especially given the few guarantees Python would be able to provide to
make such a thing work effectively.

Really, the existence of the GIL comes as an unpleasant surprise to
progrmamers used to multi-threaded programming in other languages
whose synchronization features outwardly look about the same as
Python's. Somehow those other languages manage to use multiple CPU's
based on those features, without needing a GIL. We are looking at a
Python implementation wart, not "practicalities" inherent in the
nature of concurrency.