Clarity: GIL, processes and CPUs etc

[adsheehan]

I have been reading many of the posting on the GIL and impact on
threading etc.
I have found is confusing and would welcome some clarity on this.

I understand that embedding the interpreter in a C/C++ application
limits it to one CPU.
If the application is multi-threaded (system threads) in will not use
additional CPUs as the interpreter is tied to one CPU courtesy of the
GIL.
True or False?

I understand that forking or running multiple process instances of the
above application would make use of multiple CPUs. This is because each
process would have its own interpreter and GIL that is independent of
any other process.
True or False?

Can anyone clarify the above?

Thanks
A

[Gerhard Häring]

ad*******@eircom.net wrote:

I have been reading many of the posting on the GIL and impact on
threading etc. I have found is confusing and would welcome some
clarity on this.

I understand that embedding the interpreter in a C/C++ application
limits it to one CPU. If the application is multi-threaded (system
threads) in will not use additional CPUs as the interpreter is tied
to one CPU courtesy of the GIL. True or False?
True. More than one CPU can be used if the GIL is released during calls
to external libraries, though. Well-written C extension modules and the
Python standard library do this where appropriate. Still only one thread
can execute pure Python code at a time.
I understand that forking or running multiple process instances of the
above application would make use of multiple CPUs. This is because each
process would have its own interpreter and GIL that is independent of
any other process.
True or False? [...]

True.

HTH,

-- Gerhard

[Nick Smallbone]

ad*******@eircom.net wrote:

I have been reading many of the posting on the GIL and impact on
threading etc.
I have found is confusing and would welcome some clarity on this.

I understand that embedding the interpreter in a C/C++ application
limits it to one CPU.
If the application is multi-threaded (system threads) in will not use
additional CPUs as the interpreter is tied to one CPU courtesy of the
GIL.
True or False?

As I understand it, the interpreter needs the GIL held in order to run.
That means that you can only run Python code on one CPU at once, but
plain C/C++ code can be multithreaded.
I understand that forking or running multiple process instances of the
above application would make use of multiple CPUs. This is because each
process would have its own interpreter and GIL that is independent of
any other process.
True or False?

I suppose it would, yes. But this is just like running multiple copies
of Python, in that you won't be able to use PyObject *s from one
instance in another directly, so the multiple processes would have to
communicate by sockets or IPC or some other way.

[Alan Kennedy]

[ad*******@eircom.net]

I understand that embedding the interpreter in a C/C++ application
limits it to one CPU.
If the application is multi-threaded (system threads) in will not use
additional CPUs as the interpreter is tied to one CPU courtesy of the
GIL.
True or False?
True, only when a thread is inside pure python code. C-level extensions
and library modules, e.g. I/O modules, release the GIL, thus permitting
other threads in the same process to run simultaneously. But only one
thread can be running *inside the python interpreter* at a time.
I understand that forking or running multiple process instances of the
above application would make use of multiple CPUs. This is because each
process would have its own interpreter and GIL that is independent of
any other process.
True or False?

True. Every separate process will have its own python interpreter,
meaning it has its own GIL. Python code running in multiple processes
can execute truly simultaneously.

So you can run pure python code simultaneously on multiple cpus on a
multi-cpu box by using multiple independent processes. But if your
processes need to communicate, then you need to de/serialise
objects/parameters for transmission between those processes.

--
alan kennedy
------------------------------------------------------
email alan: http://xhaus.com/contact/alan