473,544 Members | 1,778 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Will Python 3.0 remove the global interpreter lock (GIL)

I'm afraid that the GIL is killing the usefullness of python for some
types of applications now where 4,8 oder 64 threads on a chip are here
or comming soon.

What is the status about that for the future of python?

I know that at the moment allmost nobody in the scripting world has
solved this problem, but it bites and it bites hard. Only groovy as a
Java Plugin has support but i never tried it. Writing an interpreter
that does MT this seems to be extremely difficult to do it right, with
lots of advanced stuff like CAS and lock free programming.

Even Smalltalk and Common Lisp didn't get it until know (with the
exception of certain experiments).

Sep 3 '07 #1
33 3673
On 9/2/07, llothar <ll*****@web.de wrote:
I'm afraid that the GIL is killing the usefullness of python for some
types of applications now where 4,8 oder 64 threads on a chip are here
or comming soon.

What is the status about that for the future of python?

I know that at the moment allmost nobody in the scripting world has
solved this problem, but it bites and it bites hard. Only groovy as a
Java Plugin has support but i never tried it. Writing an interpreter
that does MT this seems to be extremely difficult to do it right, with
lots of advanced stuff like CAS and lock free programming.

Even Smalltalk and Common Lisp didn't get it until know (with the
exception of certain experiments).

No. http://www.artima.com/weblogs/viewpo...?thread=211430

--
http://www.advogato.org/person/eopadoan/
Bookmarks: http://del.icio.us/edcrypt
Sep 3 '07 #2
On 3 Sep., 07:38, "Eduardo O. Padoan" <eduardo.pad... @gmail.com>
wrote:
No.http://www.artima.com/weblogs/viewpo...?thread=211430

Ops, I meant:http://www.artima.com/forums/threade...&thread=211200
Thanks. I whish there would be a project for rewritting the C
interpreter
to make it better and more useable for threading use.

But the CPU infrastructure is also not perfect enough so maybe it's
good to
wait with this a few more years until Intel and AMD know what they are
doing.
Sep 3 '07 #4
On Sep 2, 11:16 pm, llothar <llot...@web.de wrote:
On 3 Sep., 07:38, "Eduardo O. Padoan" <eduardo.pad... @gmail.com>
wrote:
No.http://www.artima.com/weblogs/viewpo...?thread=211430
Ops, I meant:http://www.artima.com/forums/threade...&thread=211200

Thanks. I whish there would be a project for rewritting the C
interpreter
to make it better and more useable for threading use.

But the CPU infrastructure is also not perfect enough so maybe it's
good to
wait with this a few more years until Intel and AMD know what they are
doing.

I read somewhere that PYPY won't have the interpreter lock (I may be
wrong though).
Check it out: http://codespeak.net/pypy
Sep 3 '07 #5
On Sun, 2007-09-02 at 17:21 -0700, llothar wrote:
I'm afraid that the GIL is killing the usefullness of python for some
types of applications now where 4,8 oder 64 threads on a chip are here
or comming soon.

What is the status about that for the future of python?
The GIL is an implementation specific issue with CPython. It will not be
removed in CPython for the forseeable future, but you can already get a
GIL-free interpreter with Jython and IronPython. AFAIK there are plans
to remove the GIL in PyPy.

According to the last PyPy release announcement, they're running at
about half the speed of CPython, and have a preliminary JIT that can
translate certain integer operations into assembly, and will be expanded
upon in future releases. If you're looking for a progressive alternative
to CPython, I'd keep an eye on that project ;-)

--
Evan Klitzke <ev**@yelp.co m>

Sep 3 '07 #6
On Sep 3, 2:21 am, llothar <llot...@web.de wrote:
I'm afraid that the GIL is killing the usefullness of python for some
types of applications now where 4,8 oder 64 threads on a chip are here
or comming soon.

What is the status about that for the future of python?
This is FAQ. You will find thousands of discussion on the net about
that.
My personal opinion (and I am not the only one in the Python
community) is that
if you want to scale the way to go is to use processes, not threads,
so removing the GIL would be a waste of effort anyway.
Look at the 'processing' module in PyPI.

Michele Simionato

Sep 3 '07 #7
On Sep 3, 9:15 am, Michele Simionato <michele.simion ...@gmail.com>
wrote:
On Sep 3, 2:21 am, llothar <llot...@web.de wrote:
My personal opinion (and I am not the only one in the Python
community) is that
if you want to scale the way to go is to use processes, not threads,
so removing the GIL would be a waste of effort anyway.
Look at the 'processing' module in PyPI.

Michele Simionato

I second that. You may also look here,
http://www.parallelpython.com/

I tested it and work as expected. You can see all your processing-
cores work nicely and balanced.

Sep 3 '07 #8
Michele Simionato <mi************ ***@gmail.comwr ites:
On Sep 3, 2:21 am, llothar <llot...@web.de wrote:
I'm afraid that the GIL is killing the usefullness of python for
some types of applications now where 4,8 oder 64 threads on a chip
are here or comming soon.

This is FAQ. You will find thousands of discussion on the net about
that.
My personal opinion (and I am not the only one in the Python
community) is that if you want to scale the way to go is to use
processes, not threads, so removing the GIL would be a waste of
effort anyway.
Yes. Processes are cheap on well-designed operating systems, and using
processes to subdivide your processor usage encourages simple, modular
interfaces between parts of a program. Threads, while also cheap, are
much more difficult and fiddly to program correctly, hence are rarely
an overall win.

One common response to that is "Processes are expensive on Win32". My
response to that is that if you're programming on Win32 and expecting
the application to scale well, you already have problems that must
first be addressed that are far more fundamental than the GIL.

--
\ "Pinky, are you pondering what I'm pondering?" "Umm, I think |
`\ so, Brain, but three men in a tub? Ooh, that's unsanitary!" -- |
_o__) _Pinky and The Brain_ |
Ben Finney
Sep 3 '07 #9
I was wondering (and maybe i still do) about this GIL "problem". I am
relatively new to Python (less than a year) and when i started to
think about it i said: "Oh, this IS a problem". But when i dig a
little more, i found that "Ah, maybe it isn't".
I strongly believe that the best usage of multiple cores processor
will be achieved if programming languages are modified to support this
on their "hearts". Code blocks that would be identified by the
compiler and run in parallel and such things. Laboratories are working
on these stuff but i do not expect something in the very-near future.

So, as i mentioned above, there are solutions for that right now
("parallel python" and others) that enabled us with little effort to
spawn a new python interpreter, thus allowing the OS to schedule it on
a different core and do the job this way relatively cheap.
I wouldn't recommend going to IronPython despite the fact that the CLR
better utilize MP. The reason for this is that i would NEVER give up
the freedom that CPython gives me by exchange "better" usage of the MP
and platform lock-in.

Sep 3 '07 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

7
4488
by: Wenning Qiu | last post by:
I am researching issues related to emdedding Python in C++ for a project. My project will be running on an SMP box and requires scalability. However, my test shows that Python threading has very poor performance in terms of scaling. In fact it doesn't scale at all. I wrote a simple test program to complete given number of iterations of...
65
6666
by: Anthony_Barker | last post by:
I have been reading a book about the evolution of the Basic programming language. The author states that Basic - particularly Microsoft's version is full of compromises which crept in along the language's 30+ year evolution. What to you think python largest compromises are? The three that come to my mind are significant whitespace,...
1
3696
by: amit | last post by:
Hello, I am embedding a python script in a C++ application. The script can be called simultaneously from multiple threads. What is the correct way to implement this situation: 1) Have unique python interpreter instantiations ( Py_Initialize() ) for each thread.
1
2085
by: Carl Waldbieser | last post by:
I have been considering using Python and the Reportlab library for generating PDF reports for the back-end of a web based application. The application runs most of its background tasks on a dedicated server that is Windows based (Win2K or Win2k3). The program that launches the tasks requires a COM-based interface, so I wrote a Python COM...
0
361
by: Kurt B. Kaiser | last post by:
Patch / Bug Summary ___________________ Patches : 378 open ( +3) / 3298 closed (+34) / 3676 total (+37) Bugs : 886 open (-24) / 5926 closed (+75) / 6812 total (+51) RFE : 224 open ( +7) / 227 closed ( +7) / 451 total (+14) New / Reopened Patches ______________________
41
2309
by: km | last post by:
Hi all, Is there any PEP to introduce true threading features into python's next version as in java? i mean without having GIL. when compared to other languages, python is fun to code but i feel its is lacking behind in threading regards, KM
10
3969
by: Louise Hoffman | last post by:
Dear readers, I was wondering, if Python in the foerseeable future will allocate one CPU core just for the interpreter, so heavy Python operations does slow down the OS? It seams to me like a perfect use for a CPU core =) Lots of love, Louise
6
2930
by: nikhilketkar | last post by:
What are the implications of the Global Interpreter Lock in Python ? Does this mean that Python threads cannot exploit a dual core processor and the only advantage of using threads is in that computation and IO-bound operations can continue in parallel ? Thanks, Nikhil
7
12497
by: skip | last post by:
This question was posed to me today. Given a C/C++ program we can clearly embed a Python interpreter in it. Is it possible to fire up multiple interpreters in multiple threads? For example: C++ main thread 1 Py_Initialize() thread 2 Py_Initialize()
0
7414
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
7757
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
1
7360
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
7699
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
1
5288
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
4905
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3400
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
1
1833
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
0
651
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.