473,903 Members | 6,147 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Python is faster than C

Hi!

This is a rant against the optimization trend of the Python interpreter.

Sorting a list of 100000 integers in random order takes:

* 0.75 seconds in Python 2.1
* 0.51 seconds in Python 2.2
* 0.46 seconds in Python 2.3

Tim Peters did a great job optimizing list.sort(). If I try with a
simple, non-stable pure Python quicksort implementation, in Python 2.3:

* 4.83 seconds
* 0.21 seconds with Psyco

First step towards world domination of high-level languages :-)

The reason that Psyco manages to outperform the C implementation is not
that gcc is a bad compiler (it is about 10 times better than Psyco's).
The reason is that the C implementation must use a generic '<' operator
to compare elements, while the Psyco version quickly figures out that it
can expect to find ints in the list; it still has to check this
assumption, but this is cheap and then the comparison is done with a
single machine instruction.

Similarily, here are some results about the heapq module, which is
rewritten in C in the CVS tree for Python 2.4:

l = [random.random() for x in range(200000)]
heapq.heapify(l )

This code executes on my laptop in:

* 1.96 seconds on Python 2.3 (pure Python)
* 0.18 seconds on Python 2.4cvs (rewritten in C)
* 0.16 seconds on Python 2.3 with Psyco

So this is not so much a plug for Psyco as a rant against the current
trend of rewriting standard modules in C. Premature optimization and
all that.

Worse, and more importantly, the optimization starts to become visible
to the programmer. Iterators, for example, are great in limited cases
but I consider their introduction a significant complication in the
language; before, you could expect that some function from which you
would expect a sequence returned a list. Python was all lists and
dicts, with dicts used as namespaces here and there. Nowadays you have
to be careful. Moreover, it is harder to explain:
zip([1,2,3], [4,5,6]) # easy to understand and explain [(1, 4), (2, 5), (3, 6)]
enumerate([6,7,8,9]) # uh ?

<enumerate object at 0x401a102c>

I know you can always do list(_). My point is that this is a
user-visible optimization. enumerate() should return a normal list, and
it should be someone else's job to ensure that it is correctly optimized
away if possible (and I'm not even talking about Psyco, it could be done
in the current Python implementation with a reasonable amount of
effort).
Protesting-ly yours,

Armin
Jul 18 '05
36 2687
"Michel Claveau/Hamster" <No********@No. Spam.mclaveau.N o.Spam.com> writes:
Yes, but Psyco is writted in C & Python, and it use an C module.
Said "Psyco is faster than C", it's like said "Psyco is faster than Psyco".


You appear to be missing the point completely.

The language used to implement the tools you use is irrelevant. What
is important is the language in which you are programming.

Armin's point is that (in certain circumstances) you can achieve a
significant speedup by one of two ways

- Recode some Python code in C

- import psyco; psyco.full()

Note: in the second case, this is _all_[*] you have to do; it takes
about 2 seconds of work. In the first case it take many
hours/days/weeks of tedious and error-prone work.
The point is: Why code in C when you can get just as good program
speed performance by coding in Python.

Put another way: Why code in a low-level, tedious, rigid,
error-inducing language, when you can get just as good program speed
performance by writing in a high-level, flexible, dynamic, pleasant
language.
[*] Yes, I'm deliberately keeping things simple in order not to draw
attention away from the real point.
Jul 18 '05 #31
Raymond Hettinger wrote:
...

P.S. If some experimentation shows your code to be useful at the
interactive prompt, please submit a patch on SF so it won't be lost.


Why put this behaviour in the interactive prompt rather than in repr()?

Paul Prescod

Jul 18 '05 #32
> Yes, but Psyco is writted in C & Python, and it use an C module.
Said "Psyco is faster than C", it's like said "Psyco is faster than Psyco".


You probably meant "C is faster than C". That is a reasonable
statement, but users of Psyco don't need to write anything special
beyond a handful of extra statements in Python.

To the user, Psyco is a module that makes things fast. They wrote
everything in Python, so to them, "Python with Psyco is faster than C"
is far more accurate.

- Josiah
Jul 18 '05 #33
Paul Rubin <http://ph****@NOSPAM.i nvalid> writes:
Armin Rigo <ar***@tunes.or g> writes:
Ideally: If you do x=range(100); x[50]='hi' then the interpreter first
builds this optimized range representation and assigns it to x; and when
in the next statement you modify this list x it says 'oops! i cannot do
that with this representation' , so it reverts to an array-like
representation (i.e. it creates all 100 elements) and then changes the
50th. No gain here. If on the other hand you only ever do 'easy'
things with your list, like iterate over it or read elements, then it
can all be done with the range representation, without falling back to
the array representation.


Maybe there is something to this.


Armin is usually worth listening too :-)

You can find a step towards many of these ideas in PyPy, which should
surprise no one at all...

Cheers,
mwh

--
Indeed, when I design my killer language, the identifiers "foo" and
"bar" will be reserved words, never used, and not even mentioned in
the reference manual. Any program using one will simply dump core
without comment. Multitudes will rejoice. -- Tim Peters, 29 Apr 1998
Jul 18 '05 #34
Raymond Hettinger:
Likewise, I program in a less hostile world than Andrew Dalke who has
to contend with pandora's box iterators which ruin the lives of mortal
men who think they can call .next() with impunity ;-)


I did say "Should be rare though." :)

Andrew
da***@dalkescie ntific.com

(And now you've got me thinking about the Tomb Raider II movie.
Not a bad thing.)
Jul 18 '05 #35
Michel Claveau/Hamster wrote:
Hi !

Yes, but Psyco is writted in C & Python, and it use an C module.
Said "Psyco is faster than C", it's like said "Psyco is faster than Psyco".


This is not so much of a contradiction.

First of all, Psyco creates assembly code. Not the fastest,
but it enters an area where Python has no entrance, yet.
So what it does is to produce faster code, although not as fast
as C could, but faster than the existing C implementation
of Python.

Seconde, depending on the algorithm you use, even Python can be
faster than C. For example, a few years ago, I implemented
a fast long integer multiplication in Python. At that time,
CPython still had asymptotic behavior of O(n**2), while
the Karatsuba algorithm I used hat something like O(2**1.53).
The difference showed up with several thousands of digits, but
it was remarkable.

Later on, Karatsuba was built into the core, and it became clear
to me that *I* caused this mess, because I had the chance to
get that algorithm into the core, long time ago. I just missed it.

What I'm trying to say: Finally it is always the algorithm.

ciao - chris

--
Christian Tismer :^) <mailto:ti****@ stackless.com>
Mission Impossible 5oftware : Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/
14109 Berlin : PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/
Jul 18 '05 #36
On 03 Apr 2004 18:23:11 -0800, Paul Rubin
<http://ph****@NOSPAM.i nvalid> wrote:
Armin Rigo <ar***@tunes.or g> writes:
Ideally: If you do x=range(100); x[50]='hi' then the interpreter first
builds this optimized range representation and assigns it to x; and when
in the next statement you modify this list x it says 'oops! i cannot do
that with this representation' , so it reverts to an array-like
representation (i.e. it creates all 100 elements) and then changes the
50th. No gain here. If on the other hand you only ever do 'easy'
things with your list, like iterate over it or read elements, then it
can all be done with the range representation, without falling back to
the array representation.


Maybe there is something to this.


The problem is, once you start where do you stop.

At the moment, Armin suggests optimising the a list consisting of a
single repeating item. But what about optimising a repeating list with
one or two special cases, so that the "x[50]='hi'" above doesn't incur
a penalty?

Consider integer lists - what about optimising arithetic progressions?
Geometric progressions? Progressions with special cases? Progressions
that are the intersection, or union, or whatever of other
progressions?

If the internal implementation doesn't special-case the implementation
of operations on these, all you have is lazy evaluation. But if the
internal implementation adds special-case implementations of
operations, you either end up with an huge number of special case
implementation methods (other parameters can end up with special-case
implementations , not just 'self') or you have to implement what
amounts to a full algebraic solving engine in the Python interpreter.

Or else you can just choose to special case the really important types
and operations, which I believe Python already does to some degree (an
integer is a special case of a long integer, for instance, and an
iterator is a special case of a list with only a subset of the
operations available to a standard list) and provide the programmer
with the tools to easily implement any further special cases that he
may need.
--
Steve Horne

steve at ninereeds dot fsnet dot co dot uk
Jul 18 '05 #37

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

49
2898
by: Ville Vainio | last post by:
I don't know if you have seen this before, but here goes: http://text.userlinux.com/white_paper.html There is a jab at Python, though, mentioning that Ruby is more "refined". -- Ville Vainio http://www.students.tut.fi/~vainio24
38
3760
by: kbass | last post by:
In different articles that I have read, persons have constantly eluded to the productivity gains of Python. One person stated that Python's productivity gain was 5 to 10 times over Java in some in some cases. The strange thing that I have noticed is that there were no examples of this productivity gain (i.e., projects, programs, etc.,...). Can someone give me some real life examples of productivity gains using Python as opposed other...
7
2074
by: stormslayer | last post by:
Folks: I've been considering a shift to python. I currently use c++builder (borland) or perl. I do computational models / statistics programming, and was interested in python b/c it a. has a library that connects to the R stats package b. has code that seems way more readable than anything else There is, however, at least for my work, a hard constraint. Execution
114
9927
by: Maurice LING | last post by:
This may be a dumb thing to ask, but besides the penalty for dynamic typing, is there any other real reasons that Python is slower than Java? maurice
68
5919
by: Lad | last post by:
Is anyone capable of providing Python advantages over PHP if there are any? Cheers, L.
9
4534
by: Dieter Vanderelst | last post by:
Dear all, I'm currently comparing Python versus Perl to use in a project that involved a lot of text processing. I'm trying to determine what the most efficient language would be for our purposes. I have to admit that, although I'm very familiar with Python, I'm complete Perl noob (and I hope to stay one) which is reflected in my questions. I know that the web offers a lot of resources on Python/Perl differences. But I couldn't find a...
118
6792
by: 63q2o4i02 | last post by:
Hi, I've been thinking about Python vs. Lisp. I've been learning Python the past few months and like it very much. A few years ago I had an AI class where we had to use Lisp, and I absolutely hated it, having learned C++ a few years prior. They didn't teach Lisp at all and instead expected us to learn on our own. I wasn't aware I had to uproot my thought process to "get" it and wound up feeling like a moron. In learning Python I've...
0
273
by: Kurt B. Kaiser | last post by:
Patch / Bug Summary ___________________ Patches : 419 open ( +3) / 3410 closed ( +2) / 3829 total ( +5) Bugs : 910 open (+12) / 6185 closed ( +5) / 7095 total (+17) RFE : 235 open ( +1) / 238 closed ( +0) / 473 total ( +1) New / Reopened Patches ______________________
0
1696
by: Kurt B. Kaiser | last post by:
Patch / Bug Summary ___________________ Patches : 420 open ( +4) / 3410 closed ( +2) / 3830 total ( +6) Bugs : 915 open (+17) / 6186 closed ( +6) / 7101 total (+23) RFE : 235 open ( +1) / 238 closed ( +0) / 473 total ( +1) New / Reopened Patches ______________________
23
2560
by: Python Maniac | last post by:
I am new to Python however I would like some feedback from those who know more about Python than I do at this time. def scrambleLine(line): s = '' for c in line: s += chr(ord(c) | 0x80) return s def descrambleLine(line):
0
9999
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, weíll explore What is ONU, What Is Router, ONU & Routerís main usage, and What is the difference between ONU and Router. Letís take a closer look ! Part I. Meaning of...
0
9848
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10876
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
10501
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7208
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5894
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
6094
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4727
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
4308
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.