By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
459,203 Members | 1,623 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 459,203 IT Pros & Developers. It's quick & easy.

Is a "real" C-Python possible?

P: n/a
I understand that the standard Python distribution is considered
the C-Python. Howerver, the current C-Python is really a combination
of C and Python implementation. There are about 2000 Python files
included in the Windows version of Python distribution. I'm not sure
how much of the C-Python is implemented in C but I think the more
modules implemented in C, the better performance and lower memory
footprint it will get.

I wonder if it's possible to have a Python that's completely (or at
least for the most part) implemented in C, just like PHP - I think
this is where PHP gets its performance advantage. Or maybe I'm wrong
because the core modules that matter are already in C and those Python
files are really a think wrapper. Anyhow, if would be ideal if Python
has performance similar to Java, with both being interpreted languages.

Jack
Dec 9 '07 #1
Share this Question
Share on Google+
71 Replies


P: n/a
In article <G6******************************@comcast.com>,
Jack <no****@invalid.comwrote:
>
I understand that the standard Python distribution is considered
the C-Python. Howerver, the current C-Python is really a combination
of C and Python implementation. There are about 2000 Python files
included in the Windows version of Python distribution. I'm not sure
how much of the C-Python is implemented in C but I think the more
modules implemented in C, the better performance and lower memory
footprint it will get.
Prove it. ;-)

Seriously, switching to more C code will cause development to bog down
because Python is so much easier to write than C.
>I wonder if it's possible to have a Python that's completely (or at
least for the most part) implemented in C, just like PHP - I think
this is where PHP gets its performance advantage. Or maybe I'm wrong
because the core modules that matter are already in C and those Python
files are really a think wrapper. Anyhow, if would be ideal if Python
has performance similar to Java, with both being interpreted languages.
Could you provide some evidence that Python is slower than Java or PHP?
--
Aahz (aa**@pythoncraft.com) <* http://www.pythoncraft.com/

"Typing is cheap. Thinking is expensive." --Roy Smith
Dec 9 '07 #2

P: n/a
Jack schrieb:
I understand that the standard Python distribution is considered
the C-Python. Howerver, the current C-Python is really a combination
of C and Python implementation. There are about 2000 Python files
included in the Windows version of Python distribution. I'm not sure
how much of the C-Python is implemented in C but I think the more
modules implemented in C, the better performance and lower memory
footprint it will get.

I wonder if it's possible to have a Python that's completely (or at
least for the most part) implemented in C, just like PHP - I think
this is where PHP gets its performance advantage. Or maybe I'm wrong
because the core modules that matter are already in C and those Python
files are really a think wrapper. Anyhow, if would be ideal if Python
has performance similar to Java, with both being interpreted languages.
Writing everything in C might be possible - but is a daunting task & not
justified by the results. And wherever the standard libraries make use
of the flexibility of Python, it's questionable if there really was any
performance gain at all.

But what REALLY is questionable is the alleged performance advantage -
how do you back that up? According to the well-known (and surely
limited) computer language shootout

http://shootout.alioth.debian.org/gp...t=all&lang=all
Python is roughly 25% faster than PHP. Granted, this is just one
benchmark, with questionable real-life relevance. But where do you get
the impression from that PHP is faster than Python then?

diez
Dec 9 '07 #3

P: n/a
>I'm not sure
how much of the C-Python is implemented in C but I think the more
modules implemented in C, the better performance and lower memory
footprint it will get.

Prove it. ;-)
I guess this is subjective :) - that's what I felt in my experience
with web applications developed in Python and PHP. I wasn't able to
find a direct comparison online.
Seriously, switching to more C code will cause development to bog down
because Python is so much easier to write than C.
I understand. Python modules implemented in Python - this is how
Python gets its really rich library.
>>I wonder if it's possible to have a Python that's completely (or at
least for the most part) implemented in C, just like PHP - I think
this is where PHP gets its performance advantage. Or maybe I'm wrong
because the core modules that matter are already in C and those Python
files are really a thin wrapper. Anyhow, it would be ideal if Python
has performance similar to Java, with both being interpreted languages.

Could you provide some evidence that Python is slower than Java or PHP?
I think most Java-Python benchmarks you can find online will indicate
that Java is a 3-10 times faster. A few here:
http://mail.python.org/pipermail/pyt...ry/125789.html
http://blog.snaplogic.org/?p=55

Here's an article that shows the new version of Ruby is
faster than Python in some aspects (they are catching up :)
http://antoniocangiano.com/2007/11/2...s-python-away/
Dec 9 '07 #4

P: n/a
Jack wrote:
I wonder if it's possible to have a Python that's completely (or at
least for the most part) implemented in C, just like PHP - I think
this is where PHP gets its performance advantage. Or maybe I'm wrong
PHP is slower than Python.

Dec 9 '07 #5

P: n/a
That first article is five years old... I wouldn't give too much
weight to it.
Dec 9 '07 #6

P: n/a
Jack wrote:
I guess this is subjective :) - that's what I felt in my experience
with web applications developed in Python and PHP. I wasn't able to
find a direct comparison online.
Please compare the number of serious bugs and vulnerabilities in PHP and
Python.
I understand. Python modules implemented in Python - this is how
Python gets its really rich library.
Correct
Python code is much easier to write and multiple times easier to get
right than C code. Everybody with a few months of Python experience can
contribute to the core but it requires multiple years of C and Python
experience to contribute to the C implementation.
I think most Java-Python benchmarks you can find online will indicate
that Java is a 3-10 times faster. A few here:
http://mail.python.org/pipermail/pyt...ry/125789.html
http://blog.snaplogic.org/?p=55
There are lies, damn lies and benchmarks. :)

Pure Python code is not going to beat Java code until the Python core
gets a JIT compiler. If you want fair results you have to either
disable the JIT in Java or use Psyco for Python. Otherwise you are
comparing the quality of one language implementation to the quality of a
JIT compiler.
Here's an article that shows the new version of Ruby is
faster than Python in some aspects (they are catching up :)
http://antoniocangiano.com/2007/11/2...s-python-away/
The Ruby developers are allowed to be proud. They were able to optimize
some aspects of the implementation to get one algorithm about 14 times
faster. That's good work. But why was it so slow in the first place?

Nevertheless it is just one algorithm that beats Python in an area that
is well known to be slow. Python's numbers are several factors slower
than C code because the overhead of the dynamic language throws lots of
data out of the cache line. If you need fast and highly optimized int
and floating point operations you can rewrite the algorithm in C and
create a Python interface for it.
Dec 9 '07 #7

P: n/a
Jack wrote:
I guess this is subjective :) - that's what I felt in my experience
with web applications developed in Python and PHP. I wasn't able to
find a direct comparison online.
Please compare the number of serious bugs and vulnerabilities in PHP and
Python.
I understand. Python modules implemented in Python - this is how
Python gets its really rich library.
Correct
Python code is much easier to write and multiple times easier to get
right than C code. Everybody with a few months of Python experience can
contribute to the core but it requires multiple years of C and Python
experience to contribute to the C implementation.
I think most Java-Python benchmarks you can find online will indicate
that Java is a 3-10 times faster. A few here:
http://mail.python.org/pipermail/pyt...ry/125789.html
http://blog.snaplogic.org/?p=55
There are lies, damn lies and benchmarks. :)

Pure Python code is not going to beat Java code until the Python core
gets a JIT compiler. If you want fair results you have to either
disable the JIT in Java or use Psyco for Python. Otherwise you are
comparing the quality of one language implementation to the quality of a
JIT compiler.
Here's an article that shows the new version of Ruby is
faster than Python in some aspects (they are catching up :)
http://antoniocangiano.com/2007/11/2...s-python-away/
The Ruby developers are allowed to be proud. They were able to optimize
some aspects of the implementation to get one algorithm about 14 times
faster. That's good work. But why was it so slow in the first place?

Nevertheless it is just one algorithm that beats Python in an area that
is well known to be slow. Python's numbers are several factors slower
than C code because the overhead of the dynamic language throws lots of
data out of the cache line. If you need fast and highly optimized int
and floating point operations you can rewrite the algorithm in C and
create a Python interface for it.

Dec 9 '07 #8

P: n/a

"Jack" <no****@invalid.comwrote in message
news:G6******************************@comcast.com. ..
|I understand that the standard Python distribution is considered
| the C-Python. Howerver, the current C-Python is really a combination
| of C and Python implementation. There are about 2000 Python files
| included in the Windows version of Python distribution.

About half or fewer are modules meant to be imported into programs. The
rest comprise utility programs and test programs. The core interpreter is
all C.

| because the core modules that matter are already in C

Correct. There are about 20 'builtin' modules written is C either because
they need low level access to the machine or for speed concerns. Third
party modules not included in the standard distribution but definitely part
of the Python universe are also a mix.

If people wrote everything in C for speed, there would be no need for
Python!!

And don't say that you want everyone else to write in C while you enjoy the
pleasures of Python ;-).

tjr

Dec 9 '07 #9

P: n/a
>I think most Java-Python benchmarks you can find online will indicate
that Java is a 3-10 times faster. A few here:
http://mail.python.org/pipermail/pyt...ry/125789.html
http://blog.snaplogic.org/?p=55

There are lies, damn lies and benchmarks. :)

Pure Python code is not going to beat Java code until the Python core
gets a JIT compiler. If you want fair results you have to either
disable the JIT in Java or use Psyco for Python. Otherwise you are
comparing the quality of one language implementation to the quality of a
JIT compiler.
The second articple does have a column for Psyco. It helps in some areas
but still not good enough to stand up against Java. Plus, Psyco is not the
main stream and has stopped development.

I'm also wondering, if Psyco is the right way to do, any reason it's not
being integrated into standard Python?
Dec 10 '07 #10

P: n/a
On Dec 9, 6:07 pm, "Jack" <nos...@invalid.comwrote:
Plus, Psyco is not the
main stream and has stopped development.
<scooby-whruu??>

What makes you think it has stopped development? I just swung by the
SF project page, and its most recent news post was just 2 months ago.

Psyco may not be in the standard Python distribution, but it is
definitely a fixture of the Python landscape, which is about as close
to main stream as you can get.

-- Paul
Dec 10 '07 #11

P: n/a
On Dec 9, 10:04 pm, Paul McGuire <pt...@austin.rr.comwrote:
On Dec 9, 6:07 pm, "Jack" <nos...@invalid.comwrote:
Plus, Psyco is not the
main stream and has stopped development.

<scooby-whruu??>

What makes you think it has stopped development? I just swung by the
SF project page, and its most recent news post was just 2 months ago.

Psyco may not be in the standard Python distribution, but it is
definitely a fixture of the Python landscape, which is about as close
to main stream as you can get.

-- Paul
Maybe because of this line:

"Psyco is a reasonably complete project. I will not continue to
develop it beyond making sure it works with future versions of Python.
My plans for 2006 are to port the techniques implemented in Psyco to
PyPy. PyPy will allow us to build a more flexible JIT specializer,
easier to experiment with, and without the overhead of having to keep
in sync with the evolutions of the Python language."
Dec 10 '07 #12

P: n/a
Jack a écrit :
I understand that the standard Python distribution is considered
the C-Python. Howerver, the current C-Python is really a combination
of C and Python implementation. There are about 2000 Python files
included in the Windows version of Python distribution. I'm not sure
how much of the C-Python is implemented in C but I think the more
modules implemented in C, the better performance and lower memory
footprint it will get.

I wonder if it's possible to have a Python that's completely (or at
least for the most part) implemented in C, just like PHP - I think
this is where PHP gets its performance advantage.
Which "performance advantage" ???
Or maybe I'm wrong
because the core modules that matter are already in C and those Python
files are really a think wrapper. Anyhow, if would be ideal if Python
has performance similar to Java, with both being interpreted languages.
<mode="pedantic">
Neither Python nor Java are "interpreted languages", because there's no
such thing as an "interpreted language" - being 'interpreted' (whatever
the definition of 'interpreted') is a quality of an implementation, not
of a language. wrt/ to CPython and Sun's Java implementation, they are
both byte-code compiled - which, according to usual definitions, is not
quite the same thing !-)
</mode>

Now most of the performance difference is due to Java being much less
dynamic than Python, which allow both the compiler and the VM to do much
more optimizations - specially JIT compilation. It's quite harder to
implement such optimizations for a language as dynamic as Python (IIRC,
some language/compiler gurus here mentionned that even compiling Python
to native binary code would not buy that much gain).

Actually, it seems that taking the opposite approach - that is, trying
to implement as much as possible of Python in Python - would be more
promising wrt/ possible JIT compilation, cf the Pypy project.
Dec 10 '07 #13

P: n/a
On Dec 9, 10:43 pm, "Jack" <nos...@invalid.comwrote:
>
http://blog.snaplogic.org/?p=55
There's some choice nonsense here, albeit on a different topic:

"Coding for wxwidgets, using a QT or GTK bridge, or using TCL/TK is
hardly an optimal solution when writing complex graphical
applications, and Java wins in this area, despite there comically
being many problems with the look and feel of Java applications."

Clearly an individual who hasn't actually used any of the Python GUI
development solutions, given the choice of words: "bridge", "hardly an
optimal solution"; virtually intimating that you'd be doing malloc/
free or new/delete all the time. Plus throwaway remarks of the form
"XYZ wins" tend to suggest beliefs with little substance and a
continual need for self-reassurance on such matters.

Anyway, back to the topic at hand...
Here's an article that shows the new version of Ruby is
faster than Python in some aspects (they are catching up :)

http://antoniocangiano.com/2007/11/2...19-smokes-pyth...
It's evident that the next mainstream version of Ruby will have
various optimisations around recursive operations - something that has
generally been rejected for CPython. Of course, the mainstream Ruby
implementation has had a lot of scope for improvement:

http://shootout.alioth.debian.org/gp...t=all&lang=all

What disappoints me somewhat is that most of the people interested in
taking Python performance to the next level are all outside (or on the
outer fringes of) the CPython core development group: PyPy and Shed
Skin are mostly distinct technologies; Psyco integrates with CPython
but hasn't improved the "out of the box" situation; Pyrex is really a
distinct tool, being more like a convenient wrapper generator than a
bolt-on high performance engine for CPython. Language implementations
like that of Lua have seen more progress on integrating solutions for
performance, it would seem.

As for a C-Python of the form requested, I suppose tools like Shed
Skin and RPython fit the bill somewhat, if a transparent solution is
needed where one writes in Python and it magically becomes fairly
efficient C or C++. Otherwise, Pyrex provides more explicit control
over what gets written in C and what remains in Python.

Paul
Dec 10 '07 #14

P: n/a
On Dec 9, 3:23 pm, a...@pythoncraft.com (Aahz) wrote:
In article <G62dnbBDl_Y0x8HanZ2dnUVZ_gudn...@comcast.com>,

Jack <nos...@invalid.comwrote:
I understand that the standard Python distribution is considered
the C-Python. Howerver, the current C-Python is really a combination
of C and Python implementation. There are about 2000 Python files
included in the Windows version of Python distribution. I'm not sure
how much of the C-Python is implemented in C but I think the more
modules implemented in C, the better performance and lower memory
footprint it will get.

Prove it. ;-)

Seriously, switching to more C code will cause development to bog down
because Python is so much easier to write than C.
I wonder if it's possible to have a Python that's completely (or at
least for the most part) implemented in C, just like PHP - I think
this is where PHP gets its performance advantage. Or maybe I'm wrong
because the core modules that matter are already in C and those Python
files are really a think wrapper. Anyhow, if would be ideal if Python
has performance similar to Java, with both being interpreted languages.

Could you provide some evidence that Python is slower than Java or PHP?
--
Aahz (a...@pythoncraft.com) <* http://www.pythoncraft.com/

"Typing is cheap. Thinking is expensive." --Roy Smith
I'd like to provide some evidence that Python is *faster* than Java.
EVE online...emulate that in JAVA please.
Dec 10 '07 #15

P: n/a
On Dec 9, 1:14 pm, "Jack" <nos...@invalid.comwrote:
I wonder if it's possible to have a Python that's completely (or at
least for the most part) implemented in C, just like PHP - I think
this is where PHP gets its performance advantage. Or maybe I'm wrong
because the core modules that matter are already in C and those Python
files are really a think wrapper. Anyhow, if would be ideal if Python
has performance similar to Java, with both being interpreted languages.
-1 This would seriously muck-up the evolution of the language.
Having a few building blocks written in C provides a basis
for writing very fast pure python (for example, sets, heapq,
itertools).
Beyond those building blocks, it is a step backwards to write in C.

Also, if you really need performance, the traditional solutions are to
use tools like Psyco or Pyrex.
Raymond
Dec 10 '07 #16

P: n/a
On Dec 9, 10:07 pm, "Jack" <nos...@invalid.comwrote:
I think most Java-Python benchmarks you can find online will indicate
that Java is a 3-10 times faster. A few here:
http://mail.python.org/pipermail/pyt...ry/125789.html
http://blog.snaplogic.org/?p=55
There are lies, damn lies and benchmarks. :)
Pure Python code is not going to beat Java code until the Python core
gets a JIT compiler. If you want fair results you have to either
disable the JIT in Java or use Psyco for Python. Otherwise you are
comparing the quality of one language implementation to the quality of a
JIT compiler.

The second articple does have a column for Psyco. It helps in some areas
but still not good enough to stand up against Java. Plus, Psyco is not the
main stream and has stopped development.

I'm also wondering, if Psyco is the right way to do, any reason it's not
being integrated into standard Python?
Instead of recurring to benchmarks, I recommend that you read the
following:

http://highscalability.com/youtube-architecture

There are no comparisons there, just a sample of what python and
psyco can achieve. For a language that isn't designed with speed in
mind, I think that's quite impressive.
Dec 10 '07 #17

P: n/a
Jack a écrit :
>>>I'm not sure
how much of the C-Python is implemented in C but I think the more
modules implemented in C, the better performance and lower memory
footprint it will get.

Prove it. ;-)


I guess this is subjective :)
If yes, benchmarks are not an argument. Else, you'll have hard time
making your point !-)

(hint: doing objective benchmarking is really a difficult art)
- that's what I felt in my experience
with web applications developed in Python and PHP. I wasn't able to
find a direct comparison online.
Could it be the case that you are comparing Python CGI scripts with
mod_php ? Anyway, since php is also usable (hem... maybe not the
appropriate qualificative) outside Apache, it should quite easy to make
a more serious test ?

Seriously: I never saw any benchmark where php was faster than Python
for any kind of stuff - unless of course you're trying to compare Zope
running as a CGI script with an hello world PHP script runned by mod_php.
Dec 10 '07 #18

P: n/a
hu***********@gmail.com a écrit :
(snip)
I'd like to provide some evidence that Python is *faster* than Java.
Then benchmark the time taken for the interpreter (oops, sorry: "VM") to
start !-)
Dec 10 '07 #19

P: n/a
On 9 Des, 22:14, "Jack" <nos...@invalid.comwrote:
I understand that the standard Python distribution is considered
the C-Python. Howerver, the current C-Python is really a combination
of C and Python implementation. There are about 2000 Python files
included in the Windows version of Python distribution. I'm not sure
how much of the C-Python is implemented in C but I think the more
modules implemented in C, the better performance and lower memory
footprint it will get.
Donald Knuth, one of the fathers of modern computer science, is famous
for stating that "premature optimization is the root of all evil in
computer science." A typical computer program tends to have
bottlenecks that accounts for more than 90% of the elapsed run time.
Directing your optimizations anywhere else is futile.

Writing a program in C will not improve the speed of your hardware. If
the bottleneck is a harddisk or a network connection, using C will not
change that. Disk i/o is a typical example of that. It is not the
language that determines the speed by which Python or C can read from
a disk. It is the disk itself.

I had a data vizualization program that was slowed down by the need to
move hundreds of megabytes of vertex data to video RAM. It would
obviously not help to make the handful of OpenGL calls from C instead
of Python. The problem was the amount of data and the speed of the
hardware (ram or bus). The fact that I used Python instead of C
actually helped to make the problem easier to solve.

We have seen several examples that 'dynamic' and 'interpreted'
languages can be quite efficient: There is an implementation of Common
Lisp - CMUCL - that can compete with Fortran in efficiency for
numerical computing. There are also versions of Lisp than can compete
with the latest versions of JIT-compiled Java, e.g. SBCL and Allegro.
As it happens, SBCL and CMUCL is mostly implemented in Lisp. The issue
of speed for a language like Python has a lot to do with the quality
of the implementation. What really makes CMUCL shine is the compiler
that emits efficient native code on the fly. If it is possible to make
a very fast Lisp, it should be possible to make a very fast Python as
well. I remember people complaining 10 years ago that 'Lisp is so
slow'. A huge effort has been put into making Lisp efficient enough
for AI. I hope Python some day will gain a little from that effort as
well.

We have a Python library that allows us to perform a wide range of
numerical tasks at 'native speed': NumPy (http://www.scipy.org). How
such array libraries can be used to get excellent speedups is
explained here: http://home.online.no/~pjacklam/matl...mtt/index.html

We obviously need more effort to make Python more efficient for CPU
bound tasks. Particularly JIT compilation like Java, compilation like
Lisp or data specialization like Psyco.

But writing larger parts of the standard library in C is not a
solution.
Dec 10 '07 #20

P: n/a
On 9 Des, 23:34, Christian Heimes <li...@cheimes.dewrote:
Nevertheless it is just one algorithm that beats Python in an area that
is well known to be slow. Python's numbers are several factors slower
than C code because the overhead of the dynamic language throws lots of
data out of the cache line. If you need fast and highly optimized int
and floating point operations you can rewrite the algorithm in C and
create a Python interface for it.
Lisp is a dynamically typed language. CMUCL can compete with Fortran
for numerical work. SBCL can compete with the Java server VM. If the
current CPython VM throws data out of the cache line, then it must be
a design flaw in the VM.

Dec 10 '07 #21

P: n/a
In article <a3**********************************@l16g2000hsf. googlegroups.com>,
sturlamolden <st**********@yahoo.nowrote:
>
Donald Knuth, one of the fathers of modern computer science, is famous
for stating that "premature optimization is the root of all evil in
computer science."
From my .sig database:

"Premature optimization is the root of all evil in programming."
--C.A.R. Hoare (often misattributed to Knuth, who was himself quoting
Hoare)
--
Aahz (aa**@pythoncraft.com) <* http://www.pythoncraft.com/

"Typing is cheap. Thinking is expensive." --Roy Smith
Dec 10 '07 #22

P: n/a
sturlamolden a écrit :
On 9 Des, 23:34, Christian Heimes <li...@cheimes.dewrote:

>>Nevertheless it is just one algorithm that beats Python in an area that
is well known to be slow. Python's numbers are several factors slower
than C code because the overhead of the dynamic language throws lots of
data out of the cache line. If you need fast and highly optimized int
and floating point operations you can rewrite the algorithm in C and
create a Python interface for it.


Lisp is a dynamically typed language. CMUCL can compete with Fortran
for numerical work. SBCL can compete with the Java server VM. If the
current CPython VM throws data out of the cache line, then it must be
a design flaw in the VM.
Or a lack of time and money. Lisp is one of the older programming
languages around, and at a time had BigBucks(tm) invested on it to try
and make it practically usable.
Dec 10 '07 #23

P: n/a
On 10 Des, 23:54, Bruno Desthuilliers
<bdesth.quelquech...@free.quelquepart.frwrote:
Or a lack of time and money. Lisp is one of the older programming
languages around, and at a time had BigBucks(tm) invested on it to try
and make it practically usable.
Yes. But strangely enough, the two Lisp implementations that really
kick ass are both free and not particularly old. CMUCL and SBCL proves
that you can make a dynamic language implementation extremely
efficient if you try hard enough. There are also implementations of
Scheme (e.g. Bigloo) that shows the same.
Dec 10 '07 #24

P: n/a
On 10 Des, 23:49, a...@pythoncraft.com (Aahz) wrote:
"Premature optimization is the root of all evil in programming."
--C.A.R. Hoare (often misattributed to Knuth, who was himself quoting
Hoare)
Oh, I was Hoare? Thanks. Anyway, it doesn't change the argument that
optimizing in wrong places is a waste of effort.
Dec 10 '07 #25

P: n/a
On 9 Des, 23:34, Christian Heimes <li...@cheimes.dewrote:
http://antoniocangiano.com/2007/11/2...19-smokes-pyth...

The Ruby developers are allowed to be proud. They were able to optimize
some aspects of the implementation to get one algorithm about 14 times
faster. That's good work. But why was it so slow in the first place?
The thing to notice here is that Congiano spent 31.5 seconds computing
36 Fibonacci numbers in Python and 11.9 seconds doing the same in
Ruby. Those numbers are ridiculous! The only thing they prove is that
Congiano should not be programming computers. Anyone getting such
results should take a serious look at their algoritm instead of
blaming the language. I don't care if it takes 31.5 seconds to compute
36 Fibonacci numbers in Python 2.5.1 with the dumbest possible
algorithm.




Dec 10 '07 #26

P: n/a
We obviously need more effort to make Python more efficient for CPU
bound tasks. Particularly JIT compilation like Java, compilation like
Lisp or data specialization like Psyco.
Given that the Python core team has been mostly silent about JIT
compilation and Armin Rigos work in particular which started 5 years
ago ( Psyco will not be promoted towards Python 3.0 and there is no
indication that anyone but Armin would maintain Psyco ) I wonder about
this sudden insight. But maybe you can tell us more when you work on
such stuff for CPython yourself? Given your status in the core Python
team this would have a chance of not being just a loosely coupled
independent project as almost all the interesting stuff that happens
around Python these days.
Dec 11 '07 #27

P: n/a
sturlamolden <st**********@yahoo.nowrote:
On 9 Des, 23:34, Christian Heimes <li...@cheimes.dewrote:
>http://antoniocangiano.com/2007/11/2...19-smokes-pyth
...

The Ruby developers are allowed to be proud. They were able to
optimize some aspects of the implementation to get one algorithm
about 14 times faster. That's good work. But why was it so slow in
the first place?

The thing to notice here is that Congiano spent 31.5 seconds computing
36 Fibonacci numbers in Python and 11.9 seconds doing the same in
Ruby. Those numbers are ridiculous! The only thing they prove is that
Congiano should not be programming computers. Anyone getting such
results should take a serious look at their algoritm instead of
blaming the language. I don't care if it takes 31.5 seconds to compute
36 Fibonacci numbers in Python 2.5.1 with the dumbest possible
algorithm.
Quite so.

Take something like
http://aspn.activestate.com/ASPN/Coo.../Recipe/498110
and then modify the Python code from the "Ruby smokes Python" article by
the addition of @memoize(3) to decorate the otherwise unchanged fib
function: the Python runtime drops down to 0.002 seconds.

That is just slightly better than Ruby's 11.9 seconds although I'm sure the
Ruby code would also gain as much from a memoize decorator.
from memoize import memoize

@memoize(3)
def fib(n):
if n == 0 or n == 1:
return n
else:
return fib(n-1) + fib(n-2)

from time import clock
start = clock()
for i in range(36):
print "n=%d =%d" % (i, fib(i))
print clock()-start
When I run this (with output directed to a file: I'm not trying to time
windows console speed), the output is:

n=0 =0
n=1 =1
n=2 =1
n=3 =2
n=4 =3
n=5 =5
n=6 =8
n=7 =13
n=8 =21
n=9 =34
n=10 =55
n=11 =89
n=12 =144
n=13 =233
n=14 =377
n=15 =610
n=16 =987
n=17 =1597
n=18 =2584
n=19 =4181
n=20 =6765
n=21 =10946
n=22 =17711
n=23 =28657
n=24 =46368
n=25 =75025
n=26 =121393
n=27 =196418
n=28 =317811
n=29 =514229
n=30 =832040
n=31 =1346269
n=32 =2178309
n=33 =3524578
n=34 =5702887
n=35 =9227465
0.00226425425578

Dec 11 '07 #28

P: n/a
On Dec 11, 3:10 am, Duncan Booth <duncan.bo...@invalid.invalidwrote:
sturlamolden <sturlamol...@yahoo.nowrote:
On 9 Des, 23:34, Christian Heimes <li...@cheimes.dewrote:
http://antoniocangiano.com/2007/11/2...19-smokes-pyth
...
The Ruby developers are allowed to be proud. They were able to
optimize some aspects of the implementation to get one algorithm
about 14 times faster. That's good work. But why was it so slow in
the first place?
The thing to notice here is that Congiano spent 31.5 seconds computing
36 Fibonacci numbers in Python and 11.9 seconds doing the same in
Ruby. Those numbers are ridiculous! The only thing they prove is that
Congiano should not be programming computers. Anyone getting such
results should take a serious look at their algoritm instead of
blaming the language. I don't care if it takes 31.5 seconds to compute
36 Fibonacci numbers in Python 2.5.1 with the dumbest possible
algorithm.

Quite so.

Take something likehttp://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/498110
and then modify the Python code from the "Ruby smokes Python" article by
the addition of @memoize(3) to decorate the otherwise unchanged fib
function: the Python runtime drops down to 0.002 seconds.

That is just slightly better than Ruby's 11.9 seconds although I'm sure the
Ruby code would also gain as much from a memoize decorator.

from memoize import memoize

@memoize(3)
def fib(n):
if n == 0 or n == 1:
return n
else:
return fib(n-1) + fib(n-2)

from time import clock
start = clock()
for i in range(36):
print "n=%d =%d" % (i, fib(i))
print clock()-start

When I run this (with output directed to a file: I'm not trying to time
windows console speed), the output is:

n=0 =0
n=1 =1
n=2 =1
n=3 =2
n=4 =3
n=5 =5
n=6 =8
n=7 =13
n=8 =21
n=9 =34
n=10 =55
n=11 =89
n=12 =144
n=13 =233
n=14 =377
n=15 =610
n=16 =987
n=17 =1597
n=18 =2584
n=19 =4181
n=20 =6765
n=21 =10946
n=22 =17711
n=23 =28657
n=24 =46368
n=25 =75025
n=26 =121393
n=27 =196418
n=28 =317811
n=29 =514229
n=30 =832040
n=31 =1346269
n=32 =2178309
n=33 =3524578
n=34 =5702887
n=35 =9227465
0.00226425425578
Another point is, the reason the ruby code shows such a performance
increase is because of the way it wraps native (C) types for integers
in the the new byte compiler; i.e., it's a directed optimization,
which the example code exploits to its full extent. But with
dictionary access, for example, python still creams ruby (by a 2/1
factor in my tests). Speaking as someone who uses both python and
ruby, I can say that ruby 1.9 is approaching python's speed, which is
very cool, but is still not quite as fast as python in general (the
whole "smokes python" bit is just propaganda that utilizes a specific
feature vector, and is generally unhelpful).

Regards,
Jordan
Dec 11 '07 #29

P: n/a
sturlamolden wrote:
On 10 Des, 23:49, a...@pythoncraft.com (Aahz) wrote:
>"Premature optimization is the root of all evil in programming."
--C.A.R. Hoare (often misattributed to Knuth, who was himself quoting
Hoare)
We're ten years into Python, and it's still a naive interpreter.
It's time for a serious optimizing compiler. Shed Skin is going
in the right direction. But for some reason, people seem to dislike the
Shed Skin effort. Its author writes "Am I the only one seeing the potential
of an implicitly statically typed Python-like-language that runs at
practically the same speed as C++?"

"For a set of 27 non-trivial test programs (at about 7,000 lines in total;
.... measurements show a typical speedup of 2-40 times over Psyco, about 10 on
average, and 2-220 times over CPython, about 35 on average." So that's
what's possible.

I'm surprised that Google management isn't pushing Guido towards
doing something about the performance problem.

John Nagle
Dec 11 '07 #30

P: n/a
sturlamolden <st**********@yahoo.nowrites:
On 10 Des, 23:49, a...@pythoncraft.com (Aahz) wrote:
>"Premature optimization is the root of all evil in programming."
--C.A.R. Hoare (often misattributed to Knuth, who was himself quoting
Hoare)

Oh, I was Hoare? Thanks. Anyway, it doesn't change the argument that
optimizing in wrong places is a waste of effort.
Although sometimes people seem to think that it goes "optimisation is
the root...". The "premature" bit is significant.
Dec 11 '07 #31

P: n/a
On Dec 11, 2007 1:25 PM, John Nagle <na***@animats.comwrote:
sturlamolden wrote:
On 10 Des, 23:49, a...@pythoncraft.com (Aahz) wrote:
"Premature optimization is the root of all evil in programming."
--C.A.R. Hoare (often misattributed to Knuth, who was himself quoting
Hoare)

We're ten years into Python, and it's still a naive interpreter.
This is an absurd misrepresentation of the state of the Python VM.
It's time for a serious optimizing compiler. Shed Skin is going
in the right direction. But for some reason, people seem to dislike the
Shed Skin effort. Its author writes "Am I the only one seeing the potential
of an implicitly statically typed Python-likea-lnguage that runs at
practically the same speed as C++?"

"For a set of 27 non-trivial test programs (at about 7,000 lines in total;
... measurements show a typical speedup of 2-40 times over Psyco, about 10 on
average, and 2-220 times over CPython, about 35 on average." So that's
what's possible.
.... with roughly a hundredth of the python standard library, and a
bunch of standard python features not even possible. I like
generators, thanks.

If shedskin can actually match Pythons feature set and provide the
performance it aspires to, thats great, and I may even start using it
then. But in the meantime, hardly anything I write is CPU bound and
when it is I can easily optimize using other mechanisms. Shedskin
doesn't give me anything that's worth my time to improve on it, or the
restrictions it places on my code. I think JIT is the future of
optimization anyway.
I'm surprised that Google management isn't pushing Guido towards
doing something about the performance problem.
Assuming your conclusion (ie, that there's a performance problem to do
something about) doesn't prove your case.
Dec 11 '07 #32

P: n/a
sturlamolden <st**********@yahoo.nowrites:
On 10 Des, 23:54, Bruno Desthuilliers
<bdesth.quelquech...@free.quelquepart.frwrote:
>Or a lack of time and money. Lisp is one of the older programming
languages around, and at a time had BigBucks(tm) invested on it to try
and make it practically usable.

Yes. But strangely enough, the two Lisp implementations that really
kick ass are both free and not particularly old.
Not two, but one -- SBL is simply a fork of CMU CL. As for their age,
the CMU CL states that it has been "continually developed since the
early 1980s".
Dec 11 '07 #33

P: n/a
On Tue, 11 Dec 2007 11:25:32 -0800, John Nagle wrote:
sturlamolden wrote:
>On 10 Des, 23:49, a...@pythoncraft.com (Aahz) wrote:
>>"Premature optimization is the root of all evil in programming."
--C.A.R. Hoare (often misattributed to Knuth, who was himself quoting
Hoare)

We're ten years into Python, and it's still a naive interpreter.
It's time for a serious optimizing compiler. Shed Skin is going in the
right direction. But for some reason, people seem to dislike the Shed
Skin effort. Its author writes "Am I the only one seeing the potential
of an implicitly statically typed Python-like-language that runs at
practically the same speed as C++?"

"For a set of 27 non-trivial test programs (at about 7,000 lines in
total;
... measurements show a typical speedup of 2-40 times over Psyco, about
10 on average, and 2-220 times over CPython, about 35 on average." So
that's what's possible.

I'm surprised that Google management isn't pushing Guido towards
doing something about the performance problem.
Maybe because it isn't as much a problem as people with C envy assume it
must be? (Disclaimer: I'm not suggesting that John is one of those
people.)

Not that I'd object to anyone else doing the work to speed up Python, but
for the things I use Python for, I've never felt the need to say "Gosh
darn it, my script took twelve milliseconds to run, that's just too
slow!!!". Maybe Google are in the same boat?

Actually, in Google's case, I guess their bottleneck is not Python, but
trying to push around gigabytes of data. That will be slow no matter what
language you write in.

--
Steven.
Dec 11 '07 #34

P: n/a
John Nagle schrieb:
sturlamolden wrote:
>On 10 Des, 23:49, a...@pythoncraft.com (Aahz) wrote:
>>"Premature optimization is the root of all evil in programming."
--C.A.R. Hoare (often misattributed to Knuth, who was himself quoting
Hoare)

We're ten years into Python, and it's still a naive interpreter.
It's time for a serious optimizing compiler. Shed Skin is going
in the right direction. But for some reason, people seem to dislike the
Shed Skin effort. Its author writes "Am I the only one seeing the potential
of an implicitly statically typed Python-like-language that runs at
practically the same speed as C++?"

"For a set of 27 non-trivial test programs (at about 7,000 lines in
total; ... measurements show a typical speedup of 2-40 times over Psyco,
about 10 on average, and 2-220 times over CPython, about 35 on
average." So that's
what's possible.
No, it's not. Shedskin is interesting, but just a small subset of Python
- and without completeness, performance is useless.

The PyPy approach is much more interesting - first create a
full-featured Python itself, then create optimizing backends for it,
also for just a language subset - RPython.

And if possible - which it is only in a very limited set of cases for
not type-annotated code - identify parts that conform to RPython's
constraints, and compile that JITly.

Diez
Dec 11 '07 #35

P: n/a
John Nagle a écrit :
sturlamolden wrote:
>On 10 Des, 23:49, a...@pythoncraft.com (Aahz) wrote:
>>"Premature optimization is the root of all evil in programming."
--C.A.R. Hoare (often misattributed to Knuth, who was himself quoting
Hoare)


We're ten years into Python, and it's still a naive interpreter.
It's time for a serious optimizing compiler.
Care to provide a less "naive" one ?

(snip usual rant)
I'm surprised that Google management isn't pushing Guido towards
doing something about the performance problem.
Could it be possible they don't see Python's perfs as a "problem" ?

Ok, I don't mean there's no room for improvement here. If you feel like
addressing the problem, you're welcome - in case you didn't notice,
Python is free software.
Dec 11 '07 #36

P: n/a
On 11 Des, 20:25, John Nagle <na...@animats.comwrote:
Shed Skin effort. Its author writes "Am I the only one seeing the potential
of an implicitly statically typed Python-like-language that runs at
practically the same speed as C++?"
Don't forget about Pyrex and PyPy's RPython.

By the way, we don't need a hotspot JIT compiler. Lisp has a compiler
invoked by the programmer. We could include optional static typing in
Python, and have an optional static optimizing native compiler for
selected portions of code. That would be easier to implement in the
short run, with JIT-capabilities added later. Pyrex, ShedSkin or
RPython are all good starting points.



Dec 11 '07 #37

P: n/a
sturlamolden a écrit :
-snip)
We could include optional static typing in
Python,
Please wait - going to get my gun...
Dec 12 '07 #38

P: n/a
On Dec 12, 2:18 am, Kay Schluehr <kay.schlu...@gmx.netwrote:
On Dec 12, 7:34 am, sturlamolden <sturlamol...@yahoo.nowrote:
I am not sure why a new type annotation syntax was needed Python 3:

Because people care about a feature when there is @syntax.
Good point; the inverse is not true though: time and time again people
cared about some syntax for properties without any change so far. The
result is a handful of different ways to spell out properties; python
2.6 will add yet another variation (http://mail.python.org/pipermail/
python-dev/2007-October/075057.html).

George
Dec 12 '07 #39

P: n/a
On Dec 12, 9:04 am, George Sakkis <george.sak...@gmail.comwrote:
On Dec 12, 2:18 am, Kay Schluehr <kay.schlu...@gmx.netwrote:
On Dec 12, 7:34 am, sturlamolden <sturlamol...@yahoo.nowrote:
I am not sure why a new type annotation syntax was needed Python 3:
Because people care about a feature when there is @syntax.

Good point; the inverse is not true though: time and time again people
cared about some syntax for properties without any change so far. The
result is a handful of different ways to spell out properties; python
2.6 will add yet another variation (http://mail.python.org/pipermail/
python-dev/2007-October/075057.html).

George
Yes, I'm aware. Curiously, whenever property syntax is discussed the
discussion loses track and is dragged away by needless side
discussions. Just look at Stephen Bethards withdrawn PEP 359 [1] in
which he finally muses about replacing the class statement by the make
statement. So the PEP ended in "abstract nonsense" instead of
clarifying the point.

[1] http://www.python.org/dev/peps/pep-0359/

I vaguely remember a discussion a few years ago, where someone made
the quite reasonable suggestion of introducing some kind of
thunk_statement:

class A(object):
foo = property:
def fget(self):
return self._foo
def fset(self, value):
self._foo = value

which was translated as follows:

class A(object):
def thunk():
def fget(self):
return self._foo
def fset(self, value):
self._foo = value
return vars()
foo = propery(**thunk())
del thunk

Now people started to consider using the compound statement within
expressions as well because the equal sign is used within method
signatures and call syntax. This lead to a general discussion about
the expr/statement distinction in Python, about multiline lambdas and
functional style programming. These association graphs are almost
predictable.

Dec 12 '07 #40

P: n/a
sturlamolden wrote:
On 11 Des, 20:25, John Nagle <na...@animats.comwrote:
>Shed Skin effort. Its author writes "Am I the only one seeing the potential
of an implicitly statically typed Python-like-language that runs at
practically the same speed as C++?"

Don't forget about Pyrex and PyPy's RPython.

By the way, we don't need a hotspot JIT compiler. Lisp has a compiler
invoked by the programmer. We could include optional static typing in
Python, and have an optional static optimizing native compiler for
selected portions of code. That would be easier to implement in the
short run, with JIT-capabilities added later. Pyrex, ShedSkin or
RPython are all good starting points.
I just want to stress that adding type hints _won't_ make programs
faster if you use a good specializing JIT compiler. Psyco in particular
would not benefit from type hints at all (even if you changed Psyco take
them into account) and would give you exactly the same speed as without
them.

Cheers,

Carl Friedrich Bolz
Dec 12 '07 #41

P: n/a
sturlamolden wrote:
On 11 Des, 20:25, John Nagle <na...@animats.comwrote:
>Shed Skin effort. Its author writes "Am I the only one seeing the potential
of an implicitly statically typed Python-like-language that runs at
practically the same speed as C++?"

Don't forget about Pyrex and PyPy's RPython.

By the way, we don't need a hotspot JIT compiler. Lisp has a compiler
invoked by the programmer. We could include optional static typing in
Python, and have an optional static optimizing native compiler for
selected portions of code. That would be easier to implement in the
short run, with JIT-capabilities added later. Pyrex, ShedSkin or
RPython are all good starting points.
I just want to stress that adding type hints _won't_ make programs
faster if you use a good specializing JIT compiler. Psyco in particular
would not benefit from type hints at all (even if you changed Psyco take
them into account) and would give you exactly the same speed as without
them.

Cheers,

Carl Friedrich Bolz

Dec 12 '07 #42

P: n/a
On Dec 12, 4:09 am, Kay Schluehr <kay.schlu...@gmx.netwrote:
Curiously, whenever property syntax is discussed the
discussion loses track and is dragged away by needless side
discussions. Just look at Stephen Bethards withdrawn PEP 359 [1] in
which he finally muses about replacing the class statement by the make
statement. So the PEP ended in "abstract nonsense" instead of
clarifying the point.

[1]http://www.python.org/dev/peps/pep-0359/
Ah, the 'make' statement.. I liked (and still do) that PEP, I think it
would have an impact comparable to the decorator syntax sugar, if not
more. Alas, it was too much ahead of its time.. who knows, it might
revive on some 3.x version.

George
Dec 12 '07 #43

P: n/a
On 12 Des, 12:56, George Sakkis <george.sak...@gmail.comwrote:
Ah, the 'make' statement.. I liked (and still do) that PEP, I think it
would have an impact comparable to the decorator syntax sugar, if not
more.
I think it is one step closer to Lisp. I believe that it would be
worth considering adding defmacro statement. Any syntax, including if,
else, for, while, class, lambda, try, except, etc. would be
implemented with defmacros. We would only need a minimalistic syntax,
that would bootstrap a full Python syntax on startup. And as for
speed, we all know how Lisp compares to Python.



Dec 12 '07 #44

P: n/a
On Dec 12, 2007 8:36 AM, sturlamolden <st**********@yahoo.nowrote:
On 12 Des, 12:56, George Sakkis <george.sak...@gmail.comwrote:
Ah, the 'make' statement.. I liked (and still do) that PEP, I think it
would have an impact comparable to the decorator syntax sugar, if not
more.

I think it is one step closer to Lisp. I believe that it would be
worth considering adding defmacro statement. Any syntax, including if,
else, for, while, class, lambda, try, except, etc. would be
implemented with defmacros. We would only need a minimalistic syntax,
that would bootstrap a full Python syntax on startup. And as for
speed, we all know how Lisp compares to Python.
You say that as if "one step closer to Lisp" is a worthwhile goal.

Python has not become what it is, and achieved the success it has,
because a bunch of people really wanted to use Lisp but didn't think
other people could handle it.

The goal of these sorts of discussions should be to make Python a
better Python. But what happens far too often (especially with
Lispers, but not just them by any means) is that people want to make
Python into a clone or "better" version of whatever other language
they like.

If you're the sort of person who views lisp as the goal that other
languages should aspire to, and I know many of those people exist and
even frequent this list, then you should probably spend your time and
energy on making Lisp a better Lisp and addressing whatever weaknesses
in Lisp have you using Python instead. Trying to fix Lisp (or
whatever) by transforming Python into it isn't going to make you any
happier, and it's just going to derail any discussion of making Python
a better *Python*.
Dec 12 '07 #45

P: n/a
On Wed, Dec 12, 2007 at 06:36:49AM -0800, sturlamolden wrote regarding Re: Is a "real" C-Python possible?:
>
On 12 Des, 12:56, George Sakkis <george.sak...@gmail.comwrote:
Ah, the 'make' statement.. I liked (and still do) that PEP, I think it
would have an impact comparable to the decorator syntax sugar, if not
more.

I think it is one step closer to Lisp. I believe that it would be
worth considering adding defmacro statement. Any syntax, including if,
else, for, while, class, lambda, try, except, etc. would be
implemented with defmacros. We would only need a minimalistic syntax,
that would bootstrap a full Python syntax on startup. And as for
speed, we all know how Lisp compares to Python.
Programmable syntax is a very powerful concept. However, python is designed not only to be powerful, but simple, and this change would drastically reduce the simplicity of Python. It would cease to be a good beginner's language. If you want a language with different syntax than python, python has wonderful parsing libraries. Use those instead.

My 2Â.

Cheers,
Cliff

Dec 12 '07 #46

P: n/a
On 12 Des, 17:44, "J. Clifford Dyer" <j...@sdf.lonestar.orgwrote:
Programmable syntax is a very powerful concept.
You don't have to use the programmable syntax just because it's there.
But I do realize it would be a misfeature if it is abused.

Two points:

* Programmable syntax would make it easier to write an efficient
native compiler. The compiler would only need to know about the small
subset of language used for bootstrapping (i.e. any high-level OOP
constructs could emerge from defmacros).

* Numerical extensions like NumPy create a temporary array when
expressions like '(a+b)*(c+d)' is evaluated. This is because the
overloaded operators do not see the whole expression. Programmable
syntax is a remedy for this.

Dec 12 '07 #47

P: n/a
On 12 Des, 17:00, "Chris Mellon" <arka...@gmail.comwrote:
Python has not become what it is, and achieved the success it has,
because a bunch of people really wanted to use Lisp but didn't think
other people could handle it.

The goal of these sorts of discussions should be to make Python a
better Python.
I do not want to use Lisp. The syntax is awkward and strange, and does
not fit in my brain. I cannot read Lisp code and get a mental image of
what it does. Readability is what sets Python apart.

But I do think programmable syntax can make Python a better Python,
just as it made Lisp a better Lisp.

Dec 12 '07 #48

P: n/a
Christian Heimes <li***@cheimes.dewrites:
>
We are happy and glad for every improvement regarding speed, memory
usage or features if and only if: ...
... platform independent / supported on all platforms. Python runs
on machines from mobile phones to large main frames.
JOOI - there are things in the standard library that are not supported
on all platforms. Why would that be a basis for excluding some
psyco-like package?

Dec 12 '07 #49

P: n/a
Kay Schluehr wrote:
class A(object):
foo = property:
def fget(self):
return self._foo
def fset(self, value):
self._foo = value

which was translated as follows:

class A(object):
def thunk():
def fget(self):
return self._foo
def fset(self, value):
self._foo = value
return vars()
foo = propery(**thunk())
del thunk
Python 2.6 and 3.0 have a more Pythonic way for the problem:

class A(object):
@property
def foo(self):
return self._foo

@foo.setter
def foo(self, value)
self._foo = value

@foo.deletter
def foo(self)
del self._foo

class B(A):
# one can even overwrite the getter in a subclass
@foo.getter
def foo(self):
return self._foo * 2

Christian
Dec 12 '07 #50

71 Replies

This discussion thread is closed

Replies have been disabled for this discussion.