Python compilers?

[Svein Ove Aas]

Is anyone working on a python-to-native compiler?
I'd be interested in taking a look.

Come to think of it, is anyone working on a sexpr-enabled version of
Python, or anything similar? I really miss my macros whenever I try to
use it...

[Carl Banks]

Paul Rubin wrote:

Carl Banks <im*****@aerojo ckey.invalid> writes:

> The example above kills any attempt to turn a.bar() into a static
> procedure call.

Of course it does--but it's one method. A compiler, if it's good,
would only make the optization on methods named "bar", and it could
probably pare the number of possible classes it could happen to down
to only a few.

How could it possibly know? The reassignment of a.bar could happen
anytime, anywhere in the code. Maybe even in an eval.

And if it happens anytime, anywhere in the code, the compiler will see
it and create more general code. Or is that impossible?

As for eval, well it would be silly to even allow eval in a compiled
program; kind of defeats the purpose of compling. You might let it
run in its own namespace so it can only affect certain objects.
--
CARL BANKS http://www.aerojockey.com/software
"If you believe in yourself, drink your school, stay on drugs, and
don't do milk, you can get work."
-- Parody of Mr. T from a Robert Smigel Cartoon

[asdf as]

Svein Ove Aas wrote:

Is anyone working on a python-to-native compiler?
I'd be interested in taking a look.

Come to think of it, is anyone working on a sexpr-enabled version of
Python, or anything similar? I really miss my macros whenever I try to
use it...

[Fuzzyman]

si************@ yahoo.co.uk (simo) wrote in message news:<30******* *************** ****@posting.go ogle.com>...

Leif K-Brooks <eu*****@ecritt ers.biz> wrote:

Is anyone working on a python-to-native compiler?
I'd be interested in taking a look.
Various people on various projects... most still at the vapourware
stage.
In essence python is a dynamic interpreted language though - in order
to make it suitable for 'compilation' you have to remove some of the
advantages.

Funnily though, most people call Java a compiled language, but it only
compiles to Java bytecode which runs on the virtual machine. Python
precompiles to python bytecode which runs on the python virtual
machine. So arguably it is *as* compiled as Java.....
If it's faster code execution, the primary slowdown with a very
high-level language like Python is caused by high-level data structures
(introspection, everything being an object, etc.), not the code itself.
A native compiler would still have to use high-level data structures to
work with all Python code, so the speed increase wouldn't be very much.

I think the main slowdown most people would see is the startup time -
the interpreter takes way too long to start (why the company where I
work won't replace Perl/PHP with Python).

It's not a problem I've been particularly aware of.
Lots of projects to increase execution speed. Psyco is *very* good and
very useable of x86 machines. I use it and see a real speed increase
of approx 10 times on a couple of applications I 've done - anagram
generator and binary interleaving... both of which are processor
intensive. It doesn't address startup speed though.........

If it's ease of distribution you're looking for, I think distutils can
make standalone programs on Windows, and most Linux distros have Python
installed by default.

Oh that one is funny - do most distro's have PyQT and wxPython (and
their supporting Qt/GKT libs) installed by default, oh and which ones
come with 2.3 by default?

No - and worse PyQT is commercial only on windoze.
You need py2exe (or similar) to make standalone programs that will run
without python. It's not difficult though.

If you just think a compiler would be cool (or would like to see how it
would be done), check out Psyco, Pyrex, and probably some other
projects. Pyrex even combats the speed issue by allowing native C types
to be used in addition to Python high-level types.

I thought Pyrex was a hybrid of C and Python (like Jython/Java) not
actually a Python-to-C convertor? And Pysco is just a different VM
isn't it?

That's right - Pyrex is a new language that mixes python and C syntax.
You can also use it to easily build python extensions.
I could really push Python where I work if there was a native
compiler, my company uses C/C++/Java/Qt and were looking at QSA as a
way to allow the user to script things, but as all of our products
integrate with our software protection system, we can't be
distributing source or easily decompiled bytecode!

We could replace the whole lot with PyQt and use an embedded Python
interpreter for the scripting! Ah the frustration :-(

if you're lookibng at scripting then you'll be embedding an
interpreter *anyway*.
There *isn't* a native compiler... because python isn't a compiled
language.
*But* bytecode isn't *easily* decompiled (and decompyler is no longer
publicly available) and speed critical stuff/stuff that needs to be
compiled for security reasons can be compiled using Pyrex - which
handles all the interfacing of python to C and you get the best of
both worlds in terms of syntax.

No reason you shouldn't distribute the PyQt libraries *with* your
program - assuming you stick to the relevant licenses.

Regards,
Fuzzy

http://www.voidspace.org.uk/atlantib...thonutils.html

[Heather Coppersmith]

On Fri, 21 May 2004 03:03:51 GMT,
Carl Banks <im*****@aerojo ckey.invalid> wrote:

Paul Rubin wrote:

Carl Banks <im*****@aerojo ckey.invalid> writes:

> The example above kills any attempt to turn a.bar() into a static
> procedure call.

Of course it does--but it's one method. A compiler, if it's good,
would only make the optization on methods named "bar", and it could
probably pare the number of possible classes it could happen to down
to only a few.
How could it possibly know? The reassignment of a.bar could happen
anytime, anywhere in the code. Maybe even in an eval.

And if it happens anytime, anywhere in the code, the compiler will see
it and create more general code. Or is that impossible?
The compiler might not see it. Any function/method/module called while
a is in scope might change a.bar. It doesn't even take a perverted
introspection tool:

module foo:

import bar
class C:
pass
a = C( )
a.bar = 'bar'
bar.bar( a )
a.bar = a.bar + 'bar' # which 'add' instruction should this compile to?

module bar:

def bar( x ):
x.bar = 3
As for eval, well it would be silly to even allow eval in a compiled
program; kind of defeats the purpose of compling. You might let it
run in its own namespace so it can only affect certain objects.

I reiterate my comments regarding type declarations, add new comments
regarding backwards compatibility, and note that Lisp allows eval in
compiled code.

Regards,
Heather

--
Heather Coppersmith
That's not right; that's not even wrong. -- Wolfgang Pauli

[Peter Hansen]

Fuzzyman wrote:

Funnily though, most people call Java a compiled language, but it only
compiles to Java bytecode which runs on the virtual machine. Python
precompiles to python bytecode which runs on the python virtual
machine. So arguably it is *as* compiled as Java.....

Actually, there are compilers that produce native machine code from
Java for several CPUs available, and they are used at least in the
embedded world.

-Peter

[Svein Ove Aas]

Peter Hansen wrote:

Fuzzyman wrote:

Funnily though, most people call Java a compiled language, but it only
compiles to Java bytecode which runs on the virtual machine. Python
precompiles to python bytecode which runs on the python virtual
machine. So arguably it is *as* compiled as Java.....

Actually, there are compilers that produce native machine code from
Java for several CPUs available, and they are used at least in the
embedded world.

There is also GCJ as part of the GCC, which can compile both .class
and .java files. Its libraries aren't complete yet, but I'm sure it's
only a matter of time.

[Carl Banks]

Heather Coppersmith wrote:

On Fri, 21 May 2004 03:03:51 GMT,
Carl Banks <im*****@aerojo ckey.invalid> wrote:

Paul Rubin wrote:

Carl Banks <im*****@aerojo ckey.invalid> writes:
> The example above kills any attempt to turn a.bar() into a static
> procedure call.

Of course it does--but it's one method. A compiler, if it's good,
would only make the optization on methods named "bar", and it could
probably pare the number of possible classes it could happen to down
to only a few.

How could it possibly know? The reassignment of a.bar could happen
anytime, anywhere in the code. Maybe even in an eval.

And if it happens anytime, anywhere in the code, the compiler will see
it and create more general code. Or is that impossible?

The compiler might not see it. Any function/method/module called while
a is in scope might change a.bar. It doesn't even take a perverted
introspection tool:

module foo:

import bar
class C:
pass
a = C( )
a.bar = 'bar'
bar.bar( a )
a.bar = a.bar + 'bar' # which 'add' instruction should this compile to?

module bar:

def bar( x ):
x.bar = 3

Compiler builds a big call-tree. Compiler sees that the object "a" is
passed to a function that might rebind bar. Compiler thinks to
itself, "better mark C.bar as a dynamic attribute." Compiler sees
dynamic attirbute and addition. Compiler generates generic add code.

You could have all kinds of setattrs, evals, and unanalysable data
structures (a la pickle), that could turn the code into a nightmare
that a compiler couldn't do anything with. But surely, guarding
against all of that, a good enough static compiler can still reduce a
lot of good, maintainable Python code to it's static case.
And, frankly, I still don't see how this is different from Lisp.

(defun foo ()
(let ((x 1))
(setq x (bar x))
(setq x (+ x 1))))

In another package:

(defun bar (x)
#C(2.0 1.0))

If I recall, + can work on ints, floats, bignums, rationals, and
complex numbers, at least. What one instruction does + compile to
here?

As for eval, well it would be silly to even allow eval in a compiled
program; kind of defeats the purpose of compling. You might let it
run in its own namespace so it can only affect certain objects.

I reiterate my comments regarding type declarations, add new comments
regarding backwards compatibility, and note that Lisp allows eval in
compiled code.

It's highly frowned upon cause it interferes with everything it
touches.
--
CARL BANKS http://www.aerojockey.com/software
"If you believe in yourself, drink your school, stay on drugs, and
don't do milk, you can get work."
-- Parody of Mr. T from a Robert Smigel Cartoon

[Andrew MacIntyre]

On Fri, 21 May 2004, Svein Ove Aas wrote:

Peter Hansen wrote:

Actually, there are compilers that produce native machine code from
Java for several CPUs available, and they are used at least in the
embedded world.

There is also GCJ as part of the GCC, which can compile both .class
and .java files. Its libraries aren't complete yet, but I'm sure it's
only a matter of time.

Hmmm... anyone tried GCJ on Jython?

--
Andrew I MacIntyre "These thoughts are mine alone..."
E-mail: an*****@bullsey e.apana.org.au (pref) | Snail: PO Box 370
an*****@pcug.or g.au (alt) | Belconnen ACT 2616
Web: http://www.andymac.org/ | Australia

[Paul Rubin]

Carl Banks <im*****@aerojo ckey.invalid> writes:

If I recall, + can work on ints, floats, bignums, rationals, and
complex numbers, at least. What one instruction does + compile to
here?

Lisp supports type declarations which advise the compiler in those
situations. A few such proposals have been made for Python, but none
have taken off so far.

[Carl Banks]

Paul Rubin wrote:

Carl Banks <im*****@aerojo ckey.invalid> writes:

If I recall, + can work on ints, floats, bignums, rationals, and
complex numbers, at least. What one instruction does + compile to
here?

Lisp supports type declarations which advise the compiler in those
situations. A few such proposals have been made for Python, but none
have taken off so far.

Yes, that's been established. There's two questions remaining for me:

1. These claims that Lisp code can approach 50 percent the speed of C,
is that with or without the optional type declarations?

2. If you don't use the declarations, does compiling Lisp help? If it
does (and nothing I've read indicated that is doesn't), it
definitely casts some doubt on the claim that compiling Python
wouldn't help. That's kind of been my point all along.

I think (and I'm wrapping this up, cause I think I made my point)
compiling Python could help, even without type declarations, but
probably not as much as in Lisp. It could still make inferences or
educated guesses, like Lisp compilers do; just maybe not as often.
--
CARL BANKS http://www.aerojockey.com/software
"If you believe in yourself, drink your school, stay on drugs, and
don't do milk, you can get work."
-- Parody of Mr. T from a Robert Smigel Cartoon