Python compilers?

[Svein Ove Aas]

Is anyone working on a python-to-native compiler?
I'd be interested in taking a look.

Come to think of it, is anyone working on a sexpr-enabled version of
Python, or anything similar? I really miss my macros whenever I try to
use it...

[Grant Edwards]

On 2004-05-18, Svein Ove Aas <sv************ @brage.info> wrote:

Seems to me that you want 'smarter', not 'worse'. I can't take a language
seriously if it says that 1/3 is 0.33333... .

Dude, didn't you take high-school math? 1/3 _is_ 0.33333...

--
Grant Edwards grante Yow! My forehead feels
at like a PACKAGE of moist
visi.com CRANBERRIES in a remote
FRENCH OUTPOST!!

[Svein Ove Aas]

Brian Quinlan wrote:

Svein Ove Aas wrote:

Erk.
Seems to me that you want 'smarter', not 'worse'. I can't take a
language seriously if it says that 1/3 is 0.33333... .

What is that? Are you arguing for an integer result, a fixed result or
a rational result?

A rational result, of course. If I'm okay with losing precision, I'll
*tell* the language I'm okay with losing precision.

[Heiko Wundram]

Am Dienstag, 18. Mai 2004 13:41 schrieb Jacek Generowicz:

Native compilers for other languages just as dynamic as Python
exist. These compilers manage to achieve very significant speed
increases[*].

You are again refering to LISP as an example of a dynamic language which when
compiled gives huge speed increases. This is true in some respect, in others
it isn't. LISP has the advantage that type-inference may be used throughout
the program to create one version of each function, which can then be
compiled. Of course it still has to call into runtime functions to do the
high-level work, but there is actually only one representation of each
finished LISP program, and only one set and one proper order of
runtime-functions to call.

In Python this isn't true. Python, instead of LISP, is "completely " dynamic,
meaning that it's pretty impossible to do type-inference for each function
that is called (even checking types isn't possible). E.g. how do you expect
type-inference to work with the pickle module? string -> something/Error
would be the best description what pickle does. For the function which calls
pickle, do you want to create versions for each possible output of Pickle?
Which outputs of Pickle are possible? (depends on the loaded modules, which
can be loaded at runtime) There is no (sane) way to create machine-code which
calls into the appropriate (low-level) Python-runtime functions (such as
Py_List*, Py_Dict*, etc.) for such a method, at least not at compile-time.

At runtime, this is possible. See what psyco does. There's a nice presentation
on the psyco-website which explains what it does, and I guess you'll
understand when you see that why writing a "compiler" for Python is pretty
impossible.

Heiko.

[Tor Iver Wilhelmsen]

Grant Edwards <gr****@visi.co m> writes:

Dude, didn't you take high-school math? 1/3 _is_ 0.33333...

No, because at some point you will stop writing 3's, either out of
boredom, exhaustion or because you need to pee. At that instant, you
introduce a rounding error, making 3 * 1/3 = 0.99999999999.. . instead
of 1.0

[SeeBelow]

Leif K-Brooks wrote:

Svein Ove Aas wrote:

Is anyone working on a python-to-native compiler?
I'd be interested in taking a look.

What are you trying to achieve?

If it's faster code execution, the primary slowdown with a very
high-level language like Python is caused by high-level data structures
(introspection, everything being an object, etc.), not the code itself.
A native compiler would still have to use high-level data structures to
work with all Python code, so the speed increase wouldn't be very much.

Yes, fast execution. I have been using C. In my applications there is
a population of "chromosome s" which are arrays of floats, about 2 to 5 k
in length. Then there are subroutines which operate on a chromosome
using pointers. For example, the "crossover" routine uses two pointers
to swap portions of two chromosomes. My software sometimes runs for
hours, perform many millions of operations like these. Clearly, speed
of execution is of dramatic importance.

A related problem is that it seems to be a big deal to call C routines
from Python. I have not actually tried it, because when I read about
how its done I was not able to understand it.

Mitchell Timin

--
"Many are stubborn in pursuit of the path they have chosen, few in
pursuit of the goal." - Friedrich Nietzsche

http://annevolve.sourceforge.net is what I'm into nowadays.
Humans may write to me at this address: zenguy at shaw dot ca

[Svein Ove Aas]

Heiko Wundram wrote:

Am Dienstag, 18. Mai 2004 13:41 schrieb Jacek Generowicz:

Native compilers for other languages just as dynamic as Python
exist. These compilers manage to achieve very significant speed
increases[*].

You are again refering to LISP as an example of a dynamic language which
when compiled gives huge speed increases. This is true in some respect,
in others it isn't. LISP has the advantage that type-inference may be
used throughout the program to create one version of each function,
which can then be compiled. Of course it still has to call into runtime
functions to do the high-level work, but there is actually only one
representation of each finished LISP program, and only one set and one
proper order of runtime-functions to call.

In Python this isn't true. Python, instead of LISP, is "completely "
dynamic, meaning that it's pretty impossible to do type-inference for
each function that is called (even checking types isn't possible). E.g.
how do you expect type-inference to work with the pickle module? string
-> something/Error would be the best description what pickle does. For
the function which calls pickle, do you want to create versions for each
possible output of Pickle? Which outputs of Pickle are possible?
(depends on the loaded modules, which can be loaded at runtime) There is
no (sane) way to create machine-code which calls into the appropriate
(low-level) Python-runtime functions (such as Py_List*, Py_Dict*, etc.)
for such a method, at least not at compile-time.

I suppose that's true; pickle is an exception, and the compiler would pick
that up. (The Lisp (print) function is approximately equivalent, at least
when *print-readably* is true.)

What you're claiming, though, is that it's possible to write Python code
that can't easily be translated to equivalent Lisp code. Can you give an
example?

[Thorsten Kampe]

* Tor Iver Wilhelmsen (2004-05-18 17:26 +0100)

Grant Edwards <gr****@visi.co m> writes:

Dude, didn't you take high-school math? 1/3 _is_ 0.33333...

No, because at some point you will stop writing 3's, either out of
boredom, exhaustion or because you need to pee. At that instant, you
introduce a rounding error, making 3 * 1/3 = 0.99999999999.. . instead
of 1.0

Must have been a long time since you went to school... 1/3 is
/exactly/ 0.3...: http://mathworld.wolfram.com/RepeatingDecimal.html

Thorsten

[Yermat]

Heiko Wundram wrote:

[...]

In Python this isn't true. Python, instead of LISP, is "completely " dynamic,
meaning that it's pretty impossible to do type-inference for each function
that is called (even checking types isn't possible). E.g. how do you expect
type-inference to work with the pickle module? string -> something/Error
would be the best description what pickle does. For the function which calls
pickle, do you want to create versions for each possible output of Pickle?
Which outputs of Pickle are possible? (depends on the loaded modules, which
can be loaded at runtime) There is no (sane) way to create machine-code which
calls into the appropriate (low-level) Python-runtime functions (such as
Py_List*, Py_Dict*, etc.) for such a method, at least not at compile-time.

At runtime, this is possible. See what psyco does. There's a nice presentation
on the psyco-website which explains what it does, and I guess you'll
understand when you see that why writing a "compiler" for Python is pretty
impossible.

That is here where you are wrong !
It can be known at compile time if you know every modules that will be
imported. That is called "closed environment".

If it is the case, you would be able to compile a program even if you
would not be allowed to do incremental compilation... That just mean
that you will need to recompile everything each time you modified something.

The demonstration is quite easy:
In a "closed environment" there is a finite number of classes. So you
just have to create as much specialized functions as classes. Then where
you can infere type, call directly the good function. Elsewhere, just
call a runtime dispatcher. In fact, it is already used in some languages
like Eiffel with certain optimizations.

The problem is that many python programs are not "closed" at
compile-time, ie they import or eval stuff only known at run-time.

impossible n'est pas français ;-)

--
Yermat

[simo]

Leif K-Brooks <eu*****@ecritt ers.biz> wrote:

Is anyone working on a python-to-native compiler?
I'd be interested in taking a look.

If it's faster code execution, the primary slowdown with a very
high-level language like Python is caused by high-level data structures
(introspection, everything being an object, etc.), not the code itself.
A native compiler would still have to use high-level data structures to
work with all Python code, so the speed increase wouldn't be very much.
I think the main slowdown most people would see is the startup time -
the interpreter takes way too long to start (why the company where I
work won't replace Perl/PHP with Python).
If it's ease of distribution you're looking for, I think distutils can
make standalone programs on Windows, and most Linux distros have Python
installed by default.
Oh that one is funny - do most distro's have PyQT and wxPython (and
their supporting Qt/GKT libs) installed by default, oh and which ones
come with 2.3 by default?
If you just think a compiler would be cool (or would like to see how it
would be done), check out Psyco, Pyrex, and probably some other
projects. Pyrex even combats the speed issue by allowing native C types
to be used in addition to Python high-level types.

I thought Pyrex was a hybrid of C and Python (like Jython/Java) not
actually a Python-to-C convertor? And Pysco is just a different VM
isn't it?

I could really push Python where I work if there was a native
compiler, my company uses C/C++/Java/Qt and were looking at QSA as a
way to allow the user to script things, but as all of our products
integrate with our software protection system, we can't be
distributing source or easily decompiled bytecode!

We could replace the whole lot with PyQt and use an embedded Python
interpreter for the scripting! Ah the frustration :-(

[Yermat]

simo a écrit :

[...]

I thought Pyrex was a hybrid of C and Python (like Jython/Java) not
actually a Python-to-C convertor? And Pysco is just a different VM
isn't it?
Pyrex is python plus access to C structures and type declarations. But
then the current implementation create intermediary C files.

Psyco is not a different VM, it's like the JIT of java. It's a Just In
Time compilers, ie it runs over the CPythonVM.
[...]

--
Yermat