Can one run a 1.5 .pyc file with the 2.x version
interpreters and vice versa?
How about running a 2.x .pyc using a 2.y interpreter?
46  6175 
>>>>> "Jon" == Jon Perez <jb********@yahoo.com> writes:
Jon> Can one run a 1.5 .pyc file with the 2.x version
Jon> interpreters and vice versa?
Nope, bytecode compatibility is not guaranteed.
--
Ville Vainio http://tinyurl.com/2prnb
Jon Perez <jb********@yahoo.com> writes: Can one run a 1.5 .pyc file with the 2.x version
interpreters and vice versa?
How about running a 2.x .pyc using a 2.y interpreter?
Both are hopeless, in practice.
Cheers,
mwh
--
ZAPHOD: OK, so ten out of ten for style, but minus several million
for good thinking, eh?
-- The Hitch-Hikers Guide to the Galaxy, Episode 2
Jon Perez wrote: Can one run a 1.5 .pyc file with the 2.x version
interpreters and vice versa?
How about running a 2.x .pyc using a 2.y interpreter?
What is the specific need you have? There may be ways around
your problem.
-Peter
Peter Hansen wrote: Jon Perez wrote:
Can one run a 1.5 .pyc file with the 2.x version
interpreters and vice versa?
How about running a 2.x .pyc using a 2.y interpreter?
What is the specific need you have? There may be ways around
your problem.
-Peter
I was just curious wrt deployment issues. So I guess it is
fair to say that .pyc files are not as flexible as Java
class files which are at least forward compatible in most
cases (i.e. old class files will run on newer JVMs).
I'm sure a 2.N.x .pyc can run on a 2.N.y interpreter though,
right? :-D
The answers also make me curious about how CPython's VM works.
I suppose that, unlike Java, it is constantly being reworked
in more extensive ways from one dot-version to another. Would
that also explain why there is no detailed spec detailing how
CPython's VM works (since its design is not as nailed down as,
say, the JVM)?
As a programmer, I find installing (and uninstalling) Python
extensions to be about as easy as one can wish for. However,
for end users, it would be great if we can someday have the
equivalent of a Java Runtime Environment with all the important
extensions already included. Then Python developers can distribute
..pycs to end-users and expect them to work with few to no issues.
The things I would like to see in this hypothetical Python Runtime
Environment (PRE) would be Pygame, PyOpenGL, PyGtk (once it finally
matures on Aqua), and the other popular extensions they rely on (like
Numeric - although the current PyOpenGL is experimenting with
using numarray instead).
It may not necessarily have to be the current lightweight distro being
released by the offical Python maintainers, but a separate one
specifically geared for end users who mainly wish to use 3rd party apps
written in Python. Due to Python's nature, such a PRE would also
automatically be an effective developer environment, nullifying the
need for an SDK edition.
> I was just curious wrt deployment issues. So I guess it is fair to say that .pyc files are not as flexible as Java
class files which are at least forward compatible in most
cases (i.e. old class files will run on newer JVMs).
It is not about Python being "less flexible" than Java, Python is
evolving. It is also stated in the docs somewhere that bytecode is not
guaranteed to be forwards or backwards compatible.
I'm sure a 2.N.x .pyc can run on a 2.N.y interpreter though,
right? :-D
Yeah, they try not to do bytecode changes mid point releases.
The answers also make me curious about how CPython's VM works.
I suppose that, unlike Java, it is constantly being reworked
in more extensive ways from one dot-version to another. Would
that also explain why there is no detailed spec detailing how
CPython's VM works (since its design is not as nailed down as,
say, the JVM)?
Virtual machine, interpreter, call it what you want. There is probably
documentation somewhere, but the best idea (if you are interested) is to
just take a dive into the source. It is all browsable on sourceforge.
As a programmer, I find installing (and uninstalling) Python
extensions to be about as easy as one can wish for. However,
for end users, it would be great if we can someday have the
equivalent of a Java Runtime Environment with all the important
extensions already included. Then Python developers can distribute
..pycs to end-users and expect them to work with few to no issues.
IMO you are better off packaging them with py2exe or an equivalent for
your platform. Then you can guarantee that your users are running
python version x.y.z, and not x.y.a (minor releases are to fix
relatively important bugs, which may effect the execution of your program).
The things I would like to see in this hypothetical Python Runtime
Environment (PRE) would be Pygame, PyOpenGL, PyGtk (once it finally
matures on Aqua), and the other popular extensions they rely on (like
Numeric - although the current PyOpenGL is experimenting with
using numarray instead).
Such a PRE would not be small, and trying to get compatible versions of
all of those 3rd party modules, for every supported platform, would be a
pain in the ass.
It may not necessarily have to be the current lightweight distro being
released by the offical Python maintainers, but a separate one
specifically geared for end users who mainly wish to use 3rd party apps
written in Python. Due to Python's nature, such a PRE would also
automatically be an effective developer environment, nullifying the
need for an SDK edition.
There already exist 3rd party Python distributions with various add-ons.
I don't know of any off the top of my head (because I have no need for
them), but I have heard mention of them here.
- Josiah
Josiah Carlson wrote: It is not about Python being "less flexible" than Java, Python is
evolving.
Note that I specifically mentioned .pyc files vs. java class files,
not Java the language/environment/platform vs. Python.
It is also stated in the docs somewhere that bytecode is not
guaranteed to be forwards or backwards compatible.
Ah ok.
IMO you are better off packaging them with py2exe or an equivalent for
your platform. Then you can guarantee that your users are running
python version x.y.z, and not x.y.a (minor releases are to fix
relatively important bugs, which may effect the execution of your program).
Right. I am aware that freezing is the optimal way to distribute
apps written in Python today. After trying out Java though, I have
to say I'm impressed with the way you can take a bunch of class
files archived in the exact same jarfile to *ANY* computer with a
JRE installed, type in java -jar jarfile and have it run _identically_.
Perhaps one day .pyc files can work in such a manner.
Such a PRE would not be small, and trying to get compatible versions of
all of those 3rd party modules, for every supported platform, would be a
pain in the ass.
That it would not be small I will agree with you on, however I don't
believe version [in]compatibilities will become a PITA as it isn't so
even today.
The whole idea of a Python Runtime Environment came to me precisely
because I have tried out Pygame, PyOpenGL, PyGTK on both Win32 and
Linux and they work seamlessly and uniformly on both platforms (not sure
on OS X though). Note that these are modules that tie intimately with
_different_ low-level APIs on each platform but they work so flawlessly
that it reminds of me of Java's WORA which has been a reality for a
long time now. The main difference is that with Python, getting all
these different modules (for multimedia, 3d acceleration, GUI) installed
involves more steps (though quite easy). Also, because these work
on top of C-based APIs, the performance is significantly superior to
Java's.
There already exist 3rd party Python distributions with various add-ons.
I don't know of any off the top of my head (because I have no need for
them), but I have heard mention of them here.
ActiveState's and Pythonware's are the two I am aware of.
| ....
| ActiveState's and Pythonware's are the two I am aware of
There are also the Enthought distributions ....
Python 2.3 and Python 2.2.3
for Windows (Enthought Edition) .... http://www.enthought.com/
I've been using the 2.3 version successfully
for several months and chose it specifically
because several add-on packages were included
that I didn't have to seek out and otherwise
individually install ....
--
Cousin Stanley
Human Being
Phoenix, Arizona
"Jon Perez" <jb********@yahoo.com> wrote in message
news:c3*************@ID-180326.news.uni-berlin.de... The answers also make me curious about how CPython's VM works.
I suppose that, unlike Java, it is constantly being reworked
in more extensive ways from one dot-version to another. Would
that also explain why there is no detailed spec detailing how
CPython's VM works (since its design is not as nailed down as,
say, the JVM)?
The CPython VM is intentionally underdocumented and intentionally not part
of the language spec. It is still being developed and might radically
change in the future. There will probably be minor changes in the next
release as a result of experiments designed to find combinations of codes
that dependably run faster on all major platforms. However, the current
byte code is described in the Library Ref as part of the doc for the dis
(assembly) module.
Terry J. Reedy
In article <c3*************@ID-180326.news.uni-berlin.de>,
Jon Perez <jb********@yahoo.com> wrote:
Right. I am aware that freezing is the optimal way to distribute
apps written in Python today. After trying out Java though, I have
to say I'm impressed with the way you can take a bunch of class
files archived in the exact same jarfile to *ANY* computer with a
JRE installed, type in java -jar jarfile and have it run _identically_.
Really? I'm sorry, I have just a tiny bit of difficulty believing that,
considering all the stories i've heard about Java not being completely
cross-platform and cross-version. I am sure that simple cases work as
you describe, but I'd like a cite for Java working as you describe in
general.
--
Aahz (aa**@pythoncraft.com) <*> http://www.pythoncraft.com/
"usenet imitates usenet" --Darkhawk
> Right. I am aware that freezing is the optimal way to distribute apps written in Python today. After trying out Java though, I have
to say I'm impressed with the way you can take a bunch of class
files archived in the exact same jarfile to *ANY* computer with a
JRE installed, type in java -jar jarfile and have it run _identically_.
Modulo other dependencies also being installed as well.
You can do the same thing with Python. Just package up the source
files as a zip.
$ PYTHONPATH=example.zip python main.py
You are considering Python .pyc files and Java class files to be
the same thing. They aren't. pyc files are a behind the scenes
implementation detail of one version of the Python interpreter
that are automatically generated and maintained.
Java class files are how you distribute a collection of Java code.
A zip of Python source files is how you distribute Python code
(you can also do it more formally as packages, installers etc).
Roger
In article <c3*************@ID-180326.news.uni-berlin.de>, Jon Perez wrote: Right. I am aware that freezing is the optimal way to distribute
apps written in Python today. After trying out Java though, I have
to say I'm impressed with the way you can take a bunch of class
files archived in the exact same jarfile to *ANY* computer with a
JRE installed, type in java -jar jarfile and have it run _identically_.
Spoken like a man who hasn't used much Java.
Joe
On Sat, 20 Mar 2004, Jon Perez wrote: Right. I am aware that freezing is the optimal way to distribute
apps written in Python today. After trying out Java though, I have
to say I'm impressed with the way you can take a bunch of class
files archived in the exact same jarfile to *ANY* computer with a
JRE installed, type in java -jar jarfile and have it run _identically_.
Perhaps one day .pyc files can work in such a manner.
Perhaps.
However there is a clear differentiation between Java and Python
in this area:
- Java has always been specified in terms of the language and a
virtual machine design (which implies a portable binary form files for
executables - jar files);
- Python has always been specified in terms of the language, with a
portable reference interpreter implementation for that language (which
just happens to compile to bytecode, and save that bytecode to disk
as an optimisation).
IOW, the bytecode is an implementation detail for _one_ interpreter
implementation.
This becomes obvious when you consider the possibilities associated
with Jython, which could (already does?) compile Python source to jar
files directly.
--
Andrew I MacIntyre "These thoughts are mine alone..."
E-mail: an*****@bullseye.apana.org.au (pref) | Snail: PO Box 370 an*****@pcug.org.au (alt) | Belconnen ACT 2616
Web: http://www.andymac.org/ | Australia
Aahz wrote: Right. I am aware that freezing is the optimal way to distribute
apps written in Python today. After trying out Java though, I have
to say I'm impressed with the way you can take a bunch of class
files archived in the exact same jarfile to *ANY* computer with a
JRE installed, type in java -jar jarfile and have it run _identically_.
Really? I'm sorry, I have just a tiny bit of difficulty believing that,
considering all the stories i've heard about Java not being completely
cross-platform and cross-version. I am sure that simple cases work as
you describe, but I'd like a cite for Java working as you describe in
general.
Yes. Java works like that and has for a long time already. One of the
most notable examples i can give is the SVG viewer Batik (not a trivial
program). Worked liked a charm on both Linux and Win32, and it does not
rely on an exact JRE on which it will run on properly. Virtually all
the Swing apps you can try out will work just as reliably and identically
on different platforms as Batik.
The real problem for me is that Java's sound and graphics APIs are not
up to par. Also, the language is clearly not as nice as Python. But
Java _is_ pretty cool and it is a solid (albeit somewhat resource hungry)
piece of technology.
Joe Mason wrote: Right. I am aware that freezing is the optimal way to distribute
apps written in Python today. After trying out Java though, I have
to say I'm impressed with the way you can take a bunch of class
files archived in the exact same jarfile to *ANY* computer with a
JRE installed, type in java -jar jarfile and have it run _identically_.
Spoken like a man who hasn't used much Java.
Joe
If you're speaking from experience, then I'd be interested to
hear exactly what it is you've encountered that leads you to
this view of Java. My experience so far has been contrary to
that.
Automatically assuming that the reason others don't happen to
share your view is due to their lack of experience, just shows
that you are full of conceit.
Andrew MacIntyre wrote: - Python has always been specified in terms of the language, with a
portable reference interpreter implementation for that language (which
just happens to compile to bytecode, and save that bytecode to disk
as an optimisation).
IOW, the bytecode is an implementation detail for _one_ interpreter
implementation.
This becomes obvious when you consider the possibilities associated
with Jython, which could (already does?) compile Python source to jar
files directly.
Contrary to what you seem to be implying here, Guido never expressly designed
Python to be easily portable to run on top of other bytecode implementations.
Jython was a very difficult feat to pull and the same goes (probably moreso)
for Python.NET.
Roger Binns wrote: You are considering Python .pyc files and Java class files to be
the same thing. They aren't.
They're not 'the same thing'. They are roughly analogous to one
another though, wouldn't you say?
pyc files are a behind the scenes
implementation detail of one version of the Python interpreter
that are automatically generated and maintained.
Huh?? .pyc files are an "implementation detail of the Python
interpreter"? Care to expound further? 'Coz that sounds
pretty nonsensical...
Jon> [ expand jar file - presto! instant app ]
Aahz> .... I am sure that simple cases work as you describe, but I'd
Aahz> like a cite for Java working as you describe in general.
Jon> Yes. Java works like that ... [ mentions Batik ]
The fundamental difference between Java and Python in this regard is that
the JVM must absolutely be part of the defined API since it is delivered
separate from the compiler. Whoever does this stuff probably has to go to
great lengths to preserve compatibility. This is not the case for the
Python VM, since a new version is delivered with each distribution.
Skip
Skip Montanaro wrote: The fundamental difference between Java and Python in this regard is that
the JVM must absolutely be part of the defined API since it is delivered
separate from the compiler.
Absolutely untrue. The JVM is part of the Java /spec/, but has nothing to do
with the /API/ (e.g. Java 2 Standard Edition) that application programmers use.
Whoever does this stuff probably has to go to great lengths to preserve
compatibility. This is not the case for the Python VM, since a new version
is delivered with each distribution.
I don't get what you're trying to say here. The JVM has changed very little
(or not at all) over the different Java versions. Neither the incremental
feature-additions to the Java compiler nor modifications to the Java platform
APIs require any great effort to 'maintain compatibility' between them and the
JVM. Even fairly major additions like generics (i.e. Pizza compiler) do not
sound like they require ripping out great chunks of guts and replacing them with
something totally new.
If anything though, what you're saying might apply more to Jython. New Python
language features tend to be more radical / come faster than new Java features
but the Jython authors will have to implement these on a JVM that cannot morph
to accomodate such new features. Still, Jython's continued development proves
that it is possible implement Python's features on a fixed underlying VM.
The only reason I can think of for wanting CPython's VM to remain undocumented /
not-nailed-down is because we want to make it easier / give more freedom to the
language designers to implement the plumbing for new Python features.
Terry Reedy wrote: The CPython VM is intentionally underdocumented and intentionally not part
of the language spec. It is still being developed and might radically
change in the future. There will probably be minor changes in the next
release as a result of experiments designed to find combinations of codes
that dependably run faster on all major platforms. However, the current
byte code is described in the Library Ref as part of the doc for the dis
(assembly) module.
This is the answer I was looking for... :-) Thanks, Terry. pyc files are a behind the scenes implementation detail of one
version of the Python interpreter that are automatically generated
and maintained.
Jon> Huh?? .pyc files are an "implementation detail of the Python
Jon> interpreter"? Care to expound further? 'Coz that sounds pretty
Jon> nonsensical...
As I mentioned in my previous post, the distiction between the Java VM and
the Python VM is that the former is delivered separate from the compiler and
other tools, while the latter is delivered afresh with each Python
distribution. Guido would be free to eliminate the Python VM from the next
version of Python and instead directly interpret the abstract syntax tree
emitted by the compiler. PyChecker and other tools which operate directly
with the bytecode would break, but Python applications should still work.
Think of it this way. The Java virtual machine architecture is sort of akin
to the current Wintel architecture. The Python virtual machine architecture
would like Microsoft shipping a new, possibly incompatible, CPU with each
copy of Windows.
Skip
Andrew MacIntyre wrote: This becomes obvious when you consider the possibilities associated
with Jython, which could (already does?) compile Python source to jar
files directly.
A .jar file is just a .zip file composed of a collection of .class files
(JVM bytecodes) plus an additional text-based manifest file.
Jython _does_ compile Python source to .class files (because that is
really the only way to get it to run on the JVM), and the same will apply
for any other language, like Kawa (a Scheme implementation), that run 'on
top of Java' (i.e. gain access to the huge functionality present in
the Java libraries).
Essentially, whenever someone says so-and-so language runs on Java, it means
that they have a compiler for that langauge that emits JVM bytecodes. These
bytecodes can then be run on any implementation of the JVM, whether a JIT-compiling
one or purely interpreted. Further, you can also opt to take that bytecode / .class
file and compile it into native code via a native-code compiler like gcj.
The separation of the layers is very nice and clean and you can substitute
those coming from different sources and expect things to work:
Language Compiler (javac, Jikes, Pizza, Jython, Kawa, JIcon, etc...)
|
| emits .class files which can be packaged into .jars
|
V
JVM implementation + Runtime Environment / native code compiler
(Hotspot, Jalapeno, etc...) (gcj, JRockit, etc...)
|
|
V
native executable
Thus it is reasonable to take the exact same .class file compiled by, say,
Jython or Kawa on a Mac and expect it to run perfectly well on a JVM
implementation running on an x86 or an AS/400 minicomputer. *This*
is what would be nice to have with .pyc files.
Note that there is one other consideration besides the presence of a JVM,
and that is the Runtime Environment (the APIs). Swing may not have been
implemented on the AS/400, but it is implemented on X Window so the
Jython-Swing app you compiled on a Mac would work and look exactly alike
( sluggishly, that is... ;-) ) on a Solaris, FreeBSD, or Linux box.
Skip Montanaro wrote: Jon> [ expand jar file - presto! instant app ]
Also to clarify, describing
java -jar jarfile
as expanding a jarfile is not quite right. It works
exactly as you would run an executable. On many
platforms, you can even just type:
jarfile.jar
on the command line or double click a jar icon to start the
app(*). Not once do you have to worry about extra files
lying around. In fact, it might even be the case that the
class files are loaded into the jvm straight from the
jarfile and not decompressed behind the scenes into a temporary
directory at all.
(*) That's why I think Web Start is a stupid idea that
makes people think Java is more complicated than it really
is.
Roger Binns wrote: Modulo other dependencies also being installed as well.
You can do the same thing with Python. Just package up the source
files as a zip.
$ PYTHONPATH=example.zip python main.py
Just like Java's CLASSPATH (and possibly inspired by it)... this is a
great tip. Does it work on Windows as well? And how about .tar.gz,
..tar.bz or .gzip files?
Skip Montanaro wrote: As I mentioned in my previous post, the distiction between the Java VM and
the Python VM is that the former is delivered separate from the compiler and
other tools, while the latter is delivered afresh with each Python
distribution.
Actually, Sun always bundles Hotspot and javac together in an SDK release.
But yes, you _can_ get and use a compiler separately (Jikes, for example)
from the JVM and there is very little version dependency between the two
(the JVM has stayed the same for a long time now).
Guido would be free to eliminate the Python VM from the next
version of Python and instead directly interpret the abstract syntax tree
emitted by the compiler.
eh...? If interpreting the AST isn't what the CPython VM does, then what
does it do?
PyChecker and other tools which operate directly with the bytecode would
break, but Python applications should still work.
Does this mean that you will need a different version of PyChecker for each
Python .x release?
Skip Montanaro wrote:
Jon> Huh?? .pyc files are an "implementation detail of the Python
Jon> interpreter"? Care to expound further? 'Coz that sounds pretty
Jon> nonsensical...
"The internal structure of a .pyc file depends on the implementation
details of the particular version of the Python interpreter that it was
compiled with"
now when you phrase it that way, it makes sense...
Anyway, we can at least count on the fact that a .pyc compiled
by a particular Python version (say, Python 2.1.1) on one platform
(say, Mac), will run on another platform (say, Solaris) as long
as the interpreter has the same minor version (say, Python 2.1.0).
uh... right?
Jon> Anyway, we can at least count on the fact that a .pyc compiled by a
Jon> particular Python version (say, Python 2.1.1) on one platform (say,
Jon> Mac), will run on another platform (say, Solaris) as long as the
Jon> interpreter has the same minor version (say, Python 2.1.0).
Yup. I believe there were some problems between 32- and 64-bit platforms
ages ago, but they've been resolved.
Skip
The fundamental difference between Java and Python in this regard is
that the JVM must absolutely be part of the defined API since it is
delivered separate from the compiler.
Jon> Absolutely untrue. The JVM is part of the Java /spec/, but has
Jon> nothing to do with the /API/ (e.g. Java 2 Standard Edition) that
Jon> application programmers use.
Sorry, terminology flub. Spec, external interface, API... What I meant was
that the Java folks obligated themselves to keep it compatible. There are
no such claims for the Python VM.
Whoever does this stuff probably has to go to great lengths to
preserve compatibility. This is not the case for the Python VM,
since a new version is delivered with each distribution.
Jon> I don't get what you're trying to say here. The JVM has changed
Jon> very little (or not at all) over the different Java versions.
The Python VM changes frequently, certainly when compared to the Java VM.
The PVM is an implementation detail. Your experience with Batik could not
in general be replicated with a tar file full of Python bytecode, and I'm
not sure I'd want it to be.
Jon> The only reason I can think of for wanting CPython's VM to remain
Jon> undocumented / not-nailed-down is because we want to make it easier
Jon> / give more freedom to the language designers to implement the
Jon> plumbing for new Python features.
Exactly.
Skip Guido would be free to eliminate the Python VM from the next version
of Python and instead directly interpret the abstract syntax tree
emitted by the compiler.
Jon> eh...? If interpreting the AST isn't what the CPython VM does,
Jon> then what does it do?
It interprets the Python bytecodes.
PyChecker and other tools which operate directly with the bytecode
would break, but Python applications should still work.
Jon> Does this mean that you will need a different version of PyChecker
Jon> for each Python .x release?
Not necessarily. It doesn't know everything there is to know about Python
bytecode, but it does know some bytecodes and rummages around in code
objects a bit, so if there was no bytecode it would have nothing to rummage
through.
Skip
> > You are considering Python .pyc files and Java class files to be the same thing. They aren't.
They're not 'the same thing'. They are roughly analogous to one
another though, wouldn't you say?
No. I explained it further in the message.
Huh?? .pyc files are an "implementation detail of the Python
interpreter"? Care to expound further? 'Coz that sounds
pretty nonsensical...
One version of a compliant Python interpretter happens to use
these files, and expose that they exist. They are generated on
demand and automatically updated. The Python interpretter could
have stored them elsewhere on the filesystem (for example in
a hidden "cache" directory), but for simplicity's sake stored
them alongside the source file.
Go ahead and write protect the directories containing your
Python source (after deleting all .pyc files). You will find
everything runs just fine without any visible .pyc files.
If you write your own Python interpretter/compiler, it will be
considered compliant if it implements the language correctly.
There is *nothing* about bytecodes, or bytecode compatibility.
For example if you wanted to write your own that ran on top of
Java, you would probably just directly generate class files.
If you wrote one that ran on a Palm device you would do something
else. It is up to your implementation what it does. The
only requirement would be working with Python *source* files.
Roger
> > $ PYTHONPATH=example.zip python main.py
Just like Java's CLASSPATH (and possibly inspired by it)...
Except predating Java by several years ...
Does it work on Windows as well?
C:\WINDOWS> set PYTHONPATH=example.zip
C:\WINDOWS> python main.py
And how about .tar.gz, .tar.bz or .gzip files?
Nope. The unit of transport is zip files. You can also
point to expanded directories.
Roger
Jon Perez wrote: Anyway, we can at least count on the fact that a .pyc compiled
by a particular Python version (say, Python 2.1.1) on one platform
(say, Mac), will run on another platform (say, Solaris) as long
as the interpreter has the same minor version (say, Python 2.1.0).
uh... right?
Nope. They are an implementation specific thing. The interpretter
puts version and other information in them, and regenerates them
on the fly. There is nothing wrong with an interpretter deciding
to regenerate them every single day in a different format, or
based on the moon phase, or your login id or anything else.
Roger
> Jon> Huh?? .pyc files are an "implementation detail of the Python Jon> interpreter"? Care to expound further? 'Coz that sounds pretty
Jon> nonsensical...
"The internal structure of a .pyc file depends on the implementation
details of the particular version of the Python interpreter that it was
compiled with"
now when you phrase it that way, it makes sense...
No that is not what I meant. What I meant is that the very existence
of .pyc files is an implementation detail of one version of the Python
interpretter. Tomorrow it could use 3 different files, or none at all.
They are pretty much an optional behind the scenes performance optimisation
for CPython. There is no requirement for them to exist.
Tomorrow I could write my own Python interpretter that generates
..tok files, .optim files and .silly files. As long as it executes
Python *source* (ie .py files) then those are an implementation
detail.
In Java, the JVM only executes bytecode. In CPython it only
executes .py files. Behind the scenes as an implementation
optimisation, the .py files are tokenised and transformed into
bytecode and used as internal representation. As an implementation
specific performance measure, they are written out to .pyc
files which are loaded if not out of date in the future instead
of re-parsing the .py files.
You could delete every .pyc file on your system, and then mark
your disk as read-only and your Python programs will run fine.
Java does something similar behind the scenes with HotSpot.
It compiles things to native code, and keeps profiling information.
They are not written out to disk and hence are not persistent
between executions, but if they were the usage would be analogous
to .pyc files.
Roger
In article <c3*************@ID-180326.news.uni-berlin.de>, Jon Perez wrote: Really? I'm sorry, I have just a tiny bit of difficulty believing that,
considering all the stories i've heard about Java not being completely
cross-platform and cross-version. I am sure that simple cases work as
you describe, but I'd like a cite for Java working as you describe in
general.
Yes. Java works like that and has for a long time already. One of the
most notable examples i can give is the SVG viewer Batik (not a trivial
program). Worked liked a charm on both Linux and Win32, and it does not
rely on an exact JRE on which it will run on properly. Virtually all
the Swing apps you can try out will work just as reliably and identically
on different platforms as Batik.
This has not been remotely my experience. Recently I tried 5 different
Linux JVM's for one program, leaving the Sun JRE until last because it
was a bit harder to install, and only the Sun one worked.
When a program *does* work right on multiple JVM's (Limewire, for
instance), I attribute it to the skill of the packagers. The Limewire
team obviously put a lot of effort into that, just by looking at the
number of downloads and installers on their page.
Joe
In article <sl****************@gate.notcharles.ca>, Joe Mason wrote: This has not been remotely my experience. Recently I tried 5 different
Linux JVM's for one program, leaving the Sun JRE until last because it
was a bit harder to install, and only the Sun one worked.
(I meant to add the name of the program in question, but I can't find it
now.)
Joe
Roger Binns wrote: "The internal structure of a .pyc file depends on the implementation
details of the particular version of the Python interpreter that it was
compiled with"
now when you phrase it that way, it makes sense...
No that is not what I meant. What I meant is that the very existence
of .pyc files is an implementation detail of one version of the Python
interpretter. Tomorrow it could use 3 different files, or none at all.
They are pretty much an optional behind the scenes performance optimisation
for CPython. There is no requirement for them to exist.
Is it really the case that it isn't even recommended to distribute one's
application as .pyc files?
In commercial apps for which one might not want to distribute the source
for, it would be really nice if one can count on just requiring the end-user
to install a version of Python (but not extra extension modules anymore) and
then distribute the app as a bunch of .pyc's packaged in a .zip file and
use the PYTHONPATH method you showed. Just remember to provide a set of
..pyc's for each version of Python you support.
Java does something similar behind the scenes with HotSpot.
It compiles things to native code, and keeps profiling information.
They are not written out to disk and hence are not persistent
between executions, but if they were the usage would be analogous
to .pyc files.
In Java, profiling info may not be written out to disk, but the
bytecodes certainly are (they are what is in the class files). Which
is roughly equivalent to what happens with .pyc files, correct? .pyc
files don't contain profiling info, nor was I ever aware that the Python
VM even does run-time behaviour profiling... (Pysco may, though)
If you write your own Python interpretter/compiler, it will be
considered compliant if it implements the language correctly.
There is *nothing* about bytecodes, or bytecode compatibility.
For example if you wanted to write your own that ran on top of
Java, you would probably just directly generate class files.
If you wrote one that ran on a Palm device you would do something
else. It is up to your implementation what it does. The
only requirement would be working with Python *source* files.
Okay, I get it. "Nothing in the spec specifies that .pyc files
have to exist". Further, "even the existence of a VM representation
is not part of the Python spec".
In practice though, there is really just one implementation of Python
(with the standard libraries... excluding Jython) which works using the
same bytecode mechanism on _all_ platforms, so as far as I can tell,
it would be quite feasible to distribute applications in .pyc form.
Joe Mason wrote: In article <sl****************@gate.notcharles.ca>, Joe Mason wrote:
This has not been remotely my experience. Recently I tried 5 different
Linux JVM's for one program, leaving the Sun JRE until last because it
was a bit harder to install, and only the Sun one worked.
If you stick to console programs, you are not likely to experience
problems. However, when it comes to Swing apps, the other JREs
(JVM + J2SE API implementation) out there (with the possible
exception of IBM's) are not mature enough to be practical in use.
The bottom line is that by sticking to Sun's JRE on Linux and Windows,
and Apple's implementation on the OS X, you do have a very effective
WORA platform. The others are largely good only for experimentation.
the Sun JRE until last because it
was a bit harder to install, and only the Sun one worked.
Strange, I didn't find the Sun JRE the least bit difficult
to install on Linux... execute the script, add the j2sdk/ or
j2re/ and/or its bin directory to your PATH, and you can start
doing
java -jar jarfile
under X Window, and run Swing apps immediately.
Roger Binns wrote: Nope. They are an implementation specific thing. The interpretter
puts version and other information in them, and regenerates them
on the fly.
There is nothing wrong with an interpretter deciding
to regenerate them every single day in a different format, or
based on the moon phase, or your login id or anything else.
So we can't even count on the fact that Python 2.2 running under Windows
on an Athlon will generate a .pyc file that's identical with that
generated by the exact same version (2.2) of the interpreter running
under Linux on, say, a Pentium III, and further, is not even *guaranteed*
to run on it?!?
This does not seem to fit in with what I've been reading... it would
be a lot more logical (and convenient) if we can count on .pyc portability
across different _patch versions_ (.x.y) of the same minor version (.x)
of CPython on whatever platform it is running on, and I believe that
Python's contributors would make an effort to ensure that this is the case.
> Is it really the case that it isn't even recommended to distribute one's application as .pyc files?
In commercial apps for which one might not want to distribute the source
for, it would be really nice if one can count on just requiring the end-user
to install a version of Python (but not extra extension modules anymore) and
then distribute the app as a bunch of .pyc's packaged in a .zip file and
use the PYTHONPATH method you showed. Just remember to provide a set of
.pyc's for each version of Python you support.
In the Java world, the JVM is a seperate installable component.
Some installers like ZeroG will install them for you. In general
all Java bytecode is expected (but not required) to share the
main system JVM.
In Python, if you want to distribute an application, and you
don't want the user to have to install Python or any extensions,
you use one of the installer generators out there. They gather
up the interpretter, your code and extension libraries and dump
them all into one directory. They then include a stub that makes
them all work together. It especially has no reliance on any
pre-installed interpretter.
If you want to see this in action, try my project at
http://bitpim.sf.net The downloads give you a program that runs
on Windows, Mac and Linux without any prerequisites. If you
examine the install directory, you will find the find Python
interpretter DLL or binary, all needed extension shared libraries
and the program code (often as .pyc files in a zip archive or
appended to the main binary).
Note that the installer generator happens to know what formats
and other quirks the Python interpretter works with which is why
it does include some .pyc's.
In Java, profiling info may not be written out to disk, but the
bytecodes certainly are (they are what is in the class files). Which
is roughly equivalent to what happens with .pyc files, correct?
You are pedantically correct in that they both contain bytecode, but
the level is different. *If* the JVM wrote out HotSpot compilations
and profiling information, the resulting files would be analogous to
..pyc files. They would not be *required* for execution, and they
would be automatically regenerated if necessary (eg if you changed
source files). In theory it could then execute from them without
requiring the original class files, but that would be an implementation
specific thing.
.pyc
files don't contain profiling info, nor was I ever aware that the Python
VM even does run-time behaviour profiling... (Pysco may, though)
I don't know of any implementations that do, but the implementations
are free to do so. As I keep saying, .pyc files are an implementation
specific detail of one version of one implementation of the Python
interpretter. It can make as many intermediate file formats and contents
as it wants, as can any other implementation. The only requirement for
a conforming Python implementation is that it execute Python *source*,
and how it does it under the scenes is not specified.
In practice though, there is really just one implementation of Python
(with the standard libraries... excluding Jython) which works using the
same bytecode mechanism on _all_ platforms, so as far as I can tell,
it would be quite feasible to distribute applications in .pyc form.
If you distribute the exact interpretter that generated those exact
..pyc files then you will be fine in practise. That is what the various
installer generators out there do. Some examples are py2exe, cx_Freeze
and BundleBuilder which is what my project uses on Windows, Linux and
Mac respectively.
With the addition of zip imports in Python 2.3, I think things will
lead to a unification of the installer generators out there (there have
been several different ones, often limited to a subset of platforms for
historical reasons).
Roger
> So we can't even count on the fact that Python 2.2 running under Windows on an Athlon will generate a .pyc file that's identical with that
generated by the exact same version (2.2) of the interpreter running
under Linux on, say, a Pentium III, and further, is not even *guaranteed*
to run on it?!?
There are no guarantees that the same .pyc files will work the next day
with the same interpretter. The interpretter could change the format
every single day if the implementor so wanted.
This does not seem to fit in with what I've been reading... it would
be a lot more logical (and convenient) if we can count on .pyc portability
across different _patch versions_ (.x.y) of the same minor version (.x)
of CPython on whatever platform it is running on,
Now in practise the .pyc files are not changed every day because there
is no need, but an implementation is free to do so. In many cases they
are the same between platforms and minor versions, but that is not
a specification or a requirement.
and I believe that
Python's contributors would make an effort to ensure that this is the case.
Why? To distribute executable Python code, you distribute the .py files.
To distribute executable Java code, you distribute .class files. They
can be wrapped up in .zip and .jar archives respectively.
Java is defined as a compiled language. Python is not a compiled
language, although an implementation is free to do so. If you really
really want a compiled form of Python, try Jython which compiles to
Java .class files.
Roger
>> In practice though, there is really just one implementation of Python (with the standard libraries... excluding Jython)
That's *two* implementations of Python: CPython and Jython, one of
which uses pyc files. You're conveniently ignoring the primary example
of a Python interpreter that does not use pyc files.
which works using the
same bytecode mechanism on _all_ platforms, so as far as I can tell,
it would be quite feasible to distribute applications in .pyc form.
Yes it is feasible to distribute apps in .pyc form, as long as you
target a specific version of Python.
Ronald
"Ronald Oussoren" <ou******@cistron.nl> wrote in message
news:DD**********************************@cistron. nl... In practice though, there is really just one implementation of Python
(with the standard libraries... excluding Jython)
People who make their living with Jython would find the exclusion puzzling
;-)
That's *two* implementations of Python: CPython and Jython, one of
which uses pyc files. You're conveniently ignoring the primary example
of a Python interpreter that does not use pyc files.
There are actually two more:
3) Viper (now dead, I believe), which translated Python (+ 'enhancements')
to, I believe, with some fuzzyness, OCaml.
4) PyPy (as yet only partial, I believe), which, if funded for development
and spectacularly successful, could become the reference implementation
(but that is at best years away).
Nonetheless, CPython is now the main, reference implementation. 2.x.y .pyc
files are cross-platform portable. I believe the same is also true
regardless of the '.y' part. Reading PyDev, I have seen no evidence that
the planned 2.4 will be very much different.
Terry J. Reedy
Roger Binns wrote: In the Java world, the JVM is a seperate installable component.
Some installers like ZeroG will install them for you. In general
all Java bytecode is expected (but not required) to share the
main system JVM.
Afaik, each app will start its own copy of a JVM.
In Python, if you want to distribute an application, and you
don't want the user to have to install Python or any extensions,
you use one of the installer generators out there. They gather
up the interpretter, your code and extension libraries and dump
them all into one directory. They then include a stub that makes
them all work together. It especially has no reliance on any
pre-installed interpretter.
Like I mentioned earlier, I am aware of 'freezing' but having to bundle
an interpreter with every app is exactly what I would rather avoid...
If you want to see this in action, try my project at
http://bitpim.sf.net The downloads give you a program that runs
on Windows, Mac and Linux without any prerequisites.
What would be great is to not have to choose between a Windows,
Mac or Linux or XXX download (like in a proper Java app).
You are pedantically correct in that they both contain bytecode, but
the level is different. *If* the JVM wrote out HotSpot compilations
and profiling information, the resulting files would be analogous to
.pyc files.
But you're not saying that .pyc files contain profiling information,
right? Because that is what it's sounding like...
They would not be *required* for execution, and they
would be automatically regenerated if necessary (eg if you changed
source files). In theory it could then execute from them without
requiring the original class files, but that would be an implementation
specific thing.
Yes, this is the main difference between Java and Python. Python
prefers the source to be present, Java doesn't care.
With the addition of zip imports in Python 2.3,
Correct me if I'm wrong, but I thought you just said earlier that
zip imports predated Java by several years and now you're saying
they've just been added to Python 2.3?
"Jon Perez" <jb********@yahoo.com> wrote in message
news:c3*************@ID-180326.news.uni-berlin.de... In commercial apps for which one might not want to distribute the source
for, it would be really nice if one can count on just requiring the
end-user to install a version of Python (but not extra extension modules anymore)
and then distribute the app as a bunch of .pyc's packaged in a .zip file and
use the PYTHONPATH method you showed. Just remember to provide a set of
.pyc's for each version of Python you support.
The issue of hiding source comes up every couple of months. If there is
not a FAQ entry, there are certainly many past threads. Note that the dis
modules makes bytecode pretty easy to read, especially with some
experience.
Terry J. Reedy
> > In general all Java bytecode is expected (but not required) to share the
main system JVM.
Afaik, each app will start its own copy of a JVM.
I am not talking about runtime processes, I am talking about installed
files. The whole point of Java is that you only need one JVM installed
and all apps can use it. By that I mean one physical copy on the
disk. Now some apps find that the JVM isn't as version compatible
as they expect and do install a private copy. But my statement
above still remains true.
Like I mentioned earlier, I am aware of 'freezing' but having to bundle
an interpreter with every app is exactly what I would rather avoid...
Why? Using Windows as an example, the Python interpretter DLL is
974Kb. The JRE VM is about 1.1MB. Both Python and Java need a number
of other DLLs, but I believe the Python list is significantly smaller
(a lot more base functionality is built into that Python DLL than is
in the JRE DLL).
What would be great is to not have to choose between a Windows,
Mac or Linux or XXX download (like in a proper Java app).
Why? That then means the user can't use their normal system method
for adding and removing programs and instead have to do things
manually by copying JAR files and potentially other extensions
around.
And here is an example of what really happens in the Java world.
One of the most popular projects written in Java on SourceForge
is Azureus:
http://sf.net/project/showfiles.php?...ckage_id=88270
Note platform specific installers. You can also do things manually.
Click on the platform specific Howtos on http://azureus.sf.net/
It does also let you copy the jar file manually, but that then means
you have to manually deal with renames, uninstalls, backups, integrity
checking etc, not to mention having to make your own shortcuts.
But you're not saying that .pyc files contain profiling information,
right? Because that is what it's sounding like...
No. I am saying they *could* contain it. They are an implementation
detail of one version of one implementation of a conforming Python
interpretter. It is free to put *anything* it wants in there including
profiling information and your favourite colour, or just random numbers.
Yes, this is the main difference between Java and Python. Python
prefers the source to be present, Java doesn't care.
Java is a compiled language, just like C/C++ etc. Python is an interpretted
language without a compilation step. Some Python implementations
do not require the source to present as they can use the results of
their own earlier tokenization and parsing of Python source. But that
is implementation specific and not *specified* or *required* anywhere. With the addition of zip imports in Python 2.3,
Correct me if I'm wrong, but I thought you just said earlier that
zip imports predated Java by several years and now you're saying
they've just been added to Python 2.3?
No, I said that PYTHONPATH in Python predated CLASSPATH in Java by
several years. What they can point to has changed in both implementations
over time.
I would recommend you start this thread again, but actually state what
your goals and constraints are, rather than trying to retrofit/force something
(CPython bytecode) as a solution to whatever the unstated problem is.
For the *vast* majority of cases, end users couldn't care less what programming
language or environment is used for any application they get. They certainly
don't want to have to go around chasing random environments, interpretters,
extensions etc, and manually deal with extra components, upgrades, uninstalls
etc. They just want to go ahead and use the normal tools for their OS and
be as productive as quickly as possible with the application.
The installer generators solve this problem really well. In many cases
they even offer some obfuscation of code if used with particular interpretters.
They are also fairly compact.
For example my project includes the code, many images, help files (two diffent
formats), some auxiliary binaries, all required libraries and the Python interpretter
etc in 5MB. Azureus ships as a 3.2MB installer, but that excludes the JRE
which is a seperate 15MB download.
Note that the full Python environment (which is way closer to the JDK and even
includes an IDE, debugger etc) is a 9.3MB download.
In the Java world, there are two issues that result in a certain way of thinking.
One is that you know your code is supposed to run on any JVM. The second
is that the JVM is a large download. Consequently this leads to the mindset
of trying to use one installed JVM for all apps, and doing some manual
configuration to get things working.
Python isn't Java, and doesn't have this mindset. The Python mindset is
to make things as easy and logical as possible, especially without users
(or developers!) having to be aware of implementation details.
Roger
Thanks for all your replies, folks. Picked up quite a few
things in this thread...
This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Anon |
last post by:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hello all
I am a beginner teaching myself python, and I am enjoying it immensely :)
As a language it is great, I real treat to study, I actually...
|
by: Ognjen Bezanov |
last post by:
Hello All,
I am a third year computer science student and I'm the process of
selection for my final year project.
One option that was thought up was the idea of implement my own version
of...
|
by: Robert Kern |
last post by:
Ognjen Bezanov wrote:
Mostly. It is worth noting, though, that the Python bytecode is at a higher
level than Java bytecode. You can run other languages on top of the JVM using
Java bytecode...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: ryjfgjl |
last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
|
by: ryjfgjl |
last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
| |
