473,288 Members | 1,710 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,288 software developers and data experts.

why python is slower than java?

This may be a dumb thing to ask, but besides the penalty for dynamic
typing, is there any other real reasons that Python is slower than Java?

maurice
Jul 18 '05 #1
114 9637
Maurice LING wrote:
This may be a dumb thing to ask, but besides the penalty for dynamic
typing, is there any other real reasons that Python is slower than Java?

maurice


at work, we use java and python. we have projects using swing and others using wxpython. we have applications that do
intensive io and others that do intensive cpu. we have not found that python is slower than java. in fact, when it
comes to gui's, our swing apps take "forever" to startup and when when garbage collector starts, the whole app just
freezes for about 15 seconds. our wxpython apps, start right up and "feel" faster and snappier. can you show an
example of where python's "slow" speed as compared to java's "fast" speed has negatively impacted your application or
has been noticable in any way? i know this is a trolling question you have posted, but i'm actually very interested
knowing why you have said this.

thanks,

bryan
Jul 18 '05 #2
at work, we use java and python. we have projects using swing and
others using wxpython. we have applications that do intensive io and
others that do intensive cpu. we have not found that python is slower
than java. in fact, when it comes to gui's, our swing apps take
"forever" to startup and when when garbage collector starts, the whole
app just freezes for about 15 seconds. our wxpython apps, start right
up and "feel" faster and snappier. can you show an example of where
python's "slow" speed as compared to java's "fast" speed has negatively
impacted your application or has been noticable in any way? i know this
is a trolling question you have posted, but i'm actually very interested
knowing why you have said this.

thanks,

bryan


Thanks, lets just say that I have no interest in trolling.

1st of all, I thought it is somehow common knowledge that python is
slower than java in many cases. Though I may be wrong... When I do a
Google search, this came up...

http://twistedmatrix.com/users/glyph...n-vs-java.html

although http://page.mi.fu-berlin.de/~prechel.../jccpprtTR.pdf
showsjust the opposite.

What I need to work on now is something that requires speed (and dealing
with files), without user's intervention. So the part about users' delay
time is not in the equation. My choices boils down to Python or Java.

Cheers
maurice
Jul 18 '05 #3
Maurice LING wrote:
at work, we use java and python. we have projects using swing and
others using wxpython. we have applications that do intensive io and
others that do intensive cpu. we have not found that python is slower
than java. in fact, when it comes to gui's, our swing apps take
"forever" to startup and when when garbage collector starts, the whole
app just freezes for about 15 seconds. our wxpython apps, start right
up and "feel" faster and snappier. can you show an example of where
python's "slow" speed as compared to java's "fast" speed has
negatively impacted your application or has been noticable in any
way? i know this is a trolling question you have posted, but i'm
actually very interested knowing why you have said this.

thanks,

bryan

Thanks, lets just say that I have no interest in trolling.

1st of all, I thought it is somehow common knowledge that python is
slower than java in many cases. Though I may be wrong... When I do a
Google search, this came up...

http://twistedmatrix.com/users/glyph...n-vs-java.html

although http://page.mi.fu-berlin.de/~prechel.../jccpprtTR.pdf
showsjust the opposite.

What I need to work on now is something that requires speed (and dealing
with files), without user's intervention. So the part about users' delay
time is not in the equation. My choices boils down to Python or Java.

Cheers
maurice


but you aren't saying exactly _what_ requires speed and exactly what your requirements for the project is. you are being
way too general here for anyone to really help you. as for working with files, i much rather do that in python than
java. i may be wrong, but i thought java's file IO was pretty much a thin wrapper over c. so i don't think you will
have any speed problems with python's file IO compared to java's. also, just for fun, write the following fully
working python program in java:

import time
t = time.time()
s = open('in.txt').read()
open('out.txt', 'w').write(s)
print time.time() - t

(by the way...i was able to write the above lines of python code without referring to a manual :)

if it's not too much trouble, could you please post your code here along with the time results for the above code and
your java code? you can test a small file and a large one.

thanks,

bryan
Jul 18 '05 #4
On Fri, 05 Nov 2004 07:11:33 +0000, Maurice LING wrote:
What I need to work on now is something that requires speed (and dealing
with files), without user's intervention. So the part about users' delay
time is not in the equation. My choices boils down to Python or Java.


Speed in what area? Dealing with files, is it a heavily I/O-based program?
If so, choice of programming language will very probably hardly impact
performance (well, depending of course on exactly you want to do, there
may of course be differences...).
Or do you want to crunch numbers? Python has great extensions for that,
written in highly optimized C. Come to think of it, Python has great
built-in datatypes and extensions for all manner of things, written in
highly optimized C. So even if your application wants to do some heavy
processing of <whatever> that doesn't depend on external factors like I/O
speed, the network or user interaction, Python will generally be fast
enough.
And as for the statically typed nature of Java providing speed benefits,
take a look at Psyco, which often (but not always) helps enormously.

So everything boils down to two points: what exactly you want to do - if
you can tell us that, someone better qualified and less biased than
myself might actually be able to offer you concrete advice on what to
choose - and as always: premature optimization is the root of all evil :-)

--
Christopher

Jul 18 '05 #5
On Fri, 05 Nov 2004 05:58:09 +0000, Maurice LING wrote:
This may be a dumb thing to ask, but besides the penalty for dynamic
typing, is there any other real reasons that Python is slower than Java?

maurice


Hi Maurice,
the issue is compiling and bytecode. Python and Java compile source to
their own bytecodes. Bytecode is a binary that is portable between many
Operating Systems. Some time in 1999 (I think) IBM developed a jit or
Just In Time compiler for Java (Apple invented the JIT concept earlier to
make 68000 code work on their new line of PowerPC Macs).

The JIT compiler compiled java to both bytecode and native OS opcode. This
offered increase speed on the second + runs. The binary was suppose to be
compiled into a block containing bytecode and opcode. If the java runtime
noted that the opcode segment was 'native', it was suppose to run it, over
the bytecode. This gives the 'faster' perception.

Of course some company, afraid their developers might build there GUI apps
with java, over their native mode (thus making their application work
anywhere Java worked) went ahead and developed a version of the java
compiler that stripped out the bytecode. This made the binary
smaller, but locked it to one OS/arch. You can guess about which
corporation that was. Lawsuits followed. The company in question lost and
decided to drop Java for their own version of Java, JavaLite, sometimes
called 'see' 'sharp'.

Now as to speed... If you want speed, develop in Python or Perl to
get the form of your program. Development time is faster, even if the
execution is slow. Both provide OOP bindings. Once the program is up and
running, you can use profiling to determine which parts of your
interpreted code is slowest and most often used. These you rewrite as
C/C++ code and call them from within python.

In this way version 1.0 is all python. v1.1 has 20% C++; v1.2 is 34% C++
.... until v2.0 is all C++. Each upgrades is faster, but does not introduce
any new functionality and the risks that new functions imply.

If you were faithful to this concept, version 3 requires you to use the
oldest python branch who's functions will not be modified, to start
development. Could be v1.1 or 1.9.

C/C++ is for speed, not development.

Dan Atterton
at******@yahoo.com

Jul 18 '05 #6
Maurice LING <ma*********@acm.org> writes:
This may be a dumb thing to ask, but besides the penalty for dynamic
typing, is there any other real reasons that Python is slower than
Java?


Do you have any data to support this ?
On Windows XP, Windows NT and Linux (Gentoo ), I have consistently
found python apps using wxpython to be far faster to load
and to be far more responsive than Java apps.
Jul 18 '05 #7


http://dada.perl.it/shootout/craps.html

On the CPU score, Java scores 28.4 while Python is within cooee at 27.
(bigger is better )

On the updated version at http://shootout.alioth.debian.org/craps.php,
Python beats at least two java implementations.
Jul 18 '05 #8
EP
John Doe wrote (posthumously?):
In this way version 1.0 is all python. v1.1 has 20% C++; v1.2 is 34%
C++
... until v2.0 is all C++. Each upgrades is faster, but does not
introduce
any new functionality and the risks that new functions imply.

If you were faithful to this concept, version 3 requires you to use the
oldest python branch who's functions will not be modified, to start
development. Could be v1.1 or 1.9.

C/C++ is for speed, not development.

Am I the only one to think this works, but makes no sense?

If one is to migrate their Python code to C++, one ends up with C++ code, which is great unless there is that (not so very rare) event where one wantsto add new features, address new requirements, etc., etc. Applications need to evolve. If the finished code is going to be C++, one might just as well program in C++ from the start instead of aportioning mindshare between the two languages. Mindshare devoted to thinking in a second language might instead be devoted to thinking about the application.

Python seems to exemplify living code and since good applications have a life of their own, shouldn't the code live as well?

Dipping into C++ for program hotspots seems like good pramaticism, but at the expense of the simplicity and clarity that is otherwise Pythonic.

Fast Python is an answer, a patchwork of programming languages is a work around.
Eric

I won't say that Python needs to be faster, but imagine Python running as fast as C++ without any special tricks and you can envision the opportunities lost due to Python lacking blazing speed.

Jul 18 '05 #9
Maurice LING wrote:
This may be a dumb thing to ask, but besides the penalty for dynamic
typing, is there any other real reasons that Python is slower than Java?


Python is generally late-bound, which means much more happens at runtime
compared to Java. For instance, consider accessing an attribute,
"self.x". In Java you know what the class is shaped like, exactly.
Usually this turns into some bytecode like "access the variable of self,
at offset 3". I don't know any of the details, but that's the general gist.

Similarly, there's many other optimizations where you can know at
compile time exactly what functions are called, how variables are
stored, etc. Using that information, you can do optimizations like
inline code. In the same places in Python, you are typically doing one
(or several!) hashtable lookups based on the variable or attribute name.

But even in Java there are limits. Because of interfaces and subclasses
you can know the shape of self/this, but you can't know the exact shape
of the objects around you (C++ is more aggressive in this respect, and
often can determine the exact shape; but in the process it's dangerous
to put together different pieces of compiled code when they don't know
about each other, which is why binary interfaces there are fragile).
Anyway, Java does a lot of the same stuff as Python when the exact type
is determined at runtime (which is frequently). This is where JIT comes
in, doing the optimizations at runtime; it is still limited, as it
cannot guarantee the type of the objects in the system, but must check
them each time before using the optimized path. And, actually, Python
can do the same sort of things with psyco. It's still harder in Python,
and the end result not as effective, but it's one among many tools.

If Java wasn't doing any optimizations, I don't think it would be
significantly faster than Python.

Also note that Python assimilates external (C, Fortan, etc) libraries
much better than Java seems to. In an entire system, Python can easily
be faster because Java includes many slow libraries (slow compared to
equivalent libraries available in Python). E.g., Swing is much slower
than wxPython.

Anyway, that's my take. I'm no authority, as I've never seriously used
Java, and haven't done any tests, nor spent anytime looking at the
bytecodes, etc.

--
Ian Bicking / ia**@colorstudy.com / http://blog.ianbicking.org
Jul 18 '05 #10
Maurice LING <ma*********@acm.org> wrote in message news:<41********@news.unimelb.edu.au>...
This may be a dumb thing to ask, but besides the penalty for dynamic
typing, is there any other real reasons that Python is slower than Java?


Personally, in practice I haven't found Python to be slower for real
apps, although it's sometimes an order of magnitude slower in
microbenchmarks. That's probably because a lot of things that could
be slow (e.g. numerical manipulation) are backed by C modules behind
the scenes in Python; it's also very easy to refactor Python code,
which sometimes leads to massive algorithmic improvements.

I will say that I still have a tendency to write some projects in C
when they seem to warrant it (e.g. the mathematics and video portion
of my realtime music visualization system--the UI and database work is
in Python).

(I also found Java's I/O on our platform to be dog-slow back in the
JDK 1.3 days, but I've not used it for anything significant since
then.)
Jul 18 '05 #11

"Maurice LING" <ma*********@acm.org> wrote in message
news:41********@news.unimelb.edu.au...
This may be a dumb thing to ask, but besides the penalty for dynamic
typing, is there any other real reasons that Python is slower than Java?


To the extent that this is true, development money, though it is sometimes
amazing what unpaid or semi-paid volunteers have done with Python.

Terry J. Reedy
\

Jul 18 '05 #12
Maurice LING <ma*********@acm.org> wrote in message news:<41********@news.unimelb.edu.au>...
1st of all, I thought it is somehow common knowledge that python is
slower than java in many cases. Though I may be wrong... When I do a
Google search, this came up...

http://twistedmatrix.com/users/glyph...n-vs-java.html
Apart from the execution speed comparison, did you note that the
author took 4 x the time to write the Java, even though he had much
more Java experience than Python? Did you note that he severely bagged
Java reliability?

In any case, this article is comparing Python 1.5.2 and Java JDK 1.1
-- what does that tell you? If you need a clue, look at the date at
the bottom of the article; looks like 4.5 years old to me. Do you
think that either or both languages might have changed a little since
then?

Or is this a joke/troll?? I can't believe the "unimelb.edu.au"; surely
a forgery.

although http://page.mi.fu-berlin.de/~prechel.../jccpprtTR.pdf
showsjust the opposite.

What I need to work on now is something that requires speed (and dealing
with files), without user's intervention. So the part about users' delay
time is not in the equation. My choices boils down to Python or Java.

Cheers
maurice

Jul 18 '05 #13
John Machin wrote:
Or is this a joke/troll?? I can't believe the "unimelb.edu.au"; surely
a forgery.


Is it just me, or is the climate in c.l.py getting less friendly to
newbies? In any case,
http://www.zoology.unimelb.edu.au/staff/nicholas.htm mentions Maurice
Ling as an honor student.

--
Hans Nowak
http://zephyrfalcon.org/

Jul 18 '05 #14
Yes, wxPython is typically quicker than swing. It's not fair to use
this in a generic Python-vs-Java speed shootout, though, since
wxPython is just a wrapper around wxWidgets' C++ code; you're really
comparing C++ vs. Java here. It does highlight the fact that it's
easy to write C extensions for Python, though.

AFAIK the major two reasons for Python's relative slowness are that
it's interpreted and dynamically typed.

There's almost always a convenience-for-speed trade-off, and Python is
certainly more convenient than Java :P
Israel Raj T <ra****@bigpond.net.au> wrote in message news:<87************@pop-server.bigpond.net.au>...
Maurice LING <ma*********@acm.org> writes:
This may be a dumb thing to ask, but besides the penalty for dynamic
typing, is there any other real reasons that Python is slower than
Java?


Do you have any data to support this ?
On Windows XP, Windows NT and Linux (Gentoo ), I have consistently
found python apps using wxpython to be far faster to load
and to be far more responsive than Java apps.

Jul 18 '05 #15
In article <pa****************************@yahoo.com>,
John Doe <at******@yahoo.com> wrote:
Jul 18 '05 #16
fi**************@gmail.com (Lonnie Princehouse) writes:
Yes, wxPython is typically quicker than swing. It's not fair to use
this in a generic Python-vs-Java speed shootout, though, since
wxPython is just a wrapper around wxWidgets' C++ code; you're really
comparing C++ vs. Java here.
Is it unfair to compare Python dictionaries to Java HashTables because
Python is using hand-tuned C where Java is using Java? I'd say that's
a perfectly fair comparison. If that's fair, what's unfair about
comparing graphics toolkits that happen to be written in C rather than
Python (is there a graphics toolkit written in Python?).

That's *how* interpreted languages manage to compete with compiled
languages - they perform the common operations in hand-tuned compiled
code. That people tend to write Java libraries in Java rather than C
just means they're going to get slower libraries.
AFAIK the major two reasons for Python's relative slowness are that
it's interpreted and dynamically typed.


Java is interpreted just like Python is. The only difference is that
Java splits the compiler and VM implementation into two different
binaries.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
Jul 18 '05 #17
Mike Meyer wrote:
fi**************@gmail.com (Lonnie Princehouse) writes:

Yes, wxPython is typically quicker than swing. It's not fair to use
this in a generic Python-vs-Java speed shootout, though, since
wxPython is just a wrapper around wxWidgets' C++ code; you're really
comparing C++ vs. Java here.

Is it unfair to compare Python dictionaries to Java HashTables because
Python is using hand-tuned C where Java is using Java? I'd say that's
a perfectly fair comparison. If that's fair, what's unfair about
comparing graphics toolkits that happen to be written in C rather than
Python (is there a graphics toolkit written in Python?).

That's *how* interpreted languages manage to compete with compiled
languages - they perform the common operations in hand-tuned compiled
code. That people tend to write Java libraries in Java rather than C
just means they're going to get slower libraries.

AFAIK the major two reasons for Python's relative slowness are that
it's interpreted and dynamically typed.

Java is interpreted just like Python is. The only difference is that
Java splits the compiler and VM implementation into two different
binaries.

<mike


Actually java has not been interpreted for quite a few years. Java today
is compiled to native code at runtime (JIT), exactly like what Psyco
tries to do for Python. This means the only things that (today) makes a
java program slower than a C program are Java's added safety features (
Bounds-checking on arrays and garbage collection to name but two).

Java's speed and Python's speed are not absolute values. Java IS faster
than python when it comes to CPU-intensive operations. IF you given
problem is heavily IO-bound, then python will be just as fast as Java.
Also, if you are writing a GUI application, your program will spend most
of its time waiting for user-input and not much difference will be
noticed. Python may even win out in that case since it's GUI are
typically platform-tuned C/C++ libraries wrapped in Python.
Steve
Jul 18 '05 #18
Hans Nowak <ha**@zephyrfalcon.org> wrote:
John Machin wrote:
Or is this a joke/troll?? I can't believe the "unimelb.edu.au"; surely
a forgery.


Is it just me, or is the climate in c.l.py getting less friendly to
newbies? In any case,


Interestingly enough, I sort of share your perception -- and I have
noticed the same thing, and seen it remarked upon by others, in other,
completely unrelated newsgroups as well, such as it.comp.macintosh.

It's not, I think, about newbies in general: people who come and post
help requests, without giving the information that's quite obviously
indispensabile to let us help them, keep getting treated with
unreasonable amounts of friendliness and courtesy even after many
requests for more info go unheeded, for example.

However, newbies who are clueless enough to come blasting in with the
usual whines we've heard a zillion times -- Macs cost too much, Python
is too slow, there's no apps for Macs, Python must absolutely add
feature X or it will die, Apple's gonna go broke tomorrow, etc, etc --
do appear to get on our collective nerves worse than their essentially
indistinguishable precursors did last year, two years ago, &c.

The accusation of being a troll often follows, breaking the old advice
to "Never attribute to malice what can be adequately explained by
incompetence". Maybe it's a case of the corollary to that old advice,
"Sufficiently advanced cluelessness is indistinguishable from malice".
But why that should be so this year more than last year, say, is not
clear to me...
Alex
Jul 18 '05 #19
Interestingly enough, I sort of share your perception -- and I have
noticed the same thing, and seen it remarked upon by others, in other,
completely unrelated newsgroups as well, such as it.comp.macintosh.

It's not, I think, about newbies in general: people who come and post
help requests, without giving the information that's quite obviously
indispensabile to let us help them, keep getting treated with
unreasonable amounts of friendliness and courtesy even after many
requests for more info go unheeded, for example.

However, newbies who are clueless enough to come blasting in with the
usual whines we've heard a zillion times -- Macs cost too much, Python
is too slow, there's no apps for Macs, Python must absolutely add
feature X or it will die, Apple's gonna go broke tomorrow, etc, etc --
do appear to get on our collective nerves worse than their essentially
indistinguishable precursors did last year, two years ago, &c.

The accusation of being a troll often follows, breaking the old advice
to "Never attribute to malice what can be adequately explained by
incompetence". Maybe it's a case of the corollary to that old advice,
"Sufficiently advanced cluelessness is indistinguishable from malice".
But why that should be so this year more than last year, say, is not
clear to me...
Alex


I've already said the following and was not noticed:

1. it is a disk intensive I/O operation.
2. users delay is not in the equation (there is no user input)
3. I am not interested in the amount of time needed to develop it. But
only interested execution speed.

Thanks maurice
Jul 18 '05 #20
In this way version 1.0 is all python. v1.1 has 20% C++; v1.2 is 34% C++
... until v2.0 is all C++. Each upgrades is faster, but does not introduce
any new functionality and the risks that new functions imply.

If you were faithful to this concept, version 3 requires you to use the
oldest python branch who's functions will not be modified, to start
development. Could be v1.1 or 1.9.

I don't quite get the last part right, the version 3 part. I got lost in
it....

thanks maurice
Jul 18 '05 #21

Do you have any data to support this ?
On Windows XP, Windows NT and Linux (Gentoo ), I have consistently
found python apps using wxpython to be far faster to load
and to be far more responsive than Java apps.


well, I do not have data to support this. Just based on an impression,
perhaps seriously flawed impression. maurice
Jul 18 '05 #22
Israel Raj T <ra****@bigpond.net.au> wrote in news:878y9gbkh7.fsf@pop-
server.bigpond.net.au:
On the updated version at http://shootout.alioth.debian.org/craps.php,
Python beats at least two java implementations.


Very interesting link, great ressource!!!

What really is disturbing: within the test "hash access" perl2 comes out
first compared with java.

A know that Python dictionaries are the most heavily used datastructure
(because it is allways used within Python itself), and from looking at the
sourcecode there is the line "with help from Raymund Hettinger"; so I am
very sure it is probably THE optimal coding.

So... string processing, regular expression, blablabla no problem; but why
second to perl2 in Hash-access ???

Harald
Jul 18 '05 #23
Harald Massa ha scritto:
Israel Raj T <ra****@bigpond.net.au> wrote in news:878y9gbkh7.fsf@pop-
server.bigpond.net.au:

On the updated version at http://shootout.alioth.debian.org/craps.php,
Python beats at least two java implementations.

Very interesting link, great ressource!!!

What really is disturbing: within the test "hash access" perl2 comes out
first compared with java.

A know that Python dictionaries are the most heavily used datastructure
(because it is allways used within Python itself), and from looking at the
sourcecode there is the line "with help from Raymund Hettinger"; so I am
very sure it is probably THE optimal coding.

So... string processing, regular expression, blablabla no problem; but why
second to perl2 in Hash-access ???


maybe because perl has been polished for more years :)
Anyway, 3 things worth noting about the shootou

- you won't *ever* write heapsort and things like that in python,
obviously it is slow, and obviously you have'em builtin.
- there are some tricks like using numarray for matrix calculations
- the ranking system is named CRAP(s).

Jul 18 '05 #24

On 2004 Nov 06, at 08:31, Maurice LING wrote:
...
1. it is a disk intensive I/O operation.
2. users delay is not in the equation (there is no user input)
3. I am not interested in the amount of time needed to develop it. But
only interested execution speed.


OK, could you provide a simple toy example that meets these conditions
-- does lot of identical disk-intensive I/O "in batch" -- and the
execution speed measured (and on what platform) for what Python and
Java implementations, please?

For example, taking a trivial Copy.java from somewhere on the net:

import java.io.*;

public class Copy {
public static void main(String[] args) throws IOException {
File inputFile = new File("/usr/share/dict/web2");
File outputFile = new File("/tmp/acopy");

FileReader in = new FileReader(inputFile);
FileWriter out = new FileWriter(outputFile);
int c;

while ((c = in.read()) != -1)
out.write(c);

in.close();
out.close();
}
}

and I observe (on an iBook 800, MacOSX 10.3.5):

kallisti:~ alex$ java -version
java version "1.4.2_05"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_05-141.3)
Java HotSpot(TM) Client VM (build 1.4.2-38, mixed mode)

-r--r--r-- 1 root wheel 2486825 12 Sep 2003 /usr/share/dict/web2

kallisti:~ alex$ time java Copy

real 0m7.058s
user 0m5.820s
sys 0m0.390s

versus:

kallisti:~ alex$ time python2.4 Copy.py

real 0m0.296s
user 0m0.080s
sys 0m0.170s

with Python 2.4 beta 1 for the roughly equivalent:

inputFile = file("/usr/share/dict/web2", 'r')
outputFile = file("/tmp/acopy", 'w')

outputFile.write(inputFile.read())

inputFile.close()
outputFile.close()

which isn't all that far from highly optimized system commands:

kallisti:~ alex$ time cp /usr/share/dict/web2 /tmp/acopy

real 0m0.167s
user 0m0.000s
sys 0m0.040s

kallisti:~ alex$ time cat /usr/share/dict/web2 >/tmp/acopy

real 0m0.149s
user 0m0.000s
sys 0m0.090s
I'm sure the Java version can be optimized easily, too -- I just
grabbed the first thing I saw off the net. But surely this example
doesn't point to any big performance issue with Python disk I/O wrt
Java. So, unless you post concrete examples yourself, the smallest the
better, it's going to be pretty difficult to understand where your
doubts are coming from!
Alex

Jul 18 '05 #25
On Fri, 05 Nov 2004 18:33:10 -0500, Hans Nowak <ha**@zephyrfalcon.org> wrote:
John Machin wrote:
Or is this a joke/troll?? I can't believe the "unimelb.edu.au"; surely
a forgery.


Is it just me, or is the climate in c.l.py getting less friendly to
newbies? In any case,
http://www.zoology.unimelb.edu.au/staff/nicholas.htm mentions Maurice
Ling as an honor student.


Not only that, but Maurice Ling has started a long thread a few weeks
looking for a good research topic for his thesis (involving Java &
Python, btw). At some points he was bashed, but went ahead with the
discussion, that ended up touching in several interesting topics.

As for the climate in c.l.py, it's just interesting to note that the
climate *is* getting less friendly, and that it does coincide with
several old-timers moving away from the list. A few years ago I
remember that the likes of Tim Peters, effbot, Skip Montanaro, and
several others (sorry, I would really like to remember more names from
the top of my head) were frequent posters here. They've now moved to
other interests, or are focusing their efforts on the python-dev list.
It must not be a coincidence.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com
Jul 18 '05 #26
On Sat, 6 Nov 2004 08:12:10 +0100, Alex Martelli <al*****@yahoo.com> wrote:
"Sufficiently advanced cluelessness is indistinguishable from malice".
Although not originally yours, anyway: +100 QOTW!
But why that should be so this year more than last year, say, is not
clear to me...


I think that (at least for c.l.py) it has something to do with the
fact that several old-timers are moving away from the list. I don't
know if this is a cause or a consequence. But it's not a coincidence,
IMHO.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com
Jul 18 '05 #27
Alex Martelli wrote:
Hans Nowak <ha**@zephyrfalcon.org> wrote:

John Machin wrote:

Or is this a joke/troll?? I can't believe the "unimelb.edu.au"; surely
a forgery.


Is it just me, or is the climate in c.l.py getting less friendly to
newbies? In any case,

Interestingly enough, I sort of share your perception -- and I have
noticed the same thing, and seen it remarked upon by others, in other,
completely unrelated newsgroups as well, such as it.comp.macintosh.


I think it can be explained by donor fatigue.

I've experienced this on a couple of projects where the initial
developers/intellectual foundation are very helpful and a lot of work
and they get burned out. The Next Generation comes along and doesn't
quite know enough to be able to handout ready answers but tries real
hard and get frustrated. Users get frustrated and then as you point
out, the clueless twits start dominating the conversation. Sort of an
intellectual approximation of Gresham's law.

I know from my perspective, I have been reluctant to really talk much
about what I've done with Python in developing the camram anti-spam
system because of fatigue of a different type. It is a radically
different system and the very odd thing is that ordinary users get it in
a heartbeat and many technologists fight it with every fiber of their being.

so as a result, I ask questions trying to describe the abstract case of
what I'm trying to do so that I don't get distracted down a rathole of
defending the application. Needless to say, I suspect others are doing
the same thing and it's really really hard to describe the abstract case
in a way that's clear.

now I'm going to throw out another radical idea for dealing with donor
fatigue: I will argue that we might want to start including in the
headers of mailing list/newsgroups a dana reference. If you are
familiar with Buddhist culture, dana is an expression of generosity. A
gift given in response to a priceless lesson. There is no expectation
giving and there is no expectation of receiving.
http://www.accesstoinsight.org/ptf/dana.html therefore, with a dana
header, someone who is happy with the answer to a question could gift
some money to the recipient.

I'll argue that some form of compensation system is necessary to help
combat donor fatigue. We all live for reward stimuli be it money, sex,
chocolate. Compulsory "pay for the answer" systems don't work well
because you have no assurance that you'll get an answer you can use.
The flip side "pay for the cheapest answer that meets your needs" does
not encourage participation because it's usually a financial race to the
bottom. I think the middle path of Dana is the most likely to succeed
although it won't make anybody rich. You contribute and maybe somebody
will thank you for it. You'll know that it was given in the spirit of
generosity and kindness, not because of a contractual requirement.

another advantage of this idea is that it builds on the existing
infrastructure of mailing lists and newsgroups not a walled garden of a
web site somewhere.

There are some challenges connecting the dana id to financial systems
and e-mail/news clients. But these are known problems with solutions
and with tools like Thunderbird, we can easily make modifications and
make them available.

---eric

Jul 18 '05 #28
On Fri, Nov 05, 2004 at 05:58:09AM +0000, Maurice LING wrote:
This may be a dumb thing to ask, but besides the penalty for dynamic
typing, is there any other real reasons that Python is slower than Java?


I favor a different kind of explanation than the ones usually offered
here. While the degree to which Python is "slower than Java" will
depend in large part on what you're trying to do with both of them,
it's also the case that Java leads Python by probably $1 billion
dollars, if not more, in JIT compiler R&D.

The main reason Java is "faster" is because it's been backed by Sun
and IBM, for use in enterprise applications, on some of the biggest
hardware available, and they've put massive R&D effort (i.e., money)
into making it faster.

I suspect if MS had chosen Python instead of C#, and then spent a few
billion and 5 to 10 years of R&D effort, people would be asking why
Perl is so much slower than Python.

Common Lisp is fast because it's had similar R&D effort, spread out
over 40 years or so, going back to the first Lisps. Same goes for
Smalltalk. Java is fast because Sun & IBM and others decided it had to
be. Python is slower because the Python community is incomparably
smaller, less well-heeled, and mostly doesn't care too much about
performance.

There are *technical* reasons for all of this, but I think those
reasons exist largely because of non-technical, social reasons.

My two cents. :>

Best,
Kendall Clark
--
Sometimes it's appropriate, even patriotic, to be ashamed
of your country. -- James Howard Kunstler
Jul 18 '05 #29

"Alex Martelli" <al*****@yahoo.com> wrote in message
news:1gmtkpc.1h26kkr1rd529sN%al*****@yahoo.com...
Hans Nowak <ha**@zephyrfalcon.org> wrote:
John Machin wrote:
> Or is this a joke/troll?? I can't believe the "unimelb.edu.au"; surely
> a forgery.
Is it just me, or is the climate in c.l.py getting less friendly to
newbies? In any case,


Interestingly enough, I sort of share your perception -- and I have
noticed the same thing, and seen it remarked upon by others, in other,
completely unrelated newsgroups as well, such as it.comp.macintosh.


I'm still one of the newbies and I was not reading this newsgroup a year
ago, so I cannot draw any comparisons, but I wonder, has this group also
grown much during this time? Could this be a cultural thing like the
difference between a small town culture and a big city culture?
It's not, I think, about newbies in general: people who come and post
help requests, without giving the information that's quite obviously
indispensabile to let us help them, keep getting treated with
unreasonable amounts of friendliness and courtesy even after many
requests for more info go unheeded, for example.

However, newbies who are clueless enough to come blasting in with the
usual whines we've heard a zillion times -- Macs cost too much, Python
is too slow, there's no apps for Macs, Python must absolutely add
feature X or it will die, Apple's gonna go broke tomorrow, etc, etc --
do appear to get on our collective nerves worse than their essentially
indistinguishable precursors did last year, two years ago, &c.


I have complained myself about several aspects of Python that I was
discovering as I was learning it (although I never said Python will die).
But I have been programming for enough years to believe that I am entitled
to have an opinion even in something that I am new at and that I am entitled
to express that opinion.

On the other hand, as I become more experienced in Python and I have
opportunities to help other people who are even newer at this than I am, I
also feel some frustration when I do not get enough help from them to help
them. But I probably can still relate better to them because, like them, I
still need help in so many areas.

IMO, we all must show more tolerance. I believe though that newbies are
naturally more tolerant of experts because newbies need the experts. So I
am making a plea particularly to experts to show more tolerance to newbies.
The newbies' complaints come out of frustration, frustration just like
yours, which shows in terms like "clueless", "blasting in", and "whines
we've heard a zillion times", all in just one sentence (and that is in a
discussion on friendliness towards newbies).

Are you learning any new subject now and are you involved in a newsgroup as
a newbie? If you're not doing that already, python experts, please try it
and you'll probably see what it's like. No better way to see the other
side's point of view than being on the other side.

Dan
Jul 18 '05 #30
Dan Perl <da*****@rogers.com> wrote:
Are you learning any new subject now and are you involved in a newsgroup as
a newbie? If you're not doing that already, python experts, please try it
and you'll probably see what it's like. No better way to see the other
side's point of view than being on the other side.


What if we do, _AND_ carefully follow Eric Raymond's excellent
recommendations each and every time we ask for help? Are then we
allowed to loathe and despise the mass of clueless dweebs?-)

I can be a newbie at a bazillion subjects, easily -- but I cannot truly
be a newbie at such tasks as human interaction, social dynamics, general
information retrieval and processing. I can easily guess what will
happen if I enter any mailing list or newsgroup with both guns blazing
out of frustration, for example, and therefore I cannot easily
sympathize with anybody who _does_ behave so foolishly. It's not a
matter of expertise about any specific subject, not even exactly one of
skills, but rather one of personal maturity and character.

I know how to search mailing list archives, or google groups ones, and
considerate enough to use this easily acquired and very useful knowledge
to try and avoid wasting other people's time and energy, for example, by
airing some complaint that's been made a thousand times and answered
very comprehensively. When I can't find an answer that way, I ask with
courtesy and consideration and appreciation for the time of the people
who, I hope, will be answering my questions. Etc, etc -- reread Eric's
essay on how to ask for help, it's a great piece of work.

That doesn't mean a newbie isn't always welcome, _if_ they show any sign
whatever of being worth it. But asking for tolerance and patience
against _rude_ newbies which barge in with shrill, mostly unjustified,
repetitious complaints, is, I think, a rather far-fetched request.
Alex
Jul 18 '05 #31
Alex Martelli said unto the world upon 2004-11-06 02:12:
Hans Nowak <ha**@zephyrfalcon.org> wrote:


<SNIP>

Is it just me, or is the climate in c.l.py getting less friendly to
newbies? In any case,

Interestingly enough, I sort of share your perception -- and I have
noticed the same thing, and seen it remarked upon by others, in other,
completely unrelated newsgroups as well, such as it.comp.macintosh.

It's not, I think, about newbies in general: people who come and post
help requests, without giving the information that's quite obviously
indispensabile to let us help them, keep getting treated with
unreasonable amounts of friendliness and courtesy even after many
requests for more info go unheeded, for example.

However, newbies who are clueless enough to come blasting in with the
usual whines we've heard a zillion times -- Macs cost too much, Python
is too slow, there's no apps for Macs, Python must absolutely add
feature X or it will die, Apple's gonna go broke tomorrow, etc, etc --
do appear to get on our collective nerves worse than their essentially
indistinguishable precursors did last year, two years ago, &c.

The accusation of being a troll often follows, breaking the old advice
to "Never attribute to malice what can be adequately explained by
incompetence". Maybe it's a case of the corollary to that old advice,
"Sufficiently advanced cluelessness is indistinguishable from malice".
But why that should be so this year more than last year, say, is not
clear to me...
Alex


Hi all,

I should say up front that I'm a relative newcomer to technical
newsgroups and mail-lists myself, so these comments aren't backed-up by
years of observation. I'd also like to make clear that I am responding
with general observations to what I took to be general observations in
the quoted msgs above; I don't think any of Alex, Hans, or I intend to
be talking about any one post or poster. Last, having just finished
writing this email, I see it got a bit long. There is a concrete
proposal at the end, so I invite those who feel its a ramble to skip
down 5 or so paragraphs.

The thread has offered donor fatigue and old-hands drifting away as
partial explanations, but I'd say any decline in newbie-friendliness
must be at least partly explained by success. Many in the Python
community seem to share the laudable goal of spreading Python, and not
just to the already technically informed. From Guido's Computing For
Everybody project, to the ever-patient folks on the tutor list, to any
number of other manifestations one could point to, the Python community
seems to have a "Welcome. You don't have to know what a dip-switch is to
enter here" sign up. That's a good thing. But, if it works (and it
does--witness this post ;-), it brings in people who don't know the
social norms.

That community attitude doesn't necessarily explain the Mac list Alex
cites. (I've not read the list, so cannot say.) But there too, there
must be an upsurge of new folk who finally got tired of doing what Bill
told them to do. (Among my circle, about 1 frustrated Windows user
every month or two switches to Macs.) So, it both cases, it is likely to
be driven by continued expansion of the membership outside the bounds of
those who know what elm and pine are.

Python might also be vulnerable to a strange, paradoxical phenomena: it
a way, it seems the more people get for less ($ or effort) the more they
expect. I'm on a list for 2 shareware products, a few freeware (beer
sense) ones, and a few open-source. People do seem to demand more of
list membership and project owners the more free the project is. That
one puzzles me, but I guess helps explain why the non-free,
non-responsive Redmond machine still is widely accepted.

Of all the lists (tech and non) I have read, the Python lists are among
the most tolerant and welcoming. It certainly took me a few tries to
get reasonably close to appropriate form. But, not once was I flamed or
more than gently nudged in the right direction. This friendliness is
important, to be valued, and I think protected. (I recall a post I made
to a LUG asking for help with an aborted attempt to get Linux and my
laptop's Winmodem to co-operate. I was flamed because my mail headers
indicated that I was using Thunderbird on *Windows* -- the horror! --
Not the way to welcome people to the world of Linux.)

But, if Python's increase in user-base continues, on these issues onward
and downward seems all too possible.

So, what to do? Well, I've just subscribed to the list on another
address to check, and there is nothing in the welcome message giving
people new to newsgroups and mail-lists a "Newbie's Introduction to the
Medium". Nor is there a recap posted in the monthly subscription reminder.

I'm not a fan of 'Official List Rules' full of "Thou shalt"s and that's
not what I am suggesting. I am also aware that many of the people whose
post exasperate are also the people least likely to read or attend to
such introductions. But, still, something along the lines might help.

What I have in mind is a wiki-page or three with links prominently
located in the welcome message and the reminder message. I could see at
least three distinct pages being of use. In rapid-sketch mode:

1) Welcome to the list, here are our norms
At minimum, a short 1-2 paragraph distillation of "Smart Questions", a
link to it, a reminder of the existence of the tutor list, and links to
the other two here mentioned.

2) How to find information about Python
Links to one or more of the comp.lang.python archives with perhaps some
advice for optimal searching. This could include a link to a quick "How
to google".
Perhaps a link to Mark Hammond's wicked-cool Python Sidebar for
Firefox/Mozilla <http://starship.python.net/crew/mhammond/mozilla/> and
to any other similar aids.

3) Links to threads that answer common questions
Yes, there is an FAQ and yet people still ask why Python gets division
wrong. But I've often seen people dig up links to threads for questions
perhaps not so common to be written up in an FAQ, but perhaps common
enough to merit linking on a wiki page. I'm not under the illusion that
it would solve the problem on its own, but it might well head some
repeat questions off. For those it doesn't, it would give an easy way to
answer.

Yes, some (or even all) of this info is already on Python.org. But this
way would allow it to be audience-tailored and to be ultra-easy to
direct newbie's toward. And yes, not all newbies read Eric Raymond's
"Smart Questions" essay within the first 5 times they are pointed at it.
Clearly it won't be a silver bullet, but maybe it would help.

As a relative newbie myself, I think I might make a better person to
start off (1) and (2) than some of the old-hands who are further away
from the newbie mindset. And, on the "talk not followed by effort is
chatter" principle, I surely ought volunteer, too ;-) I'm happy to do
so, provided 1) it doesn't strike people as a silly or hopelessly naive
idea, 2) I'm not committed to providing a proof of concept within the
next week :-) , and 3) I can count on other's to do the wiki thing to my
starting effort.

So, if anyone's still reading, thanks. I'm curious to see if the
suggestion seems worthwhile. Best to all,

Brian vdB

Jul 18 '05 #32
Maurice LING <ma*********@acm.org> wrote in message news:<41********@news.unimelb.edu.au>...
at work, we use java and python. we have projects using swing and
others using wxpython. we have applications that do intensive io and
others that do intensive cpu. we have not found that python is slower
than java. in fact, when it comes to gui's, our swing apps take
"forever" to startup and when when garbage collector starts, the whole
app just freezes for about 15 seconds. our wxpython apps, start right
up and "feel" faster and snappier. can you show an example of where
python's "slow" speed as compared to java's "fast" speed has negatively
impacted your application or has been noticable in any way? i know this
is a trolling question you have posted, but i'm actually very interested
knowing why you have said this.

thanks,

bryan


Thanks, lets just say that I have no interest in trolling.

1st of all, I thought it is somehow common knowledge that python is
slower than java in many cases. Though I may be wrong... When I do a
Google search, this came up...

http://twistedmatrix.com/users/glyph...n-vs-java.html

although http://page.mi.fu-berlin.de/~prechel.../jccpprtTR.pdf
showsjust the opposite.

What I need to work on now is something that requires speed (and dealing
with files), without user's intervention. So the part about users' delay
time is not in the equation. My choices boils down to Python or Java.

Cheers
maurice


dude that "comparision" from twistedmatrix you refrence is ANCIENT!!!

it is comparing versions that are YEARS out of date and use!

you are just trolling or your don't know enough to understand the
answer to your question which is way to vague to be answered, as there
is no real correct answer.
Jul 18 '05 #33
In article <ma**************************************@python.o rg>,
Brian van den Broek <bv****@po-box.mcgill.ca> wrote:

<snip>

What I have in mind is a wiki-page or three with links prominently
located in the welcome message and the reminder message. I could see at
least three distinct pages being of use. In rapid-sketch mode:

1) Welcome to the list, here are our norms
At minimum, a short 1-2 paragraph distillation of "Smart Questions", a
link to it, a reminder of the existence of the tutor list, and links to
the other two here mentioned.

2) How to find information about Python
Links to one or more of the comp.lang.python archives with perhaps some
advice for optimal searching. This could include a link to a quick "How
to google".
Perhaps a link to Mark Hammond's wicked-cool Python Sidebar for
Firefox/Mozilla <http://starship.python.net/crew/mhammond/mozilla/> and
to any other similar aids.

3) Links to threads that answer common questions
Yes, there is an FAQ and yet people still ask why Python gets division
wrong. But I've often seen people dig up links to threads for questions
perhaps not so common to be written up in an FAQ, but perhaps common
enough to merit linking on a wiki page. I'm not under the illusion that
it would solve the problem on its own, but it might well head some
repeat questions off. For those it doesn't, it would give an easy way to
answer.

Yes, some (or even all) of this info is already on Python.org. But this
way would allow it to be audience-tailored and to be ultra-easy to
direct newbie's toward. And yes, not all newbies read Eric Raymond's
"Smart Questions" essay within the first 5 times they are pointed at it.
Clearly it won't be a silver bullet, but maybe it would help.

As a relative newbie myself, I think I might make a better person to
start off (1) and (2) than some of the old-hands who are further away
from the newbie mindset. And, on the "talk not followed by effort is
chatter" principle, I surely ought volunteer, too ;-) I'm happy to do
so, provided 1) it doesn't strike people as a silly or hopelessly naive
idea, 2) I'm not committed to providing a proof of concept within the
next week :-) , and 3) I can count on other's to do the wiki thing to my
starting effort.

So, if anyone's still reading, thanks. I'm curious to see if the
suggestion seems worthwhile. Best to all,

Brian vdB


Sounds good to me - go for it!
Jul 18 '05 #34
Maurice LING <ma*********@acm.org> wrote in message news:<ma**************************************@pyt hon.org>...
Interestingly enough, I sort of share your perception -- and I have
noticed the same thing, and seen it remarked upon by others, in other,
completely unrelated newsgroups as well, such as it.comp.macintosh.

It's not, I think, about newbies in general: people who come and post
help requests, without giving the information that's quite obviously
indispensabile to let us help them, keep getting treated with
unreasonable amounts of friendliness and courtesy even after many
requests for more info go unheeded, for example.

However, newbies who are clueless enough to come blasting in with the
usual whines we've heard a zillion times -- Macs cost too much, Python
is too slow, there's no apps for Macs, Python must absolutely add
feature X or it will die, Apple's gonna go broke tomorrow, etc, etc --
do appear to get on our collective nerves worse than their essentially
indistinguishable precursors did last year, two years ago, &c.

The accusation of being a troll often follows, breaking the old advice
to "Never attribute to malice what can be adequately explained by
incompetence". Maybe it's a case of the corollary to that old advice,
"Sufficiently advanced cluelessness is indistinguishable from malice".
But why that should be so this year more than last year, say, is not
clear to me...
Alex


I've already said the following and was not noticed:

1. it is a disk intensive I/O operation.
2. users delay is not in the equation (there is no user input)
3. I am not interested in the amount of time needed to develop it. But
only interested execution speed.

Thanks maurice

this is still not enough information to give an answer to. and as for
"disk intensive" all languages handle disk i/o about the same since
they have to rely on the underlying Operating System, its drivers.
Jul 18 '05 #35
Alex Martelli wrote:
while ((c = in.read()) != -1)
out.write(c);
Incidentally, that Java code copies one character at a time.
outputFile.write(inputFile.read())


The Python code is reading the entire string into memory and
then writing it.

The interpretter overhead vs system calls could be measured
by having the language involved in every byte transferred as
in the Java example, or negligibly as in the Python example.

Whenever anyone has an agenda, you can make all sorts of silly
claims. IMHO the best thing to do is to lead by examples.
For years many people claimed Perl was line noise, hard to
maintain etc. I never really saw much response, since the
Perl people were too busy writing real world code and helping
deliver part of the web revolution. (And selling zillions of
books for O'Reilly :-)

I couldn't find any Python success stories on python.org itself,
but if you dig you can find the stories for the Python Business
Forum as well as Pythonology.

There are however remarkably few where you can go grab the
source code and see how it is all put together. In fact I
couldn't find a single one, but didn't do an exhaustive
search from python.org.

Perhaps it is also worth linking to the projects done in
Python on SourceForge and elsewhere?

http://sf.net/softwaremap/trove_list.php?form_cat=178

Roger
Jul 18 '05 #36
[...]
As a relative newbie myself, I think I might make a better person to
start off (1) and (2) than some of the old-hands who are further away
from the newbie mindset. And, on the "talk not followed by effort is
chatter" principle, I surely ought volunteer, too ;-) I'm happy to do
so, provided 1) it doesn't strike people as a silly or hopelessly
naive
idea, 2) I'm not committed to providing a proof of concept within the
next week :-) , and 3) I can count on other's to do the wiki thing to
my
starting effort.

So, if anyone's still reading, thanks. I'm curious to see if the
suggestion seems worthwhile. Best to all,


This would be very nice in any OOS list!
+1

Fabio
Jul 18 '05 #37
Kendall Clark <ke*****@monkeyfist.com> writes:
The main reason Java is "faster" is because


I would reject the premise entirely.

When looking at desktop apps made with Python, they positively whiz when compared to Java
swing apps.

As for non desktop apps, the entire portage system of Gentoo is written in Python.
'emerge sync' causes a python app to synchronise a local application database with database at a non-local mirror.
It is i/o intensive and appears to work very well and very fast.
Jul 18 '05 #38
Brian van den Broek <bv****@po-box.mcgill.ca> writes:
(I recall a post I made to a LUG asking for help with an aborted attempt to get Linux and my laptop's Winmodem to
co-operate. I was flamed because my mail headers indicated that I was using Thunderbird on *Windows* -- the horror! --


Or perhaps because you had not read the many available resources on using winmodems under linux.

BTW, if you are looking for a decent mail client check out gnus. Of the over 20 mail clients that I have used over the
years, this is the best.
Jul 18 '05 #39
"Eric S. Johansson" <es*@harvee.org> writes:
I will argue that we might want to start including in the
headers of mailing list/newsgroups a dana reference.


I would argue that this would debase newsgroups even further.
Jul 18 '05 #40
I rarely find myself acting as an apologist for Java, and I understand
that the point Alex is making is that Python's performance for this
operation is quite good, and that the OP should post some code, but this
is really too unfair a comparison for me not to say something.

There are two major differences between these two programs:
- The Java version is doing a character by character copy; the Python
program reads the entire file into a buffer in one operation.
- The Java program is converting the entire file to and from Unicode;
the Python program is copying the literal bytes.

Here is a much more comparable Java program (that will fail if the file
size is over 2^31-1):

import java.io.*;

public class Copy {
public static void main(String[] args) throws IOException {
File inputFile = new File("/usr/share/dict/web2");
int bufferSize = (int)inputFile.length();
File outputFile = new File("/tmp/acopy");

FileInputStream in = new FileInputStream(inputFile);
FileOutputStream out = new FileOutputStream(outputFile);

byte buffer[] = new byte[bufferSize];
int len=bufferSize;

while (true)
{
len=in.read(buffer,0,bufferSize);
if (len<0 )
break;
out.write(buffer,0,len);
}

in.close();
out.close();
}
}

Here are the results I get with this program and Alex's Python program
on my G4-400 Mac:
kent% time java Copy
0.440u 0.320s 0:00.96 79.1% 0+0k 9+3io 0pf+0w
kent% time python Copy.py
0.100u 0.120s 0:00.31 70.9% 0+0k 2+4io 0pf+0w

The Python program is still substantially faster (3x), but with nowhere
near the margin Alex saw.

Kent

Alex Martelli wrote:
OK, could you provide a simple toy example that meets these conditions
-- does lot of identical disk-intensive I/O "in batch" -- and the
execution speed measured (and on what platform) for what Python and Java
implementations, please?

For example, taking a trivial Copy.java from somewhere on the net:

import java.io.*;

public class Copy {
public static void main(String[] args) throws IOException {
File inputFile = new File("/usr/share/dict/web2");
File outputFile = new File("/tmp/acopy");

FileReader in = new FileReader(inputFile);
FileWriter out = new FileWriter(outputFile);
int c;

while ((c = in.read()) != -1)
out.write(c);

in.close();
out.close();
}
}

and I observe (on an iBook 800, MacOSX 10.3.5):

kallisti:~ alex$ java -version
java version "1.4.2_05"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_05-141.3)
Java HotSpot(TM) Client VM (build 1.4.2-38, mixed mode)

-r--r--r-- 1 root wheel 2486825 12 Sep 2003 /usr/share/dict/web2

kallisti:~ alex$ time java Copy

real 0m7.058s
user 0m5.820s
sys 0m0.390s

versus:

kallisti:~ alex$ time python2.4 Copy.py

real 0m0.296s
user 0m0.080s
sys 0m0.170s

with Python 2.4 beta 1 for the roughly equivalent:

inputFile = file("/usr/share/dict/web2", 'r')
outputFile = file("/tmp/acopy", 'w')

outputFile.write(inputFile.read())

inputFile.close()
outputFile.close()

which isn't all that far from highly optimized system commands:

kallisti:~ alex$ time cp /usr/share/dict/web2 /tmp/acopy

real 0m0.167s
user 0m0.000s
sys 0m0.040s

kallisti:~ alex$ time cat /usr/share/dict/web2 >/tmp/acopy

real 0m0.149s
user 0m0.000s
sys 0m0.090s
I'm sure the Java version can be optimized easily, too -- I just grabbed
the first thing I saw off the net. But surely this example doesn't
point to any big performance issue with Python disk I/O wrt Java. So,
unless you post concrete examples yourself, the smallest the better,
it's going to be pretty difficult to understand where your doubts are
coming from!
Alex

Jul 18 '05 #41
Roger Binns <ro****@rogerbinns.com> wrote:
Alex Martelli wrote:
while ((c = in.read()) != -1)
out.write(c);
Incidentally, that Java code copies one character at a time.


Yep, I noticed -- pretty silly, but if that's how the Java designers
decided the read method should behave by default, who am I to argue?
Just an example I grabbed off the net, first google hit for (if I recall
correctly) java file reading that had the source of a complete example.
outputFile.write(inputFile.read())


The Python code is reading the entire string into memory and
then writing it.


Right, _Python_'s default.
The interpretter overhead vs system calls could be measured
by having the language involved in every byte transferred as
in the Java example, or negligibly as in the Python example.
The claim posted to this newsgroup, without any support nor examples
being given, was that Python's I/O was far slower than Java's in
_disk-intensive_ operations. I'm still waiting to see any small,
verifiable examples of that posted on this thead.

Apparently, at least in default operations on both sides, based on calls
to read without parameters, that is definitely not the case - no doubt,
as you say, that's because of the different ways those defaults are
tuned, making Python much faster. Great, then let those who claim
Java's I/O is much faster in disk intensive operation post suitable
examples, and we'll see.

Whenever anyone has an agenda, you can make all sorts of silly
claims. IMHO the best thing to do is to lead by examples.
I posted examples -- probably not "leading" ones, just one grabbed off
the net using Java's defaults, and one using Python's defaults. I'm not
the one making any claims regarding I/O performance of _properly tuned_
disk-intensive programs -- what I would like to see would be some such
examples posted by those who DO make such claims. Shouldn't the burden
of proof lay on the side making positive assertions, by ordinary rules?!
For years many people claimed Perl was line noise, hard to
maintain etc. I never really saw much response, since the
Perl people were too busy writing real world code and helping
deliver part of the web revolution. (And selling zillions of
books for O'Reilly :-)
Surely more than we have sold so far with Python, yes - Tim O'Reilly has
published data about that. Which is why I'm striving to _worsen_ the
quality of Python's free docs until they match Perl's, against heathens
who (can you imagine that!) are instead trying to *improve* them further
(thus no doubt hurting book sales).

I couldn't find any Python success stories on python.org itself,
but if you dig you can find the stories for the Python Business
Forum as well as Pythonology.
Lots of digging, yes, considering that if you google for python success
stories those pbf and pythonology hits are just at the top (other
similar links, such as those to O'Reilly's booklets covering the same
material and their PDF forms, make up the rest of the top-ten pages).
There are however remarkably few where you can go grab the
source code and see how it is all put together. In fact I
couldn't find a single one, but didn't do an exhaustive
search from python.org.
Without straying from the top google hit, you can find that some of the
listed success stories are open-source, such as mayavi -- they're
admittedly a minority of the "success stories" listed on these sites.
Perhaps it is also worth linking to the projects done in
Python on SourceForge and elsewhere?

http://sf.net/softwaremap/trove_list.php?form_cat=178


If the message you're keen to send is "Python is great for open-source",
yes. If you're focusing on "Python is great for your _business_" (as
the python *business* forum does, for example), then emphasizing
open-source projects can reasonably be considered secondary to
emphasizing projects that make or save money for the businesses which
developed them.

If you're looking for highly visible open-source projects using Python,
I'd think that bittorrent, chandler, zope and plone, mayavi, all the
scipy.org site, twisted, schooltool, ubuntu, and the like, should keep
you in reading materials for a while. Not sure how many of these
projects are specifically on sourceforge, but I believe some minority
probably is.
Alex
Jul 18 '05 #42
On Sat, 06 Nov 2004 20:53:26 +0000, Israel Raj T wrote:
As for non desktop apps, the entire portage system of Gentoo is written in
Python. 'emerge sync' causes a python app to synchronise a local
application database with database at a non-local mirror. It is i/o
intensive and appears to work very well and very fast.


"emerge sync" mostly runs an rsync command (at least on standard
installs). The python contribution to that process is minimal.

The primary slowdown with "emerge" appears to be the way the database is
stored as many thousands of little files in /usr/portage. Again, not
really python; you want to accelerate emerge, speed up your hard drive.

(They have more cached metadata now at least in the cutting-edge release,
but as that is the one I use I don't know if that is now considered
stable. It has really sped things up. A couple more iterations and Python
might actually become the bottleneck.)
Jul 18 '05 #43
Israel Raj T said unto the world upon 2004-11-06 16:01:
Brian van den Broek <bv****@po-box.mcgill.ca> writes:

(I recall a post I made to a LUG asking for help with an aborted
attempt to get Linux and my laptop's Winmodem to co-operate. I was
flamed because my mail headers indicated that I was using
Thunderbird on *Windows* -- the horror! --

Or perhaps because you had not read the many available resources on
using winmodems under linux.

BTW, if you are looking for a decent mail client check out gnus. Of
the over 20 mail clients that I have used over the years, this is the
best.


Perhaps. But the context was that I was a Linux neophyte (not much
beyond that now), identified myself as such, made sure to manifest that
I'd tried and failed to sort it out for myself, and did my best to
appear to be asking for help learning to fish, rather than to be handed
a cooked trout. :-) (As in, I'd read and tried to follow the advice of
Raymond's essay.)

I'm certain that the sort of care I put into asking the question made it
fall outside the scope of the comments about newbie questions made
up-thread. I've yet to see a Python list so react to someone who made it
clear they were trying to follow the norms, even if they didn't meet
with complete success. I think the issue is about those who don't try to
follow courteous practise, and perhaps aren't aware they aren't.

Any project, Linux, Python, whatever, that aims to get a user base
beyond the gurus has to accept that not everyone who needs help will be
able to work everything out based on what they can find on-line. (It is
clear to me that Python lists *do* accept this.) The problem isn't with
those who get stuck after trying for themselves; it those who hit send
before search :-)

But thanks for the comment, and the client pointer. Best,

Brian vdB

Jul 18 '05 #44
Kent Johnson <ke******@yahoo.com> wrote:
I rarely find myself acting as an apologist for Java, and I understand
that the point Alex is making is that Python's performance for this
operation is quite good, and that the OP should post some code, but this
is really too unfair a comparison for me not to say something.
I'm glad I posted a sufficiently silly comparison to elicit some
response, then;-)

There are two major differences between these two programs:
- The Java version is doing a character by character copy; the Python
program reads the entire file into a buffer in one operation.
- The Java program is converting the entire file to and from Unicode;
the Python program is copying the literal bytes.
Right: each is using the respective language's defaults, and Python's
are apparently tuned for speed, while Java's apparenty aren't.

Here is a much more comparable Java program (that will fail if the file
size is over 2^31-1):
I believe that the Python program, if run on a suitable 64-bit OS, on a
ridiculously-large-memory machine, could succeed for much larger files
than that. Machines with many gigabytes of physical RAM are becoming
far from absurd these days -- I can't yet afford to throw 2500 euros out
of the window to buy 8 GB of fast RAM, but if I could it would fit
snugly into my own cheap dual-G5 powermac, for example (old model: I got
it used/reconditioned many months ago; I think the current cheaper model
does top out at 4 GB). On 32-bit machines or OSs, of course, Python's
memory limits _will_ byte at fewer GB than that, too.

import java.io.*;

public class Copy {
public static void main(String[] args) throws IOException {
File inputFile = new File("/usr/share/dict/web2");
int bufferSize = (int)inputFile.length();
File outputFile = new File("/tmp/acopy");

FileInputStream in = new FileInputStream(inputFile);
FileOutputStream out = new FileOutputStream(outputFile);

byte buffer[] = new byte[bufferSize];
int len=bufferSize;

while (true)
{
len=in.read(buffer,0,bufferSize);
if (len<0 )
break;
out.write(buffer,0,len);
}

in.close();
out.close();
}
}

Here are the results I get with this program and Alex's Python program
on my G4-400 Mac:
kent% time java Copy
0.440u 0.320s 0:00.96 79.1% 0+0k 9+3io 0pf+0w
kent% time python Copy.py
0.100u 0.120s 0:00.31 70.9% 0+0k 2+4io 0pf+0w
With your program, and mine simply converted to run inside a main()
function rather than at module-level for comparison, switching to tcsh
for direct comparison with the format you're using, I see:

[kallisti:~] alex% time java Copy
0.200u 0.140s 0:00.54 62.9% 0+0k 0+0io 0pf+0w
[kallisti:~] alex% time python Copy.py
0.080u 0.020s 0:00.13 76.9% 0+0k 0+1io 0pf+0w

Which python and java versions are you using? I'm trying to use the
latest and greatest of each, 2.4b1 (I know, I know, I need to install
b2!) for Python, 1.4.2_05 for Java -- just upgraded to MacOSX 10.3.6,
and my /usr/share/dict/web2 is 2486825 bytes.
The Python program is still substantially faster (3x), but with nowhere
near the margin Alex saw.


I still see a 4:1 ratio, but, sure, nowhere like the 20:1 my originally
silly example showed. Maybe a more realistic program would use a buffer
of some fixed length, rather than 'as long as the whole file'. Say:

int bufferSize = 64 * 1024;

in a program that otherwise is just like yours, for the Java side of
things. Switching back to bash because I can't stand long exposures to
anything in the csh family;-), I see:

kallisti:~ alex$ time java Copy

real 0m0.521s
user 0m0.200s
sys 0m0.120s

after several runs, so everything gets a chance to go to cache. Oh,
btw, I did compile with 'javac -O Copy.java'.

The closest Python equivalent I know how to write is:

def main():
inputFile = file("/usr/share/dict/web2", 'r')
bufferSize = 64 * 1024
outputFile = file("/tmp/acopy", 'w')

inf = inputFile.read
ouf = outputFile.write

while 1:
buf = inf(bufferSize)
if not buf: break
ouf(buf)

inputFile.close()
outputFile.close()

and its performance:

kallisti:~ alex$ time python -O Copy.py

real 0m0.135s
user 0m0.050s
sys 0m0.050s

so we still see the 4:1 ratio in favour of Python. It's barely more
than 3:1 in actual CPU time, user and sys, but for some reason Python
seems to be able to get more of the CPU's % attention -- I don't claim
to understand this!-)

So I scp'd everything over to the powermac dual g5 1.8 GHz, and ssh'd
there to do more measurements -- no python 2.4 there (it's my production
machine -> no betas!) and again I measured after a few runs to let stuff
get into cache:

macarthur:~ alex$ time java Copy

real 0m0.163s
user 0m0.040s
sys 0m0.110s

macarthur:~ alex$ time python -O Copy.py

real 0m0.039s
user 0m0.020s
sys 0m0.020s

Far better real/cpu ratios of course (a dual-CPU machine does that;-).
And faster overall. But roughly the same 4:1 ratio in favour of Python.
So, same thing on the oldie but goodie Linux box. In that case I don't
have an updated Java -- Kaffe 1.0.7 will have to do! The CPU is oldish
(Athlon 1.2G), but the disk subsystem is really really good. So I got
(again, good repeatable results after a few runs to stabilize):

[alex@lancelot alex]$ time java Copy
0.07user 0.04system 0:00.11elapsed 98%CPU (0avgtext+0avgdata
0maxresident)k
0inputs+0outputs (1623major+659minor)pagefaults 0swaps

[alex@lancelot alex]$ time python2.4 -O Copy.py
0.02user 0.01system 0:00.02elapsed 115%CPU (0avgtext+0avgdata
0maxresident)k
0inputs+0outputs (460major+241minor)pagefaults 0swaps

don't ask ME how python got over 100% CPU on a single-CPU machine; I
guess the task is just too small, at such tiny fractions of a second, to
avoid anomalies. Still, the roughly 4:1 performance ratio appears to be
repeatable here; and an interesting clue is that _pagefaults_ also
appear to be in roughly 4:1 ratio.

But clearly we need bigger tasks to make measurements less fraught with
anomalies. Unfortunately, not knowing on exactly which tasks the OP had
observed and measured his reported results that "Python is slower than
Java" in "a disk intensive I/O operation", it's hard to guess.
Obviously, his results are anything but _trivial_ to reproduce based on
the dearth of information that he has released about them so far; I find
it hard to think of something more disk-intensive than a simple copy of
one file to another. Assuming he does have some interest in
ascertaining the truth and getting his questions answered, perhaps he'll
deign to give us Java and Python sources, and reference data file[s],
for a small benchmark that _does_ reproduce his results. Otherwise, it
appears we may be just shooting in the dark.
Alex
Jul 18 '05 #45

dude that "comparision" from twistedmatrix you refrence is ANCIENT!!!
I am wondering the impact when IBM decided that the base memory to not
exceed 64kb, in the late 1960s...

I suppose more experienced people in this list can agree that certain
decisions made can be almost an edict. So, there is a re-building
process every now and then, hopefully to by-pass such edicts. Python
itself is already such an example.

it is comparing versions that are YEARS out of date and use!
Are the codebase of Python 1.5.2 and Java 1.1 totally replaced and
deprecated?

Lisp compiler is the 1st compiler to be created (according to the
Red-Dragon book, I think) and almost all others are created by
bootstrapping to LISP compiler. What are the implications of design
decisions made in LISP compiler then affecting our compilers today? I
don't know. I repeat myself, I DO NOT KNOW.

you are just trolling or your don't know enough to understand the
answer to your question which is way to vague to be answered, as there
is no real correct answer.


Certainly I do not have 15 PhDs in computer science or computating
mathematics...... I suppose there are some syntax error in your
statement to allow me to parse it completely. "too vague", not "to vague".

Thanks
maurice
Jul 18 '05 #46
Not only that, but Maurice Ling has started a long thread a few weeks
looking for a good research topic for his thesis (involving Java &
Python, btw). At some points he was bashed, but went ahead with the
discussion, that ended up touching in several interesting topics.
Thank you. You can do a google search and find my details anyway. Don't
have to go around guessing my details. I have no idea of the motives of
whoever that highlight "unimelb.edu.au". Does he have a problem with my
institution or what, I am not bothered, as I am only a student. Email to
the Vice Chancellor if he has a point to make, don't take it out on me,
I don't have the title of a Professor prefixing my name.

There is a Chinese saying "waves do not happen without wind." If my
impression of Python and Java is flawed, I am seriously wondering where
it came from? Is it all due to benchmark data from years ago?

Before I am accused of trolling etc etc again, let me say this all over
again. I am using both Python and Java actively and had contributed to
BioPython project as well.

As for the climate in c.l.py, it's just interesting to note that the
climate *is* getting less friendly, and that it does coincide with
several old-timers moving away from the list. A few years ago I
remember that the likes of Tim Peters, effbot, Skip Montanaro, and
several others (sorry, I would really like to remember more names from
the top of my head) were frequent posters here. They've now moved to
other interests, or are focusing their efforts on the python-dev list.
It must not be a coincidence.

Maybe as the old-timers go off, the next batch of old-timers feel that
it is too much to handle. Nobody requires anyone to answer to all
threads.....

Thanks
Maurice
Jul 18 '05 #47
Message from yahoo.com.
Unable to deliver message to the following address(es).

<ja*************@yahoo.com>:
64.157.4.78 failed after I sent the message.
Remote host said: 554 delivery error: dd This user doesn't have a
yahoo.com account (ja*************@yahoo.com) [0] -
mta109.mail.sc5.yahoo.com
Jul 18 '05 #48
Israel Raj T wrote:
"Eric S. Johansson" <es*@harvee.org> writes:

I will argue that we might want to start including in the
headers of mailing list/newsgroups a dana reference.

I would argue that this would debase newsgroups even further.


debase in what way? I certainly don't see enough money flowing that it
would lead major python contributors down a road of debauchery and
depravity. at least, those that aren't already there.

First, there would be a bunch of things required to make this work. A
pay on the newsreader client, a connection to a payment system, a
registered identity for payment. This is not a trivial amount of work.

Second, much of the conflict around various forms of intellectual
property boil down to getting paid. By building a low overhead
mechanism through which people could get paid for all sorts of
intellectual property (questions answered, music, video, writings,
code), we would jumpstart a new economic engine.

initially there would be a big rush of everybody on the face the planet
trying to get that dollar payment. It would probably be not unlike the
swarms of beggars that surround rich westerners when they travel through
less well-off regions. We would evolve filters and rating systems which
would be used to cut down on the noise. Eventually some contributors
would fallaway and others would persist and get better.

nothing would compel anyone to pay anything. But I could see eventually
folks being rated on whether or not they do pay. If you've got someone
who's always asking time-consuming questions and not paying anything, it
would be totally appropriate to filter out that person.

in the same community of users, you would have some people operating on
a gift economy, others operating on a financial economy and I believe
somehow, it would work. I can't yet describe how but I believe it would
work.

---eric

Jul 18 '05 #49
Alex Martelli wrote:
Kent Johnson <ke******@yahoo.com> wrote:
I rarely find myself acting as an apologist for Java...
I'm glad I posted a sufficiently silly comparison to elicit some
response, then;-)


Yes, good success on that one :-)
Which python and java versions are you using?


Old. Unfortunately the Mac is no longer my primary machine and not
up-to-date. I just used it to get timings comparable to yours. It's
Java 1.4.1_01
Python 2.3.3
Mac OSX 10.2.4

....We now return to our regularly scheduled program of trying to
convince Java programmers to try Python...

Kent
Jul 18 '05 #50

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

7
by: Catalin | last post by:
How can I make executables with python? I found some utilities that claim they can do something like that like Installer and py2exe but they actualy pack the code in a huge arhive! This solves the...
49
by: Ville Vainio | last post by:
I don't know if you have seen this before, but here goes: http://text.userlinux.com/white_paper.html There is a jab at Python, though, mentioning that Ruby is more "refined". -- Ville...
30
by: Stuart Turner | last post by:
Hi Everyone, I'm working hard trying to get Python 'accepted' in the organisation I work for. I'm making some good in-roads. One chap sent me the text below on his views of Python. I wondered...
118
by: 63q2o4i02 | last post by:
Hi, I've been thinking about Python vs. Lisp. I've been learning Python the past few months and like it very much. A few years ago I had an AI class where we had to use Lisp, and I absolutely...
34
by: Anthony Irwin | last post by:
Hi All, I am currently trying to decide between using python or java and have a few quick questions about python that you may be able to help with. #1 Does python have something like javas...
18
by: Jens | last post by:
I'm starting a project in data mining, and I'm considering Python and Java as possible platforms. I'm conserned by performance. Most benchmarks report that Java is about 10-15 times faster than...
16
by: Raxit | last post by:
Hi, i was reading/learning some hello world program in python. I think its very simillar to Java/C++/C#. What's different (except syntax) ? what can i do easily with python which is not easy...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: marcoviolo | last post by:
Dear all, I would like to implement on my worksheet an vlookup dynamic , that consider a change of pivot excel via win32com, from an external excel (without open it) and save the new file into a...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.