fairly large webapp: from Java to Python. experiences?

[john_sips_tea]

I've got a fairly substantial webapp written in Java (plus Tomcat,
Hibernate, Struts, JSP, MySQL) that is a bit of a bear to work with. I
didn't write it. Much of it is only very sparsely documented (if at
all). No design docs anywhere. It's a large webapp with many classes
and fairly deep inheritance hierarchies. There seems to be JUnit test
cases in place.

The possibility of re-writing it has come up, and I must say that,
although it would likely be a ton of work, it could also be a real
pythonic adventure. ;) Reasons for a rewrite are to get better
flexibility, easier maintenance (easier to make changes), and the
possibility of really slimming down the size of the thing. And of
course, it would be nice to work with Python -- possibly significantly
more productive.

Does anyone here have any experience with large(ish) webapps in Python?
I was sniffing around the following A-team of toolkits: CherryPy,
Cheetah, SQLObject, all behind Apache2. Has anyone seen performance
issues with Python webapps?

Could one reasonably expect to have an easier time with maintainence
and future modifications with using Python over Java?

Any tips, stories, recommendations, and/or experiences are most
welcome.

Thanks.

[brianhray]

To replace a large framework you will probably need a framework. Take a
look at http://www.djangoproject.com or http://www.turbogears.org. They
both use some of the tools you mention but operate on a higher level.

I find Python fairly easy to maintain. Unfortunatly, I do not find it
easy to take a bunch of Java code and move it to Python. But once it is
there, Python is a good choice for web apps. Java is slow and has a big
footprint. Python is reasonably nimble and good design choices pay off.
Plus, you must write many more lines of Java to do what you can in
Python. I do not know the code you are not working with, but a lot of
large Java apps can be poorly written.

I guess you could also look at http://www.jython.org/ and see if you
can somehow make the transition but not all at once.

Good luck,
<http://brianray.chipy.org>

[John M. Gabriele]

br*******@gmail.com wrote:

To replace a large framework you will probably need a framework.
Well, I'm sure not all web frameworks are created equal, however,
CherryPy does bill itself as a "web framework".
Take a
look at http://www.djangoproject.com or http://www.turbogears.org. They
both use some of the tools you mention but operate on a higher level.
Django looks interesting to me. Would like to try out mod_python.
Taking a look...
I find Python fairly easy to maintain. Unfortunatly, I do not find it
easy to take a bunch of Java code and move it to Python.
Well, I figured it might be better to use the previous webapp
as a *model* to learn from, rather than to directly translate
code from Java to Python.
But once it is
there, Python is a good choice for web apps. Java is slow
Slow? They're both dynamic languages, but Java is statically
typed (with less work to do at runtime). For long-running processes,
I'd guess that Java bytecode executes faster than Python bytecode.
and has a big
footprint.
Yes.
Python is reasonably nimble and good design choices pay off.
Plus, you must write many more lines of Java to do what you can in
Python.
Yup. The main issue with the current Java webapp seems to be
that it takes a long time to make changes/additions.
I do not know the code you are not working with, but a lot of
large Java apps can be poorly written.

I've found that good documentation is *most* critical. At least
if the docs are good, but the code needs work, you have some
direction as to how to fix the code. If there's no docs, you
spend hours wondering why things are done the way they're done.

Currently having a look at the Django docs. :)

Thanks Brian!
---J
--
(remove zeez if demunging email address)

[Scott David Daniels]

John M. Gabriele wrote:

br*******@gmail.com wrote:

But once it is
there, Python is a good choice for web apps. Java is slow

Slow? They're both dynamic languages, but Java is statically
typed (with less work to do at runtime). For long-running processes,
I'd guess that Java bytecode executes faster than Python bytecode.

Well, perhaps yes, but not so if you count the fact that recoding
is so easy that you can race several implementations of the same
thing to see which trade-offs work. Java may run the one algorithm
you decide to write faster than Python will run the same thing,
but in Python you may race five algorithms and keep the fastest.
Using programmer resources effectively is Python's secret sauce.

--Scott David Daniels
sc***********@acm.org

[Giovanni Bajo]

John M. Gabriele wrote:

But once it is
there, Python is a good choice for web apps. Java is slow

Slow? They're both dynamic languages, but Java is statically
typed (with less work to do at runtime). For long-running processes,
I'd guess that Java bytecode executes faster than Python bytecode.

It's not the raw computing performance that counts in this case. I got this
joke in my mail today:

Python:
print "%10.2f" % x

Java:
java.text.NumberFormat formatter = java.text.NumberFormat.getNumberInstance();
formatter.setMinimumFractionDigits(2);
formatter.setMaximumFractionDigits(2);
String s = formatter.format(x);
for (int i = s.length(); i < 10; i++) System.out.print(' ');
System.out.print(s);

Let alone the time it takes to write this routine, I'm hundered percent sure
that the Python's version is also faster at runtime. Python lets you write
pretty expressive code which is easy to maintain and where the computation cost
is easily at the C level. Also Python code is pretty bare-metal, so that
file.write or socket.write go to the syscall immediately. Try that in Java and
you'll find 30 layers of complex abstractions for doubtful benefits and obvious
slowness.
--
Giovanni Bajo

[Shalabh Chaturvedi]

jo***********@yahoo.com wrote:

I've got a fairly substantial webapp written in Java (plus Tomcat,
Hibernate, Struts, JSP, MySQL) that is a bit of a bear to work with. I
didn't write it. Much of it is only very sparsely documented (if at
all). No design docs anywhere. It's a large webapp with many classes
and fairly deep inheritance hierarchies. There seems to be JUnit test
cases in place.

The possibility of re-writing it has come up, and I must say that,
although it would likely be a ton of work, it could also be a real
pythonic adventure. ;) <snip> Any tips, stories, recommendations, and/or experiences are most
welcome.

A class-to-class and method-to-method rewrite will give some but likely
not the full benefit of moving to Python. A redesign might be necessary
- making it more 'Pythonic' in the process. In my experience, many cruft
classes that exist in a Java design simply disappear when ported to
Python. See also http://dirtsimple.org/2004/12/python-is-not-java.html

Cheers,
Shalabh

[John M. Gabriele]

Shalabh Chaturvedi wrote:

jo************@yahooz.com wrote:

A class-to-class and method-to-method rewrite will give some but likely
not the full benefit of moving to Python. A redesign might be necessary
- making it more 'Pythonic' in the process. In my experience, many cruft
classes that exist in a Java design simply disappear when ported to
Python. See also http://dirtsimple.org/2004/12/python-is-not-java.html

Cheers,
Shalabh

Great link Shalabh. Thanks. :) Googling around, I'd also come across
that one.

---J

--
(remove zeez if demunging email address)

[AdSR]

Giovanni Bajo wrote:

Also Python code is pretty bare-metal, so that
file.write or socket.write go to the syscall immediately. Try that in Java and
you'll find 30 layers of complex abstractions for doubtful benefits and obvious
slowness.

+1 QOTW
(I'd recommend the whole post but it might be too long.)

Cheers,

AdSR

[Fabio Zadrozny]

I agree that python code is usually smaller... but what you did is too
unfair (the code below would be more suitable for the comparrison).

python:
print "%10.2f" % 10

java:
System.out.println(String.format("%10.2f", 10.0));

-- altough for me it would be the same, as I have defined a print
method... so, it is always something as:
print ("%10.2f", 10) in java for me :-)

What I'm trying to say here is that if you already have a big java app,
sometimes spending some time trying to refactor it for better
understanding is more useful than scrapping what already exists
(especially if you have good test cases)... altough I would reccomend
jython for new development on that same application.

I guess that the main difference is that python usually makes you do the
right thing, whereas in java you need to know a lot more about it to
manage to do the right thing...

Cheers,

Fabio

--
Fabio Zadrozny
------------------------------------------------------
Software Developer

ESSS - Engineering Simulation and Scientific Software
www.esss.com.br

PyDev - Python Development Enviroment for Eclipse
http://pydev.sf.net
http://pydev.blogspot.com


Giovanni Bajo wrote:

John M. Gabriele wrote:

But once it is
there, Python is a good choice for web apps. Java is slow

Slow? They're both dynamic languages, but Java is statically
typed (with less work to do at runtime). For long-running processes,
I'd guess that Java bytecode executes faster than Python bytecode.

It's not the raw computing performance that counts in this case. I got this
joke in my mail today:

Python:
print "%10.2f" % x

Java:
java.text.NumberFormat formatter = java.text.NumberFormat.getNumberInstance();
formatter.setMinimumFractionDigits(2);
formatter.setMaximumFractionDigits(2);
String s = formatter.format(x);
for (int i = s.length(); i < 10; i++) System.out.print(' ');
System.out.print(s);

Let alone the time it takes to write this routine, I'm hundered percent sure
that the Python's version is also faster at runtime. Python lets you write
pretty expressive code which is easy to maintain and where the computation cost
is easily at the C level. Also Python code is pretty bare-metal, so that
file.write or socket.write go to the syscall immediately. Try that in Java and
you'll find 30 layers of complex abstractions for doubtful benefits and obvious
slowness.

[Kent Johnson]

Fabio Zadrozny wrote:

I agree that python code is usually smaller... but what you did is too
unfair (the code below would be more suitable for the comparrison).

python:
print "%10.2f" % 10

java:
System.out.println(String.format("%10.2f", 10.0));

Though String.format() is new in Java 1.5 so in older code or for
backward compatibility the longer code may be found. OTOH Python has
supported % formatting since at least version 1.4 released in 1996.
http://www.python.org/doc/1.4/lib/no...00000000000000

In my experience the original point is valid - Python is usually
(always?) more concise than equivalent Java code, and often dramatically
so. Python makes common operations easy; Java sometimes seems to go out
of its way to make them awkward.

Kent

[Fabio Zadrozny]

Kent Johnson wrote:

Fabio Zadrozny wrote:

I agree that python code is usually smaller... but what you did is too
unfair (the code below would be more suitable for the comparrison).

python:
print "%10.2f" % 10

java:
System.out.println(String.format("%10.2f", 10.0));

Though String.format() is new in Java 1.5 so in older code or for
backward compatibility the longer code may be found. OTOH Python has
supported % formatting since at least version 1.4 released in 1996.
http://www.python.org/doc/1.4/lib/no...00000000000000

In my experience the original point is valid - Python is usually
(always?) more concise than equivalent Java code, and often dramatically
so. Python makes common operations easy; Java sometimes seems to go out
of its way to make them awkward.

Kent

Yeap, I agree with the original point too (as I explained in the rest of
my comment)... just didn't think it was a 'fair' comparisson. ;-)
-- me? When I use java I write 1.5 code and 'retroweave' it to be
compatible with java 1.4 (Pydev is done this way) -- and I use Python at
work -- and I'm very happy with it... I just think that a language is as
a tool, and each may be suited better for a different occasion.

The 'usually smaller' I mentioned is because sometimes java code gets
smaller because of the already-existing codebase... E.g.: for developing
an IDE, Eclipse already has a huge codebase, so, if I wanted to do all
that was already done in it in python just because Eclipse was in java,
I think I would be doing the wrong decision, and besides, I could use
jython for it if I wanted to...

And from what I understood, the original poster had a large codebase in
java already, so, I'm not sure that changing everything to python would
be the way to go (altough that DOES depend a lot on the original
codebase quality).

Cheers,

Fabio

--
Fabio Zadrozny
------------------------------------------------------
Software Developer

ESSS - Engineering Simulation and Scientific Software
www.esss.com.br

PyDev - Python Development Enviroment for Eclipse
http://pydev.sf.net
http://pydev.blogspot.com

[George Sakkis]

> Any tips, stories, recommendations, and/or experiences are most

welcome.

Just one suggestion, read the article "Things You Should Never Do, Part
I" first (
http://www.joelonsoftware.com/articl...00000069.html). Quoting
from the article:

(Netscape made) "the *single worst strategic mistake* that any software
company can make: They decided to rewrite the code from scratch."

As much as I find python more maintenable and fun to work with in the
long run, I would be *very hesitant* to start a complete rewrite,
especially given the insufficient documentation of your webapp and your
lack of experience with a pythonic web framework.

George

[bruno at modulix]

jo***********@yahoo.com wrote:

(snip)

Does anyone here have any experience with large(ish) webapps in Python?
Depends on the definition of "large(ish)". If you use KLOC as a metric,
you can expect a Python solution to be 5 to 10 time smaller, so what's a
'large' app in Java may end up as a small/medium app with Python !-)
I was sniffing around the following A-team of toolkits: CherryPy,
Cheetah, SQLObject, all behind Apache2.
If you plan to run behind Apache2, you may want to have a look at
Myghty. It's somewhere between a framework and a templating system, and
the architecture is based on Perl's HTML::Mason, which has proven to be
successful. It adds to Mason the possibility to take a clean MVC
approach (as well as the more ServerPage oriented approach of Mason),
and can be deployed as either a mod_python handler, fastcgi, plain old
cgi, WSGI, and even as a standalone app (with a Python HTTPServer) for
developpement.

You may also want to take a look at Turbogears (CherryPy + SQLObject +
Kid + Mochikit) or Subway (CherryPy + SQLObject + Cheetah).

Has anyone seen performance
issues with Python webapps?
Actually, only with Plone or CPS (or other ZopeCMF based app). Zope
itself is pretty usable, but the whole CMF part is a monstruosity IMHO.
Could one reasonably expect to have an easier time with maintainence
and future modifications with using Python over Java?
Err... It of course depends on first having well written code and a
sensible design, but when it comes to agility - and as long as you stay
away from Zope's CMF !-) -, I bet that Python beats Java hands down.
Any tips, stories, recommendations, and/or experiences are most
welcome.

It took me about one month to write my first Zope application - an
ad-hoc mix of an online catalog and basic CMS (pages, news,
downloadables files, ...), with LDAP and PostgresSQL in the mix,
fulltext and advanced search, newsletter subscription, a 'members' zone
etc. I had to learn Zope during the process, and I guess I could
actually rewrite the whole thing in less than 2 weeks with a much better
design - but what, the app is in production for 2 years with near zero
corrective maintenance (the biggest problem we had was with system
updates messing with the psycopg adapter...), gives full satisfaction to
the customer, and has for now proven to be very easy to modify and
evolve (so easy that we didn't even bothered to bill the customer for
much of the modification requests)... And Zope is certainly not known
as the most agile or lightweight Python web framework/application server !-)

--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"