Python as a Server vs Running Under Apache

[Matt Helm]

I am starting the design phase of a large project (ERP) where the
backend will mostly be Python (or Ruby) providing web services.

In this type of usage, is there any benenfit to running under Apache
as opposed to a pure Python solution using Medusa, TwistedMatrix, or
the like?

Thanks,
Matt

[sjdevnull]

Matt Helm wrote:

I am starting the design phase of a large project (ERP) where the
backend will mostly be Python (or Ruby) providing web services.

In this type of usage, is there any benenfit to running under Apache
as opposed to a pure Python solution using Medusa, TwistedMatrix, or
the like?

Running under Apache is nice if you need other apache services like
proxying, mod_rewrite, some of the accelerated static file serving
plugins, etc. Many of those now have replacements in Twisted (and
presumably other systems), but if you're coordinating with other parts
of the system running under Apache then it may be a reasonable choice.
Also, there may be some political/marketing gain (easier to pitch your
proposal if it's running under Apache).

But basically, if you're doing a pure-python system from scratch then
there's not much gained by running under Apache.

[Matt Helm]

<sj*******@yaho o.com> writes:

Running under Apache is nice if you need other apache services like
proxying, mod_rewrite, some of the accelerated static file serving
plugins, etc.
Not needed, this is web services only. That is, Apache is needed, but
only to handle the PHP consumers of the web services.

Also, there may be some political/marketing gain (easier to pitch your
proposal if it's running under Apache).
Not a problem. I get the final say-so on the design, platform, and
languages we use.

But basically, if you're doing a pure-python system from scratch then
there's not much gained by running under Apache.

Thanks, that is exactly what I was needing to know.

Matt

[Larry Bates]

Matt Helm wrote:

I am starting the design phase of a large project (ERP) where the
backend will mostly be Python (or Ruby) providing web services.

In this type of usage, is there any benenfit to running under Apache
as opposed to a pure Python solution using Medusa, TwistedMatrix, or
the like?

Thanks,
Matt

If it is a large project and you like Python as back end be sure
to evaluate using Zope (which can use Apache as a front end for
serving up static info and for SSL, etc.). With Zope you get a
lot of "free" stuff like XMPRPC, DAV, FTP, that can take quite
a lot of time to implement from scratch. It may not be for you,
but you owe it to yourself to take a look.

-Larry Bates

[fuzzylollipop]

as great as mod_python is, there are lots of restrictions and
limitations to what youc an do with it because of limitations of apache
itself, and I am refereing to apache 2.x as well as 1.x, like others
are saying if you don't need apache specific things it will just be one
more thing to work around the design constraints it causes, twisted
will be you best bet

[grahamd]

> as great as mod_python is, there are lots of restrictions and

limitations to what youc an do with it because of limitations of apache
itself, and I am refereing to apache 2.x as well as 1.x, like others
are saying if you don't need apache specific things it will just be one
more thing to work around the design constraints it causes, twisted
will be you best bet

I don't recollect that anyone had actually mentioned mod_python.
Certainly, mod_python isn't the only way of doing things via Apache
with Python.

Now, I am not trying to defend mod_python, as I probably better than
most understand many of the bugs/issues that the current version of
mod_python has, but what are the restrictions and limitations in it
that you see it has based on it being built on Apache.

Anyway, just curious to see what is behind your comment and how
deeply you truly understand the inner workings of mod_python, its
relationship to Apache and thus its problems.

Look forward to you answer.

Graham

[mojosam]

I guess I'm a little confused, and this certainly comes from not yet
having tried to do anything with Python on a web server.

I remarked once to a Python programmer that it appeared to me that if I
had a web page that called a Python program, that the server would:
1. Load Python
2. Run the program
3. Unload Python

Then the next time it has to serve up that page, it would have to
repeat the process. This seems inefficient, and it would slow the site
down. The programmer confirmed this. He said that's why I should use
mod_python. It stays resident.

Is this advice accurate? Are there other things to consider? Isn't
there just some way (short of running something like Zope) that would
keep Python resident in the server's RAM? This is a shared server, so
the web host probably doesn't like stuff sitting around in RAM.

Right now, I only need small programs to run. E.g., I'm thinking of
embedding a Live Journal blog in my web page. Live Journal gives you
several ways of doing this, one of which is three lines of Python code.

Ron Britton

(This email address will be deleted in a couple of days, once it starts
receiving spam from this posting. Please reply to the group!)

[Mike Meyer]

"mojosam" <zj*******@snea kemail.com> writes:

Is this advice accurate? Are there other things to consider? Isn't
there just some way (short of running something like Zope) that would
keep Python resident in the server's RAM? This is a shared server, so
the web host probably doesn't like stuff sitting around in RAM.
Jean-Paul Calderone has already answered most of these, so I'll get
the last one.

Stuff "sitting around in RAM" unusued on a busy server leaves RAM
pretty quickly on a modern system. The executable pages will just be
tossed, and reloaded from the executable file when they are needed
again. Data pages will be written out to disk in the swap area, and
read back in when they are needed. Unless your program's behavior is
very strange, this will generally be quicker than recreating the
program state from scratch. The end result is that your program loads
faster, and the real RAM used to support this is negligible.
Right now, I only need small programs to run. E.g., I'm thinking of
embedding a Live Journal blog in my web page. Live Journal gives you
several ways of doing this, one of which is three lines of Python code.

Trying to make your dynamic HTML content "fast" before you need to is
a premature optimization. Yes, CGI is slow compared to some of the
alternatives. But forks on Unix are still cheap, and you can handle
quite a bit of traffic with it on a modern system before you run into
a wall. Do it with CGI first, and fix it later if you need to.

<mike
--
Mike Meyer <mw*@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[fuzzylollipop]

there are lots of things you can't do or can't do easily or can't do at
efficiently in Apache using python as cgi or as anyone would more
likely assume mod_python. anything that requires any shared state or
shared resources in Apache is next to impossible. Doing similar things
in an app server or network application framework like twisted is
trivial.

[grahamd]

fuzzylollipop wrote:

there are lots of things you can't do or can't do easily or can't do at
efficiently in Apache using python as cgi or as anyone would more
likely assume mod_python. anything that requires any shared state or
shared resources in Apache is next to impossible. Doing similar things
in an app server or network application framework like twisted is
trivial.

What one really should be identifying here is that different tools are
better for different jobs.

Apache is a web server, and it isn't necessarily practical or sensible
to try and shoe horn it into service as a full blown application server
by and in itself. At the same time though, I would suggest that
something like Twisted is in itself not always the best solution
either, ie., to take an application server, and try and bolt on to it a
web server. Yes Twisted can perform as a web server, but when compared
to the functionality and configurability that Apache provides as a web
server, it is always going to be a lessor cousin in that role. If you
do not need the full power that Apache can provide and can live with
the lessor abilities inherent from running the web server as a
component within the application server, then great, but this isn't
necessarily going to suit everyone either.

Thus one is still faced with the question of whether you should embed
the application in the web server, or wrap a web server around the
application. Unfortunately from what I have seen, often the decision is
made based on the narrow focus of expertise of the developers involved.
A web developer will be dismissive of the application developer and
vice versus. Each will say that it is trivial if done in the system
which embodies their area of expertise, although usually that is said
without them really knowing how they will achieve the part they don't
know too much about. The strongest personalities thus will win as to
how the system is built.

The alternative is to accept that both Apache and application
frameworks such as Twisted are each good in their own space and use
both, bridging between the two as necessary. This could be done using
simple protocols like XML-RPC or more sophisticated messaging system
protocols where persistent connections can exist between the Apache
child processes and the backend application server thus avoiding the
creation of socket connections on every request. The use of messaging
systems also means that presence information can extend into the realm
of the web server and it can be easier to deal with failure or
unavailability of components of the backed system.

Keeping a split like this means you also have the added benefit that
you don't end up with application server code which is somehow
inherently dependent on or knowing that it is being used purely within
the context of a particular web server. You don't end up with
application server code being tied to the use of a specific means of
rendering web pages. With this sort of split it doesn't matter whether
you use CGI, mod_python, mod_perl or PHP for web page generation.
Provided that the interfaces between the two are well defined, you can
change the application server code structure as much as you like and
the web page generation code doesn't need to change. The opposite also
applies in as much as changes to the web page generation code shouldn't
affect the application server code.

Overall I would have to say that you are both right and wrong. Where
the state is served purely by the use of the database, there is nothing
wrong with Apache/mod_python in itself being used. Where the state also
embodies the need to have active agents which operate on that data,
possibly independent of web requests, then a back end application
server which embeds the core functionality of the application would be
better. In doing that though, it doesn't mean that Apache has to be
discarded as the web server, and a web server embeded in the
application server be adopted. Depending on the requirements, you will
get a more flexible, more easily maintained system if a combination of
the two technologies is used.

Anyway, that statisifies my daily need for a good rant. ;-)

Graham