Hello list,
our team is going to rewrite our existing web-site, which has a lot of
dynamic content and was quickly prototyped some time ago.
Today, as we get better idea of what we need, we're going to re-write
everything from scratch. Python is an obvious candidate for our team:
everybody knows it, everybody likes it, it has *real* objects, nice
clean syntax etc.
Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_." The
main reason for that is that we want to debug our own bugs, but not
the bugs in our tools.
Our problem is - we yet have to find any example of high-traffic,
scalable web-site written entirely in Python. We know that YouTube is
a suspect, but we don't know what specific python web solution was
used there.
TurboGears, Django and Pylons are all nice, and provides rich features
- probably too many for us - but, as far as we understand, they don't
satisfy the stability requirement - Pylons and Django hasn't even
reached 1.0 version yet. And their provide too thick layer - we want
something 'closer to metal', probably similar to web.py -
unfortunately, web.py doesn't satisfy the stability requirement
either, or so it seems.
So the question is: what is a solid way to serve dynamic web pages in
python? Our initial though was something like python + mod_python +
Apache, but we're told that mod_python is 'scary and doesn't work very
well'.
And although http://www.python.org/about/quotes/ lists many big names
and wonderful examples, be want more details. E.g. our understanding
is that Google uses python mostly for internal web-sites, and
performance is far from perfect their. YouTube is an interesting
example - anybody knows more details about that?
Your suggestions and comments are highly welcome!
Best Regards,
Victor. 34  4304 
Victor Kryukov wrote:
Hello list,
our team is going to rewrite our existing web-site, which has a lot of
dynamic content and was quickly prototyped some time ago.
....
Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_." The
main reason for that is that we want to debug our own bugs, but not
the bugs in our tools.
You may not be happy with Python, then.
Having spent the last several months implementing a reasonably complex
web site in Python, I now have an understanding of the problems involved.
Which are non-trivial.
Some key web site components, like the SSL interface and the
MySQL interface, are written in C and maintained by third parties,
often by a single person. Getting the correct version for your
platform and making it work can be difficult. Getting the right
versions of MySQL, OpenSSL, and Python to all play together is
non-trivial. Expect to have to build from source, debug the build
process, look at source repositories, submit bug reports, fix
library bugs yourself, and maintain a local set of library patches.
Few hosting companies will have these modules available for you.
It's not like Perl or PHP, where it works out of the box. WebFaction
claims to support Python well, but few other hosting companies bother.
Most Linux distributions ship with older versions of Python, and
that's what most hosting companies will give you. Python 2.4
is par for the course.
High traffic sites are a problem. There are a number of
"frameworks", all supported by different people and with different
capabilities. See http://www.polimetrix.com/pycon/slides/
for a discussion. I haven't tried most of them, so I won't
say anything about that issue.
Our problem is - we yet have to find any example of high-traffic,
scalable web-site written entirely in Python. We know that YouTube is
a suspect, but we don't know what specific python web solution was
used there.
Not that much of YouTube is really in Python. The video
codecs aren't; they'd take forever if they were. And since Google
took over, the old YouTube search engine (which was terrible) was
replaced by Google's search technology.
John Nagle
On May 16, 6:04 pm, Victor Kryukov <victor.kryu...@gmail.comwrote:
Hello list,
our team is going to rewrite our existing web-site, which has a lot of
dynamic content and was quickly prototyped some time ago.
Today, as we get better idea of what we need, we're going to re-write
everything from scratch. Python is an obvious candidate for our team:
everybody knows it, everybody likes it, it has *real* objects, nice
clean syntax etc.
Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_." The
main reason for that is that we want to debug our own bugs, but not
the bugs in our tools.
Our problem is - we yet have to find any example of high-traffic,
scalable web-site written entirely in Python. We know that YouTube is
a suspect, but we don't know what specific python web solution was
used there.
TurboGears, Django and Pylons are all nice, and provides rich features
- probably too many for us - but, as far as we understand, they don't
satisfy the stability requirement - Pylons and Django hasn't even
reached 1.0 version yet. And their provide too thick layer - we want
something 'closer to metal', probably similar to web.py -
unfortunately, web.py doesn't satisfy the stability requirement
either, or so it seems.
So the question is: what is a solid way to serve dynamic web pages in
python? Our initial though was something like python + mod_python +
Apache, but we're told that mod_python is 'scary and doesn't work very
well'.
And althoughhttp://www.python.org/about/quotes/lists many big names
and wonderful examples, be want more details. E.g. our understanding
is that Google uses python mostly for internal web-sites, and
performance is far from perfect their. YouTube is an interesting
example - anybody knows more details about that?
Your suggestions and comments are highly welcome!
Best Regards,
Victor.
Take a look at reddit.com .
It's been developed with WEBPY and it receives millions of visits
every day.
There are also many sites built with Django (check their website) with
a lot of traffic and very good performance
And I'm sure other frameworks can show other success stories... just
check their websites.
Luis
On May 16, 6:04 pm, Victor Kryukov <victor.kryu...@gmail.comwrote:
Hello list,
our team is going to rewrite our existing web-site, which has a lot of
dynamic content and was quickly prototyped some time ago.
Today, as we get better idea of what we need, we're going to re-write
everything from scratch. Python is an obvious candidate for our team:
everybody knows it, everybody likes it, it has *real* objects, nice
clean syntax etc.
Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_." The
main reason for that is that we want to debug our own bugs, but not
the bugs in our tools.
Our problem is - we yet have to find any example of high-traffic,
scalable web-site written entirely in Python. We know that YouTube is
a suspect, but we don't know what specific python web solution was
used there.
TurboGears, Django and Pylons are all nice, and provides rich features
- probably too many for us - but, as far as we understand, they don't
satisfy the stability requirement - Pylons and Django hasn't even
reached 1.0 version yet. And their provide too thick layer - we want
something 'closer to metal', probably similar to web.py -
unfortunately, web.py doesn't satisfy the stability requirement
either, or so it seems.
So the question is: what is a solid way to serve dynamic web pages in
python? Our initial though was something like python + mod_python +
Apache, but we're told that mod_python is 'scary and doesn't work very
well'.
And althoughhttp://www.python.org/about/quotes/lists many big names
and wonderful examples, be want more details. E.g. our understanding
is that Google uses python mostly for internal web-sites, and
performance is far from perfect their. YouTube is an interesting
example - anybody knows more details about that?
Your suggestions and comments are highly welcome!
Best Regards,
Victor.
Take a look at reddit.com .
It's been developed with WEBPY and it receives millions of visits
every day.
There are also many sites built with Django (check their website) with
a lot of traffic and very good performance
And I'm sure other frameworks can show other success stories... just
check their websites.
Luis
On May 16, 6:04 pm, Victor Kryukov <victor.kryu...@gmail.comwrote:
Hello list,
our team is going to rewrite our existing web-site, which has a lot of
dynamic content and was quickly prototyped some time ago.
Today, as we get better idea of what we need, we're going to re-write
everything from scratch. Python is an obvious candidate for our team:
everybody knows it, everybody likes it, it has *real* objects, nice
clean syntax etc.
Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_." The
main reason for that is that we want to debug our own bugs, but not
the bugs in our tools.
Our problem is - we yet have to find any example of high-traffic,
scalable web-site written entirely in Python. We know that YouTube is
a suspect, but we don't know what specific python web solution was
used there.
TurboGears, Django and Pylons are all nice, and provides rich features
- probably too many for us - but, as far as we understand, they don't
satisfy the stability requirement - Pylons and Django hasn't even
reached 1.0 version yet. And their provide too thick layer - we want
something 'closer to metal', probably similar to web.py -
unfortunately, web.py doesn't satisfy the stability requirement
either, or so it seems.
So the question is: what is a solid way to serve dynamic web pages in
python? Our initial though was something like python + mod_python +
Apache, but we're told that mod_python is 'scary and doesn't work very
well'.
And althoughhttp://www.python.org/about/quotes/lists many big names
and wonderful examples, be want more details. E.g. our understanding
is that Google uses python mostly for internal web-sites, and
performance is far from perfect their. YouTube is an interesting
example - anybody knows more details about that?
Your suggestions and comments are highly welcome!
Best Regards,
Victor.
Reddit.com was built with webpy, and it's amongst the top 1000
websites in volume of traffic.
So if you liked it, go with it...
I believe one of the authors of reddit is also webpy's developer.
Luis
On May 16, 5:04 pm, Victor Kryukov <victor.kryu...@gmail.comwrote:
Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_." The
main reason for that is that we want to debug our own bugs, but not
the bugs in our tools.
I think this is a requirement that is pretty much impossible to
satisfy. Only dead frameworks stay the same. I have yet to see a
framework that did not have incompatible versions.
Django has a very large user base, great documentation and is deployed
for several online new and media sites. It is fast, it's efficient and
is simple to use. Few modern frameworks (in any language) are
comparable, and I have yet to see one that is better, http://code.djangoproject.com/wiki/DjangoPoweredSites
i.
Victor Kryukov napisa³(a):
Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_." The
main reason for that is that we want to debug our own bugs, but not
the bugs in our tools.
I don't think you find anything even remotely resembling that idea here.
Moreover, I don't think you find it elsewhere. Maybe even such tools do
not exist in nature?
--
Jarek Zgoda
"We read Knuth so you don't have to."
Victor Kryukov wrote:
Hello list,
our team is going to rewrite our existing web-site, which has a lot of
dynamic content and was quickly prototyped some time ago.
And has stayed around to dog the developers, as so many quick fixes do ...
Today, as we get better idea of what we need, we're going to re-write
everything from scratch. Python is an obvious candidate for our team:
everybody knows it, everybody likes it, it has *real* objects, nice
clean syntax etc.
Yes indeedy, that's our language!
Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_." The
main reason for that is that we want to debug our own bugs, but not
the bugs in our tools.
Given that even Apache has bugs despite its venerable status I think you
are setting your sights a little high here.
Our problem is - we yet have to find any example of high-traffic,
scalable web-site written entirely in Python. We know that YouTube is
a suspect, but we don't know what specific python web solution was
used there.
Zope? Plone? Django? TurboGears? All are handling large volumes of data
on a daily basis, and I wouldn't use either of them (but that's just a
personal issue).
TurboGears, Django and Pylons are all nice, and provides rich features
- probably too many for us - but, as far as we understand, they don't
satisfy the stability requirement - Pylons and Django hasn't even
reached 1.0 version yet. And their provide too thick layer - we want
something 'closer to metal', probably similar to web.py -
unfortunately, web.py doesn't satisfy the stability requirement
either, or so it seems.
So the question is: what is a solid way to serve dynamic web pages in
python? Our initial though was something like python + mod_python +
Apache, but we're told that mod_python is 'scary and doesn't work very
well'.
Python CGI? mod_python is OK, but like all frameworks you have to be
aware of its limitations.
And although http://www.python.org/about/quotes/ lists many big names
and wonderful examples, be want more details. E.g. our understanding
is that Google uses python mostly for internal web-sites, and
performance is far from perfect their. YouTube is an interesting
example - anybody knows more details about that?
Google use Python for all sorts of stuff.
Your suggestions and comments are highly welcome!
Best Regards,
Victor.
I think you are chasing a chimera (and I wrote a *book* called "Python
Web Programming").
regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
------------------ Asciimercial ---------------------
Get on the web: Blog, lens and tag your way to fame!!
holdenweb.blogspot.com squidoo.com/pythonology
tagged items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-------------- Thank You for Reading ----------------
John Nagle wrote:
Victor Kryukov wrote:
Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_." The
When settling on an environment you need to have a clear strategy
about component versions and compatibility. Although Python provides a
fairly reasonable backwards compatibility story, one still needs to be
aware of changes between versions and their implications, but this is
the case for a large proportion of the software produced today.
Despite the baggage maintained by Microsoft so that ancient
applications might still run in recent versions of Windows, to choose
a fairly extreme example in certain respects, there are still many
things to be aware of when maintaining or upgrading your environment.
Saying that something needs "Python plus Windows/Linux plus some
database system" isn't going to be enough if you're emphasizing
stability.
main reason for that is that we want to debug our own bugs, but not
the bugs in our tools.
You may not be happy with Python, then.
Having spent the last several months implementing a reasonably complex
web site in Python, I now have an understanding of the problems involved.
Which are non-trivial.
I'm sure they are, which is why I attempt to use the tools and
services available to reduce the impact of those problems on myself.
Some key web site components, like the SSL interface and the
MySQL interface, are written in C and maintained by third parties,
often by a single person. Getting the correct version for your
platform and making it work can be difficult. Getting the right
versions of MySQL, OpenSSL, and Python to all play together is
non-trivial. Expect to have to build from source, debug the build
process, look at source repositories, submit bug reports, fix
library bugs yourself, and maintain a local set of library patches.
I think we went through this before, but to what extent did you rely
on your distribution to handle many of these issues? Or was this on
Windows? If so, you're the master of your own distribution rather than
a group of helpful people who've probably already done most of that
work.
Few hosting companies will have these modules available for you.
It's not like Perl or PHP, where it works out of the box. WebFaction
claims to support Python well, but few other hosting companies bother.
Most Linux distributions ship with older versions of Python, and
that's what most hosting companies will give you. Python 2.4
is par for the course.
There's nothing wrong with Python 2.4. Perhaps you've inadvertently
highlighted an issue with the Python community: that everyone wants
the latest and greatest, and sees the CPython distribution as the
primary means of delivering it. If so, you've just included yourself
in the group causing yourself all those problems described above.
Paul
P.S. The inquirer may wish to visit sites representing the different
frameworks in order to seek testimonials, and to look for sites using
those frameworks in order to assess their popularity. Zope, Plone,
Django, TurboGears, Webware and many others have delivered high-
profile sites at various times, as far as I recall.
I think the first question I would have is what kind of dynamic
content are you talking about? Is this a web app kind of thing, or
just a content pushing site?
While Django might not be v1.0 yet, it seems very solid and stable,
and perfect for quickly building powerful content based dynamic sites.
Other Python frameworks might be well suited to other types of dynamic
sites. (I like TurboGears for actual web apps...)
Depending on your needs you might also consider a non-python solution
(!) like Drupal. (I'm not a PHP fan, but since other people have done
all that great work....:)
As others have said, there might be no framework currently in
existence that meets all of your requirements. You'll need to work
with your team to decide what kinds of compromises you can all live
with.
Chris
--
"A little government and a little luck are necessary in life, but only
a fool trusts either of them." -- P. J. O'Rourke
Jarek Zgoda wrote:
Victor Kryukov napisa³(a):
>>Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_." The
main reason for that is that we want to debug our own bugs, but not
the bugs in our tools.
I don't think you find anything even remotely resembling that idea here.
Moreover, I don't think you find it elsewhere. Maybe even such tools do
not exist in nature?
Sure they do. I have a complex web site, "http://www.downside.com",
that's implemented with Perl, Apache, and MySQL. It automatically reads SEC
filings and parses them to produce financial analyses. It's been
running for seven years, and hasn't been modified in five, except once
when the NASDAQ changed the format of their ticker symbol file.
It's not that useful at this point, because its purpose was to predict
failing dot-coms, but it's still running and doing a sizable amount of
work every day to update itself.
John Nagle
Victor Kryukov <vi************@gmail.comwrote:
...
And although http://www.python.org/about/quotes/ lists many big names
and wonderful examples, be want more details. E.g. our understanding
is that Google uses python mostly for internal web-sites, and
performance is far from perfect their. YouTube is an interesting
example - anybody knows more details about that?
Hmmm, I do, but exactly because I _do_ know a lot more details I cannot
discuss them unless you're under the proper Non-Disclosure Agreement
with Google, Inc. The best I can do otherwise is to point you to
already existing webpages -- I'm not going to reveal Google-Confidential
information, nor go to the substantial trouble to get such info cleared
by Legal and all other Google stakeholders.
For example, it HAS been published elsewhere that YouTube uses lighttpd,
not Apache: <http://trac.lighttpd.net/trac/wiki/PoweredByLighttpd>.
Fortunately, I managed to convince my YouTube colleagues to publically
present many more details about the evolution of their architecture
which had been discussed in Google-confidential talks and presentations
-- and my wife Anna, who's on the program selection committee of OSCON,
may have helped that talk get accepted (must not have been a hard
job:-). See:
<http://conferences.oreillynet.com/cs/os2007/view/e_sess/13435>
I hope to see you (and anybody else interested in solid technical
details about using Python for websites on YouTube's scale) in Portland,
OR on July 26 -- that will also be your first and best chance to ask
Mike Solomon specific questions that his talk might not directly
address. Once that's done, maybe somebody can convince the YouTube guys
to contribute a "Python Success Story" so that future querants about
this can be easily pointed to a URL!-)
I would also encourage anybody who's so keenly interested in Python to
visit our jobs-listing web app, e.g., if within the US, at
< http://www.google.com/support/jobs/b...00&dep_id=1173
&by_loc=1>. Of course, one should *never* have the implementation
language of a web-app show up in the URL, and I believe we've carefully
avoided that error in other external-facing web-apps, such as (one I can
reveal is indeed in Python, because that was mentioned at
< http://www.sauria.com/~twl/conferenc.../Python%20at%2
0Google.html>) code.google.com. Etc, etc.
Performance of web-apps (be they internal or external) depends more on
the architecture than on the implementation language (as long as highly
optimized C or C++, NOT Java or Python or whatever, is used for the few
jobs that are extremely CPU-intensive, such as codecs, of course:-).
So, if some Python-coded internal web-apps at Google perform badly
(which may be the case as you say, though I can't think of any
off-hand), it must be an issue of architecture. For example, a heavily
used internal web-app at Google is Mondrian,
<http://www.niallkennedy.com/blog/archives/2006/11/google-mondrian.html>
, Guido van Rossum's web-app for code review -- it's got a good, solid
architecture, and its performance is so good that many Googlers, me
included, have switched to it for all the reviews we do (and, believe
me, we do MANY -- _nothing_ is submitted to the global code repository
until it's been OK'd in a code review). I can't mention other such apps
because, AFAIK, they haven't been previously talked about in public and
so they're Google Confidential by default until otherwise determined.
Alex
John Nagle a écrit :
Victor Kryukov wrote:
>Hello list,
our team is going to rewrite our existing web-site, which has a lot of
dynamic content and was quickly prototyped some time ago.
...
>Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_." The
main reason for that is that we want to debug our own bugs, but not
the bugs in our tools.
You may not be happy with Python, then.
John, I'm really getting tired of your systemic and totally
unconstructive criticism. If *you* are not happy with Python, by all
means use another language.
Istvan Albert a écrit :
On May 16, 5:04 pm, Victor Kryukov <victor.kryu...@gmail.comwrote:
>Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_." The
main reason for that is that we want to debug our own bugs, but not
the bugs in our tools.
I think this is a requirement that is pretty much impossible to
satisfy. Only dead frameworks stay the same. I have yet to see a
framework that did not have incompatible versions.
Django has a very large user base, great documentation and is deployed
for several online new and media sites. It is fast, it's efficient and
is simple to use. Few modern frameworks (in any language) are
comparable, and I have yet to see one that is better,
Then have a look at Pylons.
Jarek Zgoda <jz****@o2.usun.plwrote:
Daniel Nogradi napisa?(a):
For example, it HAS been published elsewhere that YouTube uses lighttpd,
not Apache: <http://trac.lighttpd.net/trac/wiki/PoweredByLighttpd>.
How do you explain these, then:
http://www.youtube.com/results.xxx
http://www.youtube.com/results.php
http://www.youtube.com/results.py
Server signature is usually configurable.
Yeah, but I don't know why it's configured it that way. A good example
of a question that looks perfectly appropriate for YouTube's OSCON
session.
Alex
Bruno Desthuilliers wrote:
John Nagle a écrit :
>Victor Kryukov wrote:
>>Hello list,
our team is going to rewrite our existing web-site, which has a lot of
dynamic content and was quickly prototyped some time ago.
...
>>Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_." The
main reason for that is that we want to debug our own bugs, but not
the bugs in our tools.
You may not be happy with Python, then.
John, I'm really getting tired of your systemic and totally
unconstructive criticism. If *you* are not happy with Python, by all
means use another language.
Denying the existence of the problem won't fix it.
Many of the basic libraries for web related functions do have
problems. Even standard modules like "urllib" and "SSL" are buggy,
and have been for years. Outside the standard modules, it gets
worse, especially for ones with C components. Version incompatibility
for extensions is a serious problem. That's reality.
It's a good language, but the library situation is poor. Python as
a language is better than Perl, but CPAN is better run than Cheese Shop.
As a direct result of this, neither the Linux distro builders like
Red Hat nor major hosting providers provide Python environments that
just work. That's reality.
John Nagle
Alex Martelli wrote:
Jarek Zgoda <jz****@o2.usun.plwrote:
>>Daniel Nogradi napisa?(a):
>>>>For example, it HAS been published elsewhere that YouTube uses lighttpd,
not Apache: <http://trac.lighttpd.net/trac/wiki/PoweredByLighttpd>.
How do you explain these, then:
http://www.youtube.com/results.xxx
http://www.youtube.com/results.php
http://www.youtube.com/results.py
Server signature is usually configurable.
Yeah, but I don't know why it's configured it that way. A good example
of a question that looks perfectly appropriate for YouTube's OSCON
session.
YouTube's home page is PHP. Try "www.youtube.com/index.php".
That works, while the obvious alternatives don't.
If you look at the page HTML, you'll see things like
<a href="/login?next=/index.php"
onclick="_hbLink('LogIn','UtilityLinks');">Log In</a>
So there's definitely PHP inside YouTube.
If you look at the HTML for YouTube pages, there seem to be two
drastically different styles. Some pages begin with "<!-- machid: 169 -->",
and have their CSS stored in external files. Those seem to be generated
by PHP. Other pages start with
"<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">",
with no "machine ID". It looks like the stuff associated with
accounts and logging in is on the second system (Python?) while the
search and view related functions are on the PHP system.
Shortly after Google bought YouTube, they replaced YouTube's search
engine (which was terrible) with one of their own. At that time,
Google search syntax, like "-", started working. That's probably
when the shift to PHP happened.
John Nagle
On May 16, 11:04 pm, Victor Kryukov <victor.kryu...@gmail.comwrote:
>
Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_." The
main reason for that is that we want to debug our own bugs, but not
the bugs in our tools.
Our problem is - we yet have to find any example of high-traffic,
scalable web-site written entirely in Python. We know that YouTube is
a suspect, but we don't know what specific python web solution was
used there.
TurboGears, Django and Pylons are all nice, and provides rich features
- probably too many for us - but, as far as we understand, they don't
satisfy the stability requirement - Pylons and Django hasn't even
reached 1.0 version yet. And their provide too thick layer - we want
something 'closer to metal', probably similar to web.py -
unfortunately, web.py doesn't satisfy the stability requirement
either, or so it seems.
So the question is: what is a solid way to serve dynamic web pages in
python? Our initial though was something like python + mod_python +
Apache, but we're told that mod_python is 'scary and doesn't work very
well'.
AFAIK mod_python is solid and works well, but YMMV of course.
If you want rock solid stability, you want a framework where there is
little
development going on. In that case, I have a perfect match for your
requirements: Quixote. It has been around for ages, it is the most bug
free framework I have seen and it *very* scalable. For instance http://www.douban.com
is a Quixote-powered chinese site with more than 2 millions of pages
served per
day. To quote from a message on the Quixote mailing list:
"""
Just to report-in the progress we're making with a real-world Quixote
installation: yesterday douban.com celebrated its first 2 million-
pageview day. Quixote generated 2,058,207 page views. In addition,
there're about 640,000 search-engine requests. These put the combined
requests at around 2.7 millions. All of our content pages are
dynamic, including the help and about-us pages.
We're still wondering if we're the busiest one of all the python/ruby
supported websites in the world.
Quixote runs on one dual-core home-made server (costed us US$1500).
We have three additional servers dedicated to lighttpd and mysql. We
use memcached extensively as well.
Douban.com is the most visible python establishment on the Chinese
web, so there's been quite a few django vs. quixote threads in the
Chinese language python user mailing lists.
"""
Michele Simionato
John Nagle wrote:
YouTube's home page is PHP. Try "www.youtube.com/index.php".
That works, while the obvious alternatives don't.
If you look at the page HTML, you'll see things like
<a href="/login?next=/index.php"
onclick="_hbLink('LogIn','UtilityLinks');">Log In</a>
So there's definitely PHP inside YouTube.
Not sure; that "next" field is just the URL of the page you're on,
inserted into the output HTML. It's "index.php" because the page was
"index.php".
But it's an Apache server, with all the usual Apache messages.
John Nagle
John Nagle <na***@animats.comwrites:
As a direct result of this, neither the Linux distro builders like
Red Hat nor major hosting providers provide Python environments that
just work. That's reality.
Try SuSE, OpenSUSE, Ubuntu... They "just work". I've never had any
problem installing any library or module for Python. Even the ones that
require huge libraries or compiling something.
--
Jorge Godoy <jg****@gmail.com>
John Nagle wrote:
Many of the basic libraries for web related functions do have
problems. Even standard modules like "urllib" and "SSL" are buggy,
and have been for years. Outside the standard modules, it gets
worse, especially for ones with C components. Version incompatibility
for extensions is a serious problem. That's reality.
It's a good language, but the library situation is poor. Python as
a language is better than Perl, but CPAN is better run than Cheese Shop.
You know, submitting bug reports, patches, etc., can help make Python
better. And with setuptools' easy_setup, getting modules and packages
installed from the Cheese Shop is pretty painless.
- Josiah
John Nagle a écrit :
Bruno Desthuilliers wrote:
>John Nagle a écrit :
>>Victor Kryukov wrote:
Hello list,
our team is going to rewrite our existing web-site, which has a lot of
dynamic content and was quickly prototyped some time ago.
...
Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_." The
main reason for that is that we want to debug our own bugs, but not
the bugs in our tools.
You may not be happy with Python, then.
John, I'm really getting tired of your systemic and totally
unconstructive criticism. If *you* are not happy with Python, by all
means use another language.
Denying the existence of the problem won't fix it.
Neither will keeping on systematically criticizing on this newsgroup
instead of providing bug reports and patches.
As a direct result of this, neither the Linux distro builders like
Red Hat nor major hosting providers provide Python environments that
just work. That's reality.
I've been using Python for web applications (Zope, mod_python, fast cgi
etc) on Gentoo and Debian for the 4 or 5 past years, and it works just
fine. So far, I've had much more bugs and compatibility problems with
PHP (4 and 5) than with Python.
On May 16, 5:04 pm, Victor Kryukov <victor.kryu...@gmail.comwrote:
>
Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_." The
main reason for that is that we want to debug our own bugs, but not
the bugs in our tools.
youre not going to find a web development platform in any language at
all where you will not come across bugs. you will always have to
"worry" about versions. I have a day job where we worry about bugs
and versions in struts, hibernate, mysql, and spring all day long, and
each of those products probably has more users than all python
frameworks combined (feel free to whomever to bring out numbers, id be
glad to be proven wrong).
The web platform for Python which has the longest running time and the
most thousands-of-whatever hours is Zope (and Plone). All the others
which are popular today have only a tiny fraction of the in-production
time that Zope has. so if thats your criterion, then zope is what
you'd probably have to use.
TurboGears, Django and Pylons are all nice, and provides rich features
- probably too many for us - but, as far as we understand, they don't
satisfy the stability requirement - Pylons and Django hasn't even
reached 1.0 version yet. And their provide too thick layer - we want
something 'closer to metal', probably similar to web.py -
unfortunately, web.py doesn't satisfy the stability requirement
either, or so it seems.
I would seriously reconsider the notion that Pylons is "too thick" of
a layer. Pylons is quite open ended and non-opinionated. the
approaches of Pylons and Django couldnt be more different, so I would
suggest digging a little deeper into the various frameworks before
dismissing them on based on shallow judgments. Also, I understand
reddit is built on web.py, which is pretty high-traffic/proven/etc.
So the question is: what is a solid way to serve dynamic web pages in
python? Our initial though was something like python + mod_python +
Apache, but we're told that mod_python is 'scary and doesn't work very
well'.
mod_python works fantastically in my experience. that would satisfy
your requirement of stability as well as "close to the metal". but
youre going to have to roll your own pretty much everything...theres
only the most rudimdental controller layer, not much of an idea of url
resolution, and of course youd still have to figure out database/
templating. if you built a whole lot of custom mod_python handlers,
youd be tied to a very specific kind of process model and couldnt
really branch out into something like fcgi/mod_proxy->WSGI etc.
I think you guys have to either be less rigid about your requirements
and be willing to get your hands a little dirtier...web.py does seem
to be the kind of thing you guys would like, but if it has some issues
then youd just have to ....*shudder*....*contribute!* to that project
a little bit. its sort of par for the course in the field of open
source that youre going to have to be willing to contribute, if not
patches, then at least feedback and test cases to the developers for
issues found. if youre not willing to do that, you might have to
stick with J2EE for now.
On May 16, 2:04 pm, Victor Kryukov <victor.kryu...@gmail.comwrote:
Hello list,
our team is going to rewrite our existing web-site, which has a lot of
dynamic content and was quickly prototyped some time ago.
Today, as we get better idea of what we need, we're going to re-write
everything from scratch. Python is an obvious candidate for our team:
everybody knows it, everybody likes it, it has *real* objects, nice
clean syntax etc.
Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_." The
main reason for that is that we want to debug our own bugs, but not
the bugs in our tools.
Our problem is - we yet have to find any example of high-traffic,
scalable web-site written entirely in Python. We know that YouTube is
a suspect, but we don't know what specific python web solution was
used there.
TurboGears, Django and Pylons are all nice, and provides rich features
- probably too many for us - but, as far as we understand, they don't
satisfy the stability requirement - Pylons and Django hasn't even
reached 1.0 version yet. And their provide too thick layer - we want
something 'closer to metal', probably similar to web.py -
unfortunately, web.py doesn't satisfy the stability requirement
either, or so it seems.
So the question is: what is a solid way to serve dynamic web pages in
python? Our initial though was something like python + mod_python +
Apache, but we're told that mod_python is 'scary and doesn't work very
well'.
And althoughhttp://www.python.org/about/quotes/lists many big names
and wonderful examples, be want more details. E.g. our understanding
is that Google uses python mostly for internal web-sites, and
performance is far from perfect their. YouTube is an interesting
example - anybody knows more details about that?
Your suggestions and comments are highly welcome!
Best Regards,
Victor.
Teenwag runs on Python, with a hacked up Framework and recieves about
2million visitors a day and is constantly increasing http://teenwag.com/profile?friendid=326
Use php. I am lead programmer/maintainer of big website with a lot of
interactive stuff in user's backoffice and with a lot of interraction
to our non-web apps.
PHP is a crap, but programming for web in python is a pain in the ass.
And php programmers are cheaper. Especialy avoid mod_python.
IMHO.
On May 16, 4:04 pm, Victor Kryukov <victor.kryu...@gmail.comwrote:
Hello list,
our team is going to rewrite our existing web-site, which has a lot of
dynamic content and was quickly prototyped some time ago.
See #3 below
Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_."
TurboGears, Django and Pylons are all nice, and provides rich features
- probably too many for us - but, as far as we understand, they don't
satisfy the stability requirement - Pylons and Django hasn't even
reached 1.0 version yet.
See #3 below
And their provide too thick layer - we want
something 'closer to metal', probably similar to web.py -
See #1 below
Your suggestions and comments are highly welcome!
Victor et al,
I would propose that you ask some different questions. I propose these
out of personal experience, much of it from making poor decisions and
learning the hard way. Sorry if these sound gruff, I think its hard to
avoid when using bullet points; in my mind, these are all being
phrased as kindly and gently as I can.
#1
Seek flexibility over being "closer to metal." I say this because
whenever I've thought I wanted to be closer to metal, its because I
didn't want to be constrained by a framework's assumptions. I guess
another reason would be the need for raw performance with
computations, but I think you would have said that if that was your
goal. Still, "more flexible" is not always better. "flexible and well
integrated" is slightly better than "flexible to the Nth degree."
#2
Look for projects that deal with updates and security fixes in a way
that is sensitive to users of critical applications. This is better
than programs that change little. For example, does the "vendor"
provide patches containing just critical updates? Do they have good,
clear communication about changes that may break compatibility? How
long are older versions maintained? Asking these questions will help
you find a thriving project that is actively maintained and supported.
(contrast to abandonware, which hasn't changed in ages)
#3
Why haven't you mentioned maintainability or scalability? It sounds
like you're coming from a platform that you have outgrown, either
because your app can't keep up with it's load, or because you can't
enhance it with the features you want. You're not simply refactoring
it, you're starting over from the beginning. How often do you want to
start from scratch? If the answer is, "this is the last time," then
I'd worry *way* more about this and point #2 than anything else you
mentioned.
#4 (optional)
Has your dev team built many python web-apps? I'm guessing no, or
you'd already have picked a platform because of experience. If they
have not, I'd personally also ask for a platform that is easy to
learn, works well with the dev tools (IDE, debugger, version control)
you're familiar with, and has good documentation (dead tree or
online).
The unfortunate news, I'm afraid to say, is that because of the python
culture, you're still going to face tough decisions as there are
several mature products who score high marks in the areas I've listed
above. It seems the python community at large is insanely bent on
doing things the "right" way, so there may just be too many good
options to choose from.
But its better to ask the right questions.
--
Matthew Nuzum
newz2000 on freenode
Ivan Tikhonov a écrit :
Use php. I am lead programmer/maintainer of big website with a lot of
interactive stuff in user's backoffice and with a lot of interraction
to our non-web apps.
PHP is a crap, but programming for web in python is a pain in the ass.
Strange enough, MVHO on this is that PHP is crap *and* that programming
"for the web" with it is a king-size PITA, *specially* compared to Python.
And php programmers are cheaper.
"cheaper", yes. On an hourly basis. TCO is another problem...
Especialy avoid mod_python.
Unless you have to deeply integrate with (and are willing to be forever
tied to) Apache, I totally agree on this last one.
John Nagle a écrit :
(snip)
YouTube's home page is PHP. Try "www.youtube.com/index.php".
That works, while the obvious alternatives don't.
If you look at the page HTML, you'll see things like
<a href="/login?next=/index.php"
onclick="_hbLink('LogIn','UtilityLinks');">Log In</a>
So there's definitely PHP inside YouTube.
What about learning more on web servers configuration ? http://zope.alostnet.eu/index.php
"definitively php", hu ?
Please have a look at Plone and Zope.
"During the month of January 2006, we've had approx. 167 million hits"
plone.org
On 2007-05-16 23:04:17 +0200, Victor Kryukov <vi************@gmail.comsaid:
Hello list,
our team is going to rewrite our existing web-site, which has a lot of
dynamic content and was quickly prototyped some time ago.
Today, as we get better idea of what we need, we're going to re-write
everything from scratch. Python is an obvious candidate for our team:
everybody knows it, everybody likes it, it has *real* objects, nice
clean syntax etc.
Our main requirement for tools we're going to use is rock-solid
stability. As one of our team-members puts it, "We want to use tools
that are stable, has many developer-years and thousands of user-years
behind them, and that we shouldn't worry about their _versions_." The
main reason for that is that we want to debug our own bugs, but not
the bugs in our tools.
Our problem is - we yet have to find any example of high-traffic,
scalable web-site written entirely in Python. We know that YouTube is
a suspect, but we don't know what specific python web solution was
used there.
TurboGears, Django and Pylons are all nice, and provides rich features
- probably too many for us - but, as far as we understand, they don't
satisfy the stability requirement - Pylons and Django hasn't even
reached 1.0 version yet. And their provide too thick layer - we want
something 'closer to metal', probably similar to web.py -
unfortunately, web.py doesn't satisfy the stability requirement
either, or so it seems.
So the question is: what is a solid way to serve dynamic web pages in
python? Our initial though was something like python + mod_python +
Apache, but we're told that mod_python is 'scary and doesn't work very
well'.
And although http://www.python.org/about/quotes/ lists many big names
and wonderful examples, be want more details. E.g. our understanding
is that Google uses python mostly for internal web-sites, and
performance is far from perfect their. YouTube is an interesting
example - anybody knows more details about that?
Your suggestions and comments are highly welcome!
Best Regards,
Victor.
--
João Santos ma**@thegoldenaura.com www.thegoldenaura.com
John Nagle wrote:
Sure they do. I have a complex web site, "http://www.downside.com",
that's implemented with Perl, Apache, and MySQL. It automatically reads
SEC
filings and parses them to produce financial analyses. It's been
running for seven years, and hasn't been modified in five, except [..]
Well, how can you be then sure that you don't have any security hole in
there?
Ciao, Michael.
Hello list,
thanks a lot to everybody for their suggestions. We're yet to make our
final decision, and the information you've provided is really helpful.
Best,
Victor.
This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Brandon J. Van Every |
last post by:
What's better about Ruby than Python? I'm sure there's something. What is
it?
This is not a troll. I'm language shopping and I want people's answers. I
don't know beans about Ruby or have...
|
by: mike420 |
last post by:
I think everyone who used Python will agree that its syntax is
the best thing going for it. It is very readable and easy
for everyone to learn. But, Python does not a have very good
macro...
|
by: Svein Ove Aas |
last post by:
Is anyone working on a python-to-native compiler?
I'd be interested in taking a look.
Come to think of it, is anyone working on a sexpr-enabled version of
Python, or anything similar? I really...
|
by: Lad |
last post by:
Is anyone capable of providing Python advantages over PHP if there are
any?
Cheers,
L.
|
by: Philippe C. Martin |
last post by:
I apologize in advance for launching this post but I might get enlightment
somehow (PS: I am _very_ agnostic ;-).
- 1) I do not consider my intelligence/education above average
- 2) I am very...
|
by: mystilleef |
last post by:
Hello,
What is the Pythonic way of implementing getters and setters. I've
heard
people say the use of accessors is not Pythonic. But why? And what is
the alternative? I refrain from using them...
|
by: metaperl |
last post by:
Hello, I am responsible for converting 30 loosey-goosey Perl scripts
into 30 well-documented easy to understand and use Python scripts.
No one has said anything in particular about how this...
|
by: kaens |
last post by:
Hey everyone, I'm relatively new to python - I actually picked it up
to see how quickly I could start building non-trivial apps with it.
Needless to say, I was quite pleased.
Anyhow, I'm...
|
by: CloudSolutions |
last post by:
Introduction:
For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome former...
|
by: ryjfgjl |
last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
|
by: taylorcarr |
last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
|
by: aa123db |
last post by:
Variable and constants
Use var or let for variables and const fror constants.
Var foo ='bar';
Let foo ='bar';const baz ='bar';
Functions
function $name$ ($parameters$) {
}
...
|
by: ryjfgjl |
last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
| |
