Modules for inclusion in standard library?

[Reinhold Birkenfeld]

Hello,

at the moment python-dev is discussing including Jason Orendorff's path module
into the standard library.

Do you have any other good and valued Python modules that you would think are
bug-free, mature (that includes a long release distance) and useful enough to
be granted a place in the stdlib?

For my part, ctypes seems like a suggestion to start with.

Reinhold

[Thomas Heller]

Reinhold Birkenfeld <re************************@wolke7.net> writes:

Hello,

at the moment python-dev is discussing including Jason Orendorff's path module
into the standard library.

I have never used the path module before, although I've heard good
things about it. But, it seems to have problems with unicode pathnames,
at least on windows:

C:\>mkdir späm

C:\späm>py24
Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

from path import path
path.getcwd() Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "C:\TSS5\components\_Pythonlib\path.py", line 97, in getcwd
return path(os.getcwd())
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 5: ordinal not in range(128)

Although it seems this could be fixed by adding this constructor to the
path class:

def __new__(cls, init=u""):
if isinstance(init, unicode):
return _base.__new__(cls, init)
return _base.__new__(cls, init, sys.getfilesystemencoding())

I would really appreciate if Python's unicode support would be better
;-), and maybe it's easier to add this to the path module than to
os.path.

Thomas

[Fredrik Johansson]

On 6/27/05, Reinhold Birkenfeld <re************************@wolke7.net> wrote:

Do you have any other good and valued Python modules that you would thinkare
bug-free, mature (that includes a long release distance) and useful enough to
be granted a place in the stdlib?
First of all, numeric/numarray, obviously!

I'd also like to see wxPython, pygame and SciPy, but these may be too
heavy-weight. A good distribution system for widely used packages may
be a better solution than including everything in the standard
library.
For my part, ctypes seems like a suggestion to start with.

I think ctypes simply *must* go in there sooner later. My experience
dealing with binary data in Python is that it's a pain in the neck
compared to C, and the more machine-like behaviour and elegant code
you want, the slower it gets. I have refrained from using ctypes for a
number of reasons, but I would probably use it if it was in the
standard library.

-- Fredrik Johansson

[Steven Bethard]

Reinhold Birkenfeld wrote:

For my part, ctypes seems like a suggestion to start with.

I believe this has been discussed somewhere before and the conclusion
was that ctypes should not be a candidate for inclusion in the Python
stdlib because people don't want things in the stdlib that can make
Python segfault.

STeVe

[Steven Bethard]

Fredrik Johansson wrote:

On 6/27/05, Reinhold Birkenfeld <re************************@wolke7.net> wrote:

Do you have any other good and valued Python modules that you would think are
bug-free, mature (that includes a long release distance) and useful enough to
be granted a place in the stdlib?

First of all, numeric/numarray, obviously!

There has been recent discussion about this. Check the python-dev list
archives I think. It's unlikely that all of numeric/numarray could go
into the Python stdlib because there still is disagreement between the
two camps as to the module architecture. However, there does seem to be
some agreement at the level of the basic array object, so it may be
possible that at least the array object itself might join the stdlib in
the not too distant future. I suspect there's more detailed discussion
of this in the numeric/numarray lists.

STeVe

[George Sakkis]

I'd love to see IPython replace the standard interpreter. Pychecker
seems to be a strong candidate too.

George

[Robert Kern]

Reinhold Birkenfeld wrote:

Hello,

at the moment python-dev is discussing including Jason Orendorff's path module
into the standard library.

Do you have any other good and valued Python modules that you would think are
bug-free, mature (that includes a long release distance) and useful enough to
be granted a place in the stdlib?

I would like to see the setuptools/PythonEggs/EasyInstall trifecta get
more attention and eyeballs. Once it is mature, I think that it will
obviate the desire for stdlibification of most of the packages being
requested here.

--
Robert Kern
rk***@ucsd.edu

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

[Michael Hoffman]

Robert Kern wrote:

I would like to see the setuptools/PythonEggs/EasyInstall trifecta get
more attention and eyeballs. Once it is mature, I think that it will
obviate the desire for stdlibification of most of the packages being
requested here.

Looks pretty cool!
--
Michael Hoffman

[Chris Connett]

pyparsing is the far and away the easiest general purpose parser out
there that I've encountered; BNF-style grammar parsing is a *pleasure*
with pyparsing. And it all comes in a single pure python module to
boot.

[projecktzero]

I'll 2nd the vote for Pychecker.

[bruno modulix]

George Sakkis wrote:

I'd love to see IPython replace the standard interpreter.

I dont.

--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"

[George Sakkis]

> "bruno modulix" <on***@xiludom.gro> wrote:

George Sakkis wrote:

I'd love to see IPython replace the standard interpreter.

I dont.

Care to say why ?

George

[John Roth]

"Reinhold Birkenfeld" <re************************@wolke7.net> wrote in
message news:3i************@individual.net...

Hello,

at the moment python-dev is discussing including Jason Orendorff's path
module
into the standard library.

Do you have any other good and valued Python modules that you would think
are
bug-free, mature (that includes a long release distance) and useful enough
to
be granted a place in the stdlib?

For my part, ctypes seems like a suggestion to start with.

Reinhold

I'd go with path. While I'm not at all certain that it's "the" way
to go, it'll at least break the conceptual logjam caused by simply
providing wrappers around the C library functions.

I'd definitely like to see ctypes. I can agree with the segfault
issue, but I think that some design work would eliminate that.

PyChecker, rather obviously. Likewise I'd like to see something
that would do a decent job of coverage analysis. Neither of the
modules out there at the moment is ready for prime time.

I don't have enough experience with interactive mode to have
an opinion on IPython. What I would like to see is a Python
based shell that could compete with either ksh or Monad.

John Roth

[Paul Rubin]

Reinhold Birkenfeld <re************************@wolke7.net> writes:

Do you have any other good and valued Python modules that you would think are
bug-free, mature (that includes a long release distance) and useful enough to
be granted a place in the stdlib?

How about the win32 shell extension that allows stuff like reading
the registry? I'm not sure if that's the same as win32com.shell.

[Reinhold Birkenfeld]

George Sakkis wrote:

"bruno modulix" <on***@xiludom.gro> wrote:

George Sakkis wrote:

> I'd love to see IPython replace the standard interpreter.

I dont.

Care to say why ?

For an easy, quick interactive interpreter, it's way to overloaded
with functions and too slow in startup.

However, having ipython in the Python distribution is worth a thought.

Reinhold

[George Sakkis]

"Reinhold Birkenfeld" <re************************@wolke7.net> wrote:

George Sakkis wrote:

"bruno modulix" <on***@xiludom.gro> wrote:

George Sakkis wrote:
> I'd love to see IPython replace the standard interpreter.
I dont.

Care to say why ?

For an easy, quick interactive interpreter, it's way to overloaded
with functions and too slow in startup.

Too slow ? It doesn't take more than a second or two to startup in a
two years old 1.8Ghz Athlon and an older 900Mhz sparcv9 I usually work
on. How slow is it at platforms you have experience with ?
However, having ipython in the Python distribution is worth a thought.

Reinhold

George

[Robert Kern]

George Sakkis wrote:

"Reinhold Birkenfeld" <re************************@wolke7.net> wrote:

For an easy, quick interactive interpreter, it's way to overloaded
with functions and too slow in startup.

Too slow ? It doesn't take more than a second or two to startup in a
two years old 1.8Ghz Athlon and an older 900Mhz sparcv9 I usually work
on. How slow is it at platforms you have experience with ?

I think that it loads up too much stuff to be the default interpreter,
speed issues aside. Sometimes, you really need to debug something that
requires you to control the imports.

It's a moot point anyways. The current ipython codebase isn't in any
shape to go into the stdlib, and the rewrite hasn't quite begun, yet.
Ask again in a year or two.

--
Robert Kern
rk***@ucsd.edu

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

[Simon Brunning]

On 6/28/05, John Roth <ne********@jhrothjr.com> wrote:

I'd definitely like to see ctypes. I can agree with the segfault
issue, but I think that some design work would eliminate that.

I'm not sure that it would. Ctypes allows you, as one colleague
memorably put it, to "poke the operating system with a stick". You are
always going to be able to use ctypes to provoke spectacular crashes
of the kind that you can never get with 'ordinary' Python.

Having said that, I'd like to see ctypes in the standard library
anyway, with a suitably intimidating warning in the docs about the
trouble you can get yourself into with it.

--
Cheers,
Simon B,
si***@brunningonline.net,
http://www.brunningonline.net/simon/blog/

[Thomas Heller]

Simon Brunning <si************@gmail.com> writes:

On 6/28/05, John Roth <ne********@jhrothjr.com> wrote:

I'd definitely like to see ctypes. I can agree with the segfault
issue, but I think that some design work would eliminate that.
I'm not sure that it would. Ctypes allows you, as one colleague
memorably put it, to "poke the operating system with a stick". You are
always going to be able to use ctypes to provoke spectacular crashes
of the kind that you can never get with 'ordinary' Python.

Right.
Having said that, I'd like to see ctypes in the standard library
anyway, with a suitably intimidating warning in the docs about the
trouble you can get yourself into with it.

To me, this sounds that *at least* a PEP would be needed to convince
Guido. Or, to record the reasoning why it cannot be included.

Thomas

[Gregory Piñero]

I'd like to see some database API's to the most common databases
included. It would make Python much more useful for web development.
I've come across situations where a web host supports python and
supports MySQL yet it's taken me days to get the MySQLAPI installed
with running setup in my home directory etc. And I don't know what
options you have if you don't have shell access?

It definately seems to me that some API's to popular database would be
conducive to a "batteries included" approach.

-Greg

On 6/29/05, Thomas Heller <th*****@python.net> wrote:

Simon Brunning <si************@gmail.com > writes:

On 6/28/05, John Roth <ne********@jhrothjr.com > wrote:

I'd definitely like to see ctypes. I can agree with the segfault
issue, but I think that some design work would eliminate that.

I'm not sure that it would. Ctypes allows you, as one colleague
memorably put it, to "poke the operating system with a stick". You are
always going to be able to use ctypes to provoke spectacular crashes
of the kind that you can never get with 'ordinary' Python.

Right.

Having said that, I'd like to see ctypes in the standard library
anyway, with a suitably intimidating warning in the docs about the
trouble you can get yourself into with it.

To me, this sounds that *at least* a PEP would be needed to convince
Guido. Or, to record the reasoning why it cannot be included.

Thomas
--
http://mail.python.org/mailman/listinfo/python-list

[bruno modulix]

George Sakkis wrote:

"bruno modulix" <on***@xiludom.gro> wrote:

George Sakkis wrote:

I'd love to see IPython replace the standard interpreter.

I dont.

Care to say why ?

Sorry...

it was about the "replace", not about IPython itself nor about IPython
becoming part of the stdlib.

IPython is a great *advanced* REPL, but most of the time, I just don't
need its advanced features (I usually run the REPL as an emacs
subprocess), and moreover, beginners already have enough to learn !-)

regards
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'on***@xiludom.gro'.split('@')])"

[Paul Rubin]

Gregory Piñero <gr********@gmail.com> writes:

I'd like to see some database API's to the most common databases
included.

Yes, certainly, this is a serious deficiency with Python.

[Rocco Moretti]

Paul Rubin wrote:

Gregory Piñero <gr********@gmail.com> writes:

I'd like to see some database API's to the most common databases
included.

Yes, certainly, this is a serious deficiency with Python.

Except that (please correct me if I'm wrong) there is somewhat of a
policy for not including interface code for third party programs which
are not part of the operating system. (I.e. the modules in the standard
libary should all be usable for anyone with a default OS + Python install.)

A notable exception is the dbm modules, but I seem to recall hearing
that the official position is that it was a mistake. (Now only kept for
backward compatability.)

[Gregory Piñero]

While that policy does make sense, I think a database program falls
somewhere in between an OS and an everyday third party program. For
web developers, the database might as well be the OS. I use the
database to store everything in my web app. That way I can just worry
about 1 place to access information and not have to fool with files
and other OS issues.

So I humbly suggest the policy should be :

Python will not include interface code for third party programs which
are not part of an operating system or database system.

...
But I have no experience in designing world class programming
langauges so forgive me if I am too bold.

-Greg
On 6/29/05, Rocco Moretti <ro**********@hotpop.com> wrote:

Paul Rubin wrote:

Gregory Piñero <gr********@gmail.com > writes:

I'd like to see some database API's to the most common databases
included.

Yes, certainly, this is a serious deficiency with Python.

Except that (please correct me if I'm wrong) there is somewhat of a
policy for not including interface code for third party programs which
are not part of the operating system. (I.e. the modules in the standard
libary should all be usable for anyone with a default OS + Python install..)

A notable exception is the dbm modules, but I seem to recall hearing
that the official position is that it was a mistake. (Now only kept for
backward compatability.)
--
http://mail.python.org/mailman/listinfo/python-list

[Gregory Piñero]

And 1 more argument for adding DB support, a large part of PHP's
success as a web langauge is being easily interoperable with MySQL
(out of the box I think? I haven't used it.) But I think it's tight
integration with MySQL really helped it find its niche.

I think "batteries included" means Python MUST be useful for common
tasks right out of the box. Perhaps the only debate should be, what
are the most common tasks?

Just some more ideas to consider...

Greg
On 6/29/05, Gregory Piñero <gr********@gmail.com> wrote:

While that policy does make sense, I think a database program falls
somewhere in between an OS and an everyday third party program. For
web developers, the database might as well be the OS. I use the
database to store everything in my web app. That way I can just worry
about 1 place to access information and not have to fool with files
and other OS issues.

So I humbly suggest the policy should be :

Python will not include interface code for third party programs which
are not part of an operating system or database system.

..
But I have no experience in designing world class programming
langauges so forgive me if I am too bold.

-Greg


On 6/29/05, Rocco Moretti <ro**********@hotpop.com > wrote:

Paul Rubin wrote:

Gregory Piñero <gr********@gmail.com > writes:

>I'd like to see some database API's to the most common databases
>included.

Yes, certainly, this is a serious deficiency with Python.

Except that (please correct me if I'm wrong) there is somewhat of a
policy for not including interface code for third party programs which
are not part of the operating system. (I.e. the modules in the standard
libary should all be usable for anyone with a default OS + Python install.)

A notable exception is the dbm modules, but I seem to recall hearing
that the official position is that it was a mistake. (Now only kept for
backward compatability.)
--
http://mail.python.org/mailman/listinfo/python-list

[Noah]

unzip() -- Not really a module, but a standard library function.
Why isn't it in the standard library?
It seems like I'm always adding it to my code.
I think I once heard an argument against it, but I forget what it was.
And yet I still find myself wanting unzip.

def unzip(list):
if len(list) == 0: return ()
l = []
for t in range(len(list[0])):
l.append(map( lambda x,t=t: x[t], list ))
return tuple(l)

Yours,
Noah

Reinhold Birkenfeld wrote:

Hello,

at the moment python-dev is discussing including Jason Orendorff's path module
into the standard library.

Do you have any other good and valued Python modules that you would think are
bug-free, mature (that includes a long release distance) and useful enough to
be granted a place in the stdlib?

For my part, ctypes seems like a suggestion to start with.

Reinhold

[Paul Rubin]

Rocco Moretti <ro**********@hotpop.com> writes:

Except that (please correct me if I'm wrong) there is somewhat of a
policy for not including interface code for third party programs which
are not part of the operating system. (I.e. the modules in the
standard libary should all be usable for anyone with a default OS +
Python install.)
I've never heard of Python having such a policy and I don't understand
how such a stupid policy could be considered compatible with a
proclaimed "batteries included" philosophy. Why would Python
advocates want to make Python deliberately uncompetitive with PHP,
Java, and other languages that do include database modules?
A notable exception is the dbm modules, but I seem to recall hearing
that the official position is that it was a mistake. (Now only kept
for backward compatability.)

Ahem: Tkinter. There's actually several more, looking in the lib docs.

[Steven Bethard]

Noah wrote:

def unzip(list):
if len(list) == 0: return ()
l = []
for t in range(len(list[0])):
l.append(map( lambda x,t=t: x[t], list ))
return tuple(l)

The simplest solution to this problem that I know of:

def unzip(iterable):
return zip(*iterable)

However, Guido feels that this is an abuse of the argument passing
machinery. For a version that is a little more careful in the case of
iterables, see:

http://aspn.activestate.com/ASPN/Coo.../Recipe/302325

STeVe

[Christopher Arndt]

Reinhold Birkenfeld schrieb:

Do you have any other good and valued Python modules that you would think are
bug-free, mature (that includes a long release distance) and useful enough to
be granted a place in the stdlib?

Hmmm, let's look into <pythonlib>/site-packackes, That's what I always
have installed:

- pysqlite
- Numeric
- PIL
- mxExtensions (mostly for mxDateTime)
- ClientForm
- ClientCookie
- pycrypto

Most of these are probably not elegible due to license issues but I'd
love to see SQLite support added to Python-out-of-the-box.

Also, when on windows, I always install the ActiveState distro because
it comes with the win23 extensions and more useful documentation. Maybe
there should be rather a separate "add-on-packs" distribution?

I think, this would be mostly useful for developers though. With
solutions like p2exe and python eggs becoming more widely used, caring
about what is installed and what not, shouldn't be such an issue anymore.

Chris

[Mike Meyer]

Rocco Moretti <ro**********@hotpop.com> writes:

Except that (please correct me if I'm wrong) there is somewhat of a
policy for not including interface code for third party programs which
are not part of the operating system. (I.e. the modules in the
standard libary should all be usable for anyone with a default OS +
Python install.)

A notable exception is the dbm modules, but I seem to recall hearing
that the official position is that it was a mistake. (Now only kept
for backward compatability.)

Um - dbm is bundled with both os's I use (FreeBSD and OS X). I'd be
surprised if some variant or another didn't come standard on most
Linux systems as well.

Not all builtin functions are available on all os's. And the source
tree certainly includes modules that are specific to a single
operating systems (just look in the PC and Mac directories). I'm
pretty sure there are other modules that aren't supported on various
oddball systems - like termios.

Why should a module that's usable out-of-the-box on a fair variety of
systems be excluded just because it's not usable on all platforms?

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Terry Hancock]

On Wednesday 29 June 2005 04:55 am, Simon Brunning wrote:

On 6/28/05, John Roth <ne********@jhrothjr.com> wrote:

I'd definitely like to see ctypes. I can agree with the segfault
issue, but I think that some design work would eliminate that.

I'm not sure that it would. Ctypes allows you, as one colleague
memorably put it, to "poke the operating system with a stick". You are
always going to be able to use ctypes to provoke spectacular crashes
of the kind that you can never get with 'ordinary' Python.

Gosh, I didn't know or care about ctypes, but now I'm interested! ;-D

--
Terry Hancock ( hancock at anansispaceworks.com )
Anansi Spaceworks http://www.anansispaceworks.com

[Simon Brunning]

On 6/29/05, Christopher Arndt <ch*********@web.de> wrote:

(snip)

Most of these are probably not elegible due to license issues but I'd
love to see SQLite support added to Python-out-of-the-box.

Adding sqllite to the standard library has been discussed before:
http://groups-beta.google.com/group/...150297c201f814

--
Cheers,
Simon B,
si***@brunningonline.net,
http://www.brunningonline.net/simon/blog/

[Simon Brunning]

On 6/29/05, Thomas Heller <th*****@python.net> wrote:

To me, this sounds that *at least* a PEP would be needed to convince
Guido. Or, to record the reasoning why it cannot be included.

I have a feeling that Guido won't allow ctypes into the standard
library since it can crash Python. I don't know about you, but that's
I interpret this -
<http://mail.python.org/pipermail/python-dev/2004-January/041856.html>.

I am prepared to be wrong, though. Only Tim can channel Guido!

--
Cheers,
Simon B,
si***@brunningonline.net,
http://www.brunningonline.net/simon/blog/

[Harry George]

Paul Rubin <http://ph****@NOSPAM.invalid> writes:

Rocco Moretti <ro**********@hotpop.com> writes:

Except that (please correct me if I'm wrong) there is somewhat of a
policy for not including interface code for third party programs which
are not part of the operating system. (I.e. the modules in the
standard libary should all be usable for anyone with a default OS +
Python install.)

I've never heard of Python having such a policy and I don't understand
how such a stupid policy could be considered compatible with a
proclaimed "batteries included" philosophy. Why would Python
advocates want to make Python deliberately uncompetitive with PHP,
Java, and other languages that do include database modules?

A notable exception is the dbm modules, but I seem to recall hearing
that the official position is that it was a mistake. (Now only kept
for backward compatability.)

Ahem: Tkinter. There's actually several more, looking in the lib docs.

I typically install dozens of python packages (on IRIX, Solaris, AIX,
Linux, Win2K). 21 are standalone enough to be considered for the std
library. However I wouldn't necessarily want them in there, because:

a) They have their own release cycles, and coordinating would be too
painful. We'd get a Python-1.2.3 with a package ABC-2.3.4 which is
(too late) discovered to have a bug. So everyone would have to
download ABC-2.3.5 and install it anyway.

b) Installing distutils-aware python packages is trivial. I'd rather
the energy which might go into a bigger std library go instead into
helping projects which don't have distutils-style builds.
--
ha************@boeing.com
6-6M21 BCA CompArch Design Engineering
Phone: [phone]

[Christopher Arndt]

Simon Brunning schrieb:

On 6/29/05, Christopher Arndt <ch*********@web.de> wrote:
Adding sqllite to the standard library has been discussed before:
http://groups-beta.google.com/group/...150297c201f814

Yeah, but they didn't seem to have come to a conclusion then. Also,
pysqlite 2.x has had a final release now and is still fairly DB-API
compatible (at least is seems so by looking at the docs).

I might go to their mailing list and ask about the status.

Chris

[Tom Anderson]

On Wed, 29 Jun 2005, it was written:

Rocco Moretti <ro**********@hotpop.com> writes:

Except that (please correct me if I'm wrong) there is somewhat of a
policy for not including interface code for third party programs which
are not part of the operating system.

I've never heard of Python having such a policy and I don't understand
how such a stupid policy could be considered compatible with a
proclaimed "batteries included" philosophy.

Agreed. If this is the policy, it should be reconsidered. It's silly.

tom

--
How did i get here?

[Mike Meyer]

Harry George <ha************@boeing.com> writes:

b) Installing distutils-aware python packages is trivial. I'd rather
the energy which might go into a bigger std library go instead into
helping projects which don't have distutils-style builds.

How about integrating distutils and PyPI, so that distutils can
automatically download and install packages that are required by the
package it's currently installing? In other words, C-Python-AN.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Robert Kern]

Mike Meyer wrote:

Harry George <ha************@boeing.com> writes:

b) Installing distutils-aware python packages is trivial. I'd rather
the energy which might go into a bigger std library go instead into
helping projects which don't have distutils-style builds.

How about integrating distutils and PyPI, so that distutils can
automatically download and install packages that are required by the
package it's currently installing? In other words, C-Python-AN.

http://peak.telecommunity.com/DevCenter/EasyInstall

--
Robert Kern
rk***@ucsd.edu

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

[Max M]

Reinhold Birkenfeld wrote:

Hello,

Do you have any other good and valued Python modules that you would think are
bug-free, mature (that includes a long release distance) and useful enough to
be granted a place in the stdlib?

For my part, ctypes seems like a suggestion to start with.

ctypes certainly. Even though it can crash Python. People using ctypes
would be aware of this.

Another good bet is BeautifulSoup, which is absolutely great for
scraping content from webpages.

http://crummy.com/software/BeautifulSoup/index.html
--

hilsen/regards Max M, Denmark

http://www.mxm.dk/
IT's Mad Science

[Ivan Van Laningham]

Hi All--

Max M wrote:

Another good bet is BeautifulSoup, which is absolutely great for
scraping content from webpages.

http://crummy.com/software/BeautifulSoup/index.html

Not if you want to handle HTML in anything but ASCII. BeautifulSoup
insists you change your site.py to change the default encoding if you
want to use non-ASCII. It might work beautifully, but I won't use it,
at least not until it's fixed to understand encodings.

Metta,
Ivan
----------------------------------------------
Ivan Van Laningham
God N Locomotive Works
http://www.andi-holmes.com/
http://www.foretec.com/python/worksh...oceedings.html
Army Signal Corps: Cu Chi, Class of '70
Author: Teach Yourself Python in 24 Hours

[Harry George]

Mike Meyer <mw*@mired.org> writes:

Harry George <ha************@boeing.com> writes:

b) Installing distutils-aware python packages is trivial. I'd rather
the energy which might go into a bigger std library go instead into
helping projects which don't have distutils-style builds.

How about integrating distutils and PyPI, so that distutils can
automatically download and install packages that are required by the
package it's currently installing? In other words, C-Python-AN.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

I personally don't do auto download/installs, preferring instead to
scan the code before build/install. However, getting a list of
dependencies would be nice. Maybe a setup.py --requires flag.
--
ha************@boeing.com
6-6M21 BCA CompArch Design Engineering
Phone: [phone]

[Rocco Moretti]

Paul Rubin wrote:

Rocco Moretti <ro**********@hotpop.com> writes:

Except that (please correct me if I'm wrong) there is somewhat of a
policy for not including interface code for third party programs which
are not part of the operating system. (I.e. the modules in the
standard libary should all be usable for anyone with a default OS +
Python install.)

I've never heard of Python having such a policy and I don't understand
how such a stupid policy could be considered compatible with a
proclaimed "batteries included" philosophy. Why would Python
advocates want to make Python deliberately uncompetitive with PHP,
Java, and other languages that do include database modules?

Well, since there seems to be an outpouring of disgust at my statement,
and no official confirmation/rejection, it's probably a figment of my
prematurely failing mind.

However, if there was such a policy, it would not be unequivocally
"stupid." First off, there is a bit of flexibility in what is considered
part of the OS. E.g, Linux may properly refer to just the kernel, but
rarely is just the kernel installed. Various utilities and programs
might be considered part of the OS because they are ubiquitously
installed, or are included with the Python distribution itself (as Tk is
with windows Python).

For those programs which aren't ubiquitously installed, or even for ones
that are, but require significant configuration, it is reasonable to
expect that if someone has the ability and goes to the effort of
locating, obtaining, installing, and configuring a third party program,
they can just as easily obtain and install the python module, especially
as it's usually as easy as "python setup.py install".

At any rate, I'm not advocating such a policy, I'm just saying it can
make a bit of sense if you look at it from a certain angle.

[Daniel Dittmar]

Rocco Moretti wrote:

Except that (please correct me if I'm wrong) there is somewhat of a
policy for not including interface code for third party programs which
are not part of the operating system. (I.e. the modules in the
standard libary should all be usable for anyone with a default OS +
Python install.)

There are several degrees of std-ness for Python Modules:

Example expat: The sources for this extension module are contained in
the Python tarball, so this module is guaranteed to be part of a Python
distribution

Example zlib: If the include files and libs can be found, this extension
will be built. So it's part of std-Python (the source), but not part of
std-Python (installed). See others in Modules/Setup

Binary distributions may choose additional modules which appear to be
standard. Example: zlib is part of all Python-Installations on Windows
and most Linux-distribution will have readlines as 'standard'.

There seems to be a great reluctance by the Python developers to add
modules of the expat kind, as this means responsibilities for additional
source modules. There's also the problem with incompatible licenses,
integrating a second configure, deciding when to update to the latest
version of the library etc.

Another problem is that there are often several modules for a specific
problem (e.g. several modules interfacing to PostgreSQL), so there's
always a chance to make an anemy for life, no matter which you pick.

And module authors might not be interested as they'd be then bound by
the Python release cycle.

Daniel

[Scott David Daniels]

Daniel Dittmar wrote:

Rocco Moretti wrote:

Except that (please correct me if I'm wrong) there is somewhat of a
policy for not including interface code for third party programs which
are not part of the operating system. (I.e. the modules in the
standard libary should all be usable for anyone with a default OS +
Python install.)

....

There seems to be a great reluctance by the Python developers to add
modules of the expat kind, as this means responsibilities for additional
source modules. There's also the problem with incompatible licenses,
integrating a second configure, deciding when to update to the latest
version of the library etc.

If you haven't noticed, the Python code has a substantial body of unit
tests. Arranging the tests to be easily runnable for all developers
is going to be tough for "third party programs." Making the interfaces
work for differing versions of the 3PPs as the third parties themselves
change their interfaces (see fun with Tcl/Tk versions for example), and
building testbeds to test to all of those differing versions, would
cause a nightmare that would make a knight of Ni scream.

--Scott David Daniels
Sc***********@Acm.Org

[Colin J. Williams]

Steven Bethard wrote:

Fredrik Johansson wrote:

On 6/27/05, Reinhold Birkenfeld
<re************************@wolke7.net> wrote:

Do you have any other good and valued Python modules that you would
think are
bug-free, mature (that includes a long release distance) and useful
enough to
be granted a place in the stdlib?

First of all, numeric/numarray, obviously!

There has been recent discussion about this. Check the python-dev list
archives I think. It's unlikely that all of numeric/numarray could go
into the Python stdlib because there still is disagreement between the
two camps as to the module architecture. However, there does seem to be
some agreement at the level of the basic array object, so it may be
possible that at least the array object itself might join the stdlib in
the not too distant future. I suspect there's more detailed discussion
of this in the numeric/numarray lists.

There was a flurry of exchanges about three months ago but it soon died.

One thing which needs to be sorted out is the silent truncation of
complex values:
http://sourceforge.net/tracker/index...69&atid=450446

Colin W.
STeVe

[Colin J. Williams]

Gregory Piñero wrote:

While that policy does make sense, I think a database program falls
somewhere in between an OS and an everyday third party program. For
web developers, the database might as well be the OS. I use the
database to store everything in my web app. That way I can just worry
about 1 place to access information and not have to fool with files
and other OS issues.

So I humbly suggest the policy should be :

Python will not include interface code for third party programs which
are not part of an operating system or database system.

... Isn't this where the discussion should start? There should be some
general policy guiding the types of modules which should be in the
standard library. Clearly, size is a factor as the msi version of
Python 2.4 is now 10 MB.

if there were some sort of package manager which took account of
dependencies would that reduce the need to expand the standard library?

Colin W. But I have no experience in designing world class programming
langauges so forgive me if I am too bold.

-Greg
On 6/29/05, Rocco Moretti <ro**********@hotpop.com> wrote:

Paul Rubin wrote:

Gregory Piñero <gr********@gmail.com > writes:
I'd like to see some database API's to the most common databases
included.

Yes, certainly, this is a serious deficiency with Python.

Except that (please correct me if I'm wrong) there is somewhat of a
policy for not including interface code for third party programs which
are not part of the operating system. (I.e. the modules in the standard
libary should all be usable for anyone with a default OS + Python install..)

A notable exception is the dbm modules, but I seem to recall hearing
that the official position is that it was a mistake. (Now only kept for
backward compatability.)
--
http://mail.python.org/mailman/listinfo/python-list

[Tom Anderson]

On Fri, 1 Jul 2005, Scott David Daniels wrote:

Daniel Dittmar wrote:

Rocco Moretti wrote:

> Except that (please correct me if I'm wrong) there is somewhat of a
> policy for not including interface code for third party programs
> which are not part of the operating system. (I.e. the modules in the
> standard libary should all be usable for anyone with a default OS +
> Python install.)
There seems to be a great reluctance by the Python developers to add
modules of the expat kind, as this means responsibilities for
additional source modules. There's also the problem with incompatible
licenses, integrating a second configure, deciding when to update to
the latest version of the library etc.

If you haven't noticed, the Python code has a substantial body of unit
tests. Arranging the tests to be easily runnable for all developers
is going to be tough for "third party programs."

The tests for interface modules would have to use mock objects on the back
end. This is pretty standard practice, isn't it?
Making the interfaces work for differing versions of the 3PPs as the
third parties themselves change their interfaces (see fun with Tcl/Tk
versions for example), and building testbeds to test to all of those
differing versions, would cause a nightmare that would make a knight of
Ni scream.

But given that at a number of such modules have in fact been written,
along with tests, why not add them to the standard distribution?

tom

--
REMOVE AND DESTROY

[Colin J. Williams]

Rocco Moretti wrote:

Paul Rubin wrote:

Rocco Moretti <ro**********@hotpop.com> writes:

Except that (please correct me if I'm wrong) there is somewhat of a
policy for not including interface code for third party programs which
are not part of the operating system. (I.e. the modules in the
standard libary should all be usable for anyone with a default OS +
Python install.)

I've never heard of Python having such a policy and I don't understand
how such a stupid policy could be considered compatible with a
proclaimed "batteries included" philosophy. Why would Python
advocates want to make Python deliberately uncompetitive with PHP,
Java, and other languages that do include database modules?

Well, since there seems to be an outpouring of disgust at my statement,
and no official confirmation/rejection, it's probably a figment of my
prematurely failing mind.

However, if there was such a policy, it would not be unequivocally
"stupid." First off, there is a bit of flexibility in what is considered
part of the OS. E.g, Linux may properly refer to just the kernel, but
rarely is just the kernel installed. Various utilities and programs
might be considered part of the OS because they are ubiquitously
installed, or are included with the Python distribution itself (as Tk is
with windows Python).

For those programs which aren't ubiquitously installed, or even for ones
that are, but require significant configuration, it is reasonable to
expect that if someone has the ability and goes to the effort of
locating, obtaining, installing, and configuring a third party program,
they can just as easily obtain and install the python module, especially
as it's usually as easy as "python setup.py install".

At any rate, I'm not advocating such a policy, I'm just saying it can
make a bit of sense if you look at it from a certain angle.

I agree. It makes sense to develop a policy first.

Colin W.

[Terry Reedy]

"Colin J. Williams" <cj*@sympatico.ca> wrote in message
news:ga*********************@news20.bellglobal.com ...

Isn't this where the discussion should start? There should be some
general policy guiding the types of modules which should be in the
standard library.

A couple of times, Guido has given his general policy as generally useful;
best-of-breed, tested and accepted by the community; and backed by a
developer who will adapt it and its doc up to stdlib standards (if
necessary) and commit to maintainence for a few years.

Terry J. Reedy

[Terry Reedy]

"Colin J. Williams" <cj*@sympatico.ca> wrote in message
news:Iv*****************@news20.bellglobal.com...

Terry Reedy wrote:

A couple of times, Guido has given his general policy as generally
useful;
best-of-breed, tested and accepted by the community; and backed by a
developer who will adapt it and its doc up to stdlib standards (if
necessary) and commit to maintainence for a few years.

This is a good base. Presumably "accepted by the community" means
with some minimum number of ongoing users.

Yes. Users also indicate practical (versus theoretical) usefullness and
also test beyond what a developer might (especially documentation).

Of course, modules written by the inner core group of active developers or
on their behalf may bypass these criteria, but the context of my answer was
discussion of inclusion of existing modules mostly or all written by other
people.

Part of the maintainence requirement is a willingness to work compatibly
with the Python release cycle. Related to that is something I left out on
the other side: willingness to license the module to the PSF for inclusion.
Some people seem to assume that everyone 'of course' wants their stuff
included and that it is just Guido blocking inclusion, but that just is not
so.

Terry J. Reedy