What's missing from python?

[John Burton]

One of the reasons I like python so much is that the standard
installation comes with a whole lot of libraries to do lots of useful
things but there seem to be a few things missing from the standard
library which would be useful in many projects:

Encryption - AES and 3DES. Possibly RSA and Certificate processing.
DNS - Something better than gethostbyaddr for looking up other record types.
Database - Something suitable for small projects / prototypes like
pysqlite or something in pure python even.
SSL/TLS - Some useful support...

A lot of projects seem to need some of those. It's easy enough to
install extras to do those thing but they are all things where it would
be nice to have something in the standard library that you could rely on
being there.

Is there any reason why those things are not already in the standard
library?

[Peter Hansen]

John Burton wrote:

One of the reasons I like python so much is that the standard
installation comes with a whole lot of libraries to do lots of useful
things but there seem to be a few things missing from the standard
library which would be useful in many projects:

Encryption - AES and 3DES. Possibly RSA and Certificate processing.
DNS - Something better than gethostbyaddr for looking up other record
types.
Database - Something suitable for small projects / prototypes like
pysqlite or something in pure python even.
SSL/TLS - Some useful support...

A lot of projects seem to need some of those. It's easy enough to
install extras to do those thing but they are all things where it would
be nice to have something in the standard library that you could rely on
being there.

Is there any reason why those things are not already in the standard
library?

Yes, they aren't mature enough to be in there yet (because the work
is not complete, probably because not enough people are helping).

That's at least a partial answer. Were I actually one who helped
with the core work on these things myself, I would doubtless add
"contributions are welcome!" :-)

-Peter

[Scott David Daniels]

John Burton wrote:

One of the reasons I like python so much is that the standard
installation comes with a whole lot of libraries to do lots of useful
things but there seem to be a few things missing from the standard
library which would be useful in many projects:

Encryption - AES and 3DES. Possibly RSA and Certificate processing. This almost certainly not go in. You can get into a big legal hassle
when you ship encryption binaries around, as some governments tend to
think of them as "munitions" and regulate traffic in them. Python
just wants to avoid that hassle, and so leaves it up to you to put it
together. DNS - Something better than gethostbyaddr for looking up other record
types. Hmmm -- I personally have no need, but whatever floats your boat.
The general rule is to build a module, get people using it, and if
there is both enough interest and a maintainer, propose its addition.
Database - Something suitable for small projects / prototypes like
pysqlite or something in pure python even. I'd personally use such a thing, but perhaps it would be a big
chunk of code. It would need its own maintainer. Is pysqlite
sufficiently unencumbered by license? SSL/TLS - Some useful support... If you can figure out how to finesse the encryption issue, I'm sure
something in this area might be a go.
Is there any reason why those things are not already in the standard
library?

Ah -- a volunteer to build and support -- I like that.

--
-Scott David Daniels
Sc***********@Acm.Org

[Stefan Axelsson]

Scott David Daniels wrote:

This almost certainly not go in. You can get into a big legal hassle
when you ship encryption binaries around, as some governments tend to
think of them as "munitions" and regulate traffic in them. Python
just wants to avoid that hassle, and so leaves it up to you to put it
together.

Actually with the Wassenaar agreement its not that bad anylonger, with
an "open source", i.e. freely available source implementation, which
would cover C Python quite nicely I think.

The only countries that would be excluded would more or less be excluded
(it's a gray area) from importing Python anyway, since it's 'American
technology'. Google the Wassenaar agreement.

And IMHO the world in this day and age actually needs more free high
quality crypto implementations. I'd hate to see crypto not being made
available because of some (more or less) irrational fear that one might
step on someones toes. "Damn the torpedoes, full speed ahead". Other
software (likewise exported from the US) contains crypto implementations
these days, so I see no reason that Python couldn't.

Stefan,
--
Stefan Axelsson (http://www.cs.chalmers.se/~sax)

[Peter Hickman]

John Burton wrote:

One of the reasons I like python so much is that the standard
installation comes with a whole lot of libraries to do lots of useful
things but there seem to be a few things missing from the standard
library which would be useful in many projects:
It depends on what you regard as a 'standard library'. For me it is the
minimum required to use the language in a general sense. The smallest
subset of all the possible modules. Thus access to the file system is a
requirement but Encryption, DNS, Database, etc are not.

I can use Python without them but take away access to the file system an
Python becomes less useful. Besides do we really want a humungous
standard library, a al Java, or something you could actually get your
head around.
Is there any reason why those things are not already in the standard
library?

Gadfly is a useful SQL database in Python and I am sure that the
developer(s) are fully committed to supporting it into the future.
However if it becomes part of the standard library then this would
impose an additional workload / strain of the developers.

Unless the code is perfect it will require maintainance and testing as
Python grows. As the standard library grows the amount of testing grows
with it, and it is probably not linear.

This alone would be a reason to keep the standard library small.

[Paul Rubin]

Peter Hickman <pe***@semantico.com> writes:

It depends on what you regard as a 'standard library'. For me it is
the minimum required to use the language in a general sense. The
smallest subset of all the possible modules. Thus access to the file
system is a requirement but Encryption, DNS, Database, etc are not.
Python advertises a "batteries included" approach to the library which
means it's supposed to include most of the stuff that users need,
instead of making them download the stuff from random places.
Unless the code is perfect it will require maintainance and testing as
Python grows. As the standard library grows the amount of testing
grows with it, and it is probably not linear.

This alone would be a reason to keep the standard library small.

Python aims to be competitive with other languages and implementations
including (e.g.) Java, which come with large libraries. Unless it
wants to retreat from aiming for the big time, it needs to also have
thorough library coverage, even if that means more work.

[Peter Hickman]

Paul Rubin wrote:

Python advertises a "batteries included" approach to the library which
means it's supposed to include most of the stuff that users need,
instead of making them download the stuff from random places.
The 'batteries included' stuff is more a distribution / packaging issue
than the standard library. At least coming from Tcl where we have a
'batteries included' distribution including all sorts of nice things.
However this does not mean that they are part of the standard library.

Tcl works find without an XML parser, but the 'batteries included'
distribution has an XML parser which makes the distribution much more
convenient.
Python aims to be competitive with other languages and implementations
including (e.g.) Java, which come with large libraries. Unless it
wants to retreat from aiming for the big time, it needs to also have
thorough library coverage, even if that means more work.

Simple feature / library count as a measure of progress is flawed. Big
time need not equate with bloat, besides Perl has a massive library of
modules (CPAN) but they are not part of the standard library and this
has not held Perl back.

We need to recognize the divide between language features and a basic
distribution. A 'batteries included' distribution of Python, a la Tcl,
which includes a selection of useful libraries is a good idea but there
is no need to make it part of the standard library.

I want my Python in a Nutshell to be a single volume not an encyclopedia.

[Skip Montanaro]

Peter> Paul Rubin wrote:

Python advertises a "batteries included" approach to the library
which means it's supposed to include most of the stuff that users
need, instead of making them download the stuff from random places.

Peter> The 'batteries included' stuff is more a distribution / packaging
Peter> issue than the standard library. At least coming from Tcl where
Peter> we have a 'batteries included' distribution including all sorts
Peter> of nice things. However this does not mean that they are part of
Peter> the standard library.

See:

http://www.tcl.tk/cgi-bin/tct/tip/12.html

and search for "Frank Stajano". My guess would be that Frank's paper got
the ball rolling for the "batteries included" concept in Tcl.

Skip

[Cameron Laird]

In article <ma************************************@python.org >,
Skip Montanaro <sk**@pobox.com> wrote:

Peter> Paul Rubin wrote:

>> Python advertises a "batteries included" approach to the library
>> which means it's supposed to include most of the stuff that users
>> need, instead of making them download the stuff from random places.

Peter> The 'batteries included' stuff is more a distribution / packaging
Peter> issue than the standard library. At least coming from Tcl where
Peter> we have a 'batteries included' distribution including all sorts
Peter> of nice things. However this does not mean that they are part of
Peter> the standard library.

See:

http://www.tcl.tk/cgi-bin/tct/tip/12.html

and search for "Frank Stajano". My guess would be that Frank's paper got
the ball rolling for the "batteries included" concept in Tcl.

Skip

I confirm that. That's verifiably how it happened.

I also don't see that as contradicting what Peter wrote.
--

Cameron Laird <cl****@phaseit.net>
Business: http://www.Phaseit.net

[Skip Montanaro]

Peter> The 'batteries included' stuff is more a distribution / packaging
Peter> issue than the standard library. At least coming from Tcl where
Peter> we have a 'batteries included' distribution including all sorts
Peter> of nice things. However this does not mean that they are part of
Peter> the standard library.

See:

http://www.tcl.tk/cgi-bin/tct/tip/12.html

and search for "Frank Stajano". My guess would be that Frank's paper
got the ball rolling for the "batteries included" concept in Tcl.

Cameron> I confirm that. That's verifiably how it happened.

Cameron> I also don't see that as contradicting what Peter wrote.

Peter said, "The 'batteries included' stuff is more a distribution /
packaging issue than the standard library." As I recall from Frank's talk
and the quote in the above URL, Frank was definitely using "batteries
included" to mean the stuff that was included as part of the standard
library.

Python's batteries have met my needs quite well. Refreshing my memory with
my site-packages directory, I can think of only a few things that I've
gotten from elsewhere over the years:

* timeout_socket - now supplanted by socket's own timeout feature

* PIL at one point where I was messing around with some simple image
processing

* mxDateTime - now supplanted for my meager needs by datetime

* Zope, Quixote, WebWare - all really more than you'd need/want in the
standard distribution I think, and all really cross the boundary
between library and application

* Object Craft's csv module (and a couple other csv modules) - now
supplanted by the csv module

* xmlrpclib - now in the core distribution

* psyco

* psycopg and MySQLdb

* TextFile

Of those, I don't really expect the web app framework stuff to turn up in
the core (too many to choose from, no "one best way", too big), PIL's
probably too specialized for the core, psyco is still largely experimental
(though once it stabilizes and supports at least one non-Intel CPU I think
it should be considered), database connectors probably don't belong in the
core because there are too many databases to connect to, and TextFile
(transparently gzip files on output) could have been written easily enough
had I not stumbled upon it first.

Skip

[Paul Rubin]

Peter Hickman <pe***@semantico.com> writes:

Python advertises a "batteries included" approach to the library which
means it's supposed to include most of the stuff that users need,
instead of making them download the stuff from random places.
The 'batteries included' stuff is more a distribution / packaging
issue than the standard library. At least coming from Tcl where we
have a 'batteries included' distribution including all sorts of nice
things. However this does not mean that they are part of the standard
library.

Well, the contents of the standard library certainly sounds like a
distribution/packaging issue to me.
Simple feature / library count as a measure of progress is flawed. Big
time need not equate with bloat, besides Perl has a massive library of
modules (CPAN) but they are not part of the standard library and this
has not held Perl back.
When there's enough stuff written for Python, maybe it will also get
something like CPAN. However, CPAN is somewhat of a pain in the neck
compared to just including the needed stuff. People do ask all the
time for a super-sized Perl distribution that includes everything from
CPAN, or includes lots more CPAN stuff than the standard distro does,
or even for a simple CD-ROM dump of the CPAN contents.
We need to recognize the divide between language features and a basic
distribution. A 'batteries included' distribution of Python, a la Tcl,
which includes a selection of useful libraries is a good idea but
there is no need to make it part of the standard library.

I want my Python in a Nutshell to be a single volume not an encyclopedia.

For sure, there's some attractiveness to making three separate
distros: minimal, normal (like the current one), and encyclopedic.
However, that's a lot of work, so the current strategy seems to be to
have one distro that strikes a reasonable balance between minimal and
encyclopedic, providing enough stuff to meet the needs of application
implementers in Python's targeted areas. Right now there are
significant gaps in the library but with luck that will improve over
time.

[John Burton]

John Burton wrote:

One of the reasons I like python so much is that the standard
installation comes with a whole lot of libraries to do lots of useful
things but there seem to be a few things missing from the standard
library which would be useful in many projects:

Encryption - AES and 3DES. Possibly RSA and Certificate processing.
DNS - Something better than gethostbyaddr for looking up other record
types.
Database - Something suitable for small projects / prototypes like
pysqlite or something in pure python even.
SSL/TLS - Some useful support...

A lot of projects seem to need some of those. It's easy enough to
install extras to do those thing but they are all things where it would
be nice to have something in the standard library that you could rely on
being there.

Is there any reason why those things are not already in the standard
library?

Thanks for all the comments.
The reason I chose those particular things was that they were things
that a fair number of programs seem to need but which there is no real
reason to need more than one implementation so it seems sensible to
include it. Obviously you can't put everything in.

Maybe the database thing doesn't qualify for that but it would be
*really* useful to be able to write tools and utilities that you could
put on a random manchine with python and they'd have some kind of
database available

[Cameron Laird]

In article <89*************************@posting.google.com> ,
Anthony_Barker <an************@hotmail.com> wrote:

> Encryption - AES and 3DES. Possibly RSA and Certificate processing.
> DNS - Something better than gethostbyaddr for looking up other record
> types.
> Database - Something suitable for small projects / prototypes like
> pysqlite or something in pure python even.
> SSL/TLS - Some useful support...

I agree and would add the ldap module.

My feeling is that once a module is included in the standard library
the level of documentation and testing improves significantly. I've
always prefered the python way of including libraries over perl.

If the distro is too large you could always modify the installer to
allow users to choose which modules to include(like ms office does
with apps).

Heh-heh: "you could always ..." Installers can be
bigger investments than whole collections of modules.

I am fond of LDAP, by the way.
--

Cameron Laird <cl****@phaseit.net>
Business: http://www.Phaseit.net