Documentation suggestions

[A.M. Kuchling]

Here are some thoughts on reorganizing Python's documentation, with
one big suggestion.

The tutorial seems to be in pretty good shape because Raymond
Hettinger has been keeping it up to date. It doesn't cover
everything, but it's a solid introduction, and if people don't find it
works for them, they have lots of alternative books. No suggestions
here.

There are endless minor bugs in the library reference, but that seems
unavoidable. It documents many different and shifting modules, and
what to document is itself a contentious issue, so I don't think the
stream of small problems will ever cease.

There's another struggle within the LibRef: is it a reference or a
tutorial? Does it list methods in alphabetical order so you can look
them up, or does it list them in a pedagogically useful order? I
think it has to be a reference; if each section were to be a tutorial,
the manual would be huge. Here I think the solution is to encourage
separate tutorials and HOWTOs, and link to them from the LibRef.

The library reference has so many modules that the table of contents
is very large. Again, not really a problem that we can fix; splitting
it up into separate manuals doesn't seem like it would help.

The Extending/Embedding manual isn't great, but only advanced users
will come in contact with it, so I don't think it's a critical factor.
(Michael Hudson's new manual is a promising replacement.)

I suspect the Achilles' heel of the docs is the Language Reference.
Put aside the fact that it's not up to date with new-style classes and
other stuff; that would be fixable with some effort.

To some degree, the guide is trying to be very formal; it's written
like a specification for an implementor, not a document that people
would read through. But there's no other way for people to learn
about all the special object methods like __add__; the tutorial can't
cover them all, and the LibRef doesn't describe them. So the newbie
is stuck.

For example, I noticed this passing reference in Tim Bray's weblog:
http://www.tbray.org/ongoing/When/200x/2005/08/27/Ruby

Based on first impressions and light exposure (a basis that
matters a lot) Ruby seems better-documented and easier to get
into than Python. I've actually written (a little)
production code in Python, but I always had the feeling that
there was lots of stuff going on I didn't understand; a
couple of days in, I think I have a better grasp on what
Ruby's up to, even where I'm not looking.

I don't know exactly what Bray meant, but suspect that a more readable
reference guide would have helped him understand what was going on.

Perhaps we need a friendlier counterpart to the RefGuide, something
like the 20-page introduction to Python at the beginning of Beazley's
Essential Reference:

* go over the statements one-by-one
* go over the basic types and their methods
* go over object semantics
* cover some of the lexical material in chapter 2 of the RefGuide
* overarching principles: go into a fair bit of detail, but
not every corner case; make the text readable, not meticulously
precise.

(I should point out: this suggestion is not entirely different from
some of the suggestions that X*h L** has made. He (?) makes his
suggestions in a venomous style, and with his own eccentric
terminology, but that doesn't mean he's always wrong, e.g. his
complaints about the re module's documentation are well-placed in the
main.)

One problem with such a friendly document: it might make the Ref Guide
even more irrelevant, if we always updated the friendly document
(which is easy) and left the RefGuide to drift even further out of
date (because it's hard to update). I don't know if this is an
argument for not having a friendly guide, or for dumping the RefGuide
entirely.

Dumping the RefGuide means there isn't a more formal-style description
of Python's semantics. I don't know if this matters. In theory, the
implementors of Jython or IronPython could be using the RefGuide to
know what they need to implement, but in practice I suspect
implementors use the test suite and existing library as checks. Maybe
we don't really need a tediously precise description of Python.

What do people think?

--amk

[Ben Sizer]

A.M. Kuchling wrote:

The tutorial seems to be in pretty good shape because Raymond
Hettinger has been keeping it up to date. It doesn't cover
everything, but it's a solid introduction, and if people don't find it
works for them, they have lots of alternative books. No suggestions
here.
The last time I looked at it, I thought the tutorial was the weakest
part of the documentation, because it assumed you were coming at Python
from another language, and often assumed that language was C. I mean,
the very first line says, "If you ever wrote a large shell script"...
which most people never have. The second page assumes you're using Unix
and throws in Windows as an afterthought. The third page demonstrates
string literals using the term "just as you would do in C". This is a
bit pointless because anyone familiar with C is probably bright enough
to make the connection for themselves, and anyone who isn't will not
appreciate what is being explained. This sort of theme continues
throughout.

Once upon a time I expect nearly all Python programmers came from C,
but today, I doubt that is true, especially since most universities
seem to prefer teaching Java over C++. In short, the tutorial is quite
outdated for today's prospective Python programmers. Admission: I once
offered to help clean it up, but by the time I got a reply from
do**@python.org, I had started a new job and no longer had time. Sorry.

As for the alternative books comment, I think this is very wrong...
people are surely far more likely to buy a comprehensive reference to a
language they're already hooked on by a good tutorial, than they are to
buy an entry-level text for a language they don't quite understand but
which seems to boast good reference material!
There are endless minor bugs in the library reference, but that seems
unavoidable. It documents many different and shifting modules, and
what to document is itself a contentious issue, so I don't think the
stream of small problems will ever cease.
Make the docs like PHP's docs. Users can post addendums and corrections
to the documentation, then all one of the official maintainers has to
do is fold in those corrections later.
There's another struggle within the LibRef: is it a reference or a
tutorial? Does it list methods in alphabetical order so you can look
them up, or does it list them in a pedagogically useful order? I
think it has to be a reference; if each section were to be a tutorial,
the manual would be huge.
I agree: reference foremost. However, all but the most trivial
libraries should have a short example, preferably on the first page of
their docs, so that you can quickly get an idea of whether this does
what you want it to do, and how the code is going to look. Some
libraries don't (ConfigParser comes to mind, Threading is another)
which make using them a bit of a chore to begin with. Again, if web
users could contribute examples as comments on the docs as with PHP,
they can be added in later.
The library reference has so many modules that the table of contents
is very large. Again, not really a problem that we can fix; splitting
it up into separate manuals doesn't seem like it would help.

Personally I'd be happy to see a lot of those modules removed from the
distribution, but I expect few will agree with me. ;)

--
Ben Sizer

[Ian Bicking]

A.M. Kuchling wrote:

Here are some thoughts on reorganizing Python's documentation, with
one big suggestion.
Thanks for bringing this up...
There are endless minor bugs in the library reference, but that seems
unavoidable. It documents many different and shifting modules, and
what to document is itself a contentious issue, so I don't think the
stream of small problems will ever cease.
Since the topic of php.net-style comments comes up often, I thought I'd
note I've been working on a comment system:
http://pythonpaste.org/comment/commentary/ -- this might be useful for
collecting and managing small updates and fixes to the documentation.
There's another struggle within the LibRef: is it a reference or a
tutorial? Does it list methods in alphabetical order so you can look
them up, or does it list them in a pedagogically useful order? I
think it has to be a reference; if each section were to be a tutorial,
the manual would be huge. Here I think the solution is to encourage
separate tutorials and HOWTOs, and link to them from the LibRef.
We don't have a clear place to put that material. People's personal
pages leave the potential for material disappearing, and since that
stuff usually doesn't have a clear license we can't necessarily be sure
we can move it elsewhere if the original disappears or becomes out of
date. The Wiki would probably work, and maybe we should just say that
contributions on the Wiki are the preferred form for tutorials and
howtos. Then, of course, we have to link to them. With a commenting
system that can happen more organically.
The library reference has so many modules that the table of contents
is very large. Again, not really a problem that we can fix; splitting
it up into separate manuals doesn't seem like it would help.
There's a lot of dumb modules in there. I'd love if there was a module
index that excluded modules that were not useful. Like, say,
MimeWriter or flp. I think alphabetical or categorized doesn't matter
that much, compared to just a shorter list. Or maybe categorized, but
layed out so that it still all fits on one screen; the main library ToC
is simply too long because it doesn't fit on a screen, but the
categories can still be useful.
I suspect the Achilles' heel of the docs is the Language Reference.
Put aside the fact that it's not up to date with new-style classes and
other stuff; that would be fixable with some effort.

To some degree, the guide is trying to be very formal; it's written
like a specification for an implementor, not a document that people
would read through. But there's no other way for people to learn
about all the special object methods like __add__; the tutorial can't
cover them all, and the LibRef doesn't describe them. So the newbie
is stuck.
Definitely; there's a problem for a newcomer, where the "language" is
documented one place (and hard to find), and the standard library
someplace else. As a newcomer it can be hard to know which is which.
Also, things like the functions in builtins are particularly hard to
find IMHO. I usually use pydoc for these things now, but not because I
particularly want to.
Perhaps we need a friendlier counterpart to the RefGuide, something
like the 20-page introduction to Python at the beginning of Beazley's
Essential Reference:

* go over the statements one-by-one
* go over the basic types and their methods
* go over object semantics
* cover some of the lexical material in chapter 2 of the RefGuide
* overarching principles: go into a fair bit of detail, but
not every corner case; make the text readable, not meticulously
precise.
I think that an example-based reference could be more useful than the
current layout. The corner cases *are* important, and it's valuable to
be able to read documentation to fully understand one particular
feature. Like, say, exceptions (which are undercovered right now).
But it can be easier to understand those with a series of examples
meant to demonstrate the behavior, than with a formal description.
One problem with such a friendly document: it might make the Ref Guide
even more irrelevant, if we always updated the friendly document
(which is easy) and left the RefGuide to drift even further out of
date (because it's hard to update). I don't know if this is an
argument for not having a friendly guide, or for dumping the RefGuide
entirely.

Dumping the RefGuide means there isn't a more formal-style description
of Python's semantics. I don't know if this matters. In theory, the
implementors of Jython or IronPython could be using the RefGuide to
know what they need to implement, but in practice I suspect
implementors use the test suite and existing library as checks. Maybe
we don't really need a tediously precise description of Python.

A test suite seems far more useful to implementors than any guide,
especially where the accuracy of that guide is not completely assured.
A series of examples seems more concrete than a formal description, and
as such can be not just more helpful, but also a more accurate in how
it transfers knowledge.

[D H]

Ian Bicking wrote:

There are endless minor bugs in the library reference, but that seems
unavoidable . It documents many different and shifting modules, and
what to document is itself a contentious issue, so I don't think the
stream of small problems will ever cease.

Since the topic of php.net-style comments comes up often, I thought I'd
note I've been working on a comment system:
http://pythonpaste.org/comment/commentary/ -- this might be useful for
collecting and managing small updates and fixes to the documentation.

Yes, do it the way php's documentation is done, Xah Lee's recent trolls
aside.

[skip]

Ben> Make the docs like PHP's docs.

Easier said than done. There is still - I think - a project in the works to
redo the python.org website, but I have no idea if it means to retain
ht2html+latex2h tml as the page builders or move to something else. Neither
ht2html nor latex2html support in inline annotation feature. It would need
to be added.

What was the comment here the other day? Python has more web application
frameworks than keywords. PHP only has one. ;-)

Skip

[skip]

Ian> A test suite seems far more useful to implementors than any guide,

Of course, test cases can be modified or ignored. I'd agree with you if we
had a test suite that was more strongly cast in stone.

Skip

[Fredrik Lundh]

Ian Bicking wrote:

There's another struggle within the LibRef: is it a reference or a
tutorial? Does it list methods in alphabetical order so you can look
them up, or does it list them in a pedagogically useful order? I
think it has to be a reference; if each section were to be a tutorial,
the manual would be huge. Here I think the solution is to encourage
separate tutorials and HOWTOs, and link to them from the LibRef.

We don't have a clear place to put that material. People's personal
pages leave the potential for material disappearing

so ? this "if we don't own it, we won't even pretend it exists" attitude
of the PSF is beginning to get a bit boring.

(the standard library suffers from the same problem)

</F>

[BartlebyScrivener]

>> A series of examples seems more concrete than a formal description,<<

Amen. This is why people buy the books: The good ones have lots of
examples. The wizards glance at them in passing and think, "Duh." And
the rest of us (including the intermediate folks, I'll bet) are
grateful for the chance to see a real live example.

rd

[Ian Bicking]

Fredrik Lundh wrote:

There's another struggle within the LibRef: is it a reference or a
tutorial? Does it list methods in alphabetical order so you can look
them up, or does it list them in a pedagogically useful order? I
think it has to be a reference; if each section were to be a tutorial,
the manual would be huge. Here I think the solution is to encourage
separate tutorials and HOWTOs, and link to them from the LibRef.

We don't have a clear place to put that material. People's personal
pages leave the potential for material disappearing

so ? this "if we don't own it, we won't even pretend it exists" attitude
of the PSF is beginning to get a bit boring.

Well, I ain't the PSF, so my comment here doesn't really indicate
anything with respect to that. There are already links in the
reference document to external documents. But if we start getting a
half dozen links for each page (which is not an unreasonable number)
the maintenance of trimming dead links and pages that are out of date
becomes more significant. And there's no particular coordination, or
any simple feedback process that can be applied to all content.

I don't think there should be any policy discouraging external links;
there's a lot of sources like the Python Cookbook, some free online
books (e.g., Dive Into Python, or Think Like A Computer Scientist), and
stable personal pages that should be linked in . But I do think that
we should encourage some specific process for new or revised
tutorial/howto contributions, like encouraging people put such material
in the wiki.

Ian

[skip]

A series of examples seems more concrete than a formal description,<<

rd> Amen. This is why people buy the books: The good ones have lots of
rd> examples. The wizards glance at them in passing and think, "Duh."
rd> And the rest of us (including the intermediate folks, I'll bet) are
rd> grateful for the chance to see a real live example.

ISTR that comment was made in reference to the Language Reference, which as
written is intended for "language lawyers". Examples there seem
counterproducti ve.

Skip