Tried Ruby (or, "what Python *really* needs" or "perldoc!")

[john_sips_tea]

Just tried Ruby over the past two days. I won't bore you
with the reasons I didn't like it, however one thing really
struck me about it that I think we (the Python community)
can learn from.

Ruby has ... an issue with docs. That is to say, there are
almost none. Well, actually, there are some. For example,
the "PickAxe" book (google it), and "Why's" Poignant Guide.
But there's a disturbing lack of *built-in* docs for Ruby.
Now, the situation is getting better. In fact, it's getting
better very quickly. Let me explain.

IMO, Perl has docs nailed. I learned Perl before coming
to Python, and I can tell you that their docs kick butt.
I believe the reason why is (besides Larry's excellent
and entertaining writing) because of perldoc. Here's how
it works: they write special doc directives into their .pl
files that the interpreter ignores, but that the perldoc
command processes into a manpage for you. This is a *bit*
like what pydoc does, only pydoc just regurgitates docstrings,
while perldoc formats special directives into headings, code
listings, bulleted lists, etc.

Now, the *real* magic of perldoc is that it makes writing
small self-contained little manpage style docs easy as pie.
What this means is that really good doc writers can sit down
and quickly produce something that can be readily incorporated
into the perl-doc distribution. *That's* the magic. That's
why I think they've organically grown such an amazing crop
of great docs. Anyone can quickly and easily write docs that
the community can then filter to find the best ones.

Back to Ruby's docs. Their situation is getting better. This
is because they're picking up on Perl's doc success and doing
the same sort of thing. Only, I think their solution may be
clunkier because it requires a separate program (rdoc) to
process the doc directives, and then you actually read the
docs with a program called ri (not sure). Regardless of the
minute details, it looks like their docs are starting to get
better, rapidly.

Here's what I think is currently happening with Python
(please correct me if I'm wrong): There's the docutils project
http://docutils.sourceforge.net/ which, AFAICT, is working on a
number of complicated things, one of which is to have a way to
allow you to put reStructuredText markup (aka "reST")
( http://docutils.sourceforge.net/rst.html ) into your docstrings
(and presumably have the pydoc command understand it). There's
more info in PEP 287 http://www.python.org/doc/peps/pep-0287/ .

I certainly don't understand all that Docutils is trying to
do. All I'm convinced of is, the real magic is in being able to
quickly write a .py file containing marked up docstrings and
have the pydoc command be able to render it as something that
looks like a man page in my terminal window. If it can later
also produce html, that's great too. But *that's* the essense,
IMO, of what will foster community involvement in making the
docs great, as well as allowing folks to more easily document
their own modules and contribute integrated docs to all the
great modules already available.

It looks like we have the tools do this *right now*. We've got
the markup (reST), we've got the tools to churn that into a
manpage or html (part of docutils?), and we've got the pydoc
command. I think the current hangup is that the docutils guys
(being smart and ambitious folks) want to get some sort of
inter-doc linking thing working so you can refer from one doc
page to another (?). I don't think perldoc has (or needs) that
feature... maybe we could put that on the "would be nice to
have down the road" list, and get a quick common-sense
docstring-reST pydoc setup working for us much sooner? I
don't know for sure, but my guess is "yes".

Eventually, maybe the tutorial, language reference, library ref,
etc., could even all make it into this format, with the html
versions generated from the .py/docstring/reST sources. That might
make it easier for the community to contribute to them (if their
respective maintainers are interested in that).

Please note, I certainly don't want to step on the doc-sig folks'
toes here -- but rather to generate more interest in what they're
doing, and to help make Python be even better in an area that I
see it struggling.

What do you folks think?

Yes, I'm trying to make time to look at the docutils code and the
pydoc command to see what's involved. Unfortunately, my spare
time is vanishingly close to zero right now.

Related link:
Doc-SIG http://www.python.org/community/sigs/current/doc-sig/

Thanks,
---John

[Diez B. Roggisch]

> Please note, I certainly don't want to step on the doc-sig folks'

toes here -- but rather to generate more interest in what they're
doing, and to help make Python be even better in an area that I
see it struggling.

What do you folks think?

Yes, I'm trying to make time to look at the docutils code and the
pydoc command to see what's involved. Unfortunately, my spare
time is vanishingly close to zero right now.

You heard of epydoc? http://epydoc.sourceforge.net/

It pretty much does what you say I think - and for my personal projects I
use it. Maybe we an adopt it as standard-tool.

Diez

[john_sips_tea]

Thanks Diez! Epydoc looks great.

Can we use epytext to generate output suitable for a manpage?

Do you prefer epytext or reST?

[Diez B. Roggisch]

> Thanks Diez! Epydoc looks great.

Can we use epytext to generate output suitable for a manpage?
Don't know, never tried that.
Do you prefer epytext or reST?

So far epytext suited my needs.

Diez

[john_sips_tea]

> So far epytext suited my needs.

I like it too.

Ok, now I'm starting to get excited. :)

[Colin J. Williams]

Diez B. Roggisch wrote:

Thanks Diez! Epydoc looks great.

Can we use epytext to generate output suitable for a manpage?

Don't know, never tried that.

The answer appear to be No, but there is a man.py file which indicates
that some work was done on it.

Below is the epydoc commandline guide.

Colin W.

epydoc [OPTIONS] MODULES...

MODULES... The Python modules to document.
--html Generate HTML output (default).
--latex Generate LaTeX output.
--pdf Generate pdf output, via LaTeX.
--check Run documentation completeness checks.
-o DIR, --output DIR The output directory.
-n NAME, --name NAME The documented project's name.
-u URL, --url URL The documented project's url.
-t PAGE, --top PAGE The top page for the HTML documentation.
-c SHEET, --css SHEET CSS stylesheet for HTML files.
--private-css SHEET CSS stylesheet for private objects.
--inheritance STYLE The format for showing inherited objects.
-V, --version Print the version of epydoc.
-h, -?, --help, --usage Display this usage message.
-h TOPIC, --help TOPIC Display information about TOPIC (docformat,
css, inheritance, usage, or version).

Do you prefer epytext or reST?

So far epytext suited my needs.

Diez

[john_sips_tea]

Ok. I'm going to try and make something happen. Give me a day or so.

:)
---John

[Nick Craig-Wood]

jo***********@yahoo.com <jo***********@yahoo.com> wrote:

IMO, Perl has docs nailed. I learned Perl before coming
to Python, and I can tell you that their docs kick butt.
I believe the reason why is (besides Larry's excellent
and entertaining writing) because of perldoc. Here's how
it works: they write special doc directives into their .pl
files that the interpreter ignores, but that the perldoc
command processes into a manpage for you. This is a *bit*
like what pydoc does, only pydoc just regurgitates docstrings,
while perldoc formats special directives into headings, code
listings, bulleted lists, etc.

As another perl refugee I agree with you 100% here.

"perldoc perltoc" then "perldoc xxxx" will find you anything in perl.

Its frustrating that pydoc is only half (or less) of the docs.

However having bedded into python for a few years, I now just reach
for my web browser for the global module index instead of pydoc. Its
not as good as perldoc as it doesn't cover all the modules I've got
installed, but its very good documentation. Some modules have good
pydoc-umentation but not all of them. Code examples tend to be
missing.

I think a major problem with our way of thinking about
perldoc/pydoc/man pages is that it is a) unix centric, and b) possibly
a bit old fashioned! a) and b) apply to perl certainly, but I don't
think they apply to python in the same way.

I'd love to have a unified documentation system where *all* the
documentation for *all* installed modules was available to pydoc *and*
the web browser and *all* this documentation was in .py files.

(Putting the code together with the documentation is essential in my
opinion and experience - if you seperate the two then the documention
will lag the code.)

PS I've used reST and perldoc. reST is easier to use for the easy
things, but gets complicated for the hard things.

--
Nick Craig-Wood <ni**@craig-wood.com> -- http://www.craig-wood.com/nick

[Terry Hancock]

On 14 Mar 2006 09:25:07 -0800
jo***********@yahoo.com wrote:

Do you prefer epytext or reST?

I personally prefer epytext primarily because it has support
for greek letters and math symbols. That could be very
useful in documenting certain kinds of software. OTOH, I
haven't had much occasion to use that feature (once upon a
time almost all the software I wrote was scientific, but it
seems I've pretty much given it up -- not that I
particularly planned to, but it's turned out that way).

I was under the impression that pydoc already interpreted
restructured text notation, but maybe I was wrong. I don't
like pydoc because it doesn't make useable static
documentation sets -- that's where epydoc (which I suppose
stands for "extended pydoc") shines.

Although it's great to have "manpage" type help, I
personally find HTML documentation much easier to read and
browse. Pydoc can do this if you are willing to use it as a
server, but it doesn't do so well at making an in-package
website, which is what I usually want to do.

There's also happydoc, which was better than either at
discovering documentation, but stagnated somewhere between
2.x and 3.x with bugs that make it fairly unusable. What
would be cool is if some of happydoc's unique features were
ported to epydoc (such as getting information from comments
as well as docstrings).

--
Terry Hancock (ha*****@AnansiSpaceworks.com)
Anansi Spaceworks http://www.AnansiSpaceworks.com

[aaronwmail-usenet]

I agree that more progress is needed on the Python documentation
front. For example if you look at the "codecs" module documentation
there is no hint of what a codec is anywhere that I can see. Also
the distinction between an "encoder" and a "decoder" is not explained.
Even though I've used it many times and understand it, I still find
myself using the interactive interpreter to make sure I'm sending the
bytes in the right direction...

Perhaps some faqwiz/wiki-like tool to allow the broader community to
propose documentation enhancements would be useful?

-- Aaron Watters

===

"I know what it's like to put food on my family." G.W.Bush

[john_sips_tea]

Well, we've already got a wiki, of course: http://wiki.python.org/moin/

Regarding the docs for the module you're asking about, the way it's
supposed to work is (I think), you're supposed to checkout the Python
source, add your docs to the docstrings of that module, then either
commit your changes back in (if you have access privileges) or else
send a patch to the right list (python-dev?) so your patch can be
integrated.

It seems to me that it's best to write your docstring as epytext. That
way, for now, it just looks like regular plain text (though neatly
formatted) when viewed with the pydoc tool...

[aaronwmail-usenet]

hmmm. Interesting about the wiki.
It's unusable in my version of IE. Javascript error
on almost every keystroke :(!

http://wiki.python.org/moin/

It works in Firefox, which I have, of course, but
still...

And the patch procedure you described requires
a higher degree of motivation (and free time) than
most potential contributors might have on offer, imo.

-- Aaron Watters

===
"I weep for you," the walrus said, I deeply sympathize.
With sobs and tears he sorted out those of the largest size.
Holding his pocket handkercheif before his streaming eyes.
from "The Walrus and the Carpenter", Lewis Carroll

[john_sips_tea]

> And the patch procedure you described requires

a higher degree of motivation (and free time) than
most potential contributors might have on offer, imo.

Another option is to simply email the author/maintainer
for a given module your modifications to their module.

cd ~/dev/python/modified_modules
cp /path/to/module_needs_docs.py .
vim ./module_needs_docs.py # add docs

[Paul Boddie]

jo***********@yahoo.com wrote:

[Quoting Aaron Watters - *the* Aaron Watters?!]

And the patch procedure you described requires
a higher degree of motivation (and free time) than
most potential contributors might have on offer, imo.

The patch procedure described seemed to involve mailing python-dev,
which I thought was not the accepted practice. Even making patch
descriptions in SourceForge won't apparently get patches straight into
Python, although there may be fewer restrictions on submitting
documentation patches.
Another option is to simply email the author/maintainer
for a given module your modifications to their module.

cd ~/dev/python/modified_modules
cp /path/to/module_needs_docs.py .
vim ./module_needs_docs.py # add docs

True, but I'm not convinced this is going to work when the modules are
in the standard library. Anyway, there were going to be some
improvements to the processes, last time I looked, and various editable
resources did spring up to meet the need for documentation edits - none
of them official as far as I know, though.

Paul

P.S. On the documentation-from-sources question, I've had some success
with epydoc. Some documentation systems (even those for
statically-typed languages) have you describe parameters in
mind-numbing detail, but all I would expect from a documentation tool
is that it highlight mentions of the parameter names in my docstrings.
Having some understanding of the nasty issues around deducing types, I
wouldn't expect a tool to fill in the gaps about parameter types: good
documentation for dynamic languages like Python should be able to
describe what kind of objects should be passed in, and type deduction
could actually give quite the wrong impression about acceptable
arguments to functions, although interface descriptions could be
helpful.

[Steve]

I have found the Python sidebar VERY helpful:
http://projects.edgewall.com/python-sidebar/

[msoulier]

While epydoc is nice, I'll point out that one thing that Unix people
like myself really like is to be able to check docs on a remote server
that we're logged into via a terminal session. The help() function in
the interpreter is great for this, although it seems that python eggs
broke it. :(

Please don't leave the terminal behind. Perl provides docs in all
formats, since if you can write POD (plain old documentation), you can
run perldoc, or run pod2html, or pod2man, etc.

I'll also point out that a lot of these complex solutions are ignoring
the KISS principle. POD takes about 5 minutes to learn, tops.

[msoulier]

> I'd love to have a unified documentation system where *all* the

documentation for *all* installed modules was available to pydoc *and*
the web browser and *all* this documentation was in .py files.

Seconded!

Mike

[msoulier]

> I have found the Python sidebar VERY helpful:

Personally, I can't use local docs on my desktop as they may not be the
same version of the docs for the Python distro running on the server
that I'm deploying on. I usually go to python.org and use the wayback
machine to look at the old docs for the release that I'm on.

But, if Python would match Perl for docs available on the command-line,
then I'd have it all at my fingertips. I simply don't understand why
this is not being done. When I'm coding in C, I use the manpages on the
remote host so that I know the docs are correct for my target. Why
can't I do that in Python? It's yet another thing that my Perl-using
coworkers point out as a Python weakness.

Mike

[Paul Boddie]

msoulier wrote:

But, if Python would match Perl for docs available on the command-line,
then I'd have it all at my fingertips. I simply don't understand why
this is not being done. When I'm coding in C, I use the manpages on the
remote host so that I know the docs are correct for my target. Why
can't I do that in Python?
What about pydoc?

% pydoc os

[Clear screen]

Help on module os:

NAME
os - OS routines for Mac, DOS, NT, or Posix depending on what
system we're on.

FILE
/usr/lib/python2.4/os.py

MODULE DOCS
http://www.python.org/doc/current/lib/module-os.html

DESCRIPTION

[...]
It's yet another thing that my Perl-using coworkers point out as a Python weakness.

In my experience, Perl-using coworkers tend to dredge up all sorts of
"weaknesses", although many of them are no more than cheap shots: the
name, whitespace, and so on. Sure, Python's documentation system could
be better, but let's consider the tools we already have first.

Paul

[Jason Earl]

"msoulier" <ms******@gmail.com> writes:

I have found the Python sidebar VERY helpful:
Personally, I can't use local docs on my desktop as they may not be
the same version of the docs for the Python distro running on the
server that I'm deploying on. I usually go to python.org and use the
wayback machine to look at the old docs for the release that I'm on.

Why don't you instead install the info version of the Python
documentation on your server. Then you can do "info Python2.3-lib"
and have at it. If you are hacking in emacs then this is about as
slick a documentation system as you could ask for, but even if you use
some other editor info is a much better documentation tool than man.
But, if Python would match Perl for docs available on the
command-line, then I'd have it all at my fingertips. I simply don't
understand why this is not being done. When I'm coding in C, I use
the manpages on the remote host so that I know the docs are correct
for my target. Why can't I do that in Python? It's yet another thing
that my Perl-using coworkers point out as a Python weakness.

Python does match (and exceed) Perl for docs available on the command
line. Once you get used to using the excellent info-based Python
documentation using man is downright primitive.

Jason

[john_sips_tea]

msoulier wrote:

[snip]
But, if Python would match Perl for docs available on the command-line,
then I'd have it all at my fingertips. I simply don't understand why
this is not being done. [snip]

Mike

Ok, well, here's my attempt to begin to make that happen:

http://www.simisen.com/jmg/cpd/

This "release" is as alpha as alpha gets. It's so alpha it
actually loops back around to zeta -- but it's a start, and I
think it's exactly what the Python community needs.

The command to read the docs works like pydoc but is
called dennis (extra points for getting the obscure Monty
Python reference before reading the web page). :) The dennis
command is only a few lines of Python held together by bailing
wire, so it will need lots of work, but it lets us read the docs
so it's ok for now. You're invited to fix it up and send me an
improved version. :)

Please have a look and let me know what you think. If people
start writing modules (that is, documentation) and sending
them to me, I'll probably be forced to start a mailing list and
actually put this stuff in CVS/SVN/bzr.

Have at it! :)

---John

[Marc 'BlackJack' Rintsch]

In <47************@uni-berlin.de>, Diez B. Roggisch wrote:

Yes, I'm trying to make time to look at the docutils code and the
pydoc command to see what's involved. Unfortunately, my spare
time is vanishingly close to zero right now.

You heard of epydoc? http://epydoc.sourceforge.net/

It pretty much does what you say I think - and for my personal projects I
use it. Maybe we an adopt it as standard-tool.

Epydoc seems to be "dead" and pudge not yet alive:

http://pudge.lesscode.org/

But it looks promising IMHO.

Ciao,
Marc 'BlackJack' Rintsch

[Nick Craig-Wood]

Jason Earl <je***@xmission.com> wrote:

Why don't you instead install the info version of the Python
documentation on your server. Then you can do "info Python2.3-lib"
and have at it.

I didn't know about that. Its very good.

Its still not all the documentation for all the installed modules
(a-la perltoc) but it is much better than I thought!

info on its own reveals these possible info packages for python

Python
* Python2.3-api: (python2.3-api). Python/C 2.3 API Reference Manual
* Python2.3-dist: (python2.3-dist). Distributing Python Modules (2.3)
* Python2.3-ext: (python2.3-ext). Extending & Embedding Python 2.3
* Python2.3-lib: (python2.3-lib). Python 2.3 Library Reference
* Python2.3-ref: (python2.3-ref). Python 2.3 Reference Manual
* Python2.3-tut: (python2.3-tut). Python 2.3 Tutorial

--
Nick Craig-Wood <ni**@craig-wood.com> -- http://www.craig-wood.com/nick

[Terry Hancock]

On Wed, 15 Mar 2006 23:10:16 +0100
"Marc 'BlackJack' Rintsch" <bj****@gmx.net> wrote:

In <47************@uni-berlin.de>, Diez B. Roggisch wrote:

Yes, I'm trying to make time to look at the docutils

code and the > pydoc command to see what's involved.
Unfortunately, my spare > time is vanishingly close to
zero right now.
You heard of epydoc? http://epydoc.sourceforge.net/

It pretty much does what you say I think - and for my
personal projects I use it. Maybe we an adopt it as
standard-tool.

Epydoc seems to be "dead" and pudge not yet alive:

What leads you to this conclusion? Works pretty well for
me. Maybe it's just "stable"?

--
Terry Hancock (ha*****@AnansiSpaceworks.com)
Anansi Spaceworks http://www.AnansiSpaceworks.com

[Steve Juranich]

jo***********@yahoo.com wrote:

This "release" is as alpha as alpha gets. It's so alpha it
actually loops back around to zeta -- but it's a start, and I
think it's exactly what the Python community needs.

Not to pick nits, but that should actually be "... so alpha that it actually
loops back around to *OMEGA*."

Cheers.

--
Steve Juranich
Tucson, AZ
USA

[john_sips_tea]

Jason Earl wrote:

"msoulier" <ms******@gmail.com> writes:

I have found the Python sidebar VERY helpful:

Personally, I can't use local docs on my desktop as they may not be
the same version of the docs for the Python distro running on the
server that I'm deploying on. I usually go to python.org and use the
wayback machine to look at the old docs for the release that I'm on.

Why don't you instead install the info version of the Python
documentation on your server. Then you can do "info Python2.3-lib"
and have at it. If you are hacking in emacs then this is about as
slick a documentation system as you could ask for, but even if you use
some other editor info is a much better documentation tool than man.

You know, I'm pretty used to reading man pages. There's only a
few simple keystrokes I need to remember, and the only time I
ever touch "info" is when a very short man page tells me that the
real docs are in info and I need to look there. At that point, I have
to relearn how to use that info command. Info is complicated. It's
got some concept of nodes being hierarcical, but also they seem
to have some order to them (a la next and previous) as well. No
idea what the connection between those is. Also it's got "cross-
references" as well as menus -- dunno if they're different or not.
Bah. Why not just use html and browse it with lynx/links/links2/elinks?

I can't understand why we need two separate and distinct doc
formats: man and info. My take: write your docs in some markup
that can produce both man output and html. Done and done.
Seems like two good candidates for that are epytext and
reStructuredText ("reST").

But, if Python would match Perl for docs available on the
command-line, then I'd have it all at my fingertips. I simply don't
understand why this is not being done. When I'm coding in C, I use
the manpages on the remote host so that I know the docs are correct
for my target. Why can't I do that in Python? It's yet another thing
that my Perl-using coworkers point out as a Python weakness.

Python does match (and exceed) Perl for docs available on the command
line. Once you get used to using the excellent info-based Python
documentation using man is downright primitive.

Jason

I think that may be wishful thinking. When I use the pydoc command,
I mostly get *very* short API docs with *no* example code or tutorial
notes at all.

Contrast that with perldoc which is categorized and complete docs
for the entire language plus tutorials, faqs, ports info, ... I mean,
have a look over at http://perldoc.perl.org/perl.html -- everything
there is at your fingertips from the terminal! And it's written by
folks
like Larry, Dominus, Randal Schwartz, Tom Christiansen, et al. I've
never come across a FLOSS software project with docs as good as
Perl's.

Now, the one big thing we have going for us is that since Python
is a simpler language, so it needs few docs overall anyway. :)

---John
http://www.simisen.com/jmg/cpd/

[Tim Roberts]

jo***********@yahoo.com wrote:

Just tried Ruby over the past two days. I won't bore you
with the reasons I didn't like it, however one thing really
struck me about it that I think we (the Python community)
can learn from.

Ruby has ... an issue with docs. That is to say, there are
almost none. ...

IMO, Perl has docs nailed.

I know there are people who disagree with me, but I think the PHP community
has the right idea with their online docs. The wiki-like community
participation adds a helpful extra dimension, giving practical tips and
pointing out non-obvious gotchas.

However, I'm sure it's a bear to administer.
--
- Tim Roberts, ti**@probo.com
Providenza & Boekelheide, Inc.

[Joel Hedlund]

>>This "release" is as alpha as alpha gets. It's so alpha it

actually loops back around to zeta -- but it's a start, and I
think it's exactly what the Python community needs.

Not to pick nits, but that should actually be "... so alpha that it actually
loops back around to *OMEGA*."

I think he's using extended Greek++. That got seriously big around the late B.C's.

/Joel

Sorry for this garbage post, btw... Couldn't help myself... :-)

[Marc 'BlackJack' Rintsch]

In <ma***************************************@python. org>, Terry Hancock
wrote:

Epydoc seems to be "dead" and pudge not yet alive:

What leads you to this conclusion? Works pretty well for
me. Maybe it's just "stable"?

But Docutils is a moving target and now and then the combination of both
breaks and Epydoc doesn't get fixed officially. There are patches in the
Source Forge tracker I have to apply myself and it doesn't look like they
will be merged into a new Epydoc release.

Ciao,
Marc 'BlackJack' Rintsch