How does pydoc parse code?

[Greg Kuperberg]

I plan to use pydoc for my Python project. After looking through the
standard documentation, I am not sure how pydoc interprets its input.
In its basic operation it evidently looks at the first string literal in
the module and in each function definition. But there is more to the
story than that, obviously. What other vestigial code does it detect?
Every string literal? Variables of the form __xxx__, I gather? Which of
these variables have a special meaning? How does it divide the initial
string literal into the "name" and "description" sections? What other
directives can I send to pydoc to alter its presentation?

For instance, I might want a revision history for each module listed
before the description. Is there a way to do that? Or I might want
to suppress the __xxx__ variables used by pydoc from the data section.
Is there a way to do that? What *can* I do?

Given that it is a documentation facility, pydoc isn't all that well
documented. :-) At least not at this page:

http://www.python.org/doc/current/lib/module-pydoc.html

And this is hardly better:

pydoc /usr/lib/python2.2/pydoc.py
--
/\ Greg Kuperberg (UC Davis)
/ \
\ / Visit the Math ArXiv Front at http://front.math.ucdavis.edu/
\/ * All the math that's fit to e-print *

[Michael Hudson]

gr**@conifold.math.ucdavis.edu (Greg Kuperberg) writes:

I plan to use pydoc for my Python project. After looking through the
standard documentation, I am not sure how pydoc interprets its input.
In its basic operation it evidently looks at the first string literal in
the module and in each function definition. But there is more to the
story than that, obviously. What other vestigial code does it detect?
Every string literal? Variables of the form __xxx__, I gather? Which of
these variables have a special meaning? How does it divide the initial
string literal into the "name" and "description" sections? What other
directives can I send to pydoc to alter its presentation?
It would be nice if there was a concise, simple place in the
documentation I could point you to to answer that question.

Alas, it does what it does, and that's about all that can be said.

Oh, and it *doesn't* parse the module: it imports and then introspects
it.
For instance, I might want a revision history for each module listed
before the description. Is there a way to do that? Or I might want
to suppress the __xxx__ variables used by pydoc from the data section.
Is there a way to do that? What *can* I do?
Beats me. You might be happier with one of the other documentation
tools like epydoc or happydoc (both at SF) but I know very little
about them...

Cheers,
mwh

-- I'm a little confused.

That's because you're Australian! So all the blood flows to
your head, away from the organ most normal guys think with.
-- Mark Hammond & Tim Peters, comp.lang.python

[Bengt Richter]

On Fri, 17 Oct 2003 11:34:05 GMT, Michael Hudson <mw*@python.net> wrote:

gr**@conifold.math.ucdavis.edu (Greg Kuperberg) writes:

I plan to use pydoc for my Python project. After looking through the
standard documentation, I am not sure how pydoc interprets its input.
In its basic operation it evidently looks at the first string literal in
the module and in each function definition. But there is more to the
story than that, obviously. What other vestigial code does it detect?
Every string literal? Variables of the form __xxx__, I gather? Which of
these variables have a special meaning? How does it divide the initial
string literal into the "name" and "description" sections? What other
directives can I send to pydoc to alter its presentation?

It would be nice if there was a concise, simple place in the
documentation I could point you to to answer that question.

Alas, it does what it does, and that's about all that can be said.

Oh, and it *doesn't* parse the module: it imports and then introspects
it.

I wonder if we shouldn't take that seriously. E.g., check the file's md5
against a set (or dict, so you could ask why if a name change) of known safe
module sources and issue a (untested)

if not raw_input("""
WARNING: Module is not known to system.
There is a SECURITY RISK in proceeding, because pydoc executes the module
by importing it to make the doc info available to it via inspect.

Proceed anyway? (only typing exactly "Yes" w/o quotes will proceed): """
)=='Yes': raise SystemExit, 'Unsafe pydoc inspection abandoned by user.'

if not found. You could also give the user the option to declare a given module
trusted by having the md5 set persist in site info, for convenience.

Regards,
Bengt Richter