doctest environment question

[thomas.guest]

I'm not making progress with the following and would appreciate any
help.

Here's an interpreted Python session.

>>import sys
def f(): pass

....

>>this_module = sys.modules[__name__]
delattr(this_module, 'f')
f()

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'f' is not defined

Suppose I want to doctest this behaviour. I cut and paste the above
into a file "test.txt" then run:

python -c "import doctest; doctest.testfile('test.txt')"

This gives me unexpected test failures:

python -c "import doctest; doctest.testfile('test.txt')"
************************************************** ********************
File "test.txt", line 5, in test.txt
Failed example:
delattr(this_module, 'f')
Exception raised:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/doctest.py", line 1212, in __run
compileflags, 1) in test.globs
File "<doctest test.txt[3]>", line 1, in <module>
delattr(this_module, 'f')
AttributeError: f
************************************************** ********************
File "test.txt", line 6, in test.txt
Failed example:
f()
Expected:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'f' is not defined
Got nothing
************************************************** ********************
1 items had failures:
2 of 5 in test.txt
***Test Failed*** 2 failures.

[Peter Otten]

th**********@gmail.com wrote:

I'm not making progress with the following and would appreciate any
help.

Here's an interpreted Python session.

>>>import sys
def f(): pass

...

>>>this_module = sys.modules[__name__]
delattr(this_module, 'f')
f()

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'f' is not defined

Suppose I want to doctest this behaviour. I cut and paste the above
into a file "test.txt" then run:

python -c "import doctest; doctest.testfile('test.txt')"

This gives me unexpected test failures:

python -c "import doctest; doctest.testfile('test.txt')"
************************************************** ********************
File "test.txt", line 5, in test.txt
Failed example:
delattr(this_module, 'f')
Exception raised:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/doctest.py", line 1212, in __run
compileflags, 1) in test.globs
File "<doctest test.txt[3]>", line 1, in <module>
delattr(this_module, 'f')
AttributeError: f
************************************************** ********************
File "test.txt", line 6, in test.txt
Failed example:
f()
Expected:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'f' is not defined
Got nothing
************************************************** ********************
1 items had failures:
2 of 5 in test.txt
***Test Failed*** 2 failures.

The doctest code is executed in a module without a __name__, it seems.
Unfortunately (in this case) the builtin module serves as a fallback
helping out with its own name:

>>__name__

'__main__'

>>del __name__
__name__

'__builtin__'
What to do about it? doctest could be changed to execute code snippets in a
module with a name and a sys.modules entry though I don't see much benefit
here.

Peter

[tag]

On 21 May, 18:53, Peter Otten <[e-mail]:

The doctest code is executed in a module without a __name__, it seems.
Unfortunately (in this case) the builtin module serves as a fallback
helping out with its own name:

>__name__

'__main__'

>del __name__
__name__

'__builtin__'

What to do about it? doctest could be changed to execute code snippets in a
module with a name and a sys.modules entry though I don't see much benefit
here.

Peter, thanks for the quick response, but I don't quite understand
what you're saying. I don't want to change doctest -- I want to find a
way to make my example pass using doctest.

doctest.testfile comes with lots of parameters, and I suspect if I
knew how to do it I could pass in the correct parameters to make this
example work. It's not what I actually want to document/test, it's a
minimal example which demonstrates the problem.

Thanks again.

>
Peter

[Peter Otten]

tag wrote:

On 21 May, 18:53, Peter Otten <[e-mail]:

>The doctest code is executed in a module without a __name__, it seems.
Unfortunately (in this case) the builtin module serves as a fallback
helping out with its own name:

>>__name__

'__main__'

>>del __name__
__name__

'__builtin__'

What to do about it? doctest could be changed to execute code snippets in
a module with a name and a sys.modules entry though I don't see much
benefit here.

Peter, thanks for the quick response, but I don't quite understand
what you're saying. I don't want to change doctest -- I want to find a
way to make my example pass using doctest.

The environment that doctest provides is similar to the interactive
interpreter but not identical. In particular there is no meaningful
__name__.

doctest.testfile comes with lots of parameters, and I suspect if I
knew how to do it I could pass in the correct parameters to make this
example work. It's not what I actually want to document/test, it's a
minimal example which demonstrates the problem.

Here are two alternatives:

(1)

>>def f(): pass

....

>>del f
f()

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'f' is not defined

(2)

>>def f(): pass

....

>>del globals()["f"]
f()

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'f' is not defined

If these don't work you'll have to give a bit more context.

Peter

[tag]

On 21 May, 22:17, Peter Otten <[e-mail]:

If these don't work you'll have to give a bit more context.

Peter

Thanks again Peter. Here's something much closer to what I really want
to do. You should be able to cut and paste this post into a file
"post.txt". Running the command `python -c "import doctest;
doctest.testfile('post.txt')"` gives a test failure even though
everything works fine in an interpreted Python session. I'd like to
find a way to make the doctest pass.

>>def announce(f):

.... " Return a function which announces calls to the input
function. "
.... def wrapper(*v, **k):
.... print "Calling %s" % f.__name__
.... return f(*v, **k)
.... return wrapper

We can rebind a function to announce calls to it:

>>def g(): pass

....

>>g = announce(g)
g()

Calling g

Or we can use decorator syntax:

>>@announce

.... def f(): pass
....

>>f()

Calling f

Here's a function which rebinds a function at the top level of a
module (it won't work for nested functions).

>>def announce_function(f):

.... " Rebind f within a module so that calls to f are announced. "
.... import inspect
.... setattr(inspect.getmodule(f), f.__name__, announce(f))

Let's give it a try. This next works fine in an interactive Python
session but fails when doctested.

>>def h(): pass

....

>>announce_function(h)
h()

Calling h
Here's the doctest failure:

python -c "import doctest; doctest.testfile('post.txt')"
************************************************** ********************
File "post.txt", line 37, in post.txt
Failed example:
announce_function(h)
Exception raised:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/doctest.py", line 1212, in __run
compileflags, 1) in test.globs
File "<doctest post.txt[8]>", line 1, in <module>
announce_function(h)
File "<doctest post.txt[6]>", line 4, in announce_function
setattr(inspect.getmodule(f), f.__name__, announce(f))
AttributeError: 'NoneType' object has no attribute 'h'
************************************************** ********************
File "post.txt", line 38, in post.txt
Failed example:
h()
Expected:
Calling h
Got nothing
************************************************** ********************
1 items had failures:
2 of 10 in post.txt
***Test Failed*** 2 failures.

[Peter Otten]

tag wrote:

Thanks again Peter. Here's something much closer to what I really want
to do. You should be able to cut and paste this post into a file
"post.txt". Running the command `python -c "import doctest;
doctest.testfile('post.txt')"` gives a test failure even though
everything works fine in an interpreted Python session. I'd like to
find a way to make the doctest pass.

>>>def announce(f):

... " Return a function which announces calls to the input
function. "
... def wrapper(*v, **k):
... print "Calling %s" % f.__name__
... return f(*v, **k)
... return wrapper

We can rebind a function to announce calls to it:

>>>def g(): pass

...

>>>g = announce(g)
g()

Calling g

Or we can use decorator syntax:

>>>@announce

... def f(): pass
...

>>>f()

Calling f

Here's a function which rebinds a function at the top level of a
module (it won't work for nested functions).

>>>def announce_function(f):

... " Rebind f within a module so that calls to f are announced. "
... import inspect
... setattr(inspect.getmodule(f), f.__name__, announce(f))

inspect.getmodule(f) returns None because f() is not defined in a module.
You can either move f() to a helper module and then

from helper_module import f

or modify anouncement_function() to not rely on that non-existent module

>>def announce_function(f):

.... " Rebind f within a module so that calls to f are announced. "
.... f.func_globals[f.__name__] = announce(f)
....

Let's give it a try. This next works fine in an interactive Python
session but fails when doctested.

>>>def h(): pass

...

>>>announce_function(h)
h()

Calling h

Even when it works, implicitly modifying global variables is bad style.

Peter

[tag]

On 22 May, 08:59, Peter Otten <[e-mail]:

[snip]

inspect.getmodule(f) returns None because f() is not defined in a module.

OK. But there was a module when I ran interactively?

You can either move f() to a helper module and then

from helper_module import f

Yes.

or modify anouncement_function() to not rely on that non-existent module

>def announce_function(f):

... " Rebind f within a module so that calls to f are announced. "
... f.func_globals[f.__name__] = announce(f)
...

I think this is what I should be doing. Very nice! You're modifying
f's own gloabl environment.

>

Let's give it a try. This next works fine in an interactive Python
session but fails when doctested.

>>def h(): pass

...

>>announce_function(h)
h()

Calling h

Even when it works, implicitly modifying global variables is bad style.

I have to admit it didn't feel right, but I didn't come up with the
idea you present. Thanks again for your help.

[Peter Otten]

tag wrote:

On 22 May, 08:59, Peter Otten <[e-mail]:

>inspect.getmodule(f) returns None because f() is not defined in a module.

OK. But there was a module when I ran interactively?

Yes. Looking into the doctest source, there is a -- deprecated -- class
called Tester that provides a module. I don't know why this approach was
dropped.

Peter

[Gabriel Genellina]

En Tue, 22 May 2007 04:21:06 -0300, tag <th**********@gmail.comescribió:

Here's a function which rebinds a function at the top level of a
module (it won't work for nested functions).

>>>def announce_function(f):

... " Rebind f within a module so that calls to f are announced. "
... import inspect
... setattr(inspect.getmodule(f), f.__name__, announce(f))

Let's give it a try. This next works fine in an interactive Python
session but fails when doctested.

The version given by Peter Otten may do what you want, but I'd consider if
you really need an announce_function in the first place, given all the
other ways you already have to do the same thing.
Implicitely rebinding globals does not look good.

--
Gabriel Genellina

[tag]

On 22 May, 10:11, "Gabriel Genellina" <[e-mail]:

The version given by Peter Otten may do what you want, but I'd consider if
you really need an announce_function in the first place, given all the
other ways you already have to do the same thing.
Implicitely rebinding globals does not look good.

Thanks for the advice Gabriel. The use case I have is that I'd like to
be able to decorate classes and even modules in this way:

import announce
import spam
announce.announce_module(spam)

.... code which calls into spam module

Here, "announce.announce_module" has a look in "spam", finds all the
functions and instancemethods, and decorates them (rebinds them) by
announcing calls to these functions and instancemethods.

It's something I've found useful, though clearly the behaviour of
"spam" has been drastically changed. I'd appreciate advice on better
ways to achieve this kind of thing, or why it doesn't look good.

[tag]

On 22 May, 10:11, "Gabriel Genellina" <[e-mail]:

The version given by Peter Otten may do what you want, but I'd consider if
you really need an announce_function in the first place, given all the
other ways you already have to do the same thing.
Implicitely rebinding globals does not look good.

Thanks for the advice Gabriel. The use case I have is that I'd like to
be able to decorate classes and even modules in this way:

>>import announce
import spam
announce.announce_module(spam)

Here, "announce.announce_module" has a look in "spam", finds all the
functions and instancemethods, and decorates them (rebinds them) by
announcing calls to these functions and instancemethods.

It's something I've found useful, though clearly the behaviour of
"spam" has been drastically changed. I'd appreciate advice on better
ways to achieve this kind of thing, or why it doesn't look good.

[tag]

On 22 May, 10:11, "Gabriel Genellina" <[e-mail]:

The version given by Peter Otten may do what you want, but I'd consider if
you really need an announce_function in the first place, given all the
other ways you already have to do the same thing.
Implicitely rebinding globals does not look good.

Thanks for the advice Gabriel. The use case I have is that I'd like to
be able to decorate classes and even modules in this way:

import announce
import spam
announce.announce_module(spam)

.... code which calls into spam module

Here, "announce.announce_module" has a look in "spam", finds all the
functions and instancemethods, and decorates them (rebinds them) by
announcing calls to these functions and instancemethods.

It's something I've found useful, though clearly the behaviour of
"spam" has been drastically changed. I'd appreciate advice on better
ways to achieve this kind of thing, or why it doesn't look good.

[Gabriel Genellina]

En Tue, 22 May 2007 08:57:29 -0300, tag <th**********@gmail.comescribió:

On 22 May, 10:11, "Gabriel Genellina" <[e-mail]:

>The version given by Peter Otten may do what you want, but I'd consider
if
you really need an announce_function in the first place, given all the
other ways you already have to do the same thing.
Implicitely rebinding globals does not look good.

Thanks for the advice Gabriel. The use case I have is that I'd like to
be able to decorate classes and even modules in this way:

import announce
import spam
announce.announce_module(spam)

... code which calls into spam module

Here, "announce.announce_module" has a look in "spam", finds all the
functions and instancemethods, and decorates them (rebinds them) by
announcing calls to these functions and instancemethods.

It's something I've found useful, though clearly the behaviour of
"spam" has been drastically changed. I'd appreciate advice on better
ways to achieve this kind of thing, or why it doesn't look good.

Ah, then you *have* a reference to the containing module. announce_module
might be something like this:

def announce_module(module):
for fname in find_all_functions_by_name_in_module(module):
function = getattr(module, fname)
setattr(module, fname, announce_function(function))

And no need to guess anything. It works even if the original function name
is not the same as the name used in the module (this may happen when you
assign a function with another name).
Remember "Explicit is better than implicit" and "In the face of ambiguity,
refuse the temptation to guess."

--
Gabriel Genellina