Using other tools with unittest?

[Edward K. Ream]

Has anyone used tools like pychecker (or even Python's own debugger or
profiler) during unit testing? Here are some things that might be natural
to do:

1. pychecker can run all kinds of fancy tests on code. I'm wondering
whether these tests could be incorporated somehow into individual unit
tests. For example, one might use pychecker to assert that a subclass does,
or does not, override a method of a base class. Yes, one could write that
test "by hand". The question is: how good is pychecker for this sort of
thing? Anyone have any experience?

2. Python's debugger is simple because Python so helpfully provides the
sys.settrace function. Has anyone used sys.settrace for unit testing? For
example, suppose you wanted to make a list of the methods that are exercised
by a test suite. The tracing function registered by sys.settrace could
easily do this. Etc.

3 In a slightly different direction, has someone used a simulation of
aspect-oriented programming in Python to make statements about test
coverage, or for other testing purposes?

Thanks.

Edward
--------------------------------------------------------------------
Edward K. Ream email: ed*******@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------

[Jp Calderone]

On Tue, Nov 18, 2003 at 04:11:18PM -0600, Edward K. Ream wrote:

Has anyone used tools like pychecker (or even Python's own debugger or
profiler) during unit testing? Here are some things that might be natural
to do:

1. pychecker can run all kinds of fancy tests on code. I'm wondering
whether these tests could be incorporated somehow into individual unit
tests. For example, one might use pychecker to assert that a subclass does,
or does not, override a method of a base class. Yes, one could write that
test "by hand". The question is: how good is pychecker for this sort of
thing? Anyone have any experience?
Dunno. The way I typically write tests, I'll just call the method. If
it's not implemented (or implemented incorrectly), it won't behave how I
expect it to, and the test will fail.

It might be interesting to have a "pychecker test" provided by the test
framework. Similar to the common "at least import" tests, this could run
pychecker on all modules in a project, and fail if errors (and/or warnings)
are emitted. I've never looked inside pychecker, I don't know how easy or
hard this would be to implement.

2. Python's debugger is simple because Python so helpfully provides the
sys.settrace function. Has anyone used sys.settrace for unit testing? For
example, suppose you wanted to make a list of the methods that are exercised
by a test suite. The tracing function registered by sys.settrace could
easily do this. Etc.
A few test frameworks have support for reporting code coverage. No names
spring to mind at the moment, unfortunately. Also, a project I work on uses
this module:

http://www.ravenbrook.com/project/p4...st/coverage.py

in its unit tests to generate coverage statistics (the "test" can't ever
fail; a graph is generated based on its results, though; it is run for every
checkin).

3 In a slightly different direction, has someone used a simulation of
aspect-oriented programming in Python to make statements about test
coverage, or for other testing purposes?

Not exactly sure what you're getting at here.
Jp

[John Roth]

"Edward K. Ream" <ed*******@charter.net> wrote in message
news:vr************@corp.supernews.com...

Has anyone used tools like pychecker (or even Python's own debugger or
profiler) during unit testing? Here are some things that might be natural
to do:

1. pychecker can run all kinds of fancy tests on code. I'm wondering
whether these tests could be incorporated somehow into individual unit
tests. For example, one might use pychecker to assert that a subclass does, or does not, override a method of a base class. Yes, one could write that
test "by hand". The question is: how good is pychecker for this sort of
thing? Anyone have any experience?
I don't have any experiance, but the statement on the project page
http://pychecker.sourceforge.net/
doesn't suggest that it can do what you mention. It's more a static
analyzer for common problems.

That said, it might be a good project practice that all code needs to
pass pychecker before it can be committed.

Also, I'd like to know how you write that kind of test 'by hand.'
I can certainly see checking a single class for a manditory override /
not override, but I can't see exactly how you'd check *all* subclasses
for that condition without building a lot of infrastructure. You would at
least have to *find* all the class definitions.
2. Python's debugger is simple because Python so helpfully provides the
sys.settrace function. Has anyone used sys.settrace for unit testing? For example, suppose you wanted to make a list of the methods that are exercised by a test suite. The tracing function registered by sys.settrace could
easily do this. Etc.
There are at least two coverage tools that use the trace hook, and
I suspect there are others. One is accessable through the Vaults;
I don't know the rest of them. I've frequently thought that if you're
using XP or TDD, it might be a real good practice to run a coverage
checker. TDD is supposed to get you very close to 100% statement
coverage without extra effort.

When I want to check, I simply run my test suite under the coverage
tool. It takes about three times (or more) to run that way.
3 In a slightly different direction, has someone used a simulation of
aspect-oriented programming in Python to make statements about test
coverage, or for other testing purposes?
I don't understand this one at all...

John Roth

Thanks.

Edward
--------------------------------------------------------------------
Edward K. Ream email: ed*******@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------