unittest vs py.test?

[Roy Smith]

I've used the standard unittest (pyunit) module on a few projects in the
past and have always thought it basicly worked fine but was just a little
too complicated for what it did.

I'm starting a new project now and I'm thinking of trying py.test
(http://codespeak.net/py/current/doc/test.html). It looks pretty cool from
the docs. Is there anybody out there who has used both packages and can
give a comparative review?

[Raymond Hettinger]

[Roy Smith]

I've used the standard unittest (pyunit) module on a few projects in the
past and have always thought it basicly worked fine but was just a little
too complicated for what it did.

I'm starting a new project now and I'm thinking of trying py.test
(http://codespeak.net/py/current/doc/test.html). It looks pretty cool from
the docs. Is there anybody out there who has used both packages and can
give a comparative review?

I've used both and found py.test to be effortless and much less verbose.
For more complex testing strategies, py.test is also a winner. The generative
tests are easier to write than crafting a similar strategy for unittest.

py.test does not currently integrate well with doctest; however, that will
likely
be the next feature to be added (per holger's talk at PyCon).

For output, unittest's TextTestRunner produces good looking, succinct output
on successful tests. For failed tests, it is not bad either. In contrast,
py.test
output is more highly formatted and voluminous -- it takes a while to get used
to.

unittest users have to adapt to the internal structure of the unittest module
and
become familiar with its class structure (test fixture, test case, test suite,
and test
runner objects). py.test does a good job of hiding its implementation.

py.test is relatively new and is continuing to evolve. Support tools like a
GUI test runner are just emerging. In contrast, unittest is based on a proven
model and the code is mature.

unittest module updates come up in distinct releases, often months or years
apart. py.test is subject to constant update by subversion. Personally, I like
the continuous updates, but it could be unsettling if you're depending on it
for production code.
Raymond Hettinger

[Nigel Rowe]

Roy Smith wrote:

I've used the standard unittest (pyunit) module on a few projects in the
past and have always thought it basicly worked fine but was just a little
too complicated for what it did.

I'm starting a new project now and I'm thinking of trying py.test
(http://codespeak.net/py/current/doc/test.html). It looks pretty cool
from
the docs. Is there anybody out there who has used both packages and can
give a comparative review?

Have you seen Grig Gheorghiu's 3 part comparison of unittest, and py.test?

http://agiletesting.blogspot.com/200...-unittest.html
http://agiletesting.blogspot.com/200...2-doctest.html
http://agiletesting.blogspot.com/200...test-tool.html

--
Nigel Rowe
A pox upon the spammers that make me write my address like..
rho (snail) swiftdsl (stop) com (stop) au

[bearophileHUGS]

Nigel Rowe>Have you seen Grig Gheorghiu's 3 part comparison of
unittest, and py.test?<

Very interesting articles, thank you. Testing seems something still in
quick development.

For small functions the doctests are useful, but py.test has some
advantages. Probably something even better that py.test can be
designed, taking some ideas from the doctests, or vice versa :-]
In py.test I see a couple of features useful for the Python language
too:

The raises:
py.test.raises( NameError, "self.alist.sor t(int_compare)" )
py.test.raises( ValueError, self.alist.remo ve, 6)
(A try can probably do something similar)

And the improved error messages:
"When it encounters a failed assertion, py.test prints the lines [3-4
lines?] in the method containing the assertion, up to and including the
failure. It also prints the actual and the expected values involved in
the failed assertion."

http://agiletesting.blogspot.com/200...test-tool.html

Such things can help avoid (just for simple situations/functions!)
testing frameworks in the first place, so you can use just the normal
Python code to test other code.

Bye,
Bearophile

[Roy Smith]

Nigel Rowe <rh*@see.signat ure.invalid> wrote:

Have you seen Grig Gheorghiu's 3 part comparison of unittest, and py.test?

http://agiletesting.blogspot.com/200...-unittest.html
http://agiletesting.blogspot.com/200...2-doctest.html
http://agiletesting.blogspot.com/200...test-tool.html

Just finished reading them now. Thanks for the pointer, they make an
excellent review of the space.

One thing that worries me a little is that all three seem to have
advantages and disadvantages, yet none is so obviously better than the
others that it stands out as the only reasonable way to do it. This means
some groups will adopt one, some will adopt another, and the world will
become fragmented.

[Grig Gheorghiu]

In my mind, practicing TDD is what matters most. Which framework you
choose is a function of your actual needs. The fact that there are 3 of
them doesn't really bother me. I think it's better to have a choice
from a small number of frameworks rather than have no choice or have a
single choice that might not be the best for your specific environment
-- provided of course that this doesn't evolve into a PyWebOff-like
nightmare :-)

Grig

[Raymond Hettinger]

[Roy Smith <ro*@panix.co m>]

One thing that worries me a little is that all three seem to have
advantages and disadvantages, yet none is so obviously better than the
others that it stands out as the only reasonable way to do it. This means
some groups will adopt one, some will adopt another, and the world will
become fragmented.

Worry is a natural thing for someone with "panix" in their email address ;-)

FWIW, the evolution of py.test is to also work seemlessly with existing tests
from the unittest module.

the world diversifies, the world congeals,
Raymond Hettinger

[Jeremy Bowers]

On Fri, 01 Apr 2005 16:42:30 +0000, Raymond Hettinger wrote:

FWIW, the evolution of py.test is to also work seemlessly with existing tests
from the unittest module.

Is this true now, or is this planned?

I read(/skimmed) the docs for py.test when you linked to the project, but
I don't recall seeing that. Certainly some of the features made me drool
but I have an investment in unittest. If I can relatively easily port them
over, I'd love to use py.test. (I don't care about a small per-file
change, it'd probably be one I can automate anyhow. But I can't afford to
re-write every test.) I didn't see anything like this in the docs, but I
may have missed it.

That'd be cool.

[Grig Gheorghiu]

>From what I know, the PyPy guys already have a unittest-to-py.test
translator working, but they didn't check in the code yet. You can send
an email to py-dev at codespeak.net and let them know you're interested
in this functionality.

Grig

[Colin J. Williams]

Grig Gheorghiu wrote:

In my mind, practicing TDD is what matters most. Which framework you
choose is a function of your actual needs. The fact that there are 3 of
them doesn't really bother me. I think it's better to have a choice
from a small number of frameworks rather than have no choice or have a
single choice that might not be the best for your specific environment
-- provided of course that this doesn't evolve into a PyWebOff-like
nightmare :-)

Grig

Grig,

Many thanks for your helpful essays.

unittest seems rather heavy. I don't like mixing tests with
documentation, it gives the whole thing a cluttered look.
Py.test is the more appealing but it doesn't appear to be
ready for packaging yet.

Thanks,

Colin W.