473,686 Members | 2,926 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

unittest: collecting tests from many modules?

I have a set of tests in different modules: test_foo.py, test_bar.py and so
on. All of these use the simplest possible internal layout: a number of
classes containing test*() methods, and the good old lines at the end:

if __name__ == "__main__":
unittest.main()

This is great, because each of the modules are runnable, and I can select
classes or tests to run on the commandline if I want to. However, running
all the tests from e.g. a Makefile is not that fun; I don't get a single
pass/fail summary across the modules.

What's the best way of creating a test.py which
- aggregates the tests from all the test_*.py modules?
- doesn't require me to enumerate all the test classes in test.py
(forcing each module to define test_foo.theSui te or someting would
be OK though)
- retains the ability to select tests and verbosity (-q, -v) from the
command line?

Something like:

import unittest
import test_foo
import test_bar

if __name__ == "__main__":
unittest.main(m odules = ['test_foo',
'test_bar'])

Seems to me this should be possible, since all the logic for doing it /is/
there for the local module; I'd assume there would be a way to make unittest
search additional modules for test classes. But my head starts spinning
when I read the source code ...

/Jorgen

--
// Jorgen Grahn <jgrahn@ Ph'nglui mglw'nafh Cthulhu
\X/ algonet.se> R'lyeh wgah'nagl fhtagn!
Jul 19 '05 #1
7 2070
"Jorgen Grahn" <jg*********@al gonet.se> wrote in message
news:sl******** *************** *@frailea.sa.in valid...
I have a set of tests in different modules: test_foo.py, test_bar.py and so
on. All of these use the simplest possible internal layout: a number of
classes containing test*() methods, and the good old lines at the end:

if __name__ == "__main__":
unittest.main()

This is great, because each of the modules are runnable, and I can select
classes or tests to run on the commandline if I want to. However, running
all the tests from e.g. a Makefile is not that fun; I don't get a single
pass/fail summary across the modules.

What's the best way of creating a test.py which
- aggregates the tests from all the test_*.py modules?
- doesn't require me to enumerate all the test classes in test.py
(forcing each module to define test_foo.theSui te or someting would
be OK though)
- retains the ability to select tests and verbosity (-q, -v) from the
command line?
I use your second point: I build a TestAll module. However,
I'm going to look at building the test suite using the TestLoader
class in the next version of PyFit. It sholdn't be all that difficult
to find all the Test*.py modules in a directory, import them and
use the TestLoader class to add them to the test suite.

Or, for that matter, to use reflection to find all the classes that
derive from TestCase and add them to the suite manually. That
has the advantage that one could then select classes according
to some parameter.

John Roth


Something like:

import unittest
import test_foo
import test_bar

if __name__ == "__main__":
unittest.main(m odules = ['test_foo',
'test_bar'])

Seems to me this should be possible, since all the logic for doing it /is/
there for the local module; I'd assume there would be a way to make
unittest
search additional modules for test classes. But my head starts spinning
when I read the source code ...

/Jorgen

--
// Jorgen Grahn <jgrahn@ Ph'nglui mglw'nafh Cthulhu
\X/ algonet.se> R'lyeh wgah'nagl fhtagn!


Jul 19 '05 #2
"Jorgen Grahn" wrote:
I have a set of tests in different modules: test_foo.py, test_bar.py and so
on. All of these use the simplest possible internal layout: a number of
classes containing test*() methods, and the good old lines at the end:

if __name__ == "__main__":
unittest.main()

This is great, because each of the modules are runnable, and I can select
classes or tests to run on the commandline if I want to. However, running
all the tests from e.g. a Makefile is not that fun; I don't get a single
pass/fail summary across the modules.

What's the best way of creating a test.py which
- aggregates the tests from all the test_*.py modules?
- doesn't require me to enumerate all the test classes in test.py
(forcing each module to define test_foo.theSui te or someting would
be OK though)
- retains the ability to select tests and verbosity (-q, -v) from the
command line?

Something like:

import unittest
import test_foo
import test_bar

if __name__ == "__main__":
unittest.main(m odules = ['test_foo',
'test_bar'])

Seems to me this should be possible, since all the logic for doing it /is/
there for the local module; I'd assume there would be a way to make unittest
search additional modules for test classes. But my head starts spinning
when I read the source code ...


I had written a script to do something close to this; currently it
doesn't do any kind of aggregation, but it should be easy to extend it
as you like. What I don't like is the way it currently works: it
replaces sys.modules['__main__'] for each unit test and then it
execfile()s it, which seems like a hack. I didn't look into unittest's
internals in case there is a more elegant way around this; if there
isn't, a future version of unittest should address the automatic
aggregation of tests, as py.test does already.

The link to the script is http://rafb.net/paste/results/V0y16g97.html.

Hope this helps,
George

Jul 19 '05 #3
Jorgen Grahn wrote:
I have a set of tests in different modules: test_foo.py, test_bar.py and so
on. All of these use the simplest possible internal layout: a number of
classes containing test*() methods, and the good old lines at the end:
if __name__ == "__main__":
unittest.main()
....

What's the best way of creating a test.py which
- aggregates the tests from all the test_*.py modules?
- doesn't require me to enumerate all the test classes in test.py
(forcing each module to define test_foo.theSui te or someting would
be OK though)
- retains the ability to select tests and verbosity (-q, -v) from the
command line? Something like:

import unittest
import test_foo
import test_bar

if __name__ == "__main__":
unittest.main(m odules = ['test_foo',
'test_bar'])

Seems to me this should be possible, since all the logic for doing it /is/
there for the local module; I'd assume there would be a way to make unittest
search additional modules for test classes. But my head starts spinning
when I read the source code ...

/Jorgen

How about some variant of:

import unittest
import test_foo, test_bar, ...

def set_globals(mod ules):
glbl = globals()
for module in modules:
modprefix = module.__name__[5:] + '__'
for element in dir(module):
data = getattr(module, element)
if isinstance(data , type) and issubclass(data ,
unittest.TestCa se):
glbl[modprefix + element] = data

if __name__ == "__main__":
module = type(unittest)
set_globals([mod for name, mod in globals().items ()
if name.lower().be ginswith('test' )
and isinstance(mod, module)])
unittest.main()
--Scott David Daniels
Sc***********@A cm.Org
Jul 19 '05 #4
On 12 Jun 2005 08:06:18 -0700, "George Sakkis" <gs*****@rutger s.edu> wrote:
"Jorgen Grahn" wrote:
I have a set of tests in different modules: test_foo.py, test_bar.py and so
on. All of these use the simplest possible internal layout: a number of
classes containing test*() methods, and the good old lines at the end:

if __name__ == "__main__":
unittest.main()

This is great, because each of the modules are runnable, and I can select
classes or tests to run on the commandline if I want to. However, running
all the tests from e.g. a Makefile is not that fun; I don't get a single
pass/fail summary across the modules.

What's the best way of creating a test.py which
- aggregates the tests from all the test_*.py modules?
- doesn't require me to enumerate all the test classes in test.py
(forcing each module to define test_foo.theSui te or someting would
be OK though)
- retains the ability to select tests and verbosity (-q, -v) from the
command line?

Something like:

import unittest
import test_foo
import test_bar

if __name__ == "__main__":
unittest.main(m odules = ['test_foo',
'test_bar'])

Seems to me this should be possible, since all the logic for doing it /is/
there for the local module; I'd assume there would be a way to make unittest
search additional modules for test classes. But my head starts spinning
when I read the source code ...


I had written a script to do something close to this; currently it
doesn't do any kind of aggregation, but it should be easy to extend it
as you like. What I don't like is the way it currently works: it
replaces sys.modules['__main__'] for each unit test and then it
execfile()s it, which seems like a hack. I didn't look into unittest's
internals in case there is a more elegant way around this; if there
isn't, a future version of unittest should address the automatic
aggregation of tests, as py.test does already.

The link to the script is http://rafb.net/paste/results/V0y16g97.html.

I think if you execfile a script and supply the global dict initialized
to {'__name__':'__ main__'} then I think that will satisfy the if __name__ ... condition
and the whole thing will run as if executed interactively from the command line.
So IWT you could just loop through a list of test modules doing that.
Haven't tried it though. E.g., maybe there are nasty reload problems for
shared modules among different tests?

Regards,
Bengt Richter
Jul 19 '05 #5
"Jorgen Grahn" wrote:
What's the best way of creating a test.py which
- aggregates the tests from all the test_*.py modules?


You might want to check out the test runner in Zope 3
(svn://svn.zope.org/repos/main/Zope3/trunk/src/zope/app/testing)

It aggregates test reporting, does code coverage, lets you select
subsets of the tests to run, as well as control verbosity.

And, if you feel experimental you might want to preview the new Zope
test runner currently under development
(svn://svn.zope.org/repos/main/zope.testing).
--
Benji York
Jul 19 '05 #6
"Bengt Richter" <bo**@oz.net> wrote:
On 12 Jun 2005 08:06:18 -0700, "George Sakkis" <gs*****@rutger s.edu> wrote:
I had written a script to do something close to this; currently it
doesn't do any kind of aggregation, but it should be easy to extend it
as you like. What I don't like is the way it currently works: it
replaces sys.modules['__main__'] for each unit test and then it
execfile()s it, which seems like a hack. I didn't look into unittest's
internals in case there is a more elegant way around this; if there
isn't, a future version of unittest should address the automatic
aggregation of tests, as py.test does already.

I think if you execfile a script and supply the global dict initialized
to {'__name__':'__ main__'} then I think that will satisfy the if __name__ ... condition
and the whole thing will run as if executed interactively from the command line.


Yes, execfile is called with {'__name__':'__ main__'} for globals but
that's not enough because __main__ is already bound to the test.py
script, not the tested module. Even worse, after each test,
sys.modules['__main__'] seems to have to be reset to the original
__main__; otherwise os (and I guess other globals) is bound to None !
I'm not sure if this is normal or I missed something basic.

Regards,
George
PS: I reposted the script at
http://rafb.net/paste/results/3f1PIZ70.html.

Jul 19 '05 #7
On 12 Jun 2005 10:14:50 GMT, Jorgen Grahn <jg*********@al gonet.se> wrote:

[regarding module unittest]
What's the best way of creating a test.py which
- aggregates the tests from all the test_*.py modules?
- doesn't require me to enumerate all the test classes in test.py
(forcing each module to define test_foo.theSui te or someting would
be OK though)
- retains the ability to select tests and verbosity (-q, -v) from the
command line?


Thanks for all the input. Three weeks later I stumble across this thread
again and notice I didn't report what I ended up with. I ended up doing the
thing I wanted to avoid in the first place: hardcoding everything.

import unittest

import test_bibdb
import test_citefmt
import test_person
import test_refsfmt

suite = unittest.TestSu ite()
suite.addTest(t est_bibdb.suite ())
suite.addTest(t est_citefmt.sui te())
suite.addTest(t est_person.suit e())
suite.addTest(t est_refsfmt.sui te())

if __name__ == "__main__":
unittest.TextTe stRunner(verbos ity=2).run(suit e)

My test modules are, after all, a fairly fixed set, and if I want to run
specific tests, or with a specific verbosity, I can execute the individual
test modules.

BR,
/Jorgen

--
// Jorgen Grahn <jgrahn@ Ph'nglui mglw'nafh Cthulhu
\X/ algonet.se> R'lyeh wgah'nagl fhtagn!
Jul 21 '05 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
2313
by: Will Stuyvesant | last post by:
I have a unittest testfile like this: ----------------------- test_mod.py --------------------- import sys sys.path.append('..') import unittest import mod class Test_rmlutils(unittest.TestCase):
3
1369
by: Brian | last post by:
Hello; I am writing my test modules that are separate from the modules they test. The test modules can be invoked with if __name__ == "__main__" when run from a command line. Right now, if __name__ == "__main__", the test module executes runTests(), which defines a suite of tests. From the module being tested, I am calling into the test module -- to the function runTests(), if __name__ == "__main__". Is this how people make the tests...
4
1450
by: Lol McBride | last post by:
Hi all, I'm developing an OOP program and trying to be smart by using unittest to help me to catch as many bugs as possible during development.Unfortunately I've hit a snag which I believe is to do with namespaces (specifically the os.path). The program is split into packages and modules as below: oop_lb - toplevelpackage __init__.py oop_lb.py
8
1821
by: Remy Blank | last post by:
Hello unittest users, In a project I am working on, I have a series of tests that have to be run as root, and others as a normal user. One solution is to separate the tests into two different files, and run the right one. I don't like this too much, as I prefer to group tests by "function". Another solution is to build the test suite differently, depending
0
2048
by: Remy Blank | last post by:
Ok, here we go. I added the possibility for tests using the unittest.py framework to be skipped. Basically, I added two methods to TestCase: TestCase.skip(msg): skips unconditionally TestCase.skipIf(expr, msg): skips if expr is true These can be called either in setUp() or in the test methods. I also added reporting of skipped tests to TestResult, _TextTestResult and
1
1503
by: Thomas Heller | last post by:
I'm trying to integrate some doctest tests with unittest. The tests must be exposed as one or more subclasses of unittest.TestCase, so I'm collecting them with a call to doctest.DocTestSuite(), and then add them to a TestCase class I have created. The tests seem to run, but they always seem to succeed - I have no idea why. Any ideas? Thomas ---snip---
41
10269
by: Roy Smith | last post by:
I've used the standard unittest (pyunit) module on a few projects in the past and have always thought it basicly worked fine but was just a little too complicated for what it did. I'm starting a new project now and I'm thinking of trying py.test (http://codespeak.net/py/current/doc/test.html). It looks pretty cool from the docs. Is there anybody out there who has used both packages and can give a comparative review?
18
1864
by: Scott David Daniels | last post by:
There has been a bit of discussion about a way of providing test cases in a test suite that _should_ work but don't. One of the rules has been the test suite should be runnable and silent at every checkin. Recently there was a checkin of a test that _should_ work but doesn't. The discussion got around to means of indicating such tests (because the effort of creating a test should be captured) without disturbing the development flow. ...
0
891
by: Roy Smith | last post by:
In article <87zlrzw1f5.fsf@physik.rwth-aachen.de>, Torsten Bronger <bronger@physik.rwth-aachen.dewrote: When I first started using unittest, I did the same thing you did -- I put the test files in some other directory. This forced me into having to play the same sorts of tricks you're playing with sys.path so you could import your modules into the test framework. It also turned out to be more work for me during normal development,...
0
8580
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8513
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9049
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
8931
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
8777
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
5795
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4526
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
2943
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
2203
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.