473,574 Members | 2,659 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Iterating over test data in unit tests

Howdy all,

Summary: I'm looking for idioms in unit tests for factoring out
repetitive iteration over test data. I explain my current practice,
and why it's unsatisfactory.
When following test-driven development, writing tests and then coding
to satisfy them, I'll start with some of the simple tests for a class.

import unittest

import bowling # Module to be tested

class Test_Frame(unit test.TestCase):

def test_instantiat e(self):
""" Frame instance should be created """
instance = bowling.Frame()
self.failUnless (instance)

class Test_Game(unitt est.TestCase):

def test_instantiat e(self):
""" Game instance should be created """
instance = bowling.Game()
self.failUnless (instance)

As I add tests for more interesting functionality, they become more
data dependent.

class Test_Game(unitt est.TestCase):

# ...

def test_one_throw( self):
""" Single throw should result in expected score """
game = bowling.Game()
throw = 5
game.add_throw( throw)
self.failUnless Equal(throw, game.get_score( ))

def test_three_thro ws(self):
""" Three throws should result in expected score """
game = bowling.Game()
throws = (5, 7, 4)
game.add_throw( throws[0])
game.add_throw( throws[1])
game.add_throw( throws[2])
self.failUnless Equal(sum(throw s), game.get_score( ))

This cries out, of course, for a test fixture to set up instances.

class Test_Game(unitt est.TestCase):

def setUp(self):
""" Set up test fixtures """
self.game = bowling.Game()

def test_one_throw( self):
""" Single throw should result in expected score """
throw = 5
score = 5
self.game.add_t hrow(throw)
self.failUnless Equal(score, game.get_score( ))

def test_three_thro ws(self):
""" Three throws should result in expected score """
throws = [5, 7, 4]
score = sum(throws)
for throw in throws:
game.add_throw( throw)
self.failUnless Equal(score, game.get_score( ))

def test_strike(sel f):
""" Strike should add the following two throws """
throws = [10, 7, 4, 7]
score = 39
for throw in throws:
game.add_throw( throw)
self.failUnless Equal(score, game.get_score( ))

So far, this is just following what I see to be common practice for
setting up *instances* to test.

But the repetition of the test *inputs* also cries out to me to be
refactored. I see less commonality in doing this.

My initial instinct is just to put it in the fixtures.

class Test_Game(unitt est.TestCase):

def setUp(self):
""" Set up test fixtures """
self.game = bowling.Game()

self.game_data = {
'one': dict(score=5, throws=[5]),
'three': dict(score=17, throws=[5, 7, 5]),
'strike': dict(score=39, throws=[10, 7, 5, 7]),
}

def test_one_throw( self):
""" Single throw should result in expected score """
throws = self.game_data['one']['throws']
score = self.game_data['one']['score']
for throw in throws:
self.game.add_t hrow(throw)
self.failUnless Equal(score, game.get_score( ))

def test_three_thro ws(self):
""" Three throws should result in expected score """
throws = self.game_data['three']['throws']
score = self.game_data['three']['score']
for throw in throws:
game.add_throw( throw)
self.failUnless Equal(score, game.get_score( ))

def test_strike(sel f):
""" Strike should add the following two throws """
throws = self.game_data['strike']['throws']
score = self.game_data['strike']['score']
for throw in throws:
game.add_throw( throw)
self.failUnless Equal(score, game.get_score( ))

But this now means that the test functions are almost identical,
except for choosing one data set or another. Maybe that means I need
to have a single test:

def test_score_thro ws(self):
""" Game score should be calculated from throws """
for dataset in self.game_data:
score = dataset['score']
for throw in dataset['throws']:
self.game.add_t hrow(throw)
self.failUnless Equal(score, self.game.get_s core())

Whoops, now I'm re-using a fixture instance. Maybe I need an instance
of the class for each test case.

def setUp(self):
""" Set up test fixtures """
self.game_data = {
'one': dict(score=5, throws=[5]),
'three': dict(score=17, throws=[5, 7, 5]),
'strike': dict(score=39, throws=[10, 7, 5, 7]),
}

self.game_param s = {}
for key, dataset in self.game_data. items():
params = {}
instance = bowling.Game()
params['instance'] = instance
params['dataset'] = dataset
self.game_param s[key] = params

def test_score_thro ws(self):
""" Game score should be calculated from throws """
for params in self.game_param s.values():
score = params['dataset']['score']
instance = params['instance']
for throw in params['dataset']['throws']:
instance.add_th row(throw)
self.failUnless Equal(score, instance.get_sc ore())

Good, now the tests for different sets of throws are in a dictionary
that's easy to add to. Of course, now I need to actually know which
one is failing.

def test_score_thro ws(self):
""" Game score should be calculated from throws """
for key, params in self.game_param s.items():
score = params['dataset']['score']
instance = params['instance']
for throw in params['dataset']['throws']:
instance.add_th row(throw)
self.failUnless Equal(score, instance.get_sc ore(),
msg="Score mismatch for set '%s'" % key
)

It works. It's rather confusing though, since the actual test --
iterate over the throws and check the score -- is in the midst of the
iteration over data sets.

Also, that's just *one* type of test I might need to do. Must I then
repeat all that iteration code for other tests I want to do on the
same data?

Maybe I need to factor out the iteration into a generic iteration
function, taking the actual test as a function object. That way, the
dataset iterator doesn't need to know about the test function, and
vice versa.

def iterate_test(se lf, test_func, test_params=Non e):
""" Iterate a test function for all the sets """
if not test_params:
test_params = self.game_param s
for key, params in test_params.ite ms():
dataset = params['dataset']
instance = params['instance']
test_func(key, dataset, instance)

def test_score_thro ws(self):
""" Game score should be calculated from throws """
def test_func(key, dataset, instance):
score = dataset['score']
for throw in dataset['throws']:
instance.add_th row(throw)
self.failUnless Equal(score, instance.get_sc ore())

self.iterate_te st(test_func)

That's somewhat clearer; the test function actually focuses on what
it's testing. Those layers of indirection are annoying, but they allow
the data sets to grow without writing more code to handle them.
Testing a rules-based system involves lots of data sets, and each data
set represents a separate test case; but the code for each of those
test cases is mindlessly repetitive. Factoring them out seems like it
needs a lot of indirection, and seems to make each test harder to
read. Different *types* of tests would need multiple iterators, more
complex test parameter dicts, or some more indirection. Those all
sound ugly, but so does repetitively coding every test function
whenever some new data needs to be tested.

How should this be resolved?

--
\ "I never forget a face, but in your case I'll be glad to make |
`\ an exception." -- Groucho Marx |
_o__) |
Ben Finney
Dec 6 '05 #1
6 2684
Ben Finney wrote:
Summary: I'm looking for idioms in unit tests for factoring out
repetitive iteration over test data....
How about something like:
import unittest, bowling
class Test_Game(unitt est.TestCase):
def setUp(self):
""" Set up test fixtures """
self.game = bowling.Game()
def runs(self, throws):
"""Run a series of scores and return the result"""
for throw in throws:
self.game.add_t hrow(throw)
return self.game.get_s core()
def test_one_throw( self):
""" Single throw should result in expected score """ self.assertEqua l(5, self.runs([5]))
def test_three_thro ws(self):
""" Three throws should result in expected score """ self.assertEqua l(5 + 7 + 4, self.runs([5, 7, 4]))
def test_strike(sel f):
""" Strike should add the following two throws """

self.assertEqua l(39, self.runs([10, 7, 4, 7]))
There is no reason you cannot write support functions.

--
-Scott David Daniels
sc***********@a cm.org
Dec 6 '05 #2
Scott David Daniels <sc***********@ acm.org> wrote:
Ben Finney wrote:
Summary: I'm looking for idioms in unit tests for factoring out
repetitive iteration over test data....
How about something like:
class Test_Game(unitt est.TestCase): [...] def runs(self, throws):
"""Run a series of scores and return the result"""

[...] def test_one_throw( self):
""" Single throw should result in expected score """

self.assertEqua l(5, self.runs([5]))
def test_three_thro ws(self):
""" Three throws should result in expected score """

self.assertEqua l(5 + 7 + 4, self.runs([5, 7, 4]))
def test_strike(sel f):
""" Strike should add the following two throws """

self.assertEqua l(39, self.runs([10, 7, 4, 7]))


Yes, I'm quite happy that I can factor out iteration *within* a single
data set. That leaves a whole lot of test cases identical except for
the data they use.

The question remains: how can I factor out iteration of *separate test
cases*, where the test cases are differentiated only by the data they
use? I know at least one way: I wrote about it in my (long) original
post. How else can I do it, with less ugliness?

--
\ "I went to a garage sale. 'How much for the garage?' 'It's not |
`\ for sale.'" -- Steven Wright |
_o__) |
Ben Finney
Dec 6 '05 #3
On Tue, 6 Dec 2005 12:19:40 +1100 (EST), Ben Finney <bi************ ****@benfinney. id.au> wrote:
Howdy all,

Summary: I'm looking for idioms in unit tests for factoring out
repetitive iteration over test data. I explain my current practice,
and why it's unsatisfactory.

Does this do what you want?
http://codespeak.net/py/current/doc/...ing-more-tests

Regards,
Bengt Richter
Dec 6 '05 #4
Ben Finney wrote:
Maybe I need to factor out the iteration into a generic iteration
function, taking the actual test as a function object. That way, the
dataset iterator doesn't need to know about the test function, and
vice versa.

def iterate_test(se lf, test_func, test_params=Non e):
""" Iterate a test function for all the sets """
if not test_params:
test_params = self.game_param s
for key, params in test_params.ite ms():
dataset = params['dataset']
instance = params['instance']
test_func(key, dataset, instance)

def test_score_thro ws(self):
""" Game score should be calculated from throws """
def test_func(key, dataset, instance):
score = dataset['score']
for throw in dataset['throws']:
instance.add_th row(throw)
self.failUnless Equal(score, instance.get_sc ore())

self.iterate_te st(test_func)

That's somewhat clearer; the test function actually focuses on what
it's testing. Those layers of indirection are annoying, but they allow
the data sets to grow without writing more code to handle them.


Don't know if this helps, but I'd be more likely to write this as
something like (untested)::

def get_tests(self, test_params=Non e):
""" Iterate a test function for all the sets """
if not test_params:
test_params = self.game_param s
for key, params in test_params.ite ms():
dataset = params['dataset']
instance = params['instance']
yield key, dataset, instance

def test_score_thro ws(self):
""" Game score should be calculated from throws """
for key, dataset, instance in self.get_tests( )
score = dataset['score']
for throw in dataset['throws']:
instance.add_th row(throw)
self.failUnless Equal(score, instance.get_sc ore())

That is, make an interator to the various test information, and just put
your "test_func" code inside a for-loop.

STeVe
Dec 6 '05 #5
Ben Finney <bi************ ****@benfinney. id.au> wrote:
Summary: I'm looking for idioms in unit tests for factoring out
repetitive iteration over test data.


Thanks to those who've offered suggestions, especially those who
suggested I look at generator functions. This leads to::

import unittest

import bowling # Module to be tested

class Test_Game(unitt est.TestCase):
""" Test case for the Game class """

def setUp(self):
""" Set up test fixtures """
self.game_data = {
'none': dict(score=0, throws=[], frame=1),
'one': dict(score=5, throws=[5], frame=1),
'two': dict(score=9, throws=[5, 4], frame=2),
'three': dict(score=14, throws=[5, 4, 5], frame=2),
'strike': dict(score=26, throws=[10, 4, 5, 7], frame=3),
}

self.game_param s = {}
for key, dataset in self.game_data. items():
params = {}
instance = bowling.Game()
params['instance'] = instance
params['dataset'] = dataset
self.game_param s[key] = params

def iterate_params( test_params=Non e):
""" Yield the test parameters """
if not test_params:
test_params = self.game_param s
for key, params in test_params.ite ms():
dataset = params['dataset']
instance = params['instance']
yield key, dataset, instance

def test_score_thro ws(self):
""" Game score should be calculated from throws """
for key, dataset, instance in self.iterate_pa rams():
score = dataset['score']
for throw in dataset['throws']:
instance.add_th row(throw)
self.failUnless Equal(score, instance.get_sc ore())

def test_current_fr ame(self):
""" Current frame should be as expected """
for key, dataset, instance in self.iterate_pa rams():
frame = dataset['frame']
for throw in dataset['throws']:
instance.add_th row(throw)
self.failUnless Equal(frame, instance.curren t_frame)

That's much better. Each test is now clearly about looping through the
datasets, but the infrastructure to do so is factored out. Adding a
test case modelled on the existing cases just means adding a new entry
to the game_data dictionary. Setting up a different kind of test --
e.g. for invalid game data -- just means setting up a new params
dictionary and feeding that to the same generator function.

I like it. Can it be improved? Are there readability problems that can
be fixed? Is the test fixture setup too complex? Should the iterator
become even more general, and be refactored out to a test framework
for the project?

--
\ "Those who can make you believe absurdities can make you commit |
`\ atrocities." -- Voltaire |
_o__) |
Ben Finney
Dec 6 '05 #6
Ben Finney wrote:
Ben Finney <bi************ ****@benfinney. id.au> wrote:
Summary: I'm looking for idioms in unit tests for factoring out
repetitive iteration over test data.


Thanks to those who've offered suggestions, especially those who
suggested I look at generator functions. This leads to::


Here's another way (each test should independently test one feature):

class Test_Game(unitt est.TestCase):
""" Test case for the Game class """
score = 0
throws = []
frame = 1

def setUp(self):
""" Set up test fixtures """

self.game = bowling.Game()

def test_score_thro ws(self):
""" Game score should be calculated from throws """
for throw in self.throws:
self.game.add_t hrow(throw)
self.assertEqua l(self.score, self.game.get_s core())

def test_current_fr ame(self):
""" Current frame should be as expected """
frame = dataset['frame']
for throw in self.throws:
self.game.add_t hrow(throw)
self.assertEqua l(self.frame, self.game.curre nt_frame)

class Test_one(Test_G ame):
score = 5
throws = [5]
frame = 1

class Test_two(Test_G ame):
score = 9
throws = [5, 4]
frame = 2

class Test_three(Test _Game):
score = 14
throws = [5, 4, 5]
frame = 2

class Test_strike(Tes t_Game):
score = 26
throws = [10, 4, 5, 7]
frame = 3

--Scott David Daniels
sc***********@a cm.org
Dec 6 '05 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

27
3761
by: Josh | last post by:
We have a program written in VB6 (over 100,000 lines of code and 230 UI screens) that we want to get out of VB and into a better language. The program is over 10 years old and has already been ported from VB3 to VB6, a job which took over two years. We would like to port it to Python, but we need to continue to offer upgrades and fixes to the...
3
2067
by: Water Cooler v2 | last post by:
Here's my understanding as of now. If I were writing a function bool IsValidContact(Offerer objOfferer, Accepter objAccepter, TermsAndConditions objTermsAndConditions); Before writing the function, I'd enlist all the conditions that must be met for a contract to be valid. Something along the lines of:
10
2428
by: Michael B. Trausch | last post by:
Alright, I seem to be at a loss for what I am looking for, and I am not even really all that sure if it is possible or not. I found the 'pdb' debugger, but I was wondering if there was something that would trace or log the order of line execution for a multi-module Python program. I am having a little bit of a problem tracking down a problem...
9
2062
by: Deckarep | last post by:
Hello Group, I actually have two seperate questions regarding Unit Testing with NUnit in C#. Please keep in mind that I'm new to the concept of Unit Testing and just barely coming around to feeling comfortable writing tests first and code after as in TDD style. Question 1: How can you effectively start incorporating Unit Testing in...
5
6507
by: shuisheng | last post by:
Dear All, I was told that unit test is a powerful tool for progamming. If I am writing a GUI code, is it possible to still using unit test? I have a little experience in using unittest++. But I can not work out a way to use it to test GUI code. Thanks a lot!
1
1791
by: rich_sposato | last post by:
I released version 2.0 of C++ Unit Test Library. You can download it from SourceForget.Net at http://sourceforge.net/projects/cppunittest/ .. I wrote this unit test library because other unit test frameworks always lacked features I considered important. Some only provided output in certain formats, and I had no easy way to choose my own...
48
2466
by: Ark Khasin | last post by:
Unit testing is an integral component of both "formal" and "agile" models of development. Alas, it involves a significant amount of tedious labor. There are test automation tools out there but from what limited exposure I've had, they are pricey, reasonably buggy, and require compiler/target adaptation. Out of my frustration with two out...
6
5679
by: Vyacheslav Maslov | last post by:
Hi all! I have many many many python unit test, which are used for testing some remote web service. The most important issue here is logging of test execution process and result. I strongly need following: 1. start/end timestamp for each test case (most important) 2. immediate report about exceptions (stacktrace) 3. it will be nice to...
0
8249
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
0
8107
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
0
6464
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5631
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
5307
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3748
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
0
3756
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
1352
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
1071
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.