473,320 Members | 2,145 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

testing units in a specific order?

I have used unit tests now for a number of project. One thing
that I dislike is it that the order in which the tests are done
bears no relationship to the order they appear in the source.

This makes using unit tests somewhat cumbersome. Is there some
way to force the tests being done in a particular order?

--
Antoon Pardon




















Jan 9 '06 #1
8 1281
You could use py.test

Jan 9 '06 #2
Disclaimer: I am a contributor to the py-lib, of which py.test is part
of (but have not worked on py.test until now).

Michele Simionato wrote:
You could use py.test


Indeed. It has exactly this feature (together with many nifty others):
all tests are run in the order they appear in the test-file. The website
of the py-lib is at

http://codespeak.net/py

the documentation page for py.test is

http://codespeak.net/py/current/doc/test.html

If you like it so much (and I am being optimistic here :-) that you want
to switch tests that use the stdlib unittest module over to py.test you
can use a script called utestconvert.py which converts the unittest
syntax over to py.test syntax. It can be found in the tool directory of
the py-lib.

Cheers,

Carl Friedrich Bolz

Jan 9 '06 #3
Carl Friedrich Bolz wrote:
If you like it so much (and I am being optimistic here :-) that you want
to switch tests that use the stdlib unittest module over to py.test you
can use a script called utestconvert.py which converts the unittest
syntax over to py.test syntax. It can be found in the tool directory of
the py-lib.


This looks new. Can you comment on how utestconvert.py work and how
reliable is it?
Thanks,
Michele Simionato

Jan 9 '06 #4
Hi Michele!

Michele Simionato wrote:
This looks new. Can you comment on how utestconvert.py work and how
reliable is it?


It is rather old, but was never advertised much. PyPy used it to convert
all its unit tests from unittests to py.test, when py.test was finished.
The idea is quite simple, the script does a stupid rewrite of the
typical unittest-syntax to py.test syntax:

self.assertEqual(x, y) --> assert x == y
self.assert_(some_expression) --> assert some_expression

and so on. It works very well in "regular" unittest code that does not
use any special unittests extensions. It might break for more
complicated cases but (especially custom unittest extensions), as I
said, it worked well with PyPy's unittests (which are quite many).

In general, I think that py.test will at one point grow a collector for
doctest/unittest test cases. We plan to do that since quite some time
now (it is even part an issue in our tracker) but we currently don't
have the resources to actually do so. Contributions are of course
welcome ;-)

Cheers,

Carl Friedrich Bolz

Jan 9 '06 #5
[Antoon Pardon]
I have used unit tests now for a number of project. One thing
that I dislike is it that the order in which the tests are done
bears no relationship to the order they appear in the source.

This makes using unit tests somewhat cumbersome. Is there some
way to force the tests being done in a particular order?


They're run in alphabetical order, sorting on the test methods' names.
For that reason some people name test methods like 'test_001',
'test_002', ..., although unit tests really "shouldn't" case which
order they get run in. Sometimes this is abused in a different way,
by naming a setup kind of method starting with AAA and its
corresponding teardown kind of method with zzz.

You could presumably change the sort order by subclassing TestLoader
and overriding its class-level .sortTestMethodsUsing attribute (which
is `cmp` in TestLoader). Sounds painful and useless to me, though ;-)
The source-code order isn't available in any case (unittest doesn't
analyze source code).
Jan 9 '06 #6
Tim Peters <ti********@gmail.com> writes:
They're run in alphabetical order, sorting on the test methods' names.
For that reason some people name test methods like 'test_001',
'test_002', ..., although unit tests really "shouldn't" case which
order they get run in.


This seems sort of hard to do with "good" OO design. I.e. - I have
some object foo with the requirement that "foo.setup" be run before
"foo.process". I typically do unit tests to run foo.setup first, and
the test for foo.process assumes that foo.setup has been run.

The alternative would have the setup for "foo.process" run "foo.setup"
as part of the prep. But that assumes that foo.setup worked
properly. To make sure of that, I still need the unit test for
foo.setup to run before the test for foo.process.

I supposed the test would be better if the test of foo.process didn't
expect foo.setup to be run first. But in that case, if foo.setup
fails, running foo.process is only moderatly interesting - a failure
may be a cascade from the failure of setup, and passing the test
may not mean it'll pass if setup actually finishes properly.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
Jan 10 '06 #7
Op 2006-01-09, Tim Peters schreef <ti********@gmail.com>:
[Antoon Pardon]
I have used unit tests now for a number of project. One thing
that I dislike is it that the order in which the tests are done
bears no relationship to the order they appear in the source.

This makes using unit tests somewhat cumbersome. Is there some
way to force the tests being done in a particular order?


They're run in alphabetical order, sorting on the test methods' names.
For that reason some people name test methods like 'test_001',
'test_002', ..., although unit tests really "shouldn't" case which
order they get run in.


Well maybe unit tests shouldn't care (Thats what I think you meant),
I care. Some methods are vital for the functionality of other methods.
So it the test for the first method fails it is very likely a number of
other methods will fail too. However I'm not interrested in the results
of those other tests in that case. Having to weed through all the test
results in order to check first if the vital methods are working before
checking other methods is cumbersome.

Having the vital methods tested first and ignore the rest of the results
if they fail is much easier.

--
Antoon Pardon
Jan 10 '06 #8
[Antoon Pardon]
Well maybe unit tests shouldn't care (Thats what I think you meant),
Yup!
I care. Some methods are vital for the functionality of other methods.
So it the test for the first method fails it is very likely a number of
other methods will fail too. However I'm not interrested in the results
of those other tests in that case. Having to weed through all the test
results in order to check first if the vital methods are working before
checking other methods is cumbersome.

Having the vital methods tested first and ignore the rest of the results
if they fail is much easier.


So put the tests for the different kinds of methods into different
test classes, and run the corresponding test suites in the order you
want them to run. This is easy. Code like:

test_classes = [FileStorageConnectionTests,
FileStorageReconnectionTests,
FileStorageInvqTests,
FileStorageTimeoutTests,
MappingStorageConnectionTests,
MappingStorageTimeoutTests]

def test_suite():
suite = unittest.TestSuite()
for klass in test_classes:
suite.addTest(unittest.makeSuite(klass))
return suite

is common in large projects. unittest runs tests added to a suite in
the order you add them (although _within_ a test class, the test
methods are run in alphabetical order of method name -- when you want
ordering, that's the wrong level to try to force it; forcing order is
natural & easy at higher levels).
Jan 11 '06 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
by: lig | last post by:
hi , i am trying to do something that is not working (cuz i tried that) so i hope for some ideas about it. ok, so i have this recursive function, lets call it recFun and i wanted to be able to...
3
by: KatB | last post by:
Hi, Is it possible, and how?, to append a child element into a specific order. For example: <stations> <station name="one"/> <station name="three"/> <station name="four"/> </stations>
12
by: Jozef | last post by:
Hello, Does anyone here know for sure, when you do a For Each loop on a forms controls collection, does Access cycle through them in Alphabetical order or control ID or ???. Any ideas? ...
8
by: clintonG | last post by:
Could someone provide me with a URL documenting the specific order of exceptions? -- <%= Clinton Gallagher A/E/C Consulting, Web Design, e-Commerce Software Development Wauwatosa, Milwaukee...
1
by: mrmagoo | last post by:
Is it possible to view enum values in the order that I have defined them? Public Enum Size Small Medium Large Biggest End Enum I love Enums because they make my life easier, but, because...
8
by: spohle | last post by:
hi i have a normal dictionary with key and value pairs. now i wanna sort by the keys BUT in a specific order i determine in a list !? any ideas dic = {'key1':'value1', 'key2':'value2',...
3
by: Hartmut Dippon | last post by:
Hi all, I hope somebody can help me with following problem: I have an application where I can drag&drop files/dirs from within explorer onto my form. If multiple files/dirs are selected I...
1
by: meettapan | last post by:
Anyone please suggest me that......... When we send multiple PDF documents from a loop for printing using Process class, the printing is not taking place in the order of documents created. ...
1
by: Achtland | last post by:
Hi all, I'm rather new to XSL (and this forum) and have searched everywhere to find out how to order the output of XML to text using XSLT but haven't found the answer. The XML is similar to...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.