Hi all ya unit-testing experts there :)
Code I'm working on has to parse large and complex files and detect
equally complex and large amount of errors before the contents of the file
is fed to module interpreting it. First I created a unit-test class named
TestLoad which loaded, say, 40 files of which about 10 are correct and
other 30 files contained over 20 different types of errors. Different
methods on the TestLoad class were coded so that they expected the
spesific error to occur. They looked like the following:
def test_xxx_err(se lf):
for fname in xxx_err_lst:
self.assertEqua l(get_load_stat us(fname), xxx)
However, now we've added several more tests and we have added another
function to the system - the ability to try loading the file without
actually executing instructions in it. So, I added another class
TestValidate, which has exactly the same tests as the TestLoad class, the
only difference being in the routine get_load_status , which is not
get_load_status but get_validate_st atus.
I thought I'd simplify the process of adding new tests - if it is too
cumbersome to add new tests, we'll end up with having few tests which in
turn will result in more buggy software system.
Thus I came up with the simple idea of adding two directories:
test/correct-files and test/incorrect-files. Every time the test suite is
run, it goes through every file in the test/correct-files and
test/incorrect-files and expects to receive status 0 from all files under
correct-files, and appropriate error code from all files under
test/incorrect-files. How does it deduce the correct error code? Well,
I've forced things to that any file in the test/incorrect-files must
begin with code prefix 'xx-' in the file name, and the test suite reads
expected error code from the file name prefix.
All this allows me to have only two methods (in addition to setUp and
tearDown methods) in TestLoad and TestValidate classes, and what's more
convenient - I don't have to alter unit test suite at all when I add new
file tests to the system; all I have to do is to put it under either
directory, and if the file contains an error, I prefix the file name with
appropriate code.
The system has but one annoyance: first, if I use self.assertEqua l in the
test_load method, it aborts on the first error found and as such doesn't
test all the other files. This would be bad, because when errors occur in
the code, I have to fix them one by one running test suite after every fix
to find other potential errors in the system.
So I changed the test_load routine by replacing self.assertEqua l with my
own code which won't abort the test if errors are found, but it will
report them with sufficient verbosity and continue testing. But this
approach isn't optimal either: because there is no self.assertEqua l in the
test anymore, there may be many errors in the test class but after running
all the methods, the test suite will report OK status, even though errors
did occur.
The problem I described above shows probably one of the best reasons why
each test method should test only one (type of) thing, but in my special
case I don't find that appropriate due to aforementioned reasons. Any
suggestions? I was wondering about creating test methods dynamically
according to the files found in the test directories (which would solve
all the problems), but I find it a tad too complex way of solving the
problem.
--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!
$_ = '45647661726420 4d616a616b61726 92c206120436872 69737469616e20' ; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60 281449,'es'),2, 4),"\n"; 2 2096
I'm not a unit-testing expert, and I don't necessarily recommend what
I'm about to suggest, but...
To solve the problem of having a single "test" actually run many
"internal" tests, report on the results of those many tests, and fail
(in the PyUnit sense) if any of those internal tests fail, you could
add a flag, AllTestsPassed, to the TestLoad and TestValidate classes.
Initialize that flag to True. Then, in your version of assertEqual,
if any test fails set the flag to False. Finally, after you've run
all your tests, add
self.failUnless (self.AllTestsP assed, 'Some tests failed.')
HTH,
-robert
Edvard Majakari <ed*********@ma jakari.net> wrote in message news:<87******* *****@titan.sta selog.com>... Hi all ya unit-testing experts there :)
Code I'm working on has to parse large and complex files and detect equally complex and large amount of errors before the contents of the file is fed to module interpreting it. First I created a unit-test class named TestLoad which loaded, say, 40 files of which about 10 are correct and other 30 files contained over 20 different types of errors. Different methods on the TestLoad class were coded so that they expected the spesific error to occur. They looked like the following:
def test_xxx_err(se lf): for fname in xxx_err_lst: self.assertEqua l(get_load_stat us(fname), xxx)
However, now we've added several more tests and we have added another function to the system - the ability to try loading the file without actually executing instructions in it. So, I added another class TestValidate, which has exactly the same tests as the TestLoad class, the only difference being in the routine get_load_status , which is not get_load_status but get_validate_st atus.
I thought I'd simplify the process of adding new tests - if it is too cumbersome to add new tests, we'll end up with having few tests which in turn will result in more buggy software system.
Thus I came up with the simple idea of adding two directories: test/correct-files and test/incorrect-files. Every time the test suite is run, it goes through every file in the test/correct-files and test/incorrect-files and expects to receive status 0 from all files under correct-files, and appropriate error code from all files under test/incorrect-files. How does it deduce the correct error code? Well, I've forced things to that any file in the test/incorrect-files must begin with code prefix 'xx-' in the file name, and the test suite reads expected error code from the file name prefix.
All this allows me to have only two methods (in addition to setUp and tearDown methods) in TestLoad and TestValidate classes, and what's more convenient - I don't have to alter unit test suite at all when I add new file tests to the system; all I have to do is to put it under either directory, and if the file contains an error, I prefix the file name with appropriate code.
The system has but one annoyance: first, if I use self.assertEqua l in the test_load method, it aborts on the first error found and as such doesn't test all the other files. This would be bad, because when errors occur in the code, I have to fix them one by one running test suite after every fix to find other potential errors in the system.
So I changed the test_load routine by replacing self.assertEqua l with my own code which won't abort the test if errors are found, but it will report them with sufficient verbosity and continue testing. But this approach isn't optimal either: because there is no self.assertEqua l in the test anymore, there may be many errors in the test class but after running all the methods, the test suite will report OK status, even though errors did occur.
The problem I described above shows probably one of the best reasons why each test method should test only one (type of) thing, but in my special case I don't find that appropriate due to aforementioned reasons. Any suggestions? I was wondering about creating test methods dynamically according to the files found in the test directories (which would solve all the problems), but I find it a tad too complex way of solving the problem. fe*****@diablot ech.com (Robert Ferrell) writes: add a flag, AllTestsPassed, to the TestLoad and TestValidate classes. Initialize that flag to True. Then, in your version of assertEqual, if any test fails set the flag to False. Finally, after you've run all your tests, add
self.failUnless (self.AllTestsP assed, 'Some tests failed.')
But that doesn't give instant feedback (running ValidateTest takes over a
minute), neither does it report exact cause of errors. Hmm. I think I'll
look into Python's ability to create methods on the fly.
Thanks anyway!
PS. Sorry for the provoking .signature, it dates back to the Old Times
when I didn't know of Python ;)
--
#!/usr/bin/perl -w
$h={23,69,28,'6 e',2,64,3,76,7, 20,13,61,8,'4d' ,24,73,10,'6a', 12,'6b',21,68,1 4,
72,16,'2c',17,2 0,9,61,11,61,25 ,74,4,61,1,45,2 9,20,5,72,18,61 ,15,69,20,43,26 ,
69,19,20,6,64,2 7,61,22,72};$_= join'',map{chr hex $h->{$_}}sort{$a<= >$b}
keys%$h;m/(\w).*\s(\w+)/x;$_.=uc substr(crypt(jo in('',60,28,14, 49),join'',
map{lc}($1,subs tr $2,4,1)),2,4)." \n"; print; This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Bob |
last post by:
Are they different names for the same concept ?
|
by: Praveen |
last post by:
How to get rid of this error
|
by: Shan |
last post by:
Is there any tool available for generating unit test code in .NET. This is
reverse engineering process, which we have to unit test methods in a class.
Tool such as Nunit can test the methods only if we have unit test classes. Is
there any tool available which can assist in developing these unit test
classes.
|
by: Ben Finney |
last post by:
Howdy all,
My practice when writing unit tests for a project is to make 'test/'
subdirectories for each directory containing modules I want to test.
project-foo/
+-- lib/
| +-- test/
+-- data/
+-- gui/
|
by: Stephan Rose |
last post by:
In one of the applications I am developing I have need of working with
numbers in varying unit system. I might be adding 5 mil to 1 inch or
whatever.
What I have come up with works really well, however...it moves about
as fast as a honda civic trying to tow a semi truck.
The basic definition looks like this:
public struct ScalarGeneric : IScalar
| |
by: AbrahamLincolnIllinois |
last post by:
Greetings!
I am working with a researcher who would be very happy if he could
include units in his calculations. Then, if his complicated expression
didn't result in "kg/yr" the program would stop and point out his
error.
Does Python (or SciPy or ..) offer this feature?
Thanks for your help.
|
by: shuisheng |
last post by:
Dear All,
I am using visual studipo 2005 (standard version which do not provide
unit test tool) to develop some c++ code. I want to do unit test while
I am coding. Anybody can suggest me an easy-to-use tool for unit tests?
I appreciate your suggestion.
Bests,
|
by: Lee |
last post by:
Hi,
Using the propertygrid, is it possible to have a user selectable 'unit
of measure' for a field.
For example, I am looking at using a propertygrid for defining inputs
into a calculation. Some of the inputs can be either monetary or
percantage. The ideal scenario is where the user enters the value and
immediately to the right of the value field is a drop down, defaulted
to %,which allows a currency to be selected.
|
by: batvanio |
last post by:
Hi,
I am writing unit tests in VS2005 and am having the following problem:
I am trying to test a timeout property of one of my methods. This
timeout exhibits itself in an exceptioin - i.e. I am sending a command
to a hardware device, and if it does not answer in the timeout defined
(i.e. in 2 sec), my software fires an exception.
Now, I am writing an unit test in which I start a timer and I send the
command. Note that I DO expect an...
|
by: =?Utf-8?B?cmFuZHkxMjAw?= |
last post by:
I'm working in Visual Studio 2005 Team Edition for Software Developers
I've used the wizard under the Test menu to create a bunch of unit tests.
When I click "Test -Start Selected Test Project with Debugger" the tests
run.
I'd really like to create a WinForm User Interface to run the Unit Tests.
The idea would be to give the user a list of Unit Tests in a table, and let
the user select any combination of tests they like. The user then...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
| |
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
| |
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
|
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |