473,883 Members | 1,761 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

unit testing guidelines

I have compiled a set og unit testing
recommendations based on my own experience
on the concept.

Feedback and suggestions for improvements
are appreciated:

http://geosoft.no/development/unittesting.html

Thanks.
Mar 17 '06
72 5295
Andrew McDonagh wrote:
Ian Collins wrote:
Jacob wrote:
Ian Collins wrote:

I'd add point 0 - write the tests first.
Personaly find the XP approach to unit testing a bit too restrictive

I find debugging a bit too restrictive. I can't just use Undo to make the
bug go >poof<.

Imagine if you had such a button on your debugger! You would hit it all the
time!

You have such a button; it's just a little more expensive than raw code. The
cost savings - no more debugging - overwhelmingly offsets that cost.
TDD is more than an approach to unit testing, it is an approach to the
full design-test-code cycle.

More fundamentally, TDD is Design Methodology, Not a Testing Methodology.

It just happens to use Unit tests as its means of describing the design,
much like RUP uses UML.

Indeed, some TDD practitioners are starting to call it BDD - as in

http://www.google.co.uk/search?hl=en...e+Search&meta=

And some call it Test First Programming, because TDD is position to replace
the hideous name "eXtreme Programming".

And it doesn't create "unit tests", which are a different topic entirely.

The failure of a unit test implicates only one unit - such as the Ariane V
engine controller.

The failure of a _Developer_ Test implicates the developer's last edit. Time
to hit Undo.
TDD done well will give you 100% execution coverage for free.


That's not exhaustive.

TDD done well will reduce the _odds_ that you need exhaustive unit testing.

--
Phlip
http://www.greencheese.org/ZeekLand <-- NOT a blog!!!
Mar 20 '06 #21
Phlip wrote:
Andrew McDonagh wrote:
Ian Collins wrote:
Jacob wrote:
Ian Collins wrote:

> I'd add point 0 - write the tests first.

Personaly find the XP approach to unit testing a bit too restrictive

Phlip, your mail client must have removed to much cause Ian wrote this
'too restrictive' part not me.

I find debugging a bit too restrictive. I can't just use Undo to make the
bug go >poof<.

Imagine if you had such a button on your debugger! You would hit it all the
time!

You have such a button; it's just a little more expensive than raw code. The
cost savings - no more debugging - overwhelmingly offsets that cost.
100% agree and its how I work too - make a change, it doesn't work, undo
change, make a different change, rinse & repeat.

TDD is more than an approach to unit testing, it is an approach to the
full design-test-code cycle.


More fundamentally, TDD is Design Methodology, Not a Testing Methodology.

It just happens to use Unit tests as its means of describing the design,
much like RUP uses UML.

Indeed, some TDD practitioners are starting to call it BDD - as in

http://www.google.co.uk/search?hl=en...e+Search&meta=

And some call it Test First Programming, because TDD is position to replace
the hideous name "eXtreme Programming".


;-) If only Kent had his way
Mar 20 '06 #22
Andrew McDonagh wrote:
Phlip wrote:
Andrew McDonagh wrote:
Ian Collins wrote:
Jacob wrote:
> Ian Collins wrote:
>
>> I'd add point 0 - write the tests first.
>
> Personaly find the XP approach to unit testing a bit too restrictive
Phlip, your mail client must have removed to much cause Ian wrote this
'too restrictive' part not me.
When I snip, I leave in the replied-to person, and sometimes leave in the
previous repliers. But I don't change the >>>, so count them.

(We must hope readers understand the ticks and don't come after you for
others' heretical statements...)
100% agree and its how I work too - make a change, it doesn't work, undo
change, make a different change, rinse & repeat.
If we could put that feeling into a dartgun...
And some call it Test First Programming, because TDD is position to
replace the hideous name "eXtreme Programming".

;-) If only Kent had his way


Uh, it's his idea, and it's why XPX2 uses TFP instead of TDD.

--
Phlip
http://www.greencheese.org/ZeekLand <-- NOT a blog!!!
Mar 20 '06 #23
Andrew McDonagh wrote:

I am not sure it does, and I wanted to define the two concepts
"execution coverage" and "test coverage" anyway. There is a blurred
distinction between the two in the literature as far as I have been
able to dig up.

TDD done well will give you 100% execution coverage for free.

I'd clarify that with 'TDD done *Correctly will give you 100% execution
coverage'

*Correctly = Write 1 failing Testcase,
Write only enough code to make test Pass,
Refactor to Remove Duplication,
Repeat

More commonly referred to as Red, Green, Refactor.
How good your test coverage is depends on how good you are at thinking
up edge cases to test.

Always starting with the test first, only allows for 100%.

I was using the OP's definition of "test coverage". It might just be
me, but I've always had testers or users (normally testers) find some
bizarre use case that wasn't catered for in the original user stories or
unit tests.

--
Ian Collins.
Mar 21 '06 #24
Ian Collins wrote:
I was using the OP's definition of "test coverage". It might just be me,
but I've always had testers or users (normally testers) find some bizarre
use case that wasn't catered for in the original user stories or unit
tests.


That's why, regardless of your unit testing strategy, you work to lower the
cost of acceptance tests, so anyone can write them, and they come up with
all sorts of things.

Hence all of XP is driven by tests.

--
Phlip
http://www.greencheese.org/ZeekLand <-- NOT a blog!!!
Mar 21 '06 #25
Ian Collins wrote:
Andrew McDonagh wrote:
Always starting with the test first, only allows for 100%.

I was using the OP's definition of "test coverage". It might just be
me, but I've always had testers or users (normally testers) find some
bizarre use case that wasn't catered for in the original user stories or
unit tests.


But we are talking about unit testing here - developers write and run
unit tests.

Users/testers don't unit test - they Acceptance (integration, System) Test.

Everyone runs the acceptance tests.
Mar 21 '06 #26
Jacob wrote:
I have compiled a set og unit testing
recommendations based on my own experience
on the concept.

Feedback and suggestions for improvements
are appreciated:

http://geosoft.no/development/unittesting.html

Thanks.

At no point do you suggest that both the code design and testing should
be subject to independent review, preferably by another programmer and a
user. This can often help spot potential problems early on. The
developer can frequently miss something, if only because their too close
and as the cliche goes "You can't see the wood for the trees".

Hope you approve, from an unemployed C++ developer

JB
Mar 22 '06 #27

"Jacob" <ja***@yahoo.co m> wrote:
Adam Maass wrote:
I strongly object to number 13. Unit-tests, especially in an automated
framework, should be repeatable. (When a test fails, you need to know on
what inputs it failed. Once you fix the failure, you should hard-code the
inputs it failed on so that subsequent changes do not cause a regression
of the error.)


I understand your objection, but this is actually one of the
mechanisms that have helped me found some of the hardest to
trace and most subtle errors in the code. It has proven to be
extremely helpful. Also, it gives me lots of confidence
knowing that my test suite of several thousand tests
are executed every hour with different input each time.
It is like adding another dimension to unit testing.


There is sometimes value in testing on large numbers of random inputs. But
this isn't *unit* testing; it's more akin to a system or stress test. It's
something you hope your QAs will do for you; test on inputs that you weren't
necessarily expecting and see what breaks. Unit testing is about the
correctness of code for known inputs. If you come across a failure for a
novel set of inputs in system or stress testing, by all means, take that
input and add it to your unit test suite.

Note that test frameworks can be used both for unit tests as well as other
kinds of tests. (Simply because it's called 'JUnit', for example, does not
necessarily mean that all the test cases are, in fact, unit tests.)

-- Adam Maass
Mar 23 '06 #28
Adam Maass wrote:
There is sometimes value in testing on large numbers of random inputs. But
this isn't *unit* testing; it's more akin to a system or stress test. It's
something you hope your QAs will do for you; test on inputs that you weren't
necessarily expecting and see what breaks. Unit testing is about the
correctness of code for known inputs.


Which definition of unit testing is this? I have searched the
net but hasn't been able to find any backing for this?

If I write a method void setLength(doubl e length), who define
the input "necesseril y expected", and why isn't this the entire
double range? I'd claim the latter and to cover as many inputs
as possible I use the random trick.

I don't have a problem with defining this kind of testing
differently, for instance "stress testing", but on the other
hand there isn't really any more "stress" is calling
setLength(1.230 76e+307) than setLength(2.0) as long as the
method accepts a double as input?

And why do you care about "known" input as long as the
actual (failing) input can be traced afterwards anyway?

You define this as a unit test:

for (int i = 0; i < 1000; i++)
testMyIntMethod (i);

while this is not:

for (int i = 0; i < 1000; i++)
testMyIntMethod (getRandomInt() )

even if an error on input=42 will produce identical error reports
in both cases. Only the latter will (eventually) reveal the
error for input=-100042.

Also, if I have a setLength() method which cover the "typical"
input cases just fine, but is in general crap (a common scenario),
then a testSetLength() method that verifies that setLength() work
fine for "typical" input isn't worth a lot. What you need is a test
method that test the non-typical inputs. From a black-box perspective
you don't really know what is typical or non-typical, so why not just
throw a random number genrator at it?
Mar 23 '06 #29
Jacob wrote:
Adam Maass wrote:
There is sometimes value in testing on large numbers of random inputs.
But this isn't *unit* testing; it's more akin to a system or stress
test. It's something you hope your QAs will do for you; test on inputs
that you weren't necessarily expecting and see what breaks. Unit
testing is about the correctness of code for known inputs.
Which definition of unit testing is this? I have searched the
net but hasn't been able to find any backing for this?

If I write a method void setLength(doubl e length), who define
the input "necesseril y expected", and why isn't this the entire
double range? I'd claim the latter and to cover as many inputs
as possible I use the random trick.


The programmer specifies the preconditions. If the preconditions are
not met, there is no reason for it to produce valid results. Random is
not repeatable, and is not predictable.
I don't have a problem with defining this kind of testing
differently, for instance "stress testing", but on the other
hand there isn't really any more "stress" is calling
setLength(1.230 76e+307) than setLength(2.0) as long as the
method accepts a double as input?
No, but the point is that when you unit test, you need to make informed
choices about the inputs you choose. It's usually wise to throw in a
couple of "normal", "everyday" values, but also explicitly check
boundary cases and out of range.
And why do you care about "known" input as long as the
actual (failing) input can be traced afterwards anyway?
Repeatability. It's no use relying on randomness to thoroughly test.
You have to design your test cases.
You define this as a unit test:

for (int i = 0; i < 1000; i++)
testMyIntMethod (i);
Not really. What are you testing? That it doesn't crash? Presumably
you need to check the output against an array of 1000 pre-computed
values? Not much fun.
while this is not:

for (int i = 0; i < 1000; i++)
testMyIntMethod (getRandomInt() )
How can you possibly check the output is correct for a random input?
even if an error on input=42 will produce identical error reports
in both cases. Only the latter will (eventually) reveal the
error for input=-100042.
I don't understand, are you checking for crashing?
Also, if I have a setLength() method which cover the "typical"
input cases just fine, but is in general crap (a common scenario),
then a testSetLength() method that verifies that setLength() work
fine for "typical" input isn't worth a lot. What you need is a test
method that test the non-typical inputs. From a black-box perspective
you don't really know what is typical or non-typical, so why not just
throw a random number genrator at it?


You know your preconditions. You know your postconditions.

If anything can happen on invalid input, then no point in testing. If
you want a default output, exception or whatever for out-of-range, then
check it with a unit test. You get to choose your input and you get to
check your output.

Randomness just doesn't cut it, and I don't understand how you can check
the output is correct, without knowing the input.

Ben Pope
--
I'm not just a number. To many, I'm known as a string...
Mar 23 '06 #30

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

14
2767
by: | last post by:
Hi! I'm looking for unit-testing tools for .NET. Somthing like Java has --> http://www.junit.org regards, gicio
5
6537
by: shuisheng | last post by:
Dear All, I was told that unit test is a powerful tool for progamming. If I am writing a GUI code, is it possible to still using unit test? I have a little experience in using unittest++. But I can not work out a way to use it to test GUI code. Thanks a lot!
0
11153
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10757
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
10420
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
9583
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
7975
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
7134
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
6002
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4620
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
4225
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.