473,900 Members | 4,105 Online

# unit testing guidelines

I have compiled a set og unit testing
recommendations based on my own experience
on the concept.

Feedback and suggestions for improvements
are appreciated:

http://geosoft.no/development/unittesting.html

Thanks.
Mar 17 '06
72 5297
Ben Pope wrote:
Randomness just doesn't cut it, and I don't understand how you can check
the output is correct, without knowing the input.

You *do* know the input!

Consider testing this method:

double square(double v)
{
return v * v;
}

Below is a typical unit test that verifies that the
method behaves correctly on typical input:

double v = 2.0;
double v2 = square(v); // You know the input: It is 2.0!
assertEquals(v2 , 4.0);

The same test using random input:

double v = getRandomDouble ();
double v2 = square(v); // You know the input: It is v!
assertEquals(v2 , v*v);

If the test fails, all the details will be in the error
report.

And this method actually *do* fail for a mjority of all
possible inputs (abs of v exceeding sqrt(maxDouble) ).
This will be revealed instantly using the random approach.

For an experienced programmer the limitation of square()
might be obvious so border cases are probably covered
sufficiently in both the code and the test. But for more
complex logic this might not be this apparent and throwing
in random input (in ADDITION to the typical cases and all
obvious border cases) has proven quite helpful, at least
to me.
Mar 25 '06 #31
Jacob:

You've chosen a trivial example where your assert can compute the results of
the Square() function you are calling. That is hardly a typical situation
or there would be no reason for the function to have been created.

double v = getRandomDouble ();
double v2 = AccountBalance( v );
assertEquals( v2, ? );

So explain how you get the value to type in ? given you don't know what the
input will be. Perhaps you would do the computations you read about in the
AccountBalance( ) method inline to see if those and yours matched?
"Jacob" <ja***@yahoo.co m> wrote in message
news:Jt******** ************@te lenor.com...
Ben Pope wrote:
Randomness just doesn't cut it, and I don't understand how you can check
the output is correct, without knowing the input.

You *do* know the input!

Consider testing this method:

double square(double v)
{
return v * v;
}

Below is a typical unit test that verifies that the
method behaves correctly on typical input:

double v = 2.0;
double v2 = square(v); // You know the input: It is 2.0!
assertEquals(v2 , 4.0);

The same test using random input:

double v = getRandomDouble ();
double v2 = square(v); // You know the input: It is v!
assertEquals(v2 , v*v);

If the test fails, all the details will be in the error
report.

And this method actually *do* fail for a mjority of all
possible inputs (abs of v exceeding sqrt(maxDouble) ).
This will be revealed instantly using the random approach.

For an experienced programmer the limitation of square()
might be obvious so border cases are probably covered
sufficiently in both the code and the test. But for more
complex logic this might not be this apparent and throwing
in random input (in ADDITION to the typical cases and all
obvious border cases) has proven quite helpful, at least
to me.

Mar 25 '06 #32
Tom Leylan wrote:
You've chosen a trivial example where your assert can compute the results of
the Square() function you are calling. That is hardly a typical situation
or there would be no reason for the function to have been created.

double v = getRandomDouble ();
double v2 = AccountBalance( v );
assertEquals( v2, ? );

So explain how you get the value to type in ? given you don't know what the
input will be. Perhaps you would do the computations you read about in the
AccountBalance( ) method inline to see if those and yours matched?

I chose a fairly typical example of a basic unit requiring
unit testing and I proved that by using random input it
easily identified an error that otherwise could slip through.

I never said that using random input was useful in all
cases and perhaps it isn't in your specific example. On the
other hand, how do you know what goes into "?" given you
know the input? There must be some sort of reasoning behind

Below is a different example which might not be as trivial
as my previous. It "proves" that encoding + decoding (according
to some procedure) of any string should give back the original
string:

String text = getRandomString (0,1000000); // 0 - 1MB
String encoded = Encoder.encode( text);
String decoded = Encoder.decode( encoded);
assertEquals(te xt, decoded);
Mar 25 '06 #33
Below is a different example which might not be as trivial
as my previous. It "proves" that encoding + decoding (according
to some procedure) of any string should give back the original
string:

String text = getRandomString (0,1000000); // 0 - 1MB
String encoded = Encoder.encode( text);
String decoded = Encoder.decode( encoded);
assertEquals(te xt, decoded);

This only proves that the encoding & decoding scheme is the same. I can
make this test pass like this...

public String encode(String text) {
return text;
}

public String decode(String encoded) {
return encoded;
}

Here, the unit test is testing every line of code, yet its a worthless
implementation.

The unit test is not testing the encoding mechanism as it should be
implemented as its using the decode() for the assertion.

In this case, its prudent/better to have separate unit tests for both
the encode & decode, an doing the opposite conversion locally within the
unit test itself as this prevents errors of implementation but more
importantly, forces the implementation to use the correct algorithms.

public void testEncoder() {
String text = getRandomString (0,1000000); // 0 - 1MB
String encoded = Encoder.encode( text);
String decoded = MD5.decode(enco ded);
assertEquals(te xt, decoded);
}

public void testDecoder() {
String text = getRandomString (0,1000000); // 0 - 1MB
String encoded = MD5.encode(text );
String decoded = Encoder.decode( encoded);
assertEquals(te xt, decoded);
}

Here we have separated the logic needed for the test from the unit under
test.
Mar 25 '06 #34
Forgive me but you are terming it "fairly typical" and it isn't typical of
anything I have seen. So rather than you or I decide (since we don't agree)
let others weigh in on how typical a function is that an easily-contrived
IsPrime() for instance.

The discussion hasn't been (as I read it) that random input is of no value
in all cases. It was illustrated by others that "unit tests" imply one does
know the answer and that random input means you can rarely know the answer.
To answer your question as to how one might know the value of ? it would be
"computed" by whatever method was required. A test for IsPrime() would be
fed known prime and non-prime values safely knowing which ones should return
true and which should return false. A test for AccountBalance( ) would
similarly have inputs and outputs which have been determined to test the
functionality.

I think Andrews response does a great job of pointing out how relying on the
two functions to test each other is a mistake.
"Jacob" <ja***@yahoo.co m> wrote in message
news:4p******** ************@te lenor.com...
Tom Leylan wrote:
You've chosen a trivial example where your assert can compute the results
of the Square() function you are calling. That is hardly a typical
situation or there would be no reason for the function to have been
created.

double v = getRandomDouble ();
double v2 = AccountBalance( v );
assertEquals( v2, ? );

So explain how you get the value to type in ? given you don't know what
the input will be. Perhaps you would do the computations you read about
in the AccountBalance( ) method inline to see if those and yours matched?

I chose a fairly typical example of a basic unit requiring
unit testing and I proved that by using random input it
easily identified an error that otherwise could slip through.

I never said that using random input was useful in all
cases and perhaps it isn't in your specific example. On the
other hand, how do you know what goes into "?" given you
know the input? There must be some sort of reasoning behind

Below is a different example which might not be as trivial
as my previous. It "proves" that encoding + decoding (according
to some procedure) of any string should give back the original
string:

String text = getRandomString (0,1000000); // 0 - 1MB
String encoded = Encoder.encode( text);
String decoded = Encoder.decode( encoded);
assertEquals(te xt, decoded);

Mar 25 '06 #35
Andrew McDonagh wrote:
Below is a different example which might not be as trivial
as my previous. It "proves" that encoding + decoding (according
to some procedure) of any string should give back the original
string:

String text = getRandomString (0,1000000); // 0 - 1MB
String encoded = Encoder.encode( text);
String decoded = Encoder.decode( encoded);
assertEquals(te xt, decoded);
This only proves that the encoding & decoding scheme is the same. I can
make this test pass like this...

public String encode(String text) {
return text;
}

public String decode(String encoded) {
return encoded;
}

Here, the unit test is testing every line of code, yet its a worthless
implementation.

No, an encoding that doesn't do any encoding is still an encoding. It
just happens to be the 'identity' encoding (in the same way that 1 is
the identity for multiplication and 0 is the identity for addition). If
an encoding and decoding function are presented as a pair of opposite
functions, then you are perfectly justified in testing that one is the
inverse of the other:

Decode(Encode(x )) = x

Testing that an encoding is *any good* by whatever means you judge
'good' is entirely a different matter to testing the reversibility of an
encode/decode pair (and harder to do, to boot, although testing that the
encoding did something, anything, to the data is trivial).

The unit test is not testing the encoding mechanism as it should be
implemented as its using the decode() for the assertion.

In this case, its prudent/better to have separate unit tests for both
the encode & decode, an doing the opposite conversion locally within the
unit test itself as this prevents errors of implementation but more
importantly, forces the implementation to use the correct algorithms.

public void testEncoder() {
String text = getRandomString (0,1000000); // 0 - 1MB
String encoded = Encoder.encode( text);
String decoded = MD5.decode(enco ded);
assertEquals(te xt, decoded);
}

public void testDecoder() {
String text = getRandomString (0,1000000); // 0 - 1MB
String encoded = MD5.encode(text );
String decoded = Encoder.decode( encoded);
assertEquals(te xt, decoded);
}

Here we have separated the logic needed for the test from the unit under
test.

Mar 26 '06 #36
Andrew McDonagh wrote:
Here, the unit test is testing every line of code, yet its a worthless
implementation.

The test is still useful. It can't prove that the code is correct,
but if it fails, it can prove that the code is wrong.

And as stated several times already: The test comes in ADDITION to
the test for typical cases and all obvious boundary cases, which in
this particular case would have been written quite differently.
Mar 26 '06 #37
Jacob wrote:
Andrew McDonagh wrote:
Here, the unit test is testing every line of code, yet its a worthless
implementation.

The test is still useful. It can't prove that the code is correct,
but if it fails, it can prove that the code is wrong.

But it cant tell you which of the collaborating en/de coding methods is
the cause.

Mar 26 '06 #38
Tom Leylan wrote:
Forgive me but you are terming it "fairly typical" and it isn't typical of
anything I have seen.
The most typical methods around are getters and setters which
are even less complex than the square example I used previously:

String name = getRandomString (0,1000);
A.setName(name) ;
assertEquals(A. getName(), name);

They are not the most interesting ones to test, but they should
still be tested, and using random input increase the test coverage.
Show me your assertEquals for IsPrime() for instance.

Not the best example I could come up with, but it indicates
the principle:

for (int i = 0; i < 1000; i++) {
int v1 = getRandomInt();
if (isPrime(v1)) {
for (int j = 0; j < 1000; j++) {
int v2 = getRandomInt();
if (isPrime(v2)) {
assertNotEquals (v2 % v1, 0);
assertNotEquals (v1 % v2, 0);
}
}
}
}

Again: It doesn't prove that isPrime() is correct, but it may be able
to prove that it is wrong.
Mar 26 '06 #39
Andrew McDonagh wrote:
Jacob wrote:
Andrew McDonagh wrote:
Here, the unit test is testing every line of code, yet its a
worthless implementation.

The test is still useful. It can't prove that the code is correct,
but if it fails, it can prove that the code is wrong.

But it cant tell you which of the collaborating en/de coding methods is
the cause.

Given an explicit input for which the operation fails should
give you enough information to be able to track this down yourself.
Mar 26 '06 #40

This thread has been closed and replies have been disabled. Please start a new discussion.