How to use CPPUnit effectively?

[Hooyoo]

How to use CPPUnit effectively?
When I want to use CPPUnit, I always fint it useless and waste my time.
I think that's mainly
because I don't know how to use CPPUnit effectively. So if somebody
here has some experience,
please share it to me.
And also I expect answers to these questions:
1. Should I write test codes for every member function of my class? If
not, which function should be tested?
2. What else should I consider apart from checking parameters and
return value?

Thanks.

[Phlip]

Gianni Mariani wrote:

Hiding the MT nature of a service does not help you design it properly !

You are correct that TDD is _a_ design technique, but it's not the only one.
MT requires more than emergent design. (Not much more, as Ron Jeffries's
paper noted, for one narrow situation.)

>If we are not writing a kernel, we don't need to TDD its event systems.
We won't refactor our kernel, so gaps in our test coverage are very low
risk.

I disagree. I think this is why I have seen so many abortions of MT code,
because they don't take into account the big 3:

1. resource deadlock (not just mutexes)
2. reliable destruction
3. reliable event management

Right. That's neither TDD's fault, nor a good target for TDD's power.

>>Show me what your timer test code looks like ?

TEST_(TestDialog, SetTimer)
{
CButton slowButton = m_aDlg.GetDlgItem(IDC_SLOW_OPERATION);
slowButton.SendMessage(BM_CLICK);
BOOL thereWasaTimerToKill = m_aDlg.KillTimer(0);
CPPUNIT_ASSERT(thereWasaTimerToKill);
}

Not a very useful timer interface, how can I reset the timer ? Can I
safely delete my client and see the timer get deregistered automatically?
Can I set the timer to an absolute time or an interval?

I didn't invent the Win32 timer interface. And we note with sadness that
even high-end OS APIs are often not designed for testing.

The point is, you can't design something using TDD if you don't show how
it's done.

Yes you can, for known algorithms, and for the majority of new code. Legacy
code considerations always interfere. MT is just one of the culprits. Any
kind of pre-existing code will taint the purity of your emergent design. All
this is still no excuse not to TDD your code, and not to emerge your own
designs.

If your interface is dealing with MT issues, there really is little point
in not doing an MT unit test for the purposes on TDD.

Can I get such a test to fail in a way that directs me to write new code?

Regardless of my reluctance to TDD the MT stuff, I suspect the answer is
yes!

--
Phlip
http://www.greencheese.us/ZeekLand <-- NOT a blog!!!

[Gianni Mariani]

Ian Collins wrote:

Gianni Mariani wrote:

>The point is, you can't design something using TDD if you don't show how
it's done. If your interface is dealing with MT issues, there really is
little point in not doing an MT unit test for the purposes on TDD.

I think we'll have to agree to disagree on this point, I still think you
can't write an MT unit test in a TDD context. Why? Because it's all
but impossible to write a single pass MT test that has a 100%
predictable outcome. There are too many variables, how busy the machine
is, which thread starts first, is there more than one CPU available when
the test runs... So you end up getting unexpected and often
unrepeatable test failures.

I thought you argued earlier that unit tests for the purposes of design?
As for your unpredictability argument, umm what are you talking about.
MT code had better have a predictable outcome otherwise there is no
point in MT at all.

>
But you can TDD the logic of an MT object and use a separate Monte Carlo
type acceptance stress test for the MT functionality.

Yes. I've always seen it happen that way, TDD unit test followed by a
more rigorous test. What I am saying is that in the limited number of
interfaces that are MT specific, the TDD unit test case/s should
encompass the complexities of MT, otherwise you're design is likely to
come out incomplete.

[Gianni Mariani]

Phlip wrote:

Gianni Mariani wrote:

>Hiding the MT nature of a service does not help you design it properly !

You are correct that TDD is _a_ design technique, but it's not the only one.
MT requires more than emergent design. (Not much more, as Ron Jeffries's
paper noted, for one narrow situation.)

Yes. The only point I am trying to defend is "in the limited number of
cases where the design is inherently MT, the TDD test case will help you
design better if it demonstrates how to manage it's MT nature of the
design". As for how to design as a whole, that a far more involved
discussion.

>

>>If we are not writing a kernel, we don't need to TDD its event systems.
We won't refactor our kernel, so gaps in our test coverage are very low
risk.

I disagree. I think this is why I have seen so many abortions of MT code,
because they don't take into account the big 3:

1. resource deadlock (not just mutexes)
2. reliable destruction
3. reliable event management

Right. That's neither TDD's fault, nor a good target for TDD's power.

The point I'm trying to make is that this can be considered unless the
TDD test case shows what how the problem is solved.

>

>>>Show me what your timer test code looks like ?
TEST_(TestDialog, SetTimer)
{
CButton slowButton = m_aDlg.GetDlgItem(IDC_SLOW_OPERATION);
slowButton.SendMessage(BM_CLICK);
BOOL thereWasaTimerToKill = m_aDlg.KillTimer(0);
CPPUNIT_ASSERT(thereWasaTimerToKill);
}

Not a very useful timer interface, how can I reset the timer ? Can I
safely delete my client and see the timer get deregistered automatically?
Can I set the timer to an absolute time or an interval?

I didn't invent the Win32 timer interface. And we note with sadness that
even high-end OS APIs are often not designed for testing.

"Testability" is a primary deliverable.

>

>The point is, you can't design something using TDD if you don't show how
it's done.

Yes you can, for known algorithms, and for the majority of new code. Legacy
code considerations always interfere. MT is just one of the culprits. Any
kind of pre-existing code will taint the purity of your emergent design. All
this is still no excuse not to TDD your code, and not to emerge your own
designs.

That's what wrapping code in abstractions is for.

>

> If your interface is dealing with MT issues, there really is little point
in not doing an MT unit test for the purposes on TDD.

Can I get such a test to fail in a way that directs me to write new code?

Yes. I tend to make less mistakes today but just the act of writing the
test case usually helps me realize the more elegant ways of doing things.

>
Regardless of my reluctance to TDD the MT stuff, I suspect the answer is
yes!

MT TDD unit tests should be limited to just the designs that are MT in
nature. e.g. Timers, asynchronous comms layers, low level MT facilities
like threadpools.

I might be wrong but I think we're not saying anything different.

[Ian Collins]

Gianni Mariani wrote:

Ian Collins wrote:

>Gianni Mariani wrote:

>>The point is, you can't design something using TDD if you don't show how
it's done. If your interface is dealing with MT issues, there really is
little point in not doing an MT unit test for the purposes on TDD.

I think we'll have to agree to disagree on this point, I still think you
can't write an MT unit test in a TDD context. Why? Because it's all
but impossible to write a single pass MT test that has a 100%
predictable outcome. There are too many variables, how busy the machine
is, which thread starts first, is there more than one CPU available when
the test runs... So you end up getting unexpected and often
unrepeatable test failures.

I thought you argued earlier that unit tests for the purposes of design?

They are, the design of the logic.

As for your unpredictability argument, umm what are you talking about.
MT code had better have a predictable outcome otherwise there is no
point in MT at all.

I was referring to tests that start threads. On my last project we
started with several of these, by the end they were removed and the
treading layer tested in its own Monte Carlo type test harness.

>>
But you can TDD the logic of an MT object and use a separate Monte Carlo
type acceptance stress test for the MT functionality.

Yes. I've always seen it happen that way, TDD unit test followed by a
more rigorous test. What I am saying is that in the limited number of
interfaces that are MT specific, the TDD unit test case/s should
encompass the complexities of MT, otherwise you're design is likely to
come out incomplete.

Maybe I should revisit TDD and threads. I've been bitten too many times
over the years by tests with multiple threads that I've grown accustomed
to using a different test style for that limited number of interfaces.
It may also be that I've implemented them so many times, I don't have to
design any more!

--
Ian Collins.

[Gianni Mariani]

Ian Collins wrote:

Gianni Mariani wrote:

....

>>
Yes. I've always seen it happen that way, TDD unit test followed by a
more rigorous test. What I am saying is that in the limited number of
interfaces that are MT specific, the TDD unit test case/s should
encompass the complexities of MT, otherwise you're design is likely to
come out incomplete.

Maybe I should revisit TDD and threads. I've been bitten too many times
over the years by tests with multiple threads that I've grown accustomed
to using a different test style for that limited number of interfaces.
It may also be that I've implemented them so many times, I don't have to
design any more!

They are definitely limited in number. e.g. I would not do an MT TDD
test for a matrix library (as an example), even if the library used
threads to parallelize. Unless of course it needed an asynchronous
termination interface to terminate long running computations reliably.

A nice threading or pool interface makes life easier.

[Phlip]

Gianni Mariani wrote:

"Testability" is a primary deliverable.

Yay! Now watch me try to bill hours for it! (To a client familiar with
hack-n-slash web programming ;-)

I might be wrong but I think we're not saying anything different.

I noticed that a while ago but I kept going...

--
Phlip
http://www.greencheese.us/ZeekLand <-- NOT a blog!!!

[VJ]

Phlip wrote:

VJ wrote:

>>We are using http://cxxtest.sourceforge.net/guide.html and it is good and
easy to understand

I don't understand the brief amount of documentation I read.

Given test<3>, do I guess right that we have to increment that 3 by hand,
each time we

I want to smoke crack while coding, and having to remember what number I'm
up to interferes with my lifestyle. I can't even think of a macro that fixes
this (with a compile-time constant).

So does cxxtest really make me increment that number? Or should I, like,
read more of the documentation before passing judgement?

Not sure what you are talking about - I have not read their documentation

You call a function with specific parameters, and check what it outputs

Does a unit test library you are using increment that number for you?

[Phlip]

VJ wrote:

http://cxxtest.sourceforge.net/guide.html

Not sure what you are talking about - I have not read their documentation

I will start again from scratch.

I'm looking for a pattern called "Test Collector".

cxxtest implements it using a Perl script:

cxxtestgen.pl --error-printer -o runner.cpp MyTestSuite.h

That reads the source, pulls out every function named /test*/, and writes
runner.cpp to call them all.

(Parenthetically, I propose that going to another language just to avoid
macros is kind'a pathetic...)

That is still better than the CppUnit system of making you write the name of
each test case twice. (For TDD, test cases must be _absurdly_ easy to add!)

I seem to recall recently reading the documentation of a test runner that
uses templates to do Test Collector. Maybe not; I can't find them on the
above page. No worries.

--
Phlip
http://www.greencheese.us/ZeekLand <-- NOT a blog!!!

[Gianni Mariani]

Phlip wrote:

Gianni Mariani wrote:

>"Testability" is a primary deliverable.

Yay! Now watch me try to bill hours for it! (To a client familiar with
hack-n-slash web programming ;-)

That's ok, keep on milking when they need all those urgent customer
fixes ... sad

>

>I might be wrong but I think we're not saying anything different.

I noticed that a while ago but I kept going...

[Gianni Mariani]

Phlip wrote:
....

I seem to recall recently reading the documentation of a test runner that
uses templates to do Test Collector. Maybe not; I can't find them on the
above page. No worries.

Austria C++'s unit test framework uses the Austria C++ generic factory
to collect all the unit tests into one place at startup. One of the
nice things about this is that you can place your unit tests entirely in
their own anonymous namespace so you can more-or-less cut-n-paste a test
from one file to the other and not have to worry about namespace
collisions. Depending on your debugger, you may or may not want to do this.