Writing a Coverage Tool for C#

[tjm]

Pure coverage from Rational is the only coverage tool I've been able
to find for C#, and I don't really want to pay what Rational charges
for something as simple as a coverage tool. So, I'd like to try to
write my own, but I'm having trouble getting started. The way I see
it, I need to do the following…

1) Create a debugger object or something similar that will allow me
to step through my code.

2) Start a process to run through my unit tests, and use the debugger
to determine which lines of code execute.

I've looked into the Debug and Debugger classes in the MSDN
documentation, but neither of them seem to offer the functionality I
need. Can anyone tell me how to step through code like the debugger
does so I can determine which lines of code are executed? Any help to
get moving in the right direction on this would be greatly
appreciated.

Refleftion
"tjm" <tj***@rocketmail.com> wrote in message
news:a5**************************@posting.google.c om...

Pure coverage from Rational is the only coverage tool I've been able
to find for C#, and I don't really want to pay what Rational charges
for something as simple as a coverage tool. So, I'd like to try to
write my own, but I'm having trouble getting started. The way I see
it, I need to do the following.

1) Create a debugger object or something similar that will allow me
to step through my code.

2) Start a process to run through my unit tests, and use the debugger
to determine which lines of code execute.

I've looked into the Debug and Debugger classes in the MSDN
documentation, but neither of them seem to offer the functionality I
need. Can anyone tell me how to step through code like the debugger
does so I can determine which lines of code are executed? Any help to
get moving in the right direction on this would be greatly
appreciated.

[Daniel O'Connell]

"tjm" <tj***@rocketmail.com> wrote in message
news:a5**************************@posting.google.c om...

Pure coverage from Rational is the only coverage tool I've been able
to find for C#, and I don't really want to pay what Rational charges
for something as simple as a coverage tool. So, I'd like to try to
write my own, but I'm having trouble getting started. The way I see
it, I need to do the following.

1) Create a debugger object or something similar that will allow me
to step through my code.
Best as I can tell, this isn't possible directly in .NET. cordbg.exe appears
to import from imagehlp.dll for most of its features, you will have to
research how to write debuggers in general.
There is almost certainly no directly available class in the framework to
do this, it may be possible to import the functions(or write in MC++) to
achieve that goal, I don't know. When I have a bit more time I'll see what
else I can find out and I'd appreciate it if you post any further
information you find here.

2) Start a process to run through my unit tests, and use the debugger
to determine which lines of code execute.

I've looked into the Debug and Debugger classes in the MSDN
documentation, but neither of them seem to offer the functionality I
need. Can anyone tell me how to step through code like the debugger
does so I can determine which lines of code are executed? Any help to
get moving in the right direction on this would be greatly
appreciated.

[Daniel Pratt]

Hi tjm,

"tjm" <tj***@rocketmail.com> wrote in message
news:a5**************************@posting.google.c om...

Pure coverage from Rational is the only coverage tool I've been able
to find for C#, and I don't really want to pay what Rational charges
for something as simple as a coverage tool.

<snip>

One possible way to do this would be to "hook" the C# compiler and
insert "logging" calls at the top and bottom of each method in the output
assembly. Such a technique is described here (but for a different purpose):

http://codeproject.com/dotnet/model_...nts_in_net.asp

As you'll see, it's decidedly non-trivial, which could explain why
Rational charges so much.

Regards,
Dan

[Daniel Pratt]

Here's a better article (same technique, though):

"Daniel Pratt" <ko******************@hotmail.com> wrote in message
news:ub**************@TK2MSFTNGP09.phx.gbl...

Hi tjm,

"tjm" <tj***@rocketmail.com> wrote in message
news:a5**************************@posting.google.c om...

Pure coverage from Rational is the only coverage tool I've been able
to find for C#, and I don't really want to pay what Rational charges
for something as simple as a coverage tool. <snip>

One possible way to do this would be to "hook" the C# compiler and
insert "logging" calls at the top and bottom of each method in the output
assembly. Such a technique is described here (but for a different

purpose):
http://codeproject.com/dotnet/model_...nts_in_net.asp

As you'll see, it's decidedly non-trivial, which could explain why
Rational charges so much.

Regards,
Dan

[Daniel Pratt]

Darned itch trigger finger! Anyway...here's a better article (same
technique, though):
http://msdn.microsoft.com/msdnmag/is...f/default.aspx

"Daniel Pratt" <ko******************@hotmail.com> wrote in message
news:ub**************@TK2MSFTNGP09.phx.gbl...

Hi tjm,

"tjm" <tj***@rocketmail.com> wrote in message
news:a5**************************@posting.google.c om...

Pure coverage from Rational is the only coverage tool I've been able
to find for C#, and I don't really want to pay what Rational charges
for something as simple as a coverage tool. <snip>

One possible way to do this would be to "hook" the C# compiler and
insert "logging" calls at the top and bottom of each method in the output
assembly. Such a technique is described here (but for a different

purpose):
http://codeproject.com/dotnet/model_...nts_in_net.asp

As you'll see, it's decidedly non-trivial, which could explain why
Rational charges so much.

Regards,
Dan

[Bruce Benton [MSFT]]

CoreDbg ships as a sample application which is built on the Runtime Debug API set. It supports both managed and native debugging but is primarily for debugging managed code. While it should be possible to implement some form of code coverage assessment tool using the debug API the performance would probably not be very good.

If you are seriously considering implementing your own code coverage tool their are some options but they will pretty much all require some significant compromises. The best way to achieve code coverage is to instrument the binaries of your application under test (as previously mentioned on this thread this is typically done in conjunction with the compiler or as a subsequent pass over the binary/IL code) but it is very technically challenging and time consuming. A modified approach would be to identify a subset of your code as of interest (possibly as simple as function entries) and use debug trace mechanisms to optional log to shared memory when each unique code segment is accessed. I have used techniques similar to this for performance testing of critical pieces of an application. The debug trace code would be compiled out of your shipping code.