Question on uniti testing results...

[Tom Olson]

I have written several tests for a unit and they run fine, but I do notice
extreme differences in performance when I only run one test versus running
every test in the unit.

The class that is being tested has several static methods. When I run all
the tests in this unit the timing (captured through QueryPerformanceCount)
returns in 130-140 ticks. However if I just run this test by itself the
same process takes 8800-8900 ticks.

Has anyone else encountered this?

Tom

[Jon Skeet [C# MVP]]

Tom Olson <to**@tomo.com> wrote:

I have written several tests for a unit and they run fine, but I do notice
extreme differences in performance when I only run one test versus running
every test in the unit.

The class that is being tested has several static methods. When I run all
the tests in this unit the timing (captured through QueryPerformanceCount)
returns in 130-140 ticks. However if I just run this test by itself the
same process takes 8800-8900 ticks.

Has anyone else encountered this?

Do you mean that just that test takes 130-140 ticks? If so, the rest of
the ticks are probably taken with creating the test fixture, loading
libraries etc.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

[Tom Olson]

Jon,

The timings of all the test are pretty consistent, however when any of them
are run individually the run time skyrockets. When run as a batch none of
the test show this spike.

"Jon Skeet [C# MVP]" wrote:

Tom Olson <to**@tomo.com> wrote:

I have written several tests for a unit and they run fine, but I do notice extreme differences in performance when I only run one test versus running every test in the unit.

The class that is being tested has several static methods. When I run all the tests in this unit the timing (captured through QueryPerformanceCount) returns in 130-140 ticks. However if I just run this test by itself the
same process takes 8800-8900 ticks.

Has anyone else encountered this?

Do you mean that just that test takes 130-140 ticks? If so, the rest of
the ticks are probably taken with creating the test fixture, loading
libraries etc.
...

[Marcus Andrén]

On Thu, 28 Apr 2005 22:13:57 -0500, "Tom Olson" <to**@tomo.com> wrote:

I have written several tests for a unit and they run fine, but I do notice
extreme differences in performance when I only run one test versus running
every test in the unit.

The class that is being tested has several static methods. When I run all
the tests in this unit the timing (captured through QueryPerformanceCount)
returns in 130-140 ticks. However if I just run this test by itself the
same process takes 8800-8900 ticks.

Has anyone else encountered this?

Tom

Have you made sure that all methods have been run atleast once
(including the queryperformance methods), before you do the
performance test. Methods aren't JIT compiled until they are accessed
for the first time and JIT compiling takes a good amount of time.

--
Marcus Andrén

[Tom Olson]

Marcus,

I'll give that a try, but wouldn't the timing of at least one of the test be
askew even when run as a batch?

<Marcus Andrén> :

On Thu, 28 Apr 2005 22:13:57 -0500, "Tom Olson" <to**@tomo.com> wrote:

I have written several tests for a unit and they run fine, but I do noticeextreme differences in performance when I only run one test versus runningevery test in the unit.

The class that is being tested has several static methods. When I run allthe tests in this unit the timing (captured through QueryPerformanceCount)returns in 130-140 ticks. However if I just run this test by itself the
same process takes 8800-8900 ticks.

Has anyone else encountered this?

Tom

Have you made sure that all methods have been run atleast once
(including the queryperformance methods), before you do the
performance test. Methods aren't JIT compiled until they are accessed
for the first time and JIT compiling takes a good amount of time.

--
Marcus Andrén