472,958 Members | 1,483 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,958 software developers and data experts.

Benchmarking C#, what is the most efficient way of running console code?

I'm doing some benchmarking tests to compare Microsoft's CLR against
Mono's CLR. I could use some suggestions for how to objectively
compare the code. To my surprise the few tests I've run so far have
had Mono running quite a bit faster than Vanilla .NET on my Windows XP
installation, which has me wondering if I'm doing something that's
sandbagging .NET. It's all console code, for Microsoft I'm just
running the .exe file from the command line, and for Mono I'm using
the mono command line to run the same .exe. The .exe file was
generated by Visual Studio 2005 on release mode. Is there anything
further left to do to optimize the running of the exe file under .NET?
Thanks for the help.

Mar 10 '07 #1
7 2390
On Mar 10, 1:01 am, "Wisgary" <wisg...@gmail.comwrote:
I'm doing some benchmarking tests to compare Microsoft's CLR against
Mono's CLR. I could use some suggestions for how to objectively
compare the code. To my surprise the few tests I've run so far have
had Mono running quite a bit faster than Vanilla .NET on my Windows XP
installation, which has me wondering if I'm doing something that's
sandbagging .NET. It's all console code, for Microsoft I'm just
running the .exe file from the command line, and for Mono I'm using
the mono command line to run the same .exe. The .exe file was
generated by Visual Studio 2005 on release mode. Is there anything
further left to do to optimize the running of the exe file under .NET?
Thanks for the help.
Oh yeah, to measure the time between bits of test code I'm using two
DateTime.Nows, one before the test code, one after and then
subtracting and printing the resulting TimeSpan. If there's a better
way to measure time, suggestions for that would be good too. I'm also
wondering if the version of Microsoft's .NET I'm running matters when
it comes to speed if I don't use any library code. I'm not using any
except for measuring the time it takes for a test to run.

Mar 10 '07 #2
Wisgary wrote:
Oh yeah, to measure the time between bits of test code I'm using two
DateTime.Nows, one before the test code, one after and then
subtracting and printing the resulting TimeSpan. If there's a better
way to measure time, suggestions for that would be good too.
Stopwatch w = Stopwatch.StartNew();
// code to time
elapsed time is: w.ElapsedTicks / (double) Stopwatch.Frequency

DateTime.Now has terrible resolution for short intervals. If you use
large loops (i.e. that take several seconds to run), then it won't make
much difference.
I'm also
wondering if the version of Microsoft's .NET I'm running matters when
it comes to speed if I don't use any library code. I'm not using any
except for measuring the time it takes for a test to run.
Sure it matters, why not? Use .NET 2.0 for Stopwatch, it's not in
earlier versions.

-- Barry

--
http://barrkel.blogspot.com/
Mar 10 '07 #3
Thanks for the suggestions, I switched to the Stopwatch and now my
results actually do seem to be more reasonable. Maybe there was a big
difference in the Mono/Microsoft implementations of DateTime that was
lagging everything. Mono is still faster, but not by much, at least on
Windows, on Linux the same program that took 23 seconds to run on
Microsoft's CLR, and 21.6 seconds on Windows Mono actually took around
19.3 seconds, nearly 4 seconds faster. (The test involved really big
linked list sorting). To be fair, these are just pre-tests, when I ran
it on Windows I had a number of applications open while Linux only had
xterm open, so they aren't really representative of anything yet. I'll
try it out cleanly on windows later today. Thanks again for the help
and any other people that can help me out with some suggestions here
is welcome to do so!

On Mar 10, 2:31 am, Barry Kelly <barry.j.ke...@gmail.comwrote:
Wisgary wrote:
Oh yeah, to measure the time between bits of test code I'm using two
DateTime.Nows, one before the test code, one after and then
subtracting and printing the resulting TimeSpan. If there's a better
way to measure time, suggestions for that would be good too.

Stopwatch w = Stopwatch.StartNew();
// code to time
elapsed time is: w.ElapsedTicks / (double) Stopwatch.Frequency

DateTime.Now has terrible resolution for short intervals. If you use
large loops (i.e. that take several seconds to run), then it won't make
much difference.
I'm also
wondering if the version of Microsoft's .NET I'm running matters when
it comes to speed if I don't use any library code. I'm not using any
except for measuring the time it takes for a test to run.

Sure it matters, why not? Use .NET 2.0 for Stopwatch, it's not in
earlier versions.

-- Barry

--http://barrkel.blogspot.com/

Mar 10 '07 #4
You've got to be aware that comparing Mono & the Microsoft .NET CLR isn't
apples to apples.

For example, the Mono Garbage Collector doesn't do heap compaction (at least
not in the versions that I've used). There are quite a bit examples of
things like that that aren't done - so even if it appears faster in some
cases, there are a number of cases where isn't just not yet suitable to use.

In general, for the stuff we've built and run atop of Mono, performance was
"good enough". Long term stability of a process wasn't really there though,
and the frequent versioning that Mono did really caused us nighmares. We
would have code that worked great in one version, and would completly fail
in others.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"Wisgary" <wi*****@gmail.comwrote in message
news:11*********************@p10g2000cwp.googlegro ups.com...
Thanks for the suggestions, I switched to the Stopwatch and now my
results actually do seem to be more reasonable. Maybe there was a big
difference in the Mono/Microsoft implementations of DateTime that was
lagging everything. Mono is still faster, but not by much, at least on
Windows, on Linux the same program that took 23 seconds to run on
Microsoft's CLR, and 21.6 seconds on Windows Mono actually took around
19.3 seconds, nearly 4 seconds faster. (The test involved really big
linked list sorting). To be fair, these are just pre-tests, when I ran
it on Windows I had a number of applications open while Linux only had
xterm open, so they aren't really representative of anything yet. I'll
try it out cleanly on windows later today. Thanks again for the help
and any other people that can help me out with some suggestions here
is welcome to do so!

On Mar 10, 2:31 am, Barry Kelly <barry.j.ke...@gmail.comwrote:
>Wisgary wrote:
Oh yeah, to measure the time between bits of test code I'm using two
DateTime.Nows, one before the test code, one after and then
subtracting and printing the resulting TimeSpan. If there's a better
way to measure time, suggestions for that would be good too.

Stopwatch w = Stopwatch.StartNew();
// code to time
elapsed time is: w.ElapsedTicks / (double) Stopwatch.Frequency

DateTime.Now has terrible resolution for short intervals. If you use
large loops (i.e. that take several seconds to run), then it won't make
much difference.
I'm also
wondering if the version of Microsoft's .NET I'm running matters when
it comes to speed if I don't use any library code. I'm not using any
except for measuring the time it takes for a test to run.

Sure it matters, why not? Use .NET 2.0 for Stopwatch, it's not in
earlier versions.

-- Barry

--http://barrkel.blogspot.com/


Mar 10 '07 #5
On Mar 10, 4:08 pm, "Chris Mullins [MVP]" <cmull...@yahoo.comwrote:
You've got to be aware that comparing Mono & the Microsoft .NET CLR isn't
apples to apples.

For example, the Mono Garbage Collector doesn't do heap compaction (at least
not in the versions that I've used). There are quite a bit examples of
things like that that aren't done - so even if it appears faster in some
cases, there are a number of cases where isn't just not yet suitable to use.

In general, for the stuff we've built and run atop of Mono, performance was
"good enough". Long term stability of a process wasn't really there though,
and the frequent versioning that Mono did really caused us nighmares. We
would have code that worked great in one version, and would completly fail
in others.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVPhttp://www.coversant.com/blogs/cmullins

"Wisgary" <wisg...@gmail.comwrote in message

news:11*********************@p10g2000cwp.googlegro ups.com...
Thanks for the suggestions, I switched to the Stopwatch and now my
results actually do seem to be more reasonable. Maybe there was a big
difference in the Mono/Microsoft implementations of DateTime that was
lagging everything. Mono is still faster, but not by much, at least on
Windows, on Linux the same program that took 23 seconds to run on
Microsoft's CLR, and 21.6 seconds on Windows Mono actually took around
19.3 seconds, nearly 4 seconds faster. (The test involved really big
linked list sorting). To be fair, these are just pre-tests, when I ran
it on Windows I had a number of applications open while Linux only had
xterm open, so they aren't really representative of anything yet. I'll
try it out cleanly on windows later today. Thanks again for the help
and any other people that can help me out with some suggestions here
is welcome to do so!
On Mar 10, 2:31 am, Barry Kelly <barry.j.ke...@gmail.comwrote:
Wisgary wrote:
Oh yeah, to measure the time between bits of test code I'm using two
DateTime.Nows, one before the test code, one after and then
subtracting and printing the resulting TimeSpan. If there's a better
way to measure time, suggestions for that would be good too.
Stopwatch w = Stopwatch.StartNew();
// code to time
elapsed time is: w.ElapsedTicks / (double) Stopwatch.Frequency
DateTime.Now has terrible resolution for short intervals. If you use
large loops (i.e. that take several seconds to run), then it won't make
much difference.
I'm also
wondering if the version of Microsoft's .NET I'm running matters when
it comes to speed if I don't use any library code. I'm not using any
except for measuring the time it takes for a test to run.
Sure it matters, why not? Use .NET 2.0 for Stopwatch, it's not in
earlier versions.
-- Barry
--http://barrkel.blogspot.com/
Yeah, I'm not sure if this is related to the heap compaction absence
problem you mentioned, but when I scaled up the test and added a few
million extra items to my linked list, .NET worked fine but Mono
started blurting out heap errors and crashing. Had to tone it down a
bit so it would run at all.

Mar 10 '07 #6
Wisgary wrote:
I'm doing some benchmarking tests to compare Microsoft's CLR against
Mono's CLR. I could use some suggestions for how to objectively
compare the code. To my surprise the few tests I've run so far have
had Mono running quite a bit faster than Vanilla .NET on my Windows XP
installation, which has me wondering if I'm doing something that's
sandbagging .NET. It's all console code, for Microsoft I'm just
running the .exe file from the command line, and for Mono I'm using
the mono command line to run the same .exe. The .exe file was
generated by Visual Studio 2005 on release mode. Is there anything
further left to do to optimize the running of the exe file under .NET?
That surprises me.

In all my tests MS .NET has been significantly faster than Mono.

But as usual with benchmarks then specific code used for the benchmark
van produce almost any result. If you post the code, then you may get
a more specific answer.

Arne
Mar 17 '07 #7
Barry Kelly wrote:
Wisgary wrote:
>Oh yeah, to measure the time between bits of test code I'm using two
DateTime.Nows, one before the test code, one after and then
subtracting and printing the resulting TimeSpan. If there's a better
way to measure time, suggestions for that would be good too.

Stopwatch w = Stopwatch.StartNew();
// code to time
elapsed time is: w.ElapsedTicks / (double) Stopwatch.Frequency

DateTime.Now has terrible resolution for short intervals. If you use
large loops (i.e. that take several seconds to run), then it won't make
much difference.
The test should take many seconds to be good anyway ...

Arne

Mar 17 '07 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Andy Dingley | last post by:
I've just started on a new project and inherited a huge pile of XSLT (and I use the term "pile" advisedly !) It runs at glacial speed, and I need to fix this this. Platform is MSXML 4 / ASP ...
0
by: Jerry Coffin | last post by:
This is more or less related to the (now huge) thread about Java and C++ performance. It's been noted that the C++ involved in that benchmark is hardly how most people would actually write code...
8
by: Jerry Coffin | last post by:
As promised, more benchmarking results of comparing C++ to Java. This time around, our first target will be the strcat program. This is one that the C++ version rather bothered me -- I'm...
1
by: | last post by:
I was viewing some strange behaviour in c# so I tried a comparison in vb.net A simple 1 line console application written in both c# and vb.net (One line meaning one line in the main function). ...
4
by: Robert Vasquez | last post by:
I'm currently trying to decide the best way to design my application, any suggestions or comments welcomed. In C++ to make my apps more efficient (consume less memory while running) I would...
4
by: Anders | last post by:
Hi, I was wondering what is most efficient of the two. Is it more efficient to add server controls within the Itemtemplate and use OnItemDataBound to manipulate and databind the servercontrols. ...
10
by: Michel Rouzic | last post by:
I need to determine how long does an addition take and how long does a multiplication takes. So far I've been trying to use the clock() function in my programs in order to find out how long it took...
2
by: Bjarne | last post by:
Dear all, although being a user of PHP and other scripting-languages for years, I have not taken part in any large projects based on PHP. Thus, I don't really know how real PHP-applications are...
2
by: chitranjan | last post by:
Hello All, I want to do Bechmarking of PostgreSQL on Linux (fedora) so,please tell me about benchmarking tools of postgresql which is easy to use and implement..It is very urgent for me to do...
2
by: DJRhino | last post by:
Was curious if anyone else was having this same issue or not.... I was just Up/Down graded to windows 11 and now my access combo boxes are not acting right. With win 10 I could start typing...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 4 Oct 2023 starting at 18:00 UK time (6PM UTC+1) and finishing at about 19:15 (7.15PM) The start time is equivalent to 19:00 (7PM) in Central...
0
by: Aliciasmith | last post by:
In an age dominated by smartphones, having a mobile app for your business is no longer an option; it's a necessity. Whether you're a startup or an established enterprise, finding the right mobile app...
0
tracyyun
by: tracyyun | last post by:
Hello everyone, I have a question and would like some advice on network connectivity. I have one computer connected to my router via WiFi, but I have two other computers that I want to be able to...
4
NeoPa
by: NeoPa | last post by:
Hello everyone. I find myself stuck trying to find the VBA way to get Access to create a PDF of the currently-selected (and open) object (Form or Report). I know it can be done by selecting :...
3
NeoPa
by: NeoPa | last post by:
Introduction For this article I'll be using a very simple database which has Form (clsForm) & Report (clsReport) classes that simply handle making the calling Form invisible until the Form, or all...
3
by: nia12 | last post by:
Hi there, I am very new to Access so apologies if any of this is obvious/not clear. I am creating a data collection tool for health care employees to complete. It consists of a number of...
0
NeoPa
by: NeoPa | last post by:
Introduction For this article I'll be focusing on the Report (clsReport) class. This simply handles making the calling Form invisible until all of the Reports opened by it have been closed, when it...
2
by: GKJR | last post by:
Does anyone have a recommendation to build a standalone application to replace an Access database? I have my bookkeeping software I developed in Access that I would like to make available to other...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.