473,322 Members | 1,566 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

Richards bench benchmark

Is there a Python implementation of Martin Richards benchmark Bench?

Be interesting to add it to these
www.lissett.com/ben/exp/bench1.htm
Jul 18 '05 #1
16 4403
Duncan Lissett wrote:
Is there a Python implementation of Martin Richards benchmark Bench?

Be interesting to add it to these
www.lissett.com/ben/exp/bench1.htm


I doubt one exists yet. There doesn't even appear to be a
specification. It even looks like different implementations of
that benchmark were implemented by random people porting arbitrary
other versions. For example, the Algol68 one comes from Pascal,
and the Pascal one came from C (?).

Worse than that, I can't find a specification for the benchmark
anywhere. What's it supposed to do? Writing this in Python by
reverse engineering a possibly buggy implementation from some other
language would be a wonderful exercise in futility.

There's a hint on http://www.cl.cam.ac.uk/users/mr/Bench.html
(Martin's own page on the subject) which suggests ("designed for the
comparison of System Implementation Languages") that this is not
a suitable benchmark for Python anyway. Python doesn't normally
get used in that domain, and it might be it can't even do what
is required without resorting to C extensions...

My guess is there are more modern benchmarks that are much more
effective at comparing Python performance in areas where it's really
used, but without even a paper describing this one, who could say?

-Peter
Jul 18 '05 #2
Peter Hansen <pe***@engcorp.com> wrote in message news:<jL********************@powergate.ca>...
There's a hint on http://www.cl.cam.ac.uk/users/mr/Bench.html
(Martin's own page on the subject) which suggests ("designed for the
comparison of System Implementation Languages") that this is not
a suitable benchmark for Python anyway. Python doesn't normally
get used in that domain, and it might be it can't even do what
is required without resorting to C extensions...


That would be sad, given that language implementors have used Bench to
explore the performance characteristics of Smalltalk and Self and
Java.

"Benchmarking Java with the Richards benchmark"
http://research.sun.com/people/mario.../richards.html
Jul 18 '05 #3
Duncan Lissett wrote:
Peter Hansen <pe***@engcorp.com> wrote in message news:<jL********************@powergate.ca>...

There's a hint on http://www.cl.cam.ac.uk/users/mr/Bench.html
(Martin's own page on the subject) which suggests ("designed for the
comparison of System Implementation Languages") that this is not
a suitable benchmark for Python anyway. Python doesn't normally
get used in that domain, and it might be it can't even do what
is required without resorting to C extensions...

That would be sad, given that language implementors have used Bench to
explore the performance characteristics of Smalltalk and Self and
Java.

"Benchmarking Java with the Richards benchmark"
http://research.sun.com/people/mario.../richards.html


That has useful additional info I couldn't find on Richards' own site.
(His own site refers to it as "bench", but I guess others call it
"richard" -- it helps to be searching for the right thing.)

It appears the "specification" is his original, which was written
in BCPL. It also seems that the problem is relatively generic,
just a "task scheduler", without really requiring low-level support
of the kind I feared (after looking at the Pascal and C versions).

I'd implement it in Python for kicks, but I'm not sure how I'd know
whether my version was doing the right thing. Are there any tests
for it? Ways of ensuring the output is correct?

-Peter
Jul 18 '05 #4
Peter Hansen <pe***@engcorp.com> wrote in message news:<Et********************@powergate.ca>...
It appears the "specification" is his original, which was written
in BCPL. It also seems that the problem is relatively generic,
just a "task scheduler", without really requiring low-level support
of the kind I feared (after looking at the Pascal and C versions).
Exactly, it's a simulation of a OS kernel task scheduler.
I'd implement it in Python for kicks, but I'm not sure how I'd know
whether my version was doing the right thing. Are there any tests
for it? Ways of ensuring the output is correct?


Martin Richards original spec and debug/trace information is reposted
here:
http://www.lissett.com/ben/com/benchspec.htm

There are many different ways to implement the benchmark, a Python
version of the straightforward OO implementation would be an
interesting place to start. Like these Java C# Smalltalk
implementations:

http://www.lissett.com/ben/exp/bench1-java-62.htm
http://www.lissett.com/ben/exp/bench1-csharp-30.htm
http://www.lissett.com/ben/exp/bench1-smalltalk-34.htm
Jul 18 '05 #5
Duncan Lissett wrote:
Peter Hansen wrote:
I'd implement it in Python for kicks, but I'm not sure how I'd know
whether my version was doing the right thing. Are there any tests
for it? Ways of ensuring the output is correct?


Martin Richards original spec and debug/trace information is reposted
here:
http://www.lissett.com/ben/com/benchspec.htm


Okay, as a starting point I've turned the "gold output" part of that
spec into a first acceptance test, and have a shell bench.py which
runs but fails ('pass' statements tend to do that a lot during TDD).

The code should be available with

svn co svn://fortress.engcorp.com/public/bench

I don't really expect anyone to check it out at this stage, but if you
happen to have a free moment, I wouldn't mind knowing if subversion
is properly visible through my firewall.

-Peter
Jul 18 '05 #6
Duncan Lissett wrote:
Peter Hansen wrote:
I'd implement it in Python for kicks, but I'm not sure how I'd know
whether my version was doing the right thing. Are there any tests
for it? Ways of ensuring the output is correct?


Martin Richards original spec and debug/trace information is reposted
here:
http://www.lissett.com/ben/com/benchspec.htm


Okay, as a starting point I've turned the "gold output" part of that
spec into a first acceptance test, and have a shell bench.py which
runs but fails ('pass' statements tend to do that a lot during TDD).

The code should be available with

svn co svn://fortress.engcorp.com/public/bench

I don't really expect anyone to check it out at this stage, but if you
happen to have a free moment, I wouldn't mind knowing if subversion
is properly visible through my firewall.

-Peter
Jul 18 '05 #7
Duncan Lissett wrote:
Peter Hansen wrote:
I'd implement it in Python for kicks, but I'm not sure how I'd know
whether my version was doing the right thing. Are there any tests
for it? Ways of ensuring the output is correct?


Martin Richards original spec and debug/trace information is reposted
here:
http://www.lissett.com/ben/com/benchspec.htm


Okay, as a starting point I've turned the "gold output" part of that
spec into a first acceptance test, and have a shell bench.py which
runs but fails ('pass' statements tend to do that a lot during TDD).

The code should be available with

svn co svn://fortress.engcorp.com/public/bench

I don't really expect anyone to check it out at this stage, but if you
happen to have a free moment, I wouldn't mind knowing if subversion
is properly visible through my firewall.

-Peter

Jul 18 '05 #8
Peter Hansen <pe***@engcorp.com> wrote in message news:<Ru********************@powergate.ca>...
The code should be available with

svn co svn://fortress.engcorp.com/public/bench

I don't really expect anyone to check it out at this stage, but if you
happen to have a free moment, I wouldn't mind knowing if subversion
is properly visible through my firewall.


What do we use to browse that?
Jul 18 '05 #9
Duncan Lissett wrote:
Peter Hansen <pe***@engcorp.com> wrote in message news:<Ru********************@powergate.ca>...

The code should be available with

svn co svn://fortress.engcorp.com/public/bench

I don't really expect anyone to check it out at this stage, but if you
happen to have a free moment, I wouldn't mind knowing if subversion
is properly visible through my firewall.

What do we use to browse that?


Subversion, the replacement for CVS. See http://subversion.tigris.org/
for a start, and grab the command line version for whatever OS
you are using and after an appropriate installation, the above command
should grab the source into the "public/bench" folder under your current
directory. There's also TortoiseSVN (integrated with Win Explorer
shell) or RapidSVN from the same place.

-Peter
Jul 18 '05 #10
Peter Hansen <pe***@engcorp.com> wrote in message news:<nr********************@powergate.ca>...

Done some interpreter implementations, so Python won't look out of place ;-)

http://www.lissett.com/ben/bench1.htm
Jul 18 '05 #11
Duncan Lissett wrote:
Peter Hansen <pe***@engcorp.com> wrote in message news:<nr********************@powergate.ca>...

Done some interpreter implementations, so Python won't look out of place ;-)

http://www.lissett.com/ben/bench1.htm


Okay, first working version is available by going to
http://www.engcorp.com/main/projects/PyBench .

That's about all the time I can spare on this for now.
Feel free to download, hack, improve, or ignore.

If nothing else, it would be of some interest to have
someone run one of the other versions on a common machine
and post the results. The above page is a wiki, so feel
free to add results there, expand the page, whatever...

-Peter
Jul 18 '05 #12
Peter Hansen <pe***@engcorp.com> wrote in message news:<40**************@engcorp.com>...
Okay, first working version is available by going to
http://www.engcorp.com/main/projects/PyBench .

That's about all the time I can spare on this for now.
Feel free to download, hack, improve, or ignore.


Excellent!

I've added the Python implementation timings:
http://www.lissett.com/ben/bench1.htm

I've no prior experience of Python to judge from, but for some reason
I expected it to be faster than the interpreted implementation of
Smalltalk designed for mobile devices.
Jul 18 '05 #13
Duncan Lissett wrote:
Peter Hansen <pe***@engcorp.com> wrote in message news:<40**************@engcorp.com>...
Okay, first working version is available by going to
http://www.engcorp.com/main/projects/PyBench .

That's about all the time I can spare on this for now.
Feel free to download, hack, improve, or ignore.


Excellent!

I've added the Python implementation timings:
http://www.lissett.com/ben/bench1.htm

I've no prior experience of Python to judge from, but for some reason
I expected it to be faster than the interpreted implementation of
Smalltalk designed for mobile devices.


One key item would be to examine the function calls to append().
I'm guessing they are a substantial portion of the execution
time. A little profiling is in order if anyone is interested
in optimizing it.

I made no effort to convert it to a Pythonic style. It's
basically C code translated, with one small exception being
the task list which is a simple Python list object instead of
being a linked list. That means task lookups by id using findtcb()
are quite fast, but the "work queues" used to hold the packets
are actual singly-linked lists, implemented in Python, with
a subroutine called append() used to scan the list and stick
stuff on the end. This is absolutely, without question, not
the way anyone would write this in Python if they were not
just trying to maintain the original BCPL/C flavour intact.

I'll make a prediction: with an hour or two of work, any
good Python programmer can get a 10x speedup out of the
program. I don't know how meaningful it will be as a
benchmark at that point, but it is probably fair to compare
with the object-oriented versions which already violate the
original specification (e.g. the Java version, and the
Smalltalk ones Duncan mentioned).

If anyone wants to improve it, feel free. There are full
unit tests and an acceptance test in bench_test.py, even
covering the format of the output, so this could be a good
chance for someone new to unit testing to get a feel for
it by trying to modify them to drive the code in a new
direction...

-Peter
Jul 18 '05 #14
Peter Hansen <pe***@engcorp.com> wrote in message news:<40**************@engcorp.com>...
http://www.engcorp.com/main/projects/PyBench .
The above page is a wiki ...


<remark having nothing to do with the original question>

I see that you use ZWiki; I started using it few days ago
and I like it. My question: did you succeed in getting
the testing framework (http://www.zwiki.org/FitTests) to work?
It seems it is very new and it does not work out of the box,
at least for me. Alternatively, which Web-based test framework
(Fitnless-like) would you suggest for Python?

The question is of course open to everybody having expertise
in the area.
</remark having nothing to do with the original question>

Michele Simionato
Jul 18 '05 #15
Michele Simionato wrote:
Peter Hansen <pe***@engcorp.com> wrote in message news:<40**************@engcorp.com>...
http://www.engcorp.com/main/projects/PyBench .
The above page is a wiki ...


<remark having nothing to do with the original question>

I see that you use ZWiki; I started using it few days ago
and I like it. My question: did you succeed in getting
the testing framework (http://www.zwiki.org/FitTests) to work?
It seems it is very new and it does not work out of the box,
at least for me. Alternatively, which Web-based test framework
(Fitnless-like) would you suggest for Python?

The question is of course open to everybody having expertise
in the area.
</remark having nothing to do with the original question>


I use ZWiki because four years ago when we tried to set up an
Intranet at Kaval, we got Zope and ZWiki working very easily
with little effort.

Then we tried to build on that, and improve things, and write
our own products, and we encountered what seemed like a very
tall and steep learning curve with Zope. We never got a whole
lot further with it, but it remains for me one of the simplest
combinations of things available to get a wiki set up
quickly (i.e. a couple of minutes of work, post installation).

You'll notice, perhaps, that I'm using a fairly old version
of ZWiki. I'm loath to update, at this point, partly because
"it's working" and partly because I try to make as few changes
as possible to anything we do with Zope, including attempting
to figure out how to get the "new" testing framework working.
(I say "new" because of course I'm now way behind the times
in this area and most others of Zope.) I'm not particularly
surprised to hear that you didn't get it working yourself.

-Peter
Jul 18 '05 #16
Peter Hansen <pe***@engcorp.com> wrote in message news:<fI********************@powergate.ca>...
I use ZWiki because four years ago when we tried to set up an
Intranet at Kaval, we got Zope and ZWiki working very easily
with little effort.

Then we tried to build on that, and improve things, and write
our own products, and we encountered what seemed like a very
tall and steep learning curve with Zope. We never got a whole
lot further with it, but it remains for me one of the simplest
combinations of things available to get a wiki set up
quickly (i.e. a couple of minutes of work, post installation).

You'll notice, perhaps, that I'm using a fairly old version
of ZWiki. I'm loath to update, at this point, partly because
"it's working" and partly because I try to make as few changes
as possible to anything we do with Zope, including attempting
to figure out how to get the "new" testing framework working.
(I say "new" because of course I'm now way behind the times
in this area and most others of Zope.) I'm not particularly
surprised to hear that you didn't get it working yourself.

-Peter


Well, so far my experience with Zope closely match yours; that's why
I asked here before trying touching anything ;) So, what about my
second question, i.e. other fit tests for Python?
Michele Simionato
Jul 18 '05 #17

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Charles Rush | last post by:
I have some bench time and want to pick up a new language to, perhaps, reduce the amount of bench time in the future. To that end, I've been looking at either the C#/.NET (VB/.NET?) thing with the...
15
by: Duncan Lissett | last post by:
I'd appreciate any suggestions on how to make faster Python implementations of Richards benchmark. Perhaps there are obvious problems that can be corrected? http://www.lissett.com/ben/bench1.htm
0
by: julio | last post by:
Hello Guys, AlphaServer EV6 processor 500mhz/1Gm memory with Linux mysql> SELECT BENCHMARK(1000000,ENCODE("hello","goodbye")); +----------------------------------------------+ |...
3
by: Steve | last post by:
Hi, I'm currently looking a porting our Windows Codewarrior based C++ projects to MinGW and/or Visual Studio and I'm just wondering if there is a 'standard' suite of benchmark programs I can...
0
by: Tarek Ziadé | last post by:
Hi list, I am trying to code some utilities to bench code performances, machine-independant, to be able to measure the absolute cost of a piece of code. So i came up with the pystone idea...
2
by: Matthijs van Waveren | last post by:
"Your opportunity to be a RECOGNIZED EXPERT in the HPC Community" The SPEC High Performance Group is seeking candidates for a benchmark suite based on Message Passing Interface (MPI)...
5
by: A.M-SG | last post by:
Hi, My solution consists of 20 c# projects. When I switch to class view and right click on any class, I don't have the create instance option. I am using visual studio 2005 professional...
16
by: Jorge | last post by:
Webkit r34469 vs. Opera 9.50 : 3.00x as fast 6339.6ms(Opera) 2109.8ms (Webkit) ----- FF3.0 (final) vs. Opera 9.50 : 1.94x as fast 6339.6ms (Opera) 3269.6ms (FF3)
37
by: Jack | last post by:
I know one benchmark doesn't mean much but it's still disappointing to see Python as one of the slowest languages in the test: ...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.