By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,506 Members | 1,891 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,506 IT Pros & Developers. It's quick & easy.

Richards bench benchmark

P: n/a
Is there a Python implementation of Martin Richards benchmark Bench?

Be interesting to add it to these
www.lissett.com/ben/exp/bench1.htm
Jul 18 '05 #1
Share this Question
Share on Google+
16 Replies


P: n/a
Duncan Lissett wrote:
Is there a Python implementation of Martin Richards benchmark Bench?

Be interesting to add it to these
www.lissett.com/ben/exp/bench1.htm


I doubt one exists yet. There doesn't even appear to be a
specification. It even looks like different implementations of
that benchmark were implemented by random people porting arbitrary
other versions. For example, the Algol68 one comes from Pascal,
and the Pascal one came from C (?).

Worse than that, I can't find a specification for the benchmark
anywhere. What's it supposed to do? Writing this in Python by
reverse engineering a possibly buggy implementation from some other
language would be a wonderful exercise in futility.

There's a hint on http://www.cl.cam.ac.uk/users/mr/Bench.html
(Martin's own page on the subject) which suggests ("designed for the
comparison of System Implementation Languages") that this is not
a suitable benchmark for Python anyway. Python doesn't normally
get used in that domain, and it might be it can't even do what
is required without resorting to C extensions...

My guess is there are more modern benchmarks that are much more
effective at comparing Python performance in areas where it's really
used, but without even a paper describing this one, who could say?

-Peter
Jul 18 '05 #2

P: n/a
Peter Hansen <pe***@engcorp.com> wrote in message news:<jL********************@powergate.ca>...
There's a hint on http://www.cl.cam.ac.uk/users/mr/Bench.html
(Martin's own page on the subject) which suggests ("designed for the
comparison of System Implementation Languages") that this is not
a suitable benchmark for Python anyway. Python doesn't normally
get used in that domain, and it might be it can't even do what
is required without resorting to C extensions...


That would be sad, given that language implementors have used Bench to
explore the performance characteristics of Smalltalk and Self and
Java.

"Benchmarking Java with the Richards benchmark"
http://research.sun.com/people/mario.../richards.html
Jul 18 '05 #3

P: n/a
Duncan Lissett wrote:
Peter Hansen <pe***@engcorp.com> wrote in message news:<jL********************@powergate.ca>...

There's a hint on http://www.cl.cam.ac.uk/users/mr/Bench.html
(Martin's own page on the subject) which suggests ("designed for the
comparison of System Implementation Languages") that this is not
a suitable benchmark for Python anyway. Python doesn't normally
get used in that domain, and it might be it can't even do what
is required without resorting to C extensions...

That would be sad, given that language implementors have used Bench to
explore the performance characteristics of Smalltalk and Self and
Java.

"Benchmarking Java with the Richards benchmark"
http://research.sun.com/people/mario.../richards.html


That has useful additional info I couldn't find on Richards' own site.
(His own site refers to it as "bench", but I guess others call it
"richard" -- it helps to be searching for the right thing.)

It appears the "specification" is his original, which was written
in BCPL. It also seems that the problem is relatively generic,
just a "task scheduler", without really requiring low-level support
of the kind I feared (after looking at the Pascal and C versions).

I'd implement it in Python for kicks, but I'm not sure how I'd know
whether my version was doing the right thing. Are there any tests
for it? Ways of ensuring the output is correct?

-Peter
Jul 18 '05 #4

P: n/a
Peter Hansen <pe***@engcorp.com> wrote in message news:<Et********************@powergate.ca>...
It appears the "specification" is his original, which was written
in BCPL. It also seems that the problem is relatively generic,
just a "task scheduler", without really requiring low-level support
of the kind I feared (after looking at the Pascal and C versions).
Exactly, it's a simulation of a OS kernel task scheduler.
I'd implement it in Python for kicks, but I'm not sure how I'd know
whether my version was doing the right thing. Are there any tests
for it? Ways of ensuring the output is correct?


Martin Richards original spec and debug/trace information is reposted
here:
http://www.lissett.com/ben/com/benchspec.htm

There are many different ways to implement the benchmark, a Python
version of the straightforward OO implementation would be an
interesting place to start. Like these Java C# Smalltalk
implementations:

http://www.lissett.com/ben/exp/bench1-java-62.htm
http://www.lissett.com/ben/exp/bench1-csharp-30.htm
http://www.lissett.com/ben/exp/bench1-smalltalk-34.htm
Jul 18 '05 #5

P: n/a
Duncan Lissett wrote:
Peter Hansen wrote:
I'd implement it in Python for kicks, but I'm not sure how I'd know
whether my version was doing the right thing. Are there any tests
for it? Ways of ensuring the output is correct?


Martin Richards original spec and debug/trace information is reposted
here:
http://www.lissett.com/ben/com/benchspec.htm


Okay, as a starting point I've turned the "gold output" part of that
spec into a first acceptance test, and have a shell bench.py which
runs but fails ('pass' statements tend to do that a lot during TDD).

The code should be available with

svn co svn://fortress.engcorp.com/public/bench

I don't really expect anyone to check it out at this stage, but if you
happen to have a free moment, I wouldn't mind knowing if subversion
is properly visible through my firewall.

-Peter
Jul 18 '05 #6

P: n/a
Duncan Lissett wrote:
Peter Hansen wrote:
I'd implement it in Python for kicks, but I'm not sure how I'd know
whether my version was doing the right thing. Are there any tests
for it? Ways of ensuring the output is correct?


Martin Richards original spec and debug/trace information is reposted
here:
http://www.lissett.com/ben/com/benchspec.htm


Okay, as a starting point I've turned the "gold output" part of that
spec into a first acceptance test, and have a shell bench.py which
runs but fails ('pass' statements tend to do that a lot during TDD).

The code should be available with

svn co svn://fortress.engcorp.com/public/bench

I don't really expect anyone to check it out at this stage, but if you
happen to have a free moment, I wouldn't mind knowing if subversion
is properly visible through my firewall.

-Peter
Jul 18 '05 #7

P: n/a
Duncan Lissett wrote:
Peter Hansen wrote:
I'd implement it in Python for kicks, but I'm not sure how I'd know
whether my version was doing the right thing. Are there any tests
for it? Ways of ensuring the output is correct?


Martin Richards original spec and debug/trace information is reposted
here:
http://www.lissett.com/ben/com/benchspec.htm


Okay, as a starting point I've turned the "gold output" part of that
spec into a first acceptance test, and have a shell bench.py which
runs but fails ('pass' statements tend to do that a lot during TDD).

The code should be available with

svn co svn://fortress.engcorp.com/public/bench

I don't really expect anyone to check it out at this stage, but if you
happen to have a free moment, I wouldn't mind knowing if subversion
is properly visible through my firewall.

-Peter

Jul 18 '05 #8

P: n/a
Peter Hansen <pe***@engcorp.com> wrote in message news:<Ru********************@powergate.ca>...
The code should be available with

svn co svn://fortress.engcorp.com/public/bench

I don't really expect anyone to check it out at this stage, but if you
happen to have a free moment, I wouldn't mind knowing if subversion
is properly visible through my firewall.


What do we use to browse that?
Jul 18 '05 #9

P: n/a
Duncan Lissett wrote:
Peter Hansen <pe***@engcorp.com> wrote in message news:<Ru********************@powergate.ca>...

The code should be available with

svn co svn://fortress.engcorp.com/public/bench

I don't really expect anyone to check it out at this stage, but if you
happen to have a free moment, I wouldn't mind knowing if subversion
is properly visible through my firewall.

What do we use to browse that?


Subversion, the replacement for CVS. See http://subversion.tigris.org/
for a start, and grab the command line version for whatever OS
you are using and after an appropriate installation, the above command
should grab the source into the "public/bench" folder under your current
directory. There's also TortoiseSVN (integrated with Win Explorer
shell) or RapidSVN from the same place.

-Peter
Jul 18 '05 #10

P: n/a
Peter Hansen <pe***@engcorp.com> wrote in message news:<nr********************@powergate.ca>...

Done some interpreter implementations, so Python won't look out of place ;-)

http://www.lissett.com/ben/bench1.htm
Jul 18 '05 #11

P: n/a
Duncan Lissett wrote:
Peter Hansen <pe***@engcorp.com> wrote in message news:<nr********************@powergate.ca>...

Done some interpreter implementations, so Python won't look out of place ;-)

http://www.lissett.com/ben/bench1.htm


Okay, first working version is available by going to
http://www.engcorp.com/main/projects/PyBench .

That's about all the time I can spare on this for now.
Feel free to download, hack, improve, or ignore.

If nothing else, it would be of some interest to have
someone run one of the other versions on a common machine
and post the results. The above page is a wiki, so feel
free to add results there, expand the page, whatever...

-Peter
Jul 18 '05 #12

P: n/a
Peter Hansen <pe***@engcorp.com> wrote in message news:<40**************@engcorp.com>...
Okay, first working version is available by going to
http://www.engcorp.com/main/projects/PyBench .

That's about all the time I can spare on this for now.
Feel free to download, hack, improve, or ignore.


Excellent!

I've added the Python implementation timings:
http://www.lissett.com/ben/bench1.htm

I've no prior experience of Python to judge from, but for some reason
I expected it to be faster than the interpreted implementation of
Smalltalk designed for mobile devices.
Jul 18 '05 #13

P: n/a
Duncan Lissett wrote:
Peter Hansen <pe***@engcorp.com> wrote in message news:<40**************@engcorp.com>...
Okay, first working version is available by going to
http://www.engcorp.com/main/projects/PyBench .

That's about all the time I can spare on this for now.
Feel free to download, hack, improve, or ignore.


Excellent!

I've added the Python implementation timings:
http://www.lissett.com/ben/bench1.htm

I've no prior experience of Python to judge from, but for some reason
I expected it to be faster than the interpreted implementation of
Smalltalk designed for mobile devices.


One key item would be to examine the function calls to append().
I'm guessing they are a substantial portion of the execution
time. A little profiling is in order if anyone is interested
in optimizing it.

I made no effort to convert it to a Pythonic style. It's
basically C code translated, with one small exception being
the task list which is a simple Python list object instead of
being a linked list. That means task lookups by id using findtcb()
are quite fast, but the "work queues" used to hold the packets
are actual singly-linked lists, implemented in Python, with
a subroutine called append() used to scan the list and stick
stuff on the end. This is absolutely, without question, not
the way anyone would write this in Python if they were not
just trying to maintain the original BCPL/C flavour intact.

I'll make a prediction: with an hour or two of work, any
good Python programmer can get a 10x speedup out of the
program. I don't know how meaningful it will be as a
benchmark at that point, but it is probably fair to compare
with the object-oriented versions which already violate the
original specification (e.g. the Java version, and the
Smalltalk ones Duncan mentioned).

If anyone wants to improve it, feel free. There are full
unit tests and an acceptance test in bench_test.py, even
covering the format of the output, so this could be a good
chance for someone new to unit testing to get a feel for
it by trying to modify them to drive the code in a new
direction...

-Peter
Jul 18 '05 #14

P: n/a
Peter Hansen <pe***@engcorp.com> wrote in message news:<40**************@engcorp.com>...
http://www.engcorp.com/main/projects/PyBench .
The above page is a wiki ...


<remark having nothing to do with the original question>

I see that you use ZWiki; I started using it few days ago
and I like it. My question: did you succeed in getting
the testing framework (http://www.zwiki.org/FitTests) to work?
It seems it is very new and it does not work out of the box,
at least for me. Alternatively, which Web-based test framework
(Fitnless-like) would you suggest for Python?

The question is of course open to everybody having expertise
in the area.
</remark having nothing to do with the original question>

Michele Simionato
Jul 18 '05 #15

P: n/a
Michele Simionato wrote:
Peter Hansen <pe***@engcorp.com> wrote in message news:<40**************@engcorp.com>...
http://www.engcorp.com/main/projects/PyBench .
The above page is a wiki ...


<remark having nothing to do with the original question>

I see that you use ZWiki; I started using it few days ago
and I like it. My question: did you succeed in getting
the testing framework (http://www.zwiki.org/FitTests) to work?
It seems it is very new and it does not work out of the box,
at least for me. Alternatively, which Web-based test framework
(Fitnless-like) would you suggest for Python?

The question is of course open to everybody having expertise
in the area.
</remark having nothing to do with the original question>


I use ZWiki because four years ago when we tried to set up an
Intranet at Kaval, we got Zope and ZWiki working very easily
with little effort.

Then we tried to build on that, and improve things, and write
our own products, and we encountered what seemed like a very
tall and steep learning curve with Zope. We never got a whole
lot further with it, but it remains for me one of the simplest
combinations of things available to get a wiki set up
quickly (i.e. a couple of minutes of work, post installation).

You'll notice, perhaps, that I'm using a fairly old version
of ZWiki. I'm loath to update, at this point, partly because
"it's working" and partly because I try to make as few changes
as possible to anything we do with Zope, including attempting
to figure out how to get the "new" testing framework working.
(I say "new" because of course I'm now way behind the times
in this area and most others of Zope.) I'm not particularly
surprised to hear that you didn't get it working yourself.

-Peter
Jul 18 '05 #16

P: n/a
Peter Hansen <pe***@engcorp.com> wrote in message news:<fI********************@powergate.ca>...
I use ZWiki because four years ago when we tried to set up an
Intranet at Kaval, we got Zope and ZWiki working very easily
with little effort.

Then we tried to build on that, and improve things, and write
our own products, and we encountered what seemed like a very
tall and steep learning curve with Zope. We never got a whole
lot further with it, but it remains for me one of the simplest
combinations of things available to get a wiki set up
quickly (i.e. a couple of minutes of work, post installation).

You'll notice, perhaps, that I'm using a fairly old version
of ZWiki. I'm loath to update, at this point, partly because
"it's working" and partly because I try to make as few changes
as possible to anything we do with Zope, including attempting
to figure out how to get the "new" testing framework working.
(I say "new" because of course I'm now way behind the times
in this area and most others of Zope.) I'm not particularly
surprised to hear that you didn't get it working yourself.

-Peter


Well, so far my experience with Zope closely match yours; that's why
I asked here before trying touching anything ;) So, what about my
second question, i.e. other fit tests for Python?
Michele Simionato
Jul 18 '05 #17

This discussion thread is closed

Replies have been disabled for this discussion.