New browsers : SunSpider JavaScript Benchmark Results.

[Jorge]

Webkit r34469 vs. Opera 9.50 :

3.00x as fast 6339.6ms(Opera) 2109.8ms (Webkit)

-----

FF3.0 (final) vs. Opera 9.50 :

1.94x as fast 6339.6ms (Opera) 3269.6ms (FF3)

-----

Webkit r34469 vs. FF3.0 (final) :

1.55x as fast 3269.6ms(FF3) 2109.8ms(Webkit)

-----

http://webkit.org/perf/sunspider-0.9/sunspider.html

--Jorge.

[Lasse Reichstein Nielsen]

Jorge <jo***@jorgechamorro.comwrites:

3.00x as fast 6339.6ms(Opera) 2109.8ms (Webkit)

....

1.94x as fast 6339.6ms (Opera) 3269.6ms (FF3)

....

1.55x as fast 3269.6ms(FF3) 2109.8ms(Webkit)

....

http://webkit.org/perf/sunspider-0.9/sunspider.html

Numbers on their own are meaningless. What were the configurations
used to perform the test? OS, CPU, etc? Any other pages open in the
browser? Was Opera's mail client enabled? Was WebKit used in
a browser (e.g. Safari) or run as a GUI-less batch job?

And sorry, but I don't trust a benchmark hosted on the site of the one
performing best at that benchmark. For all we know, Webkit could have
been microoptimized for exactly the tested tasks, and suck at
everything else. Probably not, but it wouldn't be the first time
something like that happened.
Is the sunspider test suite available for download and perusal, or
should we just read the page source?

/L
--
Lasse Reichstein Nielsen - lr*@hotpop.com
DHTML Death Colors: <URL:http://www.infimum.dk/HTML/rasterTriangleDOM.html>
'Faith without judgement merely degrades the spirit divine.'

[Gregor Kofler]

Jorge meinte:

Webkit r34469 vs. FF3.0 (final) :

1.55x as fast 3269.6ms(FF3) 2109.8ms(Webkit)

Safari 3.1.1 vs. FF3 (WinXP): 4909ms vs. 5602ms

Anyway, how useful is a JS benchmark nowadays, which deliberately leaves
out DOM perfomance? Since the benchmark is by Apple, I don't expect
anything but "lightning fast" performance by their in-house browser.

Gregor


--
http://photo.gregorkofler.at ::: Landschafts- und Reisefotografie
http://web.gregorkofler.com ::: meine JS-Spielwiese
http://www.image2d.com ::: Bildagentur für den alpinen Raum

[Jorge]

On Jun 18, 10:13*am, Lasse Reichstein Nielsen <[e-mail]:

Numbers on their own are meaningless.

Why ? The results show that it runs x times faster/slower on my
computer.
All the browsers were tested on the same computer.

What were the configurations
used to perform the test? OS, CPU, etc? Any other pages open in the
browser? *Was Opera's mail client enabled? Was WebKit used in
a browser (e.g. Safari) or run as a GUI-less batch job?

Browser : a browser is a browser. Webkit is a browser as well.
Open the browser, run the test. That's it. Nothing special. Try it
yourself.

>
And sorry, but I don't trust a benchmark hosted on the site of the one
performing best at that benchmark. For all we know, Webkit could have
been microoptimized for exactly the tested tasks, and suck at
everything else. Probably not, but it wouldn't be the first time
something like that happened.
Is the sunspider test suite available for download and perusal, or
should we just read the page source?

Hey, what's up ?
Take it easy...

Thanks,
--Jorge.

[Jorge]

On Jun 18, 10:12*am, Gregor Kofler <[e-mail]:

Anyway, how useful is a JS benchmark nowadays, which deliberately leaves
out DOM perfomance?

Is JS core performance a don't care, then ?

Since the benchmark is by Apple, I don't expect
anything but "lightning fast" performance by their in-house browser.

Not so.
They must have been working hard lately,
these tests show no lightning-fast-performance here : look :

http://mozillalinks.org/wp/2008/02/f...e-performance/

That was in February. 15 days later :

http://mozillalinks.org/wp/2008/03/u...pt-benchmarks/

And today we are where we are :

Webkit r34469 vs. FF3.0 (final) :
1.55x as fast 3269.6ms(FF3) 2109.8ms(Webkit)

--Jorge.

[Gregor Kofler]

Jorge meinte:

On Jun 18, 10:12 am, Gregor Kofler <[e-mail]:

>Anyway, how useful is a JS benchmark nowadays, which deliberately leaves
out DOM perfomance?

Is JS core performance a don't care, then ?

No. But only a fraction of the speed of a JS application running in a
browser window will be contributed by the core performance. Typical "Web
2.0" applications rely heavily on DOM performance.

>Since the benchmark is by Apple, I don't expect
anything but "lightning fast" performance by their in-house browser.

Not so.
They must have been working hard lately,
these tests show no lightning-fast-performance here : look :

http://mozillalinks.org/wp/2008/02/f...e-performance/

That was in February. 15 days later :

http://mozillalinks.org/wp/2008/03/u...pt-benchmarks/

And today we are where we are :

Webkit r34469 vs. FF3.0 (final) :
1.55x as fast 3269.6ms(FF3) 2109.8ms(Webkit)

One could argue, that they've tweaked their browser to outperform
competitors with their home-made benchmark...
Anyway, on WinXP the current Safari version is 10 to 15% faster than FF3
running a benchmark that measures core JS performance. IMO nothing to
write home about.

Gregor
--
http://photo.gregorkofler.at ::: Landschafts- und Reisefotografie
http://web.gregorkofler.com ::: meine JS-Spielwiese
http://www.image2d.com ::: Bildagentur für den alpinen Raum

[Jorge]

On Jun 18, 11:33*am, Gregor Kofler <[e-mail]:

Anyway, on WinXP the current Safari version is 10 to 15% faster than FF3
running a benchmark that measures core JS performance. IMO nothing to
write home about.

But you might be glad to learn that

-FF is almost as fast as the fastest.

-all of them are performing much better than before. (all but IE).

--Jorge.

[Lasse Reichstein Nielsen]

Jorge <jo***@jorgechamorro.comwrites:

On Jun 18, 10:13*am, Lasse Reichstein Nielsen <[e-mail]:

>Numbers on their own are meaningless.

Why ? The results show that it runs x times faster/slower on my
computer.

That *what* runs faster? To understand the numbers, we need to know
how they were produced. E.g., how many other applications were running
at the time, how many other pages were open in the same browser, etc.
In other words, we need to be able to reproduce the setting and test
the numbers.
Sorry, I'm a trained scientist, pedantry is an occupational hazerd.

All the browsers were tested on the same computer.

Good. That's the least to require.

>What were the configurations
used to perform the test? OS, CPU, etc? Any other pages open in the
browser? *Was Opera's mail client enabled? Was WebKit used in
a browser (e.g. Safari) or run as a GUI-less batch job?

Browser : a browser is a browser.

Meh. A browser component running in a thin wrapper doesn't have the
memory footprint or other concurrent threads to handle that a full
fledged, highly skinnable browser+mail agent+news reader+irc
client+download manager does.

Webkit is a browser as well.

Webkit, if I read correctly, is a browser component. I.e., it only
hadles the display of the page. Safari is a browser that is build
on Webkit, just as Internet Explorer is a browser build on the
Microsoft Browser Component, and Firefox is build on the

Open the browser, run the test. That's it. Nothing special. Try it
yourself.

If I did that on my browser, I would open a browser with 30+ tabs,
a mail client checking mail and an IRC client connecting to a channel.
It would be likely to give worse results than if I opened a clean profile
with everything fancy disabled and no pages open.

I don't know that what you did, which is why I say that the number is
meaningless.

You could easily have done everything perfectly, eliminating as many
spurious influences as at all possible, but then again, you might
not. It would be equally wrong of me to assume either, which is why I
merely say that the numbers, by themselves, are meaningless.

>And sorry, but I don't trust a benchmark hosted on the site of the one
performing best at that benchmark.

.... [and more of the same] ...

Hey, what's up ?

Distrust of benchmarks, mainly :)
I'm remembering things like
http://www.geek.com/ati-driver-caugh...ark03-as-well/
and have a natural distrust of any benchmark that comes from the same
source as the products that scores highest on the benchmark.
It just smells like selective sampling. In science, procedures to avoid
selective sampling are introduced not only to prevent cheating, but
also, just as importantly, to remove subcontious bias.
In that case, the producers should make a serious effort to convince
people that is a balanced and generic benchmark. That hasn't happened
here (to my knowledge).

Take it easy...

Oh, I do :)

/L
--
Lasse Reichstein Nielsen - lr*@hotpop.com
DHTML Death Colors: <URL:http://www.infimum.dk/HTML/rasterTriangleDOM.html>
'Faith without judgement merely degrades the spirit divine.'

[Lasse Reichstein Nielsen]

Lasse Reichstein Nielsen <lr*@hotpop.comwrites:

....

on Webkit, just as Internet Explorer is a browser build on the
Microsoft Browser Component, and Firefox is build on the

Gecko browser component.

(Note to self: remember to finish sentences before moving on)
/L
--
Lasse Reichstein Nielsen - lr*@hotpop.com
DHTML Death Colors: <URL:http://www.infimum.dk/HTML/rasterTriangleDOM.html>
'Faith without judgement merely degrades the spirit divine.'

[Jorge]

On Jun 18, 8:26*pm, Lasse Reichstein Nielsen <[e-mail]:

That *what* runs faster? To understand the numbers, we need to know
how they were produced. E.g., how many other applications were running
at the time, how many other pages were open in the same browser, etc.
In other words, we need to be able to reproduce the setting and test
the numbers.

On the same computer, in the same circumstances, open a browser, run
the test, close it. Open another browser, run the test, close it.
Repeat for the third browser. Paste in the results and note the
difference.

The numbers obtained serve very well to *compare* one browser against
the other. I posted : browser a ran x times faster than browser b,
etc.
That's not meaningless, I think.
It does not matter whether the actual number was 5000mS, what matters
is that 2xmS is twice as much as xmS... no ?
And (x times faster) applies most likely to most other computers
running the same OS, as well.
The numbers themselves are meaningless if you intend to compare them
against results obtained in another computer/OS/whatever, yes.

I'm glad to see that JS performance for the last generation of
browsers is several times better. Don't you ?

Thanks,
--Jorge.

[Jorge]

On Jun 18, 8:26*pm, Lasse Reichstein Nielsen <[e-mail]:

Webkit, if I read correctly, is a browser component. I.e., it only
hadles the display of the page. Safari is a browser that is build
on Webkit, just as Internet Explorer is a browser build on the
Microsoft Browser Component, and Firefox is build on the Gecko browser.

The non-release versions Safari are usually called "Webkit nightly
builds". If you go to nightly.webkit.org what you download is there an
app, that is a browser, that looks and behaves like Safari, but it's
called WebKit not Safari (it's name is WebKit). I used the name in
that sense.

Thanks,
--Jorge.

[Richard Levasseur]

On Jun 18, 12:51*pm, Jorge <[e-mail]:

On Jun 18, 8:26*pm, Lasse Reichstein Nielsen <[e-mail]:

Webkit, if I read correctly, is a browser component. I.e., it only
hadles the display of the page. Safari is a browser that is build
on Webkit, just as Internet Explorer is a browser build on the
Microsoft Browser Component, and Firefox is build on the Gecko browser.

The non-release versions Safari are usually called "Webkit nightly
builds". If you go to nightly.webkit.org what you download is there an
app, that is a browser, that looks and behaves like Safari, but it's
called WebKit not Safari (it's name is WebKit). I used the name in
that sense.

Thanks,
--Jorge.

I recently found this article: http://www.sitepen.com/blog/2008/05/...e-an-analysis/

I think it highlights the concerns people have expressed. Something
innocuous in one browser is a horrible kludge in another. JS tests
can be very sensitive, too. Simple things like adding a setTimeout()
call between test runs can make a difference.

That isn't to say the tests are completely useless, but saying safari
is 2x as opera, and 50% faster than firefox becomes questionable.
The best you can really do is say Safari >= FF Opera, but without
any quantitative measurements.

Richard

[Jorge]

On Jun 19, 10:22*am, Richard Levasseur <[e-mail]:

>
I recently found this article:http://www.sitepen.com/blog/2008/05/...e-an-analysis/

I think it highlights the concerns people have expressed. *Something
innocuous in one browser is a horrible kludge in another. *JS tests
can be very sensitive, too. *Simple things like adding a setTimeout()
call between test runs can make a difference.

And the getTime() accuracy as well. There's a browser out there that
increments it in 16mS steps... I heard.

Do you say "take it with a grain of salt" ?

Yes, the results shown are a mean value, and all the consequences of
being an average apply.
But if you want you can dig deeper and see "what really happenned" and
have a look at the invidual tests results as well :
============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total: 2247.6ms +/- 7.6%
--------------------------------------------

3d: 321.6ms +/- 23.2%
cube: 106.4ms +/- 23.3%
morph: 109.0ms +/- 36.4%
raytrace: 106.2ms +/- 10.4%

access: 325.6ms +/- 9.0%
binary-trees: 39.6ms +/- 10.8%
fannkuch: 105.2ms +/- 4.0%
nbody: 143.2ms +/- 19.3%
nsieve: 37.6ms +/- 21.6%

bitops: 223.2ms +/- 4.3%
3bit-bits-in-byte: 33.0ms +/- 6.0%
bits-in-byte: 43.8ms +/- 22.7%
bitwise-and: 63.0ms +/- 3.9%
nsieve-bits: 83.4ms +/- 12.1%

controlflow: 24.4ms +/- 12.8%
recursive: 24.4ms +/- 12.8%

crypto: 148.8ms +/- 12.7%
aes: 52.0ms +/- 5.6%
md5: 48.6ms +/- 23.6%
sha1: 48.2ms +/- 10.5%

date: 206.8ms +/- 8.4%
format-tofte: 120.8ms +/- 6.0%
format-xparb: 86.0ms +/- 12.2%

math: 240.0ms +/- 13.7%
cordic: 82.4ms +/- 11.3%
partial-sums: 109.8ms +/- 15.4%
spectral-norm: 47.8ms +/- 15.9%

regexp: 214.8ms +/- 0.8%
dna: 214.8ms +/- 0.8%

string: 542.4ms +/- 1.9%
base64: 84.0ms +/- 1.5%
fasta: 95.4ms +/- 1.5%
tagcloud: 130.2ms +/- 2.8%
unpack-code: 135.8ms +/- 3.9%
validate-input: 97.0ms +/- 3.4%

There there's and empty field to paste in other browser's results so
that you can compare them side-by-side, line-by-line, test-by-test.

I doubt that the tests themselves are architected to benefit WebKit's
results.
As JS is necessarily open-source, how would you hide such a thing ?

Thanks,
--Jorge.

[Dr J R Stockton]

In comp.lang.javascript message <870207f7-b830-409e-bc94-cd694ceea498@m3
6g2000hse.googlegroups.com>, Thu, 19 Jun 2008 02:27:01, Jorge
<jo***@jorgechamorro.composted:

>
And the getTime() accuracy as well. There's a browser out there that
increments it in 16mS steps... I heard.

Standard is that 16mS means 16 milliSiemens, which is a conductance
corresponding to about 63 Ohms. Doubtless you mean what should be
written as 16 ms - the approximate duration of one cycle of the mains in
hasty countries.

In Win98, I got intervals of 54.9 ms, rounded to 50 or 60. The
underlying interval in WinXP is generally 15.625 ms, rounded to 1 ms
unless something changes it. But in Firefox 3.0 I seem to see 1 ms
intervals.

--
(c) John Stockton, near London. *@merlyn.demon.co.uk/?.**********@physics.org
Web <URL:http://www.merlyn.demon.co.uk/- FAQish topics, acronyms, & links.
Correct <= 4-line sig. separator as above, a line precisely "-- " (SoRFC1036)
Do not Mail News to me. Before a reply, quote with ">" or "" (SoRFC1036)

[Jorge]

On Jun 19, 9:25*pm, Dr J R Stockton <[e-mail]:

>
Standard is that 16mS means 16 milliSiemens, which is a conductance
corresponding to about 63 Ohms. *Doubtless you mean what should be
written as 16 ms - the approximate duration of one cycle of the mains in
hasty countries.

Sure, yep.
"mS" is what I get when my brain executes
"milisegundos".abbreviate().toCamelCase(); :-)

The
underlying interval in WinXP is generally 15.625 ms, rounded to 1 ms
unless something changes it.

That's ~ a tick, in MacOS's tongue : TickCount().

--Jorge.

[Thomas 'PointedEars' Lahn]

Jorge wrote:

On Jun 18, 11:33 am, Gregor Kofler <[e-mail]:

>Anyway, on WinXP the current Safari version is 10 to 15% faster than FF3
running a benchmark that measures core JS performance. IMO nothing to
write home about.

But you might be glad to learn that

-FF is almost as fast as the fastest.

-all of them are performing much better than before. (all but IE).

ISTM you have yet to learn that comparisons, especially but not excluded to
be those made in numbers, need a reliable reference base for them becoming
possible to be taken under serious consideration. For example, "almost as
fast as the fastest" (what?) alone is an utterly useless and pointless
statement about everything. There are really much better ways to waste
bandwidth than posting it here.
PointedEars
--
realism: HTML 4.01 Strict
evangelism: XHTML 1.0 Strict
madness: XHTML 1.1 as application/xhtml+xml
-- Bjoern Hoehrmann

[Jorge]

On Jun 22, 7:21*pm, Thomas 'PointedEars' Lahn <[e-mail]>
wrote:

ISTM you have yet to learn that comparisons, especially but not excluded to
be those made in numbers, need a reliable reference base for them becoming
possible to be taken under serious consideration. *For example, "almostas
fast as the fastest" (what?) alone is an utterly useless and pointless
statement about everything.

Might be.
But you would be asking "What the hell ?" if FF3 ran 2x slower than
FF2.

*There are really much better ways to waste
bandwidth than posting it here.

You're welcome too.

--Jorge.