[Openmcl-devel] Fun With Data: Historical Benchmarks
bfulg at pacbell.net
Tue Oct 17 16:35:10 EDT 2006
For this benchmark run, 'Reference' == 'OpenMCL'. The benchmark clarifies this at the bottom of the page, but I just added a line at the top of the page to make this clear.
I think the page is immutable if you are not logged in to the wiki.
----- Original Message ----
From: Andrew Shalit <alms at clozure.com>
To: Brent Fulgham <bfulg at pacbell.net>
Cc: Gary Byers <gb at clozure.com>; OpenMCL <openmcl-devel at clozure.com>
Sent: Tuesday, October 17, 2006 8:12:38 AM
Subject: Re: [Openmcl-devel] Fun With Data: Historical Benchmarks
Brent --- all I see on that page is columns for "Reference" and "SBCL".
Is the second column actually OpenMCL?
Also, why is the wiki page immutable?
Brent Fulgham wrote:
> On Oct 16, 2006, at 5:42 PM, Gary Byers wrote:
>> Sorry for not responding to your original message.
>> One of the things that stuff like this (tracking changes to
>> compliance, tracking
>> performance between OpenMCL releases) more meaningful and useful is
>> if those
>> releases is if those releases were more frequent; it's been over a
>> year since
>> 1.0, 1.1 is nearing feature-completness (some Unicode support!) but
>> will need
>> some testing, and that's pretty bad.
> Well, I guess you just need some help!
>> I'm sure that this'd be useful (thanks), but it sort of goes without
>> saying that benchmarking's a black art; I'd tend to trust results
>> that're consistently bad (like I/O in 1.0) more than those that appear
>> good. I was just timing something a few minutes ago, and tended to
>> get better results when I introduced code that should have slowed
>> things down slightly.
> I posted the results of my quick run of the benchmark (http://
> openmcl.org/openmcl-wiki/HowFastAreWe#preview). I'll run it again
> with OpenMCL 1.1 soon to see where things stand...
>> (Just the same, things that we know about - like I/O performance -
>> may have gotten
>> gradually worse over time. If that's true, timing results over
>> time would have
>> shown that trend and might have caught the problem earlier, and
>> having that
>> sort of thing set up may keep other things from drifting into the
>> realm of the
>> very bad.)
> Sure. We all know [Twain/Disraeli/?]'s famous quote. But it's
> mostly sudden changes that are interesting. Even the funny tests on
> http://shootout.alioth.debian.org/ create their own drama, etc. But
> it's useful to see trends, and sometimes highlights true problems.
> At any rate, it gives me an excuse to play with the compiler and
> hopefully it provides some utility (even if only entertainment). :-)
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
More information about the Openmcl-devel