[Openmcl-devel] Speed, compilers and multi-core processors
mevins at mac.com
Tue May 19 19:37:30 EDT 2009
On May 19, 2009, at 1:12 PM, Dan Weinreb wrote:
> Alexander Repenning wrote:
>> not so fast ;-)
>> The "how can we make use of multiple cores" is currently on the
>> the hottest funding topics supported by NSF, DOE, Microsoft, .....
> For some applications, it's very easy to take advantage of all those
> cores. The airline reservation system we're building at ITA
> has its "core" (business tier) model written in Common Lisp.
> We get lots and lots of requests coming in, which are distributed
> to a load balancer to a lot (maybe 20?) of boxes, each of
> which has 8 cores and 32GB of memory. On each one,
> we run 8 CCL processes, and each one has a single
> request-handling thread. Voila, all the cores are
> being used.
> But, of course, you're talking about situations where a single
> program wants to use lots of cores.
> The approach being taken by many new languages is
> (1) do as much as possible with no side effects, so
> that nobody is writing anything, and therefore
> no two threads have conflicts, and (2) when you do
> have two threads reading and/or writing the same
> memory, use transactions.
> Guy Steele said that it's quite amazing how many new
> languages work this way. His own new language,
> Fortress is one of them, and he named several
> others. Anders Hejlsberg, the C# inventor at
> Microsoft, is also talking about this approach (I've
> seen him discuss it in two videos that I found on
> the web).
> And you know what other language works this way?
> Clojure, a very promising new dialect of Lisp.
Yup. It works pretty well. I've used it a lot for about eight months.
You can't get something that works as well simply by adding Clojure's
concurrency features to Common Lisp. Clojure *forbids* simple
assignment; you can't do it. There are four data types that support
assignment, and none of them supports simple assignment. All four use
transactional assignment that is safe under concurrent use, but
significantly more expensive than simple assignment. Clojure's basic
data structures are also different from Common Lisp's, despite looking
familiar. Its List, Vector, and Map types, for example, look like CL's
lists, vectors, and hashtables, but are fundamentally different data
structures under the covers, in order to support both immutability and
efficient updates. (They are implemented using Bagwell's ideal hash
tries, which provide an efficient way to implement "inserts" as newly-
created data structures that refer to unaltered old ones).
Clojure forbids simple assignment because all it takes to destroy its
safety guarantees under concurrency is a single SETQ in the wrong
context. Presumably, this is why some of the approaches it uses, such
as Software Transactional Memory, are widely used in languages like
Haskell and not widely used in languages like C and Common Lisp:
because all your safety guarantees go out the window in the presence
of simple assignment. Clojure and Haskell solve that problem by making
simple assignment impossible. (Actually, Clojure doesn't; because it
runs on the JVM and provides a seamless integration with Java, you can
still do simple assignment using the Java features. If you do that,
though All Bets Are Off).
More information about the Openmcl-devel