[Openmcl-devel] http servers and threads under heavy load; was: Good Video
R. Matthew Emerson
rme at clozure.com
Sat Oct 17 19:32:37 EDT 2009
On Oct 17, 2009, at 4:45 PM, Daniel Pezely wrote:
> Sometimes, however, OS-native threads are required for your particular
> needs. In those cases, try lock-free, spin-free message lists within
> the pipeline. It's no magic bullet, of course... You'll want atomic-
> push and -pop operations (and ideally, atomic-rotatef-1), which in
> SBCL can be built upon #'sb-ext:compare-and-swap function. Since CCL
> internally has #'ccl::atomic-incf and -decf, I'm sure that the other
> functions will follow soon or already exist by some other name. I
> couldn't find any with a quick check of latest source in svn; anyone
There was a discussion on the #ccl irc channel on Freenode a couple of
days ago about this.
The upshot of it is that there are several building blocks of this
nature in ccl, but they're all undocumented and all somewhat
01:52:37 <chandler> Is there an exposed CAS-style operation? (Forgive
me if I'm missing something in the documentation.)
01:54:03 <gbyers> There are several, all funky and none (AFAIK)
documented. CLX wants a CONDITIONAL-STORE, and there's something
that's slightly exposed for its benefit.
01:55:24 <sellout> chandler: Adlai was playing with a CAS earlier
today, I think ...
01:55:48 <sellout> (for CCL)
01:56:16 <sykopomp> sellout: he's been working on it for a while. I
don't know if he's gotten around to writing that latest version after
he figured out what was wrong...
01:56:22 <chandler> Ah, I just found that in the logs.
01:56:42 <gbyers> The general case of CAS is weird for a few reasons
(having to do with the GC, threads, and interrupts.)
01:56:58 <rme> Without special lisp runtime support, I don't think
it's possible to do a CAS for lisp objects.
01:58:02 <chandler> Can you elucidate some of the reasoning behind that?
02:00:54 <gbyers> on x8664, "cmpxchg" uses %rax as an implicit
operand: you generally load the contents of a memory location into
%rax, then try to store a new value there, and the conditional store
works if the location still contains the value in %rax. If the GC
runs between the load and the cmpxchg, the memory location's value may
change but the value in %rax (presumed to be a non-lisp value) won't
02:01:24 <rme> In CCL, a gc can happen at any instruction boundary.
CCL's gc thinks it always knows if a register contains a lisp
object ... what he said.
02:04:44 <chandler> Ah. Having only dealt with this at a C level, I
02:07:05 <chandler> Presumably some sort of thread state could be
updated around the instruction, with a worst case that the old value
is incorrectly marked as alive. (Or I suppose the GC could look back
an instruction and handle cmpxchg cases specially.)
02:07:20 <gbyers> rme and I walk around saying "... but if a GC
happens here, we're screwed." It may be a curse.
02:07:32 <chandler> Heh.
02:07:34 <rme> For a good time (not) read pc_luser_xp() in ccl:lisp-
02:10:34 <chandler> I'll have a look when I'm not IRCing from my phone.
02:11:00 <gbyers> We do the problematic cmpxchg's in a few known
locations in the lisp kernel, and that function recognizes when a
thread gets stopped in the middle of one of those sequences and backs
up/moves forward as appropriate.
02:15:21 <gbyers> We also do the generational write-barrier stuff in
software, so if a cmpxchg succeeds, it may have just stored a pointer
to a young object in an old object. We need to set a bit to indicate
a possible intergenerational reference, and the store and the bit-
setting have to appear to be atomic.
More information about the Openmcl-devel