[info-mcl] some issues (Rosetta, ia32 port of MCL)

[Gary Byers]

[I tried to resubscribe to info-mcl a week or so ago; I found that I
needed to hold my head in both hands to keep it from exploding and
quickly unsubscribed; I just resubscribed in digest mode, so at worst
I'll only need to assume the Edvard Munsch "Scream" pose once a day.]

To try to answer (in no particular order) a few questions that've come
up:

- MCL is a CFM application (though I think that I've heard Alice
say that 5.2 uses bridging technology to make "native" OSX Carbon
stuff accessible via #_ and friends.)  CFM (the classic MacOS
Code Fragment Manager) is PPC-only, so any ia32 port would have
to involve making MCL into a native ("Mach-o") application and 
addressing all of the issues that come up.  Those issues are
probably a good deal less significant than the issues related
to changing to a different architecture, but they're certainly
significant.

- For OpenMCL, Rosetta's inability to provide accurate exception
information made it an absolute non-starter.  Aside from the
synchronous exceptions that MCL uses (where "synchronous" means
"a thread executes some instruction that causes it to take an
exception"), OpenMCL uses asynchronous exceptions (where Thread
A asks the OS to force an exception in Thread B).  OpenMCL uses
asynchronous exceptions for things like PROCESS-INTERRUPT and
for the GC.  A thread running the GC needs to be able to stop
other threads at least long enough to be able to determine what
lisp objects are referenced from that thread's registers and
here its stack(s) is/are pointing, and if this information can't
be provided because of dynamic translation issues in Rosetta ...
well, when it became clear that Rosetta wasn't going to stop
putting random numbers in exception contexts, it became clear
that OpenMCL and Rosetta weren't going to get along: the GC just
can't "not know" what other threads are referencing.  Period.
Move on.

- It's been a few years since I looked at MCL sources and about
as long since I've thought about it, but I think that all exceptions
in MCL are synchronous (caused by various combinations of illegal/trap
instructions and by memory protection faults.)  Changing MCL's compiler
and runtime to not generate synchronous exceptions is certainly possible,
but there are lots of subtle issues and interactions among those subtle
issues.  For example, a function that takes 0 arguments probably compiles
to PPC machine code that does something like:

    (trap-if-incoming-arg-count-not-equal 0) ; (twnei nargs 0)
    ;; On entry to a function, the return address is in a register on PPC
    (build-stack-frame-and-save-return-address-and-callers's-context-in-it)


If we replace the trap with something like:

    (compare nargs 0)
    (beq+ @ok)           ; branch if equal, predict taken
    (call wrong-number of-args-error)
    @ok
    (build-stack-frame-and-save-return-address-and-callers's-context-in-it)

the CALL (probably a BL or BLA) will clobber the register (the LR on
PPC) that would tell backtrace where we came from and where we'd like
to eventually return to if we're able to resume in the break loop
(there's another register which would contain the calling function
itself that'd also get clobbered.)

So, we could change that to:

    (build-stack-frame-and-save-return-address-and-callers's-context-in-it)
    (compare nargs 0)
    (beq+ @ok)           ; branch if equal, predict taken
    (call wrong-number of-args-error)

which might work OK for things that took a fixed number of arguments 
(goodbye, leaf function optimization!) but would require some changes
to how things are done in other cases.  I'd believe that most of those
changes are pretty small in and of themselves, but I'd also believe
that there'd be something of a snowball effect (where a group of
changes winds up affecting something far removed from "inability to
use synchronous exceptions".)  I wouldn't want to estimate how long
it might take to make these changes, but "longer than it might seem
at first glance" is often a good answer to that question, and that
may be especially true in this case.  (And yes, I do realize that
replacing a trap with a comapare-and-branch sounds like a simple,
localized change.)  There'd almost certainly be a tunnel somewhere
with light at the end of it.

- Yes, MCL uses write-protected guard pages for stack overflow detection
on some stacks and for heap allocation.  There are other ways to do
both of these things, and I don't think that there's the same degree
of contagion/"subtle interactions with other things" as there is in
other cases; likewise, the use of traps for type- and bounds-checking
could probably be replaced with comparisons and branches around error
calls without requiring much of anything else to change.

I'm deliberately trying to stay out of discussions of whether this
(trying to make MCL run under Rosetta) is a good idea or of how it
should be done or financed if people decide that it is.  I did want
to point out that there'd be at least some of this snowball effect
and that some obvious changes are a little subtler than they might
first appear to be.

- To state the obvious: -if- someone made MCL run under Rosetta,
the result would probably offer a high degree (possibly total)
source compatibility with existing MCL 5.x code.  It wouldn't
offer much in the way of binary compatibility: existing fasl
files (containing code that generates exceptions) would have to
be recompiled, etc.

- There are some other MacOS PPC emulators (besides Rosetta) that
might offer the ability to run legacy MCL code on Intel Mac hardware.
(Try Googling for "SheepShaver" or "pearpc" or "QEMU"; there may be
others.)  I have no idea whether any of these things would offer a
reasonable solution for people interested in the general idea of
running PPC MCL under emulation, but that might be worth exploring.