[From nobody Wed Sep 2 17:21:50 2009 Subject: Re: [Openmcl-devel] Couple implementation questions... From: "Jon S. Anthony" <j-anthony@comcast.net> Reply-To: j-anthony@comcast.net To: Gary Byers <gb@clozure.com> In-Reply-To: <alpine.BSF.2.00.0909021323060.44822@abq.clozure.com> References: <1251919582.2415.98.camel@sirius> <alpine.BSF.2.00.0909021323060.44822@abq.clozure.com> Content-Type: text/plain Message-Id: <1251926800.2415.140.camel@sirius> Mime-Version: 1.0 X-Mailer: Evolution 2.6.0 (2.6.0-1) Date: Wed, 02 Sep 2009 17:26:40 -0400 Content-Transfer-Encoding: 7bit On Wed, 2009-09-02 at 14:08 -0600, Gary Byers wrote: > > On Wed, 2 Sep 2009, Jon S. Anthony wrote: > > > Hi, > > > > A couple of (I believe) quick questions on some implementation details > > with CCL. > > > > 1. Does CCL cache in some fashion the hash for a symbol? ACL does this > > and it can be a real win when using symbols as keys in a hashtable. > > But, I don't see a slot in the symbol def for this, so maybe this isn't > > done in CCL. If not, is there a compelling reason not too or just not > > something anyone thought that important? It wouldn't take much to build > > hack using say the property list, but that doesn't seem satisfactory if > > something better is available. > > It does; I don't think that it's very important, but a customer once > thought that it did. Just to make sure I understand - "It does" => CCL does cache the hash? > I don't know for sure which of us was right or to what degree. My > intuition was/is that the costly cases of hashing a symbol (which is > hashing its pname in CCL) involve (hardware data) cache misses. Would that still be true with the much larger caches today? I actually don't have much of a clue on this point. From past testing (admittedly with ACL), the hashing for large names can be fairly expensive (as an example, think URLs for names...) as that can involve quite a number of multiplications. > If > you cache the result of that operation, you've got to put that result > somewhere. If you just happen to have an unused word in the symbol > that's otherwise (due to alignment issues or something) wasted, you > might win; This may indeed be/have been the case for ACL. They have a fair amount of stuff in their symbol and it may be that an extra slot was there due to alignment requirements, so why not use it? > if you have to make symbols bigger or somehow pack this > cached info into some other field in the symbol, you probably introduce > more cache misses elsewhere. Hmmmm.... > I don't remember this change making any measurable difference in > performance; in that customer's case, locking on concurrent access > to hash tables tended to be a much bigger source of overhead. Well, that certainly sounds right. But, with CCL's new lockless nearly wait free hashtables that should no longer be much (if any) issue. > That > other overhead is usually less than it was a few years ago, and it > might be easier to evaluate the positive and negative effects of > this change. I've never done that, but I'm not a big fan of the > whole idea. Fair enough. It seems you are quite sure that potential induced cache misses by the extra space would kill any savings on the hash computation. I just have not studied and do not understand some of these aspects of newer machine architectures. I suppose this could also be sensitive to the hash computation - in particular, if you have jumbo names, you may lose here on both counts... > > 2. I've noticed that ivectors (created via ccl:make-heap-ivector) don't > > seem to show up in the stats from (room t). Is this as expected (i.e., > > known consequence of design) or a bug/missing functionality? Oddly, the > > space doesn't seem to show up in TOP (the RES field) either. This is on > > Linux (and 32 bit - I didn't check 64 bit). > > I can't think of any reason for ROOM not to tell you about heap-allocated > ivectors, OK, let me recheck this again. I thought I was fairly careful in not seeing a ghost, but I may have been mistaken... > but unless user code is using them heavily (IIRC, you are), there > are usually just a few KB worth of stream buffers allocated in foreign > memory. Yes, I will be using them quite "heavily", but they will be continually recycled so it isn't like they are going to be leaking or anything. Being able to really preallocate these things and just have the readers/writers just use them (without any copying!) is one of the really major wins about them! What I _thought_ I saw went something like this: 1 gc, perform room t 2 look at the info 3 preallocate a few ivectors (4-8MB each) 4 perform room t 5 look again and compare. Hmmm doesnt' seem to be different. repeat 1-5 Hmmm still looks basically the same. Certainly not megabytes different. I'll have to check again. But, oddly enough, I didn't see any change in the stats from top either. Thanks for the info! /Jon ]