00:09:49 -!- bjorkintosh [~bjork@ip70-189-66-194.ok.ok.cox.net] has quit [*.net *.split] 00:09:50 -!- luis [~luis@nhop.r42.eu] has quit [*.net *.split] 00:13:32 bjorkintosh [~bjork@ip70-189-66-194.ok.ok.cox.net] has joined #sbcl 00:13:32 luis [~luis@nhop.r42.eu] has joined #sbcl 00:14:55 rpg [~rpg@216.243.156.16.real-time.com] has joined #sbcl 01:01:24 homie` [~levgue@xdsl-78-35-161-46.netcologne.de] has joined #sbcl 01:03:25 -!- homie [~levgue@xdsl-78-35-176-145.netcologne.de] has quit [Ping timeout: 244 seconds] 01:37:41 scymtym [~user@2001:638:504:2093:21a:a0ff:fe34:2d7d] has joined #sbcl 02:38:19 We have, on average, ~84k logged writes to non-nursery addresses. 02:38:38 On average, we trigger 40.3 mprotect write barriers per GC. 02:50:55 so, between each GC, we actually do surprisingly a lot on the few pages we do mutate. 03:30:56 rpg_ [~rpg@216.243.156.16.real-time.com] has joined #sbcl 03:34:41 -!- rpg [~rpg@216.243.156.16.real-time.com] has quit [Ping timeout: 260 seconds] 03:36:55 -!- cmm [~cmm@109.65.214.78] has quit [Ping timeout: 250 seconds] 03:37:45 cmm [~cmm@109.65.214.78] has joined #sbcl 03:44:36 surprisingly, the software barrier's precision means it only registers 30 new mutated pages per GC on average (but the writes are scattered over 85 non-nursery pages total, 400 with the nursery) 04:08:11 -!- homie` [~levgue@xdsl-78-35-161-46.netcologne.de] has quit [Quit: ERC Version 5.3 (IRC client for Emacs)] 04:47:13 _8david: I'm going with card marking (modulo the card table size). We'll see if it works any better than an SSB. 04:50:59 -!- rpg_ [~rpg@216.243.156.16.real-time.com] has quit [Quit: rpg_] 05:15:16 yeah, card marking is much better. No trick, and < 5% overhead. 05:33:26 tcr1 [~tcr@80-218-247-218.dclient.hispeed.ch] has joined #sbcl 06:52:39 -!- tcr1 [~tcr@80-218-247-218.dclient.hispeed.ch] has quit [Quit: Leaving.] 06:54:47 smaller pages, and we're down to pretty much equivalent perf. 06:59:23 -!- attila_lendvai [~attila_le@unaffiliated/attila-lendvai/x-3126965] has quit [Ping timeout: 240 seconds] 06:59:57 attila_lendvai [~attila_le@catv-89-132-188-166.catv.broadband.hu] has joined #sbcl 06:59:57 -!- attila_lendvai [~attila_le@catv-89-132-188-166.catv.broadband.hu] has quit [Changing host] 06:59:57 attila_lendvai [~attila_le@unaffiliated/attila-lendvai/x-3126965] has joined #sbcl 07:03:12 pkhuong annotated #122969 "8k pages" at http://paste.lisp.org/display/122969#3 07:06:18 -!- attila_lendvai [~attila_le@unaffiliated/attila-lendvai/x-3126965] has quit [Ping timeout: 246 seconds] 07:07:37 mmapping huge pages would probably bring that in favour of soft barriers. 07:08:08 attila_lendvai [~attila_le@unaffiliated/attila-lendvai/x-3126965] has joined #sbcl 07:09:00 flip214 [~marek@2001:858:107:1:7a2b:cbff:fed0:c11c] has joined #sbcl 07:09:00 -!- flip214 [~marek@2001:858:107:1:7a2b:cbff:fed0:c11c] has quit [Changing host] 07:09:00 flip214 [~marek@unaffiliated/flip214] has joined #sbcl 07:18:05 -!- scymtym [~user@2001:638:504:2093:21a:a0ff:fe34:2d7d] has quit [Remote host closed the connection] 07:24:51 scymtym [~user@2001:638:504:2093:21a:a0ff:fe34:2d7d] has joined #sbcl 07:35:42 hlavaty [~user@91-65-223-81-dynip.superkabel.de] has joined #sbcl 07:50:13 tcr1 [~tcr@217-162-207-189.dynamic.hispeed.ch] has joined #sbcl 07:52:38 -!- pchrist [~spirit@gentoo/developer/pchrist] has quit [Quit: leaving] 08:57:02 pkhuong annotated #122969 "Enable transparent huge pages" at http://paste.lisp.org/display/122969#4 08:59:12 so.. yeah. I think that, with huge pages, it'll be worth the switch to software card marking instead of mprotect. 08:59:51 People with huge heaps will be happy, and people who benefit from smaller gencgc pages as well. 09:23:19 foom: card marking it is, except that, instead of subtracting the heap base, I take the modulo with the card table size. 09:24:09 even with false positives, a reasonable table size (e.g. 1M) results in a tiny fraction of the heap being unnecessarily scanned. 09:30:16 -!- deepfire [~deepfire@80.92.100.69] has quit [Ping timeout: 244 seconds] 10:10:45 -!- cmm [~cmm@109.65.214.78] has quit [Ping timeout: 246 seconds] 10:11:37 cmm [~cmm@109.65.214.78] has joined #sbcl 10:45:31 -!- workthrick [~mathrick@emp.nat.sdu.dk] has quit [Read error: Connection reset by peer] 10:57:57 workthrick [~mathrick@emp.nat.sdu.dk] has joined #sbcl 11:18:14 -!- workthrick [~mathrick@emp.nat.sdu.dk] has quit [Read error: Connection reset by peer] 11:18:30 pkhuong annotated #122969 "On a loaded machine (no huge page)" at http://paste.lisp.org/display/122969#5 11:26:25 deepfire [~deepfire@80.92.100.69] has joined #sbcl 12:18:10 -!- tcr1 [~tcr@217-162-207-189.dynamic.hispeed.ch] has quit [Quit: Leaving.] 12:38:52 -!- christoph_debian [~user@oteiza.siccegge.de] has quit [Remote host closed the connection] 13:02:24 homie [~levgue@xdsl-78-35-161-46.netcologne.de] has joined #sbcl 13:14:32 attila_lendvai1 [~attila_le@94.44.107.88] has joined #sbcl 13:14:40 -!- attila_lendvai [~attila_le@unaffiliated/attila-lendvai/x-3126965] has quit [Disconnected by services] 13:19:06 -!- attila_lendvai1 [~attila_le@94.44.107.88] has quit [Ping timeout: 258 seconds] 13:26:29 -!- homie [~levgue@xdsl-78-35-161-46.netcologne.de] has quit [Read error: Operation timed out] 13:27:29 homie [~levgue@xdsl-78-35-169-144.netcologne.de] has joined #sbcl 13:28:05 -!- homie [~levgue@xdsl-78-35-169-144.netcologne.de] has quit [Read error: Connection reset by peer] 13:29:36 homie [~levgue@xdsl-78-35-169-144.netcologne.de] has joined #sbcl 13:42:51 christoph_debian [~user@oteiza.siccegge.de] has joined #sbcl 13:47:11 -!- flip214 [~marek@unaffiliated/flip214] has quit [Ping timeout: 260 seconds] 13:51:01 tcr1 [~tcr@217-162-207-189.dynamic.hispeed.ch] has joined #sbcl 14:23:53 -!- antgreen [~user@CPE00222d6c4710-CM00222d6c470d.cpe.net.cable.rogers.com] has quit [Remote host closed the connection] 15:23:07 can you guys think of a better way to handle interrupts in a card marking scheme than to keep a cache of the last write-barriered address? 15:25:40 that way, code can shove the address in the cache, and, if we get an interrupt between the write barrier and the write (or vice versa, depending on the order), the cache, unlike the card table, won't be cleared during a GC. 15:25:52 ok, safe points would work very well ;) 15:45:13 -!- tcr1 [~tcr@217-162-207-189.dynamic.hispeed.ch] has quit [Quit: Leaving.] 15:45:21 <_8david> safe points sound good :-) 15:45:25 <_8david> In the sense at least that GC-causing interrupts will be simply forbidden. 15:45:46 <_8david> Other crazy ideas: Inspect the signal context for RIP to check whether it looks like a write barrier instruction, and extract the address from the register in question. 15:45:48 what about async signals? 15:47:01 <_8david> (I'm not actually suggesting the latter, since I've learned that the signal context in general and RIP in particular can happily point to an earlier signal handler rather than the original Lisp code.) 15:48:01 but we could sniff all the code objects that we pin. 15:49:00 <_8david> your async signal question refers to safepoints and p/a removal in general? 15:49:01 that wouldn't get us the registers anyway 15:49:15 erh, yeah. What happens to them, if we go with safepoints? 15:49:32 <_8david> I'm thinking that user-defined signal handlers could remain possible, but would a. inhibit GC and b. run in an alternative allocation region, so that the lack of p/a for the thread's main allocation region doesn't matter. 15:49:51 <_8david> That could be enough to get sprof running at least with minimal changes to the runtime. 15:50:33 I always figured that sprof could use more or less custom hooks in the runtime if that simplified safepoints. 15:51:34 <_8david> yeah, that'd be good option. But if user-defined lisp signal handles are possible to preserve in any sense at all, sprof would immediately benefit from that anyway. 15:51:58 but what other program do we know of that couldn't defer signal handling to a safe point? 15:54:39 (modulo sync signals) 15:55:18 <_8david> I don't know of a concrete application. I'm more thinking "let's not break stuff if we can avoid it". 15:56:31 I feel like the desire to be a systems programming language has constrained the runtime's system a lot, and I don't know that it's been worth the trouble. 15:57:27 <_8david> Also, code to disable GC is a lot shorter than yet another set of code paths to delay to a safepoint in a new kind of way. :-) 15:58:56 so, caching the last address really hurts. 15:59:13 fwiw, the only lisp-side signal handler we use is sigchld 15:59:28 and that can be delayed basically indefinitely. 15:59:53 foom: oh yeah, i got card marking working for strange heaps, with only one extra instruction. 16:00:16 pdlogan [~patrick@174-25-37-137.ptld.qwest.net] has joined #sbcl 16:00:53 wow, anding off the extra bits, that's such a obvious good idea in retrospect, nice job thinking of it. :) 16:01:12 yeah, it sounds so wrong, and then you realize that 1/1M is tiny. 16:03:34 wait. the EA can encode a 32 bit address. 16:03:42 no need for an AND. 16:04:09 erh, yes need for an AND, I don't want a 3 GB card (: 16:06:23 are you using bits or bytes to store the marks? 16:06:27 bytes 16:06:38 is that faster? 16:07:03 I don't see why not. 16:08:37 I dunno, you were talking about memory bandwidth before, using bits would increase the density of storage, but at the cost of many more instructions (...and thus code bloat so more memory bandwidth for the code) 16:08:49 bandwidth was for the SSB. 16:09:07 we only touch around 100 cards between GCs. 16:09:21 with 8k-pages? 16:09:29 yup. 16:10:33 that's after the static filtering for unboxed values and stores of immediates. 16:11:10 oh right, that's another huge advantage, I'd forgotten about that! 16:11:35 mprotect didn't offer that opportunity. 16:11:45 well, sort of. We don't protect unboxed pages. 16:11:59 right, but there's lots of objects that have both boxed and unboxed values 16:12:02 right. 16:13:56 the main advantage that I can see is with hugepages, actually. Even on something as mundane as a self build, they compensate for half of the ~5-6% slowdown due to the card marking. With large data sets, it'd probably be even better. 16:14:15 yes, hugepages are awesome. 16:14:16 Using bytes also allows marking the lowest gen the object points to after scanning it. 16:15:03 er, s/object/card/ although I suspect that using even smaller card sizes than 8k would be advantageous. 16:15:24 probably, but going < OS page would require a lot of work in the GC 16:15:43 or, maybe not, actually, now that we only mmap once. 16:16:47 -!- hlavaty [~user@91-65-223-81-dynip.superkabel.de] has quit [Read error: Operation timed out] 16:16:50 must figure out the signal issue, though. 16:17:20 -!- bjorkintosh [~bjork@ip70-189-66-194.ok.ok.cox.net] has quit [Remote host closed the connection] 16:20:06 _8david: you're serious about the fact that we don't always have a good RIP? 16:20:16 even in the handler itself? 16:20:20 attila_lendvai [~attila_le@adsl-89-135-202-10.monradsl.monornet.hu] has joined #sbcl 16:20:20 -!- attila_lendvai [~attila_le@adsl-89-135-202-10.monradsl.monornet.hu] has quit [Changing host] 16:20:20 attila_lendvai [~attila_le@unaffiliated/attila-lendvai/x-3126965] has joined #sbcl 16:22:13 <_8david> pkhuong: yes, if I understand the linux kernel mailing list discussion correctly that mega had with those guys. 16:22:31 jesus. srsly. 16:22:55 <_8david> Suppose you're about to trap into a SIGSEGV handler, but at that time an asynchronous signal strikes which has higher prioriy (= lower signal number) than SIGSEGV, then the the latter gets done first, and the SIGSEGV points to the other signal context instead of Lisp code. 16:23:19 <_8david> that's why SIG_STOP_FOR_GC must be SIGUSR2 instead of SIGUSR1 16:24:08 <_8david> So don't send ^C to an SBCL that is exhausting its main stack at that very moment! :-) 16:24:29 but *some* context that we know of has the right RIP? 16:24:41 no. 16:25:04 there's still a window :\ 16:25:20 Have I told you how much I hate signals? (: 16:30:08 ok. so, the best I can think of is to re-introduce a disassembler in the GC. 16:30:37 nope. 16:31:17 wait, of course you still have a context that has the right RIP 16:31:52 foom: how do I find it from the GC? 16:31:59 it's just somewhere on the stack, no? 16:32:02 Yes 16:32:10 the signal context for the sigint will have the original RIP 16:32:21 not necessarily. 16:32:41 well, whichever hits first gets the original RIP 16:32:48 then the second gets the first's RIP 16:32:52 and then? 16:33:05 then I don't know how you find the first one... 16:33:08 but it's there. :) 16:33:13 somewhere on the stack ;) 16:34:48 why... seriously, can't signals avoid nesting by default? 16:40:49 how does our interrupt_context hack work, on threads, anyway? 16:45:13 <_8david> I'm not claiming full accuracy for my explanation of the nested handler thing; better see for yourself at http://lkml.org/lkml/2009/3/17/4 and http://lkml.org/lkml/2009/3/17/196 16:46:09 <_8david> which hack do you mean? interrupts_contexts is "just" a thread-local stack. 16:46:23 wait, wait. 16:46:31 so I do have the contexts? 16:47:40 even on single threaded builds, they're saved in the one thread struct. And *that* is safe: there's no window in which we could lose a context to nested signals? 16:47:55 the order might be wrong, but I actually have all the contexts? 16:48:53 <_8david> no, only the contexts that are currently a. on the stack and b. were registered by fake_foreign_function_call 16:49:19 <_8david> so that's pretty much unrelated to sigreturn arranging for a call upon exit from one handler to the next 16:52:46 so, even if I were to hack all our handlers, there'd be a window for another signal to hit us before registering the context :\ 16:53:11 btw, re allocation and signals, IIRC, I managed to get that working with and XADD. 16:53:29 *with an XADD 16:54:39 no need to open a new allocation region, as long as GC is disabled, or, if nursery pages with raw pointers (aligned to double lisp words) to them are pinned. 16:55:31 you need a bit more code in the allocation region, though, so I don't know that it's worth it. 17:00:01 <_8david> speed penalty? 17:01:57 maybe, not sure. I think I tried both an XADD and a CMPXCHG based allocation sequence. 17:02:58 they seemed fine without a PA section, but not any faster, and a bit larger. 17:04:44 so, currently, we can't get both a sigsegv/sigbus and a GC signal because we use SIGUSR2? 17:05:14 <_8david> the SIGUSR2 will arrive after the SIGSEGV 17:05:28 <_8david> (IIUC) 17:11:20 so, as a Hack, I could use the marked cards from the previous GC as well. 17:11:49 but, in theory, we could have a thread that doesn't make any progress between the GCs. 17:13:54 (actually, it's currently not an issue, because pinned pages are always considered dirty... which is a really bad idea for huge vectors) 17:17:03 -!- specbot [~specbot@tiger.common-lisp.net] has quit [Ping timeout: 244 seconds] 17:19:09 -!- attila_lendvai [~attila_le@unaffiliated/attila-lendvai/x-3126965] has quit [Quit: Leaving.] 17:22:13 <_8david> I think you could mprotect the array of mark bits before stopping the world and record all violations of that protection before SIG_STOP_FOR_GC hits. 17:22:19 <_8david> Those violations are your recently set marks. 17:22:26 wow. 17:22:30 <_8david> But when violated, you need to unprotect the affected part of the array. 17:22:32 <_8david> One issue is that you'd have to consider a whole page-of-mark-bits as recently marked, since you don't know whether a second thread wrote to it after protection for that part was lifted. But in general you'd hope that only few threads mutate only little between that mprotect() and the point where they stop for GC. 17:22:56 can't you handle the violation like a STOP_FOR_GC? 17:23:54 <_8david> I think when STOP_FOR_GC hits during without-cging, the thread keeps running for a bit. 17:24:08 mm, right. 17:24:28 problem is, my card table fits in a couple pages. 17:27:00 <_8david> hmm. Still worth testing I'd say; if you're lucky this sort of mutation is rare. And a compromise could be possible and improve the odds significantly: Test if stopping for GC is OK (and consider only the si_addr as marked), and unprotect only if not (only then mark the whole page). 17:29:43 that doesn't handle interrupts, though? 17:30:28 if I get a signal between the write barrier and the actual write 17:30:55 the lisp handler can do an arbitrary amount of work, including GCs, right now. 17:42:38 on the other hand, nobody has complained about our current even worse scheme. 17:44:06 try this: (defun test (n) (let ((x (make-array n))) (setf (aref x 0) 0) (dotimes (i 10) (gc)))) 17:44:17 The runtime grows ~linearly with n. 18:01:45 <_8david> hrm, all the steps described above _plus_ a special-case that keeps marks across GC caused by async signals? Pretty annoying overall compared to safepoints... 18:03:41 well. I can do a bit simpler. 18:04:11 keep marks on pinned pages. 18:05:23 For large objects that are vectors, only do so if there's also a (raw) pointer to an address in the marked card on the stack [in the vicinity of a lisp pointer to the vector]. 18:06:36 tcr1 [~tcr@80-218-247-218.dclient.hispeed.ch] has joined #sbcl 18:06:49 writes to vectors look like: LEA tmp, ... / write barrier / write, with both tmp and a pointer to the vector throughout the sequence. 18:07:14 *both tmp and a pointer to the vector live throughout the sequence. 18:45:47 wonder why factor goes with a two-level card table. It's supposed to save time scanning the cards, but, really, 1/256 of the heap is tiny. 18:59:30 prxq [~mommer@mnhm-590c2607.pool.mediaWays.net] has joined #sbcl 19:01:42 -!- pdlogan [~patrick@174-25-37-137.ptld.qwest.net] has quit [Ping timeout: 255 seconds] 19:15:30 pdlogan [~patrick@174-25-37-137.ptld.qwest.net] has joined #sbcl 19:32:05 -!- tsuru [~charlie@74.240.217.227] has quit [Ping timeout: 258 seconds] 19:38:08 tsuru [~charlie@adsl-74-240-217-227.bna.bellsouth.net] has joined #sbcl 19:54:29 -!- prxq [~mommer@mnhm-590c2607.pool.mediaWays.net] has quit [Quit: Leaving] 20:36:28 tunes [~Fare@ita4fw1.itasoftware.com] has joined #sbcl 20:54:32 pchrist [~spirit@gentoo/developer/pchrist] has joined #sbcl 20:59:57 well, with efficient marking and scanning, you should be able to GC a lot more frequently than you can in SBCL now. 21:01:30 after a while the 1/256th of the heap must start to add up. 21:03:53 I just read that as "marketing" 21:11:10 -!- homie [~levgue@xdsl-78-35-169-144.netcologne.de] has quit [Quit: ERC Version 5.3 (IRC client for Emacs)] 21:21:32 foom: true, but then again, you usually want a large nursery. 21:21:43 So if the cards are on the order of the nursery... 22:09:55 Love the comment that card marking doesn't solve the problem, but only changes the constant factor. 22:10:39 An improvement of ~1k in the constant factor solves most of my problems ;) 22:13:26 ignis_ [~quassel@12.50.98.2] has joined #sbcl 22:23:38 -!- tcr1 [~tcr@80-218-247-218.dclient.hispeed.ch] has quit [Quit: Leaving.] 23:56:47 ccl-logbot [~ccl-logbo@setf.clozure.com] has joined #sbcl 23:56:47 23:56:47 -!- names: ccl-logbot hargettp ignis_ pchrist tunes tsuru christoph_debian deepfire cmm scymtym luis hargettp_ loke peddie fe[nl]ix ASau cow-orker pkhuong mtd Xof_ slyrus Quadrescence tWip foom redline6561_ |3b| jiacobucci _8david antoszka joshe pp206 lisppaste2 jsnell antifuchs