00:15:10 -!- TimKack [~user@c-2ec2be8e-74736162.cust.telenor.se] has quit [Ping timeout: 250 seconds] 00:30:03 -!- Phoodus [~foo@ip72-223-116-248.ph.ph.cox.net] has quit [Ping timeout: 255 seconds] 00:30:19 Phoodus [~foo@ip72-223-116-248.ph.ph.cox.net] has joined #sbcl 00:36:01 rpg [~rpg@216.243.156.16.real-time.com] has joined #sbcl 00:41:55 -!- Phoodus [~foo@ip72-223-116-248.ph.ph.cox.net] has quit [Ping timeout: 260 seconds] 00:42:12 Phoodus [~foo@ip72-223-116-248.ph.ph.cox.net] has joined #sbcl 00:47:31 -!- rpg [~rpg@216.243.156.16.real-time.com] has quit [Quit: rpg] 00:57:04 rpg [~rpg@216.243.156.16.real-time.com] has joined #sbcl 00:58:52 -!- rpg [~rpg@216.243.156.16.real-time.com] has quit [Client Quit] 00:59:23 rpg [~rpg@216.243.156.16.real-time.com] has joined #sbcl 00:59:34 -!- rpg [~rpg@216.243.156.16.real-time.com] has quit [Client Quit] 02:07:18 -!- slyrus_ [~chatzilla@99-28-161-110.lightspeed.miamfl.sbcglobal.net] has quit [Ping timeout: 256 seconds] 02:23:18 -!- huangjs [~huangjs@190.8.100.83] has quit [Quit: Leaving] 02:30:13 slyrus_ [~chatzilla@99-28-161-110.lightspeed.miamfl.sbcglobal.net] has joined #sbcl 02:45:29 -!- Phoodus [~foo@ip72-223-116-248.ph.ph.cox.net] has quit [Read error: Connection reset by peer] 02:45:44 Phoodus [~foo@ip72-223-116-248.ph.ph.cox.net] has joined #sbcl 02:46:34 christop` [~user@oteiza.siccegge.de] has joined #sbcl 02:46:53 brown``` [user@nat/google/x-rbupnllyztslcubc] has joined #sbcl 02:47:50 -!- whoops [whoops@2600:3c01::f03c:91ff:fe93:da36] has quit [Ping timeout: 260 seconds] 02:48:25 -!- christoph_debian [~user@oteiza.siccegge.de] has quit [Ping timeout: 260 seconds] 02:48:26 -!- brown`` [user@nat/google/x-zszmkognyzqmdybw] has quit [Ping timeout: 260 seconds] 02:56:53 whoops [whoops@2600:3c01::f03c:91ff:fe93:da36] has joined #sbcl 02:59:34 antgreen [~user@bas3-toronto06-1176449386.dsl.bell.ca] has joined #sbcl 03:42:14 -!- Phoodus [~foo@ip72-223-116-248.ph.ph.cox.net] has quit [Ping timeout: 245 seconds] 03:42:31 Phoodus [~foo@ip72-223-116-248.ph.ph.cox.net] has joined #sbcl 04:05:08 -!- antgreen [~user@bas3-toronto06-1176449386.dsl.bell.ca] has quit [Ping timeout: 240 seconds] 04:20:28 -!- CampinSam [~Sam@24-176-98-217.dhcp.jcsn.tn.charter.com] has quit [Quit: leaving] 05:14:17 -!- gko [~gko@114-34-168-13.HINET-IP.hinet.net] has quit [] 05:25:27 easy-iPad [~easyipad@213.47.71.36] has joined #sbcl 05:45:21 angavrilov [~angavrilo@217.71.227.190] has joined #sbcl 05:48:48 -!- easy-iPad [~easyipad@213.47.71.36] has quit [Quit: Outta here?] 06:30:31 homie` [~levgue@xdsl-87-79-192-210.netcologne.de] has joined #sbcl 06:33:08 -!- homie [~levgue@xdsl-78-35-182-220.netcologne.de] has quit [Ping timeout: 240 seconds] 07:11:41 edgar-rft [~user@HSI-KBW-078-043-123-191.hsi4.kabel-badenwuerttemberg.de] has joined #sbcl 07:26:07 sdemarre [~serge@91.176.39.207] has joined #sbcl 08:15:18 -!- homie` [~levgue@xdsl-87-79-192-210.netcologne.de] has quit [Read error: Connection reset by peer] 08:15:57 homie` [~levgue@xdsl-87-79-192-210.netcologne.de] has joined #sbcl 08:30:20 -!- sdemarre [~serge@91.176.39.207] has quit [Ping timeout: 260 seconds] 09:45:33 nikodemus [~nikodemus@cs27123025.pp.htv.fi] has joined #sbcl 09:45:33 -!- ChanServ has set mode +o nikodemus 10:01:19 -!- nikodemus [~nikodemus@cs27123025.pp.htv.fi] has quit [Ping timeout: 255 seconds] 10:09:11 nikodemus [~nikodemus@cs27123025.pp.htv.fi] has joined #sbcl 10:09:11 -!- ChanServ has set mode +o nikodemus 10:17:08 TimKack [~user@c-2ec20fcf-74736162.cust.telenor.se] has joined #sbcl 10:28:05 -!- nikodemus [~nikodemus@cs27123025.pp.htv.fi] has quit [Ping timeout: 260 seconds] 10:50:39 -!- homie` [~levgue@xdsl-87-79-192-210.netcologne.de] has quit [Read error: Connection reset by peer] 10:51:21 homie` [~levgue@xdsl-87-79-192-210.netcologne.de] has joined #sbcl 10:56:53 nikodemus [~nikodemus@87-95-52-234.bb.dnainternet.fi] has joined #sbcl 10:56:53 -!- ChanServ has set mode +o nikodemus 10:58:39 cmm- [~cmm@bzq-79-177-236-86.red.bezeqint.net] has joined #sbcl 10:59:34 -!- TimKack [~user@c-2ec20fcf-74736162.cust.telenor.se] has quit [Remote host closed the connection] 11:00:37 -!- cmm [~cmm@bzq-79-182-209-72.red.bezeqint.net] has quit [Ping timeout: 248 seconds] 11:35:49 sdemarre [~serge@91.176.39.207] has joined #sbcl 11:37:09 gko [~gko@114-34-168-13.HINET-IP.hinet.net] has joined #sbcl 11:46:26 -!- homie` [~levgue@xdsl-87-79-192-210.netcologne.de] has quit [Read error: Connection reset by peer] 11:47:05 homie` [~levgue@xdsl-87-79-192-210.netcologne.de] has joined #sbcl 11:50:18 -!- nikodemus [~nikodemus@87-95-52-234.bb.dnainternet.fi] has quit [Ping timeout: 265 seconds] 12:57:46 -!- gko [~gko@114-34-168-13.HINET-IP.hinet.net] has quit [Ping timeout: 244 seconds] 13:02:33 antgreen [~user@out-on-195.wireless.telus.com] has joined #sbcl 13:13:27 gko [~gko@114-34-168-13.HINET-IP.hinet.net] has joined #sbcl 13:16:01 -!- homie` [~levgue@xdsl-87-79-192-210.netcologne.de] has quit [Read error: Connection reset by peer] 13:16:44 homie` [~levgue@xdsl-87-79-192-210.netcologne.de] has joined #sbcl 13:19:07 -!- homie` [~levgue@xdsl-87-79-192-210.netcologne.de] has quit [Read error: Connection reset by peer] 13:33:10 homie [~levgue@xdsl-87-79-192-210.netcologne.de] has joined #sbcl 13:38:46 leuler [~user@p54903805.dip.t-dialin.net] has joined #sbcl 13:39:26 -!- leuler [~user@p54903805.dip.t-dialin.net] has left #sbcl 13:39:52 leuler [~user@p54903805.dip.t-dialin.net] has joined #sbcl 13:54:36 -!- homie [~levgue@xdsl-87-79-192-210.netcologne.de] has quit [Quit: ERC Version 5.3 (IRC client for Emacs)] 14:13:08 -!- cmm- [~cmm@bzq-79-177-236-86.red.bezeqint.net] has quit [Ping timeout: 240 seconds] 14:13:58 homie [~levgue@xdsl-87-79-192-210.netcologne.de] has joined #sbcl 14:15:31 cmm [~cmm@bzq-79-183-234-200.red.bezeqint.net] has joined #sbcl 14:27:06 CampinSam [~Sam@24-176-98-217.dhcp.jcsn.tn.charter.com] has joined #sbcl 14:32:14 -!- CampinSam [~Sam@24-176-98-217.dhcp.jcsn.tn.charter.com] has quit [Quit: leaving] 14:35:28 CampinSam [~Sam@24-176-98-217.dhcp.jcsn.tn.charter.com] has joined #sbcl 14:39:57 rpg [~rpg@216.243.156.16.real-time.com] has joined #sbcl 14:42:08 -!- rpg [~rpg@216.243.156.16.real-time.com] has quit [Client Quit] 15:19:58 -!- leuler [~user@p54903805.dip.t-dialin.net] has quit [Quit: ERC Version 5.1.2 $Revision: 1.796.2.6 $ (IRC client for Emacs)] 15:32:40 -!- sdemarre [~serge@91.176.39.207] has quit [Ping timeout: 260 seconds] 15:36:00 -!- antgreen [~user@out-on-195.wireless.telus.com] has quit [Remote host closed the connection] 16:21:20 easy-iPad [~easyipad@213.47.71.36] has joined #sbcl 17:29:26 https://github.com/pkhuong/sbcl/tree/card-2012-2 <- win. 17:32:03 -!- gko [~gko@114-34-168-13.HINET-IP.hinet.net] has quit [] 17:50:22 nyef [~nyef@c-174-63-105-188.hsd1.ma.comcast.net] has joined #sbcl 17:54:20 nyef: do you have some time for hack-review? 17:54:48 Probably. What hack do you have that needs reviewing? 17:54:56 software write barriers. 17:55:26 Ahh. 17:56:29 https://github.com/pkhuong/sbcl/commits/card-2012-2 17:57:04 From "Enable safety checks"? 17:57:19 yeah, but it's a dev branch. 17:57:27 You probably want to look at the diff from master. 17:58:42 ... how do I do that in github? 17:59:13 sigh, that was trivial on repo 17:59:59 https://github.com/pkhuong/sbcl/compare/sbcl%3Amaster...card-2012-2 18:00:04 pkhuong: what are cards? 18:00:28 slyrus_: the granularity at which we track written-to addresses. 18:00:40 nyef: the "Files changed" view 18:01:48 Thanks. 18:05:32 Is this just for x86-64? 18:05:40 for now. 18:07:29 x86 shouldn't be *too* hard (as long as there's one register I can take out of regalloc during development). PPC should be even easier (fewer instructions that write to memory), given a PPC box. 18:07:47 and it depends on gencgc's behaviour of not-reprotecting pinned pages. 18:08:50 x86 regalloc is severely constrained. I have an old series of patches to scare up a single extra register for use as a thread-base register, and it wasn't pretty. 18:09:10 k. I'll just have to think harder (: 18:09:31 I'm also not a big fan of the use of a fixup in the write barrier. 18:09:42 will this interfere with simple-arrays allocated in foreign memory ? 18:09:48 fe[nl]ix: no. 18:10:05 awesome 18:10:06 fe[nl]ix: we'll just have false-positives. 18:10:18 in what sense ? 18:10:28 I really want to try a read-barrier implementation at some point now. (-: 18:10:50 nyef: I had it thread-loca in a few other dev branches, but the fixup was simpler, and "that's how Sun's JVM does it" 18:11:37 nyef: the infrastructure is all there to detect un-annotated reads as well. 18:12:05 fe[nl]ix: some other page in the lisp heap might be marked as having been written to. 18:12:56 Many implementations subtract the heap base from the address to get an index (+ shr for sanely-sized pages) 18:13:16 I shift and mask just so you people can play dirty tricks with the foreign heap 18:14:14 On the whole, it doesn't look horrible. 18:14:53 At the same time, it looks like it still could do with a little cleanup, or some explanation for some changes (having the heap verifier enabled, for example). 18:15:10 I'm cleaning up the runtime now. 18:15:20 Okay then. 18:15:30 and that build just self-built and tested successfully 18:18:40 I'll let genesis take care of the constants, then featuritise, and find a decent way to rewrite that history for master. 18:19:11 Sounds good to me. It's going to be a build option, right? 18:19:17 yup. 18:20:50 then people can fiddle with the barrier sequence ;) 18:21:18 what would write barriers give me? performance? 18:21:31 are they used instead of mprotect? 18:22:00 yup, instead of mprotect. 18:22:38 and they are better in some respect, i presume? 18:22:47 performance, if mprotect and handling SEGV are slower than the additional code (usually a wash, 2-3%) 18:22:55 but also fork and hugepage-friendliness. 18:23:09 -!- CampinSam [~Sam@24-176-98-217.dhcp.jcsn.tn.charter.com] has quit [Quit: leaving] 18:23:15 i see 18:23:17 and finer granularity in the future, when we change our GC a bit. 18:24:00 are there some hardware write barriers which can be used for this? 18:24:13 apart from mprotect or lisp machines? no. 18:24:18 well, yes, on solaris. 18:24:46 MMUs track written-to pages to enable software swapping. 18:25:16 only solaris exposes that data directly (instead of making us implement it with mprotect), but that's also fairly slow. 18:25:17 Isn't there some windows-specific dirty-page tracking API? 18:25:35 nyef: only for file-backed mappings, iirc. 18:25:57 Oh, for the... Why must their VMM be so bloody crippled? 18:26:50 anything i search on write barriers leads me to mfence and friends 18:26:52 oh yes, also: we can have an intermediate of mprotect/segv and solaris's dirty bit proc file by forking and comparing memory mapping metadata (: 18:27:14 stassats`: "write barrier gc" seems pretty good. 18:28:50 http://www.hoelzle.org/publications/write-barrier.pdf <- pretty much this scheme, mangled to support fe[nl]ix's abuse ;) 18:29:47 i haven't yet finished reading bigsurv.ps 18:30:07 it talks about barriers too 18:37:59 homie` [~levgue@xdsl-87-79-192-69.netcologne.de] has joined #sbcl 18:41:01 -!- homie [~levgue@xdsl-87-79-192-210.netcologne.de] has quit [Ping timeout: 252 seconds] 18:44:21 for a self-build, software wb: 288.5 sec, mprotect: 268.0 sec. 18:47:06 So, within ten percent? 18:50:08 -!- homie` [~levgue@xdsl-87-79-192-69.netcologne.de] has quit [Quit: ERC Version 5.3 (IRC client for Emacs)] 18:50:08 I should test on darwin, the figures are probably reversed. 18:50:23 nyef: easily. 18:50:39 and that's without huge pages or anything. 18:50:47 and it still could be optimized? 18:51:18 the runtime could definitely be improved to exploit software wb better. 18:51:50 pkhuong: I certainly ab-use SBCL's internals but I wouldn't mind if the code in static-vectors were harmoniously integrated into make-array :) 18:52:50 fe[nl]ix: there's this branch that lets gencgc work as a mark and sweep collector on data structures in the foreign heap. 18:53:21 how would that work ? 18:54:02 anyway, that's separate from being able to allocate such arrays using make-array, is it not ? 18:54:30 it's 99% of the way 18:55:12 I don't mind having to free them manually, especially since their lifetime is tied to that of other resources that need manual de-allocation: FDs 18:55:22 at least, that's what I use them for 18:56:14 nikodemus [~nikodemus@178-55-57-180.bb.dnainternet.fi] has joined #sbcl 18:56:14 -!- ChanServ has set mode +o nikodemus 18:56:29 hi nikodemus :) 18:57:37 hi 18:58:22 fe[nl]ix: https://github.com/pkhuong/sbcl/tree/foreign-mark-sweep <- I'm surprised...it seems almost ready to be useful. 18:59:28 i'm interested in foreign callbacks from foreign threads, any progress/ideas on that? 19:00:00 fe[nl]ix: it's not just that the foreign-allocated data is GCed, but it's also scanned for pointers. 19:00:09 stassats`: not on my end. the russians have something. 19:00:20 homie [~levgue@xdsl-87-79-192-69.netcologne.de] has joined #sbcl 19:00:21 iirc they do 19:01:21 and another inquiry, when will the freeze commence? 19:01:30 pkhuong: what does that do? 19:01:46 jsnell knows_ 19:01:49 pkhuong: so that allows also non-specialized arrays in foreign memory ? 19:02:17 fe[nl]ix: yes. 19:03:02 nikodemus: mark and sweep on specially-marked regions of the foreign heap, which can be treated as binary blobs or lispobjs. 19:03:33 2-generations 19:03:58 nikodemus: but, in more recent news, software write barriers are a go. 19:04:26 \o/ 19:04:48 high coolness 19:05:21 pkhuong: if i do (fill x vector), does it mean that it'll trigger a barrier on every element of the vector? 19:05:38 stassats`: yes. 19:05:57 what do you use for the barrier? 19:05:58 now, that barrier is 3 and a half instructions. 19:06:13 could it be so optimized as to trigger it only once? 19:06:28 stassats`: not without a lot of static analysis I'm not keen on. 19:06:33 (and presumably it doesn't trigger if the write isn't a lispobj) 19:06:45 pkhuong: What about just hacking up the FILL transform? 19:06:56 nyef: that's more like it. 19:07:12 But, here's the thing: it can be interesting to note non-pointer writes as well. 19:07:22 really? 19:07:26 *nikodemus* is curious 19:07:27 it lets us do neat refcount-like tricks. 19:07:36 pkhuong: would you be interested in a indiegogo gig(maybe after you finish with your degree) ? 19:07:37 Use-case, please? 19:08:28 nyef: say we track the number of pages that refer to each page. 19:08:46 The heap is mostly acyclic (i.e. refcount is good enough, most of the time) 19:09:08 *we need to track the set, not just number, actually. 19:09:42 a page can look at its referrer set, update it quickly by looking at the cards, and know it's dead when the set is empty. 19:09:48 oh, right. so you can do the "garbage collection without paging" stuff? 19:10:01 nikodemus: some of it, anyway. 19:10:24 but i still don't see how you need non-pointer writes for that 19:10:51 nikodemus: because I can write a fixnum to what used to be a reference 19:11:03 unboxed slots aren't tracked, if that's what you mean. 19:11:08 oh, right. but it's a lispobj 19:11:13 oh, ok, then yes. 19:11:28 yeah. i was talking about unboxed slots an arrays 19:11:53 I had some other branches that performed trivial static analyses before emitting the barrier (e.g. when writing NIL, or fixnums, or non-GCed literals) 19:12:16 actually, i smell a cheapish defstruct slot option :weak in the air 19:13:00 but I think the win from eliding those barriers is probably noise, compared to getting our GC better. 19:13:13 fe[nl]ix: not sure... I could also just get a job ;) 19:13:24 fe[nl]ix: what did you have in mind? 19:14:35 all the above GC features, a precise x86_64 GC 19:15:20 All I need is reliable backtraces. 19:18:25 I've been considering attempting to partition the x86-64 register set and split the C and control stacks, but haven't actually made the attempt yet. 19:18:32 TimKack [~user@c-2ec20fcf-74736162.cust.telenor.se] has joined #sbcl 19:19:28 huangjs [~huangjs@190.8.100.83] has joined #sbcl 19:21:02 I don't think the registers are that much of an issue 19:23:51 They are for scavenging interrupt contexts. 19:27:44 re: freeze, I don't know either. if someone wants one at a specific point in time, please suggest 19:28:53 otherwise it's likely to be a some random time when I notice there hasn't been a sbcl release in a while 19:29:29 jsnell: well, it seems like it's been almost a month, and many changes 19:40:09 sdemarre [~serge@91.176.39.207] has joined #sbcl 19:55:22 -!- nikodemus [~nikodemus@178-55-57-180.bb.dnainternet.fi] has quit [Ping timeout: 272 seconds] 20:09:30 -!- specbot [~specbot@pppoe.178-66-80-40.dynamic.avangarddsl.ru] has quit [Disconnected by services] 20:10:16 specbot [~specbot@pppoe.178-66-10-200.dynamic.avangarddsl.ru] has joined #sbcl 20:10:52 -!- stassats` [~stassats@wikipedia/stassats] has quit [Read error: Operation timed out] 20:13:49 attila_lendvai [~attila_le@unaffiliated/attila-lendvai/x-3126965] has joined #sbcl 20:18:14 ok... what if our byte vector spanned the whole address space? 20:18:47 only 256 GB of address space would be needed on x86-64, with 1024 byte per card ;) 20:19:01 TimKack` [~user@c-2ec295b6-74736162.cust.telenor.se] has joined #sbcl 20:20:25 -!- TimKack [~user@c-2ec20fcf-74736162.cust.telenor.se] has quit [Ping timeout: 256 seconds] 20:20:34 upside: we would save one instruction off the barrier sequence 20:20:43 and no false positive 20:20:46 -!- homie [~levgue@xdsl-87-79-192-69.netcologne.de] has quit [Read error: Connection reset by peer] 20:21:21 downside: there's a statically-linked 256 GB array in our runtime. Some OSes might not like that. 20:21:23 homie [~levgue@xdsl-87-79-192-69.netcologne.de] has joined #sbcl 20:26:09 -!- TimKack` is now known as TimKack 20:26:48 tsuru`` [~charlie@adsl-74-179-20-245.bna.bellsouth.net] has joined #sbcl 20:29:10 -!- tsuru` [~charlie@adsl-74-240-217-131.bna.bellsouth.net] has quit [Ping timeout: 276 seconds] 20:33:29 -!- edgar-rft [~user@HSI-KBW-078-043-123-191.hsi4.kabel-badenwuerttemberg.de] has quit [Quit: ERC Version 5.3 (IRC client for Emacs)] 20:35:59 pkhuong: it's worth trying 20:37:26 At least it could sit in the bss segment. (-: 20:38:07 yup. 20:40:38 -!- angavrilov [~angavrilo@217.71.227.190] has quit [Ping timeout: 240 seconds] 20:41:33 hmm, many people run sbcl on cheap virtual servers that limit virtual memory 20:41:57 a 256GB array would probably trigger the kernel killer 20:42:32 let's try it out, just to see the perf impact. 20:43:04 sure 20:43:14 if you want a tester, here I am 20:58:01 oh wow. We need some stuff at low (< 32 bit) addresses. 20:58:43 can't have more than 4G of static data. 21:01:42 -!- easy-iPad [~easyipad@213.47.71.36] has quit [Quit: Outta here?] 21:03:47 ah right, and an EA's displacement field is only 32 bits. 21:26:22 -!- tsuru`` [~charlie@adsl-74-179-20-245.bna.bellsouth.net] has quit [Ping timeout: 276 seconds] 21:47:59 -!- homie [~levgue@xdsl-87-79-192-69.netcologne.de] has quit [Read error: Connection reset by peer] 21:48:36 homie [~levgue@xdsl-87-79-192-69.netcologne.de] has joined #sbcl 21:57:05 -!- sdemarre [~serge@91.176.39.207] has quit [Ping timeout: 260 seconds] 22:21:51 TimKack` [~user@c-2ec2a1f0-74736162.cust.telenor.se] has joined #sbcl 22:23:08 -!- TimKack [~user@c-2ec295b6-74736162.cust.telenor.se] has quit [Ping timeout: 240 seconds] 22:35:37 -!- nyef [~nyef@c-174-63-105-188.hsd1.ma.comcast.net] has quit [Quit: G'night all.]