00:02:04 -!- loke [~elias@bb115-66-85-121.singnet.com.sg] has quit [Ping timeout: 258 seconds] 00:07:01 -!- prxq [~mommer@mnhm-5f75c4f3.pool.mediaWays.net] has quit [Quit: good night] 00:17:03 -!- milanj [~milanj_@109-92-103-199.dynamic.isp.telekom.rs] has quit [Quit: Leaving] 00:18:40 loke [~elias@bb115-66-85-121.singnet.com.sg] has joined #sbcl 00:19:02 -!- Qworkescence [~quad@unaffiliated/quadrescence] has quit [Quit: Leaving] 00:36:30 homie [~levgue@xdsl-78-35-189-167.netcologne.de] has joined #sbcl 01:24:10 pchrist_ [~spirit@gentoo/developer/pchrist] has joined #sbcl 01:26:53 -!- pchrist [~spirit@gentoo/developer/pchrist] has quit [Ping timeout: 252 seconds] 02:01:52 -!- homie [~levgue@xdsl-78-35-189-167.netcologne.de] has quit [Quit: ERC Version 5.3 (IRC client for Emacs)] 03:33:42 hargettp [~hargettp@pool-71-174-136-174.bstnma.east.verizon.net] has joined #sbcl 04:36:54 -!- hargettp [~hargettp@pool-71-174-136-174.bstnma.east.verizon.net] has quit [Quit: Linkinus - http://linkinus.com] 04:46:35 drdo [~drdo@85.207.54.77.rev.vodafone.pt] has joined #sbcl 05:31:41 -!- drdo [~drdo@85.207.54.77.rev.vodafone.pt] has quit [Read error: Operation timed out] 07:56:56 ottergwc [~brian.jon@wsip-24-234-75-217.lv.lv.cox.net] has joined #sbcl 07:58:27 Hi, I've got a problem. I'm having sbcl crash hard when doing heavy threading in the check-deadlock function, in particular the label lock-owner. It's being passed nil, which comes from the detect-deadlock call at the bottom of the label. 07:58:52 Nil blows the etypcase, and it's only called (unless (consp origin) (detect-deadlock origin)) 07:59:03 I'm wondering ... should that be null instead of consp in unless? 07:59:39 target-thread.lisp line 470 08:11:29 It looks like a cons could be (timeout . lock) which would be skipped by consp which isn't null, so it may not be just that. 08:13:03 Kryztof [~user@81.174.155.115] has joined #sbcl 08:13:03 -!- ChanServ has set mode +o Kryztof 08:15:35 attila_lendvai [~attila_le@unaffiliated/attila-lendvai/x-3126965] has joined #sbcl 08:20:50 Hmm. check-deadlock seems to be used only in get-mutex and get-spinlock, and get-mutex says deprecated in favor of grab-mutex. 08:26:29 and the sbcl site seems down; skimming the report at http://random-state.net/log/3523852985.html what I'm dealing with my be already addressed. 08:28:44 *ottergwc* clones the git and pokes around 08:31:10 And ... that looks nicer. Time to go bleeding edge I guess. 08:32:06 angavrilov [~angavrilo@217.71.227.181] has joined #sbcl 08:42:02 -!- attila_lendvai [~attila_le@unaffiliated/attila-lendvai/x-3126965] has quit [Ping timeout: 276 seconds] 10:42:00 nikodemus [~nikodemus@cs181063174.pp.htv.fi] has joined #sbcl 10:42:00 -!- ChanServ has set mode +o nikodemus 10:43:43 morning 11:35:21 Blkt [~user@82.84.190.228] has joined #sbcl 11:35:49 antgreen [~user@bas3-toronto06-1177889870.dsl.bell.ca] has joined #sbcl 11:37:40 attila_lendvai [~attila_le@unaffiliated/attila-lendvai/x-3126965] has joined #sbcl 11:41:00 good morning everyone 12:28:14 i wonder... 12:28:34 what if we defaulted nursery size to eg. 5% of dynamic-space-size? 13:19:35 blackwol` [~blackwolf@ool-45763eb0.dyn.optonline.net] has joined #sbcl 13:21:04 -!- blackwolf [~blackwolf@ool-45763eb0.dyn.optonline.net] has quit [Ping timeout: 240 seconds] 13:33:15 hargettp [~hargettp@pool-71-174-136-174.bstnma.east.verizon.net] has joined #sbcl 13:38:26 LiamH [~healy@pool-108-18-166-81.washdc.east.verizon.net] has joined #sbcl 14:04:57 -!- hargettp [~hargettp@pool-71-174-136-174.bstnma.east.verizon.net] has quit [Quit: Linkinus - http://linkinus.com] 14:14:18 homie [~levgue@xdsl-78-35-153-133.netcologne.de] has joined #sbcl 14:46:58 eh. shaves sbcl build times from ~7 minutes to ~5:30... 15:05:51 doing the same to generation.bytes_consed_between_gc -> 5:20 15:29:21 nikodemus: you could also try disabling generationality completely ;) 15:31:12 pkhuong: i did at some point run it like that 15:31:33 iirc all the fixes are these -- just needs a startup option :) 15:32:15 it'll be interesting to see if someone has a workload for which 5% is worse than the old defaults 15:32:52 I could construct one ;) 15:33:11 But I don't think CL tends to be written that way, nowadays. 15:36:35 i wonder if i'll ever learn to use MISMATCH without checking clhs 15:48:24 -!- pchrist_ [~spirit@gentoo/developer/pchrist] has quit [Quit: leaving] 15:48:54 pchrist [~spirit@gentoo/developer/pchrist] has joined #sbcl 16:15:04 -!- Blkt [~user@82.84.190.228] has quit [Quit: ERC Version 5.3 (IRC client for Emacs)] 17:17:46 -!- deepfire [~deepfire@80.92.100.69] has quit [Ping timeout: 255 seconds] 17:33:57 nikodemus: do you think we could/should have primitives for atomic swap? 17:34:18 either place/value or place/place. 17:37:47 pkhuong: i think we should 17:38:18 i've been meaning to do that for a long while -- our futex/mutex release is suboptimal because we don't have it, iirc 17:54:32 \o/ reports of threads working apparently OK on freebsd-current 17:55:06 awesome. 17:56:13 nikodemus pasted "this amuses me" at http://paste.lisp.org/display/125969 17:59:29 ok, enough random hackery for the weekend, i think 17:59:52 -!- LiamH [~healy@pool-108-18-166-81.washdc.east.verizon.net] has quit [Quit: Leaving.] 17:59:58 LiamH [~healy@pool-108-18-166-81.washdc.east.verizon.net] has joined #sbcl 18:03:51 -!- nikodemus [~nikodemus@cs181063174.pp.htv.fi] has quit [Quit: This computer has gone to sleep] 18:04:18 nikodemus [~Nikodemus@cs181063174.pp.htv.fi] has joined #sbcl 18:04:18 -!- ChanServ has set mode +o nikodemus 18:14:22 -!- LiamH [~healy@pool-108-18-166-81.washdc.east.verizon.net] has quit [Quit: Leaving.] 18:31:24 -!- Kryztof [~user@81.174.155.115] has quit [Remote host closed the connection] 18:32:58 -!- nikodemus [~Nikodemus@cs181063174.pp.htv.fi] has quit [Ping timeout: 258 seconds] 18:35:48 nikodemus [~Nikodemus@GGZYYYMMCCXV.gprs.sl-laajakaista.fi] has joined #sbcl 18:35:48 -!- ChanServ has set mode +o nikodemus 18:44:56 -!- attila_lendvai [~attila_le@unaffiliated/attila-lendvai/x-3126965] has quit [Ping timeout: 244 seconds] 19:11:15 tyson1 [~Ian@69-165-217-17.dsl.teksavvy.com] has joined #sbcl 19:36:02 -!- tyson1 [~Ian@69-165-217-17.dsl.teksavvy.com] has left #sbcl 19:47:02 nikodemus` [~nikodemus@cs181063174.pp.htv.fi] has joined #sbcl 19:49:05 -!- angavrilov [~angavrilo@217.71.227.181] has quit [Ping timeout: 260 seconds] 20:05:31 -!- nikodemus` [~nikodemus@cs181063174.pp.htv.fi] has quit [Quit: This computer has gone to sleep] 20:08:55 LiamH [~healy@pool-108-18-166-81.washdc.east.verizon.net] has joined #sbcl 20:46:25 -!- homie [~levgue@xdsl-78-35-153-133.netcologne.de] has quit [Read error: Connection reset by peer] 20:56:16 -!- nikodemus [~Nikodemus@GGZYYYMMCCXV.gprs.sl-laajakaista.fi] has quit [Ping timeout: 244 seconds] 21:05:16 homie [~levgue@xdsl-78-35-153-133.netcologne.de] has joined #sbcl 21:18:19 Kryztof [~user@81.174.155.115] has joined #sbcl 21:18:19 -!- ChanServ has set mode +o Kryztof 21:28:46 nikodemus [~Nikodemus@cs181063174.pp.htv.fi] has joined #sbcl 21:28:46 -!- ChanServ has set mode +o nikodemus 21:30:25 nikodemus: re your nursery size, we've found that it can be very important to vary that by core count 21:31:21 for a heavily functional-style program on a 24-core box, running on a 9GB nursery iirc was 2-3x faster than a 1GB nursery, seemingly due to the single-threaded GC and managing all the thread start/stop foo 21:32:11 a 1GB nursery being our manually-sampled "pretty good" point on 4-core, 8GB RAM machines 21:32:38 same workload? 21:32:41 yes 21:32:49 curious 21:33:21 more GCs on many cores = lots of idle core time, more start/stop work 21:33:53 oh... had you fixed the setf bytes-consed-between-gcs bug manually? 21:34:10 locally, i mean 21:34:21 yeah, and one of the guys submitted a patch if you haven't seen that yet 21:34:49 i did, and the bug is fixed on master 21:34:51 we've been manually patching that since v1.0.30-something, finally migrated into git and figured out how to properly do its patches :-P 21:34:55 cool, thanks 21:35:57 generally speaking, the bigger the nursery, the better the thoroughout - but latency suffers 21:36:41 oh. you spawn a number of threads proportional to number of cores, right? 21:36:47 and getting into swap space or out-of-memory conditions is bigger 21:37:03 yeah, our system is a threadpool, though there are plenty of other IO and other idle threads hanging around 21:37:26 Thanks for getting the GC in. 21:37:31 The gc patch rather. 21:37:57 made a difference for you? 21:38:08 Yes, I'm the one that was re-patching each release. 21:38:18 *Phoodus* works with ottergwc 21:38:18 aha :) 21:38:36 And I tracked the bug we had to the target-thread.lisp last night. To discover you fixed it three days back. 21:38:44 anything else you have as local patches? 21:38:54 No, that was the only local one we had. 21:39:08 It was an issue because on 24 GB box we were giving gigs to the gc size 21:39:28 We're pretty thread intense. 21:39:52 the 12Mb default was getting pretty silly 21:40:16 Yes. We've been doing a test (time (sleep 5)) to see the cores at work. That also reflects gc in a useful way. 21:40:27 On a box doing heavy work, we'll see hundreds of percent of CPU with 24 cores. 21:40:55 all these people actually using sbcl 21:41:00 in your gc tuning, have you experimented with the per-generation parameters? 21:41:00 yeah, that's been one of the most useful metrics for both thread utilization and how busy GC is 21:41:03 it was more restful when everyone pointed and laughed 21:41:18 heh 21:41:36 most of our data should be very short-lived 21:41:37 Actually, I had a detractor of Lisp at a meeting with a bunch of nuclear plants say "You can't use lisp, it's interpreted and very slow." 21:41:49 So I said "Really?" created a function and then dis-assembled it for them live. 21:41:56 That person then slunk away. 21:43:24 how much dynamic space on the box where you use the 9Gb nursery? 21:44:40 We give 20 GB total space using the command-line parameter and set the gc to 9gb. 21:44:56 ok 21:45:45 Did you have any plans for processor affinity? 21:45:59 not actively 21:46:10 In theory, we could move all our threads to an affinity pool and that should improve the throughput. 21:46:20 Not a big deal; we'd also have to get every _other_ service on the box to honor it. 21:46:22 unless you count sb-cpu-affinity 21:46:38 homie` [~levgue@xdsl-78-35-148-60.netcologne.de] has joined #sbcl 21:47:00 but i haven't touched that for quite a while 21:47:32 should still work, probably 21:47:58 we'll have to try that. 21:48:09 I'm also not sure how well affinity works for a threadpool model 21:48:40 -!- homie [~levgue@xdsl-78-35-153-133.netcologne.de] has quit [Ping timeout: 260 seconds] 21:48:44 -!- homie` [~levgue@xdsl-78-35-148-60.netcologne.de] has quit [Client Quit] 21:48:44 since your jobs are arbitrarily assigned to those threads, even if the threads stick to a core you really don't get that great of cache benefit if jobs on the same data are popping around 21:49:06 But that would help prevent the pool from losing its threads in contention with other threads in the app and the rest of the system. 21:49:52 right, though I'd say putting the affinity for the "other stuff" to 1 core is more important than affinitizing (now a word) the pool itself 21:50:08 if i remember right, it only gives you an affinity mask for the whole process 21:50:49 Ah. 21:50:50 thread affinity is still whatever the kernel happens to do 21:51:11 Of course, tuning a kernel to running our stuff would be possible. 21:51:40 but you can eg. mask out a core for the rest of the world, or make sure no-ones pre-set a narrow mask for you 21:52:47 Kryztof: are hawaian shirts coming next :) 21:53:13 or tie-dyes :p 21:53:15 homie [~levgue@xdsl-78-35-148-60.netcologne.de] has joined #sbcl 21:55:21 Phoodus, ottergwc: forgive me if i've asked and forgotten, but what does your stuff do? 21:56:20 inference-based application server 21:56:31 Phoodus: you can use mostly thread-local work queues. 21:56:45 true 21:57:16 though when there's a lot of work to do, jobs tend to run longer 21:57:30 we've got other feature-based issues to work on before that specific level of optimization 21:57:42 but reducing the number of GC calls has been pretty significant, due to the core count 21:58:16 obviously, we've got some strategies to reduce the amount of functional garbage into constrained mutable scopes 21:59:53 *Phoodus* goes afk 22:12:36 -!- Phoodus [~foo@68.107.217.139] has quit [Ping timeout: 276 seconds] 22:23:40 -!- nikodemus [~Nikodemus@cs181063174.pp.htv.fi] has quit [Ping timeout: 260 seconds] 22:28:48 nikodemus [~Nikodemus@cs181063174.pp.htv.fi] has joined #sbcl 22:28:48 -!- ChanServ has set mode +o nikodemus 23:01:30 deepfire [~deepfire@80.92.100.69] has joined #sbcl 23:03:20 -!- nikodemus [~Nikodemus@cs181063174.pp.htv.fi] has quit [Ping timeout: 260 seconds] 23:05:42 nikodemus [~Nikodemus@cs181063174.pp.htv.fi] has joined #sbcl 23:05:42 -!- ChanServ has set mode +o nikodemus 23:20:52 -!- lichtblau [~user@port-92-195-126-123.dynamic.qsc.de] has quit [Ping timeout: 245 seconds] 23:30:15 lichtblau [~user@port-92-195-21-77.dynamic.qsc.de] has joined #sbcl 23:30:41 -!- nikodemus [~Nikodemus@cs181063174.pp.htv.fi] has quit [Quit: Leaving]