00:58:22 rme [~rme@50.43.163.42] has joined #ccl 02:13:03 -!- patrickwonders [~patrickwo@user-38q42ns.cable.mindspring.com] has quit [Quit: patrickwonders] 02:23:02 -!- segv- [~mb@f052236089.adsl.alicedsl.de] has quit [Ping timeout: 264 seconds] 03:12:41 patrickwonders [~patrickwo@user-38q42ns.cable.mindspring.com] has joined #ccl 03:37:36 ddp [~ddp@c75-111-101-191.amrlcmta01.tx.dh.suddenlink.net] has joined #ccl 03:38:20 -!- ddp [~ddp@c75-111-101-191.amrlcmta01.tx.dh.suddenlink.net] has quit [Client Quit] 03:54:36 -!- gbyers [~gb@2601:0:b80:238:81cb:9aa6:3fd3:fe0e] has quit [Ping timeout: 252 seconds] 03:55:32 gbyers [~gb@2601:0:b80:238:206c:7bf8:d337:8b00] has joined #ccl 04:02:38 -!- Vivitron [~Vivitron@c-50-172-44-193.hsd1.il.comcast.net] has quit [Ping timeout: 264 seconds] 04:27:27 slarti [~anonymous@67-0-129-177.albq.qwest.net] has joined #ccl 05:08:38 -!- gbyers [~gb@2601:0:b80:238:206c:7bf8:d337:8b00] has quit [Ping timeout: 264 seconds] 05:12:22 gbyers [~gb@2601:0:b80:238:206c:7bf8:d337:8b00] has joined #ccl 07:01:04 -!- slarti [~anonymous@67-0-129-177.albq.qwest.net] has quit [Quit: slarti] 07:04:35 Vivitron [~Vivitron@c-50-172-44-193.hsd1.il.comcast.net] has joined #ccl 08:01:31 -!- wws [wws@clozure-323C97FB.dsl1.rtr.chat.fpma.frpt.net] has quit [Ping timeout] 08:03:14 -!- wws [~billstcla@p-69-195-52-152.dsl1.rtr.chat.fpma.frpt.net] has quit [Ping timeout: 264 seconds] 08:07:29 -!- rme [rme@3363A27E.B0A14555.699BA7A6.IP] has quit [Quit: rme] 08:07:29 -!- rme [~rme@50.43.163.42] has quit [Quit: rme] 09:17:05 -!- Vivitron [~Vivitron@c-50-172-44-193.hsd1.il.comcast.net] has quit [Ping timeout: 265 seconds] 09:59:13 ogamita [~t@tru75-h02-31-38-72-69.dsl.sta.abo.bbox.fr] has joined #ccl 09:59:55 -!- asedeno [~asedeno@66.102.14.16] has quit [Ping timeout: 260 seconds] 10:00:39 asedeno [~asedeno@66.102.14.24] has joined #ccl 11:09:54 -!- darage [~dlrager@inet-hqmc05-o.oracle.com] has quit [Remote host closed the connection] 11:10:56 darage [~dlrager@inet-hqmc05-o.oracle.com] has joined #ccl 12:22:22 segv- [~mb@e179036192.adsl.alicedsl.de] has joined #ccl 13:00:51 -!- aftershave [~textual@h-238-41.a336.priv.bahnhof.se] has quit [Quit: Textual IRC Client: www.textualapp.com] 13:01:55 aftershave [~textual@h-238-41.a336.priv.bahnhof.se] has joined #ccl 13:25:56 STilda [~kvirc@188.162.167.38] has joined #ccl 14:33:41 jkleiser [~jkleiser@f3bmac3.uio.no] has joined #ccl 14:51:26 -!- patrickwonders [~patrickwo@user-38q42ns.cable.mindspring.com] has quit [Quit: patrickwonders] 14:54:19 billstclair [~billstcla@unaffiliated/billstclair] has joined #ccl 15:11:29 -!- jkleiser [~jkleiser@f3bmac3.uio.no] has quit [Remote host closed the connection] 15:14:56 patrickwonders [~patrickwo@67-220-27-111.usiwireless.com] has joined #ccl 16:04:48 milanj [~milanj@82.117.199.26] has joined #ccl 16:09:07 -!- patrickwonders [~patrickwo@67-220-27-111.usiwireless.com] has quit [Quit: patrickwonders] 16:12:05 patrickwonders [~patrickwo@67-220-27-111.usiwireless.com] has joined #ccl 16:26:45 rme [~rme@50.43.163.42] has joined #ccl 16:28:05 -!- aftershave [~textual@h-238-41.a336.priv.bahnhof.se] has quit [Ping timeout: 272 seconds] 16:29:19 aftershave [~textual@h-238-41.a336.priv.bahnhof.se] has joined #ccl 16:45:51 -!- patrickwonders [~patrickwo@67-220-27-111.usiwireless.com] has quit [Quit: patrickwonders] 16:55:20 Hi all, I assume many of you have seen my posts about editing the CCL source code to reduce the virtual memory footprint [ http://comments.gmane.org/gmane.lisp.openmcl.devel/8989 ]. Does anyone have a suggestion for how I might successfully compile and run CCL with a virtual memory allocation less than 128GB on linuxx86-64? 16:55:52 Maybe I've made the necessary change and I've just run into a bootstrapping issue 16:56:54 slarti [~anonymous@67-0-129-177.albq.qwest.net] has joined #ccl 16:59:55 I saw your posts, but I haven't had any time to investigate it myself, I'm afraid. 17:20:58 -!- slarti [~anonymous@67-0-129-177.albq.qwest.net] has quit [Quit: slarti] 17:25:09 patrickwonders [~patrickwo@user-38q42ns.cable.mindspring.com] has joined #ccl 17:41:21 Roger that 17:43:02 Going through the bootstrapping process in section 3.6 of the CCL manual didn't seem to help 17:43:58 (I thought it might have, but it still crashed with the same sig11 when it went to restart the image) 17:54:18 aftersha_ [~textual@h-238-41.a336.priv.bahnhof.se] has joined #ccl 17:55:43 -!- aftersha_ [~textual@h-238-41.a336.priv.bahnhof.se] has quit [Client Quit] 18:06:19 Vivitron [~Vivitron@c-50-172-44-193.hsd1.il.comcast.net] has joined #ccl 18:10:32 -!- Vivitron [~Vivitron@c-50-172-44-193.hsd1.il.comcast.net] has quit [Ping timeout: 245 seconds] 19:02:43 -!- milanj [~milanj@82.117.199.26] has quit [Ping timeout: 272 seconds] 19:04:01 Vivitron [~Vivitron@c-50-172-44-193.hsd1.il.comcast.net] has joined #ccl 19:08:02 -!- Vivitron [~Vivitron@c-50-172-44-193.hsd1.il.comcast.net] has quit [Ping timeout: 245 seconds] 19:17:23 Vivitron` [~Vivitron`@c-50-172-44-193.hsd1.il.comcast.net] has joined #ccl 19:23:31 darage: I was going to respond eventually. Does your code use HONS/hash-consing ? 19:26:02 Good deal, indeed, we use the hash-consing build of ACL2 (ACL2(h)) 19:27:02 How little room for static conses do you want ? 19:28:19 maybe 4GB? Let me see if I can come up with a plausible answer to that question 19:29:44 That's a lot of what you're talking about here: the 128GB that bothers something (or you're afraid might bother something, I'm not too clear on that ...) is effectively "reserved for static things, including static conses". 19:32:23 That makes sense; if you know which constants I can play with to toggle the size of the static areas, I could play with them and see what happens 19:33:11 The CCL code seems to be pretty nice -- e.g., MANAGED_STATIC_SIZE is calculated based upon the value of PURESPACE_RESERVE 19:33:37 milanj [~milanj@cable-178-148-12-30.dynamic.sbb.rs] has joined #ccl 19:33:53 and PURESPACE_SIZE is only 1GB, which is significnatly less than the PURESPACE_RESERVE setting of 8GB I've been trying 19:36:12 I can't easily draw a diagram in IRC (I suppose that I could paste something somewhere), but there are a few static areas in that first 128GB and they grow towards each other. You could leave a hole in the middle. 19:39:17 -!- STilda [~kvirc@188.162.167.38] has quit [Ping timeout: 245 seconds] 19:40:00 If you did that, the areas could still grow if they needed to, unless some C code somewhere allocated something in that hole. The address space is so huge that that'd rarely happen, but we used to see a case where Apple's JIT compiler for Javascript used to allocate memory in the middle of CCL's heap (without bothering to check that the memory was already mapped.) That was clearly a bug, but it would have been harder to say "well, we weren't 19:40:01 using it yet, but we were planning to". 19:40:09 STATIC_RESERVE looks like a rather small number 19:40:14 (2^12) 19:40:41 That's a few pages around NIL. 19:41:46 How bizarre (the Javascript compiler) 19:43:06 They eventually fixed it. 19:43:16 Is most of the 128GB "pure" space 19:43:46 ? (format t "~x" (ccl::%address-of nil)) 19:44:30 nice 19:45:34 jangle [~jimmy1984@pool-108-15-111-28.bltmmd.fios.verizon.net] has joined #ccl 19:46:00 I see "static space", "pure space", "reserved area" 19:48:01 Yep. There's a lot of stuff. It'd be helpful to me to know whether you can't run on some cluster because the cluster thinks that the "VM size" is too large, or whether you're just concerned that it might not. 19:48:58 CCL can run on the cluster, but the cluster then keeps any other jobs from running on the machine 19:49:23 Because the cluster's heuristics assume that CCL might at some point consume all 512 GB 19:49:31 so the cluster tries to keep that 512GB free for CCL to use 19:50:01 So it effectively prevents the other 15 cores in the cluster machine from being used 19:50:17 -!- jangle [~jimmy1984@pool-108-15-111-28.bltmmd.fios.verizon.net] has quit [Client Quit] 19:50:39 jangle [~jimmy1984@pool-108-15-111-28.bltmmd.fios.verizon.net] has joined #ccl 19:51:02 For now, if there were some constants I could tweak, that would be good enough 19:51:10 If the 512GB was reduced (soon after the lisp started), do you know whether the cluster would let other things run. 19:51:17 Yes, it would 19:51:37 And I'm running CCL with -R , so I'm already down to 129GB 19:52:55 OK, I'll try to explain how to do that. It's not just a matter of tweaking constants; there are assumed relationships about where different areas of memory are in relation to each other. 19:53:04 Indeed, that makes sense 19:55:57 We reserve 128GB at the start of the 512GB, and things at either end of the 128GB can grow towards each other. If we unmap the reserved pages in the middle, we'll suddenly not seem to be using as much memory (that we weren't really "using" in the first place) 19:57:42 Haha, how evil would that be, but it'd work :) 19:58:48 If you want to know what memory looks like, determine your process ID (PID) with (ccl::getpid) (IIRC), then look at /proc//maps 19:59:43 -!- ogamita [~t@tru75-h02-31-38-72-69.dsl.sta.abo.bbox.fr] has quit [Ping timeout: 272 seconds] 20:00:32 Ah, I see a lot at 0x00*, 0x317*, 0x30*, 0x7f* and 0xffffff* 20:00:37 Neat 20:00:48 If we did that (leave a hole in the middle), the it wouldn't be a bug if somehing (a Javascript compiler) tried to put its own stuff in the hole, and we couldn't grow stuff into the hole. 20:02:11 Is this just a javascript compiler that's called from CCL (via a FFI or something?) 20:03:19 At the moment, I'm just looking for a hack that will work most of the time 20:03:51 Well, that hopefully I won't be able to break 20:04:05 But it doesn't have to be 100% robust 20:04:14 It was part of an example; you can use the powerful Cocoa WebKit classes to basically say (make-instance 'web-browser) and that'd basically create a window with a web browser in it (in the Cocoa IDE). 20:04:35 Don't worry, it won't be 100% robust ...) 20:04:40 Haha 20:04:56 Okay, that makes sense 20:06:00 I'll try to respond to your message later today (or over the weekend if I can't) 20:06:41 That'd be great, thanks 20:06:51 32 20:06:54 (wrong window) 20:07:27 -!- jangle [~jimmy1984@pool-108-15-111-28.bltmmd.fios.verizon.net] has left #ccl 22:18:14 -!- milanj [~milanj@cable-178-148-12-30.dynamic.sbb.rs] has quit [Quit: Leaving] 22:31:59 -!- darage [~dlrager@inet-hqmc05-o.oracle.com] has quit [Remote host closed the connection] 22:32:43 darage [~dlrager@inet-hqmc05-o.oracle.com] has joined #ccl 23:50:22 -!- rme [~rme@50.43.163.42] has quit [Quit: rme]