Previous Section Next Chapter Table of Contents Glossary Index

Chapter 3. Building Clozure CL from its Source Code

3.6. Building the Heap Image

The initial heap image is loaded by the Lisp kernel, and provides most of the language implementation The heap image captures the entire state of a running Lisp (except for external resources, such as open files and TCP sockets). After it is loaded, the contents of the new Lisp process's memory are exactly the same as those of the old Lisp process when the image was created.

The heap image is how we get around the fact that we can't run Lisp code until we have a working Lisp implementation, and we can't make our Lisp implementation work until we can run Lisp code. Since the heap image already contains a fully-working implementation, all we need to do is load it into memory and start using it.

If you're building a new version of Clozure CL, you need to build a new heap image.

(You might also wish to build a heap image if you have a large program that is very complicated or time-consuming to load, so that you will be able to load it once, save an image, and thenceforth never have to load it again. At any time, a heap image capturing the entire memory state of a running Lisp can be created by calling the function ccl:save-application.)

3.6.1. Development cycle

Creating a new Clozure CL full heap image consists of the following steps:

  1. Using your existing Clozure CL, create a bootstrapping image

  2. Using your existing Clozure CL, recompile your updated Clozure CL sources

  3. Invoke Clozure CL with the bootstrapping image you just created (rather than with the existing full heap image).

When you invoke Clozure CL with the bootstrapping image, it starts up, loads all of the Clozure CL fasl files, and saves out a new full heap image. Voila. You've created a new heap image.

A few points worth noting:

  • There's a circular dependency between the full heap image and the bootstrapping image, in that each is used to build the other.

  • There are some minor implementation differences, but the environment in effect after the bootstrapping image has loaded its fasl files is essentially equivalent to the environment provided by the full heap image; the latter loads a lot faster and is easier to distribute, of course.

  • If the full heap image doesn't work (because of an OS compatibilty problem or other bug), it's very likely that the bootstrapping image will suffer the same problems.

Given a bootstrapping image and a set of up-to-date fasl files, the development cycle usually involves editing lisp sources (or updating those sources via svn update), recompiling modified files, and using the bootstrapping image to produce a new heap image.

3.6.2. Generating a bootstrapping image

The bootstrapping image isn't provided in Clozure CL distributions. It can be built from the source code provided in distributions (using a lisp image and kernel provided in those distributions) using the procedure described below.

The bootstrapping image is built by invoking a special utility inside a running Clozure CL heap image to load files contained in the ccl/level-0 directory. The bootstrapping image loads several dozen fasl files. After it's done so, it saves a heap image via save-application. This process is called "cross-dumping".

Given a source distribution, a lisp kernel, and a heap image, one can produce a bootstrapping image by first invoking Clozure CL from the shell:

shell> ccl
Welcome to Clozure CL .... !

then calling ccl:xload-level-0 at the lisp prompt:

? (ccl:xload-level-0)

This function compiles the lisp sources in the ccl/level-0 directory if they're newer than the corresponding fasl files and then loads the resulting fasl files into a simulated lisp heap contained in data structures inside the running lisp. That simulated heap image is then written to disk.

xload-level-0 should be called whenever your existing boot image is out-of-date with respect to the source files in ccl:level-0; — For example:

? (ccl:xload-level-0 :force)

forces recompilation of the level-0 sources.

3.6.3. Generating fasl files


? (ccl:compile-ccl)

at the lisp prompt compiles any fasl files that are out-of-date with respect to the corresponding lisp sources; (ccl:compile-ccl t) forces recompilation. ccl:compile-ccl reloads newly-compiled versions of some files; ccl:xcompile-ccl is analogous, but skips this reloading step.

Unless there are bootstrapping considerations involved, it usually doesn't matter whether these files are reloaded after they're recompiled.

Calling compile-ccl or xcompile-ccl in an environment where fasl files don't yet exist may produce warnings to that effect whenever files are required during compilation; those warnings can be safely ignored. Depending on the maturity of the Clozure CL release, calling compile-ccl or xcompile-ccl may also produce several warnings about undefined functions, etc. They should be cleaned up at some point.

3.6.4. Building a full image from a bootstrapping image

To build a full image from a bootstrapping image, just invoke the kernel with the bootstrapping image as an argument

$ cd ccl                        # wherever your ccl directory is
$ ./KERNEL --image-name BOOT_IMAGE --no-init

Where KERNEL and BOOT_IMAGE are the names of the kernel and boot image appropriate to the platform you are running on. See Table 3.1, “Platform-specific filename conventions”

That should load a few dozen fasl files (printing a message as each file is loaded.) If all of these files successfully load, the lisp will print a prompt. You should be able to do essentially everything in that environment that you can in the environment provided by a "real" heap image. If you're confident that things loaded OK, you can save that image:

? (ccl:save-application "image_name") ; Overwriting the existing heap image

Where image_name is the name of the full heap image for your platform. See Table 3.1, “Platform-specific filename conventions”.

If things go wrong in the early stages of the loading sequence, errors are often difficult to debug; until a fair amount of code (CLOS, the CL condition system, streams, the reader, the read-eval-print loop) is loaded, it's generally not possible for the lisp to report an error. Errors that occur during these early stages ("the cold load") sometimes cause the lisp kernel debugger (see ) to be invoked; it's primitive, but can sometimes help one to get oriented.

Previous Section Next Chapter Table of Contents Glossary Index