CCL
List of Tables
Clozure CL is a fast, mature, open source Common Lisp implementation that runs on Linux, Mac OS X, FreeBSD, and Windows. Clozure CL was forked from Macintosh Common Lisp (MCL) in 1998 and the development has been entirely separate since.
When it was forked from MCL in 1998, the new Lisp was named OpenMCL. Subsequently, Clozure renamed its Lisp to Clozure CL, partly because its ancestor MCL has been released as open source. Clozure thought it might be confusing for users if there were two independent open-source projects with such similar names. The new name also reflects Clozure CL's current status as the flagship product of Clozure Associates.
Furthermore, the new name refers to Clozure CL's ancestry: in its early years, MCL was known as Coral Common Lisp, or "CCL". For years the package that contains most of Clozure CL's implementation-specific symbols has been named "CCL", an acronym that once stood for the name of the Lisp product. It seems fitting that "CCL" once again stands for the name of the product.
Some commands and source files may still refer to "OpenMCL" instead of Clozure CL.
Clozure CL compiles to native code and supports multithreading using native OS threads. It includes a foreign-function interface, and supports both Lisp code that calls external code, and external code that calls Lisp code. Clozure CL can create standalone executables on all supported platforms.
On Mac OS X, Clozure CL supports building GUI applications that use OS X's native Cocoa frameworks, and the OS X distributions include an IDE written with Cocoa, and distributed with complete sources.
On all supported platforms, Clozure CL can run as a command-line process, or as an inferior Emacs process using either SLIME or ILISP.
Features of Clozure CL include
Very fast compilation speed.
A fast, precise, compacting, generational garbage collector written in hand-optimized C. The sizes of the generations are fully configurable. Typically, a generation can be collected in a millisecond on modern systems.
Fast execution speed, competitive with other Common Lisp implementations on most benchmarks.
Robust and stable. Customers report that their CPU-intensive, multi-threaded applications run for extended periods on Clozure CL without difficulty.
Full native OS threads on all platforms. Threads are automatically distributed across multiple cores. The API includes support for shared memory, locking, and blocking for OS operations such as I/O.
Full Unicode support.
Full SLIME integration.
An IDE on Mac OS X, fully integrated with the Macintosh window system and User Interface standards.
Excellent debugging facilities. The names of all local variables are available in a backtrace.
A complete, mature foreign function interface, including a powerful bridge to Objective-C and Cocoa on Mac OS X.
Many extensions including: files mapped to Common Lisp vectors for fast file I/O; thread-local hash tables and streams to eliminate locking overhead; cons hashing support; and much more
Very efficient use of memory
Although it's an open-source project, available free of charge under a liberal license, Clozure CL is also a fully-supported product of Clozure Associates. Clozure continues to extend, improve, and develop Clozure CL in response to customer and user needs, and offers full support and development services for Clozure CL.
As of this writing, Clozure CL 1.7 is the latest release; it was made in August 2011. For up-to-date information about releases, please see http://ccl.clozure.com/.
Clozure CL 1.7 runs on the following platforms:
Linux (x86, x86-64, ppc32, ppc64, armv7)
Mac OS X 10.5 and later (x86, x86-64)
FreeBSD 6.x and later (x86, x86-64)
Solaris (x86, x86-64)
Microsoft Windows XP and later (x86, x86-64)
Naturally, 64-bit versions of Clozure CL require 64-bit processors, for example, a G5 or Core 2. Some early Intel-based Macintoshes used processors that don't support 64-bit operation, so the 64-bit Clozure CL will not run on them, although the 32-bit Clozure CL will.
The 32-bit x86 versions of Clozure CL depend on the presence of the SSE2 instructions. Most x86 processors manufactured and sold in the last several years support SSE2 (all Apple Intel-based Macs do, for instance), but there are some exceptions. The Wikipedia article on SSE2 lists processor models that support SSE2 (and also mentions some of the more notable exceptions).
Clozure CL requires version 2.2.13 (or later) of the Linux kernel and version 2.1.3 (or later) of the GNU C library (glibc) at a bare minimum.
Because of the nature of Linux distributions, it's difficult to give precise version number requirements. In general, a "fairly modern" (no more than 2 or three years old) kernel and C library are more likely to work well than older versions.
The Linux ARM port is relatively new and is still a work-in-progress. Clozure CL needs some features (such as hardware floating-point, locking and memory-serialization primitives) that are only found in chips that implement architecture version 7 (ARMv7); technically, it needs the ARMv7 "application profile", which is sometimes called ARMv7a. In practice, most ARM consumer devices released in the last few years implement ARMv7, but there are exceptions, and it is not practical to enumerate all of the ARM devices that CCL should run on.
In addition to hardware issues, Clozure CL expects Linux to run in little-endian mode and expects software to follow "soft float" calling conventions. The latter has to do with how C functions accept floating-point arguments and return floating-point values.
Clozure CL should run on FreeBSD 6.x and 7.x. FreeBSD 7 users will need to install the "compat6x" package in order to use the distributed Clozure CL kernel, which is built on a FreeBSD 6.x system.
Clozure CL 1.7 runs on Mac OS X (x86) versions 10.5 and later, including 10.7 (Lion),
Clozure CL 1.6 runs on Mac OS X PPC as well as x86 processors.
There are three ways to obtain Clozure CL. For Mac OS X, there are disk images that can be used to install Clozure CL in the usual Macintosh way. For other OSes, Subversion is the best way to obtain Clozure CL. Mac OS X users can also use Subversion if they prefer. Tarballs are available for those who prefer them, but if you have Subversion installed, it is simpler and more flexible to use Subversion than tarballs.
There are three popular ways to use Clozure CL: as a stand-alone double-clickable application (Mac OS X only), as a command-line application, or with Emacs and SLIME.
The following sections describe these options.
If you are using Mac OS X then you can install and use Clozure CL in the usual Macintosh way. Download and mount a disk image, then drag the ccl folder to the Applications folder or wherever you wish. After that you can double-click the Clozure CL application found inside the ccl directory. The disk images for version 1.7 are available at ftp://clozure.com/pub/release/1.7/
So that Clozure CL can locate its source code, and for other
reasons explained in
Section 4.6.2, “Predefined Logical Hosts”, you keep the
Clozure CL application
in the ccl directory. If you use a shell,
you can set the value of the
CCL_DEFAULT_DIRECTORY environment variable
to explicitly indicate the location of
the ccl directory. If you choose to do
that, then the ccl directory and the Clozure CL
application can each be in any location you find
convenient.
Tarball distributions of Clozure CL release version 1.7 are available at ftp://clozure.com/pub/release/1.7/. Download and extract one on your local disk. Then edit the Clozure CL shell script to set the value of CCL_DEFAULT_DIRECTORY and start up the appropriate Clozure CL kernel. See Section 2.3.1, “The ccl Shell Script” for more information about the Clozure CL shell scripts.
It is very easy to download and configure Clozure CL to obtain sources from the Subversion repository. This is the preferred way to get either the latest, or a specific version of Clozure CL, unless you prefer the Mac Way. Subversion is a source code control system that is in wide use. Many OSes come with Subversion pre-installed. A complete, buildable and runnable set of Clozure CL sources and binaries can be retrieved with a single Subversion command.
Unless stated otherwise, examples in this chapter are given for Mac OS X in particular or Unix-based host environments in general.
For Windows, special care must be taken to install a working development environment. For more information see the Clozure CL Wiki at URL: http://trac.clozure.com/ccl/wiki/WindowsNotes
Make sure that Subversion is installed on your system. Bring up a command line shell and type:
shell> svn
If Subversion is installed, you will see something like:
Type 'svn help' for usage
If Subversion is not installed, you will see something like:
-bash: svn: command not found
If Subversion is not installed, you'll need to figure out how to install it on your OS. You can find information about obtaining and installing Subversion at the Subversion web page.
Before you download Clozure CL you should consider: Do you want to run the most recent source code, or the current stable release version? If you don't know how to answer this question, then you probably want the release version.
Day-to-day development of Clozure CL takes place in an area of the Subversion repository known as "the trunk". At most times, the trunk is perfectly usable, but occasionally it can be unstable or totally broken. If you wish to live on the bleeding edge, download sources from the trunk.
For example, the following command will fetch a copy of the trunk for Mac OS X (Darwin) with x86 processors (both 32- and 64-bit versions):
svn co http://svn.clozure.com/publicsvn/openmcl/trunk/darwinx86/ccl
To get a trunk Clozure CL for another platform, replace "darwinx86" with one of the following names (all versions include both 32- and 64-bit binaries):
darwinx86
linuxx86
freebsdx86
solarisx86
windows
linuxppc
darwinppc
Release versions of Clozure CL are intended to be stable. While bugs will be fixed in the release branches, enhancements and new features will go into the trunk. If you wish to run the stable release, the following command will fetch a copy of the release version 1.7 for Mac OS X (Darwin) with x86 processors (both 32- and 64-bit versions):
svn co http://svn.clozure.com/publicsvn/openmcl/release/1.7/darwinx86/ccl
To get the release version of Clozure CL for another platform, replace "darwinx86" with one of the following names:
darwinx86
linuxx86
freebsdx86
solarisx86
windows
linuxppc
darwinppc
These distributions contain complete sources and binaries. They use Subversion's "externals" features to share common sources; the majority of source code is the same across all versions.
This section explains how to peform a "full rebuild" of Clozure CL from a source distribution.
After downloading Clozure CL sources, you should rebuild Clozure CL as described here.
At the start of a full rebuild, object files in the ccl directory are deleted,
which causes the build script to recompile the runtime kernel (C code) and high-level sources (Lisp),
then save a new heap image.
Doing a full rebuild helps to ensure that your local installation will run properly for your host OS environment.
In an interactive shell, a command sequence like the following will rebuild Clozure CL in place:
joe> cd/path/to/installed/ccljoe:ccl> ./kernel-filename--no-init Welcome to Clozure Common Lisp Version [...] ? (rebuild-ccl :full t) <...lots of compilation output...> ? (quit) joe:ccl>
Replace /path/to/installed/ccl with the path of the ccl directory
that you downloaded.
Replace kernel-filename with the filename of the Lisp kernel program.
To find the filename of a Lisp kernel image for your particular platform, see Section 3.1.1, “Platform-specific filename conventions”.
Specifying the --no-init option ensures that personal initializations do not interfere
with rebuilding Clozure CL.
The rest of this section covers the following topics in brief:
This section does not provide comprehensive documentation on the build process. Please refer to Chapter 3, Building Clozure CL from its Source Code for more information. Those more detailed instructions are used mainly by developers who maintain, customize, and/or port Clozure CL. If you are customizing Clozure CL, or if you run into some exceptional situation, you may need to perform the individual build steps.
In order to build Clozure CL you must have a working system and development environment.
There are different requirements and setup procedures for each platform, but the main requirement is to have
a C compiler and a few other utilities:
GNU gcc or cc with ld and as;
make; and m4.
Please refer to Chapter 3, Building Clozure CL from its Source Code for details.
If you don't have the prerequisite C compiler toolchain installed, rebuild-ccl will not work.
See Section 3.3, “Kernel Build Prerequisites” for additional details.
Most distributions of Linux have all or most of the required development tools either pre-installed or readily available. On Debian-based Linux you can download and install the essential build tools using the package manager. For example:
apt-get install build-essential
(You may need to install C header files separately.)
For Mac OS X, Xcode 4 is available from the App Store.
For Windows, install Cygwin and the MinGW toolchain for the 32- or 64-bit OS. More information about installing Clozure CL on Windows is available in the Clozure CL Wiki at URL: http://trac.clozure.com/ccl/wiki/WindowsNotes
The most common scenario that requires a full rebuild is the standard installation after downloading the source tree. Users and application developers (who otherwise have no special build requirements) will generally need to run the full rebuild process just once for any given installation on a particular host system.
Another common scenario is installing a patch update:
You can use Subversion (svn update) to download a more recent set of source files.
(Be sure to download sources from the same path and branch in the source repository.)
Then run a full rebuild to create new kernel and heap images.
If you are running Clozure CL from the trunk, you may need to update sources and run the full rebuild more often.
Another reason to do a full rebuild is to ensure that Clozure CL will run properly in the host OS environment. This may be necessary, for example, when the target OS version is not identical to the one where the pre-built kernel was generated. The Lisp kernel uses some functionality defined in standard platform-provided libraries. On some platforms, applications (such as the Lisp kernel) are built in such a way as to depend on the specific versions of these libraries that were present at build time, and may not run on systems that have older or newer versions of these libraries. If you're affected by this, the simplest workaround is to build the Lisp kernel on the machine(s) that you intend to run it on and use that locally-built kernel instead of one distributed via Subversion.
Once the checkout is complete, and provided that you have a working development setup, you can build Clozure CL by running the Lisp kernel (an OS-native executable program) and running REBUILD-CCL in Lisp.
For example, to build a 64-bit Clozure CL on Mac OS X:
joe:ccl> ./dx86cl64 --no-init
Welcome to Clozure Common Lisp Version 1.7 (DarwinX8664)!
? (rebuild-ccl :full t)
Rebuilding Clozure Common Lisp using Version 1.7 (DarwinX8664)
;Building lisp-kernel ...
;Kernel built successfully.
;Compiling <...>
;Loading <...>
<...lots of compilation output...>
;Wrote bootstrapping image: #P"/Users/joe/ccl/x86-boot64.image"
;Wrote heap image: #P"/Users/joe/ccl/dx86cl64.image"
NIL
? (quit)
joe:ccl>
If the build fails for any reason, the kernel and/or heap image files may be missing or corrupted. To recover, delete the image files and update the source directory from Subversion. For example:
joe:ccl> rm dx86cl*
joe:ccl> svn update
<... lots of Subversion output...>
joe:ccl> ./dx86cl64 --no-init
Welcome to Clozure Common Lisp Version 1.7 (DarwinX8664)!
? (rebuild-ccl :full t)
<... lots of compilation output...>
? (quit)
joe:ccl>
Once the full rebuild is completed, you can run the new Lisp kernel from the command shell.
However, running the OS- and processor-specific executable directly is not recommended
for day-to-day use.
Clozure CL includes the ccl and ccl64 command shell scripts.
For details on configuring a shell script for your environment, see Section 2.3.1, “The ccl Shell Script”.
Should the build fail, your first concern should be to confirm that all requirements are in place: the C compiler, utilities, and OS header files; source files for the trunk or release branch you want to build; and the Lisp kernel and heap image files. For assistance with trouble-shooting, here is an outline of the full build process, with links to the more detailed instructions in Chapter 3, Building Clozure CL from its Source Code.
Build the Lisp kernel (Section 3.5, “Building the Kernel”)
Build the heap image (Section 3.6, “Building the Heap Image”)
Create a bootstrapping heap image (Section 3.6.2, “Generating a bootstrapping image”)
Compile Lisp code to generate fasl files (Section 3.6.3, “Generating fasl files”)
Build a full image from bootstrapping image (Section 3.6.4, “Building a full image from a bootstrapping image”)
Run new kernel with new bootstrapping image
Load Lisp code
Save a new full heap image
Sometimes it's convenient to use Clozure CL from a Unix shell command line. This is especially true when using Clozure CL as a way to run Common Lisp utilities.
Clozure CL needs to be able to find the
ccl directory in order to support features
such as require and
provide, access to foreign interface
information (see The
Interface Database) and the Lisp build process (see
Building Clozure CL from its Source
Code). Specifically, it needs to set up logical
pathname translations for the "ccl:"
logical host. If this logical host isn't defined (or isn't
defined correctly), some things might work, some things might
not, and it'll generally be hard to invoke and use Clozure CL
productively.
Clozure CL uses the value of the environment variable
CCL_DEFAULT_DIRECTORY to determine the
filesystem location of the ccl directory;
the ccl shell script is intended to provide a way to
invoke Clozure CL with that environment variable set
correctly.
There are two versions of the shell script:
"ccl/scripts/ccl" is used to invoke
32-bit implementations of Clozure CL and
"ccl/scripts/ccl64" is used to invoke
64-bit implementations.
Install one script or the other or both as needed.
To use the script:
Copy the script to a directory that is on your
PATH. This is often
/usr/local/bin or
~/bin. It is better to do this than to
add ccl/scripts to your
PATH, because the script needs to be edited,
and editing it in-place means that Subversion sees the script as
modified..
Edit the definition of
CCL_DEFAULT_DIRECTORY near the
beginning of the shell script so that it refers to
your ccl directory. Alternately, set
the value of the CCL_DEFAULT_DIRECTORY
environment variable
wherever you usually set per-user environment variables, in your
.cshrc, .tcshrc,
.bashrc, .bash_profile,
or .MacOSX/environment.plist script,
or system-wide in /etc/profile or /etc/bashrc.
When the ccl script runs, if the process environment contains
a definition of CCL_DEFAULT_DIRECTORY, the ccl
script will not override it.
Ensure that the shell script is executable, for example:
$ chmod +x
~/ccl/ccl/scripts/ccl64
This command grants execute permission to the named script. If you are using a 32-bit platform, substitute "ccl" in place of "ccl64".
The above command won't work if you are not the owner of the installed copy of Clozure CL. In that case, you can use the "sudo" command like this:
$ sudo chmod +x
~/ccl/ccl/scripts/ccl64
Give your password when prompted.
If the "sudo" command doesn't work, then you are not an administrator on the system you're using, and you don't have the appropriate "sudo" permissions. In that case you'll need to get help from the system's administrator.
Note that most people won't need both
ccl and ccl64 scripts.
You only need both if you sometimes run 32-bit Clozure CL and
sometimes run 64-bit Clozure CL. You can rename the script that
you use to whatever you want. For example, if you are on a
64-bit system, and you only use Clozure CL in 64-bit mode, then
you can rename ccl64 to
ccl so that you only need to type
"ccl" to run it.
Once this is done, it should be possible to invoke Clozure CL
by typing ccl
or ccl64 at a shell prompt:
shell> ccl
Welcome to Clozure Common Lisp Version 1.7 (DarwinX8632)!
?
The ccl shell script passes all of its arguments to the Clozure CL kernel. See Section 2.3.2, “Invocation” for more information about command-line arguments.
Assuming the shell script is configured and invoked properly, Clozure CL
should be able to initialize the "ccl:"
logical host so that its translations refer to the
"ccl" directory. To test this, you can call
probe-file in Clozure CL's read-eval-print
loop:
? (probe-file "ccl:level-1;level-1.lisp") ;returns the physical pathname of the file
#P"/Users/joe/my_lisp_stuff/ccl/level-1/level-1.lisp"
Assuming that the shell script is properly installed, it can be used to invoke Clozure CL from a shell prompt:
shell>ccl[args ...]
By convention
ccl runs a 32-bit session;
ccl64 runs a 64-bit session.
However, the name of the installed script(s) and the implementation that is invoked are customizable,
as described in Section 2.3.1, “The ccl Shell Script”.
For details about command-line options see Section 2.5, “Command Line Options”.
By default Clozure CL tries to load the file
"home:ccl-init.lisp" or the compiled
"home:ccl-init.fasl" upon starting up.
Clozure CL does this by executing (load
"home:ccl-init"). If it's unable to load the file
(for example because the file doesn't exist), Clozure CL doesn't
signal an error or warning, it just completes its startup
normally.
On Unix systems, if "ccl-init.lisp" is not
present, Clozure CL will look for ".ccl-init.lisp"
(post 1.2 versions only).
The "home:" prefix to the filename is a
Common Lisp logical host, which Clozure CL initializes to refer to
your home directory. Clozure CL therefore looks for either of the
files
~/ccl-init.lisp or
~/ccl-init.fasl.
Because the init file is loaded the same way as normal Lisp code is, you can put anything you want in it. For example, you can change the working directory, and load packages that you use frequently.
To suppress the loading of this init-file, invoke Clozure CL with the
--no-init option.
When using Clozure CL from the command line, the following
options may be used to modify its behavior. The exact set of
Clozure CL command-line arguments may vary per platform and
slowly changes over time. The current set of command line
options may be retrieved by using the
--help option.
-h (or
--help). Provides a definitive (if
somewhat terse) summary of the command line options
accepted by the Clozure CL implementation and then
exits.
-V (or
--version). Prints the version of
Clozure CL then exits. The version string is the same value
that is returned by
LISP-IMPLEMENTATION-VERSION.
-K
character-encoding-name (or
--terminal-encoding
character-encoding-name).
Specifies the character encoding to use for
*TERMINAL-IO* (see Section 4.5.4, “Character Encodings”). Specifically, the
character-encoding-name string
is uppercased and interned in the KEYWORD package. If an
encoding named by that keyword exists,
CCL:*TERMINAL-CHARACTER-ENCODING-NAME* is set to the name
of that encoding. CCL:*TERMINAL-CHARACTER-ENCODING-NAME* defaults to NIL, which
is a synonym for :ISO-8859-1.
For example:
shell> ccl -K utf-8
has the effect of making the standard CL streams use
:UTF-8 as their character
encoding.
-n (or
--no-init). If this option is given, the
init file is not loaded. This is useful if Clozure CL is being
invoked by a shell script that should not be affected by
whatever customizations a user might have in place.
-e form
(or --eval). An expression is read (via
READ-FROM-STRING) from the string
form and evaluated. If
form contains shell metacharacters,
it may be necessary to escape or quote them to prevent the
shell from interpreting them.
-l path
(or --load
path). Loads file specified by
path.
-T n (or
--set-lisp-heap-gc-threshold
n). Sets the Lisp gc threshold to
n. (see Section 16.3, “GC Page reclamation policy”
-Q (or
--quiet). Suppresses printing of
heralds and prompts when the --batch
command line option is specified.
-R n (or
--heap-reserve). Reserves
n bytes for heap expansion. The
default is 549755813888. (see Section 16.1, “Heap space allocation”)
-S n (or
--stack-size n). Sets the size of the
initial control stack to n. (see Section 7.3.1, “Thread Stack Sizes”)
-Z n (or
--thread-stack-size
n). Sets the size of the first
thread's stack to n. (see Section 7.3.1, “Thread Stack Sizes”)
-b (or --batch). Execute in "batch mode". End-of-file
from *STANDARD-INPUT* causes Clozure CL to exit, as do attempts to
enter a break loop.
--no-sigtrap An obscure option for running under GDB.
-I
image-name (or
--image-name
image-name). Specifies the image
name for the kernel to load. Defaults to the kernel name
with ".image" appended.
The --load and
--eval options can each be provided
multiple times. They're executed in the order specified on
the command line, after the init file (if there is one) is
loaded and before the toplevel read-eval-print loop is
entered.
Finally, any arguments following the pseudo-argument
-- are not processed, and are made
available to Lisp as the value of
ccl:*unprocessed-command-line-arguments*.
SLIME (see the SLIME web page) is an Emacs mode for interacting with Common Lisp systems. Clozure CL is well-supported by SLIME.
See the InstallingSlime topic on the Clozure CL wiki for some tips on how to get SLIME running with Clozure CL.
A number (ok, a small number), of example programs are distributed in the "ccl:examples;" directory of the source distribution. See the README-OPENMCL-EXAMPLES text file in that directory for information about prerequisites and usage.
Some of the example programs are derived from C examples in textbooks, etc.; in those cases, the original author and work are cited in the source code.
Unless the original author or contributor claims other rights, you're free to incorporate any of this example code or derivative thereof in any of your own works without restriction. In doing so, you agree that the code was provided "as is", and that no other party is legally or otherwise responsible for any consequences of your decision to use it.
If you've developed Clozure CL examples that you'd like to see added to the distribution, please send mail to the Clozure CL mailing lists. Any such contributions would be welcome and appreciated (as would bug fixes and improvements to the existing examples.)
Clozure CL, like many other Lisp implementations, consists of a kernel and a heap image. The kernel is an ordinary C program, and is built with a C compiler. It provides very basic and fundamental facilities, such as memory management, garbage collection, and bootstrapping. All the higher-level features are written in Lisp, and compiled into the heap image. Both parts are needed to have a working Lisp implementation; neither the kernel nor the heap image can stand alone.
You may already know that, when you have a C compiler which is written in C, you need a working C compiler to build the compiler. Similarly, the Clozure CL heap image includes a Lisp compiler, which is written in Lisp. You therefore need a working Lisp compiler in order to build the Lisp heap image.
Where will you get a working Lisp compiler? No worries; you can use a precompiled copy of a (slightly older and compatible) version of Clozure CL. This section explains how to do all this.
In principle it should be possible to use another implementation of Common Lisp as the host compiler, rather than an older Clozure CL; this would be a challenging and experimental way to build, and is not described here.
The following terms are used in subsequent sections; it may be helpful to refer to these definitions.
fasl
files are the object files produced
by compile-file. fasl files store the
machine code associated with function definitions and the
external representation of other lisp objects in a compact,
machine-readable form. fasl is short for
“FASt
Loading”. Clozure CL uses different pathname
types (extensions) to name fasl files on different platforms;
see
Table 3.1, “Platform-specific filename conventions”
The Lisp kernel is a C program with a fair amount of platform-specific assembly language code. Its basic job is to map a lisp heap image into memory, transfer control to some compiled lisp code that the image contains, handle any exceptions that occur during the execution of that lisp code, and provide various other forms of runtime support for that code. Clozure CL uses different filenames to name the lisp kernel files on different platforms; see Table 3.1, “Platform-specific filename conventions”.
A heap
image is a file that can be quickly mapped into a
process's address space. Conceptually, it's not too different
from an executable file or shared library in the OS's native
format (ELF or Mach-O/dyld format); for historical reasons,
Clozure CL's own heap images are in their own (fairly simple)
format. The term full heap image refers to a
heap image file that contains all of the code and data that
comprise Clozure CL. Clozure CL uses different filenames to name the
standard full heap image files on different platforms; see
Table 3.1, “Platform-specific filename conventions”.
A bootstrapping image is a minimal heap image used in the process of building Clozure CL itself. The bootstrapping image contains just enough code to load the rest of Clozure CL from fasl files. It may help to think of the bootstrapping image as the egg and the full heap image as the chicken. Clozure CL uses different filenames to name the standard bootstrapping image files on different platforms; see Table 3.1, “Platform-specific filename conventions” .
Each supported platform (and possibly a few
as-yet-unsupported ones) has a uniquely named subdirectory of
ccl/lisp-kernel/; each such
contains a Makefile and may contain some auxiliary files (linker
scripts, etc.) that are used to build the lisp kernel on a
particular platform.The platform-specific name of the kernel
build directory is described in
Table 3.1, “Platform-specific filename conventions”.
Table 3.1. Platform-specific filename conventions
| Platform | kernel | full-image | boot-image | fasl extension | kernel-build directory |
|---|---|---|---|---|---|
| DarwinPPC32 | dppccl | dppccl.image | ppc-boot.image | .dfsl | darwinppc |
| LinuxPPC32 | ppccl | ppccl.image | ppc-boot | .pfsl | linuxppc |
| DarwinPPC64 | dppccl64 | dppccl64.image | ppc-boot64.image | .d64fsl | darwinppc64 |
| LinuxPPC64 | ppccl64 | ppccl64.image | ppc-boot64 | .p64fsl | linuxppc64 |
| LinuxX8664 | lx86cl64 | lx86cl64.image | x86-boot64 | .lx64fsl | linuxx8664 |
| LinuxX8632 | lx86cl | lx86cl.image | x86-boot32 | .lx32fsl | linuxx8632 |
| DarwinX8664 | dx86cl64 | dx86cl64.image | x86-boot64.image | .dx64fsl | darwinx8664 |
| DarwinX8632 | dx86cl | dx86cl.image | x86-boot32.image | .dx32fsl | darwinx8632 |
| FreeBSDX8664 | fx86cl64 | fx86cl64.image | fx86-boot64 | .fx64fsl | freebsdx8664 |
| FreeBSDX8632 | fx86cl | fx86cl.image | fx86-boot32 | .fx32fsl | freebsdx8632 |
| SolarisX64 | sx86cl64 | sx86cl64.image | sx86-boot64 | .sx64fsl | solarisx64 |
| SolarisX86 | sx86cl | sx86cl.image | sx86-boot32 | .sx32fsl | solarisx86 |
| Win64 | wx86cl64.exe | sx86cl64.image | wx86-boot64.image | .wx64fsl | win64 |
| Win32 | wx86cl.exe | wx86cl.image | wx86-boot32.image | .wx32fsl | win32 |
At a given time, there are generally two versions of Clozure CL that you might want to use (and therefore might want to build from source):
The released version
The development version, called the "trunk", which may contain both interesting new features and interesting new bugs
All versions are available for download from svn.clozure.com via the Subversion source control system.
For example, to get a released version (1.7 in this example), use a command like:
svn co http://svn.clozure.com/publicsvn/openmcl/release/1.7/xxx/ccl
To get the trunk version, use:
svn co http://svn.clozure.com/publicsvn/openmcl/trunk/xxx/ccl
Change the xxx to one of the following names:
darwinx86,
linuxx86,
freebsdx86,
solarisx86,
windows,
linuxppc,
or
darwinppc.
Tarball distributions of released versions are also available for download via ftp from: ftp://clozure.com/pub/release/. For additional information about availability of source and distributions see the Clozure CL Trac.
Subversion client programs are pre-installed on Mac OS X 10.5 and
later and are typically either pre-installed or readily available
on Linux and FreeBSD platforms. The Subversion web page contains links to Subversion client programs
for many platforms.
Users of Mac OS X 10.4 or later can also
install Subversion clients via Fink or MacPorts.
On Debian Linux (and on related Linux distros such as Ubuntu) run
apt-get install subversion or equivalent in the command-line or interactive package manager.
The Clozure CL kernel can be built with the following widely available tools:
cc or gcc — the GNU C compiler
ld — the GNU linker
m4 or gm4 — the GNU m4 macro processor
as — the GNU assembler (version 2.10.1 or later)
make — either GNU make or, on FreeBSD, the default BSD make program
In general, the more recent the versions of those
tools, the better; some versions of gcc 3.x on Linux have
difficulty compiling some of the kernel source code correctly
(so gcc 4.0 should be used, if possible.) On Mac OS X, the
versions of the tools distributed with Xcode should work fine;
on Linux, the versions of the tools installed with the OS (or
available through its package management system) should work
fine if they're "recent enough". On FreeBSD, the installed
version of the m4 program doesn't support
some features that the kernel build process depends on; the
GNU version of the m4 macroprocessor (called
gm4 on FreeBSD) should be installed.
In order to build the lisp kernel on Mac OS X 10.6 Snow Leopard, you must install the optional 10.4 support when installing Xcode.
You now have everything you need. Start up
Clozure CL with the -n or --no-init
option to avoid potential interference from code in your init file,
and evaluate the following form to bring your Lisp system
up to date.
? (ccl:rebuild-ccl :full t)
That call to the function rebuild-ccl
performs the following steps:
Deletes all fasl files and other object files in the
ccl directory tree
Runs an external process that does a
make in the current platform's kernel
build directory to create a new kernel.
This step can only work if the C compiler and related
tools are installed; see Section 3.3, “Kernel Build Prerequisites”.
Does (compile-ccl t) in the running
lisp, to produce a set of fasl files from the “higher
level” lisp sources.
Does (xload-level-0 :force) in the
running lisp, to compile the lisp sources in the
“ccl:level-0;” directory into fasl files and
then create a bootstrapping image from those fasl
files.
Runs another external process, which causes the newly compiled lisp kernel to load the new bootstrapping image. The bootstrapping image then loads the “higher level” fasl files and a new copy of the platform's full heap image is then saved.
If all goes well, it'll all happen without user intervention and with some simple progress messages. If anything goes wrong during execution of either of the external processes, the process output is displayed as part of a lisp error message.
rebuild-ccl is essentially just a short
cut for running all the individual steps involved in rebuilding
the system. You can also execute these steps individually, as
described below.
The Lisp kernel is the executable that you run to use Lisp. It doesn't actually contain the entire Lisp implementation; rather, it loads a heap image which contains the specifics—the "library", as it might be called if this was a C program. The kernel also provides runtime support to the heap image, such as garbage collection, memory allocation, exception handling, and the OS interface.
The Lisp kernel file has different names on different
platforms. See
Table 3.1, “Platform-specific filename conventions”. On all
platforms the lisp kernel sources reside
in ccl/lisp-kernel.
This section gives directions on how to rebuild the Lisp kernel from its source code. Most Clozure CL users will rarely have to do this. You probably will only need to do it if you are attempting to port Clozure CL to a new architecture or extend or enhance its kernel in some way. As mentioned above, this step happens automatically when you do
? (rebuild-ccl :full t)
The initial heap image is loaded by the Lisp kernel, and provides most of the language implementation The heap image captures the entire state of a running Lisp (except for external resources, such as open files and TCP sockets). After it is loaded, the contents of the new Lisp process's memory are exactly the same as those of the old Lisp process when the image was created.
The heap image is how we get around the fact that we can't run Lisp code until we have a working Lisp implementation, and we can't make our Lisp implementation work until we can run Lisp code. Since the heap image already contains a fully-working implementation, all we need to do is load it into memory and start using it.
If you're building a new version of Clozure CL, you need to build a new heap image.
(You might also wish to build a heap image if you have a
large program that is very complicated or time-consuming to
load, so that you will be able to load it once, save an image,
and thenceforth never have to load it again. At any time, a heap
image capturing the entire memory state of a running Lisp can be
created by calling the function
ccl:save-application.)
Creating a new Clozure CL full heap image consists of the following steps:
Using your existing Clozure CL, create a bootstrapping image
Using your existing Clozure CL, recompile your updated Clozure CL sources
Invoke Clozure CL with the bootstrapping image you just created (rather than with the existing full heap image).
When you invoke Clozure CL with the bootstrapping image, it starts up, loads all of the Clozure CL fasl files, and saves out a new full heap image. Voila. You've created a new heap image.
A few points worth noting:
There's a circular dependency between the full heap image and the bootstrapping image, in that each is used to build the other.
There are some minor implementation differences, but the environment in effect after the bootstrapping image has loaded its fasl files is essentially equivalent to the environment provided by the full heap image; the latter loads a lot faster and is easier to distribute, of course.
If the full heap image doesn't work (because of an OS compatibilty problem or other bug), it's very likely that the bootstrapping image will suffer the same problems.
Given a bootstrapping image and a set of up-to-date fasl
files, the development cycle usually involves editing lisp
sources (or updating those sources via svn update),
recompiling modified files, and using the bootstrapping image
to produce a new heap image.
The bootstrapping image isn't provided in Clozure CL distributions. It can be built from the source code provided in distributions (using a lisp image and kernel provided in those distributions) using the procedure described below.
The bootstrapping image is built by invoking a special
utility inside a running Clozure CL heap image to load files
contained in the ccl/level-0 directory. The
bootstrapping image loads several dozen fasl files. After
it's done so, it saves a heap image via
save-application. This process is called
"cross-dumping".
Given a source distribution, a lisp kernel, and a heap image, one can produce a bootstrapping image by first invoking Clozure CL from the shell:
shell> ccl Welcome to Clozure CL .... ! ?
then calling ccl:xload-level-0 at the
lisp prompt:
? (ccl:xload-level-0)
This function compiles the lisp sources in the ccl/level-0
directory if they're newer than the corresponding fasl files
and then loads the resulting fasl files into a simulated lisp
heap contained in data structures inside the running
lisp. That simulated heap image is then written to
disk.
xload-level-0 should be called
whenever your existing boot image is out-of-date with respect
to the source files in ccl:level-0;
— For example:
? (ccl:xload-level-0 :force)
forces recompilation of the level-0 sources.
Calling:
? (ccl:compile-ccl)
at the lisp prompt compiles any fasl files that are
out-of-date with respect to the corresponding lisp sources;
(ccl:compile-ccl t) forces
recompilation. ccl:compile-ccl reloads
newly-compiled versions of some files;
ccl:xcompile-ccl is analogous, but skips
this reloading step.
Unless there are bootstrapping considerations involved, it usually doesn't matter whether these files are reloaded after they're recompiled.
Calling compile-ccl or
xcompile-ccl in an environment where fasl
files don't yet exist may produce warnings to that effect
whenever files are required during
compilation; those warnings can be safely ignored. Depending
on the maturity of the Clozure CL release, calling
compile-ccl or
xcompile-ccl may also produce several
warnings about undefined functions, etc. They should be
cleaned up at some point.
To build a full image from a bootstrapping image, just invoke the kernel with the bootstrapping image as an argument
$ cd ccl # wherever your ccl directory is $ ./KERNEL--image-nameBOOT_IMAGE--no-init
Where KERNEL and
BOOT_IMAGE are the names of
the kernel and boot image appropriate to the platform you are
running on. See Table 3.1, “Platform-specific filename conventions”
That should load a few dozen fasl files (printing a message as each file is loaded.) If all of these files successfully load, the lisp will print a prompt. You should be able to do essentially everything in that environment that you can in the environment provided by a "real" heap image. If you're confident that things loaded OK, you can save that image:
? (ccl:save-application "image_name") ; Overwriting the existing heap image
Where image_name is the name of
the full heap image for your platform. See
Table 3.1, “Platform-specific filename conventions”.
If things go wrong in the early stages of the loading sequence, errors are often difficult to debug; until a fair amount of code (CLOS, the CL condition system, streams, the reader, the read-eval-print loop) is loaded, it's generally not possible for the lisp to report an error. Errors that occur during these early stages ("the cold load") sometimes cause the lisp kernel debugger (see ) to be invoked; it's primitive, but can sometimes help one to get oriented.
The Common Lisp standard allows considerable latitude in the details of an implementation, and each particular Common Lisp system has some idiosyncrasies. This chapter describes ordinary user-level features of Clozure CL, including features that may be part of the Common Lisp standard, but which may have quirks or details in the Clozure CL implementation that are not described by the standard. It also describes extensions to the standard; that is, features of Clozure CL that are not part of the Common Lisp standard at all.
Clozure CL's tracing facility is invoked by an extended version of the Common Lisp trace macro. Extensions allow tracing of methods, as well as finer control over tracing actions.
TRACE {keyword
global-value}* {spec |
(spec {keyword
local-value}*)}* [Macro]
The trace macro encapsulates the functions named by
specs, causing trace actions to take place on entry and
exit from each function. The default actions print a message on function entry and
exit. Keyword/value options
can be used to specify changes in the default behavior.
Invoking (trace) without arguments returns a list of functions being traced.
A spec is either a symbol that is the name of a function, or an
expression of the form (setf symbol), or a
specific method of a generic function in the form (:method
gf-name {qualifier}*
({specializer}*)), where a
specializer can be the name of a class or an EQL
specializer.
A spec can also be a string naming a package, or equivalently a
list (:package package-name), in order to
request that all functions in the package to be traced.
By default, whenever a traced function is entered or exited, a short message is
printed on *trace-output* showing the arguments on entry and
values on exit. Options specified as key/value pairs can be used to modify this
behavior. Options preceding the function specs apply to
all the functions being traced. Options specified along with a
spec apply to that spec only and override any
global options. The following options are supported:
If true, and if applied to a spec naming a generic
function, arranges to trace all the methods of the generic function in addition to the
generic function itself.
outside-spec
| ({outside-spec}*)
Inhibits all trace actions unless the current
invocation of the function being traced is inside one of the
outside-spec's, i.e. unless a function named by one of the
outside-spec's is currently on the stack.
outside-spec can name a function, a
method, or a package, as above.
form,
:condition form
Evaluates form whenever the function being traced is
about to be entered, and inhibits all trace actions if form
returns nil. The form may reference the lexical variable ccl::args,
which is a list of the arguments in this call. :condition is just a
synonym for :if, though if both are specified, both must return non-nil.
form
Evaluates form whenever the function being traced is
about to be entered, and inhibits the entry trace actions if
form returns nil. The form may reference the lexical variable
ccl::args, which is a list of the arguments in this call. If both
:if and :before-if are specified, both must return
non-nil in order for the before entry actions to happen.
form
Evaluates form whenever the function being traced has
just exited, and inhibits the exit trace actions if form
returns nil. The form may reference the lexical variable ccl::vals,
which is a list of values returned by this call. If both :if and
:after-if are specified, both must return non-nil in order for the
after exit actions to happen.
form
Evaluates form whenever the function being traced is
about to be entered, and prints the result before printing the standard entry message.
The form may reference the lexical variable ccl::args, which is a list
of the arguments in this call. To see multiple forms, use values:
:print-before (values (one-thing) (another-thing)).
form
Evaluates form whenever the function being traced has
just exited, and prints the result after printing the standard exit message. The form may
reference the lexical variable ccl::vals, which is a list of values
returned by this call. To see multiple forms, use values:
:print-after (values (one-thing) (another-thing)).
form
Equivalent to :print-before form :print-after form.
form
Evaluates form whenever the function being traced is
about to be entered. The form may reference the lexical variable
ccl::args, which is a list of the arguments in this call.
form
Evaluates form whenever the function being has just
exited. The form may reference the lexical variable ccl::vals, which
is a list of values returned by this call.
form
Equivalent to :eval-before form
:eval-after form.
form
Evaluates form whenever the function being traced is
about to be entered, and if the result is non-nil, enters a debugger break loop. The form
may reference the lexical variable ccl::args, which is a list of the
arguments in this call.
form
Evaluates form whenever the function being traced has
just exited, and if the result is non-nil, enters a debugger break loop. The form may
reference the lexical variable ccl::vals, which is a list of values
returned by this call.
form
Equivalent to :break-before form :break-after form.
form,
:backtrace form
Evaluates form whenever the function being traced is
about to be entered. The form may reference the lexical variable
ccl::args, which is a list of the arguments in this call. The value
returned by form is intepreted as follows:
does nothing
prints a detailed backtrace to *trace-output*.
integer)
prints the top integer frames of detailed
backtrace to *trace-output*.
integer
prints top integer frames of a terse
backtrace to *trace-output*.
prints a terse backtrace to *trace-output*.
Note that unlike with the other options, :backtrace is equivalent to :backtrace-before only, not both before and after, since it's usually not helpful to print the same backtrace both before and after the function call.
form
Evaluates form whenever the function being traced has
just exited. The form may reference the lexical variable ccl::vals,
which is a list of values returned by this call. The value returned by
form is intepreted as follows:
does nothing
prints a detailed backtrace to *trace-output*.
integer)
prints the top integer frames of detailed
backtrace to *trace-output*.
integer
prints top integer frames of a terse
backtrace to *trace-output*.
prints a terse backtrace to *trace-output*.
action
specifies the action to be taken just before the traced function is entered. action is one of:
The default, prints a short indented message showing the function name and the invocation arguments
Equivalent to :before :print :break-before t
Equivalent to :before :print :backtrace-before t
function
Any other value is interpreted as a function to call on entry instead of printing the standard entry message. It is called with its first argument being the name of the function being traced, the remaining arguments being all the arguments to the function being traced, and ccl:*trace-level* bound to the current nesting level of trace actions.
action
specifies the action to be taken just after the traced function exits. action is one of:
The default, prints a short indented message showing the function name and the returned values
Equivalent to :after :print :break-after t
Equivalent to :after :print :backtrace-after t
function
Any other value is interpreted as a function to call on exit instead of printing the standard exit message. It is called with its first argument being the name of the function being traced, the remaining arguments being all the values returned by the function being traced, and ccl:*trace-level* bound to the current nesting level of trace actions.
Variable bound to the current nesting level during execution of before and after trace actions. The default printing actions use it to determine the amount of indentation.
CCL:*TRACE-MAX-INDENT* [Variable]
The default before and after print actions will not indent by more than the value of ccl:*trace-max-indent* regardless of the current trace level.
CCL:TRACE-FUNCTION spec &key {keyword value}* [Function]
This is a functional version of the TRACE macro. spec and
keywords are as for TRACE, except that all arguments are evaluated.
CCL:*TRACE-PRINT-LEVEL* [Variable]
The default print actions bind CL:*PRINT-LEVEL* to this value while printing. Note that this rebinding is only in effect during the default entry and exit messages. It does not apply to printing of :print-before/:print-after forms or any explicit printing done by user code.
CCL:*TRACE-PRINT-LENGTH* [Variable]
The default print actions bind CL:*PRINT-LENGTH* to this value while printing. Note that this rebinding is only in effect during the default entry and exit messages. It does not apply to printing of :print-before/:print-after forms or any explicit printing done by user code.
CCL:*TRACE-BAR-FREQUENCY* [Variable]
By default, this is nil. If non-nil it should be a integer, and the default entry and exit messages will print a | instead of space every this many levels of indentation.
The advise macro can be thought of as a more
general version of trace. It allows code that
you specify to run before, after, or around a given function, for
the purpose of changing the behavior of the function. Each piece
of added code is called a piece of advice. Each piece of advice
has a unique name, so that you can have multiple pieces of advice
on the same function, including multiple
:before, :after, and
:around pieces of advice.
The :name and :when
keywords serve to identify the piece of advice. A later call to
advise with the same values of
:name and :when will replace
the existing piece of advice; a call with different values will not.
spec---
A specification of the function on which to put the
advice. This is either a symbol that is the name of a
function or generic function, or an expression of the
form (setf symbol), or a
specific method of a generic function in the form
(:method symbol {qualifiers} (specializer {specializer})).
form--- A form to execute before, after, or around the advised function. The form can refer to the variable arglist that is bound to the arguments with which the advised function was called. You can exit from form with (return).
name--- A name that identifies the piece of advice.
when---
An argument that specifies when the piece of advice is
run. There are three allowable values. The default is
:before, which specifies that form is
executed before the advised function is called. Other
possible values are :after, which
specifies that form is executed after the advised
function is called, and :around,
which specifies that form is executed around the call to
the advised function. Use (:do-it)
within form to indicate invocation of the original
definition.
The function foo, already defined, does
something with a list of numbers. The following code uses a
piece of advice to make foo return zero if any of its
arguments is not a number. Using :around advice, you can do
the following:
(advise foo (if (some #'(lambda (n) (not (numberp n))) arglist) 0 (:do-it)) :when :around :name :zero-if-not-nums)
To do the same thing using a :before piece of advice:
(advise foo (if (some #'(lambda (n) (not (numberp n))) arglist) (return 0)) :when :before :name :zero-if-not-nums)
The unadvise macro removes the piece or pieces of advice
matching spec, when,
and name. When the value of
spec is t and the values of when
and name are nil, unadvise
removes every piece of advice; when spec is
t, the argument when is nil, and
name is non-nil, unadvise removes all
pieces of advice with the given name.
Clozure CL's DIRECTORY function accepts the following implementation-dependent keyword arguments:
boolean
If true, includes regular (non-directory) files in DIRECTORY's output. Defaults to T.
boolean
If true, includes directories in DIRECTORY's output. Defaults to NIL.
boolean
If true, includes files and directories whose names start with a dot character in DIRECTORY's output. (Entries whose name is "." or ".." are never included.) Defaults to T.
boolean
If true, includes the TRUENAMEs of symbolic or hard links in DIRECTORY's output; if false, includes the link filenames without attempting to resolve them. Defaults to T.
Note that legacy HFS alias files are treated as plain files.
All characters and strings in Clozure CL fully support Unicode by
using UTF-32. There is only one CHARACTER type
and one STRING type in Clozure CL. There has been a
lot of discussion about this decision which can be found by
searching the openmcl-devel archives at http://clozure.com/pipermail/openmcl-devel/. Suffice it
to say that we decided that the simplicity and speed advantages of
only supporting UTF-32 outweigh the space disadvantage.
There is one CHARACTER type in Clozure CL.
All CHARACTERs are
BASE-CHARs. CHAR-CODE-LIMIT
is now #x110000, which means that all Unicode
characters can be directly represented. As of Unicode 5.0, only
about 100,000 of 1,114,112 possible CHAR-CODEs
are actually defined. The function CODE-CHAR
knows that certain ranges of code values (notably
#xd800-#xddff) will never be
valid character codes and will return NIL for
arguments in that range, but may return a
non-NIL value (an undefined/non-standard
CHARACTER object) for other unassigned code
values.
Clozure CL supports character names of the form
u+xxxx—where x is a
sequence of one or more hex digits. The value of the hex digits
denotes the code of the character. The +
character is optional, so #\u+0020,
#\U0020, and #\U+20 all
refer to the #\Space character.
Characters with codes in the range
#xa0-#x7ff also have
symbolic names These are the names from the Unicode standard with
spaces replaced by underscores. So
#\Greek_Capital_Letter_Epsilon can be used to
refer to the character whose CHAR-CODE is
#x395. To see the complete list of supported
character names, look just below the definition for
register-character-name in
ccl:level-1;l1-reader.lisp.
OPEN, LOAD, and
COMPILE-FILE all take an
:EXTERNAL-FORMAT keyword argument. The value
of :EXTERNAL-FORMAT can be
:DEFAULT (the default value), a line
termination keyword (see Section 4.5.3, “Line Termination Keywords”), a character encoding
keyword (see Section 4.5.4, “Character Encodings”), an
external-format object created using
CCL::MAKE-EXTERNAL-FORMAT (see make-external-format), or a plist with keys:
:DOMAIN, :CHARACTER-ENCODING
and :LINE-TERMINATION. If
argument is a plist, the result of
(APPLY #'MAKE-EXTERNAL-FORMAT
will be used.argument)
If :DEFAULT is specified, then the value
of CCL:*DEFAULT-EXTERNAL-FORMAT* is used. If
no line-termination is specified, then the value of
CCL:*DEFAULT-LINE-TERMINATION* is used, which
defaults to :UNIX. If no character encoding is
specified, then
CCL:*DEFAULT-FILE-CHARACTER-ENCODING* is used
for file streams and
CCL:*DEFAULT-SOCKET-CHARACTER-ENCODING* is used
for socket streams. The default, default character encoding is
NIL which is a synonym for
:ISO-8859-1.
Note that the set of keywords used to denote CHARACTER-ENCODINGs and the set of keywords used to denote line-termination conventions is disjoint: a keyword denotes at most a character encoding or a line termination convention, but never both.
EXTERNAL-FORMATs are objects (structures) with two read-only fields that can be accessed via the functions: EXTERNAL-FORMAT-LINE-TERMINATION and EXTERNAL-FORMAT-CHARACTER-ENCODING.
The value of this variable is used when :EXTERNAL-FORMAT is unspecified or specified as :DEFAULT. It can meaningfully be given any value that can be used as an external-format (except for the value :DEFAULT.)
The initial value of this variable
in Clozure CL is :UNIX, which is equivalent to
(:LINE-TERMINATION :UNIX), among other
things.
The value of this variable is used when an external-format doesn't specify a line-termination convention (or specifies it as :DEFAULT.) It can meaningfully be given any value that can be used as a line termination keyword (see Section 4.5.3, “Line Termination Keywords”).
The initial value of this variable
in Clozure CL is :UNIX.
domain---This is used to indicate where the external
format is to be used. Its value can be almost
anything. It defaults to NIL.
There are two domains that have a pre-defined meaning in
Clozure CL: :FILE indicates
encoding for a file in the file system and
:SOCKET indicates i/o to/from a
socket. The value of domain
affects the default values for
character-encoding and
line-termination.
character-encoding---A keyword that specifies the character encoding
for the external format. Section 4.5.4, “Character Encodings”. Defaults to
:DEFAULT which means if
domain is
:FILE use the value of the variable
CCL:*DEFAULT-FILE-CHARACTER-ENCODING*
and if domain is
:SOCKET, use the value of the
variable
CCL:*DEFAULT-SOCKET-CHARACTER-ENCODING*.
The initial value of both of these variables is
NIL, which means the
:ISO-8859-1 encoding.
line-termination---A keyword that indicates a line termination
keyword Section 4.5.3, “Line Termination Keywords”.
Defaults to :DEFAULT which means
use the value of the variable
CCL:*DEFAULT-LINE-TERMINATION*.
external-format---An external-format object as described above.
Line termination keywords indicate which characters are used
to indicate the end of a line. On input, the external line
termination characters are replaced by #\Newline
and on output, #\Newlines are converted to the
external line termination characters.
Table 4.1. Line Termination Keywords
| keyword | character(s) |
|---|---|
:UNIX
|
#\Linefeed
|
:MACOS
|
#\Return
|
:CR
|
#\Return
|
:CRLF
|
#\Return #\Linefeed
|
:CP/M
|
#\Return #\Linefeed
|
:MSDOS
|
#\Return #\Linefeed
|
:DOS
|
#\Return #\Linefeed
|
:WINDOWS
|
#\Return #\Linefeed
|
:INFERRED
|
see below |
:UNICODE
|
#\Line_Separator
|
:INFERRED means that a stream's
line-termination convention is determined by looking at the contents
of a file. It is only useful for FILE-STREAMs
that're open for :INPUT or
:IO. The first buffer full of data is examined,
and if a #\Return character occurs before any
#\Linefeed character, then the line termination
type is set to :WINDOWS if that
#\Return character is immediately followed by a
#\Linefeed character and to :MACOS
otherwise. If a #\Return character isn't found in
the buffer or if #\Return is preceded by
#\Linefeed, the file's line terminationt type
is set to :UNIX.
Internally, all characters and strings in Clozure CL are in UTF-32. Externally, files or socket streams may encode characters in a wide variety of ways. The International Organization for Standardization, widely known as ISO, defines many of these character encodings. Clozure CL implements some of these encodings as detailed below. These encodings are part of the specification of external formats Section 4.5.2, “External Formats”. When reading from a stream, characters are converted from the specified external character encoding to UTF-32. When writing to a stream, characters are converted from UTF-32 to the specified character encoding.
Internally, CHARACTER-ENCODINGs are objects (structures) that are named by character encoding keywords (:ISO-8859-1, :UTF-8, etc.). The structures contain attributes of the encoding and functions used to encode/decode external data, but unless you're trying to define or debug an encoding there's little reason to know much about the CHARACTER-ENCODING objects and it's usually preferable to refer to a character encoding by its name.
On output to streams with character encodings that can encode the full range of Unicode—and on input from any stream—"unencodable characters" are represented using the Unicode #\Replacement_Character (= #\U+fffd); the presence of such a character usually indicates that something got lost in translation. Either data wasn't encoded properly or there was a bug in the decoding process.
The endianness of a character encoding is sometimes
explicit, and sometimes not. For example,
:UTF-16BE indicates big-endian, but
:UTF-16 does not specify endianness. A byte
order mark is a special character that may appear at the
beginning of a stream of encoded characters to specify the
endianness of a multi-byte character encoding. (It may also be
used with UTF-8 character encodings, where it is simply used to
indicate that the encoding is UTF-8.)
Clozure CL writes a byte order mark as the first character of a file or socket stream when the endianness of the character encoding is not explicit. Clozure CL also expects a byte order mark on input from streams where the endianness is not explicit. If a byte order mark is missing from input data, that data is assumed to be in big-endian order.
A byte order mark from a UTF-8 encoded input stream is not treated specially and just appears as a normal character from the input stream. It is probably a good idea to skip over this character.
The set of character encodings supported by Clozure CL can be retrieved by calling CCL:DESCRIBE-CHARACTER-ENCODINGS.
The list of supported encodings is reproduced here. Most
encodings have aliases, e.g. the encoding named
:ISO-8859-1 can also be referred to by the
names :LATIN1 and :IBM819,
among others. Where possible, the keywordized name of an
encoding is equivalent to the preferred MIME charset name (and
the aliases are all registered IANA charset names.)
:ISO-8859-1
An 8-bit, fixed-width character encoding in which all character codes map to their Unicode equivalents. Intended to support most characters used in most Western European languages.
Clozure CL uses ISO-8859-1 encoding for
*TERMINAL-IO* and for all streams whose
EXTERNAL-FORMAT isn't explicitly specified. The default for
*TERMINAL-IO* can be set via the
-K command-line argument (see Section 2.5, “Command Line Options”).
ISO-8859-1 just covers the first 256 Unicode code points, where the first 128 code points are equivalent to US-ASCII. That should be pretty much equivalent to what earliers versions of Clozure CL did that only supported 8-bit characters, but it may not be optimal for users working in a particular locale.
Aliases: :ISO_8859-1, :LATIN1, :L1,
:IBM819, :CP819, :CSISOLATIN1
:ISO-8859-2
An 8-bit, fixed-width character encoding in which codes #x00-#x9f map to their Unicode equivalents and other codes map to other Unicode character values. Intended to provide most characters found in most languages used in Central/Eastern Europe.
Aliases: :ISO_8859-2, :LATIN-2, :L2,
:CSISOLATIN2
:ISO-8859-3
An 8-bit, fixed-width character encoding in which codes #x00-#x9f map to their Unicode equivalents and other codes map to other Unicode character values. Intended to provide most characters found in most languages used in Southern Europe.
Aliases: :ISO_8859-3, :LATIN,3 :L3,
:CSISOLATIN3
:ISO-8859-4
An 8-bit, fixed-width character encoding in which codes #x00-#x9f map to their Unicode equivalents and other codes map to other Unicode character values. Intended to provide most characters found in most languages used in Northern Europe.
Aliases: :ISO_8859-4, :LATIN4, :L4, :CSISOLATIN4
:ISO-8859-5
An 8-bit, fixed-width character encoding in which codes #x00-#x9f map to their Unicode equivalents and other codes map to other Unicode character values. Intended to provide most characters found in the Cyrillic alphabet.
Aliases: :ISO_8859-5, :CYRILLIC, :CSISOLATINCYRILLIC,
:ISO-IR-144
:ISO-8859-6
An 8-bit, fixed-width character encoding in which codes #x00-#x9f map to their Unicode equivalents and other codes map to other Unicode character values. Intended to provide most characters found in the Arabic alphabet.
Aliases: :ISO_8859-6, :ARABIC, :CSISOLATINARABIC,
:ISO-IR-127
:ISO-8859-7
An 8-bit, fixed-width character encoding in which codes #x00-#x9f map to their Unicode equivalents and other codes map to other Unicode character values. Intended to provide most characters found in the Greek alphabet.
Aliases: :ISO_8859-7, :GREEK, :GREEK8, :CSISOLATINGREEK,
:ISO-IR-126, :ELOT_928, :ECMA-118
:ISO-8859-8
An 8-bit, fixed-width character encoding in which codes #x00-#x9f map to their Unicode equivalents and other codes map to other Unicode character values. Intended to provide most characters found in the Hebrew alphabet.
Aliases: :ISO_8859-8, :HEBREW, :CSISOLATINHEBREW,
:ISO-IR-138
:ISO-8859-9
An 8-bit, fixed-width character encoding in which codes #x00-#xcf map to their Unicode equivalents and other codes map to other Unicode character values. Intended to provide most characters found in the Turkish alphabet.
Aliases: :ISO_8859-9, :LATIN5, :CSISOLATIN5,
:ISO-IR-148
:ISO-8859-10
An 8-bit, fixed-width character encoding in which codes #x00-#x9f map to their Unicode equivalents and other codes map to other Unicode character values. Intended to provide most characters found in Nordic alphabets.
Aliases: :ISO_8859-10, :LATIN6, :CSISOLATIN6,
:ISO-IR-157
:ISO-8859-11
An 8-bit, fixed-width character encoding in which codes #x00-#x9f map to their Unicode equivalents and other codes map to other Unicode character values. Intended to provide most characters found the Thai alphabet.
:ISO-8859-13
An 8-bit, fixed-width character encoding in which codes #x00-#x9f map to their Unicode equivalents and other codes map to other Unicode character values. Intended to provide most characters found in Baltic alphabets.
:ISO-8859-14
An 8-bit, fixed-width character encoding in which codes #x00-#x9f map to their Unicode equivalents and other codes map to other Unicode character values. Intended to provide most characters found in Celtic languages.
Aliases: :ISO_8859-14, :ISO-IR-199, :LATIN8, :L8,
:ISO-CELTIC
:ISO-8859-15
An 8-bit, fixed-width character encoding in which codes #x00-#x9f map to their Unicode equivalents and other codes map to other Unicode character values. Intended to provide most characters found in Western European languages (including the Euro sign and some other characters missing from ISO-8859-1.
Aliases: :ISO_8859-15, :LATIN9
:ISO-8859-16
An 8-bit, fixed-width character encoding in which codes #x00-#x9f map to their Unicode equivalents and other codes map to other Unicode character values. Intended to provide most characters found in Southeast European languages.
Aliases: :ISO_8859-16, :ISO-IR-199, :LATIN8, :L8,
:ISO-CELTIC
:MACINTOSH
An 8-bit, fixed-width character encoding in which codes #x00-#x7f map to their Unicode equivalents and other codes map to other Unicode character values. Traditionally used on Classic MacOS to encode characters used in western languages.
Aliases: :MACOS-ROMAN, :MACOSROMAN, :MAC-ROMAN,
:MACROMAN
:UCS-2
A 16-bit, fixed-length encoding in which characters with CHAR-CODEs less than #x10000 can be encoded in a single 16-bit word. The endianness of the encoded data is indicated by the endianness of a byte-order-mark character (#u+feff) prepended to the data; in the absence of such a character on input, the data is assumed to be in big-endian order.
:UCS-2BE
A 16-bit, fixed-length encoding in which characters with CHAR-CODEs less than #x10000 can be encoded in a single 16-bit big-endian word. The encoded data is implicitly big-endian; byte-order-mark characters are not interpreted on input or prepended to output.
:UCS-2LE
A 16-bit, fixed-length encoding in which characters with CHAR-CODEs less than #x10000 can be encoded in a single 16-bit little-endian word. The encoded data is implicitly little-endian; byte-order-mark characters are not interpreted on input or prepended to output.
:US-ASCII
An 7-bit, fixed-width character encoding in which all character codes map to their Unicode equivalents.
Aliases: :CSASCII, :CP63,7 :IBM637, :US,
:ISO646-US, :ASCII, :ISO-IR-6
:UTF-16
A 16-bit, variable-length encoding in which characters with CHAR-CODEs less than #x10000 can be encoded in a single 16-bit word and characters with larger codes can be encoded in a pair of 16-bit words. The endianness of the encoded data is indicated by the endianness of a byte-order-mark character (#u+feff) prepended to the data; in the absence of such a character on input, the data is assumed to be in big-endian order. Output is written in native byte-order with a leading byte-order mark.
:UTF-16BE
A 16-bit, variable-length encoding in which characters with CHAR-CODEs less than #x10000 can be encoded in a single 16-bit big-endian word and characters with larger codes can be encoded in a pair of 16-bit big-endian words. The endianness of the encoded data is implicit in the encoding; byte-order-mark characters are not interpreted on input or prepended to output.
:UTF-16LE
A 16-bit, variable-length encoding in which characters with CHAR-CODEs less than #x10000 can be encoded in a single 16-bit little-endian word and characters with larger codes can be encoded in a pair of 16-bit little-endian words. The endianness of the encoded data is implicit in the encoding; byte-order-mark characters are not interpreted on input or prepended to output.
:UTF-32
A 32-bit, fixed-length encoding in which all Unicode characters can be encoded in a single 32-bit word. The endianness of the encoded data is indicated by the endianness of a byte-order-mark character (#u+feff) prepended to the data; in the absence of such a character on input, input data is assumed to be in big-endian order. Output is written in native byte order with a leading byte-order mark.
Alias: :UTF-4
:UTF-32BE
A 32-bit, fixed-length encoding in which all Unicode characters encoded in a single 32-bit word. The encoded data is implicitly big-endian; byte-order-mark characters are not interpreted on input or prepended to output.
Alias: :UCS-4BE
:UTF-8
An 8-bit, variable-length character encoding in which characters with CHAR-CODEs in the range #x00-#x7f can be encoded in a single octet; characters with larger code values can be encoded in 2 to 4 bytes.
:UTF-32LE
A 32-bit, fixed-length encoding in which all Unicode characters can encoded in a single 32-bit word. The encoded data is implicitly little-endian; byte-order-mark characters are not interpreted on input or prepended to output.
Alias: :UCS-4LE
:Windows-31j
An 8-bit, variable-length character encoding in which character code points in the range #x00-#x7f can be encoded in a single octet; characters with larger code values can be encoded in 2 bytes.
Aliases: :CP932, :CSWINDOWS31J
:EUC-JP
An 8-bit, variable-length character encoding in which character code points in the range #x00-#x7f can be encoded in a single octet; characters with larger code values can be encoded in 2 bytes.
Alias: :EUCJP
:GB2312
An 8-bit, variable-length character encoding in which character code points in the range #x00-#x80 can be encoded in a single octet; characters with larger code values can be encoded in 2 bytes.
Alias: :GB2312-80 :GB2312-1980 :EUC-CN :EUCCN
:CP936
An 8-bit, variable-length character encoding in which character code points in the range #x00-#x80 can be encoded in a single octet; characters with larger code values can be encoded in 2 bytes.
Alias: :GBK :MS936 :WINDOWS-936
Clozure CL provides functions to encode and decode strings to and from vectors of type (simple-array (unsigned-byte 8)).
Decodes the octets in vector (or the subsequence of it delimited by start and end) into a string according to external-format.
If string is supplied, output will be written into it. It must be large enough to hold the decoded characters. If string is not supplied, a new string will be allocated to hold the decoded characters.
Returns, as multiple values, the decoded string and the position in vector where the decoding ended.
Sequences of octets in vector that cannot be decoded into characters according to external-format will be decoded as #\Replacement_Character.
encode-string-to-octets
string
&key
start
end
external-format
use-byte-order-mark
vector
vector-offset
Encodes string (or the substring delimited by start and end) into external-format and returns, as multiple values, a vector of octets containing the encoded data and an integer that specifies the offset into the vector where the encoded data ends.
When use-byte-order-mark is true, a byte-order mark will be included in the encoded data.
If vector is supplied, output will be written to it. It must be of type (simple-array (unsigned-byte 8)) and be large enough to hold the encoded data. If it is not supplied, the function will allocate a new vector.
If vector-offset is supplied, data will be written into the output vector starting at that offset.
Characters in string that cannot be encoded into external-format will be replaced with an encoding-dependent replacement character (#\Replacement_Character or #\Sub) before being encoded and written into the output vector.
Leading tilde (~) characters in physical pathname namestrings are expanded in the way that most shells do:
"~user/..." can be used to refer to an absolute pathname rooted
at the home directory of the user named "user".
"~/..." can be used to refer to an absolute pathname rooted at
the home directory of the current user.
Clozure CL sets up logical pathname translations for logical hosts: ccl and home
The CCL logical host should point to the
ccl directory. It is used for a variety of
purposes by Clozure CL including: locating Clozure CL source code,
require and provide, accessing
foreign function information, and the Clozure CL build process. It
is set to the value of the environment variable
CCL_DEFAULT_DIRECTORY, which is set by the
openmcl shell script Section 2.3.1, “The ccl Shell Script”. If
CCL_DEFAULT_DIRECTORY is not set, then it is set
to the directory containing the current heap image.
The syntax of namestrings is implementation-defined in Common Lisp. Portable programs cannot assume much of anything about them. (See section 19.1.1 of the Common Lisp standard for more information.)
When translating a namestring into a pathname object, most
implementations seem to follow the convention that a dot
character in the namestring separates the
pathname-name and
the pathname-type. When there is more
than one dot in involved, or when dots appear at the beginning
or end of the namestrings, what to do is less clear: does
".emacs" describe a pathname whose name is
nil and whose type is emacs
or something else? Similarly, given "a.b.c", the question
is which parts are parsed as the pathname name, and which are
parsed as the pathname type?
When generating a namestring from a pathname object (as happens, for example, when printing a pathname), Clozure CL tries to avoid some potential ambiguity by escaping characters that might otherwise be used to separate pathname components. The character used to quote or escape the separators is a backlash on Unix systems, and a #\> character on Windows. So, for example, "a\\.b.c" has name "a.b" and type "c", whereas "a.b\\.c" has name "a" and type "b.c".
To get a native namestring suitable for passing to an
operating system command, use the function
ccl:native-translated-namestring.
This function returns a namestring that represents a pathname using the native conventions of the operating system. Any quoting or escaping of special characters will be removed.
For example, suppose that p is a pathname made
by (make-pathname :name "a.b" :type "c").
Then, (native-translated-namestring p) evaluates
to "a.b.c". By contrast, (namestring p) evaluates
to "a\\.b.c".
Executes forms in an environemt in which each var is bound to a stack-allocated foreign pointer which refers to a C-style string suitable for passing to foreign code which expects a filename argument.
For example, one might use this macro in the following way:
(with-filename-cstrs ((s (native-translated-namestring pathname))) (#_unlink s))
Various operating systems have different conventions for how they expect native pathname strings to be encoded. Darwin expects then to be decomposed UTF-8. The Unicode variants to Windows file-handling functions expect UTF-16. Other systems just treat them as opaque byte sequences. This macro ensures that the correct encoding is used, whatever the host operating system.
Pathname strings are treated as null-terminated strings coded in the encoding named by the value returned by the function CCL:PATHNAME-ENCODING-NAME. This value may be changed with SETF.
In release 1.2 and later, Clozure CL supports memory-mapped files. On operating systems that support memory-mapped files (including Mac OS X, Linux, and FreeBSD), the operating system can arrange for a range of virtual memory addresses to refer to the contents of an open file. As long as the file remains open, programs can read values from the file by reading addresses in the mapped range.
Using memory-mapped files may in some cases be more efficient than reading the contents of a file into a data structure in memory.
Clozure CL provides the functions CCL:MAP-FILE-TO-IVECTOR and CCL:MAP-FILE-TO-OCTET-VECTOR to support memory-mapping. These functions return vectors whose contents are the contents of memory-mapped files. Reading an element of such a vector returns data from the corresponding position in the file.
Without memory-mapped files, a common idiom for reading the contents of files might be something like this:
(let* ((stream (open pathname :direction :input :element-type '(unsigned-byte 8)))
(vector (make-array (file-size-to-vector-size stream)
:element-type '(unsigned-byte 8))))
(read-sequence vector stream))
Using a memory-mapped files has a result that is the same in that, like the above example, it returns a vector whose contents are the same as the contents of the file. It differs in that the above example creates a new vector in memory and copies the file's contents into it; using a memory-mapped file instead arranges for the vector's elements to point to the file's contents on disk directly, without copying them into memory first.
The vectors returned by CCL:MAP-FILE-TO-IVECTOR and CCL:MAP-FILE-TO-OCTET-VECTOR are read-only; any attempt to change an element of a vector returned by these functions results in a memory-access error. Clozure CL does not currently support writing data to memory-mapped files.
Vectors created by CCL:MAP-FILE-TO-IVECTOR and CCL:MAP-FILE-TO-OCTET-VECTOR are required to respect Clozure CL's limit on the total size of an array. That means that you cannot use these functions to create a vector longer than ARRAY-TOTAL-SIZE-LIMIT, even if the filesystem supports file sizes that are larger. The value of ARRAY-TOTAL-SIZE-LIMIT is (EXPT 2 24) on 32-but platforms; and (EXPT 2 56) on 64-bit platforms.
CCL:MAP-FILE-TO-IVECTOR
pathname
element-type
[Function]
The pathname of the file to be memory-mapped.
The element-type of the vector to be created. Specified as a type-specifier that names a subtype of either SIGNED-BYTE or UNSIGNED-BYTE.
The map-file-to-ivector function tries to
open the file at pathname for reading. If
successful, the function maps the file's contents to a range of
virtual addresses. If successful, it returns a read-only vector
whose element-type is given
by element-type, and whose contents are
the contents of the memory-mapped file.
The returned vector is a displaced-array whose element-type is (UPGRADED-ARRAY-ELEMENT-TYPE element-type). The target of the displaced array is a vector of type (SIMPLE-ARRAY element-type (*)) whose elements are the contents of the memory-mapped file.
Because of alignment issues, the mapped file's contents start a few bytes (4 bytes on 32-bit platforms, 8 bytes on 64-bit platforms) into the vector. The displaced array returned by CCL:MAP-FILE-TO-IVECTOR hides this overhead, but it's usually more efficient to operate on the underlying simple 1-dimensional array. Given a displaced array (like the value returned by CCL:MAP-FILE-TO-IVECTOR), the function ARRAY-DISPLACEMENT returns the underlying array and the displacement index in elements.
Currently, Clozure CL supports only read operations on memory-mapped files. If you try to change the contents of an array returned by map-file-to-ivector, Clozure CL signals a memory error.
CCL:UNMAP-IVECTOR
displaced-array
[Function]
If the argument is a displaced-array returned by map-file-to-ivector, and if it has not yet been unmapped by this function, then unmap-ivector undoes the memory mapping, closes the mapped file, and changes the displaced-array so that its target is an empty vector (of length zero).
CCL:MAP-FILE-TO-OCTET-VECTOR
pathname
[Function]
This function is a synonym for (CCL:MAP-FILE-TO-IVECTOR pathname '(UNSIGNED-BYTE 8)) It is provided as a convenience for the common case of memory-mapping a file as a vector of bytes.
CCL:UNMAP-OCTET-VECTOR
displaced-array
[Function]
This function is a synonym for (CCL:UNMAP-IVECTOR)
Clozure CL supports the definition of static variables, whose values are the same across threads, and which may not be dynamically bound. The value of a static variable is thus the same across all threads; changing the value in one thread changes it for all threads.
Attempting to dynamically rebind a static variable (for instance, by using LET, or using the variable name as a parameter in a LAMBDA form) signals an error. Static variables are shared global resources; a dynamic binding is private to a single thread.
Static variables therefore provide a simple way to share mutable state across threads. They also provide a simple way to introduce race conditions and obscure bugs into your code, since every thread reads and writes the same instance of a given static variable. You must take care, therefore, in how you change the values of static variables, and use normal multithreaded programming techniques, such as locks or semaphores, to protect against race conditions.
In Clozure CL, access to a static variable is usually faster than access to a special variable that has not been declared static.
DEFSTATIC
var
value
&key
doc-string
[Macro]
The name of the new static variable.
The initial value of the new static variable.
A documentation string that is assigned to the new variable.
Proclaims the variable special, assigns the variable the supplied value, and assigns the doc-string to the variable's VARIABLE documentation. Marks the variable static, preventing any attempt to dynamically rebind it. Any attempt to dynamically rebind var signals an error.
Clozure CL provides the
function CCL:SAVE-APPLICATION, which creates a file
containing an archived Lisp memory image.
Clozure CL consists of a small executable called the Lisp kernel, which implements the very lowest level features of the Lisp system, and an image, which contains the in-memory representation of most of the Lisp system, including functions, data structures, variables, and so on. When you start Clozure CL, you are launching the kernel, which then locates and reads an image file, restoring the archived image in memory. Once the image is fully restored, the Lisp system is running.
Using CCL:SAVE-APPLICATION, you can create a
file that contains a modified image, one that includes any changes
you've made to the running Lisp system. If you later pass your
image file to the Clozure CL kernel as a command-line parameter, it
then loads your image file instead of its default one, and Clozure CL
starts up with your modifications.
If this scenario seems to you like a convenient way to
create an application, that's just as intended. You can create an
application by modifying the running Lisp until it does what you
want, then use CCL:SAVE-APPLICATION to preserve your
changes and later load them for use.
In fact, you can go further than that. You can replace Clozure CL's toplevel function with your own, and then, when the image is loaded, the Lisp system immediately performs your tasks rather than the default tasks that make it a Lisp development system. If you save an image in which you have done this, the resulting Lisp system is your tool rather than a Lisp development system.
You can go a step further still. You can
tell CCL:SAVE-APPLICATION to prepend the Lisp kernel
to the image file. Doing this makes the resulting image into a
self-contained executable binary. When you run the resulting file,
the Lisp kernel immediately loads the attached image file and runs
your saved system. The Lisp system that starts up can have any
behavior you choose. It can be a Lisp development system, but with
your customizations; or it can immediately perform some task of
your design, making it a specialized tool rather than a general
development system.
In other words, you can develop any application you like by
interactively modifying Clozure CL until it does what you want, then
using CCL:SAVE-APPLICATION to preserve your changes
in an executable image.
On Mac OS X,
the application builder
uses CCL:SAVE-APPLICATION to create the executable
portion of the application
bundle. Double-clicking the application bundle runs
the executable image created
by CCL:SAVE-APPLICATION.
Also on Mac OS X, Clozure CL supports an object type
called MACPTR, which is the type of pointers into the
foreign (Mac OS) heap. Examples of
commonly-user MACPTR objects are Cocoa windows and
other dynamically-allocated Mac OS system objects.
Because a MACPTR object is a pointer into a
foreign heap that exists for the lifetime of the running Lisp
process, and because a saved image is used by loading it into a
brand new Lisp process, saved MACPTR objects cannot
be relied on to point to the same things when reconstituted from a
saved image. In fact, a restored MACPTR object might
point to anything at all—for example an arbitrary location
in the middle of a block of code, or a completely nonexistent
virtual address.
For that reason, CCL:SAVE-APPLICATION converts
all MACPTR objects to DEAD-MACPTR
objects when writing them to an image
file. A DEAD-MACPTR is functionally identical to
a MACPTR, except that code that operates
on MACPTR objects distinguishes them
from DEAD-MACPTR objects and can handle them
appropriately—signaling errors, for example.
As of Clozure CL 1.2, there is one exception to the conversion
of MACPTR to DEAD-MACPTR objects:
a MACPTR object that points to the address 0 is not
converted, because address 0 can always be relied upon to refer to
the same thing.
As of Clozure CL 1.2, the constant CCL:+NULL-PTR+
refers to a MACPTR object that points to address 0.
On all supported platforms, you can
use CCL:SAVE-APPLICATION to create a command-line
tool that runs the same way any command-line program
does. Alternatively, if you choose not to prepend the kernel, you
can save an image and then later run it by passing it as a
command-line parameter to the ccl
or ccl64 script.
SAVE-APPLICATION
filename
&key
toplevel-function
init-file
error-handler
application-class
clear-clos-caches
(purify t)
impurify
(mode #o644)
prepend-kernel
native
[Function]
The pathname of the file to be created when Clozure CL saves the application.
The function to be executed after startup is complete. The toplevel is a function of no arguments that performs whatever actions the lisp system should perform when launched with this image.
If this parameter is not supplied, Clozure CL uses its default toplevel. The default toplevel runs the read-eval-print loop.
The pathname of a Lisp file to be loaded when the image starts up. You can place initialization expressions in this file, and use it to customize the behavior of the Lisp system when it starts up.
The error-handling mode for the saved image. The
supplied value determines what happens when an error is not
handled by the saved image. Valid values
are :quit (Lisp exits with an error
message); :quit-quietly (Lisp exits without an
error message); or :listener (Lisp enters a
break loop, enabling you to debug the problem by interacting
in a listener). If you don't supply this parameter, the
saved image uses the default error handler
(:listener).
The CLOS class that represents the saved Lisp
application. Normally you don't need to supply this
parameter; CCL:SAVE-APPLICATION uses the
class CCL:LISP-DEVELOPMENT-SYSTEM. In some
cases you may choose to create a custom application class;
in that case, pass the name of the class as the value for
this parameter.
If true, ensures that CLOS caches are emptied before saving the image. Normally you don't need to supply this parameter, but if for some reason you want to ensure the CLOS caches are clear when the image starts up, you can pass any true value.
When true, calls (in effect) purify before
saving the heap image. This moves certain objects that
are unlikely to become garbage to a special memory area
that is not scanned by the GC (since it is expected that
the GC wouldn't find anything to collect).
If true, calls (in effect) impurify before
saving the heap image. (If both :impurify
and :purify are true, first
impurify is done, and then purify.)
impurify moves objects in certain special memory
areas into the regular dynamic heap, where they will be scanned
by the GC.
A number specifying the mode (permission bits) of the output file.
Specifies the file to prepend to the saved heap
image. A value of t means to prepend
the lisp kernel binary that the lisp started with.
Otherwise, the value of :prepend-kernel
should be a pathname designator for the file to be
prepended.
If the prepended file is execuatable, its execute mode bits will be copied to the output file.
This argument can be used to prepend any kind of file to the saved heap image. This can be useful in some special cases.
If true, saves the image as a native (ELF, Mach-O, PE) shared library. (On platforms where this isn't yet supported, a warning is issued and the option is ignored.)
Multiple fasl files can be concatenated into a single file.
out-file--- Name of the file in which to store the concatenation.
fasl-files--- List of names of fasl files to concatenate.
:if-exists---
As for OPEN, defaults to
:error
Creates a fasl file which, when loaded, will have the same effect as loading the individual input fasl files in the specified order. The single file might be easier to distribute or install, and loading it may be at least a little faster than loading the individual files (since it avoids the overhead of opening and closing each file in succession.)
The PATHNAME-TYPE of the output file and of each input file defaults to the current platform's fasl file type (.dx64fsl or whatever.) If any of the input files has a different type/extension an error will be signaled, but it doesn't otherwise try too hard to verify that the input files are real fasl files for the current platform.
In Clozure CL, the Common Lisp types short-float and single-float are implemented as IEEE single precision values; double-float and long-float are IEEE double precision values. On 64-bit platforms, single-floats are immediate values (like fixnums and characters).
Floating-point exceptions are generally enabled and detected. By default, threads start up with overflow, division-by-zero, and invalid enabled, and the rounding mode is set to nearest. The functions SET-FPU-MODE and GET-FPU-MODE provide user control over floating-point behavior.
mode--- One of the keywords :rounding-mode, :overflow, :underflow, :division-by-zero, :invalid, :inexact.
If mode is supplied, returns the value of the corresponding control flag for the current thread.
Otherwise, returns a list of keyword/value pairs which describe the floating-point exception-enable and rounding-mode control flags for the current thread.
rounding-mode--- One of :nearest, :zero, :positive, :negative
overflow, underflow, division-by-zero, invalid, inexact --- If true, the floating-point exception is signaled. If NIL, it is masked.
As of release 1.4, Clozure CL provides a way for lisp objects to be watched so that a condition will be signaled when a thread attempts to write to the watched object. For a certain class of bugs (someone is changing this value, but I don't know who), this can be extremely helpful.
The WATCH function arranges for the specified object to be monitored for writes. This is accomplished by copying the object to its own set of virtual memory pages, which are then write-protected. This protection is enforced by the computer's memory-management hardware; the write-protection does not slow down reads at all.
When any write to the object is attempted, a WRITE-TO-WATCHED-OBJECT condition will be signaled.
When called with no arguments, WATCH returns a freshly-consed list of the objects currently being watched.
WATCH returns NIL if the object cannot be watched (typically because the object is in a static or pure memory area).
WATCH operates at a fairly low level; it is not possible to avoid the details of the internal representation of objects. Nevertheless, as a convenience, WATCHing a standard-instance, a hash-table, or a multi-dimensional or non-simple CL array will watch the underlying slot-vector, hash-table-vector, or data-vector, respectively.
WATCH can monitor any memory-allocated lisp object.
In Clozure CL, a memory-allocated object is either a cons cell or a uvector.
WATCH operates on cons cells, not lists. In order to watch a chain of cons cells, each cons cell must be watched individually. Because each watched cons cell takes up its own own virtual memory page (4 Kbytes), it's only feasible to watch relatively short lists.
If a memory-allocated object isn't a cons cell, then it is a vector-like object called a uvector. A uvector is a memory-allocated lisp object whose first word is a header that describes the object's type and the number of elements that it contains.
So, a hash table is a uvector, as is a string, a standard instance, a double-float, a CL array or vector, and so forth.
Some CL objects, like strings and other simple vectors, map in a straightforward way onto the uvector representation. It is easy to understand what happens in such cases. The uvector index corresponds directly to the vector index:
? (defvar *s* "xxxxx")
*S*
? (watch *s*)
"xxxxx"
? (setf (char *s* 3) #\o)
> Error: Write to watched uvector "xxxxx" at index 3
> Faulting instruction: (movl (% eax) (@ -5 (% r15) (% rcx)))
> While executing: SET-CHAR, in process listener(1).
> Type :POP to abort, :R for a list of available restarts.
> Type :? for other options.
In the case of more complicated objects (e.g., a hash-table, a standard-instance, a package, etc.), the elements of the uvector are like slots in a structure. It's necessary to know which one of those "slots" contains the data that will be changed when the object is written to.
As mentioned above, watch knows about arrays, hash-tables, and standard-instances, and will automatically watch the appropriate data-containing element.
An example might make this clearer.
? (defclass foo ()
(slot-a slot-b slot-c))
#<STANDARD-CLASS FOO>
? (defvar *a-foo* (make-instance 'foo))
*A-FOO*
? (watch *a-foo*)
#<SLOT-VECTOR #xDB00D>
;;; Note that WATCH has watched the internal slot-vector object
? (setf (slot-value *a-foo* 'slot-a) 'foo)
> Error: Write to watched uvector #<SLOT-VECTOR #xDB00D> at index 1
> Faulting instruction: (movq (% rsi) (@ -5 (% r8) (% rdi)))
> While executing: %MAYBE-STD-SETF-SLOT-VALUE-USING-CLASS, in process listener(1).
> Type :POP to abort, :R for a list of available restarts.
> Type :? for other options.
Looking at a backtrace would presumably show what object and slot name were written.
Note that even though the write was to slot-a, the uvector index was 1 (not 0). This is because the first element of a slot-vector is a pointer to the instance that owns the slots. We can retrieve that to look at the object that was modified:
1 > (uvref (write-to-watched-object-object *break-condition*) 0)
#<FOO #x30004113502D>
1 > (describe *)
#<FOO #x30004113502D>
Class: #<STANDARD-CLASS FOO>
Wrapper: #<CLASS-WRAPPER FOO #x300041135EBD>
Instance slots
SLOT-A: #<Unbound>
SLOT-B: #<Unbound>
SLOT-C: #<Unbound>
1 >
This condition is signaled when a watched object is written to. There are three slots of interest:
object--- The actual object that was the destination of the write.
offset--- The byte offset from the tagged object pointer to the address of the write.
instruction--- The disassembled machine instruction that attempted the write.
A few restarts are provided: one will skip over the faulting write instruction and proceed; another offers to unwatch the object and continue.
There is also an emulate restart. In some common cases, the faulting write instruction can be emulated, enabling the write to be performed without having to unwatch the object (and therefore let other threads potentially write to it). If the faulting instruction isn't recognized, the emulate restart will not be offered.
Although some care has been taken to minimize potential problems arising from watching and unwatching objects from multiple threads, there may well be subtle race conditions present that could cause bad behavior.
For example, suppose that a thread attempts to write to a watched object. This causes the operating system to generate an exception. The lisp kernel figures out what the exception is, and calls back into lisp to signal the write-to-watched-object condition and perhaps handle the error.
Now, as soon lisp code starts running again (for the callback), it's possible that some other thread could unwatch the very watched object that caused the exception, perhaps before we even have a chance to signal the condition, much less respond to it.
Having the object unwatched out from underneath a handler may at least confuse it, if not cause deeper trouble. Use caution with unwatch.
Here are a couple more examples in addition to the above examples of watching a string and a standard-instance.
? (defvar *f* (make-array '(2 3) :element-type 'double-float)) *F* ? (watch *f*) #(0.0D0 0.0D0 0.0D0 0.0D0 0.0D0 0.0D0) ;;; Note that the above vector is the underlying data-vector for the array ? (setf (aref *f* 1 2) pi) > Error: Write to watched uvector #<VECTOR 6 type DOUBLE-FLOAT, simple> at index 5 > Faulting instruction: (movq (% rax) (@ -5 (% r8) (% rdi))) > While executing: ASET, in process listener(1). > Type :POP to abort, :R for a list of available restarts. > Type :? for other options. 1 >
In this case, uvector index in the report is the row-major index of the element that was written to.
Hash tables are surprisingly complicated. The representation of a hash table includes an element called a hash-table-vector. The keys and values of the elements are stored pairwise in this vector.
One problem with trying to monitor hash tables for writes is that the underlying hash-table-vector is replaced with an entirely new one when the hash table is rehashed. A previously-watched hash-table-vector will not be the used by the hash table after rehashing, and writes to the new vector will not be caught.
? (defvar *h* (make-hash-table)) *H* ? (setf (gethash 'noise *h*) 'feep) FEEP ? (watch *h*) #<HASH-TABLE-VECTOR #xDD00D> ;;; underlying hash-table-vector ? (setf (gethash 'noise *h*) 'ding) > Error: Write to watched uvector #<HASH-TABLE-VECTOR #xDD00D> at index 35 > Faulting instruction: (lock) > (cmpxchgq (% rsi) (@ (% r8) (% rdx))) > While executing: %STORE-NODE-CONDITIONAL, in process listener(1). > Type :POP to abort, :R for a list of available restarts. > Type :? for other options. ;;; see what value is being replaced... 1 > (uvref (write-to-watched-object-object *break-condition*) 35) FEEP ;;; backtrace shows useful context 1 > :b *(1A109F8) : 0 (%STORE-NODE-CONDITIONAL ???) NIL (1A10A50) : 1 (LOCK-FREE-PUTHASH NOISE #<HASH-TABLE :TEST EQL size 1/60 #x30004117D47D> DING) 653 (1A10AC8) : 2 (CALL-CHECK-REGS PUTHASH NOISE #<HASH-TABLE :TEST EQL size 1/60 #x30004117D47D> DING) 229 (1A10B00) : 3 (TOPLEVEL-EVAL (SETF (GETHASH # *H*) 'DING) NIL) 709 ...
As previously mentioned, WATCH only watches individual cons cells.
? (defun watch-list (list)
(maplist #'watch list))
WATCH-LIST
? (defvar *l* (list 1 2 3))
*L*
? (watch-list *l*)
((1 2 3) (2 3) (3))
? (setf (nth 2 *l*) 'foo)
> Error: Write to the CAR of watched cons cell (3)
> Faulting instruction: (movq (% rsi) (@ 5 (% rdi)))
> While executing: %SETNTH, in process listener(1).
> Type :POP to abort, :R for a list of available restarts.
> Type :? for other options.
In Clozure CL 1.4 and later, code coverage provides information about which paths through generated code have been executed and which haven't. For each source form, it can report one of three possible outcomes:
Not covered: this form was never entered.
Partly covered: This form was entered, and some parts were executed and some weren't.
Fully covered: Every bit of code generated from this form was executed.
While the information gathered for coverage of generated code is complete and precise, the mapping back to source forms is of necessity heuristic, and depends a great deal on the behavior of macros and the path of the source forms through compiler transforms. Source information is not recorded for variables, which further limits the source mapping. In practice, there is often enough information scattered about a partially covered function to figure out which logical path through the code was taken and which wasn't. If that doesn't work, you can try disassembling to see which parts of the compiled code were not executed: in the disassembled code there will be references to #<CODE-NOTE [xxx] ...> where xxx is NIL if the code that follows was never executed and non-NIL if it was.
Sometimes the situation can be improved by modifying macros to try to preserve more of the input forms, rather than destructuring and rebuilding them.
Because the code coverage information is associated with compiled functions, code coverage information is not available for load-time toplevel expressions. You can work around this by creating a function and calling it. I.e. instead of
(progn (do-this) (setq that ...) ...))
do:
(defun init-this-and-that () (do-this) (setq that ...) ...) (init-this-and-that)
Then you can see the coverage information in the definition of
init-this-and-that.
In order to gather code coverage information, you first have to
recompile all your code to include code coverage
instrumentation. Compiling files will generate code coverage
instrumentation if CCL:*COMPILE-CODE-COVERAGE*
is true:
(setq ccl:*compile-code-coverage* t) (recompile-all-your-files)
The compilation process will be many times slower than normal, and the fasl files will be many times bigger.
When you execute functions loaded from instrumented fasl files, they
will record coverage information every time they are executed.
You can examine that information by calling ccl:report-coverage
or ccl:coverage-statistics.
While recording coverage, you can collect incremental coverage deltas between any two points in time. You might do this while running a test suite, to record the coverage for each test, for example:
(ccl:reset-incremental-coverage)
(loop with coverage = (make-hash-table)
for test in (tests-to-run)
do (run-test test)
do (setf (gethash test coverage) (ccl:get-incremental-coverage))
finally (return coverage))
creates a hash table mapping a test to a representation of all coverage recorded while running the
test. This hash table can then be passed to ccl:report-coverage, ccl:incremental-coverage-svn-matches
or ccl:incremental-coverage-source-matches.
The following functions can be used to manage the coverage data:
report-coverage output-file &key
(tags nil) (external-format :default) (statistics t) (html t)
output-file--- Pathname for the output index file.
html--- If non-nil (the default), this will generate an HTML report, consisting of an index file in output-file and, in the same directory, one html file for each instrumented source file that has been loaded in the current session.
tags--- If non-nil, this should be a hash table mapping arbitrary keys (tags) to incremental coverage deltas. The HTML report will show a list of tags, and allow selection of an arbitrary subset of them to show the coloring and statistics for coverage by that subset.
external-format--- Controls the external format of the html files.
statistics--- If non-nil (the default), a comma-separated file is generated with the summary of statistics. You can specify a filename for the statistics argument, otherwise "statistics.csv" is created in the directory of output-file. See documentation of coverage-statistics below for a description of the values in the statistics file.
Restores the coverage data previously saved with
ccl:save-coverage-in-file, for the set of instrumented fasls
that were loaded both at save and restore time. I.e. coverage
info is only restored for files that have been loaded in this
session. For example if in a previous session you had loaded
"foo.lx86fsl" and then saved the coverage info, in this session
you must load the same "foo.lx86fsl" before calling
restore-coverage-from-file in order to retrieve the stored
coverage info for "foo". Equivalent to (ccl:restore-coverage
(ccl:read-coverage-from-file pathname)).
Returns a sequence of ccl:coverage-statistics objects, one for each
source file, containing the same information as that written to
the statistics file by ccl:report-coverage. The following
accessors are defined for ccl:coverage-statistics objects:
the name of the source file corresponding to this information
the total number of expressions
the number of source expressions that have been entered (i.e. at least partially covered)
the number of source expressions that were fully covered
the number of conditionals with one branch taken and one not taken
the total number of code forms. A code form is an expression in the final stage of compilation, after all macroexpansion and compiler transforms and simplification
the number of code forms that have been entered
the total number of functions
the number of functions that were fully covered
the number of functions that were partly covered
the number of functions never entered
Returns the delta of coverage since the last reset of incremental coverage.
If reset is true (the default), it also resets incremental coverage
now, so that the next call to get-incremental-coverage will return
the delta from this point.
Incremental coverage deltas are represented differently than the full coverage snapshots
returned by functions such as ccl:get-coverage. Incremental
coverage uses an abbreviated format
and is missing some of the information in a full snapshot, and therefore cannot be passed to
functions documented to accept a snapshot, only to functions
specifically documented to accept incremental coverage deltas.
collection--- A hash table mapping arbitrary keys to incremental coverage deltas, or a sequence of incremental coverage deltas.
sources--- A list of pathnames and/or source-notes, the latter representing a range within a file.
Given a hash table collection whose values are incremental coverage
deltas, return a list of all keys corresponding to those deltas that intersect any region
in sources.
For example if the deltas represent tests, then the returned value is a list of all tests that cover some part of the source regions.
collection can also be a sequence of deltas, in which case a subsequence
of matching deltas is returned. In particular you can test whether any particular delta
intersects the sources by passing it in as a single-element list.
incremental-coverage-svn-matches collection &key (directory (current-directory)) (revision :base)
collection--- A hash table mapping arbitrary keys to incremental coverage deltas, or a sequence of incremental coverage deltas.
directory--- The pathname of a subversion working directory.
revision---
The revision to compare to the working directory, an integer or another
value whose printed representation is suitable for passing as the
--revision argument
to svn.
Given a hash table collection whose values are incremental coverage
deltas, return a list of all keys corresponding to those deltas that intersect any changed
source in directory since revision revision in subversion.
For example if the deltas represent tests, then the returned value is a list of all tests that might be affected by the changes.
collection can also be a sequence of deltas, in which case a subsequence
of matching deltas is returned. In particular you can test whether any particular delta
is affected by the changes by passing it in as a single-element list.
The output of ccl:report-coverage consists of formatted source code, with coverage indicated by coloring. Four colors are used: dark green for forms that compiled to code in which every single instruction was executed, light green for forms that have been entered but weren't totally covered, red for forms that were never entered, and the page background color for toplevel forms that weren't instrumented.
The source coloring is applied from outside in. So for example if you have
(outer-form ... (inner-form ...) ...)
first the whole outer form is painted with whatever color expresses the outer form coverage, and then the inner form color is replaced with whatever color expresses the inner form coverage. One consequence of this approach is that every part of the outer form that is not specifically inside some executable inner form will have the outer form's coverage color. If the syntax of outer form involves some non-executable forms, or forms that do not have coverage info of their own for whatever reason, then they will just inherit the color of the outer form, because they don't get repainted with a color of their own.
One case in which this approach can be confusing is in the case of symbols. As noted in the Limitations section, coverage information is not recorded for variables; hence the coloring of a variable does not convey information about whether the variable was evaluated or not -- that information is not available, and the variable just inherits the color of the form that contains it.
Cleanly exit from lisp. If the exit argument is a value of type (signed-byte 32), that value will be passed to the C library function _exit() as the status code. (A value of nil is treated as a zero.)
Alternatively, exit may be a function of no arguments; this function will be called instead of _exit() to exit the lisp.
The error-handler argument, if supplied, must be a function of one argument, the condition, that will be called if an error occurs when preparing to quit. The error-handler function should exit the lisp.
Wait for the signal with signal number s to be received, or until duration seconds have elapsed. If duration is nil, wait for an indeterminate "very long time" (many years).
If signal number s is outside the range of valid signals, or is reserved by the lisp for its own use, an error is signaled. (An error is always signaled on Windows systems.)
In Clozure CL, the cleanup forms are always executed as if they were wrapped with without-interrupts. To allow interrupts, use with-interrupts-enabled.
Clozure CL ships with the complete source code for an integrated development environment written using Cocoa on Mac OS X. This chapter describes how to build and use that environment, referred to hereafter simply as "the IDE".
The IDE provides a programmable text editor, listener windows, an inspector for Lisp data structures, and a means of easily building a Cocoa application in Lisp. In addition, its source code provides an example of a fairly complex Cocoa application written in Lisp.
The current version of the IDE has seen the addition of numerous features and many bugfixes. Although it's by no means a finished product, we hope it will prove more useful than previous versions, and we plan additional work on the IDE for future releases.
Building the Clozure CL IDE is now a very simple process.
In a shell session, cd to the ccl directory.
Run ccl from the shell. The easiest way to do this is generally to execute the ccl or ccl64 command.
Evaluate the form (require :cocoa-application)
For example, assuming that the Clozure CL distribution is installed in "/usr/local/ccl", the following sequence of shell interactions builds the IDE:
oshirion:ccl mikel$ ccl64
Welcome to Clozure Common Lisp Version 1.2-r9198M-trunk (DarwinX8664)!
? (require :cocoa-application)
;Loading #P"ccl:cocoa-ide;fasls;cocoa-utils.dx64fsl.newest"...
;Loading #P"ccl:cocoa-ide;fasls;cocoa-defaults.dx64fsl.newest"...
[...many lines of "Compiling" and "Loading" omitted...]
Saving application to /usr/local/ccl/Clozure CL.app/
oshirion:ccl mikel$
Clozure CL compiles and loads the various subsystems that make up the IDE, then constructs a Cocoa application bundle named "Clozure CL.app" and saves the Lisp image into it. Normally Clozure CL creates the application bundle in the root directory of the Clozure CL distribution.
After it has been built, you can run the "Clozure CL.app" application normally, by double-clicking its icon. When launched, the IDE initially displays a single listener window that you can use to interact with Lisp. You can type Lisp expressions for evaluation at the prompt in the listener window. You can also use Hemlock editing commands to edit the text of expressions in the listener window.
You can open an editor window either by choosing Open from
the File menu and then selecting a text file, or by choosing
New from the File menu. You can also evaluate the
expression (ed) in the listener window; in that
case Clozure CL creates a new window as if you had chosen New from
the File menu.
Editor windows implement Hemlock editing commands. You can use all the editing and customization features of Hemlock within any editor window (including listener windows).
The Lisp menu provides several commands for interacting with the running Lisp session, in addition to the ways you can interact with it by evaluating expressions. You can evaluate a selected range of text in any editing buffer. You can compile and load the contents of editor windows (please note that in the current version, Clozure CL compiles and loads the contents of the file associated with an editor window; that means that if you try to load or compile a window that has not been saved to a file, the result is an error).
You can interrupt computations, trigger breaks, and select restarts from the Lisp menu. You can also display a backtrace or open the Inspector window.
At the bottom of the Lisp menu is an item entitled "Check for Updates". If your copy of Clozure CL came from the Clozure Subversion server (which is the preferred source), and if your internet connection is working, then you can select this menu item to check for updates to your copy of Clozure CL.
When you select "Check for Updates", Clozure CL uses the svn program to query the Clozure Subversion repository and determine whether new updates to Clozure CL are available. (This means that on Mac OS X versions earlier than 10.5, you must ensure that the Subversion client software is installed before using the "Check for Updates" feature. See the wikiHow page on installing Subversion for more information.) If updates are available, Clozure CL automatically downloads and installs them. After a successful download, Clozure CL rebuilds itself, and then rebuilds the IDE on the newly-rebuilt Lisp. Once this process is finished, you should quit the running IDE and start the newly built one (which will be in the same place that the old one was).
Normally, Clozure CL can install updates and rebuild itself without any problems. Occasionally, an unforeseen problem (such as a network outage, or a hardware failure) might interrupt the self-rebuilding process, and leave your copy of Clozure CL unusable. If you are expecting to update your copy of Clozure CL frequently, it might be prudent to keep a backup copy of your working environment ready in case of such situtations. You can also always obtain a full, fresh copy of Clozure CL from Clozure's repository..
The tools menu provides access to the Apropos and Processes windows. The Apropos window searches the running Lisp image for symbols that match any text you enter. You can use the Apropos window to quickly find function names and other useful symbols. The Processes window lists all threads running in the current Lisp session. If you double-click a process entry, Clozure CL opens an Inspector window on that process.
The Inspector window displays information about a Lisp value. The information displayed varies from the very simple, in the case of a simple data value such as a character, to the complex, in the case of structured data such as lists or CLOS objects. The left-hand column of the window's display shows the names of the object's attributes; the righthand column shows the values associated with those attributes. You can inspect the values in the righthand column by double-clicking them.
Inspecting a value in the righthand column changes the Inspector window to display the double-clicked object. You can quickly navigate the fields of structured data this way, inspecting objects and the objects that they refer to. Navigation buttons at the top left of the window enable you to retrace your steps, backing up to return to previously-viewed objects, and going forward again to objects you navigated into previously.
You can change the contents of a structured object by evaluating expressions in a listener window. The refresh button (marked with a curved arrow) updates the display of the Inspector window, enabling you to quickly see the results of changing a data structure.
Clozure CL builds the IDE from sources in the "objc-bridge" and "cocoa-ide" directories in the Clozure CL distribution. The IDE as a whole is a relatively complicated application, and is probably not the best place to look when you are first trying to understand how to build Cocoa applications. For that, you might benefit more from the examples in the "examples/cocoa/" directory. Once you are familiar with those examples, though, and have some experience building your own application features using Cocoa and the Objective-C bridge, you might browse through the IDE sources to see how it implements its features.
The search path for Clozure CL's REQUIRE feature
includes the "objc-bridge" and "cocoa-ide" directories. You can
load features defined in these directories by
using REQUIRE. For example, if you want to use the
Cocoa features of Clozure CL from a terminal session (or from an Emacs
session using SLIME or ILISP), you can evaluate (require
:cocoa).
One important feature of the IDE currently has no Cocoa user interface: the application builder. The application builder constructs a Cocoa application bundle that runs a Lisp image when double-clicked. You can use the application builder to create Cocoa applications in Lisp. These applications are exactly like Cocoa applications created with XCode and Objective-C, except that they are written in Lisp.
To make the application builder available, evaluate the
expression (require :build-application). Clozure CL loads
the required subsystems, if necessary.
BUILD-APPLICATION &key
(name "MyApplication")
(type-string "APPL")
(creator-string "OMCL")
(directory (current-directory))
(copy-ide-resources t)
(info-plist NIL)
(nibfiles NIL)
(main-nib-name NIL)
(application-class 'GUI::COCOA-APPLICATION)
(toplevel-function NIL)
[Function]
The build-application function constructs an application bundle, populates it with the files needed to satisfy Mac OS X that the bundle is a launchable application, and saves an executable Lisp image to the proper subdirectory of the bundle. Assuming that the saved Lisp image contains correct code, a user can subsequently launch the resulting Cocoa application by double-clicking its icon in the Finder, and the saved Lisp environment runs.
The keyword arguments control various aspects of application
bundle as BUILD-APPLICATION builds it.
Specifies the application name of the
bundle. BUILD-APPLICATION creates an application
bundle whose name is given by this parameter, with the
extension ".app" appended. For example, using the default
value for this parameter results in a bundle named
"MyApplication.app".
Specifies type of bundle to create. You should normally never need to change the default value, which Mac OS X uses to identify application bundles.
Specifies the creator code, which uniquely identifies the application under Mac OS X. The default creator code is that of Clozure CL. For more information about reserving and assigning creator codes, see Apple's developer page on the topic.
The directory in which BUILD-APPLICATION
creates the application bundle. By default, it creates the
bundle in the current working directory. Unless you
use CURRENT-DIRECTORY to set the working
directory, the bundle may be created in some unexpected place,
so it's safest to specify a full pathname for this argument. A
typical value might be "/Users/foo/Desktop/"
(assuming, of course, that your username is "foo").
Whether to copy the resource files from the IDE's
application bundle. By
default, BUILD-APPLICATION copies nibfiles
and other resources from the IDE to the newly-created
application bundle. This option is often useful when you
are developing a new application, because it enables your
built application to have a fully-functional user
interface even before you have finished designing one. By
default, the application uses the application menu and
other UI elements of the IDE until you specify
otherwise. Once your application's UI is fully
implemented, you may choose to pass NIL
for the value of this parameter, in which case the IDE
resources are not copied into your application
bundle.
A user-supplied NSDictionary object that defines the
contents of the Info.plist file to be written to the
application bundle. The default value
is NIL, which specifies that the
Info.plist from the IDE is to be used
if copy-ide-resources is true,
and a new dictionary created with default values is to be
used otherwise. You can create a suitable NSDictionary
object using the
function make-info-dict. For details on
the parameters to this function, see its definition in
"ccl/cocoa-ide/builder-utilities.lisp".
A list of pathnames, where each pathname identifies
a nibfile created
with
Apple's InterfaceBuilder
application. BUILD-APPLICATION copies each
nibfile into the appropriate place in the application bundle,
enabling the application to load user-interface elements from
them as-needed. It is safest to provide full pathnames to the
nibfiles in the list. Each nibfile must be in ".nib" format,
not ".xib" format, in order that the application can load
it.
The name of the nibfile to load initially when launching. The user-interface defined in this nibfile becomes the application's main interface. You must supply the name of a suitable nibfile for this parameter, or the resulting application uses the Clozure CL user interface.
The name of the application's CLOS class. The default value is the class provided by Clozure CL for graphical applications. Supply the name of your application class if you implement one. If not, Clozure CL uses the default class.
The toplevel function that runs when the application
launches. Normally the default value, which is Clozure CL's
toplevel, works well, but in some cases you may wish to
customize the behavior of the application's toplevel. The best
source of information about writing your own toplevel is the
Clozure CL source code, especially the implementations
of TOPLEVEL-FUNCTION in
"ccl/level-1/l1-application.lisp"
BUILD-APPLICATION creates a folder named
"name.app" in the
directory directory. Inside that
folder, it creates the "Contents" folder that Mac OS X
application bundles are expected to contain, and populates it
with the "MacOS" and "Resources" folders, and the "Info.plist"
and "PkgInfo" files that must be present in a working
application bundle. It takes the contents of the "Info.plist"
and "PkgInfo" files from the parameters
to BUILD-APPLICATION. If copy-ide-resources
is true then it copies the contents of the "Resources" folder
from the "Resources" folder of the running IDE.
The work needed to produce a running Cocoa application is
very minimal. In fact, if you
supply BUILD-APPLICATION with a valid nibfile and
pathnames, it builds a running Cocoa application that displays
your UI. It doesn't need you to write any code at all to do
this. Of course, the resulting application doesn't do anything
apart from displaying the UI defined in the nibfile. If you want
your UI to accomplish anything, you need to write the code to
handle its events. But the path to a running application with your
UI in it is very short indeed.
Please note that BUILD-APPLICATION is a work in
progress. It can easily build a working Cocoa application, but it
still has limitations that may in some cases prove
inconvenient. For example, in the current version it provides no
easy way to specify an application delegate different from the
default. If you find the current limitations
of BUILD-APPLICATION too restrictive, and want to try
extending it for your use, you can find the source code for it in
"ccl/cocoa-ide/build-application.lisp". You can see the default
values used to populate the "Info.plist" file in
"ccl/cocoa-ide/builder-utilities.lisp".
For more information on how to
use BUILD-APPLICATION, see the Currency Converter
example in "ccl/examples/cocoa/currency-converter/".
It's possible to automate use of the application builder
by running a call to CCL:BUILD-APPLICATION
from the terminal command line. For example, the following
command, entered at a shell prompt in Mac OS X's Terminal
window, builds a working copy of the Clozure CL environment called
"Foo.app":
ccl -b -e "(require :cocoa)" -e "(require :build-application)" -e "(ccl::build-application :name \"Foo\")"
You can use the same method to automate building your
Lisp/Cocoa applications. Clozure CL handles each Lisp expressions
passed with a -e argument in order, so you
can simply evaluate a sequence of Lisp expressions as in the
above example to build your application, ending with a call
to CCL:BUILD-APPLICATION. The call
to CCL:BUILD-APPLICATION can process all the
same arguments as if you evaluated it in a Listener window in
the Clozure CL IDE.
Building a substantial Cocoa application (rather than just reproducing the Lisp environment using defaults, as is done in the above example) is likely to involve a relatively complicated sequence of loading source files and perhaps evaluating Lisp forms. You might be best served to place your command line in a shell script that you can more easily edit and test.
One potentially complicated issue concerns loading all
your Lisp source files in the right order. You might consider
using ASDF to define and load a system that includes all the
parts of your application before
calling CCL:BUILD-APPLICATION. ASDF is a
"another system-definition facility", a sort
of make for Lisp, and is included in the
Clozure CL distribution. You can read more about ASDF at the ASDF
home
page.
Alternatively, you could use the standard features of Common Lisp to load your application's files in the proper order.
Hemlock is the text editor used in Clozure CL. It was originally based on the CMU Hemlock editor, but has since diverged from it in various ways. We continue to call the editor part of our IDE Hemlock to give credit where credit is due, but we make no attempt at source or API compatibility with the original Hemlock.
Like the code, this documentation is based on the original Hemlock documentation, modified as necessary.
Hemlock follows in the tradition of Emacs-compatible editors, with a rich set of extensible commands. This document describes the API for implementing new commands. The basic editor consists of a set of Lisp utility functions for manipulating buffers and the other data structures of the editor. All user level commands are written in terms of these functions. To find out how to define commands see Commands.
In Hemlock, text is represented as a sequence of lines. Newline characters
are never stored but are implicit between lines. The
implicit newline character is treated as the single character #\Newline by the
text primitives.
Text is broken into lines when it is first introduced into Hemlock. Text enters Hemlock from the outside world in two ways: reading a file, or pasting text from the system clipboard. Hemlock uses heuristics (which should be documented here!) to decide what newline convention to use to convert the incoming text into its internal representation as a sequence of lines. Similarly it uses heuristics (which should be documented here!) to convert the internal representation into a string with embedded newlines in order to write a file or paste a region into the clipboard.
A line is an object representing a sequence of characters with no line breaks.
Given a line, this function returns as a simple string the characters in the line. This is setf'able to set the line-string to any string that does not contain newline characters. It is an error to destructively modify the result of line-string or to destructively modify any string after the line-string of some line has been set to that string.
This function returns an object that serves as a signature for a line's contents. It is guaranteed that any modification of text on the line will result in the signature changing so that it is not eql to any previous value. The signature may change even when the text remains unmodified, but this does not happen often.
A mark indicates a specific position within the text represented by a
line and a character position within that line. Although a mark is
sometimes loosely referred to as pointing to some character, it in
fact points between characters. If the charpos is zero, the previous
character is the newline character separating the previous line from
the mark's line. If the charpos is equal to the number of characters
in the line, the next character is the newline character separating
the current line from the next. If the mark's line has no previous
line, a mark with charpos of zero has no previous character; if the
mark's line has no next line, a mark with charpos equal to the length of
the line has no next character.
This section discusses the very basic operations involving marks, but a lot of Hemlock programming is built on altering some text at a mark. For more extended uses of marks see Altering And Searching Text.
A mark may have one of two lifetimes: temporary or permanent. Permanent marks remain valid after arbitrary operations on the text; temporary marks do not. Temporary marks are used because less bookkeeping overhead is involved in their creation and use. If a temporary mark is used after the text it points to has been modified results will be unpredictable. Permanent marks continue to point between the same two characters regardless of insertions and deletions made before or after them.
There are two different kinds of permanent marks which differ only in their behavior when text is inserted at the position of the mark; text is inserted to the left of a left-inserting mark and to the right of right-inserting mark.
These functions destructively modify marks to point to new positions. Other sections of this document describe mark moving routines specific to higher level text forms than characters and lines, such as words, sentences, paragraphs, Lisp forms, etc.
This function changes mark to point n lines after (n before if n is negative) the current position. The character position of the resulting mark is (min (line-length resulting-line) (mark-charpos mark)) if charpos is unspecified, or (min (line-length resulting-line) charpos) if it is. As with character-offset, if there are not n lines then nil is returned and mark is not modified.
A region is simply a pair of marks: a starting mark and an ending
mark. The text in a region consists of the characters following the
starting mark and preceding the ending mark (keep in mind that a mark
points between characters on a line, not at them). By modifying the
starting or ending mark in a region it is possible to produce regions
with a start and end which are out of order or even in different
buffers. The use of such regions is undefined and may result in
arbitrarily bad behavior.
This function returns the number of lines in the region, first and last lines inclusive. A newline is associated with the line it follows, thus a region containing some number of non-newline characters followed by one newline is one line, but if a newline were added at the beginning, it would be two lines.
A buffer is an object consisting of:
A name.
A piece of text.
The insertion point.
An associated file (optional).
A write protect flag.
Some variables.
Some key bindings.
A collection of modes.
A list of modeline fields (optional).
Because of the way Hemlock is currently integrated in Cocoa, all modifications
to buffer contents must take place in the GUI thread. Hemlock commands always
run in the GUI thread, so most of the time you do not need to worry about it.
If you are running code in another thread that needs to modify a buffer, you
should perform that action using gui::execute-in-gui or gui::queue-for-gui.
There are no intrinsic limitations on examining buffers from any thread, however, Hemlock currently does no locking, so you risk seeing the buffer in an inconsistent state if you look at it outside the GUI thread.
Hemlock has the concept of the "current buffer". The current buffer is defined during Hemlock commands as the buffer of the hemlock view that received the key events that invoked the command. Many hemlock function operate on the current buffer rather than taking an explicit buffer argument. In effect, the current buffer is an implicit argument to many text manipulation functions.
This function pops the current buffer's mark stack, returning the mark. If the stack becomes empty, this pushes a new mark on the stack pointing to the buffer's start. This always deactivates the current region (see Active Regions).
This function pushes mark into the current buffer's mark stack, ensuring that the mark is right-inserting. If mark does not point into the current buffer, this signals an error. Optionally, the current region is made active, but this never deactivates the current region (see Active Regions). Mark is returned.
This variable holds a string-table mapping the name of a buffer to the corresponding buffer object.
make-buffer creates and returns a buffer with the given name. If a buffer named name already exists, nil is returned. Modes is a list of modes which should be in effect in the buffer, major mode first, followed by any minor modes. If this is omitted then the buffer is created with the list of modes contained in Default Modes. Modeline-fields is a list of modeline-field objects (see the Modelines section) which may be nil. delete-hook is a list of delete hooks specific to this buffer, and delete-buffer invokes these along with Delete Buffer Hook.
Buffers created with make-buffer are entered into the list (all-buffers), and their names are inserted into the string-table *buffer-names*. When a buffer is created the hook Make Buffer Hook is invoked with the new buffer.
Returns the buffer's region. Note this is the region that contains all the text in a buffer, as opposed to the current-region.
This can be set with setf to replace the buffer's text.
buffer-pathname returns the pathname of the file associated with the given buffer, or nil if it has no associated file. This is the truename of the file as of the most recent time it was read or written. There is a setf form to change the pathname. When the pathname is changed the hook Buffer Pathname Hook is invoked with the buffer and new value.
Returns the mark which is the current location within buffer. To move the point, use move-mark or move-to-position
This function returns t if you can modify the buffer, nil if you cannot. If a buffer is not writable, then any attempt to alter text in the buffer results in an error. There is a setf method to change this value. The setf method invokes the functions in Buffer Writable Hook on the buffer and new value before storing the new value.
buffer-modified returns t if the buffer has been modified, nil if it hasn't. This attribute is set whenever a text-altering operation is performed on a buffer. There is a setf method to change this value. The setf method invokes the functions in Buffer Modified Hook with the buffer whenever the value of the modified flag changes.
This function returns a string-table containing the names of the buffer's local variables.
This function returns the list of the names of the modes active in buffer. The major mode is first, followed by any minor modes. See the Modes chapter.
delete-buffer removes buffer from (all-buffers) and its name from *buffer-names*. Before buffer is deleted, this invokes the functions on buffer returned by buffer-delete-hook and those found in Delete Buffer Hook. If buffer is the current-buffer, or if it is displayed in any view, then this function signals an error.
A Buffer may specify a modeline, a line of text which is displayed across the bottom of a view to indicate status information. Modelines are described by a list of modeline-field objects which have individual update functions and are optionally fixed-width. These have an eql name for convenience in referencing and updating, but the name must be unique for all created modeline-field objects. All modeline-field functions must take a buffer as an argument and return a string. When displaying a modeline-field with a specified width, the result of the update function is either truncated or padded on the right to meet the constraint.
Whenever one of the following changes occurs, all of a buffer's modeline fields are updated:
A buffer's major mode is set.
One of a buffer's minor modes is turned on or off.
A buffer is renamed.
A buffer's pathname changes.
A buffer's modified status changes.
The policy is that whenever one of these changes occurs, it is guaranteed that the modeline will be updated before the next trip through redisplay. Furthermore, since the system cannot know what modeline-field objects the user has added whose update functions rely on these values, or how he has changed Default Modeline Fields, we must update all the fields.
The user should note that modelines can be updated at any time, so update functions should be careful to avoid needless delays (for example, waiting for a local area network to determine information).
This function returns a modeline-field object with name, width, and function. Width defaults to nil meaning that the field is variable width; otherwise, the programmer must supply this as a positive integer. Function must take a buffer as an arguments and return a string. If name already names a modeline-field object, then this signals an error.
Returns the function called when updating the modeline-field. When this is set with setf, the setf method updates modeline-field for all views on all buffers that contain the given field, so the next trip through redisplay will reflect the change. All modeline-field functions must take a buffer as an argument and return a string.
Returns the width to which modeline-field is constrained, or nil indicating that it is variable width. When this is set with setf, the setf method updates all modeline-fields for all views on all buffers that contain the given field, so the next trip through redisplay will reflect the change.
Returns a copy of the list of buffer's modeline-field objects. This list can be destructively modified without affecting display of buffer's modeline, but modifying any particular field's components (for example, width or function) causes the changes to be reflected the next trip through redisplay in every modeline display that uses the modified modeline-field. When this is set with setf, the setf method method updates all modeline-fields on all views on the buffer, so next trip through the redisplay will reflect the change.
A note on marks and text alteration: :temporary marks are invalid after any change has been made to the buffer the mark points to; it is an error to use a temporary mark after such a change has been made.
If text is deleted which has permanent marks pointing into it then they are left pointing to the position where the text was.
Like insert-region, inserts the region at the mark's position,
destroying the source region. This must be used with caution, since
if anyone else can refer to the source region bad things will
happen. In particular, one should make sure the region is not linked
into any existing buffer. If region is empty, and mark is in some
buffer, then Hemlock leaves buffer-modified of mark's buffer unaffected.
This deletes n characters after the mark (or -n before if n is negative). If n characters after (or -n before) the mark do not exist, then this returns nil; otherwise, it returns t. If n is zero, and mark is in some buffer, then Hemlock leaves buffer-modified of mark's buffer unaffected.
Destructively modifies region by replacing the text of each line with the result of the application of function to a string containing that text. Function must obey the following restrictions:
The argument may not be destructively modified.
The return value may not contain newline characters.
The return value may not be destructively modified after it is returned from function.
The strings are passed in order.
Using this function, a region could be uppercased by doing:
(filter-region #'string-upcase region)
Returns t if line contains only characters with a Whitespace attribute of 1. See the Character Attributes chapter for discussion of character attributes.
These predicates test the relative ordering of two marks in a piece of text, that is a mark is mark> another if it points to a position after it. An error is signalled if the marks do not point into the same buffer, except that for such marks mark= is always false and mark/= is always true.
<