Building MCL from Scratch

Shannon Spires and Gary Byers, 12-2-2003
Last updated 2-24-2004

Contents

 

Overview

MCL must be built not just from source code but from a previous version of itself, in much the same way that OpenMCL is built. An application called ccl (or sometimes ppccl) is used to compile all the source code and generate a small application called PPC-boot. PPC-boot, in turn, is used to generate a new ccl. You must have either PPC-boot (and a full collection of current FASL files) or ccl/ppccl available before you can begin the build process.

In addition, MCL requires kernels called pmcl-kernel (in the case of both OS9 and OSX) and pmcl-OSX-kernel (in the case of OSX). The kernels are C and assembly-language programs, so they can be built purely from their C and assembly-language source code files. The pmcl-kernel is required for MCL running on either platform. On OSX, the pmcl-OSX-kernel is additionally required for reliable operation, but MCL will run in a pinch without it.

You must have a directory of MCL source code (usually called MCL-dev) somewhere on your system before you start. This may come from a CVS repository or other file server, but that's beyond the scope of this document.

 

Building

To build the OS9 (CFM) pmcl-kernel (this also brings in .r resource files):

Obtain MPW (still on an Apple FTP server ?) and a copy of the GNU C Compiler for MPW and a customized version of MPW's "buildprogam" script. [we should provide a .dmg of a fully configured MPW on a local server.] Install MPW somehere where there are no spaces in the path name (this confuses the preprocessor's attempts to find header files.).

Launch MPW. In the Directory menu, Set Directory to the PMCL directory within MCL-DEV

Then type in the following two commands into the MPW command window. Note that each of these must be terminated by the <enter> key (at the far right end of your keyboard, or down near your spacebar in the case of a laptop) and not the <return> key.

buildprogram clean

buildprogram -f makefile-carbon

(this will fail if there are any spaces in the complete pathname to MPW or the PMCL directory)

Drag the new pmcl-kernel up to the toplevel MCL-DEV directory.

The OS9 (or CFM -- Code Fragment Manager) kernel (pmcl-kernel) is chiefly intended for use under OS9 or Classic. For native use under OSX, the native kernel (pmcl-OSX-kernel) is preferred, but the OS9 kernel is still required to find and install the native kernel at runtime. The OS9 kernel also provides some resources that MCL requires, even on OSX. This "two-kernel" situation on OSX is not likely to be resolved any time soon because MCL is still a CFM application and cannot link to Mach-O libraries, which is what the native kernel is. The OS9 kernel (which is a CFM library) provides the "glue" that enables MCL to use the native kernel. The native kernel is essential for reliable operation of MCL on OSX, and it provides debugging symbols that are helpful when debugging MCL under GDB.

There's an MCL function that returns the version number of the OSX kernel:

(ccl:native-mcl-kernel-version)

returns major/minor kernel version or (values NIL NIL), so:

(if (and (osx-p) (null (native-mcl-kernel-version)))
(error "Abandon all hope, all ye who enter here."))

doesn't seem too unreasonable.

The native kernel has to be in the same directory as the CFM kernel and can't be renamed. (Yes, this is stupid; if MCL were bundled, the application bundle would be a good place to look for it.)

There are some "standard" places where CFM libraries are looked for, including

/System/Library/CFMSupport,

/Library/CFMSupport, and (IIRC)

~/Library/CFMSupport.

Keeping the kernels in one of these places makes it easier to save MCL images in arbitrary directories (without having to copy the kernels to those places as well.) Things that're in /System may get wiped out by OS upgrades, so /Library/CFMSupport (or ~/Library/CFMSupport, if that works ...) is probably a better choice.

 

To build the native kernel under OSX:

1) In the OSX terminal, cd to the pmcl/OSX subdirectory of the CCL directory.

2) at the Unix prompt, do

make -f Makefile.unix

That should compile about a dozen C and assembler sources and link them into pmcl-OSX-kernel in the current directory.

The makefile (Makefile.unix) and some assembler source files in that directory are LF-terminated and need to stay that way.

Drag the new pmcl-OSX-kernel up to the toplevel MCL-DEV directory.

 

To build OpenTransportSupport library:

Launch MPW (see above for how to get it). In the Directory menu, Set Directory to the OpenTransportSupport-Folder directory within MCL-DEV

Then type in the following two commands into the MPW command window. Note that each of these must be terminated by the <enter> key (at the far right end of your keyboard, or down near your spacebar in the case of a laptop) and not the <return> key.

buildprogram clean

buildprogram -f makefile

(this will fail if there are any spaces in the complete pathname to MPW or the OpenTransportSupport-Folder directory)

If you see a message like the following, don't worry about it

   ### duplicate - "" is not a valid target name.

Drag the new OpenTransportSupport library up to the toplevel MCL-DEV directory.

 

To build ppc-boot

Make sure you don't have an init.lisp file before doing the following. If you do have one, rename it so it won't be found.

Launch ccl (or ppccl. Two different naming conventions for the same thing)

(reindex-interfaces)

(setf *fasl-save-definitions* nil) ; this seems to be necessary for some reason

(compile-ccl t) ; generates fasl files for everything except level-0. Note that we're using
; the term "fasl" generically. These files will probably end up being named "*.cfsl" or some such.

Sometimes you can get away with (compile-ccl) --without the argument--if you've recently compiled all the source. If you get errors during compile-ccl, try getting rid of your cfsl files. Even doing (compile-ccl t) doesn't always ensure that old cfsl files will be ignored (e.g. the ones in ccl:Library are sometimes required inadvertently.)

(xload-level-0-carbon :force) ; generates fasls for level-0 and generates ppc-boot

This may generate a lot of warnings and "Can't find..." messages. Don't worry about it.

(quit)

 

To build a new ccl (or ppccl)

Make sure you don't have an init.lisp file before doing the following. If you do have one, rename it so it won't be found.

Run ppc-boot. In the old days, this would die quickly after launch if you ran it in OSX, and you had to force it to run in Classic. That shouldn't happen any more.

It's possible that running ppc-boot will create a console window (which may be called "AltConsole" on OSX) showing lines like ";;;Loading foo" in red scrolling upwards. This is the MCL debugging console, but in this case it's just here for informative messages. If all goes well, a listener window will be created after all the files load. If no listener window is created, or the console ends showing a message indicating a bug, breakpoint, or error, that indicates a serious problem with ppc-boot which is beyond the scope of this document.

(require :ppc-init-ccl-carbon) ; loads a few dozen files that haven't been loaded yet

(save-application "ccl") ; Or "ppccl". This image will have the new plst

Increase the memory partition of "ccl" to something reasonable (this will be unnecessary once we get automatic heap allocation working). To do this, Get Info on the application, check "Open in the Classic Environment," change the memory allocation, then close the Get Info window. Do Get Info on it again and uncheck "Open in the Classic Environment."

 

To make an MCL, run ccl (or ppccl) and then

(load "ccl:Lib;prepare-mcl-environment.lisp")

Then save the application with the name "MCL" or "MCL 5.0" or whatever.

 

To build the MCL help file and index (map) file, (load "ccl:help-maker;make-help-files")

 

To create a distribution of MCL, use gather-sources-2.lisp and ship-files.lisp.