Foreign Function Calls in OpenMCL

Overview.

OpenMCL provides a number of constructs for calling foreign functions from Lisp code (all of them based on the function CCL:%FF-CALL). In many cases, OpenMCL's interface translator provides information about the foreign function's entrypoint name and argument and return types; this enables the use of the #_ reader macro (described below), which may be more concise and/or more readable than other constructs.

OpenMCL also provides a mechanism for defining callbacks: lisp functions which can be called from foreign code.

There's no supported way to directly pass lisp data to foreign functions: scalar lisp data must be coerced to an equivalent foreign representatation, and lisp arrays (notably strings) must be copied to non-GCed memory.

The types of foreign argument and return values in foreign function calls and callbacks can be specified by any of the following keywords:

:UNSIGNED-BYTE
The argument/return value is of type (UNSIGNED-BYTE 8)
:SIGNED-BYTE
The argument/return value is of type (SIGNED-BYTE 8)
:UNSIGNED-HALFWORD
The argument/return value is of type (UNSIGNED-BYTE 16)
:SIGNED-HALFWORD
The argument/return value is of type (SIGNED-BYTE 16)
:UNSIGNED-FULLWORD
The argument/return value is of type (UNSIGNED-BYTE 32)
:SIGNED-FULLWORD
The argument/return value is of type (SIGNED-BYTE 32)
:UNSIGNED-DOUBLEWORD
The argument/return value is of type (UNSIGNED-BYTE 64)
:SIGNED-DOUBLEWORD
The argument/return value is of type (SIGNED-BYTE 64)
:SINGLE-FLOAT
The argument/return value is of type SINGLE-FLOAT
:DOUBLE-FLOAT
The argument/return value is of type DOUBLE-FLOAT
:ADDRESS>
The argument/return values is a MACPTR
:VOID or NIL
Not valid as an argument type specifier; specifies that there is no meaningful return value

Under Darwin (only), an small positive integer N can also be used as an argument specifier; it indicates that the corresponding argument is a pointer to an N-word structure or union which should be passed by value to the foreign function.

External entrypoints and named external entrypoints.

PowerPC machine instructions are always aligned on 32-bit boundaries, so the two least significant bits of the first instruction ("entrypoint") of a foreign function are always 0. OpenMCL often represents an entrypoint address as a fixnum that's binary-equivalent to the entrypoint address: if E is an entrypoint address expressed as a signed 32-bit integer, then (ash E -2) is an equivalent fixnum representation of that address. An entrypoint address can also be encapsulated in a MACPTR, but that's somewhat less efficient.

Although it's possible to use fixnums or macptrs to represent entrypoint addresses, it's somewhat cumbersome to do so. OpenMCL can cache the addresses of named external functions in structure-like objects of type CCL::EXTERNAL-ENTRY-POINT (sometimes abbreviated as EEP). Through the use of LOAD-TIME-VALUE, compiled lisp functions are able to reference EEPs as constants; the use of an indirection allows the OpenMCL runtime system to ensure that the EEP's address is current and correct.

Functional Reference

All of the symbols mentioned below are defined in the CCL package; most (but not all) are exported from that package.

ccl::%ff-call [Function]

Syntax
%ff-call entrypoint {arg-type-keyword arg}* &optional result-type-keyword
Description Calls the foreign function at address entrypoint passing the values of each arg as a foreign argument of type indicated by the corresponding arg-type-keyword. Returns the foreign function result (coerced to a Lisp object of type indicated by result-type-keyword), or NIL if result-type-keyword is :VOID or NIL
Arguments
entrypoint
A fixnum or MACPTR
arg-type-keyword
One of the foreign argument-type keywords, described above
arg
A lisp value of type indicated by the corresponding arg-type-keyword
result-type-keyword
One of the foreign argument-type keywords, described above

ff-call [Macro]

Syntax
ff-call entrypoint {arg-type-specifier arg}* &optional result-type-specifier
Description Calls the foreign function at address entrypoint passing the values of each arg as a foreign argument of type indicated by the corresponding arg-type-specifier. Returns the foreign function result (coerced to a Lisp object of type indicated by result-type-specifier), or NIL if result-type-specifer is :VOID or NIL
Arguments
entrypoint
A fixnum or MACPTR
arg-type-specifer
One of the foreign argument-type keywords, described above, or an equivalent foreign type specifier
arg
A lisp value of type indicated by the corresponding arg-type-keyword
result-type-specifier
One of the foreign argument-type keywords, described above, or an equivalent foreign type specifier

external [Macro]

Syntax
external name
Description Maps name to an EXTERNAL-ENTRY-POINT object; tries to resolve the entrypoint address of the foreign symbol name if it's not already resolved.
Arguments
name
A lisp string. Note that, under Darwin, the names of C-callable external functions have underscores prepended to them.

ccl::%reference-external-entry-point [Function]

Syntax
ccl::%reference-external-entry-point eep
Description Tries to resolve the address of the EXTERNAL-ENTRY-POINT eep; returns a fixnum representation of that address if successful, else signals an error.
Arguments
eep
An EXTERNAL-ENTRY-POINT, as obtained by the EXTERNAL macro.

ccl::external-call [Macro]

Syntax
ccl::external-call name {arg-type-specifier arg}* &optional result-type-specifier
Description Calls the foreign function at the address obtained by resolving the external-entry-point associated with name, passing the values of each arg as a foreign argument of type indicated by the corresponding arg-type-specifier. Returns the foreign function result (coerced to a Lisp object of type indicated by result-type-specifier), or NIL if result-type-specifer is :VOID or NIL
Arguments
name
A lisp string. See external, above.
arg-type-specifer
One of the foreign argument-type keywords, described above, or an equivalent foreign type specifier
arg
A lisp value of type indicated by the corresponding arg-type-keyword
result-type-specifier
One of the foreign argument-type keywords, described above, or an equivalent foreign type specifier

ccl::foreign-symbol-entry [Function]

Syntax
ccl::foreign-symbol-entry name
Description Tries to resolve the address of the foreign symbol name. If successful, returns a fixnum representation of that address, else returns NIL.
Arguments
name
A lisp string.

ccl::foreign-symbol-address [Function]

Syntax
ccl::foreign-symbol-address name
Description Tries to resolve the address of the foreign symbol name. If successful, returns that address encapsulated in a MACPTR, else returns NIL.
Arguments
name
A lisp string.

defcallback [Macro]

Syntax
defcallback name ({arg-type-specifier var}* &optional result-type-specifier) &body body
Description

Proclaims name to be a special variable; sets its value to a MACPTR which, when called by foreign code, calls a lisp function which expects foreign arguments of the specified types and which returns a foreign value of the specified result type. Any argument variables which correspond to foreign arguments of type :ADDRESS are bound to stack-allocated MACPTRs.

If name is already a callback function pointer, its value is not changed (though it's arranged that an updated version of the lisp callback function will be called.) This feature allows for incremental redefinition of callback functions.

DEFCALLBACK returns the callback pointer, e.g., the value of name

Arguments
name
A symbol which can be made into a special variable
arg-type-specifer
One of the foreign argument-type keywords, described above, or an equivalent foreign type specifier In addition, if the keyword :WITHOUT-INTERRUPTS is specified, the callback will be executed with lisp interrupts disabled if the corresponding var is non-NIL. If :WITHOUT-INTERRUPTS is specified more than once, the rightmost instance wins.
var
A symbol (lisp variable), which will be bound to a value of the specified type.
body
A sequence of lisp forms, which should return a value which can be coerced to the specified result-type.

#_ [Reader macro]

The #_ reader macro:

  1. Reads a symbol from the current input stream, with *PACKAGE* bound to the "OS" package and with readtable-case preserved.

  2. Does a lookup on that symbol in the OpenMCL interface database, signalling an error if no foreign function information can be found for the symbol in any active interface directory.

  3. Notes the foreign function information, including the foreign function's return type, the number and type of the foreign function's required arguments, and an indication of whether or not the function accepts additional arguments (via e.g., the "varargs" mechanism in C).

  4. Defines a macroexpansion function on the symbol, which expand macro calls involving the symbol into EXTERNAL-CALL forms where foreign argument type specifiers for required arguments and the return value specifer are provided from the information in the database.

  5. Returns the symbol.

The effect of these steps is that it's possible to call foreign functions that take fixed numbers of arguments by simply providing argument values, as in:

(#_isatty fd)
(#_read fd buf n)
    
and to call foreign functions that take variable numbers of arguments by specifying the types of non-required args, as in:
(with-cstrs ((format-string "the answer is: %d"))
  (#_printf format-string :int answer))
    

C structure return conventions

On both LinuxPPC and DarwinPPC, C functions that are defined to return structures or unions do by reference: they actually accept a first parameter of type "pointer to returned struct/union" - which must be allocated by the caller - and don't return a meaningful value. OpenMCL's interface translator make this explicit: it effectively prepends an :ADDRESS to the foreign function's argument list and changes its return type to :VOID.

DarwinPPC C functions actually follow a slightly different convention: if the size of the returned structure is 4 bytes or less, the "structure" is returned by value. This irregularity isn't handled by the interface translator; fortunately, it seems to very rarely occur in practice (nameservice functions that return a (:STRUCT :IN_ADDR) being a notable exception.)


Last modified: Sun Jun 23 01:46:55 MDT 2002