Previous Section Next Chapter Table of Contents Glossary Index

Chapter 12. The Foreign-Function Interface

12.12. The Foreign-Function-Interface Dictionary

[Reader Macro]

#_

Description:

Reads a symbol from the current input stream, with *PACKAGE* bound to the "OS" package and with readtable-case preserved.

Does a lookup on that symbol in the CCL interface database, signalling an error if no foreign function information can be found for the symbol in any active interface directory.

Notes the foreign function information, including the foreign function's return type, the number and type of the foreign function's required arguments, and an indication of whether or not the function accepts additional arguments (via e.g., the "varargs" mechanism in C).

Defines a macroexpansion function on the symbol, which expand macro calls involving the symbol into EXTERNAL-CALL forms where foreign argument type specifiers for required arguments and the return value specifer are provided from the information in the database.

Returns the symbol.

The effect of these steps is that it's possible to call foreign functions that take fixed numbers of arguments by simply providing argument values, as in:

(#_isatty fd)
          (#_read fd buf n)

and to call foreign functions that take variable numbers of arguments by specifying the types of non-required args, as in:

(with-cstrs ((format-string "the answer is: %d"))
          (#_printf format-string :int answer))

You can query whether a given name is defined in the interface databases by appending the '?' character to the reader macro; for example:

          CL-USER> #_?printf
          T
          CL-USER> #_?foo
          NIL
        

[Reader Macro]

#&

Description:

In CCL 1.2 and later, the #& reader macro can be used to access foreign variables; this functionality depends on the presence of "vars.cdb" files in the interface database. The current behavior of the #& reader macro is to:

Read a symbol from the current input stream, with *PACKAGE* bound to the "OS" package and with readtable-case preserved.

Use that symbol's pname to access the CCL interface database, signalling an error if no appropriate foreign variable information can be found with that name in any active interface directory.

Use type information recorded in the database to construct a form which can be used to access the foreign variable, and return that form.

Please note that the set of foreign variables declared in header files may or may not match the set of foreign variables exported from libraries (we're generally talking about C and Unix here ...). When they do match, the form constructed by the #& reader macro manages the details of resolving and tracking changes to the foreign variable's address.

Future extensions (via prefix arguments to the reader macro) may offer additional behavior; it might be convenient (for instance) to be able to access the address of a foreign variable without dereferencing that address.

Foreign variables in C code tend to be platform- and package-specific (the canonical example - "errno" - is typically not a variable when threads are involved. )

In LinuxPPC,

? #&stderr

returns a pointer to the stdio error stream ("stderr" is a macro under OSX/Darwin).

On both LinuxPPC and DarwinPPC,

? #&sys_errlist

returns a pointer to a C array of C error message strings.

You can query whether a given name is defined in the interface databases by appending the '?' character to the reader macro; for example:

          CL-USER> #&?sys_errlist
          T
          CL-USER> #&?foo
          NIL
        

[Reader Macro]

#$

Description:

In CCL 0.14.2 and later, the #? reader macro can be used to access foreign constants; this functionality depends on the presence of "constants.cdb" files in the interface database. The current behavior of the #$ reader macro is to:

Read a symbol from the current input stream, with *PACKAGE* bound to the "OS" package and with readtable-case preserved.

Use that symbol's pname to access the CCL interface database, signalling an error if no appropriate foreign constant information can be found with that name in any active interface directory.

Use type information recorded in the database to construct a form which can be used to access the foreign constant, and return that form.

Please note that the set of foreign constants declared in header files may or may not match the set of foreign constants exported from libraries. When they do match, the form constructed by the #$ reader macro manages the details of resolving and tracking changes to the foreign constant's address.

You can query whether a given name is defined in the interface databases by appending the '?' character to the reader macro; for example:

          CL-USER> #$?SO_KEEPALIVE
          T
          CL-USER> #$?foo
          NIL
        

[Reader Macro]

#/

Description:

In CCL 1.2 and later, the #/ reader macro can be used to access foreign functions on the Darwin platform. The current behavior of the #/ reader macro is to:

Read a symbol from the current input stream, with *PACKAGE* bound to the "NEXTSTEP-FUNCTIONS" package, with readtable-case preserved, and with any colons included.

Do limited sanity-checking on the resulting symbol; for example, any name that contains at least one colon is required also to end with a colon, to conform to Objective-C method-naming conventions.

Export the resulting symbol from the "NEXTSTEP-FUNCTIONS" package and return it.

For example, reading "#/alloc" interns and returns NEXTSTEP-FUNCTIONS:|alloc|. Reading "#/initWithFrame:" interns and returns NEXTSTEP-FUNCTIONS:|initWithFrame:|.

A symbol read using this macro can be used as an operand in most places where an Objective-C message name can be used, such as in the (OBJ:@SELECTOR ...) construct.

Please note: the reader macro is not rigorous about enforcing Objective-C method-naming conventions. Despite the simple checking done by the reader macro, it may still be possible to use it to construct invalid names.

The act of interning a new symbol in the NEXTSTEP-FUNCTIONS package triggers an interface database lookup of Objective-C methods with the corresponding message name. If any such information is found, a special type of dispatching function is created and initialized and the new symbol is given the newly-created dispatching function as its function definition.

The dispatching knows how to call declared Objective-C methods defined on the message. In many cases, all methods have the same foreign type signature, and the dispatching function merely passes any arguments that it receives to a function that does an Objective-C message send with the indicated foreign argument and return types. In other cases, where different Objective-C messages have different type signatures, the dispatching function tries to choose a function that handles the right type signature based on the class of the dispatching function's first argument.

If new information about Objective-C methods is introduced (e.g., by using additional interface files or as Objective-C methods are defined from lisp), the dispatch function is reinitialized to recognize newly-introduced foreign type signatures.

The argument and result coercion that the bridge has traditionally supported is supported by the new mechanism (e.g., :<BOOL> arguments can be specified as lisp booleans and :<BOOL> results are returned as lisp boolean values, and an argument value of NIL is coerced to a null pointer if the corresponding argument type is :ID.

Some Objective-C methods accept variable numbers of arguments; the foreign types of non-required arguments are determined by the lisp types of those arguments (e.g., integers are passed as integers, floats as floats, pointers as pointers, record types by reference.)

Examples:

          ;;; #/alloc is a known message.
          ? #'#/alloc
          #<OBJC-DISPATCH-FUNCTION NEXTSTEP-FUNCTIONS:|alloc| #x300040E94EBF>
          ;;; Sadly, #/foo is not ...
          ? #'#/foo
          > Error: Undefined function: NEXTSTEP-FUNCTIONS:|foo|

          ;;; We can send an "init" message to a newly-allocated instance of
          ;;; "NSObject" by:

          (send (send ns:ns-object 'alloc) 'init)

          ;;; or by

          (#/init (#/alloc ns:ns-object))
        

Objective-C methods that "return" structures return them as garbage-collectable pointers when called via dispatch functions. For example, if "my-window" is an NS:NS-WINDOW instance, then

          (#/frame my-window)
        

returns a garbage-collectable pointer to a structure that describes that window's frame rectangle. This convention means that there's no need to use SLET or special structure-returning message send syntax; keep in mind, though, that #_malloc, #_free, and the GC are all involved in the creation and eventual destruction of structure-typed return values. In some programs these operations may have an impact on performance.

[Reader Macro]

#>

Description:

In CCL 1.2 and later, the #> reader macro reads the following text as a keyword, preserving the case of the text. For example:

          CL-USER> #>FooBar
          :<F>OO<B>AR
        

The resulting keyword can be used as the name of foreign types, records, and accessors.

[Function]

close-shared-library library &key completely
Stops using a shared library, informing the operating system that it can be unloaded if appropriate.

Values:

library---either an object of type SHLIB, or a string which designates one by its so-name.

completely---a boolean. The default is T.

Description:

If completely is T, sets the reference count of library to 0. Otherwise, decrements it by 1. In either case, if the reference count becomes 0, close-shared-library frees all memory resources consumed library and causes any EXTERNAL-ENTRY-POINTs known to be defined by it to become unresolved.

[Macro]

defcallback name ({arg-type-specifier var}* &optional result-type-specifier) &body body

Values:

name---A symbol which can be made into a special variable

arg-type-specifer---One of the foreign argument-type keywords, described above, or an equivalent foreign type specifier. In addition, if the keyword :WITHOUT-INTERRUPTS is specified, the callback will be executed with lisp interrupts disabled if the corresponding var is non-NIL. If :WITHOUT-INTERRUPTS is specified more than once, the rightmost instance wins.

var---A symbol (lisp variable), which will be bound to a value of the specified type.

body---A sequence of lisp forms, which should return a value which can be coerced to the specified result-type.

Description:

Proclaims name to be a special variable; sets its value to a MACPTR which, when called by foreign code, calls a lisp function which expects foreign arguments of the specified types and which returns a foreign value of the specified result type. Any argument variables which correspond to foreign arguments of type :ADDRESS are bound to stack-allocated MACPTRs.

If name is already a callback function pointer, its value is not changed; instead, it's arranged that an updated version of the lisp callback function will be called. This feature allows for callback functions to be redefined incrementally, just like Lisp functions are.

defcallback returns the callback pointer, e.g., the value of name.

[Macro]

def-foreign-type name foreign-type-spec

Values:

name---NIL or a keyword; the keyword may contain escaping constructs.

foreign-type-spec---A foreign type specifier, whose syntax is (loosely) defined above.

Description:

If name is non-NIL, defines name to be an alias for the foreign type specified by foreign-type-spec. If foreign-type-spec is a named structure or union type, additionally defines that structure or union type.

If name is NIL, foreign-type-spec must be a named foreign struct or union definition, in which case the foreign structure or union definition is put in effect.

Note that there are two separate namespaces for foreign type names, one for the names of ordinary types and one for the names of structs and unions. Which one name refers to depends on foreign-type-spec in the obvious manner.

[Macro]

external name => entry
Resolves a reference to an external symbol which is defined in a shared library.

Values:

name--- a simple-string which names an external symbol. Case-sensitive.

entry--- an object of type EXTERNAL-ENTRY-POINT which maintains the address of the foreign symbol named by name.

Description:

If there is already an EXTERNAL-ENTRY-POINT for the symbol named by name, finds it and returns it. If not, creates one and returns it.

Tries to resolve the entry point to a memory address, and identify the containing library.

Be aware that under Darwin, external functions which are callable from C have underscores prepended to their names, as in "_fopen".

[Macro]

external-call name {arg-type-specifier arg}* &optional result-type-specifier

Values:

name---A lisp string. See external, above.

arg-type-specifer---One of the foreign argument-type keywords, described above, or an equivalent foreign type specifier.

arg---A lisp value of type indicated by the corresponding arg-type-specifier

result-type-specifier---One of the foreign argument-type keywords, described above, or an equivalent foreign type specifier.

Description:

Calls the foreign function at the address obtained by resolving the external-entry-point associated with name, passing the values of each arg as a foreign argument of type indicated by the corresponding arg-type-specifier. Returns the foreign function result (coerced to a Lisp object of type indicated by result-type-specifier), or NIL if result-type-specifer is :VOID or NIL

[Function]

%ff-call entrypoint {arg-type-keyword arg}* &optional result-type-keyword

Values:

entrypoint---A fixnum or MACPTR

arg-type-keyword---One of the foreign argument-type keywords, described above

arg---A lisp value of type indicated by the corresponding arg-type-keyword

result-type-keyword---One of the foreign argument-type keywords, described above

Description:

Calls the foreign function at address entrypoint passing the values of each arg as a foreign argument of type indicated by the corresponding arg-type-keyword. Returns the foreign function result (coerced to a Lisp object of type indicated by result-type-keyword), or NIL if result-type-keyword is :VOID or NIL

[Macro]

ff-call entrypoint {arg-type-specifier arg}* &optional result-type-specifier

Values:

entrypoint---A fixnum or MACPTR

arg-type-specifer---One of the foreign argument-type keywords, described above, or an equivalent foreign type specifier.

arg---A lisp value of type indicated by the corresponding arg-type-specifier

result-type-specifier---One of the foreign argument-type keywords, described above, or an equivalent foreign type specifier.

Description:

Calls the foreign function at address entrypoint passing the values of each arg as a foreign argument of type indicated by the corresponding arg-type-specifier. Returns the foreign function result (coerced to a Lisp object of type indicated by result-type-specifier), or NIL if result-type-specifer is :VOID or NIL

[Function]

foreign-symbol-address name

Values:

name---A lisp string.

Description:

Tries to resolve the address of the foreign symbol name. If successful, returns that address encapsulated in a MACPTR, else returns NIL.

[Function]

foreign-symbol-entry name

Values:

name---A lisp string.

Description:

Tries to resolve the address of the foreign symbol name. If successful, returns a fixnum representation of that address, else returns NIL.

[Function]

free ptr

Values:

ptr---A MACPTR that points to a block of foreign, heap-allocated memory.

Description:

In CCL 1.2 and later, the CCL:FREE function invokes the foreign free function from the platform's standard C library to deallocate a block of foreign memory.

Previous versions of CCL implemented this function, but it was not exported.

If the argument to CCL:FREE is a gcable pointer (for example, an object returned by MAKE-GCABLE-RECORD) then CCL:FREE informs the garbage collector that the foreign memory has been deallocated before calling the foreign free function.

[Function]

make-heap-ivector element-count element-type => vector macptr size

Values:

element-count---A positive integer.

element-type---A type specifier.

vector---A lisp vector. The initial contents are undefined.

mactpr---A pointer to the first byte of data stored in the vector.

size---The size of the returned vector in octets.

Description:

An "ivector" is a one-dimensional array that's specialized to a numeric or character element type.

MAKE-HEAP-IVECTOR allocates an ivector in foreign memory. The GC will never move this vector, and will in fact not pay any attention to it at all. The returned pointer to it can therefore be passed safely to foreign code.

The vector must be explicitly deallocated with DISPOSE-HEAP-IVECTOR.

[Macro]

make-gcable-record typespec &rest initforms => result

Values:

typespec---A foreign type specifier, or a keyword which is used as the name of a foreign struct or union.

initforms---If the type denoted by typespec is scalar, a single value appropriate for that type; otherwise, a list of alternating field names and values appropriate for the types of those fields.

result--- A macptr which encapsulates the address of a newly-allocated record on the foreign heap. The foreign object returned by make-gcable-record is freed when the garbage collector determines that the MACPTR object that describes it is unreachable.

Description:

Allocates a block of foreign memory suitable to hold the foreign type described by typespec, in the same manner as MAKE-RECORD. In addition, MAKE-GCABLE-RECORD marks the returned object gcable; in other words, it informs the garbage collector that it may reclaim the object when it becomes unreachable.

In all other respects, MAKE-GCABLE-RECORD works the same way as MAKE-RECORD

When using gcable pointers, it's important to remember the distinction between a MACPTR object (which is a lisp object, more or less like any other) and the block of foreign memory that the MACPTR object points to. If a gcable MACPTR object is the only thing in the world (lisp world or foreign world) that references the underlying block of foreign memory, then freeing the foreign memory when it becomes impossible to reference it is convenient and sane. If other lisp MACPTRs reference the underlying block of foreign memory or if the address of that foreign memory is passed to and retained by foreign code, having the GC free the memory may have unpleasant consequences if those other references are used.

Take care, therefore, not to create a gcable record unless you are sure that the returned MACPTR will be the only reference to the allocated memory that will ever be used.

[Macro]

make-record typespec &rest initforms => result

Values:

typespec---A foreign type specifier, or a keyword which is used as the name of a foreign struct or union.

initforms---If the type denoted by typespec is scalar, a single value appropriate for that type; otherwise, a list of alternating field names and values appropriate for the types of those fields.

result--- A macptr which encapsulates the address of a newly-allocated record on the foreign heap.

Description:

Expands into code which allocates and initializes an instance of the type denoted by typespec, on the foreign heap. The record is allocated using the C function malloc, and the user of make-record must explicitly call the function CCL:FREE to deallocate the record, when it is no longer needed.

If initforms is provided, its value or values are used in the initialization. When the type is a scalar, initforms is either a single value which can be coerced to that type, or no value, in which case binary 0 is used. When the type is a struct, initforms is a list, giving field names and the values for each. Each field is treated in the same way as a scalar is: If a value for it is given, it must be coerceable to the field's type; if not, binary 0 is used.

When the type is an array, initforms may not be provided, because make-record cannot initialize its values. make-record is also unable to initialize fields of a struct which are themselves structs. The user of make-record should set these values by another means.

A possibly-significant limitation is that it must be possible to find the foreign type at the time the macro is expanded; make-record signals an error if this is not the case.

Notes:

It is inconvenient that make-record is a macro, because this means that typespec cannot be a variable; it must be an immediate value.

If it weren't for this requirement, make-record could be a function. However, that would mean that any stand-alone application using it would have to include a copy of the interface database (see Section 12.4, “The Interface Database”), which is undesirable because it's large.

[Function]

open-shared-library name => library
Asks the operating system to load a shared library for CCL to use.

Values:

name---A SIMPLE-STRING which is presumed to be the so-name of or a filesystem path to the library.

library---An object of type SHLIB which describes the library denoted by name.

Description:

If the library denoted by name can be loaded by the operating system, returns an object of type SHLIB that describes the library; if the library is already open, increments a reference count. If the library can't be loaded, signals a SIMPLE-ERROR which contains an often-cryptic message from the operating system.

Examples:
;;; Try to do something simple.
          ? (open-shared-library "libgtk.so")
          > Error: Error opening shared library "libgtk.so": /usr/lib/libgtk.so: undefined symbol: gdk_threads_mutex
          > While executing: OPEN-SHARED-LIBRARY

          ;;; Grovel around, curse, and try to find out where "gdk_threads_mutex"
          ;;; might be defined. Then try again:

          ? (open-shared-library "libgdk.so")
          #<SHLIB libgdk.so #x3046DBB6>

          ? (open-shared-library "libgtk.so")
          #<SHLIB libgtk.so #x3046DC86>

          ;;; Reference an external symbol defined in one of those libraries.

          ? (external "gtk_main")
          #<EXTERNAL-ENTRY-POINT "gtk_main" (#x012C3004) libgtk.so #x3046FE46>

          ;;; Close those libraries.

          ? (close-shared-library "libgtk.so")
          T

          ? (close-shared-library "libgdk.so")
          T

          ;;; Reference the external symbol again.

          ? (external "gtk_main")
          #<EXTERNAL-ENTRY-POINT "gtk_main" {unresolved} libgtk.so #x3046FE46>
Notes:

It would be helpful to describe what an soname is and give examples of one.

Does the SHLIB still get returned if the library is already open?

[Macro]

pref ptr accessor-form

Values:

ptr---a MACPTR.

accessor-form---a keyword which names a foreign type or record, as described in Section 12.8.3, “Foreign type, record, and field names”.

Description:

References an instance of a foreign type (or a component of a foreign type) accessible via ptr.

Expands into code which references the indicated scalar type or component, or returns a pointer to a composite type.

PREF can be used with SETF.

RREF is a deprecated alternative to PREF. It accepts a :STORAGE keyword and rather loudly ignores it.

[Function]

%reference-external-entry-point eep

Values:

eep---An EXTERNAL-ENTRY-POINT, as obtained by the EXTERNAL macro.

Description:

Tries to resolve the address of the EXTERNAL-ENTRY-POINT eep; returns a fixnum representation of that address if successful, else signals an error.

[Macro]

rlet (var typespec &rest initforms)* &body body

Values:

var---A symbol (a lisp variable)

typespec---A foreign type specifier or foreign record name.

initforms---As described above, for make-record

Description:

Executes body in an environment in which each var is bound to a MACPTR encapsulating the address of a stack-allocated foreign memory block, allocated and initialized from typespec and initforms as per make-record. Returns whatever value(s) body returns.

Record fields that aren't explicitly initialized have unspecified contents.

[Macro]

rletz (var typespec &rest initforms)* &body body

Values:

var---A symbol (a lisp variable)

typespec---A foreign type specifier or foreign record name.

initforms---As described above, for ccl:make-record

Description:

Executes body in an environment in which each var is bound to a MACPTR encapsulating the address of a stack-allocated foreign memory block, allocated and initialized from typespec and initforms as ccl:make-record.

Returns whatever value(s) body returns.

Unlike rlet, record fields that aren't explicitly initialized are set to binary 0.

[Function]

terminate-when-unreachable object

Values:

object---A CLOS object of a class for which there exists a method of the generic function ccl:terminate.

Description:

The "termination" mechanism is a way to have the garbage collector run a function right before an object is about to become garbage. It is very similar to the "finalization" mechanism which Java has. It is not standard Common Lisp, although other Lisp implementations have similar features. It is useful when there is some sort of special cleanup, deallocation, or releasing of resources which needs to happen when a certain object is no longer being used.

When the garbage collector discovers that an object is no longer referred to anywhere in the program, it deallocates that object, freeing its memory. However, if ccl:terminate-when-unreachable has been called on the object at any time, the garbage collector first invokes the generic function ccl:terminate, passing it the object as a parameter.

Therefore, to make termination do something useful, you need to define a method on ccl:terminate.

Because calling ccl:terminate-when-unreachable only affects a single object, rather than all objects of its class, you may wish to put a call to it in the initialize-instance method of a class. Of course, this is only appropriate if you do in fact want to use termination for all objects of a given class.

Example:
          (defclass resource-wrapper ()
            ((resource :accessor resource)))

          (defmethod initialize-instance :after ((x resource-wrapper) &rest initargs)
             (ccl:terminate-when-unreachable x))

          (defmethod ccl:terminate ((x resource-wrapper))
             (when (resource x)
                (deallocate (resource x))))

[Function]

unuse-interface-dir dir-id

Values:

dir-id---A keyword whose pname, mapped to lower case, names a subdirectory of "ccl:headers;" (or "ccl:darwin-headers;")

Description:

Tells CCL to remove the interface directory denoted by dir-id from the list of interface directories which are consulted for foreign type and function information. Returns T if the directory was on the search list, NIL otherwise.

[Function]

use-interface-dir dir-id

Values:

dir-id---A keyword whose pname, mapped to lower case, names a subdirectory of "ccl:headers;" (or "ccl:darwin-headers;")

Description:

Tells CCL to add the interface directory denoted by dir-id to the list of interface directories which it consults for foreign type and function information. Arranges that that directory is searched before any others.

Note that use-interface-dir merely adds an entry to a search list. If the named directory doesn't exist in the file system or doesn't contain a set of database files, a runtime error may occur when CCL tries to open some database file in that directory, and it will try to open such a database file whenever it needs to find any foreign type or function information. unuse-interface-dir may come in handy in that case.

Examples:

One typically wants interface information to be available at compile-time (or, in many cases, at read-time). A typical idiom would be:

(eval-when (:compile-toplevel :execute)
          (use-interface-dir :GTK))

Using the :GTK interface directory makes available information on foreign types, functions, and constants. It's generally necessary to load foreign libraries before actually calling the foreign code, which for GTK can be done like this:

(load-gtk-libraries)

It should now be possible to do things like:

(#_gtk_widget_destroy w)


Previous Section Next Chapter Table of Contents Glossary Index