32.3. The Foreign Function Call Facility

Platform Dependent: Many UNIX, Win32 platforms only.

32.3.1. Introduction
32.3.2. Overview
32.3.3. (Foreign) C types
32.3.4. The choice of the C flavor
32.3.5. Foreign variables
32.3.6. Operations on foreign places
32.3.7. Foreign functions
32.3.7.1. Callbacks and memory management
32.3.8. Argument and result passing conventions
32.3.9. Parameter Mode
32.3.10. Examples
32.3.10.1. More examples

List of Examples

32.2. Simple declarations and access
32.3. External C variable and some accesses
32.4. Calling an external function
32.5. Another example for calling an external function
32.6. Accessing cpp macros
32.7. Calling Lisp from C
32.8. Calling Lisp from C dynamically
32.9. Variable size arguments: calling gethostname from CLISP
32.10. Accessing variables in shared libraries
32.11. Controlling validity of resources
32.12. Floating point arrays

32.3.1. Introduction

This facility, also known as Foreign Language Interface, allows one to call a function implemented in C from inside CLISP and to do many related things, like inspect and modify foreign memory, define a callback (i.e., make a lisp function available to the C world), etc. To use this facility, one writes a foreign function description into an ordinary Lisp file, which is then compiled and loaded as usual; or just evaluates the appropriate form in the read-eval-print loop.

There are two basic ways to do define a foreign function:

  1. Use dlopen and dlsym to get to the location of the function code in a dynamic library. To access this facility, pass the :LIBRARY option to FFI:DEF-CALL-OUT and FFI:DEF-C-VAR.

    Unfortunately, this functionality is not available on some operating systems, and, also, it offers only a part of the foreign functionality: cpp macros and inline functions cannot be accessed this way. On the other hand, this functionality is available in the read-eval-print loop and does not require a C compiler.

  2. Use a somewhat less direct way: when you do not use the :LIBRARY argument, COMPILE-FILE produces a #P".c" file (in addition to a #P".fas" and a #P".lib"). Then you compile (with a C compiler) and link it into CLISP (statically, linking it into lisp.a, or dynamically, loading it into a running CLISP using dlopen and dlsym). This way you can use any functionality your foreign library exports, whether using ordinary functions, inline functions, or cpp macros (see Example 32.6, “Accessing cpp macros”).

All symbols relating to the foreign function interface are exported from the package FFI. To use them, (USE-PACKAGE FFI).

Special FFI forms may appear anywhere in the Lisp file.

32.3.2. Overview

These are the special FFI forms. We have taken a pragmatic approach: the only foreign languages we support for now are C and ANSI C.

Note

Unless specifically noted otherwise, type specification parameters are not evaluated, so that they can be compiled by FFI:PARSE-C-TYPE into the internal format at macroexpansion time.

High-level FFI forms; name is any Lisp SYMBOL; c-name is a STRING

(FFI:DEF-C-TYPE name &OPTIONAL c-type)

This form makes name a shortcut for c-type. Note that c-type may already refer to name. Forward declarations of types are not possible, however.

When c-type is omitted, the type is assumed to be an integer, and its size and signedness are determined at link time, e.g., (FFI:DEF-C-TYPE size_t).

(FFI:DEF-C-VAR name {option}*)

This form defines a FFI:FOREIGN-VARIABLE. name is the Lisp name, a regular Lisp SYMBOL.

Options for FFI:DEF-C-VAR

(:NAME c-name)
specifies the name as seen from C, as a STRING. If not specified, it is derived from the print name of the Lisp name.
(:TYPE c-type)
specifies the variable's foreign type.
(:READ-ONLY BOOLEAN)
If this option is specified and non-NIL, it will be impossible to change the variable's value from within Lisp (using SETQ or similar).
(:ALLOC ALLOCATION)
This option can be either :NONE or :MALLOC-FREE and defaults to :NONE. If it is :MALLOC-FREE, any values of type FFI:C-STRING, FFI:C-PTR, FFI:C-PTR-NULL, FFI:C-ARRAY-PTR within the foreign value are assumed to be pointers to malloc-allocated storage, and when SETQ replaces an old value by a new one, the old storage is freed using free and the new storage allocated using malloc. If it is :NONE, SETQ assumes that the pointers point to good storage (not NULL!) and overwrites the old values by the new ones. This is dangerous (just think of overwriting a string with a longer one or storing some data in a NULL pointer...) and deprecated.
(:LIBRARY name)
Specifies the (optional) dynamic library which contains the variable, the default is set by FFI:DEFAULT-FOREIGN-LIBRARY.
(:VERSION version)
Specifies the (optional) symbol version in the library (therefore, if :VERSION is supplied, :LIBRARY must also be supplied)
(:DOCUMENTATION string)
Specifies the (optional) VARIABLE documentation.
(FFI:DEF-C-CONST name {option}*)

This form defines a Lisp constant variable name whose value is determined at build time using an internal FFI:FOREIGN-FUNCTION.

Options for FFI:DEF-C-CONST

(:NAME c-name)
specifies the name as seen from C, as a STRING. If not specified, it is derived from the print name of the Lisp name.
(:TYPE c-type)

specifies the constant's foreign type, one of

FFI:INT
FFI:C-STRING
FFI:C-POINTER
(:GUARD string)

specifies the cpp check to wrap around c-name, defaults to "defined(c-name)"; can be NIL to omit the test. When the test fails, name is unbound.

(:DOCUMENTATION string)
Specifies the (optional) VARIABLE documentation.

See also Example 32.6, “Accessing cpp macros”.

(FFI:DEF-CALL-OUT name {option}*)

This form defines a named call-out function (a foreign function called from Lisp: control flow temporarily leaves Lisp).

Options for FFI:DEF-CALL-OUT

(:NAME c-name)
Any Lisp function call to #'name is redirected to call the C function c-name.
(:ARGUMENTS {(argument c-type [PARAM-MODE [ALLOCATION]])}*)
(:RETURN-TYPE c-type [ALLOCATION])
Argument list and return value, see Section 32.3.8, “Argument and result passing conventions” and Section 32.3.9, “Parameter Mode”.
(:LANGUAGE language)
See Section 32.3.4, “The choice of the C flavor”.
(:BUILT-IN BOOLEAN)
When the function is a C built-in, the full prototype will be output (unless suppressed by FFI:*OUTPUT-C-FUNCTIONS*).
(:LIBRARY name)
Specifies the (optional) dynamic library which contains the function, the default is set by FFI:DEFAULT-FOREIGN-LIBRARY.
(:VERSION version)
Specifies the (optional) symbol version in the library (therefore, if :VERSION is supplied, :LIBRARY must also be supplied).
(:DOCUMENTATION string)
Specifies the (optional) FUNCTION documentation.

See also Section 32.3.7, “Foreign functions”.

(FFI:DEF-CALL-IN function {option}*)

This form defines a callback - a named call-in function (i.e., a Lisp function called from the foreign language: control flow temporary enters Lisp)

Options for FFI:DEF-CALL-IN

(:NAME c-name)
Any C function call to the C function c-name is redirected to call the Common Lisp function function, which should be a function name.
(:ARGUMENTS {(argument c-type [PARAM-MODE [ALLOCATION]])}*)
(:RETURN-TYPE c-type [ALLOCATION])
Argument list and return value, see Section 32.3.8, “Argument and result passing conventions” and Section 32.3.9, “Parameter Mode”.
(:LANGUAGE language)
See Section 32.3.4, “The choice of the C flavor”.

See also Section 32.3.7, “Foreign functions”.

(FFI:OPEN-FOREIGN-LIBRARY name &KEY :REQUIRE)

Open (load) a shared foreign library.

Some shared libraries depend on other shared libraries and this dependency can be specified using the :REQUIRE argument.

Unless the library has dependencies, this is only needed if you want to test for presence of a library without creating a foreign object referencing its contents. When you create a FFI:FOREIGN-VARIABLE or a FFI:FOREIGN-FUNCTION using FFI:DEF-C-VAR or FFI:DEF-CALL-OUT with a :LIBRARY argument, the library name is opened automatically.

E.g., libgsl.so requires libgslcblas.so:

(FFI:OPEN-FOREIGN-LIBRARY "libgsl.so")
*** - FFI:OPEN-FOREIGN-LIBRARY: Cannot open library "libgsl.so": "/usr/lib64/libgsl.so: undefined symbol: cblas_ctrmv"

so a common way is to pre-open the dependency:

(FFI:OPEN-FOREIGN-LIBRARY "libgslcblas.so")
(FFI:DEF-CALL-OUT gsl_cheb_alloc (:LIBRARY "libgsl.so") (:language :stdc)
  (:arguments (n ffi:int)) (:return-type ffi:c-pointer))
⇒ GSL_CHEB_ALLOC

Alas, this would work in the current image only: if you save the image, GSL_CHEB_ALLOC will not work there because CLISP will try to re-open libgsl.so and fail as above. However, using the :REQUIRE argument will tell CLISP to re-open both libraries in the right order:

$ clisp
> (FFI:OPEN-FOREIGN-LIBRARY "libgsl.so" :require '("libgslcblas.so"))
> (FFI:DEF-CALL-OUT gsl_cheb_alloc (:library "libgsl.so") (:language :stdc)
  (:arguments (n ffi:int)) (:return-type ffi:c-pointer))
> (EXT:SAVEINITMEM "foo" :executable t)
> (EXT:EXIT)
$ ./foo
> (gsl_cheb_alloc 10)
#<FFI:FOREIGN-ADDRESS #x0000000017AC38A0>
(FFI:CLOSE-FOREIGN-LIBRARY name)

Close (unload) a shared foreign library (opened by FFI:OPEN-FOREIGN-LIBRARY or the :LIBRARY argument to FFI:DEF-CALL-OUT or FFI:DEF-C-VAR).

If you want to modify your shared library, you need to close it using FFI:CLOSE-FOREIGN-LIBRARY first. When you use a FFI:FOREIGN-VARIABLE or a FFI:FOREIGN-FUNCTION which resides in the library name, it will be re-opened automatically.

(FFI:DEFAULT-FOREIGN-LIBRARY library-name)

This macro sets the default :LIBRARY argument for FFI:DEF-CALL-OUT and FFI:DEF-C-VAR. library-name should be NIL (meaning use the C file produced by COMPILE-FILE), a STRING, or, depending on the underlying dlsym or dlvsym implementation, :DEFAULT or :NEXT.

The default is set separately in each compilation unit, so, if you are interfacing to a single library, you can set this variable in the beginning of your lisp file and omit the :LIBRARY argument throughout the file.

(FFI:DEF-C-STRUCT name (symbol c-type)*)

This form defines name to be both a STRUCTURE-CLASS and a foreign C type with the given slots. If this class representation overhead is not needed one should consider writing (FFI:DEF-C-TYPE name (FFI:C-STRUCT {LIST | VECTOR} (symbol c-type)*)) instead. name is a SYMBOL (structure name) or a LIST whose FIRST element is the structure name and the REST is options. Two options are supported at this time:

Options for FFI:DEF-C-STRUCT

:TYPEDEF
means that the name of this structure is a C type defined with typedef elsewhere.
:EXTERNAL
means that this structure is defined in a C header file that you include with, e.g., (FFI:C-LINES "#include <filename.h>~%").

These options determine how the struct is written to the #P".c".

(FFI:DEF-C-ENUM name {symbol | (symbol [value])}*)

This form defines symbols as constants, similarly to the C declaration enum { symbol [= value], ... };

You can use (FFI:ENUM-FROM-VALUE name value) and (FFI:ENUM-TO-VALUE name symbol) to convert between the numeric and symbolic representations (of course, the latter function boils down to SYMBOL-VALUE plus a check that the symbol is indeed a constant defined in the FFI:DEF-C-ENUM name).

(FFI:C-LINES format-string {argument}*)

This form outputs the string (FORMAT NIL format-string {argument}*) to the C output file's top level. This is usually used to include the relevant header files, see :EXTERNAL and FFI:*OUTPUT-C-FUNCTIONS*.

When format-string is not a STRING, is should be a SYMBOL, and then the STRING (FORMAT NIL {argument}*) is added to the appropriate C function:

(FFI:ELEMENT c-place index1 ... indexn)
Array element: If c-place is of foreign type (FFI:C-ARRAY c-type (dim1 ... dimn)) and 0 ≤ index1 < dim1, ..., 0 ≤ indexn < dimn, this will be the place corresponding to (AREF c-place index1 ... indexn) or c-place[index1]...[indexn]. It is a place of type c-type. If c-place is of foreign type (FFI:C-ARRAY-MAX c-type dim) and 0 ≤ index < dim, this will be the place corresponding to (AREF c-place index) or c-place[index]. It is a place of type c-type.
(FFI:DEREF c-place)
Dereference pointer: If c-place is of foreign type (FFI:C-PTR c-type), (FFI:C-PTR-NULL c-type) or (FFI:C-POINTER c-type), this will be the place the pointer points to. It is a place of type c-type. For (FFI:C-PTR-NULL c-type), the c-place may not be NULL.
(FFI:SLOT c-place slot-name)
Struct or union component: If c-place is of foreign type (FFI:C-STRUCT class ... (slot-name c-type) ...) or of type (FFI:C-UNION ... (slot-name c-type) ...), this will be of type c-type.
(FFI:CAST c-place c-type)
Type change: A place denoting the same memory locations as the original c-place, but of type c-type.
(FFI:OFFSET c-place offset c-type)
Type change and displacement: return a place denoting a memory locations displaced from the original c-place by an offset counted in bytes, with type c-type. This can be used to resize an array, e.g. of c-type (FFI:C-ARRAY uint16 n) via (FFI:OFFSET c-place 0 '(FFI:C-ARRAY uint16 k)).
(FFI:C-VAR-ADDRESS c-place)
Return the address of c-place as a Lisp object of type FFI:FOREIGN-ADDRESS. This is useful as an argument to foreign functions expecting a parameter of C type FFI:C-POINTER.
(FFI:C-VAR-OBJECT c-place)
Return the FFI:FOREIGN-VARIABLE object underlying the c-place. This is also an acceptable argument type to a FFI:C-POINTER declaration.
(FFI:TYPEOF c-place)
returns the c-type corresponding to the c-place.
(FFI:SIZEOF c-type)
(FFI:SIZEOF c-place)

The first form returns the size and alignment of the C type c-type, measured in bytes.

The second form returns the size and alignment of the C type of c-place, measured in bytes.

(FFI:BITSIZEOF c-type)
(FFI:BITSIZEOF c-place)

The first form returns the size and alignment of the C type c-type, measured in bits.

The second form returns the size and alignment of the C type of c-place, measured in bits.

(FFI:FOREIGN-ADDRESS-UNSIGNED foreign-entity)
(FFI:UNSIGNED-FOREIGN-ADDRESS number)

FFI:FOREIGN-ADDRESS-UNSIGNED returns the INTEGER address embodied in the Lisp object of type FFI:FOREIGN-ADDRESS, FFI:FOREIGN-POINTER, FFI:FOREIGN-VARIABLE or FFI:FOREIGN-FUNCTION.

FFI:UNSIGNED-FOREIGN-ADDRESS returns a FFI:FOREIGN-ADDRESS object pointing to the given INTEGER address.

(FFI:FOREIGN-ADDRESS foreign-entity)

FFI:FOREIGN-ADDRESS is both a type name and a selector/constructor function. It is the Lisp object type corresponding to a FFI:C-POINTER external type declaration, e.g. a call-out function with (:RETURN-TYPE FFI:C-POINTER) yields a Lisp object of type FFI:FOREIGN-ADDRESS.

The function extracts the object of type FFI:FOREIGN-ADDRESS living within any FFI:FOREIGN-VARIABLE or FFI:FOREIGN-FUNCTION object. If the foreign-entity already is a FFI:FOREIGN-ADDRESS, it returns it. If it is a FFI:FOREIGN-POINTER (e.g. a base foreign library address), it encapsulates it into a FFI:FOREIGN-ADDRESS object, as suitable for use with a FFI:C-POINTER external type declaration. It does not construct addresses out of NUMBERs, FFI:UNSIGNED-FOREIGN-ADDRESS must be used for that purpose.

(FFI:FOREIGN-VARIABLE foreign-entity c-type-internal &KEY name)
This constructor creates a new FFI:FOREIGN-VARIABLE from the given FFI:FOREIGN-ADDRESS or FFI:FOREIGN-VARIABLE and the internal C type descriptor (as obtained from FFI:PARSE-C-TYPE). name, a STRING, is mostly useful for documentation and interactive debugging since it appears in the printed representation of the FFI:FOREIGN-VARIABLE object, as in #<FFI:FOREIGN-VARIABLE "foo" #x0ADD4E55>. In effect, this is similar to FFI:CAST (or rather (FFI:OFFSET ... 0 ...) for places), except that it works with FFI:FOREIGN-ADDRESS objects and allows caching of the internal C types.
(FFI:FOREIGN-FUNCTION foreign-entity c-type-internal &KEY name)

This constructor creates a FFI:FOREIGN-FUNCTION from the given FFI:FOREIGN-ADDRESS or FFI:FOREIGN-FUNCTION and the internal C type descriptor (as obtained from (FFI:PARSE-C-TYPE '(FFI:C-FUNCTION ...)), in which case it is important to specify the :LANGUAGE because the expressions are likely to be evaluated at run time, outside the compilation unit). The name, a STRING, is mostly useful for documentation and interactive debugging since it appears in the printed representation of the FFI:FOREIGN-FUNCTION object, e.g., #<FFI:FOREIGN-FUNCTION "foo" #x0052B060>. It is inherited from the given FFI:FOREIGN-FUNCTION object when available.

See also Section 32.3.7, “Foreign functions”.

(FFI:VALIDP foreign-entity)
(SETF (FFI:VALIDP foreign-entity) value)

This predicate returns NIL if the foreign-entity (e.g. the Lisp equivalent of a FFI:C-POINTER) refers to a pointer which is invalid (e.g., because it comes from a previous Lisp session). It returns T if foreign-entity can be used within the current Lisp process (thus it returns T for all non-foreign arguments).

You can invalidate a foreign object using (SETF FFI:VALIDP). You cannot resurrect a zombie, nor can you kill a non-foreign object.

(FFI:FOREIGN-POINTER foreign-entity)
FFI:FOREIGN-POINTER returns the FFI:FOREIGN-POINTER associated with the Lisp object of type FFI:FOREIGN-ADDRESS, FFI:FOREIGN-POINTER, FFI:FOREIGN-VARIABLE or FFI:FOREIGN-FUNCTION.
(FFI:SET-FOREIGN-POINTER foreign-entity {foreign-entity | :COPY})
FFI:SET-FOREIGN-POINTER changes the FFI:FOREIGN-POINTER associated with the Lisp object of type FFI:FOREIGN-ADDRESS, FFI:FOREIGN-VARIABLE or FFI:FOREIGN-FUNCTION to that of the other entity. With :COPY, a fresh FFI:FOREIGN-POINTER is allocated. The original foreign-entity still points to the same object and is returned. This is particularly useful with (SETF FFI:VALIDP), see Example 32.11, “Controlling validity of resources”.
(FFI:WITH-FOREIGN-OBJECT (variable c-type [initarg]) body)
(FFI:WITH-C-VAR (variable c-type [initarg]) body)

These forms allocate space on the C execution stack, bind respectively a FFI:FOREIGN-VARIABLE object or a local SYMBOL-MACRO to variable and execute body.

When initarg is not supplied, they allocate space only for (FFI:SIZEOF c-type) bytes. This space is filled with zeroes. E.g., using a c-type of FFI:C-STRING or even (FFI:C-PTR (FFI:C-ARRAY uint8 32)) (!) both allocate space for a single pointer, initialized to NULL.

When initarg is supplied, they allocate space for an arbitrarily complex set of structures rooted in c-type. Therefore, FFI:C-ARRAY-MAX, #() and "" are your friends for creating a pointer to the empty arrays:

(with-c-var (v '(c-ptr (c-array-max uint8 32)) #())
  (setf (element (deref v) 0) 127) v)

c-type is evaluated, making creation of variable sized buffers easy:

(with-c-var (fv `(c-array uint8 ,(length my-vector)) my-vector)
  (print fv))

(FFI:FOREIGN-VALUE FFI:FOREIGN-VARIABLE)
(SETF (FFI:FOREIGN-VALUE FFI:FOREIGN-VARIABLE) ...)

This functions converts the reference to a C data structure which the FFI:FOREIGN-VARIABLE describes, to Lisp. Such a reference is typically obtained from FFI:ALLOCATE-SHALLOW, FFI:ALLOCATE-DEEP, FFI:FOREIGN-ALLOCATE or via a (FFI:C-POINTER c-type) C type description. Alternatively, macros like FFI:WITH-C-PLACE or FFI:WITH-C-VAR and the concept of foreign place hide many uses of this function.

The SETF form performs conversion from Lisp to C, following to the FFI:FOREIGN-VARIABLE's type description.

(FFI:WITH-FOREIGN-STRING (foreign-address char-count byte-count string &KEY encoding null-terminated-p start end) &BODY body)

This forms converts a Lisp string according to the encoding, allocating space on the C execution stack. encoding can be any EXT:ENCODING, e.g. CHARSET:UTF-16 or CHARSET:UTF-8, whereas CUSTOM:*FOREIGN-ENCODING* must be an ASCII-compatible encoding.

body is then executed with the three variables foreign-address, char-count and byte-count respectively bound to an untyped FFI:FOREIGN-ADDRESS (as known from the FFI:C-POINTER foreign type specification) pointing to the stack location, the number of CHARACTERs of the Lisp string that were considered and the number of (UNSIGNED-BYTE 8) bytes that were allocated for it on the C stack.

When null-terminated-p is true, which is the default, a variable number of zero bytes is appended, depending on the encoding, e.g. 2 for CHARSET:UTF-16, and accounted for in byte-count, and char-count is incremented by one.

The FFI:FOREIGN-ADDRESS object bound to foreign-address is invalidated upon the exit from the form.

A stupid example (a quite costly interface to mblen):

(with-foreign-string (fv elems bytes string
                      :encoding charset:jis... :null-terminated-p nil
                      :end 5)
 (declare (ignore fv elems))
 (format t "This string would take ~D bytes." bytes))

(FFI:PARSE-C-TYPE c-type)
(FFI:DEPARSE-C-TYPE c-type-internal)

Convert between the external (LIST) and internal (VECTOR) C type representations (used by DESCRIBE).

Note

Although you can memoize a c-type-internal (see Section 31.11.3, “Macro EXT:MEMOIZED - but do not expect type redefinitions to work across memoization!), you cannot serialize it (write to disk) because deserialization loses object identity.

(FFI:ALLOCATE-SHALLOW c-type &KEY :COUNT :READ-ONLY)
(FFI:ALLOCATE-DEEP c-type contents &KEY :COUNT :READ-ONLY)
(FFI:FOREIGN-FREE foreign-entity &KEY :FULL)
(FFI:FOREIGN-ALLOCATE c-type-internal &KEY :INITIAL-CONTENTS :COUNT :READ-ONLY)

Macro FFI:ALLOCATE-SHALLOW allocates (FFI:SIZEOF c-type) bytes on the C heap and zeroes them out (like calloc). When :COUNT is supplied, c-type is substituted with (FFI:C-ARRAY c-type count), except when c-type is CHARACTER, in which case (FFI:C-ARRAY-MAX CHARACTER count) is used instead. When :READ-ONLY is supplied, the Lisp side is prevented from modifying the memory contents. This can be used as an indication that some foreign side is going to fill this memory (e.g. via read).

Returns a FFI:FOREIGN-VARIABLE object of the actual c-type, whose address part points to the newly allocated memory.

FFI:ALLOCATE-DEEP will call C malloc as many times as necessary to build a structure on the C heap of the given c-type, initialized from the given contents.

E.g., (FFI:ALLOCATE-DEEP 'FFI:C-STRING "ABCDE") performs 2 allocations: one for a C pointer to a string, another for the contents of that string. This would be useful in conjunction with a char** C type declaration. (FFI:ALLOCATE-SHALLOW 'FFI:C-STRING) allocates room for a single pointer (probably 4 bytes).

(FFI:ALLOCATE-DEEP 'CHARACTER "ABCDEF" :count 10) allocates and initializes room for the type (FFI:C-ARRAY-MAX CHARACTER 10), corresponding to char* or, more specifically, char[10] in C.

Function FFI:FOREIGN-FREE deallocates memory at the address held by the given foreign-entity. If :FULL is supplied and the argument is of type FFI:FOREIGN-VARIABLE, recursively frees the whole complex structure pointed to by this variable.

If given a FFI:FOREIGN-FUNCTION object that corresponds to a CLISP callback, deallocates it. Callbacks are automatically created each time you pass a Lisp function via the FFI.

Use (SETF FFI:VALIDP) to disable further references to this address from Lisp. This is currently not done automatically. If the given pointer is already invalid, FFI:FOREIGN-FREE (currently) SIGNALs an ERROR. This may change to make it easier to integrate with EXT:FINALIZE.

Function FFI:FOREIGN-ALLOCATE is a lower-level interface as it requires an internal C type descriptor as returned by FFI:PARSE-C-TYPE.

(FFI:WITH-C-PLACE (variable foreign-entity) body)

Create a place out of the given FFI:FOREIGN-VARIABLE object so operations on places (e.g. FFI:CAST, FFI:DEREF, FFI:SLOT etc.) can be used within body. FFI:WITH-C-VAR appears as a composition of FFI:WITH-FOREIGN-OBJECT and FFI:WITH-C-PLACE.

Such a place can be used to access memory referenced by a foreign-entity object:

(setq foo (allocate-deep '(c-array uint8 3) rgb))
(with-c-place (place foo) (element place 0))

FFI:*OUTPUT-C-FUNCTIONS*
FFI:*OUTPUT-C-VARIABLES*
CLISP will write the extern declarations for foreign functions (defined with FFI:DEF-CALL-OUT) and foreign variables (defined with FFI:DEF-C-VAR) into the output #P".c" (when the Lisp file is compiled with COMPILE-FILE) unless these variables are NIL. They are NIL by default, so the extern declarations are not written; you are encouraged to use FFI:C-LINES to include the appropriate C headers. Set these variables to non-NIL if the headers are not available or not usable.
FFI:*FOREIGN-GUARD*

When this variable is non-NIL at compile time, CLISP will guard the C statements in the output file with cpp conditionals to take advantage of GNU autoconf feature detection. E.g.,

(EVAL-WHEN (compile) (setq *foreign-guard* t))
(FFI:DEF-CALL-OUT some-function (:name "function_name") ...)

will produce

# if defined(HAVE_FUNCTION_NAME)
  register_foreign_function((void*)&function_name,"function_name",1024);
# endif

and will compile and link on any system.

This is mostly useful for product delivery when you want your module to build on any system even if some features will not be available.

FFI:*FOREIGN-GUARD* is initialized to NIL for backwards compatibility.

FFI:FOREIGN-POINTER-INFO
This is an interface to dladdr and it returns the 4 fields of Dl_info as multiple values.

Low-level FFI forms

(FFI:MEMORY-AS foreign-address c-type-internal &OPTIONAL offset)
(SETF (FFI:MEMORY-AS foreign-address c-type-internal &OPTIONAL offset) value)

This accessor is useful when operating with untyped foreign pointers (FFI:FOREIGN-ADDRESS) as opposed to typed ones (represented by FFI:FOREIGN-VARIABLE). It allows to type and dereference the given pointer without the need to create an object of type FFI:FOREIGN-VARIABLE.

Alternatively, one could use (FFI:FOREIGN-VALUE (FFI:FOREIGN-VARIABLE foreign-entity c-type-internal)) (also SETFable).

Note that c-type-internal is the internal representation of a foreign type, thus FFI:PARSE-C-TYPE is required with literal names or types, e.g. (FFI:MEMORY-AS foreign-address (FFI:PARSE-C-TYPE '(FFI:C-ARRAY uint8 3))) or (SETF (FFI:MEMORY-AS foreign-address (FFI:PARSE-C-TYPE 'uint32)) 0).

32.3.3. (Foreign) C types

Foreign C types are used in the FFI. They are not regular Common Lisp types or CLOS classes.

A c-type is either a predefined C type or the name of a type defined by FFI:DEF-C-TYPE.

the predefined C types (c-type)

simple-c-type

the simple C types

Lisp nameLisp equivalentC equivalentILU equivalentComment
NILNILvoid as a result type only
BOOLEANBOOLEANintBOOLEAN 
CHARACTERCHARACTERcharSHORT CHARACTER 
charINTEGERsigned char  
ucharINTEGERunsigned char  
shortINTEGERshort  
ushortINTEGERunsigned short  
intINTEGERint  
uintINTEGERunsigned int  
longINTEGERlong  
ulongINTEGERunsigned long  
uint8(UNSIGNED-BYTE 8)uint8BYTE 
sint8(SIGNED-BYTE 8)sint8  
uint16(UNSIGNED-BYTE 16)uint16SHORT CARDINAL 
sint16(SIGNED-BYTE 16)sint16SHORT INTEGER 
uint32(UNSIGNED-BYTE 32)uint32CARDINAL 
sint32(SIGNED-BYTE 32)sint32INTEGER 
uint64(UNSIGNED-BYTE 64)uint64LONG CARDINALdoes not work on all platforms
sint64(SIGNED-BYTE 64)sint64LONG INTEGERdoes not work on all platforms
SINGLE-FLOATSINGLE-FLOATfloat  
DOUBLE-FLOATDOUBLE-FLOATdouble  
FFI:C-POINTER
This type corresponds to what C calls void*, an opaque pointer. When used as an argument, NIL is accepted as a FFI:C-POINTER and treated as NULL; when a function wants to return a NULL FFI:C-POINTER, it actually returns NIL.
(FFI:C-POINTER c-type)
This type is equivalent to what C calls c-type *: a pointer to a single item of the given c-type. It differs from (FFI:C-PTR-NULL c-type) (see below) in that no conversion to and from Lisp will occur (beyond the usual one of the C NULL pointer to or from Lisp NIL). Instead, an object of type FFI:FOREIGN-VARIABLE is used to represent the foreign place. It is assimilable to a typed pointer.
FFI:C-STRING
This type corresponds to what C calls char*, a zero-terminated string. Its Lisp equivalent is a string, without the trailing zero character.
(FFI:C-STRUCT class (ident1 c-type1) ... (identn c-typen))

This type is equivalent to what C calls struct { c-type1 ident1; ...; c-typen identn; }. Its Lisp equivalent is: if class is VECTOR, a SIMPLE-VECTOR; if class is LIST, a proper list; if class is a symbol naming a structure or CLOS class, an instance of this class, with slots of names ident1, ..., identn.

class may also be a CONS of a SYMBOL (as above) and a LIST of FFI:DEF-C-STRUCT options.

(FFI:C-UNION (ident1 c-type1) ... (identn c-typen))
This type is equivalent to what C calls union { c-type1 ident1; ...; c-typen identn; }. Conversion to and from Lisp assumes that a value is to be viewed as being of c-type1.
(FFI:C-ARRAY c-type dim1)
(FFI:C-ARRAY c-type (dim1 ... dimn))
This type is equivalent to what C calls c-type [dim1] ... [dimn]. Note that when an array is passed as an argument to a function in C, it is actually passed as a pointer; you therefore have to write (FFI:C-PTR (FFI:C-ARRAY ...)) for this argument's type.
(FFI:C-ARRAY-MAX c-type maxdimension)
This type is equivalent to what C calls c-type [maxdimension], an array containing up to maxdimension elements. The array is zero-terminated if it contains less than maxdimension elements. Conversion from Lisp of an array with more than maxdimension elements silently ignores the extra elements.
(FFI:C-FUNCTION (:ARGUMENTS {(argument a-c-type [PARAM-MODE [ALLOCATION]])}*) (:RETURN-TYPE r-c-type [ALLOCATION]) (:LANGUAGE language))
This type designates a C function that can be called according to the given prototype (r-c-type (*) (a-c-type1, ...)). Conversion between C functions and Lisp functions is transparent, and NULL/NIL is recognized and accepted.
(FFI:C-PTR c-type)
This type is equivalent to what C calls c-type *: a pointer to a single item of the given c-type.
(FFI:C-PTR-NULL c-type)
This type is also equivalent to what C calls c-type *: a pointer to a single item of the given c-type, with the exception that C NULL corresponds to Lisp NIL.
(FFI:C-ARRAY-PTR c-type)
This type is equivalent to what C calls c-type (*)[]: a pointer to a zero-terminated array of items of the given c-type.

The conversion of FFI:C-STRING, (FFI:C-ARRAY CHARACTER dim1), (FFI:C-ARRAY-MAX CHARACTER maxdimension), (FFI:C-ARRAY-PTR CHARACTER) is governed by CUSTOM:*FOREIGN-ENCODING* and dimensions are given in bytes. The conversion of CHARACTER, and as such of (FFI:C-PTR CHARACTER), or (FFI:C-PTR-NULL CHARACTER), as well as that of multi-dimensional arrays (FFI:C-ARRAY CHARACTER (dim1 ... dimn)), are governed by CUSTOM:*FOREIGN-ENCODING* if the latter is a 1:1 encoding, or by the ASCII encoding otherwise.

Note

Remember that the C type char is a numeric type and does not use CHARACTER EXT:ENCODINGs.

32.3.4. The choice of the C flavor

FFI:C-FUNCTION, FFI:DEF-CALL-IN, FFI:DEF-CALL-OUT take a :LANGUAGE argument. The language is either :C (denotes K&R C) or :STDC (denotes ANSI C) or :STDC-STDCALL (denotes ANSI C with the stdcall calling convention). It specifies whether the C function (caller or callee) has been compiled by a K&R C compiler or by an ANSI C compiler, and possibly the calling convention.

The default language is set using the macro FFI:DEFAULT-FOREIGN-LANGUAGE . If this macro has not been called in the current compilation unit (usually a file), a warning is issued and :STDC is used for the rest of the unit.

32.3.5. Foreign variables

Foreign variables are variables whose storage is allocated in the foreign language module. They can nevertheless be evaluated and modified through SETQ, just as normal variables can, except that the range of allowed values is limited according to the variable's foreign type.

Equality of foreign values

For a foreign variable x the form (EQL x x) is not necessarily true, since every time x is evaluated its foreign value is converted to a fresh Lisp value. Ergo, (SETF (AREF x n) y) modifies this fresh Lisp value (immediately discarded), not the foreign data. Use FFI:ELEMENT et al instead, see Section 32.3.6, “Operations on foreign places”.

Foreign variables are defined using FFI:DEF-C-VAR and FFI:WITH-C-VAR.

32.3.6. Operations on foreign places

A FFI:FOREIGN-VARIABLE name defined by FFI:DEF-C-VAR, FFI:WITH-C-VAR or FFI:WITH-C-PLACE defines a place, i.e., a form which can also be used as argument to SETF. (An lvalue in C terminology.) The following operations are available on foreign places:

FFI:ELEMENTFFI:C-VAR-ADDRESS
FFI:DEREFFFI:C-VAR-OBJECT
FFI:SLOTFFI:TYPEOF
FFI:CASTFFI:SIZEOF
FFI:OFFSETFFI:BITSIZEOF

32.3.7. Foreign functions

Foreign functions are functions which are defined in the foreign language. There are named foreign functions (imported via FFI:DEF-CALL-OUT or created via FFI:DEF-CALL-IN) and anonymous foreign functions; they arise through conversion of function pointers using FFI:FOREIGN-FUNCTION.

A call-out function is a foreign function called from Lisp: control flow temporarily leaves Lisp. A call-in function (AKA callback) is a Lisp function called from the foreign language: control flow temporary enters Lisp.

The following operators define foreign functions:

FFI:DEF-CALL-IN
FFI:DEF-CALL-OUT
FFI:FOREIGN-FUNCTION

32.3.7.1. Callbacks and memory management

Callbacks (C function calling Lisp function) create so-called trampolines. A trampoline is a piece of C code which knows how to call a particular Lisp function. (That is how all foreign language interfaces work, not just ours). The C pointer that the foreign library receives is the pointer to this piece of code. These are not subject to garbage-collection, as there is no protocol to tell the garbage collector when a given callback is not needed anymore (unlike with Lisp objects).

With callbacks to named functions (i.e., created by a FFI:DEF-CALL-IN form where function is a function name) this is mostly harmless, since function is unlikely to be redefined.

With callbacks to anonymous functions (i.e., created by FFI:FOREIGN-FUNCTION or a FFI:DEF-CALL-IN form where function argument is a lambda expression), this might become an issue when they are produced dynamically and en masse, e.g. inside a loop, so that many trampolines are generated.

You can use FFI:FOREIGN-FREE to free the trampoline associated with a FFI:FOREIGN-FUNCTION object, but when you pass a lambda expression to a FFI:DEF-CALL-OUT as an argument of type FFI:C-FUNCTION, such a trampoline is allocated, but you do not get hold of the associated trampoline object, and thus you cannot (trivially) free it. Thus you may find it easier to create the FFI:FOREIGN-FUNCTION object first, pass it to the FFI:DEF-CALL-OUT, and then call FFI:FOREIGN-FREE manually.

32.3.8. Argument and result passing conventions

When passed to and from functions, allocation of arguments and results is handled as follows:

Values of SIMPLE-C-TYPE, FFI:C-POINTER are passed on the stack, with dynamic extent. The ALLOCATION is effectively ignored.

Values of type FFI:C-STRING, FFI:C-PTR, FFI:C-PTR-NULL, FFI:C-ARRAY-PTR need storage. The ALLOCATION specifies the allocation policy:

:NONE
no storage is allocated.
:ALLOCA
allocation of storage on the stack, which has dynamic extent.
:MALLOC-FREE
storage will be allocated via malloc and released via free.

If no ALLOCATION is specified, the default ALLOCATION is :NONE for most types, but :ALLOCA for FFI:C-STRING and FFI:C-PTR and FFI:C-PTR-NULL and FFI:C-ARRAY-PTR and for :OUT arguments. The :MALLOC-FREE policy provides the ability to pass arbitrarily nested structures within a single conversion.

Call-out function arguments: 

For arguments passed from Lisp to C:
:MALLOC-FREE
Lisp allocates the storage using malloc and never deallocates it. The C function is supposed to call free when done with it.
:ALLOCA
Lisp allocates the storage on the stack, with dynamic extent. It is freed when the C function returns.
:NONE

Lisp assumes that the pointer already points to a valid area of the proper size and puts the result value there.

This is dangerous and deprecated.

For results passed from C to Lisp:
:MALLOC-FREE
Lisp calls free on it when done.
:NONE
Lisp does nothing.

Call-in function arguments: 

For arguments passed from C to Lisp:
:MALLOC-FREE
Lisp calls free on it when done.
:ALLOCA
:NONE
Lisp does nothing.
For results passed from Lisp to C:
:MALLOC-FREE
Lisp allocates the storage using malloc and never deallocates it. The C function is supposed to call free when done with it.
:NONE

Lisp assumes that the pointer already points to a valid area of the proper size and puts the result value there.

This is dangerous and deprecated.

Warning

Passing FFI:C-STRUCT, FFI:C-UNION, FFI:C-ARRAY, FFI:C-ARRAY-MAX values as arguments (not via pointers) is only possible to the extent the C compiler supports it. Most C compilers do it right, but some C compilers (such as gcc on hppa, x86_64 and Win32) have problems with this. The recommended workaround is to pass pointers; this is fully supported. See also clisp-devel (SFmail/200307141526.26925.bruno%40clisp.org/Gmane/devel/10089).

32.3.9. Parameter Mode

A function parameter's PARAM-MODE may be

:IN (means: read-only):
The caller passes information to the callee.
:OUT (means: write-only):
The callee passes information back to the caller on return. When viewed as a Lisp function, there is no Lisp argument corresponding to this, instead it means an additional return value. Requires ALLOCATION = :ALLOCA.
:IN-OUT (means: read-write):
Information is passed from the caller to the callee and then back to the caller. When viewed as a Lisp function, the :OUT value is returned as an additional return value.

The default is :IN.

32.3.10. Examples

Example 32.2. Simple declarations and access

The C declaration

struct foo {
  int a;
  struct foo * b[100];
};

corresponds to

(FFI:DEF-C-STRUCT foo
  (a int)
  (b (c-array (c-ptr foo) 100)))

The element access

struct foo f;
f.b[7].a

corresponds to

(declare (type foo f))
(foo-a (aref (foo-b f) 7)) or
(slot-value (aref (slot-value f 'b) 7) 'a)

Example 32.3. External C variable and some accesses

struct bar {
  short x, y;
  char a, b;
  int z;
  struct bar * n;
};

extern struct bar * my_struct;

my_struct->x++;
my_struct->a = 5;
my_struct = my_struct->n;

corresponds to

(FFI:DEF-C-STRUCT bar
  (x short)
  (y short)
  (a char)
  (b char) or (b character) if it represents a character, not a number
  (z int)
  (n (c-ptr bar)))

(FFI:DEF-C-VAR my_struct (:type (c-ptr bar)))

(setq my_struct (let ((s my_struct)) (incf (slot-value s 'x)) s)) or
(incf (slot my_struct 'x))
(setq my_struct (let ((s my_struct)) (setf (slot-value s 'a) 5) s)) or
(setf (slot my_struct 'a) 5)
(setq my_struct (slot-value my_struct 'n)) or
(setq my_struct (deref (slot my_struct 'n)))

Example 32.4. Calling an external function

On ANSI C systems, <stdlib.h> contains the declarations:

typedef struct {
  int quot;   /* Quotient */
  int rem;    /* Remainder */
} div_t;
extern div_t div (int numer, int denom);

This translates to

(FFI:DEF-C-STRUCT (div_t :typedef)
  (quot int)
  (rem int))
(FFI:DEFAULT-FOREIGN-LANGUAGE :stdc)
(FFI:DEF-CALL-OUT div (:ARGUMENTS (numer int) (denom int))
  (:RETURN-TYPE div_t))

Sample call from within Lisp (after running clisp-link):

(div 20 3)
⇒ #S(DIV_T :QUOT 6 :REM 2)

Example 32.5. Another example for calling an external function

Suppose the following is defined in a file cfun.c:

struct cfunr { int x; char *s; };
struct cfunr * cfun (int i,char *s,struct cfunr * r,int a[10]) {
  int j;
  struct cfunr * r2;
  printf("i = %d\n", i);
  printf("s = %s\n", s);
  printf("r->x = %d\n", r->x);
  printf("r->s = %s\n", r->s);
  for (j = 0; j < 10; j++) printf("a[%d] = %d.\n", j, a[j]);
  r2 = (struct cfunr *) malloc (sizeof (struct cfunr));
  r2->x = i+5;
  r2->s = "A C string";
  return r2;
}

It is possible to call this function from Lisp using the file callcfun.lisp (do not call it cfun.lisp - COMPILE-FILE will overwrite cfun.c) whose contents is:

(DEFPACKAGE "TEST-C-CALL" (:use COMMON-LISP FFI))
(IN-PACKAGE "TEST-C-CALL")
(EVAL-WHEN (compile) (setq FFI:*OUTPUT-C-FUNCTIONS* t))
(FFI:DEF-C-STRUCT cfunr (x int) (s c-string))
(FFI:DEFAULT-FOREIGN-LANGUAGE :stdc)
(FFI:DEF-CALL-OUT cfun (:RETURN-TYPE (c-ptr cfunr))
  (:ARGUMENTS (i int)
              (s c-string)
              (r (c-ptr cfunr) :in :alloca)
              (a (c-ptr (c-array int 10)) :in :alloca)))
(defun call-cfun ()
  (cfun 5 "A Lisp string" (make-cfunr :x 10 :s "Another Lisp string")
        '#(0 1 2 3 4 5 6 7 8 9)))

Use the module facility:

$ clisp-link create cfun callcfun.c
$ cc -O -c cfun.c
$ cd cfun
$ ln -s ../cfun.o cfun.o
Add cfun.o to NEW_LIBS and NEW_FILES in link.sh.
$ cd ..
$ base/lisp.run -M base/lispinit.mem -c callcfun.lisp
$ clisp-link add base base+cfun cfun
$ base+cfun/lisp.run -M base+cfun/lispinit.mem -i callcfun
> (test-c-call::call-cfun)
i = 5
s = A Lisp string
r->x = 10
r->s = Another Lisp string
a[0] = 0.
a[1] = 1.
a[2] = 2.
a[3] = 3.
a[4] = 4.
a[5] = 5.
a[6] = 6.
a[7] = 7.
a[8] = 8.
a[9] = 9.
#S(TEST-C-CALL::CFUNR :X 10 :S "A C string")
>
$ rm -r base+cfun

Note that there is a memory leak here: The return value r2 of cfun() is malloced but never freed. Specifying

(:RETURN-TYPE (c-ptr cfunr) :malloc-free)

is not an alternative because this would also free(r2->x) but r2->x is a pointer to static data.

The memory leak can be avoided using

(:RETURN-TYPE (c-pointer cfunr))

instead, in conjunction with

(defun call-cfun ()
  (let ((data (cfun ...)))
    (UNWIND-PROTECT (FFI:FOREIGN-VALUE data)
      (FFI:FOREIGN-FREE data :FULL nil))))

Example 32.6. Accessing cpp macros

Suppose you are interfacing to a library mylib.so which defines types, macros and inline functions in mylib.h:

#define FOO(x)  .....
#define BAR ...
struct zot { ... }
inline int bar (int x) { ... }

To make them available from CLISP, write these forms into the lisp file my.lisp:

(FFI:C-LINES "#include <mylib.h>
int my_foo (int x) { return FOO(x); }
int my_bar (int x) { return bar(x); }~%")
(FFI:DEF-C-CONST bar)
(FFI:DEF-C-CONST zot-size (:name "sizeof(struct zot)") (:guard nil))
(FFI:DEF-CALL-OUT my-foo (:NAME "my_foo") (:ARGUMENTS (x ffi:int)) (:RETURN-TYPE ffi:int))
(FFI:DEF-CALL-OUT my-bar (:NAME "my_bar") (:ARGUMENTS (x ffi:int)) (:RETURN-TYPE ffi:int))

Compiling this file will produce my.c and my.fas and you have two options:

  1. Compile my.c into my.o with

    $ gcc -c my.c -lmylib

    and use clisp-link to create a new CLISP linking set.

  2. Add (:LIBRARY "my.dll") to the FFI:DEF-CALL-OUT forms, compile my.c into my.so (or my.dll on Win32) with

    $ gcc -shared -o my.so my.c -lmylib

    and load my.fas.

Of course, you could have created my1.c containing

#include <mylib.h>
int my_foo (int x) { return FOO(x); }
int my_bar (int x) { return bar(x); }

manually, but FFI:C-LINES allows you to keep the definitions of my_foo and my-foo close together for easier maintenance.


Example 32.7. Calling Lisp from C

To sort an array of double-floats using the Lisp function SORT instead of the C library function qsort, one can use the following interface code sort1.c. The main problem is to pass a variable-sized array.

extern void lispsort_begin (int);
void* lispsort_function;
void lispsort_double (int n, double * array) {
  double * sorted_array;
  int i;
  lispsort_begin(n); /* store #'sort2 in lispsort_function */
  sorted_array = ((double * (*) (double *)) lispsort_function) (array);
  for (i = 0; i < n; i++) array[i] = sorted_array[i];
  free(sorted_array);
}

This is accompanied by sort2.lisp:

(DEFPACKAGE "FFI-TEST" (:use COMMON-LISP FFI))
(IN-PACKAGE "FFI-TEST")
(EVAL-WHEN (compile) (SETQ FFI:*OUTPUT-C-FUNCTIONS* t))
(FFI:DEF-CALL-IN lispsort_begin (:ARGUMENTS (n int))
  (:RETURN-TYPE nil)
  (:LANGUAGE :stdc))
(FFI:DEF-C-VAR lispsort_function (:type c-pointer))
(DEFUN lispsort_begin (n)
  (SETF (FFI:CAST lispsort_function
              `(FFI:C-FUNCTION
                 (:ARGUMENTS (v (c-ptr (c-array double-float ,n))))
                 (:RETURN-TYPE (c-ptr (c-array double-float ,n))
                               :malloc-free)))
        #'sort2))
(DEFUN sort2 (v)
  (declare (type vector v))
  (sort v #'<))

To test this, use the following test file sorttest.lisp:

(EVAL-WHEN (compile) (setq FFI:*OUTPUT-C-FUNCTIONS* t))
(FFI:DEF-CALL-OUT sort10
  (:NAME "lispsort_double")
  (:LANGUAGE :stdc)
  (:ARGUMENTS (n int)
              (array (c-ptr (c-array double-float 10)) :in-out)))

Now try

$ clisp-link create sort sort2.c sorttest.c
$ cc -O -c sort1.c
$ cd sort
$ ln -s ../sort1.o sort1.o

Add sort1.o to NEW_LIBS and NEW_FILES in link.sh. Create a file package.lisp containing the form

(MAKE-PACKAGE "FFI-TEST" :use '(COMMON-LISP FFI))

and add package.lisp to TO_PRELOAD in link.sh. Proceed:

$ cd ..
$ base/lisp.run -M base/lispinit.mem -c sort2.lisp sorttest.lisp
$ clisp-link add base base+sort sort
$ base+sort/lisp.run -M base+sort/lispinit.mem -i sort2 sorttest
> (sort10 10 '#(0.501d0 0.528d0 0.615d0 0.550d0 0.711d0
                0.523d0 0.585d0 0.670d0 0.271d0 0.063d0))
#(0.063d0 0.271d0 0.501d0 0.523d0 0.528d0 0.55d0 0.585d0 0.615d0 0.67d0 0.711d0)
$ rm -r base+sort

Example 32.8. Calling Lisp from C dynamically

Create a dynamic library lispdll (#P".dll" on Win32, #P".so" on UNIX) with the following function:

typedef int (*LispFunc)(int parameter);
int CallInFunc(LispFunc f) {
  return f(5)+11;
}

and call it from Lisp:

(ffi:def-call-out callout
  (:NAME "CallInFunc")
  (:LIBRARY "lispdll.dll")
  (:ARGUMENTS (function-arg
               (FFI:C-FUNCTION (:ARGUMENTS (number ffi:int))
                               (:RETURN-TYPE ffi:int) (:LANGUAGE :stdc))))
  (:RETURN-TYPE ffi:int)
  (:LANGUAGE :stdc))
(defun f (x) (* x 2))
⇒ F
(callout #'f)
⇒ 21

Example 32.9. Variable size arguments: calling gethostname from CLISP

The standard UNIX function

int fsfuncgethostname(name,  
 length); 
char* name;
size_t length;
 

follows a typical pattern of C out-parameter convention: it expects a pointer to a buffer it is going to fill. So you must view this parameter as either :OUT or :IN-OUT. Additionally, one must tell the function the size of the buffer. Here length is just an :IN parameter. Sometimes this will be an :IN-OUT parameter, returning the number of bytes actually filled in.

So name is actually a pointer to an array of up to length characters, regardless of what the poor char* C prototype says, to be used like a C string (NULL-termination). UNIX specifies that host names are limited to HOST_NAME_MAX bytes, which is, of course, system dependent, but it appears that 256 is sufficient.

In the present example, you can use allocation :ALLOCA, like you would do in C: stack-allocate a temporary:

(FFI:DEF-CALL-OUT gethostname
  (:ARGUMENTS (name (FFI:C-PTR (FFI:C-ARRAY-MAX ffi:char 256))
                    :OUT :ALLOCA)
              (length ffi:int))
  (:LANGUAGE :stdc) (:LIBRARY :default)
  (:RETURN-TYPE ffi:int))
⇒ GETHOSTNAME
(defun myhostname ()
  (multiple-value-bind (success name):OUT and :IN-OUT parameters are returned as multiple values
      (gethostname 256)
    (if (zerop success) name
      (error "~S: ~S: ~S" 'myhostname (os:errno) (os:strerror)))))See Section 33.2.15, “Error handling”
⇒ MYHOSTNAME
(myhostname)
⇒ #(97 98 97 122 111 110 107)

It is a SIMPLE-VECTOR, not a STRING, because the name argument is an array of char (an INTEGER type, see Section 32.3.3, “(Foreign) C types”), not character.

(FFI:DEF-CALL-OUT gethostname
  (:ARGUMENTS (name (FFI:C-PTR (FFI:C-ARRAY-MAX character 256))
                    :OUT :ALLOCA)
              (length ffi:int))
  (:LANGUAGE :stdc) (:LIBRARY :default)
  (:RETURN-TYPE ffi:int))
⇒ GETHOSTNAME
(myhostname)
⇒ "abazonk"

Now we have a different problem: if gethostname fails, then the buffer allocated for name will be filled with garbage, but it will still go through the string conversion before we can check the success status. If CUSTOM:*FOREIGN-ENCODING* is CHARSET:ISO-8859-1, this is not a problem since no real conversion is happening, but with CHARSET:UTF-8 an ERROR may be SIGNALed. A safe approach is to pass to the foreign function our own stack-allocated buffer, and only convert the buffer to a string when the foreign function succeeds:

(FFI:DEF-CALL-OUT gethostname
  (:ARGUMENTS (name FFI:C-POINTER)
              (length ffi:int))
  (:LANGUAGE :stdc) (:LIBRARY :default)
  (:RETURN-TYPE ffi:int))
⇒ GETHOSTNAME
(defun myhostname ()
  (FFI:WITH-FOREIGN-OBJECT (name '(FFI:C-ARRAY-MAX character 256))
    (let ((success (gethostname name 256)))
      (if (zerop success) (FFI:FOREIGN-VALUE name)
        (error "~S: ~S: ~S" 'myhostname (os:errno) (os:strerror))))))
⇒ MYHOSTNAME
(myhostname)
⇒ "abazonk"

Note that the type argument of FFI:WITH-FOREIGN-OBJECT is evaluated, so we do not have to make any assumptions about HOST_NAME_MAX:

(defun myhostname ()
  (let ((host-name-max (os:sysconf :host-name-max)))
    (FFI:WITH-FOREIGN-OBJECT (name `(FFI:C-ARRAY-MAX character ,host-name-max))
      (let ((success (gethostname name host-name-max)))
        (if (zerop success) (FFI:FOREIGN-VALUE name)
          (error "~S: ~S: ~S" 'myhostname (os:errno) (os:strerror)))))))
⇒ MYHOSTNAME
(myhostname)
⇒ "abazonk"


Example 32.10. Accessing variables in shared libraries

Suppose one wants to access and modify variables that reside in shared libraries:

struct bar {
  double x, y;
  double out;
};

struct bar my_struct = {10.0, 20.5, 0.0};

double test_dll(struct bar *ptr)
{
  return ptr->out = ptr->out + ptr->x + ptr->y;
}

This is compiled to libtest.so (or libtest.dll, depending on your platform).

Use the following lisp code:

(USE-PACKAGE FFI)

(FFI:DEF-C-STRUCT bar
  (x double-float)
  (y double-float)
  (out double-float))

(FFI:DEF-CALL-OUT get-own-c-float
  (:LIBRARY "libtest.so")
  (:LANGUAGE :stdc)
  (:NAME "test_dll")
  (:ARGUMENTS (ptr c-pointer :in :alloca))
  (:RETURN-TYPE double-float))

(FFI:DEF-C-VAR my-c-var (:name "my_struct")
  (:LIBRARY "libtest.so") (:type (c-ptr bar)))

Note that get-own-c-float takes a FFI:C-POINTER, not a (FFI:C-PTR bar) as the argument.

Now you can access call get-own-c-float on my-c-var:

(FFI:C-VAR-ADDRESS my-c-var)
⇒ #<FOREIGN-ADDRESS #x282935D8>
(get-own-c-float (FFI:C-VAR-ADDRESS my-c-var))
⇒ 30.5d0
(get-own-c-float (FFI:C-VAR-ADDRESS my-c-var))
⇒ 61.0d0
(get-own-c-float (FFI:C-VAR-ADDRESS my-c-var))
⇒ 91.5d0
(get-own-c-float (FFI:C-VAR-ADDRESS my-c-var))
⇒ 122.0d0

Example 32.11. Controlling validity of resources

FFI:SET-FOREIGN-POINTER is useful in conjunction with (SETF FFI:VALIDP) to limit the extent of external resources. Closing twice can be avoided by checking FFI:VALIDP. All pointers depending on this resource can be disabled at once upon close by sharing their FFI:FOREIGN-POINTER using FFI:SET-FOREIGN-POINTER.

(FFI:DEF-C-TYPE PGconn c-pointer)opaque pointer
(FFI:DEF-CALL-OUT PQconnectdb (:RETURN-TYPE PGconn)
  (:ARGUMENTS (conninfo c-string)))
(defun sql-connect (conninfo)
  (let ((conn (PQconnectdb conninfo)))
    (unless conn (error "NULL pointer"))
    ;; may wish to use EXT:FINALIZE as well
    (FFI:SET-FOREIGN-POINTER conn :COPY)))
(defun sql-dependent-resource (conn arg1)
  (FFI:SET-FOREIGN-POINTER (PQxxx conn arg1) conn))
(defun sql-close (connection)
  (when (FFI:VALIDP connection)
    (PQfinish connection)
    (setf (FFI:VALIDP connection) nil)
    T))

Warning

Sharing FFI:FOREIGN-POINTER goes both ways: invalidating the dependent resource will invalidate the primary one.

Note

An alternative approach to resource management, more suitable to non-FFI modules, is implemented in the berkeley-db module, see Section 33.7.2, “Closing handles”.


Example 32.12. Floating point arrays

Save this code into sum.c:

double sum (int len, double *vec) {
  int i;
  double s=0;
  for (i=0; i<len; i++) s+= vec[i];
  return s;
}

and compile it with

$ gcc -shared -o libsum.so sum.c

Now you can sum doubles:

(FFI:DEF-CALL-OUT sum (:NAME "sum") (:LIBRARY "libsum.so") (:LANGUAGE :stdc)
  (:RETURN-TYPE ffi:double-float)
  (:ARGUMENTS (len ffi:int) (vec (FFI:C-ARRAY-PTR ffi:double-float))))
(sum 3 #(1d0 2d0 3d0))
⇒ 6d0

32.3.10.1. More examples

You can find more information and examples of the CLISP FFI in the following clisp-list messages:

Even more examples can be found in the file tests/ffi.tst in the CLISP source distribution.


These notes document CLISP version 2.49+Last modified: 2012-05-04