Support generating API specifications automatically using sb-introspect

[kartik-s]

Problem

Writing the API specification for a library can be tedious and potentially error-prone, especially when exporting a large amount of functions. This decreases the velocity at which we can deliver new and existing Lisp projects to downstream alien language users.

Solution

SBCL's sb-introspect extension contains two functions that we can leverage to synthesize API export specifications directly from Lisp function objects:

• function-lambda-list, which we can use to get the arity and argument names for a function.
• function-type, which we can use to get the compiler-inferred type signature for the function.

Inferring types

A clear issue with using compiler-inferred type information for writing function export specifications is that in many cases the compiler cannot infer a more specific type than t for arguments or return values. This does not preclude the generation of bindings, since the t type can just be mapped to a generic handle type in the function export spec.

Nevertheless, better bindings can be generated if the compiler can infer more specific types since we can replace generic handles with more specific foreign types such as :int or :string. This can be achieved by declaring types more extensively throughout Lisp code, which has the added benefits of improving documentation and potentially improving performance.

Possible interface

One possible way this could look is like this:

(defmacro define-api-from-package (name (&key package ...))
  ...)

The macro can iterate through the external symbols of the provided package using do-external-symbols and produce an export specification for each symbol bound to a function using sb-introspect as described before.

[kartik-s]

One big disadvantage with this way of generating bindings as opposed to writing them out by hand is that we lose the ability to version control API export specifications.

This can be mitigated by dumping the generated specification to a file that gets committed to the VCS.