(rontolisp) docs

Compile to a Native Executable

--native with -o compiles a program to ONE self-contained executable: the WASM output, precompiled to machine code by wasmtime and appended to a small runner. Running it needs neither wasmtime nor a JVM:

echo '(print (+ 1 2))' > hello.lisp
rontolisp hello.lisp --native -o hello
./hello
3

The executable is the WASI command module -o hello.wasm writes, run the way wasmtime run runs it: the same standard output, and the same exit status -- the program's own (uiop:quit n) code, 0 when it returns, 134 after an uncaught error. Its arguments are the program's (uiop:command-line-arguments) and it sees the process environment and standard input. Nothing else is written: the .wasm and its precompiled form stay in memory.

Files

The current directory and / are open to the program, so relative paths (../x included) and absolute paths work as they do in any native program.

HTTP

rontolisp:fetch works in an executable as it does on the interpreter and the JVM: the request is running when fetch returns, several requests overlap, await answers (:status :headers :body) with :body a stream of the reply's octets, and a request that fails (a refused connection, a certificate no trusted root vouches for) signals at the await. The runner makes the requests itself, over HTTP/1.1 and one connection each, and like the JDK's client it neither follows redirects nor decompresses.

HTTPS trusts Mozilla's root certificates, which are built into the executable, so it needs no certificate bundle on the machine. SSL_CERT_FILE names a PEM file of certificates to trust instead, for a private or TLS-inspecting certificate authority:

SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt ./fetcher

A program that fetches starts with a larger runner, the one that carries HTTP and TLS; every other program keeps the small one (Size and Speed).

Serving HTTP

A Clack application started with (clack:clackup app :server :rontolisp), a Ring handler started with run-server, or a handler served by rontolisp:http-handler serves from the executable: the runner binds :port and :address, accepts the connections and speaks HTTP/1.1, and the program answers the requests.

rontolisp app.lisp --native -o app
./app        # serves until stopped (Ctrl-C)

The runner reads every connection on a thread of its own (a request's head and its whole body, Content-Length or chunked) and keeps connections alive as HTTP/1.1 says, closing one that stays idle for a minute. The application answers one request at a time, on the program's one thread, and its response goes out whole, with a Content-Length. So clackup, run-server and http-handler never return: :use-thread and :join? false have no second thread to hand the program back on, and the server stops with the process. The request is the one the interpreter and the JVM build, the peer's address and port included; the files a handler serves are read from the file system as in any program (Files).

TCP Sockets

The rontolisp:tcp-* functions and the usocket shim work in an executable as they do on the interpreter and the JVM: the runner opens the sockets, host may be a name or an address literal, every call blocks until it is done, writes are buffered until the program waits on a socket, flushes, closes or ends, tcp-set-timeout sets a real read deadline, and a failure signals an error with the same text (tcp-connect: cannot connect to 127.0.0.1:1: Connection refused), which the usocket shim re-signals as usocket:socket-error. So a program that talks to PostgreSQL (cl-postgres) or Redis over the shim runs from an executable. Like the component, the executable reads a socket one chunk at a time; listen answers from the chunk and then from the kernel's receive buffer, as on the interpreter, so usocket:wait-for-input waits for data or its :timeout. The TLS functions stay interpreter, JVM and, for the client half, component only: in an executable tls-connect and tls-upgrade signal an error when the call runs (so a client whose https branch is never taken still builds), and tls-listen is refused when the program is compiled.

"hello"

A program that opens sockets starts with the same larger runner a fetching one does (Size and Speed).

Flags

Every flag of the default WASM output applies (--simd, --optimize, --dynamic, ...). The flags that ask for a different kind of module are refused by name: --component, --no-wasi, --no-gc, --host-random, --host-fetch, --host-boundary, --reentrant and --emit-js-glue. So is an -o name ending in .wasm, .class, .jar or .war. A program that fetches, serves or opens sockets needs no flag: the runner answers fetch, the server and the sockets itself (HTTP, Serving HTTP, TCP Sockets).

Where It Runs

By default the executable is built for the platform the compiler runs on (Linux or macOS, x86_64 or aarch64) and uses only the CPU features every processor of that platform has, so it runs on any machine of the platform however old. On Linux it is statically linked, so it runs on any distribution of that architecture, whatever its C library. The released binaries and the executable JAR carry the precompiler; a build of rontolisp that carries none for the host says --native is not available for <os>-<arch> (building it from source: Build & Install).

--native-target builds for another platform: linux-x86_64, linux-aarch64 or macos-aarch64. The released binaries and the executable JAR carry the runner of each; a build from source carries its own platform's only, and names what it carries when asked for another.

rontolisp hello.lisp --native --native-target linux-aarch64 -o hello-arm64

--native-cpu chooses the CPU features the machine code may use: baseline (the default), host (every feature of the compiling machine's CPU), or on x86_64 a level, x86-64-v2, x86-64-v3 or x86-64-v4. An executable started on a CPU that lacks one of its features refuses to run and names the feature. Measured on the benchmark programs, the newer features did not make the code rontolisp generates faster, so the default costs nothing in practice.

The precompiler is a shared library that rontolisp extracts once to $XDG_CACHE_HOME/rontolisp (else ~/.cache/rontolisp, or ~/Library/Caches/rontolisp on macOS); the system property rontolisp.native.cache moves it (-Drontolisp.native.cache=DIR).

macOS GUI Programs

On Apple silicon (macos-aarch64) an executable also runs programs that use the objc, appkit, metal and scene packages: its runner calls the Objective-C runtime itself, and the program runs on the process's first thread, where AppKit wants it. See macOS GUI. For every other target such a program is a compile error naming the reference.

Licenses

The runner links wasmtime and the Rust crates it builds on, whose licences ask for their notices to go with the executable: rontolisp notices --native -o NOTICES.txt writes them with rontolisp's own (Licenses of a Compiled Program).

Size and Speed

An executable is a runner (2.1 MB on Linux x86_64, 1.9 MB on Linux aarch64, 1.7 MB on macOS) plus roughly 11 times the .wasm: 2.1 MB for hello on Linux x86_64, 2.3 MB for a 28 KB module. The runner of a program that fetches, serves HTTP or opens TCP sockets is 3.3 MB on Linux x86_64 and 2.9 MB on Linux aarch64. It starts in about 10 ms and runs at about the speed of wasmtime run on the same module.