--- name: rontolisp version: 0.1.620 description: >- Write, run and debug rontolisp programs -- the Common Lisp subset that runs identically on an interpreter, the JVM and two WebAssembly backends. Use this whenever you touch a .lisp or .asd file in a rontolisp project, whenever the user says rontolisp, and whenever you are about to write Common Lisp that will run under rontolisp -- even if the request never uses the word "skill" or names a backend. Your Common Lisp knowledge is a PRIOR here, not the truth: this skill carries the delta -- exactly which operators exist, which CL features are missing or partial, the rontolisp-only extensions (fetch, async/await, linalg/vec, java:, wit-import, wasm-export, ql:quickload), and how to actually run a program on each backend. --- # rontolisp rontolisp is a Common Lisp subset with four backends that must agree: the interpreter, a JVM bytecode compiler (`-o Prog.class`), and two WebAssembly compilers (wasm-GC by default, `--no-gc` for a linear-memory module). Programs are ordinary `.lisp` files and systems are ordinary `.asd` files, so a real Common Lisp library often loads verbatim -- and so does a real Common Lisp mistake. This bundle is generated from the rontolisp documentation (668 functions, 105 macros, 29 special forms); every file under `references/` is the same text published at https://making.github.io/rontolisp/docs/en/. Skill version 0.1.620. ## How to work in this language Your Common Lisp knowledge is right about most of the core and wrong in specific, recurring places. It fails in exactly two ways, and both are cheap to check before you write the code rather than after the error: 1. **Reaching for an operator that is not there.** rontolisp ships a fixed set. Before using an operator you have not already seen working in this project, look it up in `references/operators.md` -- every name it has, by category. Not listed means not there: reshape the code, or write the helper yourself. When you can run code, `(rontolisp:list-functions)`, `(rontolisp:list-macros)` and `(rontolisp:list-special-forms)` answer the same question at runtime. 2. **Assuming full CL semantics behind a name that IS there.** `loop`, `format`, `defpackage`, CLOS, `values`, string mutation and non-local exits all exist and all stop short of the standard somewhere. The whole delta is inlined below -- read it before you lean on any of them. Then run the program. An implementation with four backends is not a place to reason about output; a program that has not been executed at least on the interpreter is not finished. If it is meant to be compiled, run it on the backend it targets, because that is where the compiled-path restrictions bite (`progv`, `java:`, a `return-from` crossing `flet`, redefining a CL function). When something fails, prefer narrowing to the smallest form that still fails and running it in the REPL over rereading the code -- the error messages name the operator and the position. ## Running a program ```bash rontolisp prog.lisp # interpret rontolisp prog.lisp -o Prog.class && java Prog # JVM (class name = file stem, no directories) rontolisp prog.lisp -o prog.wasm && wasmtime run -W gc prog.wasm rontolisp prog.lisp -o prog.wasm --component && wasmtime run -W gc=y prog.wasm # WASI 0.3 rontolisp # REPL rontolisp -e '(print (+ 1 2))' # one form rontolisp format prog.lisp # re-indent in place (whitespace only) rontolisp test tests/main.lisp # run a rove suite; exit 0 passed, 1 failed ``` `rontolisp` is the native binary; `references/getting-started/build.md` says how to get one, and `java -jar target/rontolisp-*-exec.jar` stands in for it inside a build tree. The `-o` extension picks the backend. Add `-W exceptions=y` to both wasmtime commands when the program uses `handler-case` / `ignore-errors` / `unwind-protect` / `catch` / `throw`, any async form, or a `return-from` or `go` that crosses a `lambda`; a fetch component also needs `-S http=y`. Other flags that change the emitted artifact -- `--dynamic`, `--optimize=off`/`=size`, `--no-gc`, `--simd`, `--no-prune`, `--emit-wit` -- are described under `references/compiling/`. ## Guides What Common Lisp knowledge cannot supply: the surfaces rontolisp adds (HTTP, async, sockets, `java:`, the numeric kernels, WASM contracts, systems and libraries), and the places where the subset spells its own behavior out in full. Read the guide before using one of these. | Topic | Guide | | --- | --- | | Vectors & Matrices (linalg) | `references/guides/linear-algebra.md` | | Vector Kernels & SIMD (vec, linalg) | `references/guides/simd-acceleration.md` | | Neural Networks (torch) | `references/guides/neural-networks.md` | | Java Interop | `references/guides/java-interop.md` | | Asynchronous Programming (async / await) | `references/guides/async.md` | | HTTP Requests (fetch) | `references/guides/http-fetch.md` | | Serving HTTP (http-handler) | `references/guides/http-handler.md` | | TCP Sockets | `references/guides/tcp-sockets.md` | | Gray Streams | `references/guides/gray-streams.md` | | Systems (asdf) | `references/guides/asdf-systems.md` | | Clack Web Applications | `references/guides/clack.md` | | O/R Mapping (mito, sxql) | `references/guides/mito.md` | | Testing (rove) | `references/guides/testing.md` | | Reader Case (Upcasing) | `references/guides/reader-case.md` | | Math Function Backends | `references/guides/math-backends.md` | | The Clock and Randomness | `references/guides/clock-and-random.md` | | WASM Host Boundary (wasm-export / wasm-import) | `references/guides/wasm-host-boundary.md` | | WIT Contracts (wit-export / wit-import) | `references/guides/wit-contracts.md` | | wasm-GC Core Module (Default Output) | `references/guides/wasm-gc-module.md` | | WASI 0.3 Component (--component) | `references/guides/wasm-component.md` | | WASM Non-GC Output (--no-gc) | `references/guides/wasm-nogc.md` | | Running WASM in a Browser | `references/guides/wasm-browser.md` | | Unsupported CL Features | `references/guides/missing-features.md` | | Compiled eval Limitations | `references/guides/eval-limitations.md` | | Compiled read/load Limitations | `references/guides/read-load-limitations.md` | ## Unsupported Common Lisp Features rontolisp is a deliberately small subset of Common Lisp that runs identically on three backends (interpreter, JVM, WASM). To keep the language compilable to plain bytecode without a runtime metaobject protocol, many features of full Common Lisp are intentionally left out. This page lists **only what is missing or partial**. For what *is* available, see the [Language Reference](references/reference/special-forms.md), or list it at runtime with `rontolisp:list-special-forms`, `rontolisp:list-macros`, and `rontolisp:list-functions`. | Feature | Status | | --- | --- | | restarts | available; no debugger integration (`break`, `*debugger-hook*`) and no condition-restart association | | `define-symbol-macro` | not available (the lexical `symbol-macrolet` is) | | `&environment` | accepted in a `defmacro` lambda list but always bound to `nil` (there is no macro-expansion environment object). `&whole` works, in `defmacro` and `destructuring-bind` alike | | `loop` (extended) | partial (see below) | | CLOS | partial (static subset + a definition-time MOP subset) | | `defstruct` `:include` | single inheritance only; slot-overrides `(:include parent (slot default) ...)` work | | `declare` / `declaim` / `proclaim` / `the` | never change a result; on WASM an array `type` declaration directs the element-accessor emission (smaller, faster modules), everywhere else parsed no-ops | | `typep` / `subtypep` / `coerce` / `concatenate` | literal (quoted) type specifiers only; `coerce` targets `'list` / `'vector` / `'string` (or a float type), `concatenate` builds those same three sequence families | | `make-package` / `rename-package` / `delete-package` / `unintern` / `shadow` (runtime) | not available; `export` / `unexport` / `import` / `use-package` ARE, as read/compile-time directives like `in-package`; `defpackage` `:shadow` / `:shadowing-import-from` are errors | | `eval-when` | treated as `progn` (no phase distinction) | | `#:name` | reads as a plain symbol, without gensym-style freshness | | `*modules*` | not available (`require`/`provide` are) | | complex numbers | not available | | `catch` / `throw` / `unwind-protect` / conditions under `--no-gc` | compile error (available on every other backend) | ### Multiple values [`values`](references/reference/functions/values.md) and its consumers are available, including the values of user functions. The remaining deviations from Common Lisp: - a producer that calls `values` in a **non-tail** position and then returns normally may leave stale extra values behind, so keep `values` in result position (a consumer clears the values it received, so only a `values` call that nothing consumes can leave leftovers); - `funcall #'values` (the first-class value) yields the primary value only in compiled programs; - `multiple-value-call` with a built-in `#'name` keeps the wrapper's fixed arity — pass a user function or `lambda` for other argument counts; - other built-ins with secondary values in CL (`read-from-string`, `subtypep`, ...) remain single-value — [`find-symbol`](references/reference/functions/find-symbol.md) and [`intern`](references/reference/functions/intern.md) do answer the accessibility status, and [`macroexpand-1`](references/reference/functions/macroexpand-1.md) / [`macroexpand`](references/reference/functions/macroexpand.md) do answer `expanded-p`. ### Non-local exit [`catch`](references/reference/special-forms/catch.md) / [`throw`](references/reference/special-forms/throw.md), [`block`](references/reference/macros/block.md) / [`return-from`](references/reference/macros/return-from.md) and [`tagbody`](references/reference/special-forms/tagbody.md) / [`go`](references/reference/special-forms/go.md) are available, with two gaps on the **compiled** backends (the interpreter is unaffected): - a `return-from` that would cross an `flet`/`labels` local function is not yet supported (one crossing a `lambda` is, as a non-local exit); - `go` must target a tag of a `tagbody` that lexically encloses it; the interpreter additionally supports dynamic `go` across function-call boundaries, i.e. a tag established by the *caller*. A tag reached from inside a nested `lambda` -- the shape a [`handler-bind`](references/reference/macros/handler-bind.md) handler that resumes the protected loop with a `go` produces, and what quri's `:lenient` percent-decoding does -- is lowered like a cross-`lambda` `return-from`: a non-local exit that re-enters the `tagbody` at the tag and carries on. A cross-`lambda` `return-from` or `go`, `catch`/`throw`, `unwind-protect`, and condition catching all compile in exception-handling mode, so the emitted wasm-GC modules need `wasmtime -W exceptions=y` (37+); under `--no-gc` `catch`/`throw`, `unwind-protect` and the condition forms are a compile error. ### Restarts The condition system is complete through the restart layer: [`handler-bind`](references/reference/macros/handler-bind.md) handlers run at the signal point before unwinding, [`restart-case`](references/reference/macros/restart-case.md) / [`restart-bind`](references/reference/macros/restart-bind.md) / [`with-simple-restart`](references/reference/macros/with-simple-restart.md) establish restarts, and [`find-restart`](references/reference/functions/find-restart.md) / [`invoke-restart`](references/reference/functions/invoke-restart.md) / [`compute-restarts`](references/reference/functions/compute-restarts.md) / [`muffle-warning`](references/reference/functions/muffle-warning.md) / [`abort`](references/reference/functions/abort.md) / [`continue`](references/reference/functions/continue.md) drive them; [`cerror`](references/reference/macros/cerror.md) is continuable. What is missing is the **interactive debugger**: `break` and `*debugger-hook*` do not exist, a restart's `:report` is stored but never rendered and its `:interactive` function never runs, and restarts are not associated with conditions (the optional condition argument of `find-restart`/`compute-restarts` is ignored). [`check-type`](references/reference/macros/check-type.md) / [`assert`](references/reference/macros/assert.md) / [`ccase`](references/reference/macros/ccase.md) still signal without offering a `store-value` restart. Under `--no-gc` the restart forms degrade to the primary form (that backend has no condition objects at all); on the wasm-GC backends only **signaled** conditions are catchable — a runtime trap still aborts. ### The `loop` macro A bounded subset of the extended [`loop`](references/reference/macros/loop.md) is available -- that page lists the supported clauses, which include destructuring patterns, parallel `and`, the anaphoric `it`, `loop-finish` and `thereis`/`always`/`never`. What is out of scope: `named` (and the `return-from` it would name); a destructuring pattern does not recognize lambda-list keywords (`&optional` and friends bind as ordinary variables instead of signalling); `being` drives hash tables, but its package form (`being the external-symbols of ...`) parses and iterates the EMPTY sequence, because there is no runtime intern table. ### Structures and objects [`defstruct`](references/reference/special-forms/defstruct.md) supports `:include` inheritance in its single-inheritance form only. Slot-overrides work: `(:include parent (slot new-default) ...)` re-defaults an inherited slot in the child's layout while it keeps its inherited index, so the parent's accessors still read it. An instance prints in the standard `#S(...)` syntax, and a `#S(...)` literal reads back into an instance -- in source and through the runtime `read` / `read-from-string` on every backend (a compiled program's reader has frontend parity; only `#.`, `#+`/`#-` and `#n=`/`#n#` signal there). A structure that carries a `(:print-object fn)` / `(:print-function fn)` option prints through that function instead; both options are supported. CLOS is a **static subset** ([`defclass`](references/reference/special-forms/defclass.md), [`defgeneric`](references/reference/special-forms/defgeneric.md) / [`defmethod`](references/reference/special-forms/defmethod.md) dispatching on the first argument, [`make-instance`](references/reference/macros/make-instance.md) and [`slot-value`](references/reference/macros/slot-value.md) with literal quoted names). A slot written with no `:initform` starts UNBOUND, as in CL: [`slot-boundp`](references/reference/macros/slot-boundp.md) reports it, [`slot-makunbound`](references/reference/macros/slot-makunbound.md) restores it, and a read signals `unbound-slot`. [`change-class`](references/reference/macros/change-class.md) changes an instance's class in place (the target may be a runtime symbol or a class metaobject), and `reinitialize-instance` / `shared-initialize` are callable with no user method — the system defaults fill the supplied initargs, as in CL. A **definition-time MOP subset** is in: [`find-class`](references/reference/functions/find-class.md) and [`class-of`](references/reference/functions/class-of.md) answer real `standard-class` metaobjects, [`allocate-instance`](references/reference/functions/allocate-instance.md) works, and a `(:metaclass M)` class option runs the class-definition protocol at definition time (see [`defclass`](references/reference/special-forms/defclass.md)) — this is what loads postmodern's DAO layer verbatim. Multiple inheritance works (class precedence list, slot merge across superclasses). Out of scope: runtime class construction (`ensure-class` from computed data, a non-top-level `defclass`, `add-method`, `compute-applicable-methods`, class redefinition, `update-instance-for-different-class`) — the class and method sets of a compiled program are fixed at compile time. ### User-defined packages [`defpackage`](references/reference/special-forms/defpackage.md) is a literal, top-level, read/compile-time directive supporting `:use`, `:export`, `:nicknames` and `:import-from` (`:documentation`/`:size` are accepted and ignored). `:shadow` and `:shadowing-import-from` are errors (there is no symbol shadowing). `use-package`, [`export`](references/reference/functions/export.md), `unexport` and [`import`](references/reference/functions/import.md) exist as the same kind of read/compile-time directive `in-package` is: a literal top-level call takes effect for the forms that follow it, on every backend, and a runtime-computed call works on the interpreter only. Creating or renaming a package at run time does not: `make-package`, `rename-package` and `delete-package` are not available. `unintern` (and the runtime `shadow` / `shadowing-import`) cannot exist here at all — a symbol IS its name, so there is no intern table to remove it from. The queries are real: [`find-package`](references/reference/functions/find-package.md), [`package-name`](references/reference/functions/package-name.md), [`list-all-packages`](references/reference/functions/list-all-packages.md), [`package-use-list`](references/reference/functions/package-use-list.md), [`package-used-by-list`](references/reference/functions/package-used-by-list.md) and [`package-shadowing-symbols`](references/reference/functions/package-shadowing-symbols.md) (always `nil`), with the compiled backends answering from a table baked in at compile time — so a package a compiled program creates later is invisible there. When several used packages export the same name, the first package in `:use` order wins instead of signaling a conflict. ### Dynamic (special) variables Dynamic binding through `let`/`let*` and [`progv`](references/reference/special-forms/progv.md) is supported, with one limitation on the **compiled** backends (the interpreter is unaffected): while normal exit and a `return`/`return-from` that unwinds *across* a special `let` boundary both restore the binding, an error caught by a handler outside the `let` (a `go` across it, and on the WASM backends a `return` that also crosses an `unwind-protect`/`handler-case`) does not. `progv` restores on every exit an `unwind-protect` covers, including those cases. ### Numeric tower rontolisp supports integers (including arbitrary-precision bignums), ratios (`1/3`), and double floats, but **not complex numbers**. A negative square root yields a float `NaN` rather than a complex result: ```console > (sqrt -1) NaN ; full Common Lisp would return #C(0.0 1.0) ``` ### Other omissions - lambda lists: an extended `defmacro` lambda list (`&whole`, `&optional`, `&key`, `&aux`, nested destructuring patterns) routes through `destructuring-bind`, which is deliberately lenient -- a missing argument is `nil` and a surplus one is ignored rather than signalling; and a function is limited to 10 physical parameters on the funcall/apply path. - user macros are unknown to the runtime `eval` of compiled programs, and a `lambda` built at runtime by that `eval` does not parse lambda-list keywords (see [Compiled eval Limitations](references/guides/eval-limitations.md)). - the PRETTY PRINTER produces the text a wide enough line holds, but never changes the LAYOUT: no rontolisp stream carries a column, so a logical block never wraps, every conditional line break (`pprint-newline` with `:linear` / `:fill` / `:miser`, the format directives `~_` / `~:_` / `~@_` / `~i`) is a no-op, and `*print-right-margin*` / `*print-miser-width*` / `*print-lines*` are accepted and ignored. Only `(pprint-newline :mandatory)` and `~:@_` break a line. Every other `*print-*` variable exists and holds the value the printer really behaves as -- binding one to a non-default value is what has no effect, except for `*print-escape*` / `*print-readably*` / `*print-pretty*` and `*print-case*`, which are honored. `*print-case*` converts the case of the symbols the printer spells but leaves a symbol nested in a structure, a CLOS instance, a hash table or an array of rank other than one at its stored spelling ([Reader Case](references/guides/reader-case.md)). The ordinary printing operators do not consult `*print-pprint-dispatch*`: an entry fires where the program calls the entry function itself. - `#.` read-time eval is skipped with a warning inside `.asd` files. - built-in macro names (`cond`, `case`, `when`, `setf`, `push`, ...) cannot be redefined; list them with `(rontolisp:list-macros)`. This list is not exhaustive; rontolisp implements a focused core rather than the full standard. ## Where everything is - `references/operators.md` -- every operator by category. The existence check. - `references/examples.md` -- every worked example in the repository, by directory, with the whole program mirrored under `references/examples/`. Before writing a program of a shape you have not written here before -- a WASM component, an HTTP handler, a Clack app, an `.asd` system, a browser demo -- open the closest one: it shows the imports, the entry point and the build command that no reference page states. - `references/contents.md` -- every documentation page by title. - `references/reference/functions/.md`, `references/reference/macros/.md`, `references/reference/special-forms/.md` -- one page per operator, with a runnable example and its own deviations from Common Lisp. `operators.md` links each name to its page (the slug is not always the name: `+` is `plus.md`). - `references/reference/data-types.md` -- arrays, hash tables, the numeric tower, the packed float literals `#d(...)` / `#f(...)`. - `references/reference/packages.md`, `references/reference/function-namespace.md` -- the package roster, and what `#'name` may and may not name on a compiled backend. - `references/getting-started/`, `references/compiling/` -- the CLI, the REPL, the output shapes and their flags. --- # FILE: references/compiling/dynamic.md # Dynamic (Late Binding) By default the JVM and WASM compilers resolve every call and variable reference statically and reject anything they cannot find at compile time (`Cannot compile: cube`). A top-level `(load "lib.lisp")` with a literal path is handled for you: its forms are spliced into the program at compile time (a [compile-time include](#compile-time-include)), so the functions it defines compile natively and need no special handling. But a `load` whose path is computed at runtime, or one nested inside another form, runs only at runtime -- the compiler cannot see what it defines, so a static call to one of those functions still fails to compile. The `--dynamic` flag relaxes this: a call or reference that cannot be resolved statically is deferred to the runtime `eval` environment (late binding) instead of failing. This lets a program you tested in the interpreter -- one that loads code by a computed path, or otherwise defines functions only at runtime -- compile unchanged, without rewriting `(cube 3)` into `(eval '(cube 3))`. ```bash echo '(defun cube (n) (* n n n))' > lib.lisp echo '(load (concatenate (quote string) "li" "b.lisp")) (print (cube 3))' > prog.lisp rontolisp prog.lisp -o Prog.class --dynamic # the computed-path load is invisible to the compiler; (cube 3) resolves at runtime rontolisp prog.lisp -o prog.wasm --dynamic ``` A call `(f a b)` compiles to `_apply(_eval('(function f), null), (list a b))`: the operator is resolved against the runtime function namespace while the arguments are compiled normally, so locals of the enclosing compiled function stay visible (e.g. `(defun caller (n) (cube n))` works). A bare reference `x` compiles to `_eval('x, null)`, which resolves the variable namespace only. Because the fallback uses the embedded `eval` runtime, `--dynamic` always emits it (as if the program used `eval`), and an unknown symbol that is never defined at runtime errors when it is reached rather than at compile time. Functions resolved this way run on the runtime `eval` interpreter, so they are subject to the [Compiled `eval` limitations](../guides/eval-limitations.md) above. ## Compile-time include You usually do not need `--dynamic` just to split a program across files. A **top-level** `(load "lib.lisp")` whose path is a string literal is treated as a *compile-time include*: the compiler reads `lib.lisp` and splices its forms into the program before compiling (recursively, with a guard against circular loads). The definitions are compiled natively -- no `--dynamic`, no `eval` runtime, and no slowdown -- exactly as if the files had been concatenated. ```bash echo '(defun cube (n) (* n n n))' > lib.lisp echo '(load "lib.lisp") (print (cube 3))' > prog.lisp rontolisp prog.lisp -o Prog.class # (load "lib.lisp") is inlined; cube compiles natively rontolisp prog.lisp -o prog.wasm ``` Only a top-level `load` with a literal path is inlined; a `load` whose path is computed, or one nested inside another form, stays a runtime call and needs `--dynamic` (above) for the compiler to accept calls into the loaded code. The interpreter is unaffected -- it always loads at runtime -- and the include resolves paths the same way the runtime `load` does: a relative path resolves against the directory of the file doing the load (the entry file for a top-level `load`), so a program can be compiled or run from any working directory and still find its companion files. --- # FILE: references/compiling/jvm.md # Compile to JVM Bytecode Give `rontolisp` an output path ending in `.class` with `-o`, and it compiles the source straight to JVM bytecode instead of interpreting it -- no ASM or other library, the bytecode is emitted by hand. The output extension is what selects the backend (`.class` for JVM, `.wasm` for WASM). ```bash echo '(print (+ 1 2))' > hello.lisp rontolisp hello.lisp -o Hello.class java Hello ``` The generated class is named after the output file, so the name you pass to `java` is the file's stem: `-o Hello.class` produces a class `Hello` you run with `java Hello`. Keep the path free of directories (use a plain `Hello.class`, not `out/Hello.class`), since the class name must match. The program's top-level forms become the class's entry point and run in order when you launch it. Example (`hello.lisp`): ```lisp (print (+ 1 2)) ``` ``` 3 ``` ## Optimize (Dead-Code Elimination) Compilation drops every method unreachable from `main`, along with any static field only they referenced, and compacts the constant pool accordingly. You get that without asking: ```bash echo '(defun fact (n) (if (<= n 1) 1 (* n (fact (- n 1))))) (print (fact 10))' > fact.lisp rontolisp fact.lisp -o Fact.class java Fact ``` For a small program like `fact` the class is ~6.5 KB. Pass `--optimize=off` and the class instead embeds the **entire** runtime (printer, numeric, reader and `eval` helper methods, plus a first-class wrapper for every built-in) regardless of what the program actually uses, which for the same `fact` is ~190 KB. The elimination is behavior-preserving: reachability follows the actual `invoke` instructions in the bytecode, so anything a first-class function value, `funcall`, or an embedded `eval`/`load` can dispatch to is kept, and the `java:` interop bridge's reflective entry point survives as an explicit root. The same levels also tree-shake the [WASM output](wasm.md). The dispatch methods `funcall` goes through list only the functions your program can actually obtain as a value — `#'name`, a quoted `'name` designator, a `lambda`, or (while the program holds a symbol builder such as `intern` or `find-symbol`) a string or keyword constant spelling the name — so everything else becomes ordinary dead code the shaker removes. That listing switches off, and every function stays reachable, only when the program can name a function out of data this compile never sees: any use of `eval`, `read`, `read-from-string`, a runtime `load` or a `~/name/` [`format`](../reference/macros/format.md) directive — including one inside a library you loaded — as does `--dynamic`. Compile with `-Drontolisp.debug.dispatchgate=true` to have the compiler name the operator responsible. One carve-out follows from that: a designator assembled at run time out of **computed** pieces — `(funcall (intern (concatenate 'string "gre" suffix)))` — is no constant the compiler can read, so the call signals the ordinary "undefined function" error. `--dynamic` is the way back. `--optimize=off` is not: the listing is not part of what the level switches, so declining the optimizer does not bring such a name back. `--optimize` takes an optional level, shared with the [WASM backend](wasm.md). `--optimize` and `--optimize=default` both spell what an absent flag already selects — everything above — for a build script that wants it written down. `--optimize=off` declines it, and emits what a build before the flag was on by default emitted. `--optimize=size` asks for the smallest output a backend can give; this backend accepts it and emits a byte-for-byte identical class, because what that level declines are the wasm-GC emissions that spend bytes on speed and there is no counterpart here -- the same program's JVM bytecode is about a third the size of its WASM to begin with. So one build script can pass `--optimize=size` for every target. `--optimize=off` exists for two jobs, and neither of them is making a program work: comparing an artifact against one built before a compiler change, and bisecting a suspected shaker bug by asking whether the unshaken class behaves differently. A program whose functions are reached only through a name the compiler cannot read needs `--dynamic`, as above. Independently of the level, compilation always tree-shakes the libraries it splices in: the bundled Lisp-source ones (`linalg:`, `vec:`, JSON, URL, `equalp`/`string<`) and every system loaded with [`asdf:load-system` / `ql:quickload`](../guides/asdf-systems.md). A function, variable or constant your program never mentions -- by name anywhere in the source, including quoted symbols and string literals -- is not compiled in. Your own code is never pruned, and neither is anything a `load`/`require` splices in: only a library that came from a system is subject to it. Classes, generic functions, methods, conditions and structures are pruned by the same rule: a class nothing references leaves together with its methods, and a method on a generic your program does call is still dropped when no reachable code can create an instance of the class it specializes on. Methods on the standard protocol names (`initialize-instance`, `print-object`, `close`, ...) follow their class alone, since those calls are implicit. The one consequence: a library function whose name is only assembled at runtime from computed strings and called through `eval`/`apply` signals the usual "undefined function" error. Compile with `--no-prune` (or `--dynamic`) to keep every library definition in that case. The generated `.class` file targets Java 17 (class version 61), so running it requires a Java 17 or newer JRE. Beyond `java.lang` and `java.io`, the emitted runtime helpers reference `java.math` (`BigInteger`/`BigDecimal`/`MathContext`, for the overflow-promoting integer and exact ratio arithmetic) and `java.util` (`ArrayList`/`Arrays`, and `HashMap` for hash tables); a program that calls `rontolisp:fetch` additionally references `java.net`/`java.net.http`, and `rontolisp:await` / `rontolisp:futurep` represent futures as `java.util.concurrent` futures -- all of which are part of Java 17, so none of these raise the requirement. The one exception is a program that uses the [`java:` interop package](../guides/java-interop.md): the compiler embeds a reflection bridge (compiled with the project's own Java release) into the class, so it needs a JRE at least as new as the one rontolisp was built with. --- # FILE: references/compiling/self-hosted-repl.md # Self-Hosted REPL Because `read-line`, `read-from-string`, `eval` and `print` are available in every backend, a REPL can be written in RontoLisp itself and compiled to a standalone `.class` or `.wasm`: Example (`repl.lisp`): ```console (princ "> ") (setq line (read-line)) (while line (print (eval (read-from-string line))) (princ "> ") (setq line (read-line))) ``` ```bash rontolisp repl.lisp # interpret rontolisp repl.lisp -o repl.class java repl # REPL on the JVM rontolisp repl.lisp -o repl.wasm wasmtime run -W gc repl.wasm # REPL on WASM ``` The self-hosted REPL parses each input line with the embedded runtime reader (`read-from-string`), which upcases symbols like Common Lisp (see the [reader case guide](../guides/reader-case.md)), so `(defun square ...)` echoes `SQUARE`, the same as the native REPL. ```console > (defun square (x) (* x x)) SQUARE > (mapcar #'square '(1 2 3)) (1 4 9) > () NIL > (- 5) -5 ``` `read-line` returns `nil` only at end of input, so the loop exits on Ctrl-D. Entering `nil` or `()` evaluates to `NIL` and the loop keeps going, because the line is read with `read-line` (which distinguishes end of input from a datum) rather than reusing the read value as the loop's exit sentinel. Each line entered at the prompt is parsed by the runtime reader and evaluated by the embedded `eval` runtime, so the [Compiled `eval` limitations](../guides/eval-limitations.md) and [Compiled `read`/`load` limitations](../guides/read-load-limitations.md) apply. --- # FILE: references/compiling/wasm.md # Compile to WASM Give `rontolisp` an output path ending in `.wasm` with `-o`, and it compiles the source to a WebAssembly binary instead of interpreting it. As with the JVM backend, the output extension selects the target, and the binary is emitted by hand without a third-party assembler: ```bash echo '(print (+ 1 2))' > hello.lisp rontolisp hello.lisp -o hello.wasm wasmtime run -W gc hello.wasm ``` ```lisp (print (+ 1 2)) ``` ``` 3 ``` ## Choosing an Output Two independent choices determine the shape of the output: - **Value model.** By default, values live on the WebAssembly **GC heap** (integers as `i31ref`, boxed as a signed 64-bit struct past the fixnum range and as a limb-based big integer past that, floats boxed in a struct), which supports the **full language** but requires a wasm-GC capable runtime (wasmtime 14+, Node 22+, current browsers). `--no-gc` instead lowers a **pure-compute subset** of the language onto unboxed `i64`/`f64` scalars and linear-memory strings — the result is a plain MVP module that runs on **any** WebAssembly engine and is orders of magnitude smaller. - **Packaging.** By default the output is a **WASI Preview 1 core module**. `--component` wraps it as a **component**: on the GC path a WASI 0.3 component with full I/O over the async canonical ABI, on the `--no-gc` path a compact typed reactor component that runs with no host flags at all. `--no-wasi` drops the WASI imports, turning either packaging into a pure-compute library ("reactor"): a Preview 1 module a host instantiates with no import object, or — with `--component` — a **reactor component that imports nothing** and runs its top-level forms at instantiation. Crossing the two axes gives the six shapes: | Output shape | Flags | Language | Runs on | Details | | --- | --- | --- | --- | --- | | WASI command module | (none) | full | wasm-GC engine with WASI Preview 1 (`wasmtime run -W gc`) | [wasm-GC core module](../guides/wasm-gc-module.md) | | Library (reactor) module | `--no-wasi` | full (pure-compute exports) | any wasm-GC engine, no imports needed (Node 22+, current browsers; `--host-random` adds one host import, `--host-fetch` two) | [`--no-wasi` reactor mode](../guides/wasm-gc-module.md#no-wasi-reactor-mode) | | WASI 0.3 component | `--component` | full, plus component-only I/O (`rontolisp:fetch`, TCP sockets) | wasmtime 46+ or another component host with wasm-GC | [WASI 0.3 component](../guides/wasm-component.md) | | Reactor component | `--component --no-wasi` | full (pure-compute exports) | any component host with wasm-GC, empty import object | [Reactor components](../guides/wasm-component.md#reactor-components---component---no-wasi) | | Plain core module | `--no-gc` | numeric/string [subset](../guides/wasm-nogc.md#eligible-subset) | **any** WebAssembly engine, even without wasm-GC or SIMD | [Non-GC output](../guides/wasm-nogc.md) | | Compact typed component | `--no-gc --component` | numeric/string [subset](../guides/wasm-nogc.md#eligible-subset) | any component host, **zero flags** | [Compact component output](../guides/wasm-nogc.md#compact-component-output---no-gc---component) | Rule of thumb: pick the **value model** by what the code needs — the full language means the GC heap; a numeric/string kernel that fits the subset gains universal portability and a hundreds-of-bytes binary from `--no-gc` — then pick the **packaging** by the host: a component host gets `--component`, a plain engine or JavaScript embedder gets a core module. ## Host Boundaries Two complementary directives declare what crosses the module/host boundary: - [**`rontolisp:wasm-export` / `rontolisp:wasm-import`**](../guides/wasm-host-boundary.md) spell out the boundary by hand, in rontolisp's own type designators (`:int`, `:float`, `:string`, `:s-expr`, ...). The same directive compiles into four different host contracts depending on the output shape (raw core function, typed component-model export, ...). Under `--no-wasi`, [`--emit-js-glue`](../guides/wasm-host-boundary.md#generating-the-host-glue---emit-js-glue) writes the JavaScript half of that boundary from the same declarations. - [**WIT contracts (`wit-export` / `wit-import`)**](../guides/wit-contracts.md) drive the boundary from a `.wit` file — one contract, checked on every backend, with per-backend implementations (typed component-model exports under `--component`, provider callbacks on the interpreter and the JVM). Also covers [`--emit-wit`](../guides/wit-contracts.md#emitting-the-wit-world---emit-wit) and [`--scaffold-wit`](../guides/wit-contracts.md#scaffolding-an-implementation---scaffold-wit). ## Running a Component in a Browser `jco transpile` turns a component into plain JavaScript that runs in a page: see the [browser guide](../guides/wasm-browser.md) for what works today (a `--no-gc --component` needs nothing at all, a wasm-GC `--component` loads and computes but cannot yet print) and, at the end, a complete Node + browser walkthrough for calling a `--no-wasi` / `--no-gc` reactor module by hand. ## Cross-Cutting Flags ### Optimize (Tree Shaking) Compilation drops every function unreachable from the module's roots (its exports and the `_start`/`_initialize` entry) and renumbers the survivors. Unused WASI imports are removed too, so a pure-compute reactor module is a handful of functions: ```bash echo "(defun fact (n) (if (<= n 1) 1 (* n (fact (- n 1))))) (rontolisp:wasm-export 'fact :params '(:int) :returns :int)" > fact.lisp rontolisp fact.lisp --no-wasi -o fact.wasm wasmtime run --invoke fact -W gc fact.wasm 5 # => 120, from a ~2.5 KB module ``` Pass `--optimize=off` and the module instead embeds the **entire** runtime (printer, rational, string, reader and `eval` helpers, the WASI import slots, …) regardless of what the program actually uses, because function indices are then held fixed: the same `fact` module is ~155 KB rather than ~2.5 KB. The shaking is behavior-preserving: it walks the call graph from the actual `call` instructions, so anything reachable (including code an embedded `eval`/`load` dispatches to) is kept. It applies on **every** output shape, `--component` included. The same levels also dead-code-eliminate the [JVM output](jvm.md). The dead functions take their baggage with them: the WASI imports only they used, the type definitions nothing left names, and the static string data no surviving code still addresses — a printed literal's module is a few hundred bytes rather than the whole runtime's string table. Definitions that compile to byte-for-byte identical bodies — typically the accessors a `defstruct` or `define-condition` generates — are emitted once, with every call redirected to the shared body. Only the code is shared: each function keeps its own identity, so `(eq #'f #'g)` stays `NIL`. Naming the function outright helps here. `(mapcar #'double xs)`, `(reduce #'+ xs)`, `(sort xs #'<)` and `(funcall #'double x)` compile to the same direct call `(double x)` does, rather than making `double` a first-class value and calling it through the runtime's per-arity dispatcher. Two things follow: the call itself is cheaper, and a function nothing else reaches stops being reachable through that dispatcher — which is often what keeps whole swathes of a library in the artifact. Pass the same function as a computed value (a variable, a `lambda`, a designator built at run time) and the dispatcher comes back, as it must. That floor does not depend on how the program spells the write. A constant text is rendered at compile time and emitted as bytes, so `print`, `princ` + `terpri`, `write-string`, `write-line` and `(format t "Hello, ~a!~%" "World")` all leave the runtime printer behind and land within a few dozen bytes of each other (under 600 B as a core module, under 1.8 KB as a component). What is left of the static data is only what the program itself writes: the printer's own fixed strings — `NIL`, the list punctuation, the float specials, the character names — go with the printer. Print a computed value and both come back, as they must. A value the compiler can work out for itself is not a computed value, though. A call to a **pure built-in whose every argument is a literal** — `(* 6 7)`, `(length "Hello World!")`, `(concatenate 'string "Hello" " " "World!")`, `(string-upcase "hi")` — is evaluated at compile time and the call is deleted, on every backend that compiles (the interpreter still evaluates it at run time, which is the same answer). That happens before the printer fold above, so `(princ (* 6 7))` reaches the very same floor as `(princ 42)` and `(format t "~a~%" (length "abc"))` the same as `(format t "~a~%" 3)`. Redefine one of those names — a `defun`, a `defmethod`, an `flet` — and your definition wins: the compiler stops folding that name anywhere in the program. `--dynamic` turns the whole thing off, since every name there resolves at run time. A literal lookup table is folded the same way, and there the saving is the table itself: `(coerce '(0 #x77073096 …) '(vector (unsigned-byte 32)))` and `(make-array n :element-type '(unsigned-byte 8) :initial-contents '(…))` become the specialized vector they build, which the module carries as static data at the element width instead of building a list of boxed integers at startup — around 4 bytes an element for a 32-bit table rather than 12. Each evaluation of the form still yields a fresh, independently mutable vector, exactly as the call did. An element that does not fit the declared width is not folded; the run-time builder masks it, as it always has. On the `--component` path the **wrapper shrinks with the core**, not just the core itself. Which WASI 0.3 interfaces a component imports follows from what the program can actually reach: `(print "Hello World!")` compiles to a component importing `wasi:cli/types` and `wasi:cli/stdout` and nothing else — no `wasi:filesystem`, no `wasi:clocks`, no `wasi:random`, and not even `wasi:cli/stderr`, since nothing in that program can write to standard error — while a program that opens a file, reads the clock and draws random bytes keeps them all, and one that calls [`warn`](../reference/macros/warn.md) or writes to `*error-output*` gets `wasi:cli/stderr` back — and so does one that uses a condition-handling form ([`handler-case`](../reference/macros/handler-case.md) and friends), because the report an uncaught condition prints before it traps goes there too. `--emit-wit` prints the world the component really has, so the emitted `.wit` shrinks with it. ```bash echo '(print "Hello World!")' > hello.lisp rontolisp hello.lisp --component -o hello.wasm # ~1.7 KB rontolisp hello.lisp --component --optimize=off -o hello-full.wasm # ~165 KB ``` At `--optimize=off` a component always declares the full fixed WASI surface, which is what makes the two builds comparable byte-for-byte across releases. Tree shaking also decides how much of a **loaded library** it can reach. A compiled program calls most functions directly, but a `funcall` needs a dispatch table, and a function listed there counts as reachable whether or not anything ever calls it that way. So a function is listed only when your program can actually obtain it as a value — `#'name`, a quoted `'name` designator, a `lambda` — and everything else becomes ordinary dead code the shaker removes. On a program that loads `md5` and calls one function, that is the difference between about 1.1 MB and 582 KB. A program that holds a symbol **builder** — `intern`, `find-symbol`, `make-symbol`, `uiop:symbol-call` — keeps the listing, and instead widens it: a string or keyword constant the module carries can become a designator at run time, so the compiler probes those spellings of each function's name as well. That is what lets a Worker whose handler discovery is `(find-symbol "RUN" pkg)` still shake out everything it calls only directly. The listing is all-or-nothing, and what switches it off is a program that can name a function out of data this compile never sees: any use of `eval`, `read`, `read-from-string` or a runtime `load` — including one inside a library you loaded. When the build does not shrink as much as you expected, ask the compiler which operator it was: ```bash rontolisp -Drontolisp.debug.dispatchgate=true app.lisp -o app.wasm # => [dispatch-gate] every function stays dispatchable because of: EVAL ``` A `~/name/` directive in a format control string counts as well, because it names its function at run time — but only a control string the compiler can see brings it in, so a program that spells no such directive is unaffected (see [`format`](../reference/macros/format.md)). `--dynamic` switches it off too, by design: late binding resolves any name at run time. One carve-out follows from that: a designator assembled at run time out of **computed** pieces — `(funcall (intern (concatenate 'string "gre" suffix)))` — is no constant the compiler can read, so the call signals the ordinary "undefined function" error. `--dynamic` is the way back, and `--optimize=off` is not: the listing is not part of what the level switches, so declining the optimizer does not bring such a name back. For a much smaller module still, the same `fact.lisp` compiled with [`--no-gc`](../guides/wasm-nogc.md) lowers `fact` to unboxed `i32` and drops the whole GC runtime that made the 2.5 KB (the condition hierarchy, cons cells, the printer): ```bash rontolisp fact.lisp --no-gc -o fact.wasm wasmtime run --invoke fact fact.wasm 5 # => 120, from a ~108 byte module (no -W gc) ``` The source is unchanged — `wasm-export` works identically on both value models — and the resulting module also drops the `-W gc` runtime requirement. Independently of the level (and on every output mode, `--component` included), compilation always tree-shakes the libraries it splices in: the bundled Lisp-source ones (`linalg:`, `vec:`, JSON, URL, `equalp`/`string<`) and every system loaded with [`asdf:load-system` / `ql:quickload`](../guides/asdf-systems.md). A function, variable or constant your program never mentions -- by name anywhere in the source, including quoted symbols and string literals -- is not compiled into the module. Your own code is never pruned, and neither is anything a `load`/`require` splices in: only a library that came from a system is subject to it. Classes, generic functions, methods, conditions and structures are pruned by the same rule: a class nothing references leaves together with its methods, and a method on a generic your program does call is still dropped when no reachable code can create an instance of the class it specializes on. Methods on the standard protocol names (`initialize-instance`, `print-object`, `close`, ...) follow their class alone, since those calls are implicit. The one consequence: a library function whose name is only assembled at runtime from computed strings and called through `eval`/`apply` signals the usual "undefined function" error. Compile with `--no-prune` (or `--dynamic`) to keep every library definition in that case. The flag takes an optional level. `--optimize` and `--optimize=default` are the same thing — everything above, and what an absent flag already selects — and the bare spelling keeps that meaning permanently; `--optimize=size` is that plus the trades in the next section; `--optimize=off` is none of it, and emits what a build before the flag was on by default emitted. `--optimize=off` exists for two jobs, and neither of them is making a program work: comparing a module against one built before a compiler change, and bisecting a suspected tree-shaker bug by asking whether the unshaken module behaves differently. A program whose functions are reached only through a name the compiler cannot read needs `--dynamic` — the funcall dispatch listing above is not part of what the level switches, so `off` does not bring such a name back. ### Optimizing for Size (`--optimize=size`) Two wasm-GC emissions deliberately spend bytes to gain speed, and both are on at `--optimize=off` and `--optimize=default` alike: - an integer expression tree like `(logand (+ (ash x 7) i) #xFFFFFFFF)` compiles **twice** — once as a single unboxed `i64` computation, and once through the generic helpers, as the fallback a float, a ratio or an overflow into bignum territory takes; - a `let` binding whose assignments are integer arithmetic gets an unboxed `i64` slot beside its ordinary boxed one. `--optimize=size` declines both. Nothing the program computes changes — the fast path only ever existed as an alternative to the fallback, which stays — but the arithmetic now runs through the generic helpers, so the price is real, and how much you pay depends on how integer-heavy the program is: | program | `--optimize=default` | `--optimize=size` | run time | | --- | --- | --- | --- | | ironclad SHA-256/HMAC/PBKDF2, 4096 rounds | 2,078,195 B | 1,562,816 B (**-24.8%**) | 1.4 s -> 5.2 s (**3.8x**) | | a `vec:`-kernel neural-net training loop | 271,233 B | 214,169 B (-21.0%) | 1.07 s -> 1.26 s (+18%) | | a float MLP training loop (no `vec:`) | 159,747 B | 125,496 B (-21.4%) | 5.6 s -> 6.1 s (+9%) | | `cl-postgres` hello world (`--component`) | 8,024,998 B | 6,384,099 B (-20.4%) | — | (wasmtime 47, best of three runs.) The size win barely varies; the run-time price does, because only integer arithmetic fuses — a float kernel pays it on its loop indices alone, while a crypto round pays it on everything. So reach for it when the module has to travel — an edge deploy, a browser download, a registry with a size limit — unless the program's hot loop is integer arithmetic (hashing, crypto, bit twiddling), where the same win costs several times the run time. The level is accepted on every backend, so a build script need not know which one it targets, but only wasm-GC (Preview 1 and `--component`) has anything to trade: the [JVM](jvm.md) and [`--no-gc`](../guides/wasm-nogc.md) outputs are byte-for-byte what `--optimize=default` produces. ### SIMD Acceleration (`--simd`) `--simd` is the one acceleration switch shared by every backend: it lowers the vectorizable [`vec:` and `linalg:` kernels](../guides/simd-acceleration.md) to real vector instructions. On WASM it is orthogonal to the value model: - **wasm-GC + `--simd`** lowers the kernels to native fixed-width SIMD (`f64x2`/`f32x4`) over GC-managed lane-group arrays — packed float arrays stay ordinary GC objects, and memory behaves exactly as without the flag. Composes with `--component` and every `--optimize` level; run as usual with `wasmtime run -W gc` (wasmtime enables the SIMD proposal by default). - **`--no-gc` + `--simd`** lowers the same kernels to `v128` over the packed linear-memory blocks. Without `--simd`, `--no-gc` emits plain scalar loops instead — a v128-free MVP module that also runs on a runtime lacking the SIMD proposal. The full story — which kernels vectorize, precision rules for single-float reductions, measured effects, and the `linalg` interception — lives in the [SIMD acceleration guide](../guides/simd-acceleration.md). --- # FILE: references/getting-started/agent-skill.md # Agent Skill An AI coding agent that already knows Common Lisp still writes wrong rontolisp: it reaches for operators the subset does not have, misses the extensions it does have, and does not know how to run the result on each backend. The **agent skill** closes that gap. It is this same manual, packaged as a skill: a `SKILL.md` that carries the delta from Common Lisp plus every page here as a bundled reference, generated on each deploy so it cannot drift from the documentation you are reading. ## Install into Claude Code The skill is published as a plugin. Add the marketplace and install it: ```bash claude plugin marketplace add https://making.github.io/rontolisp/skill/marketplace.json claude plugin install rontolisp@rontolisp ``` The same two steps work inside a session as `/plugin marketplace add ...` and `/plugin install rontolisp@rontolisp`. Add `--scope project` to the first command to declare the marketplace in the repository instead of your user settings, so everyone working on the checkout gets the same offer. Nothing else is needed: the agent consults the skill by itself when a task involves rontolisp -- a `.lisp` or `.asd` file, or a request that names the language. ```bash claude plugin update rontolisp@rontolisp # take a newer version claude plugin uninstall rontolisp@rontolisp # remove it claude plugin list # what is installed ``` Claude Code re-reads the marketplace file from that URL, so a new version becomes available without you changing anything -- see [Staying current](#staying-current) for how the version moves. ## Install without the plugin system A skill is a directory, and Claude Code reads every skill under `~/.claude/skills` (yours) or `.claude/skills` (the repository's). If you would rather drop it in than register a marketplace: ```bash mkdir -p ~/.claude/skills && \ curl -sSL https://making.github.io/rontolisp/skill/rontolisp-skill.tar.gz | tar xz -C ~/.claude/skills ``` Swap the target for `.claude/skills` to install it into the project instead. Either way you end up with `skills/rontolisp/SKILL.md` and `skills/rontolisp/references/`; `/skills` lists what is loaded, and removing the `rontolisp` directory uninstalls it. Nothing updates it for you, so this is the path where you check the version yourself. ## Other agents and hosts | File | Use | | --- | --- | | [marketplace.json](https://making.github.io/rontolisp/skill/marketplace.json) | the plugin marketplace, added by URL as above | | [rontolisp-plugin.zip](https://making.github.io/rontolisp/skill/rontolisp-plugin.zip) | the plugin itself, if you install plugins some other way | | [rontolisp-skill.tar.gz](https://making.github.io/rontolisp/skill/rontolisp-skill.tar.gz) | the bare skill directory, for a `skills` folder | | [rontolisp.skill](https://making.github.io/rontolisp/skill/rontolisp.skill) | the same tree as a zip, to upload where a skill is uploaded rather than unpacked | | [SKILL.md](https://making.github.io/rontolisp/skill/rontolisp/SKILL.md) | the skill body alone, readable in place | | [rontolisp-full.md](https://making.github.io/rontolisp/skill/rontolisp-full.md) | manual and skill as ONE Markdown file, for a tool that has no skill loader | ## Staying current The skill is versioned `.`, so it moves exactly when the documentation or the generator does. As a plugin, `claude plugin update rontolisp@rontolisp` takes the new one. Installed by hand, compare your copy against the published version and reinstall when they differ: ```bash head -3 ~/.claude/skills/rontolisp/SKILL.md # version: 0.1.391 curl -sSL https://making.github.io/rontolisp/skill/VERSION ``` Reinstalling is the same command as installing -- it overwrites in place. [version.json](https://making.github.io/rontolisp/skill/version.json) carries the same version plus the commit it was built from, if you would rather check it from a script. ## What is inside `SKILL.md` states the working rule -- Common Lisp knowledge is a *prior* here, not the truth -- and inlines [Unsupported Common Lisp Features](../guides/missing-features.md), because that is what those priors get wrong most often. Under `references/`: - `operators.md`, an index of every operator in the language by category. One lookup answers whether rontolisp has something, which is the question a Common Lisp background answers wrongly. - `contents.md`, every page of this manual by title. - every page of this manual, verbatim, at the same relative paths -- so a detail the skill needs is a file read away, with no network. - `examples.md` and `examples/`, the repository's example programs as they are, minus the build outputs: a compiled `.wasm` or a `.bin` of weights is a link to the repository rather than a file. Documentation says what an operator does; an example says what a whole program of some shape looks like, down to the build command. --- # FILE: references/getting-started/build.md # Build & Install ## Download a prebuilt binary (recommended) Prebuilt native-image binaries are published on the [GitHub releases page](https://github.com/making/rontolisp/releases/tag/0.1.0-SNAPSHOT). They start instantly and need no JVM installed. Pick the asset for your platform, make it executable, and put it on your `PATH` as `rontolisp`. macOS (Apple Silicon): ```bash wget https://github.com/making/rontolisp/releases/download/0.1.0-SNAPSHOT/rontolisp-darwin-arm64 chmod +x rontolisp-darwin-arm64 sudo mv rontolisp-darwin-arm64 /usr/local/bin/rontolisp ``` Linux (x86-64): ```bash wget https://github.com/making/rontolisp/releases/download/0.1.0-SNAPSHOT/rontolisp-linux-amd64 chmod +x rontolisp-linux-amd64 sudo mv rontolisp-linux-amd64 /usr/local/bin/rontolisp ``` Linux (ARM64): ```bash wget https://github.com/making/rontolisp/releases/download/0.1.0-SNAPSHOT/rontolisp-linux-arm64 chmod +x rontolisp-linux-arm64 sudo mv rontolisp-linux-arm64 /usr/local/bin/rontolisp ``` Verify the install: ```bash rontolisp --version ``` A prebuilt binary needs nothing else installed. To run `.wasm` output you also need a wasm-GC capable runtime such as [wasmtime](https://wasmtime.dev/) 14+ (optional). The rest of the documentation uses the `rontolisp` command. If you build from source instead (below), substitute `java -jar target/rontolisp-0.1.0-SNAPSHOT-exec.jar` for `rontolisp`. ## Download the executable JAR The same [releases page](https://github.com/making/rontolisp/releases/tag/0.1.0-SNAPSHOT) also publishes the self-contained executable JAR (`rontolisp-0.1.0-SNAPSHOT-exec.jar`). It needs a Java 25+ runtime but works on any platform, and it is the only artifact that can *interpret* the JVM-only [`java:` interop package](../guides/java-interop.md) — the native binaries carry no reflection metadata, so interpreting `java:` forms fails there (both artifacts can still compile a `java:` program to a `.class` that runs under `java`). ```bash wget https://github.com/making/rontolisp/releases/download/0.1.0-SNAPSHOT/rontolisp-0.1.0-SNAPSHOT-exec.jar java -jar rontolisp-0.1.0-SNAPSHOT-exec.jar --version ``` ## Build from source Requires **Java 25+**. ```bash ./mvnw clean package ``` This produces `target/rontolisp-0.1.0-SNAPSHOT-exec.jar`, an executable JAR with all dependencies included. Run it with `java -jar target/rontolisp-0.1.0-SNAPSHOT-exec.jar`. ### Native Image (GraalVM) Build a native executable yourself using GraalVM: ```bash ./mvnw -Pnative clean package ``` This produces `target/rontolisp`, a standalone native binary with instant startup. **Requirements:** - GraalVM 25+ (with `native-image` tool) **Usage:** ```bash # REPL rontolisp # File interpretation rontolisp program.lisp # Compile to JVM bytecode rontolisp hello.lisp -o Hello.class # Compile to WASM rontolisp hello.lisp -o hello.wasm ``` --- # FILE: references/getting-started/file-interpretation.md # File Interpretation Pass a path to a `.lisp` file and rontolisp interprets it directly, without producing any compiled artifact. The file's top-level forms are read and evaluated in order on the tree-walking interpreter, sharing one environment, so a function or variable defined near the top is available to everything below it. ```bash rontolisp program.lisp ``` Unlike the REPL, a script does not echo the value of each form -- output is whatever the program writes explicitly with `print`, `format`, and the like. The process exits with status `0` when the file runs to completion, or non-zero if a form signals an error. Example (`program.lisp`): ```lisp (defun square (x) (* x x)) (print (square 5)) (print (square 12)) ``` ``` 25 144 ``` This is the same source you would later hand to the JVM or WASM compiler; running it through the interpreter first is the fastest way to check a program's behavior before compiling it. ## Programs Given on the Command Line `-e` (long form `--eval`) takes the program from the argument itself instead of a file. It runs exactly like a file -- no value is echoed, only what the program prints: ```bash rontolisp -e "(print (+ 1 2))" ``` ``` 3 ``` The option is repeatable, and the occurrences make up one program evaluated in order, as if the forms were written on successive lines of a single file: ```bash rontolisp -e "(defun square (x) (* x x))" -e "(print (square 5))" ``` ``` 25 ``` It combines with everything a file accepts, including the compilers (`rontolisp -e "(print (+ 1 2))" -o Prog.class`), but not with an input file -- give the program one way or the other. There being no source file, a relative `(load "...")` resolves against the working directory. --- # FILE: references/getting-started/format.md # Formatting Source Code `rontolisp format` re-indents Lisp source files in place. Point it at a file or at a directory and every `.lisp` and `.asd` file under it is rewritten to one canonical layout, so indentation stops being something anyone has to think about or review. ```bash rontolisp format app.lisp # one file rontolisp format src/ # every .lisp / .asd under src/ rontolisp format src/ tests/ # several paths ``` Only whitespace changes. Every token is reproduced exactly as written -- including its case, so `Foo` stays `Foo` -- and strings, character literals, block comments and `#+`/`#-` guards are copied through untouched. The formatted file reads as precisely the same program; nothing is macroexpanded, evaluated, or loaded, so a file formats whether or not its dependencies are installed. Files that are already formatted are left alone, not even rewritten, so the command is safe to run over a whole tree repeatedly. ## Options | Option | Meaning | | --- | --- | | `--check` | Do not write anything. List the files that are not formatted and exit `1` if there are any. | | `--stdout` | Write the result to standard output instead of the file (one file only). | | `--width=N` | Right margin to wrap to. Default `80`. | | `-h`, `--help` | Show the command's help. | A `-` in place of a path formats standard input to standard output, which is what an editor's "format buffer" command wants: ```bash echo '(let ((a 1)(b 2))(+ a b))' | rontolisp format - ``` ``` (let ((a 1) (b 2)) (+ a b)) ``` `--check` writes nothing and fails when the tree is not formatted, which makes it a one-line CI gate: ```bash rontolisp format --check src/ || { echo "run: rontolisp format src/"; exit 1; } ``` ## What the layout looks like A form that fits within the margin goes on one line. One that does not breaks according to what its operator is. A definition keeps its name and lambda list on the first line and indents the body by two: ```lisp (defun fizzbuzz (n) (cond ((zerop (mod n 15)) "FizzBuzz") ((zerop (mod n 3)) "Fizz") ((zerop (mod n 5)) "Buzz") (t (write-to-string n)))) ; => FIZZBUZZ ``` `cond` clauses line up under the first clause. A clause too wide for one line puts its body below its test -- unless the test is a single token like `t` and the body is one form, in which case the body stays beside it rather than leaving a line to that token, so long as it needs no extra line and no extra width there. `if` puts its two branches under the test, so they read as a pair rather than as a body: ```lisp (let ((threshold (* 10 10)) (small-label "small") (measured (list 1 2 3 4 5 6))) (if (< (length measured) threshold) (list small-label (length measured)) (list "large" (length measured)))) ; => ("small" 6) ``` `let` bindings line up under the first binding, once they stop fitting on the `let`'s own line: ```lisp (let ((numbers (list 3 1 4 1 5 9 2 6)) (sorted (sort (list 3 1 4 1 5) #'<)) (total (+ 1 2 3 4 5))) (list (length numbers) sorted total)) ; => (8 (1 1 3 4 5) 15) ``` A function call puts its arguments under the first one, and keeps each `:keyword value` option together on a line of its own: ```console (with-open-file (out "report.txt" :direction :output :if-exists :supersede :if-does-not-exist :create) (write-line "done" out)) ``` `loop` gets one line per clause, aligned under the first: ```lisp (loop for i from 1 to 10 when (evenp i) collect (* i i) into squares finally (return squares)) ; => (4 16 36 64 100) ``` ### Two body forms are always two lines A body of two or more forms is a sequence performed in order, so it gets a line each however short it is -- the same reason no formatter of a C-like language will put two statements on one line. A body of exactly one form may share the header's line: ```lisp (defun double (x) (* 2 x)) ; => DOUBLE ``` but a body of two may not, even though it would fit: ```lisp (defun report (x) (print x) (terpri)) ; => REPORT ``` This is also what keeps the output stable: a two-form body will not silently join because a rename made it two characters shorter. ### Comments A comment that started its own line keeps one, at the indentation of the code around it. A comment that trailed code stays on that code's line, and the trailing comments of consecutive lines are lined up into a column: ```console (setq width 80) ; the right margin (setq body-indent 2) ; body indentation (setq tabs nil) ; spaces only ``` ### Blank lines A blank line is the only paragraph break Lisp source has, so wherever you left one it stays -- inside a body as well as between top-level forms. A run of several collapses to one, and no blank line is ever added. ## Limits Line comments and string literals are never re-wrapped: their content is yours, not the formatter's. A line whose content simply cannot be broken -- a long string, a deeply nested expression with no shorter arrangement -- may therefore end up past the margin. A macro the formatter has not been told about is laid out from its name: `with-...` and `do-...` take one argument then a body, `def...` takes a name and then a body, and anything else is laid out as a function call. A `def...` macro keeps a lambda list on its first line the way `defun` does, but only when its second element could BE a lambda list -- a list of plain parameter names, possibly empty or with `&optional`/`&rest`/`&key` markers. A keyword, a string, a number, `nil`, `t` or a nested form in that position means the element is the first form of the body instead, and it gets a line of its own: ```console (define-get "/hello" () (ok "hello world")) (define-routes *app* (define-get "/hello" () (ok "hello world")) (define-any "*" () (not-found "nope"))) ``` If your own macro takes a different shape, write it in the layout you want and the formatter will keep it as long as it fits on one line. --- # FILE: references/getting-started/repl.md # REPL Run `rontolisp` with no file argument to start an interactive read-eval-print loop. It reads one expression at a time, evaluates it on the tree-walking interpreter, and prints the result -- the quickest way to explore the language. ```bash rontolisp ``` ``` > (+ 1 2) 3 > (* 3 (+ 4 5)) 27 > (defun fact (n) (if (= n 0) 1 (* n (fact (- n 1))))) FACT > (fact 10) 3628800 > (quit) ``` Each top-level form is evaluated as soon as it is complete, and its value is echoed back. Definitions persist across inputs: a `defun`, `defvar`, or `setq` entered at one prompt is visible at every later one, so you can build up state incrementally within a session. A form that returns [multiple values](../reference/functions/values.md) echoes every value, one per line -- the quotient and the remainder of `floor`, the value and the present-p flag of `gethash`. `(values)` returns no value at all and echoes nothing: ```console > (floor 10 3) 3 1 > (gethash 'b (make-hash-table)) NIL NIL > (values 1 2 3) 1 2 3 > (values) > ``` The prompt accepts multi-line input -- if an expression has unbalanced parentheses, the REPL keeps reading until it is closed before evaluating. It also supports line editing, history navigation with the up/down arrow keys, and Ctrl-C to cancel the current input. Type `(quit)` or press Ctrl-D to exit. Try a quick expression here: ```lisp (let ((x 10) (y 20)) (+ x y)) ; => 30 ``` --- # FILE: references/guides/asdf-systems.md # Systems (asdf) The `asdf` package provides a **limited, API-compatible subset of ASDF**, the Common Lisp build facility: you describe a multi-file project once in a `NAME.asd` file with [`asdf:defsystem`](../reference/functions/asdf-defsystem.md), and [`asdf:load-system`](../reference/functions/asdf-load-system.md) loads the files in dependency order — on every backend. Real ASDF is not ported (it depends on CLOS, the condition system and the pathname API, none of which exist here); instead, `.asd` files are parsed as plain data and the supported `defsystem` subset drives the same machinery as `load`/`require`. A `.asd` that stays inside the subset works unchanged. | Operator | Purpose | |----------|---------| | [`asdf:defsystem`](../reference/functions/asdf-defsystem.md) | Define a system: `:depends-on`, `:serial`, `:components` | | [`asdf:load-system`](../reference/functions/asdf-load-system.md) | Load a system (dependencies first, files in order, idempotent) | ## A complete project ```console app/ my-app.asd package.lisp main.lisp run.lisp registry/base/ base.asd base.lisp ``` ```console ;; app/my-app.asd (defsystem :my-app :version "0.1.0" :depends-on (:base) :serial t :components ((:file "package") (:file "main"))) ;; app/package.lisp (defpackage :my-app (:use :cl) (:export :run)) ;; app/main.lisp (in-package :my-app) (defun run () (print (base:double 21))) ;; app/run.lisp (asdf:load-system :my-app) (my-app:run) ``` Run or compile the entry file; the same directive works on all four backends: ```console rontolisp app/run.lisp --system-path registry/base # interpret rontolisp app/run.lisp --system-path registry/base -o Prog.class # JVM rontolisp app/run.lisp --system-path registry/base -o app.wasm # WASM ``` `my-app.asd` is found next to `run.lisp`; the `:base` dependency system is found through `--system-path`. On the compile path the whole system (its dependency first) is spliced into the program at compile time, exactly like the compile-time `load` include, so the JVM and WASM compilers see every `defun` natively. ## The system search path `asdf:load-system` looks for `NAME.asd` in, in order: 1. the directory of the file doing the `load-system` (like `load`), 2. the directories given with `--system-path` (several can be joined with the platform path separator, like `PATH`), 3. the directories in the `RONTOLISP_SOURCE_REGISTRY` environment variable (same format). A dependency system's `.asd` is searched starting from the depending system's directory, so sibling systems in one registry directory find each other. ## Downloading with quickload To skip the manual download, [`ql:quickload`](../reference/functions/ql-quickload.md) fetches a system (and its dependencies) from the real [Quicklisp](https://www.quicklisp.org/) distribution and then loads it through exactly the machinery above: ```console $ rontolisp > (ql:quickload "split-sequence") (split-sequence) > (split-sequence:split-sequence #\, "a,b,c") ("a" "b" "c") ``` The Quicklisp dist metadata drives the download (`systems.txt` for dependency resolution, `releases.txt` for the tarball URLs); each release is extracted and cached under `~/.rontolisp/quicklisp/` (override with `RONTOLISP_QUICKLISP_HOME`), so a repeat `quickload` does no network I/O. The download runs at interpret time or compile time (Java-side): a compiled program has the sources spliced in and never fetches at runtime, so `ql:quickload` works on all four backends. Because loading still goes through the `asdf` subset, the same limitations apply — a downloaded library only loads if its sources stay inside the supported subset below. ## Adding a dist (Ultralisp) Quicklisp's dist format is spoken by more than one distribution, and [Ultralisp](https://ultralisp.org/) — rebuilt every few minutes, so a library lands in it the day it is published — is the usual second one. It is **opt-in**: a program installs it with [`ql-dist:install-dist`](../reference/functions/ql-dist-install-dist.md), the same call real Quicklisp takes, ```console $ rontolisp > (ql-dist:install-dist "http://dist.ultralisp.org/" :prompt nil) "ultralisp" > (ql:quickload "circular-buffer") (circular-buffer) ``` and an invocation with nowhere to put a form (`rontolisp test SYSTEM`, a build script that must not edit the sources it compiles) names it on the command line instead — `--dist ultralisp`, several comma-separated, or the `RONTOLISP_DISTS` environment variable. Either channel also takes the URL of any other Quicklisp-format distinfo. The dists are searched **in installation order, per system**: `ql:quickload` takes each system — and each dependency — from the first dist that lists it, so adding one supplies the names Quicklisp does not have and changes where nothing else comes from. Quicklisp is installed first unless it is named explicitly, so `--dist ultralisp,quicklisp` is how Ultralisp's copy of a library both dists carry wins. A dist's index is downloaded only when a lookup actually reaches it, and each dist caches under its own `~/.rontolisp//` (`RONTOLISP_DIST_HOME` overrides the base; `RONTOLISP_QUICKLISP_HOME` still overrides the quicklisp one). Because the index is then cached forever, [`ql:update-dist`](../reference/functions/ql-update-dist.md) is what makes a fast-moving dist worth having: ```console $ rontolisp -e '(ql:update-dist "ultralisp")' ``` Everything else is unchanged: the download happens at interpret time or compile time, so an installed dist works the same on all four backends, and a downloaded library still has to stay inside the supported subset below. ## What is (and is not) supported - `.asd` files are parsed as **data**: `defsystem` (bare or `asdf:`-qualified), `in-package`/`defpackage` forms (skipped), `register-system-packages` forms (which record "this package lives in that system" — read when a package-inferred system turns a `defpackage` dependency into a system name, and otherwise inert, since a package is found through its own `defpackage`), and top-level `defparameter`s of pure literal/conditional values (evaluated into a parse-time environment) may appear. `#+`/`#-` feature conditionals work (evaluated against the target backend's features, see [Data Types](../reference/data-types.md#comments-feature-conditionals-and-features)), a `#.` read-time-eval form is resolved where its value is **used**: in a clause that decides what gets loaded it is resolved against those `defparameter`s (the `(:file #.*string-file*)` idiom), and an unresolvable one is an error naming the file and the clause; in ignored metadata (`:long-description`, `:version`, `:perform`, …) and at top level (an ASDF version guard) it is dropped unevaluated and unremarked. A `:depends-on` entry may be `(:feature EXPR DEP)`, contributing its dependency only when the feature expression holds, or `(:version NAME "1.2.3")`, resolving to the plain dependency (the version constraint is not checked — the `:version` option is ignored metadata here, so there is nothing to check against). A top-level `(defmethod perform ...)` hook is tolerated and ignored (there is no `operate` machinery for it to run on; any other method name is an error), and a top-level `defclass` whose superclasses are documentation component classes (ASDF's `doc-file`, or one declared earlier in the same file) declares its name as a component type whose entries participate in ordering but contribute no source, like `:static-file` — `:doc-file` and `:html-file` work without a `defclass`. Any other top-level form is an error naming the file. - A `.asd` may **announce features**: a top-level `(eval-when (:load-toplevel :execute) (pushnew :my-feature *features*))` (or a bare `pushnew`/`push`) before a `defsystem` declares that feature for every system defined after it in the file — the same effect as writing `:rontolisp-features (:my-feature)` on those systems. The declaration reaches the system's own `:if-feature` / `(:feature ...)` clauses and the reading of its component files — carrying the announcement out of the `.asd`, which is the half the reader cannot do for itself. (A `#+`/`#-` in the **same** `.asd` sees the push too, but through the reader's own handling of a source's announcement — see [Data Types](../reference/data-types.md#comments-feature-conditionals-and-features).) It does not reach a dependency, which declares its own. An `eval-when` whose situations are only `(:compile-toplevel)` is inert (ASDF loads a `.asd`, it never compiles one), and any other form inside the `eval-when` is an error naming it. - `defsystem` supports the metadata options (ignored, except that a plain-string `:version` is read back by [`asdf:component-version`](../reference/functions/asdf-component-version.md)), `:depends-on`, `:defsystem-depends-on` (systems real ASDF loads while the `.asd` is **read**: located the same way, loaded before `:depends-on`, never a sideway dependency of the system, and a built-in shim named there announces its features to this system — `("trivial-features")` puts `:unix` and `:little-endian` in force for its clauses and component files), `:serial`, `:pathname` (a literal directory prefixed to every component, so a system whose sources live in `src/` can name them bare) and `:components` with `:file`/`:module`/`:static-file` entries; a component may carry `:if-feature expr`, which drops the component's files when the feature expression does not hold (how libraries gate CLOS-only files behind `(:or :sbcl ...)`) while keeping its place in the dependency order. The test-op wiring options are the one op with machinery behind them: `:in-order-to ((test-op (test-op ...)))` and `:perform (test-op (o c) ...)` are recorded and driven by [`asdf:test-system`](../reference/functions/asdf-test-system.md) (any other operation, a qualified `test-op :after` method, or a `#.` in the body stays tolerated and ignored — there is still no general `operate` machinery). A `:version` value may be any literal form including ASDF's `(:read-file-form ...)` indirection (never inspected — only a plain string is recorded). Anything else (`:around-compile`, ...) is an error naming the clause. - `:class :package-inferred-system` is supported — the style ningle, rove and array-operations use. Such a system has **no `:components` at all**: a sub-system name is a file path under the system's directory (`my-lib/main` is `main.lisp`, `my-lib/util/text` is `util/text.lisp`, both below `:pathname` when the system has one), and that file's own **`defpackage`** names its dependencies — every package in `:use`, `:mix`, `:reexport`, `:use-reexport` and `:mix-reexport`, plus the first argument of each `:import-from` / `:shadowing-import-from`. A package name becomes a system name: what a `register-system-packages` form declared, otherwise the downcased package name itself (`cl` and friends drop out). Only the forms up to the first package definition form of each file are read, so the common `(in-package #:cl-user)` header before the `defpackage` is fine; a file with no `defpackage`/`uiop:define-package` at all is an error. No other `:class` is supported, and a package-inferred system that also lists `:components` is an error. - Loading a system twice is a no-op; circular `:depends-on` chains are detected and reported — including one written as a cycle between two sub-systems' `defpackage` forms. - The compile path requires a literal, top-level `(asdf:load-system NAME)`; the interpreter also accepts a computed name at runtime. Both accept and ignore trailing keyword options (`:verbose nil`, `:force t`, `:silent t`), which real libraries pass when they load a system at runtime. A nested or computed `load-system`/`ql:quickload` in a compiled program answers `nil` when the system was already spliced and signals otherwise — nothing can be loaded at run time there. - **The component metaobjects are real at run time.** [`asdf:find-system`](../reference/functions/asdf-find-system.md) answers a memoized CLOS instance per system (`eq` across calls) over real classes — `asdf:component`, `asdf:child-component`/`asdf:parent-component`, `asdf:module`, `asdf:system`, `asdf:package-inferred-system`, `asdf:source-file`, `asdf:cl-source-file`, `asdf:static-file` — so `typep`, `typecase` and `defmethod` specializers over them work on every backend. The readers [`asdf:component-name`](../reference/functions/asdf-component-name.md), [`asdf:component-version`](../reference/functions/asdf-component-version.md) (the declared `:version` string, when it was written as a plain string), [`asdf:component-pathname`](../reference/functions/asdf-component-pathname.md), [`asdf:component-children`](../reference/functions/asdf-component-children.md) (one `cl-source-file` per component file, in load order), [`asdf:component-sideway-dependencies`](../reference/functions/asdf-component-sideway-dependencies.md), [`asdf:component-parent`](../reference/functions/asdf-component-parent.md) and [`asdf:component-system`](../reference/functions/asdf-component-system.md) walk the model; [`asdf:registered-systems`](../reference/functions/asdf-registered-systems.md) lists every registered name, and `asdf:*user-cache*` is external and `nil` (there is no fasl cache). This is the component model rove's system-driven test runner reads. - **`asdf:test-system` runs the recorded test-op wiring.** [`asdf:test-system`](../reference/functions/asdf-test-system.md) loads the system, follows its `:in-order-to` test-op chain, and runs each recorded `:perform (test-op (o c) ...)` body with the component bound to the system metaobject. On the compile paths a literal top-level call splices the test systems too. - **Compiling tree-shakes the system.** A function, variable or constant a loaded system defines but your program never reaches — following names through the source, including quoted symbols and whole string literals — is left out of the `.class`/`.wasm` — classes, generic functions, methods, conditions and structures included (a method also leaves when no reachable code can create an instance it applies to). Compile with `--no-prune` (or `--dynamic`) to keep every definition; see [Compiling to the JVM](../compiling/jvm.md) for the one consequence. - **The load-context variables hold the file being loaded, on every backend.** While a file is being loaded, `*load-pathname*` holds the path `load` was called with and `*load-truename*` the path it resolved to; the enclosing file's values come back when it finishes, and outside a load both are `nil` — read at top level or from a function the load defined and your program calls later. **A component is loaded by its resolved path**, so both variables hold what [`asdf:component-pathname`](../reference/functions/asdf-component-pathname.md) answers for it, which is what lets a test framework correlate the definitions of a file with the system that owns it. This works the same on the compile backends: a spliced file's forms are compiled with its own load context, because nothing is being loaded at run time there for them to read otherwise. `*compile-file-pathname*` and `*compile-file-truename*` are always `nil` — there is no `compile-file` here. Libraries use the `(or *compile-file-truename* *load-truename*)` idiom to find data files beside their own sources. **That works at read time too**: a `#.` form — the way that idiom is normally spelled, so the file is read while the source is being read — sees its own file's load context on every backend, so a library can bake in a data file that ships next to it. ## Built-in shim systems Some Quicklisp libraries depend on per-implementation portability layers that cannot know rontolisp from their side. rontolisp ships those as **built-in ASDF systems**: `asdf:load-system`/`ql:quickload` (and a `:depends-on` from a real library) resolve the name to a bundled shim instead of downloading it. | System | What the shim provides | |--------|------------------------| | `usocket` | the socket API over `rontolisp:tcp-*` (see the [TCP guide](tcp-sockets.md#the-usocket-compatible-shim)) | | `trivial-gray-streams` | the portable Gray-stream classes/generics (base-class hierarchy incl. the binary/input classes and `trivial-gray-stream-mixin`, the read/write/sequence generics, `stream-file-position` + its `(setf ...)` writer), adapting onto rontolisp's own protocol — what the stream-taking built-ins dispatch to for a CLOS instance stream (see [Gray Streams](gray-streams.md)) | | `closer-mop` | the class-introspection readers over the real class metaobjects ([`find-class`](../reference/functions/find-class.md) / [`class-of`](../reference/functions/class-of.md) answers): `classp`, `class-slots`, `slot-definition-name`/`-initargs`/`-type`/`-readers`/`-initfunction`, `class-name`, `class-direct-superclasses`, `class-direct-slots`, `class-direct-subclasses`, `class-finalized-p`, `ensure-finalized`. A legacy tag-symbol designator still answers `(name declared-type)` pairs. The flat `closer-common-lisp` re-export package (nickname `c2cl`: all of `cl` overlaid with these, closer-mop winning collisions) is always registered, so a `(:use :closer-common-lisp)` package works | | `flexi-streams` | pass-through streams (a flexi stream IS the underlying stream) | | `babel` | the UTF-8 codec in both of babel's layers. The everyday one: `babel:string-to-octets`/`octets-to-string` (with `:start`/`:end`/`:errorp`), `babel:string-size-in-octets` (`:max` included), `babel-encodings:*default-character-encoding*` (`:utf-8`) and `babel:list-character-encodings`. The MAPPING layer underneath, for a library that decodes incrementally and has no whole octet vector to convert: `babel-encodings:get-character-encoding`, `enc-max-units-per-char`, `lookup-mapping` over `babel:*string-vector-mappings*`, and the `code-point-counter` / `octet-counter` / `decoder` / `encoder` it answers, plus the `babel:unicode-char` type. Malformed input signals babel's own conditions (`babel:character-decoding-error` and the leaves `end-of-input-in-character`, `character-out-of-range`, `invalid-utf8-starter-byte`, `invalid-utf8-continuation-byte`, `overlong-utf8-sequence`; `babel:character-encoding-error` on the way out), and `babel-encodings:*suppress-character-coding-errors*` — which every `:errorp` defaults from — replaces the signal with the substitution character. An encoding and a mapping are both just the encoding NAME here: real babel generates 40+ code pages from 20,000 lines of tables, while rontolisp has one character model (a character IS a Unicode code point, the wire form is UTF-8), so the shim implements that codec, treats the `:latin-1`/`:us-ascii` aliases as the code-point identity they are for the octets they can represent, and **signals on any other `:encoding`** rather than handing back mis-coded bytes | | `float-features` | `single-float-bits`/`bits-single-float` and the double variants over the IEEE 754 bit primitives (interpreter + JVM; the WASM numeric model cannot carry 64-bit bit patterns) | | `bordeaux-threads` (nicknames `bt` and `bt2`) | both API namespaces of the one shim. The locking subset — `make-lock`, `acquire-lock`, `release-lock`, `with-lock-held`, `*supports-threads-p*` — rides [`rontolisp:make-mutex`](../reference/functions/rontolisp-make-mutex.md) and friends; thread creation — `bt2:make-thread` (with `:initial-bindings`), `join-thread`, `threadp`, `thread-alive-p`, `destroy-thread` — rides [`rontolisp:make-thread`](../reference/functions/rontolisp-make-thread.md), a real virtual thread on the interpreter and the JVM. On the single-threaded WASM backends the thread entry points signal at call time and `bt:*supports-threads-p*` is `nil`. `make-lock` returns a reentrant lock (upstream's is not), `acquire-lock`'s `:wait-p` is ignored — the acquisition always blocks — and an `:initial-bindings` value form must be a quote form or self-evaluating (anything else would need the new thread's dynamic environment and signals) | | `uiop` | ASDF's portability layer, registered as 15 sub-packages and 429 exports. See **[The uiop Package](../reference/uiop.md)** for what is implemented; every other export resolves and signals `uiop:not-implemented-error` naming the operation, so a library that merely names one still loads | | `swank` | a stub, and only so that a library depending on it can load: `swank:create-server` signals ("rontolisp cannot serve a remote REPL") and `swank:stop-server` is a `nil` no-op. Real swank is SLIME's server half, whose own `.asd` is a program the defsystem-as-data front-end cannot read -- without the stub, `(ql:quickload "clack")` fetches the SLIME tarball and dies on it | | `mgl-pax-bootstrap` | the `mgl-pax` package (nickname `pax`) as a stub, so a library documented with [mgl-pax](https://github.com/melisgl/mgl-pax) can load (trivial-utf-8, a uuid dependency, hard-depends on it; the real system's `.asd` declares `:around-compile`, a compile hook outside the defsystem-as-data subset). `pax:define-package` acts as `defpackage`, `pax:defsection` defines its section name as a `nil` variable **and exports the section's `(symbol locative)` entries** — mgl-pax's documented default, and how such libraries export their public API — and the PAX-World registration helpers are `nil` no-ops. No documentation is generated | | `trivial-features` | the platform-feature announcement upstream trivial-features exists to make portable, without the CFFI probing: naming it in `:defsystem-depends-on` (or `:depends-on`) declares **`:unix`** and **`:little-endian`** for the depending system's own clauses and component files, and pushes both onto `*features*` at run time. `:unix` because every backend's file/path/environment surface is POSIX-shaped and none is Windows; `:little-endian` because the only places a program can see machine layout (WASM linear memory, a reactor's `:bytes` boundary) are little-endian by the wasm spec. The CPU names (`:x86-64`, `:64-bit`) are deliberately absent — rontolisp has no machine-word surface to describe. Upstream's own `.asd` cannot load here at all: it ends in `(error "Sorry, your Lisp is not supported")` for an implementation it does not recognize | | `trivial-garbage` (nickname `tg`) | GC finalizers as honest no-ops: `tg:finalize` registers nothing and returns the object, `tg:cancel-finalization` is a `nil` no-op. No backend exposes GC hooks — and Common Lisp gives finalizers no guarantee of ever running, so a conforming consumer must already work when they never fire. Practical consequence for `dbd-postgres` (its consumer): a leaked prepared statement lives until the connection closes; call `dbi:disconnect` explicitly | | `cl+ssl` | the CLIENT side of the TLS library every CL HTTP client (dexador, drakma, ...) reaches TLS through, over [`rontolisp:tls-upgrade`](../reference/functions/rontolisp-tls-upgrade.md) — the real cl+ssl is a CFFI binding to OpenSSL and cannot load here. `cl+ssl:make-ssl-client-stream` upgrades an already-connected stream to TLS against its `:hostname` (which is required); `make-context :verify-mode` + `with-global-context` + `ssl-check-verify-p` carry the verify mode, so a client's "insecure" knob (e.g. `dex:*not-verify-ssl*`) reaches the primitive's `:insecure`; `ensure-initialized` is a no-op. What has no backing **signals** instead of being accepted and ignored: client certificates (`:key`/`:certificate`/`:password`, and `use-certificate-chain-file`) and a `:verify-location` CA path — point the `javax.net.ssl.trustStore` system properties at your trust store instead. Runs on the interpreter, the JVM and the WASM `--component` backend (`tls-upgrade` rides `wasi:tls@0.3.0-draft` there — add `-S tls=y` to the run flags; a non-`nil` insecure knob signals, since the draft exposes no verification opt-out); WASM Preview 1 keeps the compile error | | `clack-handler-rontolisp` | the [Clack](https://github.com/fukamachi/clack) handler backend: package `clack.handler.rontolisp` exporting `run`/`stop`, bridging the Clack application protocol onto rontolisp's embedded HTTP server. You never load it by hand — `(clack:clackup app :server :rontolisp)` resolves it by name at run time (the system also answers to the dotted spelling `clack.handler.rontolisp` that clack derives from the package name). See the [Clack guide](clack.md) | | `clack-handler-reactor` | the Clack handler backend for a **host-driven reactor**: a Cloudflare Worker, a browser page, a node or JVM embedding — any host that has already parsed the request and calls an exported function instead of handing the program a socket. Package `clack.handler.reactor` exports `run`/`stop` and, under them, `handle` (an application and a JSON request string in, a JSON response string out) and `dispatch` (the same over the application `clackup` stored). Resolved by `(clack:clackup app :server :reactor)`, the dotted spelling included, exactly like the backend above. See the [Clack guide](clack.md#driving-the-reactor-by-hand-clack-handler-reactor) | The shims are deliberately thin: they satisfy what the loadable libraries actually call, not the full upstream APIs. ## What can I actually load? The real-world libraries below load unmodified today. The **Backends** column says where each one is verified — "all four" means the interpreter, the JVM, WASM Preview 1 and `--component`. **Notes** covers what is special about the load and what does not work. | Library | Backends | Notes | |---------|----------|-------| | [alexandria](https://gitlab.common-lisp.net/alexandria/alexandria) 1.0.1 | all four | The ecosystem's most-depended-on utility library, both packages (`alexandria`/`alexandria-1` and `alexandria-2`), from its real sources. Every library below with dependencies pulls it in. Absent are the members standing on a primitive that is still missing: `type=` (`subtypep`'s second value). `format-symbol`/`ensure-symbol` and `ensure-function` on a **symbol** work on the interpreter only (a compile-backend error, not a wrong answer). `shuffle`/`random-elt`/`gaussian-random` draw each backend's own entropy, so their output is not comparable across backends | | [split-sequence](https://github.com/sharplispers/split-sequence) v2.0.1 | all four | The whole API on strings and lists, including the second return value (the resume index). Its CLOS-only `extended-sequence.lisp` is gated behind `:if-feature (:or :sbcl :abcl)` and drops out automatically | | [parse-number](https://github.com/sharplispers/parse-number) v1.8 | all four | The whole API over integers, ratios, floats, radix-prefixed literals (`#xFF`, `#3r12`) and exponent markers; the `invalid-number` condition signals with the intended diagnostics | | [cl-utilities](https://common-lisp.net/project/cl-utilities/) v1.2.4 | all four | The whole public API — its own `split-sequence`, the `extremum` family, `read-delimited`, `expt-mod`, `collecting`/`with-collectors`, `with-unique-names`/`with-gensyms`/`once-only` (three-level nested backquote) usable from your own macros, `rotate-byte`, `copy-array`, `compose` | | [cl-who](https://edicl.github.io/cl-who/) v1.1.5 | all four | (X)HTML generation macros — `with-html-output(-to-string)` with attributes, nested tags and the local `str`/`esc`/`fmt`/`htm` operators; `:xml` and `:html5` both render correctly. **`:indent` (pretty-printed output) is unsupported**, and the output mode must be switched with **`(setf (html-mode) :html5)`**: cl-who reads it at macro-expansion time, so a runtime `let` on `*html-mode*` is not observed | | [cl-mustache](https://github.com/kanru/cl-mustache) 0.12.3 | all four | Mustache template renderer, verbatim — `render`/`render*` over string AND file templates, `compile-template` (parse once, render many), `define` (bind a renderer to a name) and `make-context` with `:data`/`:partials`. A context is an alist, a hash table or a chain of them; sections, inverted sections, partials, dynamic partial names (`{{>*name}}`) and lambda sections all render. Its own copy of the **194-case mustache spec suite scores 158/194 on every backend — the identical set SBCL passes**, so the 36 are upstream limits (null interpolation, dotted names pushing a context frame, and the whole 26-case inheritance module, which postdates the 1.1.2 spec cl-mustache targets), and `t/test-api.lisp` is 20/20. A missing partial `signal`s `partial-cant-be-found` offering a `use-value` restart, so an artifact that handles it compiles in EH mode (`-W exceptions=y` on both WASM backends) | | [assoc-utils](https://github.com/fukamachi/assoc-utils) | all four | Alist utilities, whole API — `aget` (settable), the alist/plist/hash conversions, `remove-from-alist`/`delete-from-alistf`, `with-keys`, `alist-get`, `alist=`, `alistp` | | [cl-base64](https://github.com/darabi/cl-base64) v3.4 | all four | Base64 over strings, `(unsigned-byte 8)` arrays and integers, with `:columns` wrapping and the `:uri` alphabet; a bad input character signals `bad-base64-character`. That condition's `:input`/`:position`/`:code` slots are readable on the interpreter only — the compiled backends signal a plain condition, caught by the same `handler-case` | | [md5](https://github.com/pmai/md5) v2.0.4 | all four | MD5 (RFC 1321) — `md5sum-sequence`/`md5sum-string` and the incremental API, matching the RFC test vectors identically on all four backends | | [chipz](https://github.com/froydnj/chipz) 0.8 | all four | Decompression — `chipz:decompress` for the `gzip`, `zlib` and `deflate` formats (to a fresh vector, into a supplied one, or incrementally through `make-dstate`), plus the CRC32/Adler-32 checksum entry points. The inflate state machine is a `labels` whose transitions store `#'local-function` in a struct slot, and it exits through `catch`/`throw`, so a compiled artifact is always in EH mode (`-W exceptions=y` on both WASM backends). bzip2 loads with it — `decompress`'s own `typecase` names `bzip2-state` — but is untested here. [`size-report/programs/zlib`](https://github.com/making/rontolisp/tree/develop/size-report/programs/zlib) is built on it | | [cl-ppcre](https://github.com/edicl/cl-ppcre) v2.1.2 | all four | Perl-compatible regular expressions from its real sources — `scan`, `scan-to-strings`, `split`, `regex-replace(-all)`, `all-matches`, `count-matches`, the `do-scans`/`do-matches` macros, `register-groups-bind`, `quote-meta-chars`, parse-tree regexes and inline modifiers like `(?i)` | | [com.inuoe.jzon](https://github.com/Zulu-Inuoe/jzon) v1.1.4 | all four | JSON parsing and stringification including the README walkthrough — hash-table / vector round-trips, `:key-fn`, the Gray-stream `:stream` writer, `jzon:writer`, CLOS-instance stringification; its dependencies resolve to the built-in shim systems above. Its three numeric leaf components (the eisel-lemire reader and Schubfach printer) are replaced by shims over rontolisp's own float arithmetic — float output is rontolisp's own shortest round-trip decimal, identical on every backend — and an extreme exponent can be a few ulps off when parsing (a decimal exponent of magnitude 22 or less rounds exactly). The usual WASM caveats apply: hash-table iteration order, non-ASCII `\u` escapes | | [ironclad](https://github.com/sharplispers/ironclad) v0.61 (the SHA-2 / HMAC / PBKDF2 / HKDF / SCRAM / RSA slice) | all four | From its real sources, reproducing the published FIPS 180-2, RFC 4231, RFC 5869 and RFC 7677 vectors — including SCRAM-SHA-256's client proof end to end, the sequence a PostgreSQL client authenticates with. The digests are SHA-224/256/384/512 and the MACs and KDFs over them (`hmac`, `hmac-kdf`, `pbkdf2`, `pbkdf2-hash-password`); the RSA stack is real too — `generate-key-pair`, `sign-message`/`verify-signature` and `encrypt-message`/`decrypt-message`, with and without PSS/OAEP. Only that slice loads (its own `.asd` is an executable program, so a bundled replacement declares the slice): **the ciphers, the AEAD modes, the other public-key algorithms (DSA, ElGamal, the elliptic curves, ed25519) and the other digest families are absent**, and requesting one signals at the call. `prng.lisp` is narrowed to the OS-entropy surface over `rontolisp:random-bytes` — nonces, RSA key generation and the PSS salt are cryptographically strong everywhere, but `:fortuna` and the seed-file operations are gone. PBKDF2 runs on a native kernel on the interpreter — the same bytes, roughly three orders of magnitude faster — so password hashing and SCRAM authentication are not interpreter-bound | | [uax-15](https://github.com/sabracrolleton/uax-15) v0.1.3 | all four | Unicode normalization (UAX #15) in all four forms from its real sources; **`--system-path` needs three directories** (uax-15, split-sequence, cl-ppcre, joined with `:`). Upstream builds its tables by parsing 2.7 MB of bundled Unicode text at load time (minutes interpreted); rontolisp derives the same tables from the same files at compile/load time and builds each one on first read, leaving every normalization function verbatim — so loading is nearly free and a program that never normalizes pays nothing. One deliberate difference, and it is a fix: `(unicode-letter-p #\A)` answers `T` where the real load answers `NIL` (upstream's loop reads `#+utf-32`). `get-mapping` signals on every backend — it is broken upstream and nothing calls it | | [quri](https://github.com/fukamachi/quri) v0.7.0 | all four | URI library from its real sources via `(ql:quickload "quri")` — parsing into the scheme-specific structs, the accessors, `render-uri`, `merge-uris`, `uri-query-params`, percent-encoding, the public-suffix API and the address predicates. Its `babel` dependency resolves to the built-in UTF-8 shim, so a non-UTF-8 `:encoding` signals; the effective-TLD tables build on first read from the bundled 152 KB list, so `(load-etld-data OTHER-FILE)` reads that list rather than `OTHER-FILE`. `:lenient` percent-decoding skips a bad escape with a `go` out of a `handler-bind` handler, which the compile backends lower to a non-local exit, so it answers the same on all four. Needs alexandria, split-sequence, cl-utilities and idna on `--system-path` | | [local-time](https://github.com/dlowe-net/local-time) v1.0.6 | all four | Date/time library from its real sources via `(ql:quickload "local-time")` — `encode-timestamp`/`decode-timestamp`, `now`/`today`, the unix and universal-time conversions, `parse-timestring`, `format-timestring` over every bundled format (ISO 8601, RFC 3339, RFC 1123, asctime, ISO week date) and custom format lists, the comparison family, `timestamp+`/`timestamp-`/`adjust-timestamp`/`timestamp-minimize-part`, the julian-date pair and `print-object`. Its only dependency is the built-in `uiop`. **Real TZif zone files load wherever the host has a filesystem** — `(local-time:define-timezone tokyo #p"/usr/share/zoneinfo/Asia/Tokyo" :load t)` — and the load-time `/etc/localtime` read that seeds `*default-timezone*` works the same way, falling back to `+utc-zone+` where the file cannot be read (which is what the WASM backends do without `--dir`). **`reread-timezone-repository` walks the bundled `zoneinfo/` tree on all four backends** now that `directory` exists, so `find-timezone-by-location-name` resolves `"Asia/Tokyo"` and friends; on the compiled backends pass the repository explicitly (`(local-time:reread-timezone-repository :timezone-repository "zoneinfo/")`) because its default is computed at load time from `asdf:component-pathname` through a run-time `eval`, with `*load-truename*` as the fallback — neither of which the compiled backends can answer, so the default is `nil` there | | [trivia](https://github.com/guicho271828/trivia) (the `trivia.trivial` route) | all four | Optima-compatible pattern matching from its real sources via `(ql:quickload "trivia")` — `match`/`match*`/`ematch` (failure signals `match-error`), constant / variable / `cons` / `list` / `list*` / `vector` patterns, `guard`, `or`/`and`/`not` patterns, `defpattern`, struct patterns (keyword and conc-name shapes), class patterns (keyword slot and `(class name (slot var))` shapes) and `(type spec)` patterns. System `trivia` is mapped to `trivia.trivial` — upstream's own base system for extensions — so clauses run under the `:trivial` optimizer: identical semantics, no balland2006 clause optimization (which would need `iterate` + `type-i`). Its dependencies (alexandria, lisp-namespace, the closer-mop / trivial-cltl2 shims) load with it. Note the interpreter re-expands macros per evaluation, so a hot `match` loop belongs on a compiled backend | | [sxql](https://github.com/fukamachi/sxql) | all four | SQL generator from its verbatim sources via `(ql:quickload "sxql")` — `sxql:yield` returns the SQL string plus the bind-value list as multiple values, byte-identically on every backend (and identically to SBCL on the same sources): `select` with `from`/`where` (incl. `:and`/`:or`/`:in`/`:like`), `order-by` (`:desc`, `nulls`), `limit`/`offset`, `left-join ... :on`, `insert-into` with `set=`, `update`, `delete-from`, `create-table` with column options (the mito `deftable` shape), `drop-table` and `alter-table`. Its dependencies (trivia via the `trivia.trivial` route, alexandria, cl-package-locks — the last a no-op-shaped lock library) load with it. Like every macro-heavy library, hot query construction belongs on a compiled backend (the interpreter re-expands macros per evaluation). The [O/R mapping guide](mito.md) walks through `yield` and the statement builders | | [esrap](https://github.com/scymtym/esrap) 0.19 | all four | Packrat / PEG parser from its verbatim sources via `(ql:quickload "esrap")` — `esrap:parse` over an inline expression or a named rule, `defrule` with `:lambda` / `:destructure` / `:text` transforms, `add-rule` / `make-instance 'esrap:rule`, case-insensitive `(~ "lit")` terminals, `and` / `or` / `not` / `*` / `+` / `?` sequencing, semantic predicates (`(oddp decimal)`), `:junk-allowed`, and the accurate parse-error report (`esrap:esrap-parse-error`, whose text is byte-identical to SBCL's apart from SBCL's non-standard Unicode character NAMES). The parser is pure computation, so **Preview 1 WASM is in** — no sockets, no flags beyond `-W gc`. Its dependencies (alexandria, trivial-with-current-source-form) load with it. `esrap:trace-rule` needs `break`, which does not exist here, and the swank indentation hook needs `set`; both are dead unless called | | [postmodern](https://github.com/marijnh/Postmodern) v1.33.12 (the MOP build) | interpreter, JVM, WASM component | PostgreSQL stack — s-sql included — from its verbatim upstream sources via `(ql:quickload "postmodern")`: `with-connection`/`connect` and the pool, `query`/`execute` over S-SQL forms or strings in every result style, `doquery`, prepared statements with the `:reconnect`/`reset-prepared-statement` restarts, transactions and savepoints, `execute-file`, `deftable`, and `:postmodern-thread-safe` ON so its locks really serialize. The **DAO layer is in**: the build takes `:postmodern-use-mop` ON, so `table.lisp` loads verbatim over the static metaobject subset — `(defclass ... (:metaclass pomo:dao-class))` with `:col-type`/`:keys`/`:table-name`, `dao-table-definition`, `deftable`'s `!dao-def`, `insert-dao`/`get-dao`/`update-dao`/`upsert-dao`/`delete-dao`/`save-dao`/`select-dao`/`query-dao` and `make-dao`. The metaclass protocol runs at DEFINITION time, so DAO classes must be top-level `defclass` forms with literal options (classes built from runtime data signal), and `finalize-inheritance` runs eagerly at class definition rather than at first use — definition errors surface earlier, results are unchanged. Connecting needs cl-postgres' socket layer, so **Preview 1 WASM is out**; both wasm run commands need `-W exceptions=y` and a `--component` one additionally `-S tcp=y -S inherit-network=y`. The s-sql layer alone (`(ql:quickload "s-sql")`) opens no sockets and renders identical SQL on all four backends | | [clack](https://github.com/fukamachi/clack) v2.1.0 (with [lack](https://github.com/fukamachi/lack)) | interpreter, JVM, WASM component | Web application environment from its verbatim upstream sources via `(ql:quickload "clack")`, served by the built-in `clack-handler-rontolisp` backend — see the [Clack guide](clack.md). The lack side loads too: `lack:builder`, `lack-util`'s `generate-random-id` (over the ironclad slice) and the backtrace middleware, which `clackup`'s default `:use-default-middlewares t` exercises end to end. `clackup`'s default `:use-thread t` runs the acceptor on a real thread ([`rontolisp:make-thread`](../reference/functions/rontolisp-make-thread.md)) on the interpreter and the JVM; the WASM component serves under `wasmtime serve` instead (the host owns the socket). Preview 1 WASM has no incoming TCP by design, so `clackup` signals at call time there | | [tiny-routes](https://github.com/jeko2000/tiny-routes) v0.1.1 | all four | A routing layer for Clack applications, from its verbatim sources via `(ql:quickload "tiny-routes")` — the piece between `clack:clackup` and an application with routes. `define-get`/`define-post`/`define-put`/`define-delete`/`define-any`/`define-route` and `define-routes`, the `:id`-style path template (and a regex one with `:regex t`) over cl-ppcre, `path-parameter`, `with-request`/`with-path-parameters`, the `pipe` middleware combinator with `wrap-request-body` (the Clack `:raw-body` stream), `wrap-query-parameters`, `wrap-request-predicate`/`-mapper`, the response wrappers and the whole `ok`/`created`/`not-found`/… constructor set. Its companion system `tiny-routes-middleware-cookie` loads too (`parse-cookie-header`, `write-set-cookie-header`, `wrap-request-cookies`, `wrap-response-cookies`), pulling in cl-cookie, quri, local-time and proc-parse. Routing itself is pure computation, so **all four backends are in**; SERVING the routes needs `clackup`, which rules Preview 1 out — see the [Clack guide](clack.md). Its only dependency is cl-ppcre, so `--system-path` needs two directories when you load it from disk. The test system needs fiveam, which does not load. For a size-constrained compiled module there is a ppcre-free **opt-in**, `tiny-routes/lite` — the subsection right below | | [ningle](https://github.com/fukamachi/ningle) v0.3.0 | all four | The "super micro framework" over Clack, from its verbatim sources via `(ql:quickload "ningle")` — the second routing layer here, and a genuinely different model from tiny-routes rather than another spelling. The application is a CLOS OBJECT (`(make-instance 'ningle:app)`, a `lack-component`), every route is a `setf` (`(setf (ningle:route app "/x") controller)`), a controller receives the matched PARAMETERS rather than the environment, and a controller that is not a function is answered as the response body. Path templates with `:name` tokens and `*` splats, `:regexp t` routes, `:method`, `:accept` content negotiation and user-defined requirements (`(setf (ningle:requirement app :key) fn)`) — a route can therefore be selected by something that is not the path at all — plus the `ningle:*request*`/`*response*`/`*session*` specials, `ningle:context` and `with-context-variables`, `ningle:next-route`, and `ningle:not-found`, the overridable 404 method. Its router [myway](https://github.com/fukamachi/myway) and myway's `map-set` load with it, as does the whole lack request chain it reads every request through (http-body, fast-http, smart-buffer, circular-streams, quri, yason, trivial-mimes) — which is why a compiled ningle module is an order of magnitude larger than the same routes through tiny-routes, and there is **no size opt-in** to offer: myway compiles every rule to a cl-ppcre scanner, so the regex engine is genuinely reachable. Routing itself is pure computation, so **all four backends are in**; SERVING the routes needs `clackup`, which rules Preview 1 out — see the [Clack guide](clack.md) | | [cl-dbi](https://github.com/fukamachi/cl-dbi) 0.11.1 (`dbd-postgres` only) | interpreter, JVM, WASM component | Database-independent interface from its verbatim sources via `(ql:quickload "dbd-postgres")`: `dbi:connect` (the driver resolves over the already-loaded system — a compiled program must contain the `ql:quickload` itself, since it cannot load a system at run time), `dbi:do-sql`, `dbi:prepare`/`execute`/`fetch`/`fetch-all`, `dbi:with-transaction` (commit and rollback), `dbi:connect-cached` and `dbi:disconnect`. The `:mysql` and `:sqlite3` drivers need FFI and are absent. On the thread-capable backends the connection cache is per-thread (`cache/thread.lisp` over the `bt2` shim's real locks and [`rontolisp:current-thread`](../reference/functions/rontolisp-current-thread.md)); the single-threaded WASM backends use upstream's own threadless cache. Its `trivial-garbage` dependency resolves to the no-op finalizer shim above, so call `dbi:disconnect` explicitly. Same socket constraints as postmodern: Preview 1 WASM is out, a component needs `-W exceptions=y -S tcp=y -S inherit-network=y` | | [mito](https://github.com/fukamachi/mito) 0.2.0 | interpreter, JVM, WASM component | O/R mapper from its verbatim sources via `(ql:quickload "mito")` — the **full** system (`mito-core` + `mito-migration` + `lack-middleware-mito`), covered by the [O/R mapping guide](mito.md). The DAO layer: `connect-toplevel`/`disconnect-toplevel`, `deftable` (the `dao-table-class` metaclass over the static metaobject protocol — auto-pk `:serial` and `:uuid`, `record-timestamps-mixin`'s `created-at`/`updated-at`), `table-definition`, `ensure-table-exists`, `create-dao`/`insert-dao`/`save-dao`/`delete-dao`, `find-dao`, `select-dao` with sxql clauses, `object-id`, `retrieve-by-sql` and `execute-sql`. The migration layer: `migration-expressions` and `migrate-table` diff a class against the live schema on all three backends, while `generate-migrations` / `migrate` over migration FILES are interpreter + JVM (the WASM backends import no directory-creation or file-removal call, so they signal at the call); `migrate` re-reads the generated `.sql` with esrap, and the advisory lock rides a CRC32-only slice of chipz. PostgreSQL only (`dbd-postgres`, which must be quickloaded explicitly); like every metaclass consumer, `deftable` forms must be top level with literal options. Known gaps, all in the guide: the `:conc-name` accessors are not generated (`slot-value` works), and sxql's SQL FUNCTION operators — `(:count ...)` and therefore `mito:count-dao` — are interpreter-only. Two shapes fail identically on SBCL and are upstream defects, not gaps: a bare `:references` without a `:col-type`, and adding a NOT NULL column with an `:initform`. The uuid dependency loads (its v1/v4 generation draws the backend's own entropy) and `dissect`'s stack introspection is the no-op interface. Same socket constraints as cl-dbi: Preview 1 WASM is out, a component needs `-W exceptions=y -S tcp=y -S inherit-network=y` | | [rove](https://github.com/fukamachi/rove) v0.10.0 | all four | Testing framework from its verbatim sources via `(ql:quickload "rove")`, covered by the [testing guide](testing.md) — `deftest`/`testing`/`ok`/`ng`/`signals`/`outputs`/`expands`/`pass`/`fail`/`skip`/`failing`/`setup`/`teardown`/`defhook`/`diag` with the `:spec` (default) and `:dot` reporters, and every entry point: `rove:run` over a `:package-inferred-system` or a plain `defsystem` test system, `run-test`/`run-tests`, and `run-suite` at the end of a test file. A test body that signals becomes a recorded failure instead of ending the run (on the WASM backends a raw trap — `(car 1)`, `(/ 1 0)` — still ends it), and `rontolisp test TARGET` (or `(uiop:quit (if (rove:run ...) 0 1))` inside your own runner) turns the result into a CI exit code on every backend. Its dissect dependency loads from its real sources with the stack introspection empty, so failure reports carry no backtraces, and assertion descriptions print symbols package-qualified where SBCL prints them bare — details and the run commands are in the guide | | [jose](https://github.com/fukamachi/jose) | all four | JSON Object Signing and Encryption (JWS/JWT) from its verbatim sources via `(ql:quickload "jose")` — `encode`/`decode`/`inspect-token` over HS256/384/512, RS256/384/512, PS256/384/512 and the unsecured `none`, with the registered claim checks (`iat`/`nbf`/`exp`/`jti`, plus the `:issuer`/`:audience`/`:subject` keywords). The tokens are byte-identical to SBCL's on the same sources and to Python's `hmac`/`hashlib` — the HS256 one is the token jose's own README publishes. RSA keys are ironclad objects, so `ironclad:generate-key-pair :rsa` (or any parser that produces one) supplies them; PS512 needs a modulus of at least 1040 bits, which is ironclad's own assertion, not a limit here. An expired `exp`, an `nbf` in the future and a failed signature all signal through `cerror`, so a `handler-bind` that `continue`s decodes anyway — and any program handling them compiles in EH mode (`-W exceptions=y` on both WASM backends). Upstream's own `jose/tests/jwt` suite runs green under `rontolisp test` on all four backends; its `jose/tests/jws` sibling runs nowhere, on any implementation — it `(:use #:pem)`, and `pem` is not in the Quicklisp distribution. Loading from vendored sources instead of the Quicklisp cache needs **eight directories** in `--system-path`: jose, cl-json, ironclad, cl-base64, split-sequence, assoc-utils, alexandria and trivial-utf-8 | ### The size opt-in: `tiny-routes/lite` `(ql:quickload "tiny-routes/lite")` loads the same tiny-routes tree with one component substituted — `path-template.lisp`, whose matcher upstream is a cl-ppcre scanner — and the `:cl-ppcre` dependency dropped with it. It exists because routing keeps the regex engine **live**: a route template compiles to a scanner when the route is *built*, so in a compiled module no amount of tree-shaking can remove cl-ppcre, and on a size-limited target that is most of the module — the [routed Worker example](https://github.com/making/rontolisp/tree/develop/examples/cloudflare-workers/httpbin-tiny-routes) measures 974,530 B raw with the full system and 408,448 B with the lite one, same routes, same answers, request for request. The substitution never changes what a template *matches* — it matches identically to the full system, or it refuses loudly when the route is built: - **Accepted**: templates made of literal characters and `:name` tokens — a token is `:` followed by a letter or `_`, continuing over letters, digits, `_` and `-`, anywhere in the template: `/users/:id`, `/files/v:version`, `/pair/:a/:b`. Within this subset the lite matcher reproduces the full system's semantics exactly, greedy backtracking and upstream's greedy token-*name* scan included (in `/a/:x-:y` the first token is named `x-`); the two engines are pinned template-for-template by the test suite. - **Refused at route-build time**, with an error naming the escape: a template containing any regex metacharacter — `.` `\` `[` `]` `(` `)` `{` `}` `|` `^` `$` `*` `+` `?` — and every `:regex t` template. (A template with no `:name` token is never a regex upstream either — it is compared with `string=` — so metacharacters there are fine on both systems.) Plain `(ql:quickload "tiny-routes")` is untouched — the verbatim library, cl-ppcre included — and the two systems refuse to load into one program, in either order (whichever loaded last would silently redefine the matcher). `tiny-routes/lite` is not in the Quicklisp index; the name downloads the tiny-routes release and resolves against its `.asd`. cl-ppcre's load drove the widest feature batch so far — local `(declare (special ...))`, CLOS slot accessors as generics, `initialize-instance :after`, `&environment` + `get-setf-expansion`, `psetf`, `(setf (subseq ...))`, `subst`/`search`/`copy-tree` and the descending/case-insensitive character comparisons. uax-15's load drove the second widest: compile-time folding of the ASDF/UIOP pathname primitives, inlining a bundled data file read with `with-open-file` into the artifact, a per-clause rewrite of the `LOOP` macro, and the UTF-8 byte model behind WASM GC strings. alexandria's is the batch every other library inherits, because everything above that has dependencies depends on it: `&whole` in `defmacro`/`destructuring-bind`, a destructuring pattern after `&rest`/`&body` (`if-let`), `lambda-list-keywords`, `do-external-symbols`, `intern` with a package designator, the hash-table introspection readers (`hash-table-test`/`-size`/`-rehash-size`/`-rehash-threshold`), `mismatch`, `arrayp`, `with-open-stream` and `#'open` as a first-class value — plus, for `mappend`, `#'mapcar` as a value over more than one list. Runnable demos for sixteen of them — with the per-backend commands and expected output — live in [`examples/asdf/`](https://github.com/making/rontolisp/tree/develop/examples/asdf). A library qualifies today roughly when it stays inside: plain `defun`/`defmacro`/`defpackage` code, `loop`, `multiple-value-bind` over `values`-tailed functions, `check-type`/`etypecase` with the supported type specifiers, declarations (parsed no-ops, `deftype` included), the CLOS static subset (`defclass`/`defgeneric`/`defmethod`/`make-instance`/`slot-value` with single dispatch, plus `(defun (setf name) ...)` setf functions), the condition and restart system (`define-condition`/`handler-case`/`handler-bind`/`restart-case`/ `invoke-restart`), `return-from`, and dynamic (special) variable binding (`let`/`let*` over a `defvar` special). Libraries built on the full metaobject protocol or the interactive debugger (`break`, `*debugger-hook*`) do not load yet (see [Unsupported CL Features](missing-features.md)). For anything else, the practical use is structuring **your own** multi-file rontolisp projects — with `.asd` files that real ASDF can read too. --- # FILE: references/guides/async.md # Asynchronous Programming (async / await / futures) The `rontolisp` package provides a small asynchronous surface modeled on JavaScript promises and `async`/`await`, expressed in Lisp. None of it is part of Common Lisp; reference every operator with the `rontolisp:` qualifier (see [Packages](../reference/packages.md)). The unit of the model is a **future**: a value standing in for a computation that may not have finished yet. Calling an [`rontolisp:async-defun`](../reference/special-forms/rontolisp-async-defun.md) returns one, [`rontolisp:await`](../reference/special-forms/rontolisp-await.md) resolves it, and a handful of combinators build on top. | Operator | Purpose | |----------|---------| | [`rontolisp:async-defun`](../reference/special-forms/rontolisp-async-defun.md) | Define an asynchronous function (returns a future) | | [`rontolisp:async-lambda`](../reference/special-forms/rontolisp-async-lambda.md) | The anonymous counterpart | | [`rontolisp:async`](../reference/special-forms/rontolisp-async.md) | `(async (defun ...))` / `(async (lambda ...))` — a JavaScript-style spelling of the two above | | [`rontolisp:await`](../reference/special-forms/rontolisp-await.md) | Suspend until a future settles and return its value | | [`rontolisp:futurep`](../reference/functions/rontolisp-futurep.md) | `t` if a value is a future | | [`rontolisp:wait-for`](../reference/functions/rontolisp-wait-for.md) | A future that settles to `nil` after N milliseconds (the async counterpart of `cl:sleep`) | | [`rontolisp:then`](../reference/functions/rontolisp-then.md) / [`then*`](../reference/functions/rontolisp-then-star.md) | Attach a transform to a future *as a value* | | [`rontolisp:catch`](../reference/functions/rontolisp-catch.md) | Attach an error fallback to a future as a value | | [`rontolisp:finally`](../reference/functions/rontolisp-finally.md) | Run a cleanup thunk on both the success and error channels | | [`rontolisp:make-stream`](../reference/functions/rontolisp-make-stream.md) / [`stream-read`](../reference/functions/rontolisp-stream-read.md) / [`stream-write`](../reference/functions/rontolisp-stream-write.md) / [`stream-close`](../reference/functions/rontolisp-stream-close.md) / [`read-all`](../reference/functions/rontolisp-read-all.md) | Asynchronous byte/string streams | > **Backend support.** The whole surface works on the interpreter, the JVM > backend and the WASM `--component` backend, but the machinery under it > differs. On the **interpreter and JVM** an async body runs on a virtual > thread — after its first suspension it runs in *real parallelism* with the > caller. On **`--component`** the body compiles into a cooperative, > single-threaded state machine over the WASI 0.3 component-model async ABI > ([see below](#under-the-hood-wasi-preview-3-futures--streams)); such a > component must run with `wasmtime -W exceptions=y`. **Preview 1** WASM has no > asynchronous host I/O, so async bodies run to completion eagerly (a > degenerate-but-observably-consistent synchronous mode), and `wait-for` / > the guest stream operations are compile errors there. **`--no-gc`** rejects > the whole async surface at compile time. ## Futures and eager start `rontolisp:async-defun` defines a function whose *call* starts the body immediately and hands back a future rather than a value. The body runs until its first `await` of an unsettled future (or until it finishes) — "eager start" — then the caller resumes: ```lisp (rontolisp:async-defun add-later (a b) (+ a b)) (rontolisp:await (add-later 20 22)) ; => 42 ``` The call itself is an opaque future — `rontolisp:futurep` recognizes it, and it prints as `#`: ```lisp (rontolisp:futurep (add-later 1 2)) ; => T ``` The future settles with the value of the last body form, or with the error the body signaled (re-signaled when the future is awaited — see [Errors](#errors-across-the-await-barrier)). The anonymous counterpart is [`rontolisp:async-lambda`](../reference/special-forms/rontolisp-async-lambda.md), and `(rontolisp:async (defun ...))` / `(rontolisp:async (lambda ...))` is an equivalent JavaScript-flavored spelling of the two. ## Awaiting `rontolisp:await` suspends the current asynchronous function until a future settles and returns its settled value. It is *generic*: a settled future never suspends, nested futures flatten, and a value that is not a future passes through unchanged — so `await` can be applied uniformly to a value that may or may not be a future. ```lisp (rontolisp:await 42) ; => 42 ``` `await` placement is **lexical**: it is legal only inside an `async-defun`/`async-lambda` body, or at top level (which is implicitly asynchronous). In any plain `defun`/`lambda` — even one nested inside an asynchronous body — it is an error at definition time: ```console > (defun bad () (rontolisp:await 1)) rontolisp:await is only allowed inside rontolisp:async-defun/async-lambda or at top level ``` ### Errors across the await barrier An error signaled by an async body does not escape at call time; it settles the future and re-signals the condition at the `await`. Catch it with a `handler-case` around the await — condition-type dispatch works across the barrier: ```lisp (rontolisp:async-defun failing () (error "boom")) (handler-case (rontolisp:await (failing)) (error (e) (declare (ignore e)) "caught")) ; => "caught" ``` ## Overlapping work Because a call is already running when it returns its future, several asynchronous operations overlap — start them all, then await each (in any order). The clearest illustration is `rontolisp:wait-for`, which returns a future settling after a delay: it is the asynchronous counterpart of `cl:sleep` (which *blocks* the whole program and takes seconds). Timers run concurrently, so two started together settle in delay order, not start order, and awaiting both takes about the longer delay, not the sum: ```lisp (rontolisp:async-defun delayed (ms tag) (rontolisp:await (rontolisp:wait-for ms)) tag) (let ((slow (delayed 200 "slow")) (fast (delayed 20 "fast"))) ; both timers now running (list (rontolisp:await fast) (rontolisp:await slow))) ; => ("fast" "slow") ``` The same overlap is what makes several [`rontolisp:fetch`](http-fetch.md) requests run in parallel: start them, then await the responses. ## Composing futures as values (then / then* / catch / finally) `await` is the right tool when the future is right there in your async body. But a future is also a first-class value that can cross a boundary — be returned, stored, passed around — and the caller on the other side need not itself be an `async-defun` just because its callee is one. The combinator quartet transforms a future *as a value*, each returning a fresh future: - [`rontolisp:then`](../reference/functions/rontolisp-then.md) attaches a success transform. On the input's successful settlement it invokes the function with the settled value and settles to the result; on an upstream error the callback is skipped and the condition propagates unchanged. If the function itself returns a future, `await` flattens it (no `future>`): ```lisp (rontolisp:async-defun some-future-producer () 21) (defun caller () ; a PLAIN defun, not async (rontolisp:then (some-future-producer) (lambda (v) (* 2 v)))) (rontolisp:await (caller)) ; => 42 ``` - [`rontolisp:then*`](../reference/functions/rontolisp-then-star.md) is variadic chain sugar — thread a value through several stages without the nesting a manual chain would need: ```lisp (rontolisp:async-defun produce () 40) (rontolisp:await (rontolisp:then* (produce) #'1+ #'1+)) ; => 42 ``` - [`rontolisp:catch`](../reference/functions/rontolisp-catch.md) attaches an error fallback (JavaScript `.catch`); a successful value passes through unchanged: ```lisp (rontolisp:async-defun boom () (error "nope")) (rontolisp:await (rontolisp:catch (boom) (lambda (c) (declare (ignore c)) :fallback))) ; => :FALLBACK ``` - [`rontolisp:finally`](../reference/functions/rontolisp-finally.md) runs a zero-argument cleanup thunk on *both* the success and error channels; the original outcome carries through (like `unwind-protect`): ```lisp (defvar *cleanup-log* nil) (rontolisp:async-defun make-value () 5) (let ((v (rontolisp:await (rontolisp:finally (make-value) (lambda () (push :done *cleanup-log*)))))) (list v (reverse *cleanup-log*))) ; => (5 (:DONE)) ``` A non-future first argument to any of the four is a `type-error` — there is no JavaScript-style auto-coercion to a resolved promise. And note that `rontolisp:catch` is *not* Common Lisp's [`catch`](../reference/special-forms/catch.md)/[`throw`](../reference/special-forms/throw.md) tag-based special form: they live in different packages and qualified names never collide (see the [catch reference page](../reference/functions/rontolisp-catch.md) for the naming details). ## Asynchronous streams Where a future settles once, a **stream** delivers a sequence of chunks over time. A guest-created stream is one value owning both ends: producers append with [`rontolisp:stream-write`](../reference/functions/rontolisp-stream-write.md) and finish with [`rontolisp:stream-close`](../reference/functions/rontolisp-stream-close.md); consumers take chunks with [`rontolisp:stream-read`](../reference/functions/rontolisp-stream-read.md) (each read yields a future) or drain the chunks into one string in one await with [`rontolisp:read-all`](../reference/functions/rontolisp-read-all.md): ```lisp (let ((s (rontolisp:make-stream))) (rontolisp:stream-write s "hello ") (rontolisp:stream-write s "world") (rontolisp:stream-close s) (rontolisp:await (rontolisp:read-all s))) ; => "hello world" ``` `stream-read` returns a future settling to the next chunk, or to `nil` once the stream is closed and drained — chunks are never `nil`, so a `nil` result always means end of stream. A read on an open, empty stream stays pending until a write arrives; that pending read is the suspension an awaiting async function parks on. Guest-created streams (`make-stream` / `stream-write`) exist on the interpreter and the JVM backend. On `--component` the stream *operations* work too, but the streams themselves arrive from the host: a [`rontolisp:fetch`](http-fetch.md) response `:body` and a [`rontolisp:http-handler`](http-handler.md) request `:raw-body` (in its default `:stream` mode) are asynchronous streams on every backend. Those HTTP body streams are **byte streams**: each chunk is an `(unsigned-byte 8)` vector holding the octets as they came off the wire, so a handler that answers a fetched `:body` as its own response body relays it byte-exact, and `read-all` is where the bytes become text (one UTF-8 decode of the whole body, so a chunk boundary inside a code point costs nothing). ## Under the hood: WASI Preview 3 futures & streams The `--component` backend is the one place where the async model maps onto a platform primitive rather than onto host threads. A WASI 0.3 (Preview 3) component builds on the component-model **async canonical ABI**, whose two built-in parametric types are `future` (a one-shot asynchronous result) and `stream` (a sequence of chunks). rontolisp's futures and asynchronous streams lower directly onto them: - An `async-defun` / `async-lambda` body (and a top level containing `await`) compiles into an **entry + resume state machine** over first-class component-model futures. An `await` of a value that is already settled just continues; an `await` of a pending host operation genuinely **suspends the task**, and the component's event loop resumes it when the awaited event arrives. Tasks are **cooperative and single-threaded** — two in-flight operations of one component instance interleave, but never preempt each other. This is the deliberate divergence from the interpreter/JVM's virtual threads, where a body runs in *real* parallelism after its first suspension and racing on shared global state is the program's own responsibility. - `rontolisp:wait-for` lowers to the host timer, `wasi:clocks/monotonic-clock@0.3.0`'s `wait-for`, returned as a pending future the event loop settles — which is why timers genuinely overlap in a component too. - A fetch response's `:body` / a served request's `:raw-body` is a component-model `stream` wrapped as a rontolisp stream; `stream-read` of a chunk the host still has in flight is a pending future, so a slow body read parks only its own task while another task's timer or fetch keeps running. Because the async ABI uses the component-model exception mechanism, any async component must run with `wasmtime -W exceptions=y` on top of `-W gc=y`. All of this rides the base component-model async support enabled by default in wasmtime 46+ — no experimental feature flags remain. See the [WASI 0.3 Component guide](wasm-component.md) for the component runtime as a whole. **Preview 1** WASM has none of this — there is no asynchronous host I/O in a Preview 1 core module — so an async body simply runs to completion the moment it is called, and its future is born already settled. The observable behavior matches the other backends whenever an `await` is adjacent to the call that produced the future (the common shape); it diverges only in that an error signals at the *call* rather than at the `await`, and `wait-for` / the guest stream operations are rejected at compile time. **`--no-gc`** rejects the entire async surface by name. A **`--no-wasi` reactor** is a Preview 1 module, so those degenerate futures are what it has — and yet it is the one Preview 1 build that does real asynchronous host work, because the *host* does the waiting rather than the guest. [`--host-fetch`](http-fetch.md#fetching-from-a-reactor---no-wasi---host-fetch) routes `rontolisp:fetch` at a pair of host imports (the head, then the body a chunk at a time), and a JavaScript host implements them with `WebAssembly.Suspending` (JSPI): the whole wasm stack parks until the promise settles, so `(await (fetch ...))` reads exactly as it does everywhere else, and by the time `fetch` returns its future — the reply's HEAD — is already settled. The price is paid on the host side, not in the Lisp — every export must be entered through `WebAssembly.promising` and calls must be serialised (a re-entered export refuses with a trap), and nothing on the **load path** may fetch, because `_initialize` is the one stack a suspending host cannot park. ## Where async shows up The async surface is small on purpose; most programs meet it through one of the I/O features built on it: - [HTTP Requests (fetch)](http-fetch.md) — `fetch` returns a future; a request body is drained with `read-all`. - [Serving HTTP (http-handler)](http-handler.md) — a handler that awaits (for example, one that fetches) must itself be an `async-defun`. - [TCP Sockets](tcp-sockets.md) — a pending `tcp-accept` or socket read suspends only its own task inside a component. - [Host-driven reactors](wasm-gc-module.md#no-wasi-reactor-mode) — a synchronous handler cannot `await`, so it returns the FUTURE an `async-defun` produced and the reactor transport resolves it at the boundary. --- # FILE: references/guides/clack.md # Clack Web Applications [Clack](https://github.com/fukamachi/clack) — a web application environment for Common Lisp — loads verbatim via `(ql:quickload "clack")`, and `clack:clackup` runs a Clack application on the built-in `clack-handler-rontolisp` backend: ```console $ cat app.lisp (ql:quickload "clack") (clack:clackup (lambda (env) (list 200 '(:content-type "text/plain") (list (format nil "Hello, Clack! ~A ~A~%" (getf env :request-method) (getf env :path-info))))) :server :rontolisp :port 5000 :use-thread nil) $ rontolisp app.lisp # interpret; or -o App.class / -o app.wasm --component $ curl http://127.0.0.1:5000/hello Hello, Clack! GET /hello ``` There is no adaptation layer behind this: rontolisp's own server protocol *is* Clack's (see [Serving HTTP](http-handler.md)), so the backend hands the application to the server as the handler and converts nothing per request — a Clack application is a valid `rontolisp:http-handler` handler, and vice versa. The first run downloads clack, [lack](https://github.com/fukamachi/lack) and their dependencies into `~/.rontolisp/quicklisp`; the dependencies resolve to real libraries (alexandria, the ironclad slice) and to the [built-in shim systems](asdf-systems.md#built-in-shim-systems) (bordeaux-threads, usocket, swank, uiop). ## clackup semantics The defaults work the way Clack users expect: - **`:use-thread t` (the default)** returns a handler object while the server answers on a background thread ([`rontolisp:make-thread`](../reference/functions/rontolisp-make-thread.md)), and `(clack:stop handler)` shuts that server down. - **`:use-thread nil`** serves in the foreground: `clackup` blocks until the process is stopped (Ctrl-C) — the script shape used above. - **`:use-default-middlewares t` (the default)** wraps the application in lack's backtrace middleware through `lack:builder`. - `:address` binds the listener (default `127.0.0.1`); `:silent t` suppresses the banner and `:debug nil` the debug notice. ## The application protocol The application is a function from the standard Clack env plist to the standard `(status headers body)` list: | env key | value | |---------|-------| | `:request-method` | the method as an upcased interned keyword (`:GET`, `:POST`, ...) | | `:script-name` | `""` | | `:path-info` | the percent-decoded request path | | `:query-string` | the raw query string, or `nil` | | `:request-uri` | the raw request target verbatim (still encoded, query included) | | `:server-name` / `:server-port` | from the `Host` header when present, otherwise the listener's | | `:server-protocol` | a keyword, e.g. `:HTTP/1.1` | | `:url-scheme` | `"http"` or `"https"` | | `:headers` | a hash table (`:test 'equal`) keyed by lowercased header names; duplicate request headers join with `", "` in wire order | | `:content-type` / `:content-length` | from that table (`nil` when absent; `:content-length` an integer) | | `:raw-body` | the request body as a synchronous in-memory bivalent stream — `read-line`/`read-char` and `read-byte`/`read-sequence` both work, with a real `file-position` (what lack-request and http-body need); `nil` for a bodiless request | | `:remote-addr` / `:remote-port` | the real peer on the interpreter and the JVM; `nil` on the WASI component (`wasi:http@0.3.0` exposes no peer accessor) | The response `body` may be a list of strings, a `(vector (unsigned-byte 8))` (written byte for byte, so a binary response is byte-exact), a rontolisp stream, or `nil`; the two-element `(status headers)` form is valid too. A bare string signals a clear error, as Clack itself refuses strings; a pathname body (lack's file-serving form) is a distinct value here and is refused too, until the transport can serve it. A function body is supported in Clack's delayed-response form (the responder is called with the final response list); the streaming-writer form signals. ## Getting from one handler to a set of routes The application above is ONE function for the whole site. A routing library is what turns it into a set of routes, and [tiny-routes](https://github.com/jeko2000/tiny-routes) loads unmodified (see the [ASDF systems guide](asdf-systems.md)): ```console $ cat routes.lisp (ql:quickload "clack") (ql:quickload "tiny-routes") (defpackage :demo (:use :cl :tiny-routes)) (in-package :demo) (define-routes *app* (define-get "/hello" () (ok "hello world")) (define-get "/users/:id" (req) (ok (format nil "user ~A" (path-parameter req :id)))) (define-post "/echo" (req) (ok (format nil "echo:~A" (request-body req)))) (define-any "*" () (not-found "nope"))) (clack:clackup (pipe *app* (wrap-request-body) (wrap-query-parameters)) :server :rontolisp :port 5000 :use-thread nil) $ rontolisp routes.lisp $ curl http://127.0.0.1:5000/hello hello world $ curl http://127.0.0.1:5000/users/42 user 42 $ curl -XPOST -d abc http://127.0.0.1:5000/echo echo:abc $ curl -o /dev/null -w '%{http_code}\n' http://127.0.0.1:5000/zzz 404 ``` Its request IS the env plist above and its response IS the response list, so nothing is converted at the boundary: `wrap-request-body` reads the `:raw-body` stream, `wrap-query-parameters` parses `:query-string`, the path template matches `:path-info`, and `ok`/`not-found` build `(status headers body)`. The routes are read inside the application's own package, which is where the library is meant to be used from. The same routes run WITHOUT a server on every backend — call the composed handler with a request plist you build yourself, which is what [`examples/asdf/tiny-routes-demo.lisp`](https://github.com/making/rontolisp/blob/develop/examples/asdf/tiny-routes-demo.lisp) does. Serving them has the backend constraints below. ### The other answer: ningle [ningle](https://github.com/fukamachi/ningle) loads unmodified too, and it is a different model rather than a different spelling. The application is a CLOS **object** you hang routes on, each route is a `setf`, a controller receives the matched **parameters** (the request itself is in a special variable), and a controller that is not a function at all is answered as the body: ```console $ cat ningle-app.lisp (ql:quickload "clack") (ql:quickload "ningle") (defpackage :demo (:use :cl)) (in-package :demo) (defvar *app* (make-instance 'ningle:app)) (setf (ningle:route *app* "/") "Welcome to ningle!") (setf (ningle:route *app* "/hello/:name") (lambda (params) (format nil "Hello, ~A" (cdr (assoc :name params))))) (setf (ningle:route *app* "/submit" :method :POST) (lambda (params) (format nil "posted ~A" (cdr (assoc "q" params :test #'string=))))) (clack:clackup *app* :server :rontolisp :port 5000 :use-thread nil) $ rontolisp ningle-app.lisp $ curl http://127.0.0.1:5000/ Welcome to ningle! $ curl http://127.0.0.1:5000/hello/Eitaro Hello, Eitaro $ curl -XPOST -d q=abc http://127.0.0.1:5000/submit posted abc $ curl -o /dev/null -w '%{http_code}\n' http://127.0.0.1:5000/zzz 404 ``` Four differences are worth knowing before picking one: - **Routes are added, not listed.** `(setf (ningle:route ...))` mutates the application, so routes can come from anywhere — including from run-time data. - **Query and body parameters arrive in the same alist** as the template's `:name` bindings (keyed by the string name), because ningle reads every request through `lack-request`. tiny-routes never touches that chain, and that is most of the size difference in a compiled module — an order of magnitude for the same two routes, with no ppcre-free opt-in to fall back on, since ningle's router compiles every rule to a scanner. - **The 404 is a method**, `ningle:not-found`, rather than a catch-all route, and `ningle:*response*` is mutable — which is how a controller answers a status other than 200. - **A route can be chosen by something that is not the path.** `:accept` negotiation is built in, and `(setf (ningle:requirement app :key) fn)` registers your own; the closure runs on every dispatch. ## Backends The `:server :rontolisp` line does not change between these — it means "serve on this target's native inbound transport", chosen at compile time: - **Interpreter** — everything above. - **JVM class** — the same program compiled with `-o App.class`; like every served program it needs the rontolisp jar on the runtime classpath (`java -cp rontolisp-exec.jar:. App`). - **WASM component** (`--component`) — the host owns the socket: run with `wasmtime serve -W gc=y -W exceptions=y -S cli=y -S tcp=y -S inherit-network=y app.wasm`. The `:port` argument is ignored, `:use-thread` is effectively `nil` (the WASM backends are single-threaded, so it defaults to `nil` there) and `clack:stop` is meaningless — the host controls the server's lifecycle. - **WASM reactor** (`--no-wasi`, or `--no-gc`) — the host **calls** the module instead of handing it a socket: the same program compiles to a module exporting `handle-request` (a JSON request string in, a JSON response string out), which a Cloudflare Worker, a browser page, node or a JVM host calls per request. `:port` is ignored and `clackup` returns at once — the next section has the details. - **WASM Preview 1** has no incoming TCP by design: the program compiles, and `clackup` signals `HTTP-HANDLER requires --component ...` at run time (catchable with `handler-case`). ## A host that calls you: the reactor build Some hosts never hand you a socket. A Cloudflare Worker, a browser page, node and a JVM embedding all parse the request themselves and then **call an exported function**. There is nothing for `clackup` to start there — but you still write `clackup`, and since `:server :rontolisp` picks the transport per target, *nothing* in the source has to change: compile the very same program with `--no-wasi` and the handler backend takes its reactor shape. ```console $ rontolisp app.lisp -o worker.wasm --no-wasi --optimize=size ``` `run` starts nothing here: it stores the application, and the compiler synthesizes the export the host calls (`handle-request`, a JSON request string in and a JSON response string out) from a marker the handler backend leaves behind. Nothing in your source names it, and the module imports nothing — no WASI shim on the JavaScript side. One keyword is a property of the *other* backends, not boilerplate: `:use-thread nil` — on the interpreter and the JVM `clackup` defaults to running the backend on its own thread, and a script wants to serve in the foreground. `clackup`'s default middlewares stay on everywhere: lack's `backtrace` middleware writes its report to `*error-output*`, which under `--no-wasi` is a discarding sink and on every other backend is real standard error. ### A handler that fetches: `--host-fetch` A reactor imports nothing, which also means it has no HTTP client — so an application that calls [`rontolisp:fetch`](../reference/functions/rontolisp-fetch.md) (a proxy, an API gateway) needs one more flag. `--host-fetch` lowers `fetch` onto the host's own client as an `env.fetch` import for the request and the reply's head, plus an `env.readResponseBody` import the reply's body is pulled through; those two imports are the whole difference to the module above: ```console $ rontolisp worker.lisp -o worker.wasm --no-wasi --host-fetch --optimize=size ``` The route bodies stay synchronous. Only an `async-defun` / `async-lambda` body may `await`, so a route that needs a fetched value calls one and returns its **future** — the reactor transport resolves a future-valued response at the boundary, exactly as `wasmtime serve` does under `--component`. The [fetch guide](http-fetch.md#fetching-from-a-reactor---no-wasi---host-fetch) has what else is particular to this transport (a body pulled after the head, a future settled at the headers, and the JSPI obligation on the JavaScript side), and [`examples/cloudflare-workers/dog-fetcher`](https://github.com/making/rontolisp/tree/develop/examples/cloudflare-workers/dog-fetcher) is a routed Worker built this way — one source that also serves a socket on the interpreter and the JVM. ### Driving the reactor by hand: `clack-handler-reactor` A second built-in handler backend makes the reactor shape **explicit and host-driven on every backend**: ```console $ cat worker.lisp (ql:quickload "clack-handler-reactor") (load "app.lisp") ; defines app, an ordinary Clack application (clack:clackup #'app :server :reactor :use-thread nil) ``` Where `:rontolisp` binds a socket on the interpreter and the JVM, this designator stores the application there too, and the host calls `(clack.handler.reactor:dispatch request-json)` — the same function the synthesized export calls — directly. That is how a Worker can be developed and tested without the Worker: the whole edit/run loop happens on the interpreter. A Worker itself no longer needs this designator; both ride the same machinery and the same application store, so the two cannot drift. Underneath both is `handle`, and it needs no `clackup` and no Worker to try — it is an ordinary function of two arguments: ```lisp (ql:quickload "clack-handler-reactor") (defun app (env) (list 200 '(:content-type "text/plain") (list (format nil "~a ~a ~a" (getf env :request-method) (getf env :path-info) (getf env :query-string))))) (princ (clack.handler.reactor:handle #'app "{\"method\":\"GET\",\"target\":\"/hi?a=1\"}")) ``` ```text {"status":200,"headers":[["content-type","text/plain"]],"body":"GET /hi a=1"} ``` `handle` takes the application and one JSON request string and answers one JSON response string. It builds the Clack environment and normalizes the Clack response through the same code path a served request takes, so the application sees exactly what Clack promises — and it **catches**: on a host like this an uncaught error would take the whole instance down, so it answers 500 with the condition's report instead. The envelope, in both directions: ```json { "method": "GET", "target": "/path?a=1", "headers": {"host": "..."}, "body": "", "scheme": "https", "remote-addr": "203.0.113.7" } ``` ```json { "status": 200, "headers": [["content-type", "text/plain"]], "body": "..." } ``` Two details the host side must get right: - `target` is the **raw** request target — path and query still joined and still percent-encoded. The split and the decoding happen on the Lisp side, and `:path-info` / `:query-string` have to come from there for the application to see what Clack promises. - Send `content-length` for a request with a body. `lack/request` parses nothing without it, and a request that arrived chunked carries none — set it from the bytes you actually read. Response headers cross as an **array of pairs**, not an object, so an application that sets two cookies still answers two `Set-Cookie` headers. ### Passing the body separately The JSON above is the request **head**. `handle` and `dispatch` take one more optional argument, the **body source**, so the body does not have to ride inside it: - `nil` — no body; - a string — the body, already read; - a function of no arguments — a **pull source**: each call answers the next chunk — a string, or an `(unsigned-byte 8)` vector for a host that hands over raw octets — with `nil` or an empty chunk for the end. It may answer a future, so a host that suspends while it reads can hand one over. Octets stay octets. Both `:raw-body` shapes are byte streams -- the default asynchronous stream answers each chunk as an `(unsigned-byte 8)` vector, the buffered Gray stream stores the octets as they came -- so a chunk that arrived as octets reaches the application unchanged and a binary upload is byte-exact; a source handing over text is UTF-8 encoded once. Nothing decodes per chunk, so a chunk boundary inside a UTF-8 sequence (a host reading a socket knows nothing about code points) costs nothing: `read-all` decodes the whole body once. A source that is empty at its **first** call is no body at all — `:raw-body` stays `nil`, exactly as for a request whose `"body"` is absent, because that is what upstream's `(when raw-body ...)` guards expect and a bodiless `GET` must not pay for a stream it would only find empty. The envelope's `"body"` key is exactly the string case, and it is what is used when no source is passed — or when the source turns out to be empty, so a host may start handing a reader over without also having to stop filling the envelope. A host written against the shape above keeps working unchanged. ### Taking the response body separately Symmetrically, `handle` and `dispatch` take a fourth optional argument, the **body sink**: a function of one argument, called with each chunk of the response body. It may answer a future, so a host that suspends while it writes can hand one over. Given a sink, the JSON answer is the response **head** and its `"body"` key is **absent** — so a host can tell "the body crossed out of band" from "the body is the empty string". A **stream** response body (a proxied `fetch`) is then forwarded chunk at a time instead of being collected into one string first. ```lisp (ql:quickload "clack-handler-reactor") (defun app (env) (declare (ignore env)) (list 200 '(:content-type "text/plain") (list "hello"))) (defvar *chunks* nil) (defun sink (chunk) (setq *chunks* (cons chunk *chunks*)) nil) (princ (clack.handler.reactor:handle #'app "{\"method\":\"GET\",\"target\":\"/hi\"}" nil #'sink)) (terpri) (princ (car *chunks*)) ``` ``` {"status":200,"headers":[["content-type","text/plain"]]} hello ``` The chunks cross **before** the head, because the head is the return value. So a head that does carry a `"body"` key wins over anything already written: that is how a handler that fails halfway through its body still answers one clean document — the 500 the transport catches into carries its report in band, and the host discards the chunks it already took. An `(unsigned-byte 8)` response body reaches the sink as **octets**, not as text: a sink can write bytes, and a JSON head cannot carry them. Passing no sink keeps the old shape exactly: the body rides the head, a stream body is drained into it, and octets -- an octet body, or a stream's octet chunks -- are rendered as the text their UTF-8 bytes spell, so a page a Clack application answered as octets crosses as the page it was. A JSON string is text, so that is the ONE place a *binary* response is not byte-exact, and it is the reason the sink exists: a host that answers binary passes one. ### The WASM boundary: a head export and two body imports On a `--no-wasi` WASM module neither a source nor a sink is a Lisp value the host can pass, so the boundary is three entries and the compiler writes the two imports for you: ```text module -> host handle-request(headPtr, headLen) -> (ptr, len) ; the JSON head host -> module env.readRequestBody(ptr, cap) -> n ; up to cap octets ; at ptr; 0 = end host -> module env.writeResponseBody(ptr, len) ; take these octets ``` The head is the JSON above **without** the `"body"` key, in either direction. The bodies cross as raw octets — in, into a buffer the module owns and reuses; out, straight out of the module's own memory — which is what a JSON string could not do: a **binary** body crosses exactly either way (the string boundary decodes UTF-8, and does not validate), and *crossing* costs the module no linear memory at all — the envelope used to hold the body several times over. Reading the request body is not yet free: whichever way a handler drains `:raw-body`, decoding the octets to text currently costs about fifteen times the body in linear memory, reclaimed for reuse at the end of the request. Note the direction flip in the two imports. A chunk crossing *in* is a result written into a buffer the module passes; one crossing *out* is a parameter the host reads and must copy before the call returns. Both are the same rule — the caller owns the memory — and both mean the host may not hold on to a pointer. Both imports are declared `:async t`, so the host chooses how it answers. Answering synchronously (read the body first, then call in; collect the response chunks as they arrive) is the simple host, and it is what the Worker examples do. Wrapping an import in `WebAssembly.Suspending` — pulling from the request's own reader, or writing to a stream that applies backpressure — is the streaming host: it must then enter `handle-request` through `WebAssembly.promising` and serialise its calls, because a suspended module can be re-entered — the module refuses that with a trap rather than corrupting both calls, and the build prints the obligation. Under `--component` both bodies stay inside the envelope: a component's host functions cross the canonical ABI rather than a core import. So does a plain WASI command module that drives its own `dispatch` in-process (what the examples' `check.lisp` files do), whose host is `wasmtime run` and satisfies no `env.*` import. Everything above this section is unchanged either way, which is the point of the source and the sink being abstract values. What the application then sees is the `:raw-body` mode. `clackup` and `handle` ask for the buffered one, the synchronous stream Clack promises (the source is drained into it whatever shape it had). A reactor built from a bare `rontolisp:http-handler` keeps **that directive's** default instead — a rontolisp stream, drained the same way as on every other backend: ```lisp (rontolisp:async-defun handle (env) (let ((body (rontolisp:await (rontolisp:read-all (getf env :raw-body))))) (list 200 '(:content-type "text/plain") (list body)))) ``` A complete Worker built this way — the JavaScript side and the measurements included — is [`examples/cloudflare-workers/httpbin-clack/`](https://github.com/making/rontolisp/tree/develop/examples/cloudflare-workers/httpbin-clack). Beside it, [`examples/cloudflare-workers/httpbin-clack-one-source/`](https://github.com/making/rontolisp/tree/develop/examples/cloudflare-workers/httpbin-clack-one-source) deploys [`examples/net/httpbin-clack.lisp`](https://github.com/making/rontolisp/blob/develop/examples/net/httpbin-clack.lisp) *itself* — the file that binds a socket when you interpret it — and so contains no Lisp file at all: one source, four hosts. If the module size matters more than the `clackup` line, this adapter is small enough to write out by hand and skip loading clack entirely. [`examples/cloudflare-workers/httpbin/`](https://github.com/making/rontolisp/tree/develop/examples/cloudflare-workers/httpbin) is that: the same application, the same envelope, the same JavaScript side, and about half the module. The two directories are a measured pair — the per-request cost turns out to be identical, and what clack costs on a host like this is module size and isolate startup. ## Current limits - One Clack server per process: a second concurrent `clackup` replaces the first one's application. - `clack.socket` (WebSocket) and `:swank-port` are unsupported (`:swank-port` reaches the `swank` stub, which signals). - Streaming-writer responses and bare-string/pathname bodies signal, as noted above (delayed function responses work). See also: [Serving HTTP (http-handler)](http-handler.md) for the underlying server, and [`examples/asdf/clack-hello.lisp`](https://github.com/making/rontolisp/blob/develop/examples/asdf/clack-hello.lisp) for the runnable demo with all per-backend commands. --- # FILE: references/guides/clock-and-random.md # The Clock and Randomness Two values a program cannot work out for itself: what time it is, and a number nobody can predict. Both come from outside, which is why they are the pair whose behaviour depends on the backend — and the pair a module with **no** host has to have an answer for. ## Where the values come from | build | `(random n)` | `rontolisp:random-bytes` | the clock | | --- | --- | --- | --- | | interpreter, JVM | the JVM's generator | works | the machine's clock | | WASM (default, Preview 1) | the host's WASI `random_get` | works | WASI `clock_time_get` | | WASM `--component` | the host's `wasi:random` | works | `wasi:clocks` | | WASM `--no-wasi` | a built-in generator, **same sequence every instance** | signals | what the host wrote through `__ronto_set_time`; signals until it does | | WASM `--no-wasi --host-random` | the host's `env.random_get` | works | as above | Only the `--no-wasi` rows need a decision from you; everywhere else both values are the host's own, and the rest of this page is about that last case. ## Randomness Common Lisp's [`random`](../reference/functions/random.md) is a pseudo-random draw from `*random-state*`, not an entropy API: nothing in its contract promises unpredictability, and an image may start from a fixed state. [`rontolisp:random-bytes`](../reference/functions/rontolisp-random-bytes.md) is the separate API that does promise it, and it is available only where a real entropy source is. ```lisp (print (list (random 1) (< (random 10) 10))) ; => (0 T) ``` The result type follows the limit — an integer limit yields an integer, a float limit a float — so `(random 1)` is always `0`. ## The clock [`get-universal-time`](../reference/functions/get-universal-time.md) reports seconds since 1900-01-01 GMT; [`get-internal-real-time`](../reference/functions/get-internal-real-time.md) and [`get-internal-run-time`](../reference/functions/get-internal-run-time.md) report milliseconds, and only their differences are meaningful. All three return an **integer on every backend**. ```lisp (print (list (integerp (get-universal-time)) (>= (get-internal-real-time) 0))) ; => (T T) ``` [`encode-universal-time`](../reference/functions/encode-universal-time.md) and [`decode-universal-time`](../reference/functions/decode-universal-time.md) convert between that integer and calendar fields with pure arithmetic, so they behave identically everywhere — with one deliberate deviation: a missing time zone means **GMT, not the machine's local zone**, because no backend-portable source of the local zone exists (WASI exposes no timezone at all). Waiting is the clock's other half. [`sleep`](../reference/functions/sleep.md) parks the thread on the interpreter and the JVM, waits on the real host timer under `--component` (costing no CPU), and busy-waits on the clock on WASM Preview 1, whose imports include a clock but no timer. On `--no-wasi` it signals — see below. ## A module with no host — `--no-wasi` A [`--no-wasi` module](wasm-gc-module.md#no-wasi-reactor-mode) imports nothing, so neither value has anywhere to come from. What it does about that follows the rule the whole flag follows: **a stub answers when the answer is true of the module, and refuses when answering would mean inventing a value you could not tell from a real one** — and a value the *host* hands in is not an invention, which is what the two hooks below are for. Randomness lands on the answering side by itself. The module carries its own generator, which is inside `random`'s contract — `make-random-state` here answers `nil`, so no state object is observable and "the sequence repeats" is a property of the contract rather than a claim about the host. The consequence is worth stating plainly: **unseeded, every instance of one module produces the same sequence.** Because that generator is not entropy, `rontolisp:random-bytes` signals rather than draw from it. The clock lands on the refusing side by itself: a reading of 0 is not "no time", it is 1970, and nothing the module could invent would *be* the time. So until a host sets it, all three built-ins signal a catchable error naming the operator. A library that reads the clock while it *loads* has no caller to catch that, so the **build** names it for you instead of leaving it to the first run — see [what the build tells you](wasm-gc-module.md#what-the-build-tells-you-before-you-run-it). ### Seeding the generator — `__ronto_seed_random` The module cannot *import* the host's random by default: a core WebAssembly import is not optional, so asking for one would break the very thing the flag is for (instantiating with `{}`). It exports a hook instead. Call it once, **before `_initialize`**, and even a library's load-time `(random ...)` draws from your seed: ```js const instance = new WebAssembly.Instance(module, {}); instance.exports.__ronto_seed_random( new BigUint64Array(crypto.getRandomValues(new Uint8Array(8)).buffer)[0], ); instance.exports._initialize(); ``` Skip the call and you get the deterministic sequence, unchanged. The hook is on the core-module shape only — a reactor component (`--component --no-wasi`) runs its top level at instantiation, so there is no window before the first draw. Seeding makes the sequence unpredictable per instance but **does not** re-enable `rontolisp:random-bytes`: the generator is invertible from a single output, so a seeded stream is not cryptographically strong, and the API that promises entropy keeps saying no rather than handing you something that only looks like a CSPRNG. ### Setting the clock — `__ronto_set_time` The clock's hook is the same shape, and takes **nanoseconds since the Unix epoch**: ```js const instance = new WebAssembly.Instance(module, {}); instance.exports.__ronto_set_time(BigInt(Date.now()) * 1000000n); instance.exports._initialize(); ``` Calling it before `_initialize` is what makes a library that timestamps while it *loads* loadable at all — `lack-middleware-session` reads the clock from a top-level form, and without this the module dies during initialization rather than at the first request. The clock does not tick on its own: it holds the value you wrote until you write another. That is less of a restriction than it sounds — a Cloudflare Worker's own clock is frozen for the duration of a request as a timing-attack mitigation — and the natural rhythm is to set it once per request, which is what [the Worker examples](https://github.com/making/rontolisp/tree/develop/examples/cloudflare-workers) do. The one thing it cannot support is waiting: `(sleep n)` signals here, because nothing can make an interval elapse while your call is running. Like the seed hook, it is on the core-module shape only. A reactor component runs its top level at instantiation, so there is no moment at which a host could set the time first; the clocks signal there, and say so. ### Drawing from the host — `--host-random` `--host-random` replaces the built-in generator with a host call, so every draw is the host's entropy — including draws inside a quickloaded library, which never learns where the bytes came from: ```bash rontolisp app.lisp --no-wasi --host-random -o app.wasm ``` The module then imports exactly one function, `env.random_get(buf, len) -> errno`. That is preview1's signature, so a host that already has a WASI implementation can forward it unchanged; from JavaScript it is one property: ```js const instance = await WebAssembly.instantiate(module, { env: { random_get(ptr, len) { crypto.getRandomValues(new Uint8Array(instance.exports.memory.buffer, ptr, len)); return 0; // errno 0 = success }, }, }); instance.exports._initialize(); ``` Because the entropy really is the host's, `rontolisp:random-bytes` works here. No `__ronto_seed_random` is exported — there is no module-local state left to seed. `__ronto_set_time` is unaffected: the two services are independent, and only one of them has a module-local generator to make redundant. The zero-import default is unchanged; this is the opt-in, and the module now has an import the host **must** provide. The tree shaker still drops it if the program never draws. The flag is core-module only: a reactor component imports nothing by contract, and a plain `--component` build already has `wasi:random`. There is no `--host-clock` counterpart, because the export answers the same question without costing the zero-import property. A live clock that advances *during* a call would need one; nothing has needed that yet. ## Redefining `random` A program's own `(defun random ...)` is called by the interpreter and ignored by the compile backends, which emit the standard operator at the call site and warn that they did — see [Redefining a COMMON-LISP function](../reference/function-namespace.md#redefining-a-common-lisp-function). --- # FILE: references/guides/eval-limitations.md # Compiled eval Limitations `eval` works in all three backends. In the interpreter it is the full tree-walking evaluator. The WASM and JVM compilers each emit a small tree-walking interpreter into their output (`_eval`/`_apply`/`_store` plus the helpers `_envLookup`/`_lookup`) that runs the form at runtime, so no separate evaluator or parser is needed. The compiled `eval` (WASM and JVM) implements a lexical environment plus a persistent global environment, and aims for parity with the interpreter: self-evaluating atoms, variable references, closures, the special forms and higher-order functions (`let`, `lambda`, `cond`, `while`, `dotimes`, `setq`, `setf`, `push`, `pop`, `funcall`, `mapcar`, `mapc`, `reduce`, nested `eval`, ...), and application of any function or interpreted closure all behave as in the interpreter. Rather than enumerate everything, the differences are listed below. ## Compiled `eval` limitations The compiled `eval` (WASM and JVM) differs from the interpreter only in these cases: - **`let` binding lists must use the `((name value) ...)` form** (a bare `(let (x) ...)` is not supported). - **Comparison operators are binary inside `eval`.** Compiled top-level code supports variadic `=`, `<`, `>`, `<=`, `>=` and variadic `min`/`max`/`gcd`/`lcm` (desugared into nested binary operations at compile time), but that desugaring does not reach forms interpreted at runtime by `eval`, where these operators take two arguments and extra arguments are ignored (so `(eval '(= 1 1 2))` evaluates `(= 1 1)` and returns true). `+ - * / list` are fully variadic everywhere. User functions with more than 10 parameters return `nil`. - **Edge cases that fail.** A zero-argument `(+)`/`(-)`/`(*)`/`(/)` fails at runtime (a trap in WASM, an exception in JVM). Unary `(- x)` and `(/ x)` negate/invert like the interpreter. - **A `lambda` built at runtime does not parse lambda-list keywords.** Compiled `defun`/`lambda` forms support `&optional`/`&rest`/`&key` (desugared at compile time), and calling such a compiled variadic function from `eval` works. But a lambda that only exists inside an eval'd form, e.g. `(eval '(funcall (lambda (&rest r) r) 1 2))`, binds its parameters positionally — `&rest` is treated as an ordinary parameter name. - **An unbound variable evaluates to the symbol itself.** The interpreter signals `The variable x is unbound`; the runtime `eval` has no error channel and returns the symbol instead. An undefined function in call position returns `nil`. - **Top-level global variables are shared write-through from compiled code.** A top-level `setq`/`defvar`/`defparameter`/`defconstant` mirrors its value into the runtime `eval` global environment, so an eval'd expression can read a global the compiled program defined (e.g. `(setq add10 (make-adder 10))` then `(eval '(funcall add10 100))` returns `110`). The mirror is one-way: if `eval` later reassigns such a variable, compiled code keeps reading its own copy. - **`let*`, `do`, `do*`, `dolist`, `return`, `defvar`, `defparameter`, `defconstant`, `incf`, `decf`, `format`, `error`, `ecase`, `etypecase`, `ccase`, `concatenate`, `with-open-file` and the file-stream functions (`open`, `close`, `write-line`) are not supported.** These forms are expanded or handled at compile time only; the runtime `eval` interpreter does not recognize them. The sequence functions (`length`, `reverse`, `member`, `member-if`, `find`, `find-if`, `position`, `count`, `assoc`, `assoc-if`, `getf`, `last`, `butlast`, `remove`, `remove-if`, `remove-if-not`, `remove-duplicates`, `delete`, `delete-if`, `delete-if-not`, `substitute`, `nsubstitute`, `nconc`, `copy-list`, `nreverse`, `make-list`, `union`, `intersection`, `set-difference`, `adjoin`, `identity`, `mapcan`, `sort`, `every`, `some`) and `princ-to-string`/`prin1-to-string` work, since they resolve through the compiled function registry. The `:test`/`:key` keywords of the sequence and alist functions are a compile-time expansion, however: inside a runtime `eval` form they are silently ignored and the comparison stays `eql` (only the interpreter backend applies them inside `eval`). - **`defmacro` and backquote are compile-time only.** On the compilation path, user macros are fully expanded (and their definitions consumed) before the compilers run, and backquote templates are expanded by the reader; the runtime `eval`/`read` of a compiled program recognizes neither `defmacro` nor the backquote character. The same holds for `macroexpand`/`macroexpand-1`: a call with a literal quoted argument is folded to its expansion at compile time, and the runtime `eval` does not know these functions (`gensym`, by contrast, works — it has a first-class wrapper). - **`defstruct` is compile-time only.** A top-level `defstruct` is expanded into its generated functions before compilation, so calling a constructor/accessor/predicate from `eval` works, but an eval'd form can neither define a new structure nor use an accessor as a `setf` place. A `#S(...)` literal is likewise resolved before compilation, so it is not recognized inside `eval` either. - **The CLOS subset is compile-time only.** Like `defstruct`, top-level `defclass`/`defgeneric`/`defmethod` forms are expanded before compilation — calling a generic function, a reader/accessor, or a constructor from `eval` works, but an eval'd form cannot define classes or methods, and `make-instance`/`slot-value` are not recognized inside `eval` (they resolve through the compile-time class registry). - **The `rontolisp` package functions are not supported.** `rontolisp:version`, `rontolisp:list-functions`, `rontolisp:list-macros`, `rontolisp:list-special-forms`, `rontolisp:fetch`, `rontolisp:http-handler`, `rontolisp:await`, `rontolisp:futurep`, `rontolisp:json-parse` and `rontolisp:json-stringify` are compiled directly (constants, inline calls or spliced-in library functions); the runtime `eval`/`load` does not recognize them. These differences come from the design: the runtime `eval` resolves operators by name against a compile-time registry of the functions that were actually compiled into the output, and built-in functions are shared with the compiled code. --- # FILE: references/guides/gray-streams.md # Gray Streams (user-defined streams) rontolisp ships its own small Gray-stream extension, mirroring how real implementations expose their native Gray support: a user class extends one of the `rontolisp:fundamental-*-stream` base classes and defines methods on the `rontolisp:stream-*` generics, and the stream-taking built-ins dispatch to those methods when handed such an instance instead of a stream handle. This works on every backend (interpreter, JVM, both WASM variants). The base classes form the CL-shaped hierarchy: `fundamental-stream` at the root, `fundamental-input-stream` / `fundamental-output-stream` below it, and `fundamental-character-input-stream` / `fundamental-character-output-stream` / `fundamental-binary-input-stream` / `fundamental-binary-output-stream` as the leaves (all in the `rontolisp` package). | built-in | dispatches to | | --- | --- | | `write-char` | `rontolisp:stream-write-char` | | `write-string`, `format` (stream destination) | `rontolisp:stream-write-string` | | `princ`, `prin1`, `print` | `rontolisp:stream-write-string` of the rendered text (`print` then `stream-terpri`) | | `terpri` | `rontolisp:stream-terpri` (default method writes a newline through `stream-write-char`) | | `fresh-line` | `rontolisp:stream-fresh-line` (default method: `stream-terpri` unless `stream-start-line-p`) | | `write-line` | `rontolisp:stream-write-string` then `rontolisp:stream-terpri` | | `force-output` / `finish-output` / `clear-output` | `rontolisp:stream-force-output` / `-finish-output` / `-clear-output` (default methods answer `nil`) | | `close` | answers `t` -- see below | | `write-byte` | `rontolisp:stream-write-byte` | | `read-byte` | `rontolisp:stream-read-byte` | | `read-char` | `rontolisp:stream-read-char` | | `read-char-no-hang` | `rontolisp:stream-read-char-no-hang` (default method IS `stream-read-char`) | | `peek-char` | `rontolisp:stream-peek-char` (default method: read one, hand it back through `stream-unread-char`); the `peek-type` skipping forms loop over it | | `unread-char` | `rontolisp:stream-unread-char` (default method parks the character in the protocol's one-slot pushback) | | `read-line` | `rontolisp:stream-read-line` (default method loops `stream-read-char`) | | `listen` | `rontolisp:stream-listen` (default method answers `nil`) | | `open-stream-p` | answers `t` -- like `close`, a name a program may own | | `stream-element-type` | `character`, or `(unsigned-byte 8)` for a binary base class -- a class subclassing BOTH (a bivalent stream) answers `character`, because the answer is which buffer to allocate. A name a program may own | | `read-sequence` / `write-sequence` | `rontolisp:stream-read-sequence` / `-write-sequence` (default methods loop the element generics) | | `file-position` | `rontolisp:stream-file-position`; the two-argument form calls the `(setf rontolisp:stream-file-position)` writer generic | A character output stream defines **`stream-write-char` or `stream-write-string` -- either one is enough**. Each has a default method written in terms of the other, so the rest of the output protocol composes out of whichever you wrote. (Defining neither is the one broken shape: the two defaults then call each other.) Two more generics have no built-in of their own but are what the line-oriented operators consult: `rontolisp:stream-line-column` answers the stream's current column, or `nil` (the default) for a stream that tracks none, and `rontolisp:stream-start-line-p` answers from it. A stream with no column cannot tell whether it is at the start of a line, so `fresh-line` on it always writes the newline. `rontolisp:stream-advance-to-column` rounds out the protocol for a program that calls it directly. Closing a Gray stream answers `t` and does nothing else -- there is nothing to release. A stream that DOES hold something writes CL's own spelling, a method on `close` itself: ```lisp (defclass closing-stream (rontolisp:fundamental-character-output-stream) ((acc :initform "") (openp :initform t))) (defmethod rontolisp:stream-write-char ((s closing-stream) c) (setf (slot-value s 'acc) (concatenate 'string (slot-value s 'acc) (string c))) c) (defmethod close ((s closing-stream) &key abort) (declare (ignore abort)) (setf (slot-value s 'openp) nil) t) (let ((s (make-instance 'closing-stream))) (write-string "bye" s) (list (close s) (slot-value s 'openp))) ; => (T NIL) ``` Such a method dispatches on every backend. A program that defines one owns `close` outright: the Gray default steps aside for it. A character INPUT stream defines **`stream-read-char` -- that one method is enough** (a binary one defines `stream-read-byte`). Everything else on the read side is written over it: `stream-read-line` and `stream-read-sequence` loop it, `stream-read-char-no-hang` is it, and `stream-peek-char` reads one character and hands it back through `stream-unread-char`, whose own default parks the character in the protocol's one-slot pushback. A class that can rewind its own source defines `stream-unread-char` and owns the pushback instead — the pushback cell is then never written. On the read side the methods answer the keyword `:eof` at end of stream; the built-ins translate that through the usual `eof-error-p` / `eof-value` contract. `stream-read-line` answers a partial last line as that line — `:eof` means "no characters left at all". ```lisp (defclass upcase-stream (rontolisp:fundamental-character-output-stream) ((acc :initform ""))) (defmethod rontolisp:stream-write-string ((s upcase-stream) str) (setf (slot-value s 'acc) (concatenate 'string (slot-value s 'acc) (string-upcase str))) str) (let ((s (make-instance 'upcase-stream))) (write-string "hello" s) (write-char #\! s) (slot-value s 'acc)) ; => "HELLO!" ``` A `stream-write-char`-only stream that tracks its column, so `fresh-line` can tell whether it has to break the line: ```lisp (defclass column-stream (rontolisp:fundamental-character-output-stream) ((acc :initform "") (col :initform 0))) (defmethod rontolisp:stream-write-char ((s column-stream) c) (setf (slot-value s 'acc) (concatenate 'string (slot-value s 'acc) (string c))) (setf (slot-value s 'col) (if (char= c #\Newline) 0 (+ (slot-value s 'col) 1))) c) (defmethod rontolisp:stream-line-column ((s column-stream)) (slot-value s 'col)) (let ((s (make-instance 'column-stream))) (princ "one" s) (fresh-line s) ; column 3 -> writes the newline (fresh-line s) ; column 0 -> writes nothing (write-line "two" s) ;; newlines shown as / so the whole answer fits on one line (substitute #\/ #\Newline (slot-value s 'acc))) ; => "one/two/" ``` A binary input stream with the `file-position` protocol: ```lisp (defclass byte-source (rontolisp:fundamental-binary-input-stream) ((items :initarg :items) (pos :initform 0))) (defmethod rontolisp:stream-read-byte ((s byte-source)) (let ((items (slot-value s 'items)) (pos (slot-value s 'pos))) (if (>= pos (length items)) :eof (progn (setf (slot-value s 'pos) (+ pos 1)) (nth pos items))))) (defmethod rontolisp:stream-file-position ((s byte-source)) (slot-value s 'pos)) (defmethod (setf rontolisp:stream-file-position) (position (s byte-source)) (setf (slot-value s 'pos) position)) (let ((in (make-instance 'byte-source :items (list 10 20 30)))) (read-byte in) ; 10 (file-position in) ; 1 (file-position in 0) (list (read-byte in) (read-byte in nil :done))) ; => (10 20) ``` A character input stream defining only `stream-read-char`, driven through the rest of the read protocol: ```lisp (defclass text-source (rontolisp:fundamental-character-input-stream) ((text :initarg :text) (pos :initform 0))) (defmethod rontolisp:stream-read-char ((s text-source)) (let ((text (slot-value s 'text)) (pos (slot-value s 'pos))) (if (>= pos (length text)) :eof (progn (setf (slot-value s 'pos) (+ pos 1)) (char text pos))))) (let ((in (make-instance 'text-source :text "ab cd"))) (list (peek-char nil in) ; look, do not consume (read-char in) (progn (unread-char #\a in) (read-char in)) (read-char-no-hang in) (peek-char t in) ; skip whitespace (read-line in) (open-stream-p in) (stream-element-type in))) ; => (#\a #\a #\a #\b #\c "cd" T CHARACTER) ``` ## The trivial-gray-streams shim Portable libraries are written against [trivial-gray-streams](https://github.com/trivial-gray-streams/trivial-gray-streams) rather than an implementation's own protocol. rontolisp bundles a built-in `trivial-gray-streams` ASDF system adapting the portable API onto the protocol above (see [Systems](asdf-systems.md#built-in-shim-systems)): the `trivial-gray-streams` package mirrors every base class (plus `trivial-gray-stream-mixin`) and every generic, including `stream-read-sequence` / `stream-write-sequence` `(stream sequence start end &key)`, `stream-file-position` with its `(setf ...)` writer, and the output family `stream-line-column` / `stream-start-line-p` / `stream-terpri` / `stream-fresh-line` / `stream-advance-to-column` / `stream-force-output` / `stream-finish-output` / `stream-clear-output` — this is how jzon's `:stream` writer API runs, and the class shape fast-io and circular-streams define loads unchanged. The defaults are the same ones the rontolisp protocol has, so a portable class that defines only `trivial-gray-streams:stream-write-char` still answers every operator above. ```lisp (asdf:load-system "trivial-gray-streams") (defclass upcase-stream (trivial-gray-streams:fundamental-character-output-stream) ((acc :initform ""))) (defmethod trivial-gray-streams:stream-write-string ((s upcase-stream) str &optional start end) (declare (ignore start end)) (setf (slot-value s 'acc) (concatenate 'string (slot-value s 'acc) (string-upcase str))) str) (defmethod trivial-gray-streams:stream-write-char ((s upcase-stream) c) (trivial-gray-streams:stream-write-string s (string c)) c) (let ((s (make-instance 'upcase-stream))) (write-string "hello" s) (write-char #\! s) (slot-value s 'acc)) ; => "HELLO!" ``` ## Limits - `rontolisp:stream-advance-to-column` exists as a protocol generic but no built-in dispatches to it (`format`'s `~T` does not consult the column). - The protocol's pushback holds ONE character for ONE stream at a time, which is what CL promises for `unread-char`. It is drained by every read that goes through the protocol's own defaults; a class that overrides `stream-read-line` or `stream-read-sequence` outright reads past it, so such a class should define `stream-unread-char` too. - `input-stream-p` / `output-stream-p` answer the DIRECTION base class the instance extends, not a predicate method per class: a `fundamental-input-stream` descendant answers `t` to the first and `nil` to the second, and a subclass of the bare `fundamental-stream` answers `nil` to both. A class may still define a method on either name and own the answer. - `unread-char` on a stream HANDLE — a file, a string input stream, a socket — parks the character in a handle-side pushback of its own, which `read-char`, `peek-char` and `read-line` drain. It holds one character for one stream, like the protocol's; a second `unread-char` with the cell still full signals. `read-byte`, `read-sequence` and `read` do not consult it. - The read generics return primary values only: `stream-read-line` has no `(values line missing-newline-p)` pair — `:eof` is the whole EOF signal. - `listen` on a Gray instance works on the interpreter and the JVM; the Preview 1 WASM backend rejects any `listen` call at compile time (a pre-existing platform limit, Gray or not). - A `(write-string s instance :start ... :end ...)` call with bounding keywords does not dispatch the bounds to the instance. - Dispatch happens at the built-in call sites: a first-class `(funcall #'read-byte instance)` does not dispatch on the compiled backends. --- # FILE: references/guides/http-fetch.md # HTTP Requests (fetch) The `rontolisp` package provides outgoing HTTP modeled on the JavaScript `fetch` API, plus the JSON functions that pair naturally with it. None of these are part of Common Lisp; reference them with the `rontolisp:` qualifier (see [Packages](../reference/packages.md)). `rontolisp:fetch` starts a request and immediately returns a **future**; you resolve it with `rontolisp:await`. The future / `await` mechanics themselves are not specific to HTTP — they are the subject of the [Asynchronous Programming guide](async.md), which this page assumes; here we cover only what is particular to making requests. | Function | Purpose | |----------|---------| | [`rontolisp:fetch`](../reference/functions/rontolisp-fetch.md) | Start an HTTP request: `(rontolisp:fetch url &optional options)` | | [`rontolisp:read-all`](../reference/functions/rontolisp-read-all.md) | Drain a response body stream into one string (async) | | [`rontolisp:json-parse`](../reference/functions/rontolisp-json-parse.md) | Parse a JSON string into Lisp values | | [`rontolisp:json-stringify`](../reference/functions/rontolisp-json-stringify.md) | Serialize a Lisp value to a JSON string | > **Backend support.** The interpreter and JVM-compiled classes use the JDK > `java.net.http.HttpClient`; the request runs on a background thread from the > moment `fetch` returns. On WASM `fetch` needs a host that can make the call > for it, which is either a **component** (`--component`, importing the async > `wasi:http@0.3.0`, run with `-S http=y` on top of the usual flags) or a > **`--no-wasi` reactor built with `--host-fetch`**, which lowers the same > source onto the host's own HTTP client through an `env.fetch` import (plus > `env.readResponseBody` for the reply body) — that > is how a Cloudflare Worker or a node embedding fetches > ([the section below](#fetching-from-a-reactor---no-wasi---host-fetch)). With > neither, `fetch` is a compile error in Preview 1 (core-module) mode. In the > **browser playground** `fetch` > runs the real browser `fetch()` (subject to CORS) while the program > continues. The JSON functions work on **every** backend and in every WASM > mode; only `fetch` itself is restricted. `await`, `futurep` and the future > combinators are covered in the [async guide](async.md). ## A first request `fetch` returns as soon as the request is in flight. Passing the future to `rontolisp:await` suspends until the response arrives and yields the result property list `(:status :headers :body )` — on every backend `:body` is an [asynchronous stream](async.md#asynchronous-streams) of the body's octets (each chunk an `(unsigned-byte 8)` vector), drained to one decoded string with [`rontolisp:read-all`](../reference/functions/rontolisp-read-all.md): ```lisp (let ((p (rontolisp:fetch "https://httpbin.ik.am/get"))) (getf (rontolisp:await p) :status)) ; => 200 ``` Reading the individual fields: ```lisp (defparameter *res* (rontolisp:await (rontolisp:fetch "https://httpbin.ik.am/get"))) (getf *res* :status) ; => 200 (stringp (rontolisp:await (rontolisp:read-all (getf *res* :body)))) ; => T (cdr (assoc "content-type" (getf *res* :headers) :test #'string=)) ; => "application/json" ``` Because the request is already running when `fetch` returns, several requests overlap — start them all, then await each (in any order). This is just the general [overlapping-work](async.md#overlapping-work) behavior of futures: ```lisp (let ((p1 (rontolisp:fetch "https://httpbin.ik.am/status/200")) (p2 (rontolisp:fetch "https://httpbin.ik.am/status/201"))) ; both requests running (list (getf (rontolisp:await p1) :status) (getf (rontolisp:await p2) :status))) ; => (200 201) ``` ## Request options The optional second argument is an options property list with `:method` (a string, default `"GET"`), `:headers` (an alist of `(name . value)` string pairs) and `:body` (a string): ```lisp ;; GET with request headers (an alist of (name . value) string pairs) (rontolisp:await (rontolisp:fetch "https://httpbin.ik.am/get" '(:headers (("Accept" . "application/json"))))) ;; POST with a request body (rontolisp:await (rontolisp:fetch "https://httpbin.ik.am/post" '(:method "POST" :headers (("Content-Type" . "application/json")) :body "{\"name\":\"rontolisp\"}"))) ``` The supported methods are `GET`, `HEAD`, `POST`, `PUT`, `DELETE`, `OPTIONS` and `PATCH`; see the [fetch](../reference/functions/rontolisp-fetch.md) reference page for validation timing and error behavior per backend (a failed request surfaces at `await`, not at `fetch` — every backend signals an error there; `nil` comes back only for a request that cannot be *started*). ## Working with JSON `rontolisp:json-parse` turns a JSON document into Lisp values following [`com.inuoe.jzon`](asdf-systems.md)'s defaults: a JSON object becomes a hash table with string keys, an array a vector, and `true`/`false`/`null` become `t`/`nil`/the symbol `null`: ```lisp (gethash "name" (rontolisp:json-parse "{\"name\": \"rontolisp\", \"n\": 2}")) ; => "rontolisp" ``` ```lisp (gethash "b" (gethash "a" (rontolisp:json-parse "{\"a\": {\"b\": [1, true, null]}}"))) ; => #(1 T NULL) ``` `rontolisp:json-stringify` is the inverse: a hash table becomes an object, a vector or list an array, and `nil`/`t`/the symbol `null` become `false`/`true`/`null`: ```lisp (let ((h (make-hash-table :test 'equal))) (setf (gethash "name" h) "rontolisp") (rontolisp:json-stringify h)) ; => "{\"name\":\"rontolisp\"}" ``` ```lisp (rontolisp:json-stringify (list 1 (list 2 3) nil)) ; => "[1,[2,3],false]" ``` Both functions are written in rontolisp itself and compile into the program on every backend, and are a lightweight subset of jzon — a program can switch to it unchanged. The full value mappings and the edge cases (integer width, key order) are on the [json-parse](../reference/functions/rontolisp-json-parse.md) and [json-stringify](../reference/functions/rontolisp-json-stringify.md) reference pages. When building the hash table by hand — `make-hash-table` then a `setf gethash` per key — is awkward, four utilities convert to and from the usual list shapes. [`rontolisp:plist-hash-table`](../reference/functions/rontolisp-plist-hash-table.md) and [`rontolisp:alist-hash-table`](../reference/functions/rontolisp-alist-hash-table.md) build a hash table from a property list or an association list (a keyword key like `:name` down-cases to `"name"`), so a JSON object is one expression from a quoted literal: ```lisp (rontolisp:json-stringify (rontolisp:plist-hash-table '(:name "rontolisp" :stars 1))) ; => "{\"name\":\"rontolisp\",\"stars\":1}" ``` ```lisp (rontolisp:json-stringify (rontolisp:alist-hash-table '(("name" . "rontolisp") ("stars" . 1)))) ; => "{\"name\":\"rontolisp\",\"stars\":1}" ``` The inverses [`rontolisp:hash-table-plist`](../reference/functions/rontolisp-hash-table-plist.md) and [`rontolisp:hash-table-alist`](../reference/functions/rontolisp-hash-table-alist.md) flatten a parsed object back into a list you can walk with `getf` or `assoc` (a parsed object has string keys, so `assoc` with `:test 'equal`): ```lisp (rontolisp:hash-table-plist (rontolisp:json-parse "{\"n\": 1}")) ; => ("n" 1) ``` ```lisp (rontolisp:hash-table-alist (rontolisp:json-parse "{\"n\": 1}")) ; => (("n" . 1)) ``` They are lightweight subsets of the same-named `alexandria` functions and, like the JSON functions, compile in on every backend. ## A complete program The pieces combine into the typical JSON-API round trip: build the request body with `json-stringify`, POST it, await the response and parse the body with `json-parse`. Save the following as `fetch-post.lisp`: ```lisp (let ((req (make-hash-table :test 'equal))) (setf (gethash "name" req) "rontolisp") (setf (gethash "stars" req) 1) (let* ((payload (rontolisp:json-stringify req)) (res (rontolisp:await (rontolisp:fetch "https://httpbin.ik.am/post" `(:method "POST" :headers (("Content-Type" . "application/json")) :body ,payload)))) (body (rontolisp:await (rontolisp:read-all (getf res :body)))) (json (rontolisp:json-parse body))) (print (getf res :status)) (write-line (or (gethash "data" json) body)))) ``` ```console 200 {"name":"rontolisp","stars":1} ``` ### Running it On the interpreter: ```bash rontolisp fetch-post.lisp ``` Compiled to a JVM class (the class is named after the output file): ```bash rontolisp fetch-post.lisp -o FetchPost.class java FetchPost ``` Compiled to a WASM component (wasmtime 46+; note `-S http=y`, which grants outgoing HTTP — without it instantiation fails because the `wasi:http` imports are unavailable): ```bash rontolisp fetch-post.lisp -o fetch-post.wasm --component wasmtime run -W gc=y -W exceptions=y -S http=y fetch-post.wasm ``` ## Fetching from a reactor (`--no-wasi --host-fetch`) A [`--no-wasi` reactor](wasm-gc-module.md#no-wasi-reactor-mode) imports no WASI, so it has no `wasi:http` to fetch through — but the hosts that drive one (a Cloudflare Worker, node, a browser page) have an HTTP client of their own. `--host-fetch` routes `rontolisp:fetch` at it, as two injected imports — `env.fetch(request-json) -> response-head-json` for the request and the reply's head, and `env.readResponseBody(ptr, cap) -> i32` for the reply's body: ```bash rontolisp worker.lisp -o worker.wasm --no-wasi --host-fetch --optimize=size ``` Nothing in the Lisp changes — same options, same `(:status :headers :body)` answer, `:body` an asynchronous stream like everywhere else — but three things are particular to this backend: - **The fetch belongs inside an export, not at the top level.** A reactor has no `_start`: the host instantiates it and calls an exported function. A JavaScript host implements `env.fetch` with `WebAssembly.Suspending` (JSPI), which parks the whole wasm stack until the promise settles, and `_initialize` is the one stack it may not park — so a fetch the *load path* reaches is refused there. The build prints a warning naming it. - **The body arrives after the head, one chunk at a time.** `env.fetch` answers status and headers; the octets are pulled through `env.readResponseBody` as the drain asks for them, into a buffer the module passes (the host answers how many it wrote, `0` for end of stream). So a large reply never becomes a JSON string, a *binary* reply crosses as the octets it is, and a Worker can forward a streamed upstream response straight to its own client. - **Started == settled, and settled means the HEAD.** The future is settled the moment `fetch` returns (the stack was parked for the round trip to the headers), so `await` never suspends and two fetches never overlap — the [degenerate async shape](async.md#under-the-hood-wasi-preview-3-futures--streams) Preview 1 has everywhere. A transport failure *before* the head therefore signals at the `fetch` call; one *during* the body signals at the drain, like every other backend. One reply body is live at a time: starting the next `fetch` before draining the previous one makes that drain signal rather than answer the new reply's octets. (Under [`--reentrant`](wasm-host-boundary.md#overlapping-calls---reentrant) each reply head carries its own `"body-id"` and every pull names it, so replies are drained independently and nothing is superseded.) The host side owes one obligation in return, which the build also prints: enter every export through `WebAssembly.promising` and serialise the calls (or compile [`--reentrant`](wasm-host-boundary.md#overlapping-calls---reentrant) to overlap them on one instance). A suspended handler returns control to the event loop, and a second request entering the same instance would share its globals and its allocator — the module refuses that re-entry with a trap rather than corrupting both calls. A synchronous `env.fetch` (node without JSPI, a test stub) needs none of this and is equally valid. Adding [`--emit-js-glue`](wasm-host-boundary.md#generating-the-host-glue---emit-js-glue) writes that obligation as JavaScript beside the module — both imports, the `promising` entry and the queue — leaving the host only what its `fetch` does. The usual shape is a served reactor: an [`http-handler`](http-handler.md) or a [Clack application](clack.md#a-host-that-calls-you-the-reactor-build) compiled with these flags exports `handle-request`, and its handler is what fetches. [`examples/cloudflare-workers/dog-fetcher`](https://github.com/making/rontolisp/tree/develop/examples/cloudflare-workers/dog-fetcher) is exactly that, JavaScript side included — one source that also runs on the interpreter, the JVM and a `wasi:http` component. For raw TCP instead of HTTP — or to implement the *server* side — see the [TCP Sockets guide](tcp-sockets.md). --- # FILE: references/guides/http-handler.md # Serving HTTP (http-handler) Hand-rolling HTTP over `read-line`/`write-line` (as the [TCP Sockets guide](tcp-sockets.md) demonstrates with `http-hello.lisp`) is instructive, but for a plain request/response server [`rontolisp:http-handler`](../reference/functions/rontolisp-http-handler.md) does the parsing for you. You write a handler that takes the Clack environment property list (`:request-method` / `:path-info` / `:query-string` / `:headers` / `:raw-body` / ...) and returns the Clack response list `(status headers body)` — the protocol of [Clack Web Applications](clack.md), which is why a Clack application is served with zero per-request conversion: ```console (defun handle (env) (list 200 '(:content-type "text/plain") (list (format nil "Hello from rontolisp!~%~a ~a~%" (getf env :request-method) (getf env :path-info))))) (rontolisp:http-handler 'handle 8080) ``` Save it as `app.lisp` (also shipped as [`examples/net/http-handler.lisp`](https://github.com/making/rontolisp/blob/develop/examples/net/http-handler.lisp)), then run it on any of the three supported backends below. ## The handler contract The handler receives Clack's environment property list, with these keys — always all present: | env key | value | |---------|-------| | `:request-method` | the method as an upcased interned keyword (`:GET`, `:POST`, ...), so `(eq m :POST)` works | | `:script-name` | always `""` | | `:path-info` | the percent-decoded request path | | `:query-string` | the raw text after the first `?`, or `nil` when there is none | | `:server-name` / `:server-port` | from the `Host` header when present, otherwise the listener's | | `:server-protocol` | a keyword, e.g. `:HTTP/1.1` | | `:request-uri` | the raw request target verbatim (still percent-encoded, query included) | | `:url-scheme` | `"http"` or `"https"` | | `:remote-addr` / `:remote-port` | the real peer on the interpreter and the JVM; `nil` on the WASI component (`wasi:http@0.3.0` exposes no peer accessor) | | `:headers` | an `equal` hash table keyed by **lowercased** header names — look up with `(gethash "content-type" (getf env :headers))`; repeated headers join with `", "`; never `nil` | | `:content-type` / `:content-length` | from that table (`nil` when absent; `:content-length` an integer) | | `:raw-body` | the request body (below) | By default `:raw-body` is rontolisp's **asynchronous** stream: a handler that reads it drains it with `(rontolisp:await (rontolisp:read-all (getf env :raw-body)))` and must be an [`rontolisp:async-defun`](../reference/special-forms/rontolisp-async-defun.md). With the optional directive argument `(rontolisp:http-handler 'handle 8080 :raw-body :buffered)` the body is instead read in full up front and handed over as a **synchronous** in-memory bivalent stream — readable with `read-line`/`read-char` *and* `read-byte`/`read-sequence`, with a real `file-position` — which is what a Clack application (lack-request, http-body) needs; a bodiless request then gets `:raw-body nil`. The handler returns Clack's positional response list `(status headers body)`: - `status` — a **required** integer; a non-integer car signals an error. - `headers` — a keyword plist (`'(:content-type "text/plain")`, the idiomatic form) or a dotted alist — accepted so a [`rontolisp:fetch`](http-fetch.md) result's `:headers` can be passed straight through. Repeated names each become their own header line (repeated `:set-cookie` is correct by construction); `content-length`/`transfer-encoding` are dropped (the server computes them); `nil` is fine. - `body` — a **list of strings** (joined), `nil` or omitted (an empty body — the two-element `(status headers)` form is valid), an `(unsigned-byte 8)` vector (written byte for byte — a binary response is byte-exact), or a rontolisp stream (e.g. a proxied fetch body). A **bare string signals an error** — deliberately, and faithfully to Clack, which refuses strings too. A pathname body means "serve this file" (lack's static-file middleware answers one); it is a distinct value here and is refused as unsupported until the transport can serve it. A function response is supported in Clack's delayed form only — `(lambda (responder) ... (funcall responder (list 200 nil (list "later"))))` — and the streaming-writer form is refused. One migration hazard is worth spelling out for handlers written against the pre-Clack contract: the response side fails loudly (the errors above), but the request side fails silently — `(getf env :method)` in a half-migrated handler just returns `nil`. The *client* side is unchanged: [`rontolisp:fetch`](http-fetch.md) still yields its `(:status :headers :body )` result plist. ## On the interpreter `http-handler` starts a blocking embedded HTTP server on port 8080 (one virtual thread per request) and serves until the process is stopped with `Ctrl-C`: ```console $ rontolisp app.lisp $ curl http://127.0.0.1:8080/hello Hello from rontolisp! GET /hello ``` ## Compiled to a JVM class The same source compiles to a **JVM class** serving the same way. Unlike other compiled rontolisp programs, the class is not self-contained: it implements the embedded server's handler interface, so the rontolisp executable JAR (`rontolisp-0.1.0-SNAPSHOT-exec.jar`, the same download as in [Build & Install](../getting-started/build.md)) must be on the classpath when running it: ```console $ rontolisp app.lisp -o App.class $ java -cp rontolisp-0.1.0-SNAPSHOT-exec.jar:. App $ curl http://127.0.0.1:8080/hello Hello from rontolisp! GET /hello ``` ## Compiled to a WASI HTTP component It also compiles to a **WASI HTTP component** that runs under `wasmtime serve` (wasmtime 47+ — 46 also serves it, but collapses under concurrent load; see the throughput section below): ```console $ rontolisp app.lisp -o app.wasm --component $ wasmtime serve -W gc=y -W exceptions=y app.wasm $ curl http://127.0.0.1:8080/hello Hello from rontolisp! GET /hello ``` There the module exports `wasi:http/handler@0.3.0` (the async WASI 0.3 HTTP world) and the host owns the socket, so the `port` argument is ignored. The flags enable what the component actually uses: the WebAssembly GC proposal (`-W gc=y`) and the exception-handling proposal (`-W exceptions=y`, which the Lisp-written HTTP glue uses to detect end-of-body). The handler is lifted as a **callback async** export: a handler that suspends (awaiting a timer, a fetch or a body read) hands control back to the host, which delivers each completion event through the component's callback — all of it part of the base component-model async ABI, which is default-on in wasmtime 46+, so no gated feature flags are needed. The response is still delivered mid-task through `canon task.return`, and the body streams after it. ## Other WASI HTTP runtimes The component asks its host for `wasi:http` **0.3** (async) plus wasm-GC. wasmtime 46+ serves it, and so does **wasmCloud**: `wash` 2.5.2 runs it with `wash dev`, given `dev.wasm_proposals: [gc, exception-handling, component-model-async]` in the project manifest. Install it with `curl -fsSL https://wasmcloud.com/sh | bash` — verified on wash 2.6.1. (Binaries picked by tag from the separate `wasmCloud/wash` repository are a different, older line: 2.0.0-rc.x offers `wasi:http` **0.2** only and rejects the component while extracting its interfaces.) **Spin** runs it too, from the [canary build](https://github.com/spinframework/spin/releases/tag/canary) (4.1.0-pre0) on — its embedded wasmtime is 47, which enables the WebAssembly GC and exception-handling proposals by default, so no flag is needed. Drop a `spin.toml` beside the program: ```toml spin_manifest_version = 2 [application] name = "rontolisp-http-handler" version = "0.1.0" [[trigger.http]] route = "/..." component = "hello" [component.hello] source = "app.wasm" [component.hello.build] command = "rontolisp app.lisp -o app.wasm --component" ``` ```console $ spin build && spin up Serving http://127.0.0.1:3000 $ curl http://127.0.0.1:3000/hello Hello from rontolisp! GET /hello ``` Spin owns the socket and listens on **3000**, so the `port` argument is ignored here as well. A handler that calls [`rontolisp:fetch`](http-fetch.md) also needs the upstream host on the component's `allowed_outbound_hosts` — Spin denies outbound HTTP by default: ```toml [component.dog] source = "app.wasm" allowed_outbound_hosts = ["https://dog.ceo"] ``` Released Spin **4.0.2 cannot** run the component. Its embedded wasmtime is 44, which speaks the `wasi:http@0.3.0-rc-2026-03-15` snapshot rather than the released `wasi:http@0.3.0`, so the imports fail to link even with GC turned on (and the released 4.0.2 binary has no switch to turn GC on: the `--experimental-wasm-feature` option is compiled into canary builds only). **jco** cannot run it either — it does not implement the 0.3 async ABI. ## Query strings `:path-info` carries the (percent-decoded) path only, so route comparisons are exact. When the request has a query string it arrives separately under `:query-string` — the raw text after the first `?`, or `nil` when there is none. Parse it with the query-string functions of the URL library, [`rontolisp:query-param`](../reference/functions/rontolisp-query-param.md) and [`rontolisp:query-params`](../reference/functions/rontolisp-query-params.md) (both url-decode keys and values, and both accept `nil`): ```console (defun handle (env) (list 200 '(:content-type "text/plain") (list (format nil "Hello, ~a!~%" (or (rontolisp:query-param (getf env :query-string) "name") "world"))))) (rontolisp:http-handler 'handle 8080) ``` ```console $ curl 'http://127.0.0.1:8080/greet?name=ronto%20lisp' Hello, ronto lisp! $ curl http://127.0.0.1:8080/greet Hello, world! ``` ## Calling other services from a handler [`rontolisp:fetch`](http-fetch.md) works inside a served handler on all three backends, enabling the classic proxy / aggregator shape. A handler that awaits is an asynchronous function, so define it with [`rontolisp:async-defun`](../reference/special-forms/rontolisp-async-defun.md) instead of `defun`: ```console (rontolisp:async-defun handle (env) (let ((res (rontolisp:await (rontolisp:fetch "http://127.0.0.1:9000/upstream")))) (list (getf res :status) (getf res :headers) (getf res :body)))) (rontolisp:http-handler 'handle 8080) ``` The fetch result's `:headers` alist goes into the response's `headers` slot as is, and its `:body` stream into the `body` slot — the server drains it, and because the stream's chunks are the upstream's octets and nothing decodes them on the way through, the relay is byte-exact: an image comes out as the image that went in. On the WASI component backend the outgoing-request machinery rides along in the same component — serve and serve+fetch are one component shape, importing `wasi:http/client@0.3.0`, which `wasmtime serve` provides by default (no `-S http=y` needed): ```console $ rontolisp proxy.lisp -o proxy.wasm --component $ wasmtime serve -W gc=y -W exceptions=y proxy.wasm ``` A complete example is [`examples/net/dog-fetcher.lisp`](https://github.com/making/rontolisp/blob/develop/examples/net/dog-fetcher.lisp), a reproduction of [wasmCloud's dog-fetcher example](https://wasmcloud.com/docs/v1/examples/rust/component/dog-fetcher/): every request fetches a random dog picture URL from the dog.ceo API and answers it as JSON. ## Keeping State: a store, not a global On the interpreter and the JVM the server is one long-lived process, so a global hash table survives between requests. **A served component's does not** — and the way it does not is worse than "it resets every time". How long an instance lives is the host's decision, and the hosts disagree: | host | instance lifetime | |---|---| | `wasmtime serve` | 128 requests, then retired (`--max-instance-reuse-count`) | | Spin | 128 requests (it inherits wasmtime's default) | | wasmCloud `wash dev` | 1 request — always a fresh instance | So a global neither survives the run nor resets per request: under wasmtime and Spin the top level runs again on every 128th request, and everything the handler accumulated in a global vanishes with it. Treat top-level side effects as idempotent, and keep anything that must survive outside the component. The way to keep state is therefore to put it outside the component — in a WIT interface the handler *calls*, bound with [`rontolisp:wit-import`](../reference/functions/rontolisp-wit-import.md). A served component imports it alongside its fixed `wasi:http` surface: ```console (rontolisp:wit-import "wit/keyvalue.wit" :interface "wasi:keyvalue/store@0.2.0-draft" :package kv) (defun handle (env) (let* ((page (getf env :path-info)) (bucket (kv:open "")) (seen (kv:bucket-get bucket page)) (hits (+ 1 (if seen (parse-integer seen) 0)))) (kv:bucket-set bucket page (princ-to-string hits)) (list 200 nil (list (format nil "~a: ~a hits~%" page hits))))) (rontolisp:http-handler 'handle 8080) ``` ```console $ rontolisp page-hits-server.lisp -o server.wasm --component $ wasmtime serve -W gc=y -W exceptions=y -S keyvalue=y server.wasm ``` The same source runs on the interpreter and the JVM, where a [provider](../reference/functions/rontolisp-wit-provide.md) written in Lisp answers the interface instead. Whether the counts *survive* on a component is the host's business: wasmtime's built-in key-value provider is an in-memory store that starts empty on every request (verified: each request reports 1 hit), while a host that links an out-of-process provider — wasmCloud (`wash dev`), say — keeps them. The worked example is [`examples/wit/keyvalue`](https://github.com/making/rontolisp/tree/develop/examples/wit/keyvalue). ## Throughput, and what the component pays for The three backends are in the same league on a trivial handler. Measured on one machine (16 concurrent connections, 10 s closed loop, a handler that answers `"Hello " + :path-info`; wasmtime 47.0.2, `wasmtime serve` at its defaults): | backend | requests/s | mean | p99 | |---|---|---|---| | interpreter | 33 900 | 0.47 ms | 0.99 ms | | JVM class | 36 600 | 0.44 ms | 0.88 ms | | WASI component | 24 500 | 0.65 ms | 1.19 ms | The component row needs **wasmtime 47 or newer**. On wasmtime 46 a *failing* runtime type test — the ordinary misses of a dynamic language's type dispatch — is a call into the host that takes an engine-global lock. One connection merely pays it (about 10%); concurrent connections contend for it, and throughput *falls* as connections are added — to roughly a fifteenth at 16 connections. wasmtime 47 checks these types inline (every type rontolisp emits is final, which is exactly the shape its fast path needs), and the collapse disappears. The component's gap is **instantiation**, not the handler: the host runs the whole top level (`_start`) once per instance, and it retires an instance every `--max-instance-reuse-count` requests. Lowering that knob makes the cost visible — at `--max-instance-reuse-count 1` the same component drops to roughly a third of the throughput above, because every request pays a full instantiation. Two consequences worth knowing: - **Where the top level goes matters, and how much depends on the host.** Work done at top level is paid once per instance — amortized over 128 requests under wasmtime and Spin, but paid on *every* request under wasmCloud, where the same handler serves 7 900 rps against wasmtime's 24 500. A `ql:quickload "clack"` program and a bare `rontolisp:http-handler` one serve at nearly the same rate under wasmtime for exactly this reason, and at visibly different rates under wasmCloud. - **Tree shaking is about size, not speed here.** It shakes the compiled core module (a serve component loses a few percent; a non-serve component can lose 90%), which shortens instantiation slightly, but it does not change the steady-state per-request cost. ## Limitations Request and response headers are marshalled on every backend, including the WASI component: the handler reads `:headers` (an `equal` hash table keyed by lowercased header names) from the environment and the response's `headers` element is written back. Inside a served component handler, `random`, the time built-ins and `print` (to the host's stdout) all work — the component bridges them to the `wasi:random`, `wasi:clocks` and `wasi:cli` interfaces every `wasi:http` host provides. `uiop:getenv` reads the host environment too — a served component imports `wasi:cli/environment@0.3.0`, so `wasmtime serve --env NAME=value` (or `-S inherit-env=y`) reaches the handler — and file streams are unavailable. See the [`rontolisp:http-handler`](../reference/functions/rontolisp-http-handler.md) reference page for the details. For the *client* side of HTTP, use `rontolisp:fetch` — see the [HTTP Requests guide](http-fetch.md). To work at the raw socket level instead (any TCP protocol, or TLS), see the [TCP Sockets guide](tcp-sockets.md). --- # FILE: references/guides/java-interop.md # Java Interop The `java` package lets rontolisp drive arbitrary Java APIs by reflection — construct objects, call instance and static methods, read fields, and turn a rontolisp lambda into a Java interface instance. It is how the Swing demos in `examples/` (`java-interop.lisp`, `swing.lisp`, `life-gui.lisp`) put a window on the screen without any bespoke Java glue. > **JVM only (interpreter and compiled `.class`).** Interop values are opaque > host-object references resolved by reflection, so the feature needs a real > JVM: it works under the **JVM-hosted interpreter** (`java -jar rontolisp.jar > program.lisp`) and in a **JVM-compiled program** (`-o Prog.class`, run with > `java Prog`) — the compiler embeds a small reflection bridge into the > generated class, so the output stays a single self-contained `.class` file > (running one that uses `java:` requires a JRE at least as new as the one > rontolisp was built with). The WASM backend cannot lower host references, so > compiling `java:` to `.wasm` remains a `Cannot compile: java:...` error. The > GraalVM native binary (`rontolisp program.lisp`) can **compile** a `java:` > program to a `.class`, but cannot **interpret** one: a native image only > contains the classes and members its build registered for reflection, and > rontolisp's build registers none for interop, so there even `(java:static > "java.lang.Math" "max" 3 7)` fails with `No such class`. ## The functions The package is not part of Common Lisp, so its functions are referenced with the `java:` qualifier (or unqualified after `(in-package java)`). | Function | Purpose | |----------|---------| | `java:new` | Construct a host object: `(java:new "fqcn" args...)` | | `java:call` | Invoke an instance method: `(java:call obj "method" args...)` | | `java:static` | Invoke a static method: `(java:static "fqcn" "method" args...)` | | `java:field` | Read a static or instance field: `(java:field class-or-obj "name")` | | `java:proxy` | Adapt a callable to an interface: `(java:proxy "iface" callable)` | A constructed or returned object prints opaquely as `#>` and can be passed back into `java:call`/`java:field`: ```lisp (java:call (java:new "java.lang.StringBuilder" "ab") "length") ; => 2 ``` ```lisp (java:static "java.lang.Math" "max" 3 7) ; => 7 ``` ```lisp (java:field "java.lang.Integer" "MAX_VALUE") ; => 2147483647 ``` ## Value marshalling Arguments and results are converted between rontolisp and Java automatically: | rontolisp | Java (in) | Java (out) | |-----------|-----------|------------| | integer | `int`/`long`/`short`/`byte`/`float`/`double` (and their boxes) | `int`/`long`/... → integer | | float | `double`/`float` (and boxes) | `double`/`float` → float | | string | `String`, or `char` if length 1 | `String` → string | | character | `char`/`Character` | `Character` → character | | `t` / `nil` | `boolean` (`nil` also → any `null` reference) | `boolean` → `t`/`nil` | | a `java` object | the wrapped host object | any other object → a `java` object | | a function/lambda | a `java:proxy` over the matching interface | — | | a proper list / a vector | `T[]` (element-wise, incl. primitives), or `List`/`Collection`/`Iterable` | any Java array → a list | A Java `null` (and a `void` method) comes back as `nil`. A proper list — or a rank-1 array made with `make-array` — passed where a Java array is expected is converted element-wise to the component type (including primitive arrays like `int[]`), and where a `List`/`Collection`/`Iterable` is expected it becomes a `java.util.List`; nested lists convert recursively. In the other direction a Java **array** result becomes a Lisp list, while a returned `java.util.List` stays an opaque `java` object whose methods you call: ```lisp ;; in: the list becomes a Collection (java:static "java.util.Collections" "max" (list 3 9 4)) ; => 9 ``` ```lisp ;; in: (1 2 3) -> int[]; out: the int[] result -> a list (java:static "java.util.Arrays" "copyOf" (list 1 2 3) 2) ; => (1 2) ``` Symbols, hash tables, dotted (improper) lists and multidimensional (rank-2+) arrays are **not** bridged. ## Overload resolution When a class has several constructors or methods of the same name and arity, `java` picks the overload whose arguments convert at the **lowest total cost** — an exact match beats a widening conversion, which beats a lossy/boxed one — with ties broken by a stable signature ordering. So an integer argument prefers an `int` parameter over `long`/`double`, and the choice never depends on the order reflection happens to return methods: ```lisp ;; Math.max is overloaded for int/long/float/double; an integer picks int, ;; so the result is an integer, not a float. (java:static "java.lang.Math" "max" 3 7) ; => 7 ``` When no integer overload exists the integer is converted to the available type: ```lisp (java:static "java.lang.Math" "sqrt" 16) ; => 4.0 ``` ## Varargs A varargs method (e.g. `String.format(String, Object...)`) accepts any number of trailing arguments; they are packed into the varargs array automatically. A fixed-arity overload is preferred when both match, and a list/vector passed in the varargs position can also supply the whole array itself: ```lisp ;; 1 and "x" are packed into the Object... array (java:static "java.lang.String" "format" "%s-%s" 1 "x") ; => "1-x" ``` ```lisp ;; the list is the CharSequence[] varargs array itself (java:static "java.lang.String" "join" "-" (list "a" "b" "c")) ; => "a-b-c" ``` ## Callbacks via java:proxy `java:proxy` makes a host interface instance backed by a rontolisp callable. The callable is applied as `(callable "method-name" arg...)` for every interface method, so a single lambda can implement the whole interface and dispatch on the method name. Its return value is marshalled back to the method's return type (`void` methods ignore it): ```lisp ;; A java.util.function.Supplier whose get() returns a rontolisp value. (java:call (java:proxy "java.util.function.Supplier" (lambda (method) 42)) "get") ; => 42 ``` A callable passed directly where an interface is expected is wrapped in a proxy automatically, which is what lets a Swing `ActionListener` be a plain lambda: ```console (java:call button "addActionListener" (lambda (method event) (handle-click))) ``` ## A Swing example `examples/jvm/java-interop.lisp` builds a small window directly through the package (interpret it -- or compile it to a `.class` -- on a machine with a display): ```console (defvar *frame* (java:new "javax.swing.JFrame" "java interop")) (defvar *label* (java:new "javax.swing.JLabel" "click count: 0")) (defvar *button* (java:new "javax.swing.JButton" "Increment")) (defvar *panel* (java:new "javax.swing.JPanel" (java:new "java.awt.BorderLayout" 12 12))) (defvar *count* 0) (java:call *button* "addActionListener" (java:proxy "java.awt.event.ActionListener" (lambda (method event) (setq *count* (+ *count* 1)) (java:call *label* "setText" (concatenate 'string "click count: " (princ-to-string *count*)))))) (java:call *panel* "add" *label* (java:field "java.awt.BorderLayout" "CENTER")) (java:call *panel* "add" *button* (java:field "java.awt.BorderLayout" "SOUTH")) (java:call *frame* "setContentPane" *panel*) (java:call *frame* "setDefaultCloseOperation" (java:field "javax.swing.WindowConstants" "DISPOSE_ON_CLOSE")) (java:call *frame* "setSize" 360 180) (java:call *frame* "setVisible" t) ``` `examples/jvm/swing.lisp` builds a reusable grid-window helper on top of these five functions -- wrapped in a `swing` [package](../reference/packages.md) of its own, spliced in with `(require :swing "swing.lisp")` -- and `examples/jvm/life-gui.lisp` animates Conway's Game of Life with it (`swing:grid-window`, `swing:paint`, ...). ## Limitations - **JVM only**: the interpreter (`java -jar rontolisp.jar`) and JVM-compiled classes (`java Prog`). Not on the WASM backend, and not when interpreting in the GraalVM native binary, whose image carries no reflection metadata for the interop classes (the native binary can still *compile* a `java:` program to a `.class`). - In a compiled class the five functions work in call position only: they have no first-class value, so `#'java:call` or `(funcall 'java:new ...)` is a compile error (wrap them in your own `defun` instead), and the embedded `eval` runtime does not know them either. A compiled program that uses `java:` needs a JRE at least as new as the one rontolisp was built with. - Symbols, hash tables, dotted (improper) lists and multidimensional (rank-2+) arrays are not marshalled — pass them as Java collections you build with `java:new`/`java:call` instead. - A returned `java.util.List` (unlike a Java array) stays an opaque `java` object: it keeps its identity and mutability, so read it with `java:call` (`"get"`, `"size"`, ...) rather than list functions. - Overload resolution is by argument cost, not the full Java type-inference rules; an ambiguous call resolves to the lowest-cost (then lowest-signature) candidate rather than signalling an ambiguity error. - It is a full host-reflection bridge, so it can run arbitrary Java code: treat a program that uses `java:` with the same trust as any other JVM program. --- # FILE: references/guides/linear-algebra.md # Vectors & Matrices (linalg) The `linalg` package provides a numpy-style API for vectors and matrices: constructors, shape manipulation, elementwise arithmetic, products, reductions, discrete calculus (differences and numerical derivatives), and linear algebra (determinant, inverse, linear solving). Like the JSON library, `linalg` is implemented once in Lisp source (`linalg.lisp`): the interpreter loads the definitions lazily on the first use of a `linalg:` function, and the compile path splices them into the program when it references the package. There is no per-backend code, so every function behaves identically on the interpreter, the JVM compiler, WASM Preview 1 and the WASM component. ## Data representation linalg constructors build [packed float arrays](../reference/data-types.md): unboxed `(array double-float)` values, the same representation as an `#d(...)` literal. A vector is a rank-1 array, printed `#d(1.0 2.0 ...)`, and a matrix is a rank-2 array, printed with the nested `#d((...) ...)` form -- the `#d` marks the unboxed packed representation, so the printed form reads back as a packed array. Individual elements are read and written with `aref`, and any array built elsewhere -- packed or a general boxed array -- can be handed to a linalg function. Arrays of higher rank work too: the elementwise operations, the reductions, `reshape`/`flatten` and `array-equal` walk the elements in flat row-major order and accept any rank, while `matmul` stacks rank >= 3 on its last two axes (numpy's own `np.matmul` rule) and `dot`/`outer`/`det`/`inv`/`solve`/`trace`/`transpose` stay defined for vectors and matrices (rank <= 2), like numpy's specialized routines. [`linalg:from-list`](../reference/functions/linalg-from-list.md) / [`linalg:to-list`](../reference/functions/linalg-to-list.md) convert between arrays and lists. linalg computes in floating point, prioritizing speed: every constructor and array-building operation returns a packed double-float array by default (single-float is available too -- see [Single-float precision](#single-float-precision)), and [`linalg:det`](../reference/functions/linalg-det.md), [`linalg:inv`](../reference/functions/linalg-inv.md) and [`linalg:solve`](../reference/functions/linalg-solve.md) run in floating point (like numpy), so a general inverse carries the usual rounding and a nearly singular determinant can be a small epsilon rather than exactly `0`. A reduction follows the element type numpy-style: a reduction over a packed or float array is a double, while a reduction over a plain integer array (a bare `#(1 2 3)` literal) stays an integer or exact ratio; [`linalg:norm`](../reference/functions/linalg-norm.md) is always a float because `sqrt` is. One cross-backend caveat: the WASM backends print a non-terminating float at fewer significant digits than the interpreter and JVM, so a rounded inverse or an irrational norm can look different across backends even though the underlying `double` is identical. ## A worked example ```lisp (linalg:eye 3) ; => #d((1.0 0.0 0.0) (0.0 1.0 0.0) (0.0 0.0 1.0)) (linalg:arange 5) ; => #d(0.0 1.0 2.0 3.0 4.0) (linalg:linspace 0 1 5) ; => #d(0.0 0.25 0.5 0.75 1.0) (linalg:matmul #2A((1 2) (3 4)) #2A((5 6) (7 8))) ; => #d((19.0 22.0) (43.0 50.0)) (linalg:det #2A((1 2) (3 4))) ; => -2.0 (linalg:inv #2A((4 0) (2 4))) ; => #d((0.25 0.0) (-0.125 0.25)) (linalg:solve #2A((4 0) (2 4)) #(8 8)) ; => #d(2.0 1.0) ``` The `inv` and `solve` matrices above are chosen so their float results are exact and print identically on every backend; a general inverse such as `(linalg:inv #2A((1 2) (3 4)))` computes the same values but carries floating-point rounding. `la` is a built-in nickname for `linalg`, so every `linalg:` call can also be written with the shorter `la:` prefix: ```lisp (la:arange 5) ; => #d(0.0 1.0 2.0 3.0 4.0) ``` ## Elementwise arithmetic and broadcasting [`linalg:add`](../reference/functions/linalg-add.md), [`linalg:sub`](../reference/functions/linalg-sub.md), [`linalg:mul`](../reference/functions/linalg-mul.md) and [`linalg:div`](../reference/functions/linalg-div.md) operate elementwise and broadcast by numpy's rules: a scalar operand on either side is broadcast over the other operand's shape, and two arrays of different shapes align their trailing axes -- each aligned pair of extents must be equal or contain a 1 (a missing leading axis counts as 1), and the axis of extent 1 is stretched over the other operand's extent. A pair that fits neither rule signals a shape-mismatch error. The result keeps the first array operand's element type, matching the mixed-width rule. Note that `mul` is the Hadamard (elementwise) product -- the matrix product is [`linalg:matmul`](../reference/functions/linalg-matmul.md) (or the rank-dispatching [`linalg:dot`](../reference/functions/linalg-dot.md)). Arbitrary per-element transformations go through [`linalg:emap`](../reference/functions/linalg-emap.md). The four also answer to their CL operator spellings -- [`linalg:+`](../reference/functions/linalg-plus.md), [`linalg:-`](../reference/functions/linalg-minus.md), [`linalg:*`](../reference/functions/linalg-star.md) and [`linalg:/`](../reference/functions/linalg-slash.md) -- which are n-ary left folds of `add` / `sub` / `mul` / `div`, so `(linalg:+ a b c)` broadcasts step by step and each step is still the accelerated kernel. The degenerate arities follow CL: no argument gives the identity (`0` / `1`), a single argument to `+` and `*` is itself, and a single argument to `-` and `/` is the negation / reciprocal. (The [`vec:`](simd-acceleration.md) package's operator aliases are strictly binary instead -- its kernels are fixed-arity by design.) The frequent per-element operations also exist under their numpy ufunc names: [`linalg:exp`](../reference/functions/linalg-exp.md), [`linalg:log`](../reference/functions/linalg-log.md), [`linalg:tanh`](../reference/functions/linalg-tanh.md), [`linalg:sin`](../reference/functions/linalg-sin.md), [`linalg:cos`](../reference/functions/linalg-cos.md), [`linalg:tan`](../reference/functions/linalg-tan.md), [`linalg:asin`](../reference/functions/linalg-asin.md), [`linalg:acos`](../reference/functions/linalg-acos.md), [`linalg:atan`](../reference/functions/linalg-atan.md), [`linalg:sinh`](../reference/functions/linalg-sinh.md), [`linalg:cosh`](../reference/functions/linalg-cosh.md), [`linalg:sqrt`](../reference/functions/linalg-sqrt.md), [`linalg:abs`](../reference/functions/linalg-abs.md), [`linalg:square`](../reference/functions/linalg-square.md), [`linalg:negative`](../reference/functions/linalg-negative.md), [`linalg:sign`](../reference/functions/linalg-sign.md) and [`linalg:reciprocal`](../reference/functions/linalg-reciprocal.md), the binary [`linalg:power`](../reference/functions/linalg-power.md), plus the comparison selects [`linalg:maximum`](../reference/functions/linalg-maximum.md), [`linalg:minimum`](../reference/functions/linalg-minimum.md), [`linalg:clip`](../reference/functions/linalg-clip.md) and [`linalg:relu`](../reference/functions/linalg-relu.md) (defined by the strict comparison `(if (> x y) x y)` and its mirrors, so the second operand or the bound wins any false comparison -- ties and `NaN` included, identically on every backend). Each is equivalent to the obvious `emap` (or `mul` / `div` / `maximum` / `minimum` call), but as named functions they are accelerated under [`--simd`](simd-acceleration.md#accelerating-linalg), which `emap` with an arbitrary callback never is. [`linalg:softmax`](../reference/functions/linalg-softmax.md), [`linalg:log-softmax`](../reference/functions/linalg-log-softmax.md) and [`linalg:erf`](../reference/functions/linalg-erf.md) sit here for the same reason `relu` does -- they are not in numpy proper (they are scipy's / torch's), but they are the array-level primitive an activation layer needs. The two softmaxes are the max-subtracted, numerically stable forms; `erf` is what the exact GELU is built from, and it sums the all-positive-term series rather than the alternating Maclaurin series, so it stays accurate to a double's last ulps where the naive form loses every digit by `|x| ~ 3`. One caveat about the flag, since this line began with it: the two softmaxes are built entirely out of accelerated members, so `--simd` reaches them, but `erf` is an `emap` over a scalar series and is the one name here the flag does not accelerate at all. ```lisp (linalg:add #(1 2 3) 10) ; => #d(11.0 12.0 13.0) (linalg:mul 2 #2A((1 2) (3 4))) ; => #d((2.0 4.0) (6.0 8.0)) (linalg:div #(1 2 3) 2) ; => #d(0.5 1.0 1.5) (linalg:sqrt #(4 9 16)) ; => #d(2.0 3.0 4.0) (linalg:square #2A((1 2) (3 4))) ; => #d((1.0 4.0) (9.0 16.0)) (linalg:mul #2A((1 2) (3 4)) #(10 20)) ; => #d((10.0 40.0) (30.0 80.0)) (linalg:add #2A((1 2) (3 4)) #2A((100) (200))) ; => #d((101.0 102.0) (203.0 204.0)) (linalg:+ #(1 2) #(3 4) #(10 10)) ; => #d(14.0 16.0) (linalg:- #(5 5)) ; => #d(-5.0 -5.0) ``` ## Reductions along an axis The reductions [`linalg:sum`](../reference/functions/linalg-sum.md), [`linalg:mean`](../reference/functions/linalg-mean.md), [`linalg:amax`](../reference/functions/linalg-amax.md) and [`linalg:amin`](../reference/functions/linalg-amin.md) take numpy's keyword arguments: an integer `:axis` (negative counts from the end) reduces along that axis instead of over the whole array -- the axis is dropped from the result, or kept as extent 1 when `:keepdims` is non-nil -- the shape that broadcasts back over the input, which is how a batch softmax subtracts its row maxima. [`linalg:argmax`](../reference/functions/linalg-argmax.md) and [`linalg:argmin`](../reference/functions/linalg-argmin.md) take the same `:axis` keyword and return per-slice indices (a packed double array for matrices, since linalg arrays have no integer width). [`linalg:var`](../reference/functions/linalg-var.md) and [`linalg:std`](../reference/functions/linalg-std.md) take the same `:axis` / `:keepdims` plus a `:ddof` divisor correction (`0` by default -- numpy's `np.var` and torch's `unbiased=False`; `1` gives the sample variance), which with `mean` along the same axis is the LayerNorm normalizer. [`linalg:reshape`](../reference/functions/linalg-reshape.md) accepts one `-1` extent and infers it from the element count. ```lisp (linalg:sum #2A((1 2 3) (4 5 6)) :axis 0) ; => #d(5.0 7.0 9.0) (linalg:sum #2A((1 2 3) (4 5 6)) :axis 1) ; => #d(6.0 15.0) (linalg:sum #2A((1 2 3) (4 5 6)) :axis -1 :keepdims t) ; => #d((6.0) (15.0)) (linalg:mean #2A((1 2 3) (4 5 6)) :axis 0) ; => #d(2.5 3.5 4.5) (linalg:argmax #2A((1 9 3) (7 5 6)) :axis 1) ; => #d(1.0 0.0) (linalg:std #2A((0 1 2) (3 4 5)) :axis 0) ; => #d(1.5 1.5 1.5) (linalg:softmax #2A((0 0) (1 1)) :axis 1) ; => #d((0.5 0.5) (0.5 0.5)) (linalg:shape (linalg:reshape (linalg:arange 12) '(3 -1))) ; => (3 4) ``` ## Rank-N shapes: joins, slices and stacked products Everything above is rank-generic, and a handful of operations exist to *reshape* that rank. [`linalg:expand-dims`](../reference/functions/linalg-expand-dims.md) inserts an extent-1 axis and [`linalg:squeeze`](../reference/functions/linalg-squeeze.md) removes one (numpy's `expand_dims` / `squeeze`, torch's `unsqueeze` / `squeeze`); [`linalg:concatenate`](../reference/functions/linalg-concatenate.md) joins a list of arrays along an axis that already exists and [`linalg:stack`](../reference/functions/linalg-stack.md) along a new one; [`linalg:slice`](../reference/functions/linalg-slice.md) is basic numpy slicing, and [`linalg:triu`](../reference/functions/linalg-triu.md) / [`linalg:tril`](../reference/functions/linalg-tril.md) keep one triangle of a matrix. rontolisp has no `x[:, :n]` syntax, so `slice` spells it as a list with one spec per axis: `nil` leaves that axis whole, `(start end)` or `(start end step)` selects along it, a negative index counts from the end, `nil` in the `start` or `end` position means "from the beginning" / "to the end", and a missing trailing spec leaves the remaining axes whole. Every axis is kept, exactly as numpy's `x[:, 0:3]` keeps both -- dropping an axis is what `linalg:row` does. [`linalg:matmul`](../reference/functions/linalg-matmul.md) is rank-generic too: at rank >= 3 on either side it is the **stacked** product (torch's `bmm` / `matmul`), where the last two axes are the matrix and every leading axis broadcasts. That is the shape a batched attention score has, so `(batch heads n d)` times `(batch heads d n)` gives `(batch heads n n)` in one call. ([`linalg:dot`](../reference/functions/linalg-dot.md) stays rank <= 2 on purpose: numpy's `np.dot` contracts against a different axis at higher rank, so passing it a stack signals an error pointing here rather than returning a wrong answer.) ```lisp (linalg:expand-dims #(1 2 3) 0) ; => #d((1.0 2.0 3.0)) (linalg:squeeze #2A((1 2 3))) ; => #d(1.0 2.0 3.0) (linalg:concatenate (list #(1 2) #(3))) ; => #d(1.0 2.0 3.0) (linalg:stack (list #(1 2) #(3 4)) :axis 1) ; => #d((1.0 3.0) (2.0 4.0)) (linalg:slice #2A((0 1 2) (3 4 5)) '(nil (0 2))) ; => #d((0.0 1.0) (3.0 4.0)) (linalg:slice #(0 1 2 3 4 5) '((nil nil 2))) ; => #d(0.0 2.0 4.0) (linalg:triu (linalg:ones '(3 3)) :k 1) ; => #d((0.0 1.0 1.0) (0.0 0.0 1.0) (0.0 0.0 0.0)) (linalg:shape (linalg:matmul (linalg:zeros '(2 3 4)) (linalg:zeros '(2 4 5)))) ; => (2 3 5) ``` ## Indexing, selection and masks [`linalg:take-rows`](../reference/functions/linalg-take-rows.md) selects axis-0 slices by an index vector (numpy's `x[mask]`, any rank) and keeps axis 0, while [`linalg:row`](../reference/functions/linalg-row.md) takes one slice by an integer and drops it (numpy's `x[i]`, so one image of a batch arrives at a forward pass as a plain vector). [`linalg:gather`](../reference/functions/linalg-gather.md) picks one element per row (`y[np.arange(n), t]`), and [`linalg:one-hot`](../reference/functions/linalg-one-hot.md) builds a label matrix. The elementwise comparisons [`linalg:equal`](../reference/functions/linalg-equal.md), [`linalg:greater`](../reference/functions/linalg-greater.md), [`linalg:greater-equal`](../reference/functions/linalg-greater-equal.md), [`linalg:less`](../reference/functions/linalg-less.md) and [`linalg:less-equal`](../reference/functions/linalg-less-equal.md) return 0.0/1.0 masks (with scalar operands and broadcasting): multiply by a mask where numpy would boolean-index, or -- better -- pass it to [`linalg:where`](../reference/functions/linalg-where.md), which *selects* between two operands on a non-zero mask (numpy's `np.where`). Selecting rather than multiplying is what lets a `-infinity` mask reach `linalg:softmax` as a weight of exactly zero: multiplying an infinity by zero would give a `NaN`. [`linalg:zeros-like`](../reference/functions/linalg-zeros-like.md) allocates a zero array of the same shape and width. ```lisp (linalg:take-rows #2A((10 11) (20 21) (30 31)) #(2 0)) ; => #d((30.0 31.0) (10.0 11.0)) (linalg:row #2A((10 11) (20 21) (30 31)) 2) ; => #d(30.0 31.0) (linalg:gather #2A((10 11 12) (20 21 22)) #(2 0)) ; => #d(12.0 20.0) (linalg:one-hot #(1 0) 3) ; => #d((0.0 1.0 0.0) (1.0 0.0 0.0)) (linalg:greater #(1 5 3) 2) ; => #d(0.0 1.0 1.0) (linalg:where (linalg:greater #(1 5 3) 2) #(1 5 3) 0) ; => #d(0.0 5.0 3.0) ``` ## Random numbers The `np.random` analog is seeded and cross-backend deterministic: [`linalg:seed`](../reference/functions/linalg-seed.md) resets a Wichmann-Hill generator whose draws are exact integer and IEEE double arithmetic, so a seeded sequence of [`linalg:rand`](../reference/functions/linalg-rand.md), [`linalg:randn`](../reference/functions/linalg-randn.md), [`linalg:uniform`](../reference/functions/linalg-uniform.md), [`linalg:choice`](../reference/functions/linalg-choice.md) and [`linalg:permutation`](../reference/functions/linalg-permutation.md) is bit-identical on the interpreter, the JVM and both WASM targets -- weight initialization and mini-batch sampling reproduce exactly everywhere. `randn` uses the Irwin-Hall sum of twelve uniforms rather than Box-Muller (whose `log`/`cos` would diverge on WASM), so its tails clip at six standard deviations; fine for initialization, but not a distribution-exact `np.random.randn`. ```lisp (linalg:seed 42) ; => 42 (linalg:choice 60000 4) ; => #d(26833.0 11120.0 29256.0 22347.0) (linalg:permutation 5) ; => #d(0.0 4.0 2.0 3.0 1.0) ``` ## Discrete calculus [`linalg:diff`](../reference/functions/linalg-diff.md) and [`linalg:gradient`](../reference/functions/linalg-gradient.md) are numpy's discrete-calculus pair (`np.diff` / `np.gradient`). `diff` takes the `:n`-th discrete difference (default 1) along `:axis` (default the last axis): each step shortens that axis by one, so a matrix differences within each row by default and down each column with `:axis 0`. `gradient` estimates the derivative of a vector of samples with second-order central differences (first-order one-sided at the two ends), so the result keeps the input's length; the optional second argument is either a uniform sample spacing (a number, default 1) or a coordinate vector of the same length for non-uniformly spaced samples. Both preserve the input's width like every other linalg transform. The arithmetic is floating point as usual, but sample values that differentiate exactly -- polynomials read at integer coordinates, like every example below -- print identically on every backend. ```lisp (linalg:diff #(1 2 4 7 0)) ; => #d(1.0 2.0 3.0 -7.0) (linalg:diff #(1 2 4 7 0) :n 2) ; => #d(1.0 1.0 -10.0) (linalg:diff #2A((1 3 6) (0 5 6))) ; => #d((2.0 3.0) (5.0 1.0)) (linalg:gradient #(0 1 4 9 16)) ; => #d(1.0 2.0 4.0 6.0 7.0) (linalg:gradient #(0 1 4 9 16) 2) ; => #d(0.5 1.0 2.0 3.0 3.5) (linalg:gradient #(0 1 9) #(0 1 3)) ; => #d(1.0 2.0 4.0) ``` The gradient of `#(0 1 4 9 16)` -- the parabola `y = x^2` sampled at `x = 0..4` -- recovers the true derivative `2x` exactly at the interior points (central differences are exact for quadratics; the two ends are first-order estimates), and the coordinate-vector form stays exact even for the unevenly spaced samples on the last line. [`examples/ml/numerical-calculus.lisp`](https://github.com/making/rontolisp/blob/develop/examples/ml/numerical-calculus.lisp) works these ideas through a projectile-motion walkthrough. ## Single-float precision linalg computes in `double-float` by default, but it is **width-polymorphic**: it accepts and preserves packed **single-float** (`#f`) arrays, which use half the memory and twice the SIMD lane count. Every constructor takes an `:element-type` keyword (the default is `'double-float`; pass `:element-type 'single-float` for a `#f` result), and every transform -- `add`/`sub`/`mul`/`div`/`emap`, `transpose`/`reshape`, `dot`/`matmul`/`outer`, `inv`/`solve` -- preserves its input's width. A single-float value therefore stays single-float all the way through: a functional weight update `(linalg:sub W grad)` keeps `W`'s width rather than silently widening it back to double (which, on the JVM [`--simd`](simd-acceleration.md) path, would force a mixed-width error on the following `vec:matvec`). Reach for single-float when you want the speed and memory of `f32` and can accept its lower precision, and keep the double-float default for precision-critical work such as `det`/`inv`/`solve`. ```lisp (linalg:zeros 3 :element-type 'single-float) ; => #f(0.0 0.0 0.0) (linalg:from-list '((1 2) (3 4)) :element-type 'single-float) ; => #f((1.0 2.0) (3.0 4.0)) (linalg:add (linalg:from-list '(1 2 3) :element-type 'single-float) 10) ; => #f(11.0 12.0 13.0) (array-element-type (linalg:transpose (linalg:eye 2 :element-type 'single-float))) ; => SINGLE-FLOAT ``` ## SIMD acceleration `linalg` needs no flag to be correct anywhere, but the [`--simd` flag](simd-acceleration.md) accelerates it: thirty-two functions -- `add`, `sub`, `mul`, `div`, `sum`, `norm`, `amax`, `amin`, `argmax`, `argmin`, `trace`, `transpose`, `reshape`, `dot`, `outer`, the unary ufuncs `exp`, `log`, `tanh`, `sin`, `cos`, `tan`, `asin`, `acos`, `atan`, `sinh`, `cosh`, `sqrt`, `abs`, `negative`, `sign`, and the comparison selects `maximum`, `minimum` -- are routed to native vector kernels (`jdk.incubator.vector` on the interpreter and the JVM, WebAssembly `v128` on wasm-GC), and `mean`, `matmul`, `flatten`, `solve`, `square`, `reciprocal`, `clip` and `relu` are accelerated with them because they are written in terms of them. Nothing changes in what a program accepts or rejects: an input a kernel cannot handle (a general boxed array, mixed widths, a plain number) simply runs the portable `linalg.lisp` definition instead, with the same result and the same error messages. The only observable difference is the [single-float precision rule](simd-acceleration.md#accelerating-linalg), which covers the reductions and the matrix product; element-wise results stay bit-identical. There is no reason to switch packages for speed: under `--simd`, `linalg` and `vec` land on the same kernels. See [Choosing between vec and linalg](simd-acceleration.md#choosing-between-vec-and-linalg) -- the short version is: write against `linalg` by default, and reach for `vec` only for its `-into` destination-passing loops, a `--no-gc` target (where `linalg` does not compile), or its fail-fast width strictness. ## First-class functions linalg functions are ordinary `defun`s, so `#'linalg:norm` and friends work as first-class values wherever a function is expected: ```lisp (mapcar #'linalg:norm (list #(3 4) #(6 8))) ; => (5.0 10.0) ``` Because arrays compare by identity (`eq`) only, results are compared with [`linalg:array-equal`](../reference/functions/linalg-array-equal.md), which checks shape and numeric equality (`1` and `1.0` compare equal). ## Packages `linalg` is a [package](../reference/packages.md) of its own and does not use `cl`: inside `(in-package linalg)` the standard functions are not visible under their bare names and would need `cl:` qualification (`cl:print`, `cl:mapcar`, ...). Most programs should therefore stay in the default `cl-user` package and call the qualified `linalg:` names, as every example on this page does. --- # FILE: references/guides/math-backends.md # Math Function Backends Arithmetic and comparison operators work on both integers and doubles. When any operand is a double, the result is promoted to double (e.g., `(+ 1 1.5)` returns `2.5`). `+`, `-`, `*`, `/` accept two or more arguments. `mod` supports doubles in the interpreter and JVM compiler but not in the WASM compiler. ## Math function backend support The math built-ins differ in how widely they are supported, because the WASM backend only has native instructions for a few operations: - **`sqrt`, `isqrt`, `gcd`, `lcm`, `signum`, `expt`** are supported on all three backends (interpreter, JVM, WASM) and through the compiled `eval`. `sqrt` uses the native `f64.sqrt` instruction. - **`exp`** is supported on all three backends. The interpreter and JVM use `Math.exp`; WASM has no native transcendental instruction, so it emits a software approximation in f64 (argument reduction by repeated squaring plus a Taylor polynomial), accurate to roughly 1e-6 relative error. The WASM result is therefore close to but not bit-identical to `Math.exp` (`(exp 0)` is exactly `1.0`). This is enough to run, e.g., a sigmoid `(/ 1.0 (+ 1.0 (exp (- 0 x))))` compiled to WASM. - **`log` and `tanh`** are supported on all three backends the same way as `exp`: the interpreter and JVM use `Math.log`/`Math.tanh`, and WASM emits software approximations -- `log` extracts the binary exponent and evaluates a short polynomial series on the normalized mantissa (`ln(x) = e*ln(2) + ln(m)`), and `tanh` is derived from the software `exp` as `(e^(2x)-1)/(e^(2x)+1)` with the argument clamped so large inputs saturate to exactly `±1.0`. Both match the interpreter/JVM values closely but not bit-identically (`(log 1)` and `(tanh 0)` are exactly `0.0`). The IEEE edges of `log` match `Math.log` (`log(0.0)` is `-Infinity`, a negative argument is `NaN`); WASM's `(tanh -0.0)` is `0.0` where the interpreter and JVM keep `-0.0`, the same class of edge as the WASM `signum`. - **`sin`, `cos` and `tan`** are supported on all three backends the same way: the interpreter and JVM use `Math.sin`/`Math.cos`/`Math.tan`, and WASM emits a software approximation -- Cody-Waite argument reduction (`k = nearest(x * 2/pi)`, `r = x - k*pi/2` via a two-part split of `pi/2`, the quadrant `k mod 4` selecting the sign/swap) plus Taylor polynomials for `sin(r)`/`cos(r)` on the reduced argument; `tan` is their ratio. Accuracy is ~1e-11 relative for `|x|` up to about `1e6`, after which the reduction's absolute error grows with the argument (documented like `exp`'s low-digit divergence; beyond `|x| > 2^30` a crude fold keeps the result finite but the value progressively loses significance, where the JVM's argument reduction stays exact). The zero and quadrant anchors are exact (`(sin 0)` is `0.0`, `(cos 0)` is `1.0`, `(sin (/ pi 2))` is `1.0`, `(cos pi)` is `-1.0`), and `NaN`/`±Infinity` arguments give `NaN` everywhere; WASM's `(sin -0.0)` and `(tan -0.0)` are `0.0` where the interpreter and JVM keep `-0.0`, the same class of edge as the WASM `signum` and `tanh`. - **`asin`, `acos` and `atan`** are supported on all three backends: the interpreter and JVM use `Math.asin`/`Math.acos`/`Math.atan`, and WASM emits a software approximation -- `atan` folds the argument by its odd symmetry, by the reciprocal identity `atan(x) = pi/2 - atan(1/x)` for `|x| > 1` (which also maps `±Infinity` to `±pi/2` for free), and by two half-angle steps, then evaluates a short Taylor series (~1e-15 relative error over the whole range); `asin(x) = atan(x / sqrt((1-x)(1+x)))` and `acos(x) = 2*atan(sqrt((1-x)/(1+x)))` derive from it. `(atan 0)`, `(asin 0)` and `(acos 1)` are exactly `0.0`, `(asin ±1)` is exactly `±pi/2` and `(acos -1)` exactly `pi`; an `asin`/`acos` argument outside `[-1, 1]` is `NaN` everywhere. - **`sinh` and `cosh`** are supported on all three backends: the interpreter and JVM use `Math.sinh`/`Math.cosh`, and WASM derives both from the software `exp` -- `e = exp(|x|)`, then `(e - 1/e)/2` with the sign restored, or `(e + 1/e)/2`. For tiny arguments the subtraction would cancel, so `sinh` switches to its odd Taylor series below `|x| = 0.25`. Accuracy follows the software `exp`: ~1e-7 relative for `|x|` up to ~20, degrading as the argument grows (like `exp`'s own divergence), and overflowing to `Infinity` slightly later than the JVM's 710.5. `(sinh 0)` is exactly `0.0` and `(cosh 0)` exactly `1.0`; `(sinh ±Infinity)` is `±Infinity` and `(cosh ±Infinity)` is `Infinity` everywhere. With these, **every transcendental built-in now works on all three backends**. - **`expt`** keeps an exact rational result for an integer or ratio base raised to an integer exponent (with big-integer promotion on every backend -- the WASM `expt`, like all WASM integer arithmetic, stays exact at any magnitude); a negative exponent yields the reciprocal (`(expt 2 -1)` is `1/2`). A float base with an integer exponent multiplies as a float (`(expt 2.0 3)` is `8.0`), and a float or ratio exponent gives a float: the interpreter and JVM use `Math.pow`, and WASM computes a fractional power as `exp(y * log(x))` over its software `exp`/`log` -- so `(expt 10000.0 0.75)` is `1000.0` up to the low-order digits of that approximation -- with the `Math.pow` edges (`x^0.0` is `1.0`, `0^y` is `0.0` for a positive and `Infinity` for a negative `y`, a negative base to a fractional power is `NaN`). An integer-valued float exponent (`(expt 2 3.0)`) takes the exact multiplication path and is `8.0` exactly. The dispatch happens at run time on every backend, so an exponent that arrives through a variable or a function call behaves exactly like a literal one. - **`gcd`, `lcm`, `signum`** are exact at any magnitude on every backend; **`isqrt`** still operates on the i31 integer range in the WASM backend. - **`random`** is supported on all three backends. It returns a non-negative random number below the (positive) limit, of the same type as the limit (an integer limit yields an integer, a float limit a float). The integer and float paths are chosen from the literal shape of the argument, so use a float literal (`(random 1.0)`) when a float result is wanted. The interpreter and JVM draw from `Math.random()`; the WASM backend draws real entropy from the WASI `random_get` host function (the real `wasi_snapshot_preview1.random_get` in Preview 1 mode, `wasi:random@0.3.0` in `--component` mode), so the sequence differs each run. `random` is not available inside the compiled `eval`. - **`logand`, `logior`, `logxor`, `lognot`, `ash`** are supported on all three backends and compute on exact integers of any magnitude (`ash` shifts left for a non-negative count, right otherwise). --- # FILE: references/guides/missing-features.md # Unsupported Common Lisp Features rontolisp is a deliberately small subset of Common Lisp that runs identically on three backends (interpreter, JVM, WASM). To keep the language compilable to plain bytecode without a runtime metaobject protocol, many features of full Common Lisp are intentionally left out. This page lists **only what is missing or partial**. For what *is* available, see the [Language Reference](../reference/special-forms.md), or list it at runtime with `rontolisp:list-special-forms`, `rontolisp:list-macros`, and `rontolisp:list-functions`. | Feature | Status | | --- | --- | | restarts | available; no debugger integration (`break`, `*debugger-hook*`) and no condition-restart association | | `define-symbol-macro` | not available (the lexical `symbol-macrolet` is) | | `&environment` | accepted in a `defmacro` lambda list but always bound to `nil` (there is no macro-expansion environment object). `&whole` works, in `defmacro` and `destructuring-bind` alike | | `loop` (extended) | partial (see below) | | CLOS | partial (static subset + a definition-time MOP subset) | | `defstruct` `:include` | single inheritance only; slot-overrides `(:include parent (slot default) ...)` work | | `declare` / `declaim` / `proclaim` / `the` | never change a result; on WASM an array `type` declaration directs the element-accessor emission (smaller, faster modules), everywhere else parsed no-ops | | `typep` / `subtypep` / `coerce` / `concatenate` | literal (quoted) type specifiers only; `coerce` targets `'list` / `'vector` / `'string` (or a float type), `concatenate` builds those same three sequence families | | `make-package` / `rename-package` / `delete-package` / `unintern` / `shadow` (runtime) | not available; `export` / `unexport` / `import` / `use-package` ARE, as read/compile-time directives like `in-package`; `defpackage` `:shadow` / `:shadowing-import-from` are errors | | `eval-when` | treated as `progn` (no phase distinction) | | `#:name` | reads as a plain symbol, without gensym-style freshness | | `*modules*` | not available (`require`/`provide` are) | | complex numbers | not available | | `catch` / `throw` / `unwind-protect` / conditions under `--no-gc` | compile error (available on every other backend) | ## Multiple values [`values`](../reference/functions/values.md) and its consumers are available, including the values of user functions. The remaining deviations from Common Lisp: - a producer that calls `values` in a **non-tail** position and then returns normally may leave stale extra values behind, so keep `values` in result position (a consumer clears the values it received, so only a `values` call that nothing consumes can leave leftovers); - `funcall #'values` (the first-class value) yields the primary value only in compiled programs; - `multiple-value-call` with a built-in `#'name` keeps the wrapper's fixed arity — pass a user function or `lambda` for other argument counts; - other built-ins with secondary values in CL (`read-from-string`, `subtypep`, ...) remain single-value — [`find-symbol`](../reference/functions/find-symbol.md) and [`intern`](../reference/functions/intern.md) do answer the accessibility status, and [`macroexpand-1`](../reference/functions/macroexpand-1.md) / [`macroexpand`](../reference/functions/macroexpand.md) do answer `expanded-p`. ## Non-local exit [`catch`](../reference/special-forms/catch.md) / [`throw`](../reference/special-forms/throw.md), [`block`](../reference/macros/block.md) / [`return-from`](../reference/macros/return-from.md) and [`tagbody`](../reference/special-forms/tagbody.md) / [`go`](../reference/special-forms/go.md) are available, with two gaps on the **compiled** backends (the interpreter is unaffected): - a `return-from` that would cross an `flet`/`labels` local function is not yet supported (one crossing a `lambda` is, as a non-local exit); - `go` must target a tag of a `tagbody` that lexically encloses it; the interpreter additionally supports dynamic `go` across function-call boundaries, i.e. a tag established by the *caller*. A tag reached from inside a nested `lambda` -- the shape a [`handler-bind`](../reference/macros/handler-bind.md) handler that resumes the protected loop with a `go` produces, and what quri's `:lenient` percent-decoding does -- is lowered like a cross-`lambda` `return-from`: a non-local exit that re-enters the `tagbody` at the tag and carries on. A cross-`lambda` `return-from` or `go`, `catch`/`throw`, `unwind-protect`, and condition catching all compile in exception-handling mode, so the emitted wasm-GC modules need `wasmtime -W exceptions=y` (37+); under `--no-gc` `catch`/`throw`, `unwind-protect` and the condition forms are a compile error. ## Restarts The condition system is complete through the restart layer: [`handler-bind`](../reference/macros/handler-bind.md) handlers run at the signal point before unwinding, [`restart-case`](../reference/macros/restart-case.md) / [`restart-bind`](../reference/macros/restart-bind.md) / [`with-simple-restart`](../reference/macros/with-simple-restart.md) establish restarts, and [`find-restart`](../reference/functions/find-restart.md) / [`invoke-restart`](../reference/functions/invoke-restart.md) / [`compute-restarts`](../reference/functions/compute-restarts.md) / [`muffle-warning`](../reference/functions/muffle-warning.md) / [`abort`](../reference/functions/abort.md) / [`continue`](../reference/functions/continue.md) drive them; [`cerror`](../reference/macros/cerror.md) is continuable. What is missing is the **interactive debugger**: `break` and `*debugger-hook*` do not exist, a restart's `:report` is stored but never rendered and its `:interactive` function never runs, and restarts are not associated with conditions (the optional condition argument of `find-restart`/`compute-restarts` is ignored). [`check-type`](../reference/macros/check-type.md) / [`assert`](../reference/macros/assert.md) / [`ccase`](../reference/macros/ccase.md) still signal without offering a `store-value` restart. Under `--no-gc` the restart forms degrade to the primary form (that backend has no condition objects at all); on the wasm-GC backends only **signaled** conditions are catchable — a runtime trap still aborts. ## The `loop` macro A bounded subset of the extended [`loop`](../reference/macros/loop.md) is available -- that page lists the supported clauses, which include destructuring patterns, parallel `and`, the anaphoric `it`, `loop-finish` and `thereis`/`always`/`never`. What is out of scope: `named` (and the `return-from` it would name); a destructuring pattern does not recognize lambda-list keywords (`&optional` and friends bind as ordinary variables instead of signalling); `being` drives hash tables, but its package form (`being the external-symbols of ...`) parses and iterates the EMPTY sequence, because there is no runtime intern table. ## Structures and objects [`defstruct`](../reference/special-forms/defstruct.md) supports `:include` inheritance in its single-inheritance form only. Slot-overrides work: `(:include parent (slot new-default) ...)` re-defaults an inherited slot in the child's layout while it keeps its inherited index, so the parent's accessors still read it. An instance prints in the standard `#S(...)` syntax, and a `#S(...)` literal reads back into an instance -- in source and through the runtime `read` / `read-from-string` on every backend (a compiled program's reader has frontend parity; only `#.`, `#+`/`#-` and `#n=`/`#n#` signal there). A structure that carries a `(:print-object fn)` / `(:print-function fn)` option prints through that function instead; both options are supported. CLOS is a **static subset** ([`defclass`](../reference/special-forms/defclass.md), [`defgeneric`](../reference/special-forms/defgeneric.md) / [`defmethod`](../reference/special-forms/defmethod.md) dispatching on the first argument, [`make-instance`](../reference/macros/make-instance.md) and [`slot-value`](../reference/macros/slot-value.md) with literal quoted names). A slot written with no `:initform` starts UNBOUND, as in CL: [`slot-boundp`](../reference/macros/slot-boundp.md) reports it, [`slot-makunbound`](../reference/macros/slot-makunbound.md) restores it, and a read signals `unbound-slot`. [`change-class`](../reference/macros/change-class.md) changes an instance's class in place (the target may be a runtime symbol or a class metaobject), and `reinitialize-instance` / `shared-initialize` are callable with no user method — the system defaults fill the supplied initargs, as in CL. A **definition-time MOP subset** is in: [`find-class`](../reference/functions/find-class.md) and [`class-of`](../reference/functions/class-of.md) answer real `standard-class` metaobjects, [`allocate-instance`](../reference/functions/allocate-instance.md) works, and a `(:metaclass M)` class option runs the class-definition protocol at definition time (see [`defclass`](../reference/special-forms/defclass.md)) — this is what loads postmodern's DAO layer verbatim. Multiple inheritance works (class precedence list, slot merge across superclasses). Out of scope: runtime class construction (`ensure-class` from computed data, a non-top-level `defclass`, `add-method`, `compute-applicable-methods`, class redefinition, `update-instance-for-different-class`) — the class and method sets of a compiled program are fixed at compile time. ## User-defined packages [`defpackage`](../reference/special-forms/defpackage.md) is a literal, top-level, read/compile-time directive supporting `:use`, `:export`, `:nicknames` and `:import-from` (`:documentation`/`:size` are accepted and ignored). `:shadow` and `:shadowing-import-from` are errors (there is no symbol shadowing). `use-package`, [`export`](../reference/functions/export.md), `unexport` and [`import`](../reference/functions/import.md) exist as the same kind of read/compile-time directive `in-package` is: a literal top-level call takes effect for the forms that follow it, on every backend, and a runtime-computed call works on the interpreter only. Creating or renaming a package at run time does not: `make-package`, `rename-package` and `delete-package` are not available. `unintern` (and the runtime `shadow` / `shadowing-import`) cannot exist here at all — a symbol IS its name, so there is no intern table to remove it from. The queries are real: [`find-package`](../reference/functions/find-package.md), [`package-name`](../reference/functions/package-name.md), [`list-all-packages`](../reference/functions/list-all-packages.md), [`package-use-list`](../reference/functions/package-use-list.md), [`package-used-by-list`](../reference/functions/package-used-by-list.md) and [`package-shadowing-symbols`](../reference/functions/package-shadowing-symbols.md) (always `nil`), with the compiled backends answering from a table baked in at compile time — so a package a compiled program creates later is invisible there. When several used packages export the same name, the first package in `:use` order wins instead of signaling a conflict. ## Dynamic (special) variables Dynamic binding through `let`/`let*` and [`progv`](../reference/special-forms/progv.md) is supported, with one limitation on the **compiled** backends (the interpreter is unaffected): while normal exit and a `return`/`return-from` that unwinds *across* a special `let` boundary both restore the binding, an error caught by a handler outside the `let` (a `go` across it, and on the WASM backends a `return` that also crosses an `unwind-protect`/`handler-case`) does not. `progv` restores on every exit an `unwind-protect` covers, including those cases. ## Numeric tower rontolisp supports integers (including arbitrary-precision bignums), ratios (`1/3`), and double floats, but **not complex numbers**. A negative square root yields a float `NaN` rather than a complex result: ```console > (sqrt -1) NaN ; full Common Lisp would return #C(0.0 1.0) ``` ## Other omissions - lambda lists: an extended `defmacro` lambda list (`&whole`, `&optional`, `&key`, `&aux`, nested destructuring patterns) routes through `destructuring-bind`, which is deliberately lenient -- a missing argument is `nil` and a surplus one is ignored rather than signalling; and a function is limited to 10 physical parameters on the funcall/apply path. - user macros are unknown to the runtime `eval` of compiled programs, and a `lambda` built at runtime by that `eval` does not parse lambda-list keywords (see [Compiled eval Limitations](eval-limitations.md)). - the PRETTY PRINTER produces the text a wide enough line holds, but never changes the LAYOUT: no rontolisp stream carries a column, so a logical block never wraps, every conditional line break (`pprint-newline` with `:linear` / `:fill` / `:miser`, the format directives `~_` / `~:_` / `~@_` / `~i`) is a no-op, and `*print-right-margin*` / `*print-miser-width*` / `*print-lines*` are accepted and ignored. Only `(pprint-newline :mandatory)` and `~:@_` break a line. Every other `*print-*` variable exists and holds the value the printer really behaves as -- binding one to a non-default value is what has no effect, except for `*print-escape*` / `*print-readably*` / `*print-pretty*` and `*print-case*`, which are honored. `*print-case*` converts the case of the symbols the printer spells but leaves a symbol nested in a structure, a CLOS instance, a hash table or an array of rank other than one at its stored spelling ([Reader Case](reader-case.md)). The ordinary printing operators do not consult `*print-pprint-dispatch*`: an entry fires where the program calls the entry function itself. - `#.` read-time eval is skipped with a warning inside `.asd` files. - built-in macro names (`cond`, `case`, `when`, `setf`, `push`, ...) cannot be redefined; list them with `(rontolisp:list-macros)`. This list is not exhaustive; rontolisp implements a focused core rather than the full standard. --- # FILE: references/guides/mito.md # O/R Mapping (mito, sxql) [Mito](https://github.com/fukamachi/mito) — an O/R mapper for Common Lisp — loads verbatim via `(ql:quickload "mito")`, and that pulls the whole system: `mito-core` (the DAO layer), `mito-migration` (schema diffs and migration files) and `lack-middleware-mito`. Its query clauses are written in [SxQL](https://github.com/fukamachi/sxql), which also loads and works on its own. **PostgreSQL is the only database.** `dbd-mysql` and `dbd-sqlite3` need FFI and are absent; the PostgreSQL driver rides the [cl-postgres stack](asdf-systems.md), so mito reaches every backend that can open a TCP socket — the interpreter, a JVM class and a WASM component — and not WASM Preview 1. ## Defining a table and connecting `dbd-postgres` must be quickloaded **explicitly**, next to `mito`: `dbi:connect` resolves its driver over the already-loaded systems, and a compiled program cannot load a system at run time. `deftable` is a `defclass` over mito's `dao-table-class` metaclass. The metaclass protocol runs at DEFINITION time, so a `deftable` must be a top-level form with literal options. `mito:table-definition` renders the DDL mito would create — it asks the driver for its type, so it needs a live connection: ```console $ cat blog.lisp (ql:quickload '("mito" "dbd-postgres")) (mito:deftable article () ((title :col-type (:varchar 64)) (body :col-type (or :text :null)))) (mito:connect-toplevel :postgres :database-name "blog" :username "postgres" :password "secret" :host "127.0.0.1" :port 5432) (dolist (statement (mito:table-definition 'article)) (format t "~a~%" (sxql:yield statement))) $ rontolisp blog.lisp CREATE TABLE article ( id BIGSERIAL NOT NULL PRIMARY KEY, title VARCHAR(64) NOT NULL, body TEXT, created_at TIMESTAMPTZ, updated_at TIMESTAMPTZ ) ``` The `id` primary key and the `created_at` / `updated_at` pair are mito's defaults (`:auto-pk`, `record-timestamps-mixin`); `(:auto-pk :uuid)` switches the key to a generated v4 UUID stored as `VARCHAR(36)`, and `(:table-name "...")` overrides the derived table name. `mito:disconnect-toplevel` closes the connection. Call it: the `trivial-garbage` dependency resolves to a no-op finalizer shim, so nothing reclaims a connection for you. ## CRUD ```console $ cat crud.lisp (ql:quickload '("mito" "dbd-postgres")) ;; ... the deftable and connect-toplevel from above ... (mito:ensure-table-exists 'article) (let ((a (mito:create-dao 'article :title "Hello" :body "First post"))) (format t "~a ~a~%" (mito:object-id a) (slot-value a 'title))) (mito:insert-dao (make-instance 'article :title "Hello again" :body nil)) (let ((found (mito:find-dao 'article :title "Hello"))) (setf (slot-value found 'body) "Edited") (mito:save-dao found)) (format t "~a~%" (mapcar (lambda (a) (slot-value a 'title)) (mito:select-dao 'article (sxql:where (:like :title "Hello%")) (sxql:order-by :id)))) (mito:delete-dao (mito:find-dao 'article :title "Hello again")) (format t "~a~%" (length (mito:select-dao 'article))) $ rontolisp crud.lisp 1 Hello (Hello Hello again) 1 ``` Slots are read with `slot-value`: mito injects its `:conc-name` readers from a metaclass hook that rontolisp records as metaobject data only, so the `article-title`-style accessors are **not** defined (see "Current limits"). `mito:retrieve-by-sql` and `mito:execute-sql` are there for the raw-SQL cases. ## Schema migration There are two ways in, and they differ in which backends can run them. **Diff against the live schema** — all three backends. `mito.migration:migration-expressions` compares the current class definition with the table as it exists and answers the statements that would close the gap; `migrate-table` runs them in a transaction: ```console $ cat migrate-table.lisp (ql:quickload '("mito" "dbd-postgres")) ;; ... the same connect-toplevel, and `article` redefined one column wider ... (mito:deftable article () ((title :col-type (:varchar 64)) (body :col-type (or :text :null)) (tag :col-type (or (:varchar 16) :null)))) (dolist (statement (mito.migration:migration-expressions 'article)) (format t "~a~%" (sxql:yield statement))) (mito.migration:migrate-table 'article) (format t "~a~%" (mito.migration:migration-expressions 'article)) $ rontolisp migrate-table.lisp ALTER TABLE article ADD COLUMN tag character varying(16) NIL ``` **Migration files** — interpreter and JVM only (see "Current limits"). `generate-migrations` writes `db/schema.sql` plus a timestamped `.up.sql` / `.down.sql` pair, `migration-status` reports what is applied, and `migrate` applies the pending files (parsing them with esrap): ```console $ cat db.lisp (ql:quickload '("mito" "dbd-postgres")) ;; ... the deftable and connect-toplevel from above ... (mito:generate-migrations #P"db/") (mito:migration-status #P"db/") (mito:migrate #P"db/") $ rontolisp db.lisp CREATE TABLE "article" ( "id" BIGSERIAL NOT NULL PRIMARY KEY, "title" VARCHAR(64) NOT NULL, "body" TEXT, "created_at" TIMESTAMPTZ, "updated_at" TIMESTAMPTZ ); Successfully generated: db/migrations/20260804003900.up.sql Status Migration ID -------------------------- down 20260804003900 Applying 'db/schema.sql'... -> CREATE TABLE "article" ( "id" BIGSERIAL NOT NULL PRIMARY KEY, "title" VARCHAR(64) NOT NULL, "body" TEXT, "created_at" TIMESTAMPTZ, "updated_at" TIMESTAMPTZ ); -> CREATE TABLE IF NOT EXISTS "schema_migrations" ( "version" BIGINT PRIMARY KEY, "applied_at" TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP, "dirty" BOOLEAN NOT NULL DEFAULT false ); Successfully updated to the version 20260804003900. ``` ## SQL generation with SxQL SxQL is what `select-dao`'s clauses are, and it stands on its own — `(ql:quickload "sxql")` opens no sockets and renders identical SQL on **all four** backends. `sxql:yield` answers the SQL string and the bind-value list as two values: ```console $ cat query.lisp (ql:quickload "sxql") (multiple-value-bind (sql binds) (sxql:yield (sxql:select :* (sxql:from :article) (sxql:where (:and (:like :title "%lisp%") (:in :status '("published" "draft")))) (sxql:order-by (:desc :id)) (sxql:limit 10))) (format t "~a~%~a~%" sql binds)) (format t "~a~%" (sxql:yield (sxql:insert-into :article (sxql:set= :title "Hello" :body "First post")))) (format t "~a~%" (sxql:yield (sxql:update :article (sxql:set= :title "Hi") (sxql:where (:= :id 1))))) (format t "~a~%" (sxql:yield (sxql:delete-from :article (sxql:where (:= :id 1))))) $ rontolisp query.lisp SELECT * FROM article WHERE ((title LIKE ?) AND (status IN (?, ?))) ORDER BY id DESC LIMIT 10 (%lisp% published draft) INSERT INTO article (title, body) VALUES (?, ?) UPDATE article SET title = ? WHERE (id = ?) DELETE FROM article WHERE (id = ?) ``` `create-table`, `drop-table`, `alter-table` with `add-column`, `left-join ... :on`, `limit` / `offset` and `order-by` with `:desc` / `nulls` are all there. Binding `sxql:*use-placeholder*` to `nil` renders the values inline instead of as `?` placeholders. Like every macro-heavy library, hot query construction belongs on a compiled backend — the interpreter re-expands macros on every evaluation. ## Backends - **Interpreter** — everything above. - **JVM class** — `rontolisp blog.lisp -o Blog.class && java Blog`. The compiled class is self-contained. - **WASM component** (`--component`) — needs both wasmtime features and both socket permissions: ```bash rontolisp blog.lisp -o blog.wasm --component wasmtime run -W gc=y -W exceptions=y -S tcp=y -S inherit-network=y blog.wasm ``` The `:host` must be an **IPv4 literal** — hostname lookup is unwired on WASM. - **WASM Preview 1** has no host socket API by design. A mito program compiles there (socket call sites become **call-time errors**, so spliced dead code builds), and the first socket call fails loudly at run time with a message naming the backends that work. ## Current limits - **PostgreSQL only.** `dbd-mysql` / `dbd-sqlite3` need FFI. mito's own `mysql` / `sqlite3` source files still load; they are simply never selected. - **`:conc-name` accessors are not generated.** mito adds its readers and writers from a metaclass hook, and accessor methods here come from the original `defclass` form, which carried none. Use `slot-value`. - **SQL function operators are interpreter-only**: `(:count ...)`, `(:sum ...)`, `(:max ...)` and anything else SxQL resolves as a function call — and therefore `mito:count-dao` — signal on the JVM and WASM backends. Count with `(length (mito:select-dao ...))` there. - **Writing migration files is interpreter + JVM.** `generate-migrations` and the file-deleting branch of `migrate` need directory creation and file removal, which the WASM backends do not import; they signal at the call. The diff-and-apply path (`migration-expressions`, `migrate-table`, `migration-status`, and `migrate` over files that already exist) runs everywhere. - **A bare `:references` needs a `:col-type`.** `(other-id :references (other id))` alone signals — an upstream defect, reproduced identically by SBCL on the same sources. Write `(other-id :col-type :bigint :references (other id))`. - **Adding a NOT NULL column that has an `:initform`** makes `migrate-table` emit `DEFAULT ?` with an empty bind list, and PostgreSQL answers `there is no parameter $1` — again upstream, again identical on SBCL. Wrap the call: `(let ((sxql:*use-placeholder* nil)) (mito.migration:migrate-table 'article))`. See also: [Systems (asdf)](asdf-systems.md) for the whole library catalogue and the cl-dbi / cl-postgres layers underneath, [TCP Sockets](tcp-sockets.md) for the per-backend socket rules, and [Clack Web Applications](clack.md) for the web side `lack-middleware-mito` plugs into. --- # FILE: references/guides/neural-networks.md # Neural Networks (torch) The `torch` package is a PyTorch-style layer over [linalg](linear-algebra.md): a **tensor** that remembers how it was computed, and a `torch:backward` that walks that history in reverse to fill in gradients. Everything a hand-written backpropagation pass used to do -- tracking which arrays fed which, deriving each operation's adjoint, summing gradients over broadcast axes -- happens automatically, one operation at a time. The package is implemented once in Lisp source and behaves identically on every backend. Every operation computes through the `linalg` kernels, so a torch program is accelerated under [`--simd`](simd-acceleration.md) for free, and the numerical results are the linalg results. How much the flag buys depends on which kernels a model leans on: batched (rank 3 or more) matmul is not in the accelerated set yet, so a transformer -- whose attention layers are almost entirely that call -- gains far less from it than a plain multilayer perceptron does. See [Accelerating linalg](simd-acceleration.md#accelerating-linalg). ## Tensors `torch:tensor` builds a leaf tensor from a number, a list, an array or a linalg array. `:requires-grad t` marks a parameter -- a tensor whose gradient the backward pass should fill in. A tensor prints as `#` (` :REQUIRES-GRAD T` appended for a parameter) -- the same text on every backend, since the printer shows only the data. Read values back with `torch:data` (the array), `torch:item` (the number inside a one-element tensor) and `torch:shape`: ```lisp (defparameter *w* (torch:tensor '(1.0 2.0) :requires-grad t)) (torch:data *w*) ; => #d(1.0 2.0) (torch:shape *w*) ; => (2) (torch:requires-grad-p *w*) ; => T (torch:item (torch:tensor 2.5)) ; => 2.5 ``` Operations accept tensors, numbers, raw arrays and lists interchangeably; non-tensors become constants that no gradient flows to: ```lisp (torch:data (torch:add *w* 10)) ; => #d(11.0 12.0) (torch:data (torch:matmul #2A((1.0 2.0) (3.0 4.0)) *w*)) ; => #d(5.0 11.0) ``` ## Recording and the backward pass An operation whose operand participates in autograd records the operation on a tape. `torch:backward` on a scalar (one-element) tensor seeds its gradient with `1.0`, visits the recorded operations in reverse topological order, and accumulates each input's gradient -- so a tensor used twice (a residual connection, a reused embedding row) collects the **sum** of both paths. Read the result with `torch:grad`: ```lisp (defparameter *loss* (torch:sum (torch:mul *w* *w*))) (torch:item *loss*) ; => 5.0 (torch:backward *loss*) (torch:grad *w*) ; => #d(2.0 4.0) ``` Gradients accumulate across backward calls (`+=`), which is what a mini-batch loop wants; `torch:zero-grad` clears a tensor's slot between steps: ```lisp (torch:backward (torch:sum (torch:mul *w* 3.0))) (torch:grad *w*) ; => #d(5.0 7.0) (torch:grad (torch:zero-grad *w*)) ; => NIL ``` ## Broadcasting and gradients Elementwise operations broadcast like numpy, and the backward pass reduces each gradient back to its operand's shape by summing over the broadcast axes. A `(d)` bias added to an `(n d)` activation therefore gets a `(d)` gradient -- the sum over the batch axis: ```lisp (defparameter *b* (torch:tensor '(0.5 0.5) :requires-grad t)) (defparameter *y* (torch:add (torch:tensor '((1.0 2.0) (3.0 4.0))) *b*)) (torch:backward (torch:sum *y*)) (torch:grad *b*) ; => #d(2.0 2.0) ``` ## Staying off the tape `torch:no-grad` runs its body with recording disabled -- the values are computed, nothing is remembered. This is how a training loop's parameter update (and inference in general) stays off the tape. `torch:detach` is the per-tensor spelling: a leaf sharing the same data, cut off from its history: ```lisp (torch:no-grad (torch:requires-grad-p (torch:mul *w* 2.0))) ; => NIL (torch:requires-grad-p (torch:detach (torch:mul *w* 2.0))) ; => NIL (torch:requires-grad-p (torch:mul *w* 2.0)) ; => T ``` ## A training loop: fit y = 2x Gradient descent needs nothing beyond what is above: a forward pass building the loss, `torch:backward`, and an update inside `torch:no-grad`. Fitting `y = 2x` by minimizing the mean squared error (the values are chosen so every quantity is an exact dyadic rational -- the printed result is identical on every backend): ```lisp (defparameter *wf* (torch:tensor '(0.0) :requires-grad t)) (defparameter *x* (torch:tensor '(1.0 2.0))) (defparameter *t* (torch:tensor '(2.0 4.0))) (dotimes (i 10) (let* ((diff (torch:sub (torch:mul *x* *wf*) *t*)) (loss (torch:mean (torch:mul diff diff)))) (torch:backward loss) (torch:no-grad (setq *wf* (torch:tensor (linalg:sub (torch:data *wf*) (linalg:mul 0.125 (torch:grad *wf*))) :requires-grad t))))) (torch:data *wf*) ; => #d(1.999890012666583) ``` ## Modules A **module** owns parameters, composes, and has a forward pass. `torch:module` builds one from a kind keyword, a plist of **fields** and a forward function; `torch:forward` runs it. The fields plist is the parameter registration -- `torch:parameters` walks it -- so a layer's forward reads its parameters back with `torch:field` rather than from a closed-over variable, and a parameter that exists cannot be missing from the walk: ```lisp (defun scale-layer (n) (torch:module :scale (list :gain (torch:parameter (linalg:ones (list n)))) (lambda (self x) (torch:mul x (torch:field self :gain))))) (defparameter *scale* (scale-layer 2)) (torch:data (torch:forward *scale* (torch:tensor '(3.0 4.0)))) ; => #d(3.0 4.0) (length (torch:parameters *scale*)) ; => 1 ``` The walk descends into submodules **and into lists of them**, and deduplicates by identity -- a weight shared by two layers is one parameter, and a list of N blocks needs no `ModuleList` type. A tensor field without `requires-grad` is a buffer and is skipped: ```lisp (defparameter *stack* (torch:module :stack (list :blocks (list *scale* (scale-layer 2)) :buffer (torch:tensor '(9.0 9.0))) (lambda (self x) x))) (length (torch:parameters *stack*)) ; => 2 ``` `torch:train` and `torch:eval` switch the training flag through the same walk, and `torch:zero-grad` accepts a module -- it clears every parameter's gradient. ## The built-in layers `torch:linear`, `torch:embedding`, `torch:layer-norm`, `torch:dropout` and `torch:sequential` are ordinary callers of `torch:module`. Their parameters are initialized exactly as PyTorch's are, from the seeded `linalg:seed` generator, so a seeded run reproduces on every backend. Replacing a parameter with `torch:set-field` pins a layer to given weights: ```lisp (defparameter *lin* (torch:linear 3 2)) (torch:set-field *lin* :weight (torch:parameter '((1.0 0.0) (0.0 1.0) (1.0 1.0)))) (torch:set-field *lin* :bias (torch:parameter '(0.5 -0.5))) (torch:data (torch:forward *lin* (torch:tensor '((1.0 2.0 3.0))))) ; => #d((4.5 4.5)) ``` `torch:sequential` threads its argument through each element, and an element may be a module **or a plain function** -- which is why there is no activation-module type, and why a reshape can sit in a chain: ```lisp (defparameter *net* (torch:sequential (torch:linear 4 8) (function torch:relu) (torch:linear 8 2))) (torch:shape (torch:forward *net* (torch:tensor (linalg:zeros '(3 4))))) ; => (3 2) (length (torch:parameters *net*)) ; => 4 ``` The activations are [`torch:relu`](../reference/functions/torch-relu.md), [`torch:tanh`](../reference/functions/torch-tanh.md) and [`torch:gelu`](../reference/functions/torch-gelu.md), each an ordinary function of a tensor. `torch:gelu` is the exact `x * (1 + erf(x / sqrt(2))) / 2` by default (`nn.GELU`'s own default), over the differentiable [`torch:erf`](../reference/functions/torch-erf.md); `:approximate :tanh` selects the GPT/BERT formulation instead. [`torch:fields`](../reference/functions/torch-fields.md) answers a module's whole fields plist, which is what makes the tree WALKABLE from outside: `nn.Module.apply` and `nn.Module.named_parameters` have no counterpart here because a walk is written over that plist plus [`torch:module-kind`](../reference/functions/torch-module-kind.md) -- what a layer IS, rather than a substring of a dotted parameter name. ## Losses `torch:mse-loss` and `torch:cross-entropy-loss` are plain functions returning a scalar tensor. Cross entropy takes raw **logits** (never softmax outputs -- it is computed from `-log-softmax`, the numerically stable form) and flattens all but the last axis, so `(batch seq vocab)` works directly. Its target is either integer class indices, with `:ignore-index` dropping padding positions from both the sum and the mean's denominator, or a full probability distribution of the logits' own shape -- PyTorch's soft-label form, `-sum(target * log-softmax(logits))`: ```lisp (torch:item (torch:mse-loss (torch:tensor '(1.0 2.0)) '(0.0 0.0))) ; => 2.5 (torch:item (torch:cross-entropy-loss (torch:tensor '((0.0 0.0))) #(0))) ; => 0.6931471805599453 (torch:item (torch:cross-entropy-loss (torch:tensor '((0.0 0.0))) (torch:tensor '((0.5 0.5))))) ; => 0.6931471805599453 ``` A LIST target is always class indices, so the probability spelling needs a tensor or an array; a one-hot distribution and the matching index give the same loss. ## Optimizers An **optimizer** owns the update rule and its state. `torch:sgd`, `torch:adam` and `torch:adamw` take a model (or a plain list of parameters), keep their hyper-parameters and buffers in a fields plist exactly as a module does, and apply the rule to every parameter when `torch:step` runs: ```lisp (defparameter *p* (torch:parameter '(1.0 2.0))) (defparameter *opt* (torch:sgd (list *p*) :lr 0.125 :momentum 0.5)) (torch:backward (torch:sum (torch:mul *p* *p*))) (torch:step *opt*) (torch:data *p*) ; => #d(0.75 1.5) (torch:step-count *opt*) ; => 1 ``` The update writes each parameter's data **in place** and uses no torch operation, so it records nothing on the tape and needs no `torch:no-grad` around it -- unlike a hand-written update built from `torch:set-data`. The state (a momentum buffer, Adam's two moments, the step count its bias correction divides by) lives in the optimizer and never on the parameter, so two optimizers over the same weights keep separate state. Hyper-parameters are ordinary fields, which is all a learning-rate schedule needs, and `torch:zero-grad` accepts an optimizer as well as a model: ```lisp (defparameter *adam* (torch:adam (torch:linear 2 2) :lr 0.001)) (torch:field *adam* :lr) ; => 0.001 (torch:field (torch:set-field *adam* :lr 0.0005) :lr) ; => 5.0e-4 ``` `torch:optimizer` is the constructor all three are built on -- a kind keyword, the parameters, a fields plist and a step function -- so a rule this package does not ship is a plain defun over the same record. [`torch:adam`](../reference/functions/torch-adam.md) and [`torch:adamw`](../reference/functions/torch-adamw.md) are the SAME rule with the decay in a different place: Adam's `:weight-decay` adds `wd * param` to the gradient, AdamW's shrinks the parameter directly so the adaptive denominator never rescales it. There is no parameter-GROUP object; two optimizers over disjoint parameter lists are what a group is here, which is how a transformer decays its weight matrices and leaves its biases, LayerNorm gains and embedding tables alone. [`torch:clip-grad-norm`](../reference/functions/torch-clip-grad-norm.md) goes between `torch:backward` and `torch:step`: it returns the total L2 norm of every gradient -- as measured, so the loop can log it -- and scales them in place when that exceeds the bound. ## Training a network Everything above composes into the loop PyTorch writes: forward, loss, `torch:zero-grad`, `torch:backward`, `torch:step`. ```lisp (linalg:seed 3) (defparameter *mlp* (torch:sequential (torch:linear 2 8) (function torch:relu) (torch:linear 8 1))) (defparameter *xs* (torch:tensor '((0.0 0.0) (0.0 1.0) (1.0 0.0) (1.0 1.0)))) (defparameter *ys* (torch:tensor '((0.0) (1.0) (1.0) (0.0)))) (defparameter *sgd* (torch:sgd *mlp* :lr 0.2)) (dotimes (i 200) (let ((loss (torch:mse-loss (torch:forward *mlp* *xs*) *ys*))) (torch:zero-grad *sgd*) (torch:backward loss) (torch:step *sgd*))) (< (torch:item (torch:mse-loss (torch:forward *mlp* *xs*) *ys*)) 1.0e-6) ; => T ``` Writing the update by hand instead needs `torch:no-grad` around it, because `torch:sub` on a parameter would record on the tape; `torch:set-data` then writes the new value into the very tensor the layer's fields point at, so the model keeps using it: ```lisp (defun sgd-step (model lr) (torch:no-grad (dolist (p (torch:parameters model)) (torch:set-data p (linalg:sub (torch:data p) (linalg:mul lr (torch:grad p))))))) (sgd-step *mlp* 0.2) (torch:training-p *mlp*) ; => T ``` ## Batching, padding and masks There is no `Dataset`/`DataLoader` hierarchy: a batch is an ordinary list. `torch:shuffled-batches` cuts a list of examples -- or an integer `n`, standing for the index list `0..n-1`, which is how several parallel arrays get batched at once -- into mini-batches ordered by the seeded generator, so an epoch reproduces on every backend: ```lisp (linalg:seed 1) (torch:shuffled-batches 7 3) ; => ((6 0 5) (1 4 3) (2)) (torch:shuffled-batches '(a b c d) 2 :shuffle nil) ; => ((A B) (C D)) ``` `torch:pad-sequence` turns a batch of variable-length index sequences into one padded rank-2 tensor, batch first, and the two mask constructors build the constants an attention layer fills with `-infinity`: ```lisp (defparameter *tokens* (torch:pad-sequence '((1 2 3) (4 5)))) (torch:data *tokens*) ; => #d((1.0 2.0 3.0) (4.0 5.0 0.0)) (torch:padding-mask *tokens*) ; => #d(((0.0 0.0 0.0)) ((0.0 0.0 1.0))) (torch:subsequent-mask 3) ; => #d(((0.0 1.0 1.0) (0.0 0.0 1.0) (0.0 0.0 0.0))) ``` Both masks are **raw linalg arrays** -- a mask carries no gradient -- shaped to broadcast over a `(batch query-length key-length)` score: `(batch 1 length)` for the padding mask, `(1 n n)` for the causal one. They combine with `linalg:add`, since `torch:masked-fill` treats every non-zero as masked. The padding value chosen here is also the `:ignore-index` to pass to `torch:cross-entropy-loss`, so the padded positions leave the loss alone. ## Masked attention scores `torch:masked-fill` writes a constant where a mask is non-zero; filling with `-infinity` before `torch:softmax` is the masked-attention idiom, and the masked weight comes out as exactly `0.0` -- including through the backward pass: ```lisp (defparameter *sc* (torch:tensor '((1.0 2.0) (3.0 3.0)) :requires-grad t)) (defparameter *att* (torch:softmax (torch:masked-fill *sc* #2A((0 1) (0 0)) (/ -1.0 0.0)) :axis 1)) (torch:data *att*) ; => #d((1.0 0.0) (0.5 0.5)) ``` ## A worked example [`examples/llm-from-scratch/`](https://github.com/making/rontolisp/blob/develop/examples/llm-from-scratch/README.md) is chapter 2 of 『作ってわかる大規模言語モデルの仕組み』 rewritten on this package: scaled dot-product and multi-head attention, sinusoidal positional encoding, the encoder/decoder Transformer with its padding and causal masks, and a Japanese-English training loop with greedy decoding. Its README carries the PyTorch-to-`torch` mapping table. ## Packages `torch` does not use `cl`, so programs stay in `cl-user` and call the qualified names; `#'torch:name` works (every function is a plain defun). The differentiable operations mirror their `linalg` counterparts -- the full list is in the [function reference](../reference/functions.md#torch-package-functions), and `torch:no-grad` on the [macros page](../reference/macros/torch-no-grad.md). --- # FILE: references/guides/read-load-limitations.md # Compiled read/load Limitations The reader compiled into a JVM class or WASM module parses the same syntax as the interpreter's reader: integers, big integers (JVM), floats, strings, symbols, `nil`/`t`, lists (dotted pairs included), `'quote`, `#'function`, ratios (`1/3`), radix integers (`#x10`/`#o17`/`#b101`), character literals (`#\a`, `#\Space`, ...), vectors (`#(1 2)`), rank-n arrays (`#2A((1 2) (3 4))`), bit vectors (`#*101`, read as a general vector like the frontend), packed float arrays (`#f(...)`/`#d(...)`), structure literals (`#S(NAME :SLOT value ...)`), pathname literals (`#P"dir/file"`, read as the pathname VALUE carrying that namestring), `;` line comments and nesting `#| ... |#` block comments. A token no `#` dispatch claims reads as a symbol (`#foo` is the symbol `#FOO`), exactly as it does in source. Three `#` forms are permanent exceptions, because they need an evaluator or the feature set at read time: `#.` read-time evaluation, `#+`/`#-` feature conditionals, and `#n=`/`#n#` reader labels. The compiled reader SIGNALS a catchable error on them instead of misreading (the interpreter's runtime `read` still resolves `#.` and `#+`/`#-` like the frontend, and reads labels). A `#S(...)` datum resolves against the structure types the compiled program defines; a slot the datum omits takes its `nil` initform, a simple constant initform (numbers, strings, characters, symbols, quoted lists, nested literals) is re-read from its baked printed form, and an initform outside that set signals rather than silently substituting a wrong value. In every backend `read` parses one S-expression from a line of stdin: blank and comment-only lines are skipped (it keeps reading until a line contains a datum), EOF returns `nil`, and a form must fit on a single line. The WASM reader has a hand-written parser and is narrower in its NUMBERS and its error MESSAGES: - **Integers parse exactly at any magnitude.** Integer and radix tokens promote through the same boxed-integer tiers the frontend uses (a value past the 31-bit fixnum range becomes a boxed integer, past the signed 64-bit range a limb-based big integer); ratio components stay 31-bit. - **Floats have no exponent.** A decimal token (optional leading `-` or `+`, digits, one `.`, e.g. `1.0`, `-2.5`, `.5`, `5.`) parses to an `f64`-backed float. There is no exponent (`1e3`) support, and a token with two dots or any non-digit (e.g. `1.2.3`, `foo.bar`) stays a symbol. - **Error messages are static.** A reader error signals (catchably under `handler-case`), but the message is a fixed text without the offending name interpolated -- the JVM and the interpreter carry the frontend's exact messages. - **Symbol interning is runtime-backed.** Symbols that appear in the compiled program resolve to the same offset the compiled `eval` uses; symbols seen only at runtime (e.g. a lambda parameter inside a loaded file) are interned in a runtime table so repeated occurrences stay consistent. - **`load` requires a preopened directory.** It opens the file via WASI `path_open`, resolving the path against the preopened directories: a relative path against the first one (fd 3), an absolute path against the preopened directory whose name is its longest prefix. Either way, run with `--dir` -- and with a `--dir` that COVERS an absolute path, since `--dir .` preopens a directory named `.`, which covers none. Backquote templates and a source's own feature announcement (see [Data Types](../reference/data-types.md#comments-feature-conditionals-and-features)) are frontend read-time constructs the runtime reader of compiled output does not resolve: a file read at runtime via `read`/`read-from-string`/a computed `load` must not use them. (The `*features*` VARIABLE is an ordinary special and works everywhere; what the runtime reader does not do is let a push in the text it is reading affect its own `#+`.) (`#| ... |#` block comments ARE skipped; `#+`/`#-` signal, as above.) `require`/`provide` are compile-time directives on the compiled backends, so they are **not** understood by the runtime `load` of compiled output: a file read at runtime via a computed or nested `load` must not contain them (only a literal, top-level `require`/`provide` works, consumed at compile time) — the same limitation as a runtime-loaded file's package directives (see [Packages](../reference/packages.md)). `read-from-string` reuses the same runtime reader, so on the compiled backends it parses the same syntax as `read`, and `(read-from-string (prin1-to-string x))` round-trips for every printable-readable value kind on all backends. `parse-integer` is independent of the reader and works on all three backends; its `:start`/`:end` keywords are interpreter-only, and on the compiled backends the keyword names must be literal. Both are also usable as first-class values (`#'parse-integer`, `#'read-from-string`) on all three backends — passed via the single-argument wrapper, so the keyword/optional arguments are not available through the function value. --- # FILE: references/guides/reader-case.md # Reader Case (Upcasing) rontolisp's reader upcases symbols the way standard Common Lisp's does: every unescaped character of a symbol token is converted to upper case while reading (the `:upcase` readtable case), so `foo`, `Foo` and `FOO` in source all name the same symbol `FOO`. Escaped characters keep their case: `|mixed Case|` and `\(` read verbatim. The upcased name is the canonical one -- there is no fold back to a lowercase spelling. Standard names, `t`/`nil`, lambda-list markers and built-in package members are all upper case like everything else: - standard `cl` names (`defun` and `DEFUN` both read as `DEFUN`, `list` as `LIST`), including type-specifier and condition-type names (`HASH-TABLE`, `TYPE-ERROR`), - `T` / `NIL` / `PI` and the other read-time constants, - lambda-list markers (`&OPTIONAL`, `&KEY`, ...), - built-in package prefixes and their members (`rl:fetch` reads as `RL:FETCH`, `ql:quickload` as `QL:QUICKLOAD`), - keyword or `#:` designators (`(in-package :cl-user)` reads `:CL-USER`, `(:use #:cl)` reads `#:CL`). Your symbols, your packages and your data keywords upcase the same way, self-consistently, exactly like Common Lisp: ```lisp (defun greet (name) (format nil "Hello, ~a!" name)) (greet "world") ; => "Hello, world!" 'foo ; => FOO (symbol-name 'foo) ; => "FOO" (symbol-name 'car) ; => "CAR" (eq 'foo 'FOO) ; => T (cdr (assoc :note '((:note . "hi")))) ; => "hi" ``` An escaped name keeps its case and is therefore a *distinct* symbol from the upcased one, as in Common Lisp: `|car|` is not `CAR`. Built-in keyword parameters match case-insensitively (`:TEST` works where `:test` does), and `(intern "TIME")` names the standard `TIME`, so the `(intern (string-upcase ...))` name-synthesis idiom lines up with body references, the pattern behind macros like assoc-utils' `with-keys`: ```console $ cat keys.lisp (ql:quickload :assoc-utils) (print (assoc-utils:with-keys ("name") '(("name" . "eitaro")) name)) $ rontolisp keys.lisp "eitaro" ``` Libraries loaded with `load`, `asdf:load-system` or `ql:quickload` are read the same way, so their definitions and your references upcase consistently. A symbol system designator is downcased like ASDF's `coerce-name` (`(ql:quickload :ASSOC-UTILS)` finds the `assoc-utils` system). The runtime reader upcases too, so a datum read at run time behaves like the same datum written in source: `read` and `read-from-string` upcase your symbols identically on the interpreter, the JVM and both WASM backends. ```lisp (read-from-string "foo") ; => FOO (symbol-name (read-from-string "foo")) ; => "FOO" (eq (read-from-string "list") 'list) ; => T (eval (read-from-string "(reverse (list 1 2 3))")) ; => (3 2 1) ``` ## Printing back: `*print-case*` The printer writes the stored (upper-case) spelling, which is what `*print-case*`'s standard value `:upcase` says. Binding the variable converts the case of every symbol a printing operator spells -- `princ`, `prin1`, `print`, `princ-to-string`, `prin1-to-string`, `write-to-string`, `write` and the `~A` / `~S` format directives -- on the interpreter, the JVM and both WASM backends: ```lisp (let ((*print-case* :downcase)) (princ-to-string 'add-test)) ; => "add-test" (let ((*print-case* :capitalize)) (princ-to-string 'add-test)) ; => "Add-Test" (let ((*print-case* :downcase)) (format nil "~a ~s" 'foo :foo)) ; => "foo :foo" (let ((*print-case* :downcase)) (prin1-to-string '(foo "Str" nil))) ; => "(foo \"Str\" nil)" ``` Only symbols convert: a string keeps its own characters, a character prints as itself, and `nil` / `t` are symbols and do convert. `:capitalize` keeps each word's first character as it stands and downcases the rest of the word (a word is a run of alphanumerics); it never upcases a lower-case character, because the standard converts only the upper-case ones -- which is where the rule parts company with `string-capitalize`. Deviation: a symbol nested inside a structure, a CLOS instance, a hash table or an array of rank other than one keeps its stored spelling. The conversion walks lists and vectors and hands those containers to the ordinary renderer. ## Deviations from Common Lisp - `intern`, `make-symbol` and `find-symbol` take the name verbatim (there is no separate intern table; a symbol *is* its name). `(find-symbol "car")` is `NIL` because the standard symbol is named `"CAR"`, and `(make-symbol "X")` twice yields `eq` symbols. Reading is unaffected -- `car` in source still upcases to `CAR`. - A keyword or `#:` designator that spells a member in mixed case names that exact (mixed-case) symbol; write built-in members upper case (`#:CL`) or let the reader upcase a bare name. --- # FILE: references/guides/simd-acceleration.md # Vector Kernels and SIMD Acceleration (vec, linalg) The `vec` package provides portable packed-`f64` vector kernels: constructors, element access, element-wise arithmetic and reductions over the [packed float array type](../reference/data-types.md). It is the go-to package for tight numeric loops over vectors of doubles, and it carries an optional hardware-acceleration (SIMD) layer on every backend. The package names the portable abstraction; the `--simd` flag names how it is accelerated. `--simd` is not a `vec`-only flag. It accelerates the [`linalg` package](linear-algebra.md) too, over the very same arrays. This guide covers `vec` first, then the flag, then what it does for `linalg`. Like the JSON and `linalg` libraries, `vec` is implemented once in Lisp source (`vec.lisp`): the interpreter loads the definitions lazily on the first use of a `vec:` function, and the compile path splices them into the program when it references the package. This scalar definition is the implementation on the interpreter, the JVM compiler and the WASM (wasm-GC) backends, and the correctness oracle for the accelerated paths, so every function behaves identically everywhere. ## Choosing between vec and linalg `vec` and `linalg` are not two implementations of the same idea. They are two **contracts** over the same packed float arrays -- and under `--simd` they land on the same accelerated kernels, so the choice is never about speed. It is about how a function should behave at the edges: | | `linalg` | `vec` | |---|---|---| | accepted inputs | packed arrays, general boxed arrays such as `#(1 2 3)`, plain numbers | packed float arrays only | | mixed widths (`#d` with `#f`) | allowed -- both are widened, the first operand's width wins | hard error | | broadcasting | numpy rules -- a scalar on either side, and arrays of different shapes along their trailing axes | only the scalar of `vec:scale` | | shapes | rank-n arrays and matrices, descriptive shape errors | rank-1 vectors (plus `vec:matvec`'s rank-2 matrix) | | allocation control | every result is a fresh array | `-into` siblings write into a caller-supplied destination | | `--no-gc` | does not compile | fully supported (the only vector package there) | Rule of thumb: **write against `linalg` by default.** It is the broader, numpy-style API, it forgives mixed inputs, and with `--simd` it is accelerated by the same kernels. Reach for `vec` when one of its three exclusives is the point: an allocation-free hot loop (the `-into` kernels), a `--no-gc` target, or the fail-fast strictness that turns a width mistake into an immediate error instead of a silent widening. ## Data representation A vector is a rank-1 [packed float array](../reference/data-types.md): the `double-float`-typed, unboxed array that `#d(...)` and `(make-array n :element-type 'double-float)` produce. The built-in `aref` / `length` interoperate with it, and any packed vector built elsewhere can be handed to a `vec` function. Element-wise kernels return a fresh vector; reductions return a scalar `double`. The kernels are width-polymorphic: they also accept single-float vectors (`#f(...)` / `:element-type 'single-float`, which store elements as `f32` -- half the memory, twice the SIMD lanes). The element-wise kernels preserve the input width on every backend (a `#f` in gives a `#f` out), while the reductions always fold to a scalar `double`. ```lisp (vec:arange 5) ; => #d(0.0 1.0 2.0 3.0 4.0) (vec:add #d(1.0 2.0 3.0) #d(4.0 5.0 6.0)) ; => #d(5.0 7.0 9.0) (vec:dot #d(1.0 2.0 3.0) #d(4.0 5.0 6.0)) ; => 32.0 (vec:scale #d(1.0 2.0 3.0) 10) ; => #d(10.0 20.0 30.0) ``` ## The API Construction: `vec:zeros` / `vec:ones` build a filled vector of length *n*, `vec:arange` builds `[0.0, 1.0, ..., n-1]`, and `vec:from-list` / `vec:to-list` convert between a vector and a Lisp list (the list forms run on the interpreter, the JVM and wasm-GC only, not `--no-gc`). `vec:zeros` / `vec:ones` / `vec:arange` also take an `:element-type` keyword: pass `:element-type 'single-float` for a packed single-float (`#f`) vector (the default is double-float), mirroring the [linalg constructors](linear-algebra.md#single-float-precision) and honored on every backend including the JVM and WASM `--simd` v128 paths. Access: `vec:aref` reads an element (a `setf` place via `vec:aset`), and `vec:length` returns the element count. These are thin wrappers over the generic packed-array operators, so plain `aref` / `length` work too. Element-wise (a fresh vector): `vec:add`, `vec:sub`, `vec:mul` (Hadamard product), `vec:div` and `vec:scale` (multiply by a scalar). The first four also answer to their CL operator spellings -- `vec:+`, `vec:-`, `vec:*` and `vec:/` -- which are exact aliases and compile to the same code, accelerated paths included. Unlike their n-ary [`linalg:`](linear-algebra.md) counterparts they are **strictly binary**: every `vec:` kernel is fixed-arity and allocation-explicit (the reason the `-into` family below exists), so an n-ary spelling that silently allocated one intermediate vector per extra operand would work against the point of the package. Write `(vec:+ (vec:+ a b) c)`, or better, an `-into` loop. Element-wise unary, under their numpy ufunc names (a fresh vector): `vec:exp`, `vec:log`, `vec:tanh`, `vec:sin`, `vec:cos`, `vec:tan`, `vec:asin`, `vec:acos`, `vec:atan`, `vec:sinh`, `vec:cosh`, `vec:sqrt`, `vec:abs`, `vec:square`, `vec:negative`, `vec:sign` and `vec:reciprocal` (`1 / x`). Each applies the backend's own scalar operation per element, so the transcendental members (`vec:exp` / `vec:log` / `vec:tanh` / `vec:sin` / `vec:cos` / `vec:tan` / `vec:asin` / `vec:acos` / `vec:atan` / `vec:sinh` / `vec:cosh`) on the WASM backends use their software approximations (whose low-order digits differ from the JVM's), and the `-0.0` edges of `vec:abs` / `vec:negative` / `vec:sign` / `vec:tanh` / `vec:sin` / `vec:tan` follow each backend's own scalar operation. On `--no-gc`, the transcendental members and `vec:sign` run the same software sequences as the other WASM backends, so all seventeen work everywhere. Comparison selects: `vec:maximum` / `vec:minimum` (the element-wise larger / smaller of two vectors), `vec:relu` (element-wise `max(x, 0.0)`) and `vec:clip` (element-wise `min(max(x, lo), hi)` with scalar bounds). All four are defined by the strict comparison select `(if (> x y) x y)` and its mirrors -- never an IEEE min/max primitive -- so the second operand (or the bound) wins whenever the comparison is false: a `-0.0` element against `0.0` takes the second, a `NaN` follows the same rule (`vec:relu` maps it to `0.0`, `vec:clip` to `lo`), and every backend agrees exactly, `--no-gc` included. Reductions (a scalar): `vec:sum`, `vec:dot`, `vec:mean` and `vec:norm` (the Euclidean norm, `sqrt` of the self-dot). ```lisp (vec:sum (vec:arange 5)) ; => 10.0 (vec:mean #d(2.0 4.0 6.0)) ; => 4.0 (vec:norm #d(3.0 4.0)) ; => 5.0 (vec:to-list (vec:mul #d(1.0 2.0 3.0) #d(4.0 5.0 6.0))) ; => (4.0 10.0 18.0) ``` Matrix times vector (a fresh vector): `vec:matvec` is GEMV -- a rank-2 packed matrix `W` (shape *d* x *n*) times a rank-1 vector `x` of length *n*, giving a length-*d* vector whose *i*-th element is the dot product of row *i* of `W` with `x` (no transpose). It is the workhorse of a neural network's forward pass -- every projection, feed-forward and classifier layer is a `vec:matvec` -- so it is the one kernel run once per matrix row rather than element-wise. The result follows the input width. On `--no-gc`, build `W` with `(make-array (list d n) :element-type ...)` plus `setf` of a two-subscript `aref` -- a rank-2 `#d((...))` literal is not supported there, and `x` must be the same width as `W` (the usual `vec` strictness). ```lisp (vec:matvec #d((1.0 2.0) (3.0 4.0)) #d(5.0 6.0)) ; => #d(17.0 39.0) ``` The [`ml/nn-vec.lisp` example](https://github.com/making/rontolisp/blob/develop/examples/ml/nn-vec.lisp) is a small XOR network whose single-float forward pass is built from `vec:matvec`. ## Memory: where vectors live, and what reclaims them A packed float array is an ordinary garbage-collected value on three of the four targets, and a block of WebAssembly linear memory on the fourth. Only the last one asks you to think about memory growth. | target | packed arrays live in | reclaimed automatically? | |---|---|---| | interpreter (no `-o`) | the JVM heap | yes, by the JVM's collector | | JVM (`-o prog.class`) | the JVM heap | yes, by the JVM's collector | | wasm-GC (`-o prog.wasm`) | the WebAssembly GC heap | yes, by the engine's collector | | `--no-gc` (`-o prog.wasm --no-gc`) | linear memory, bump-allocated | **no -- nothing is ever freed, so you must watch memory growth** | On the three garbage-collected targets, a loop that discards its intermediates keeps a flat footprint. Building a fresh 1024-element vector 200000 times on wasm-GC (`wasmtime run -W gc`) peaks at the same ~123 MB as doing it 50000 times, even though 1.5 GB passed through the allocator. `linalg` arrays are the same packed type and behave identically. `--no-gc` is different by design -- the name says it, there is no collector. `__ronto_alloc` is a bump allocator with no free, so **every kernel that returns a vector permanently consumes memory**. Reclamation happens only by discarding the whole arena at an export-call boundary, which a `--no-gc` module always has (its top level is nothing but `defun`s and `rontolisp:wasm-export` directives -- there is no `_start`): - an export whose return type is a non-memory scalar (`:int` / `:long` / `:float` / `:bool` / `:void`) resets the bump pointer automatically when it returns; - a resident host can bracket a call with the exported `__ronto_alloc_mark` / `__ronto_alloc_reset` pair; - **inside a single export call nothing is freed.** A loop of `(setq acc (vec:add acc d))` grows linear memory until `memory.grow` fails. That last point is what the destination-passing kernels below are for. Strings on `--no-gc` (`concatenate`, `subseq`, `princ-to-string`) bump-allocate the same way; `linalg` does not compile under `--no-gc` at all, so only `vec` is affected. wasm-GC has a linear memory too, but packed arrays never touch it: it holds interned symbol names and string-stream buffers, and runtime strings were moved onto the GC heap precisely so that it would stop growing. ## Destination-passing kernels (allocation-free loops) Every kernel above that returns a vector returns a **fresh** one, so a loop over them allocates one vector per iteration. Each has an `-into` sibling that writes into a caller-supplied destination and returns it, letting you hoist the allocation out of the loop. The destination comes first, mirroring Common Lisp's own `map-into`. | allocating | destination-passing | |---|---| | `(vec:add a b)` | `(vec:add-into out a b)` | | `(vec:sub a b)` | `(vec:sub-into out a b)` | | `(vec:mul a b)` | `(vec:mul-into out a b)` | | `(vec:div a b)` | `(vec:div-into out a b)` | | `(vec:scale v s)` | `(vec:scale-into out v s)` | | `(vec:matvec w x)` | `(vec:matvec-into out w x)` | | `(vec:exp v)` | `(vec:exp-into out v)` | | `(vec:log v)` | `(vec:log-into out v)` | | `(vec:tanh v)` | `(vec:tanh-into out v)` | | `(vec:sin v)` | `(vec:sin-into out v)` | | `(vec:cos v)` | `(vec:cos-into out v)` | | `(vec:tan v)` | `(vec:tan-into out v)` | | `(vec:asin v)` | `(vec:asin-into out v)` | | `(vec:acos v)` | `(vec:acos-into out v)` | | `(vec:atan v)` | `(vec:atan-into out v)` | | `(vec:sinh v)` | `(vec:sinh-into out v)` | | `(vec:cosh v)` | `(vec:cosh-into out v)` | | `(vec:sqrt v)` | `(vec:sqrt-into out v)` | | `(vec:abs v)` | `(vec:abs-into out v)` | | `(vec:square v)` | `(vec:square-into out v)` | | `(vec:negative v)` | `(vec:negative-into out v)` | | `(vec:sign v)` | `(vec:sign-into out v)` | | `(vec:reciprocal v)` | `(vec:reciprocal-into out v)` | | `(vec:maximum a b)` | `(vec:maximum-into out a b)` | | `(vec:minimum a b)` | `(vec:minimum-into out a b)` | | `(vec:relu v)` | `(vec:relu-into out v)` | | `(vec:clip v lo hi)` | `(vec:clip-into out v lo hi)` | The reductions (`vec:sum`, `vec:dot`, `vec:mean`, `vec:norm`) return a scalar and never allocated, so they have no sibling. ```lisp (let ((acc (vec:zeros 3)) (d #d(1.0 2.0 3.0))) (dotimes (i 3) (vec:add-into acc acc d)) acc) ; => #d(3.0 6.0 9.0) ``` In the element-wise kernels -- binary and unary alike -- the destination **may alias** an operand: element *i* of the result depends only on element *i* of the inputs, so `(vec:add-into acc acc d)` above and `(vec:exp-into v v)` are well-defined in-place updates. `vec:matvec-into` is the exception -- each output element folds over all of `x`, so writing into `x` would clobber a value a later row still has to read. Passing the same array as both `out` and `x` (or `w`) signals an error rather than corrupting it. All operands must share an element type, and `out` must be at least as long as the inputs (its length is not checked, exactly as `vec:add` does not check its operands'). This is what makes `--no-gc` usable for real numeric loops (see the memory table above): with `-into`, peak memory equals the vectors you actually keep alive. Measured on `--no-gc --simd`, accumulating a 65536-element vector 12000 times peaks at 13.7 MB with `vec:add-into`, against 4.31 GB -- and then a trap -- with `vec:add`. On the three garbage-collected targets `-into` changes nothing about correctness; there it is an allocation-rate optimization. On `--no-gc`, `vec:matvec-into`'s aliasing guard is a WebAssembly trap (an `unreachable` instruction) rather than a Lisp error -- the backend has no error channel -- and it matters most there: a decode loop of GEMVs would otherwise bump-allocate a fresh output vector per step with nothing ever freed. ## Hardware acceleration (optional) The scalar `vec.lisp` reference is correct on every backend. `--simd` is the single, backend-independent switch that additionally lowers the vectorizable kernels (`add` / `sub` / `mul` / `div` / `scale` / `dot` / `sum` / `matvec` and the four operator aliases, the unary ufuncs `exp` / `log` / `tanh` / `sin` / `cos` / `tan` / `asin` / `acos` / `atan` / `sinh` / `cosh` / `sqrt` / `abs` / `negative` / `sign` / `reciprocal`, the comparison selects `maximum` / `minimum` / `relu` / `clip`, and all their `-into` siblings, plus `mean` / `norm` / `square` transitively) to real CPU vector instructions or de-boxed loops. It is opt-in. The element-wise kernels stay byte-for-byte identical to the scalar reference; the reductions sum in a different order, and a single-float reduction also accumulates in single precision, so those can differ from it -- see the two paragraphs on precision below. The same flag accelerates a set of `linalg` functions, listed in the next section. Which memory model you compile for (`.class`, wasm-GC `.wasm`, or `--no-gc` `.wasm`) and whether you pass `--simd` are **orthogonal** axes: | target | without `--simd` | with `--simd` | |---|---|---| | interpreter (no `-o`) | scalar `vec.lisp` | `jdk.incubator.vector` (baked into the native binary; `java -jar` needs `--add-modules`) | | JVM (`-o prog.class`) | scalar `vec.lisp` | `jdk.incubator.vector` bridge | | wasm-GC (`-o prog.wasm`) | scalar `vec.lisp` | native v128 (`f64x2` / `f32x4`) | | `--no-gc` (`-o prog.wasm --no-gc`) | scalar linear-memory loops | native v128 (`f64x2` / `f32x4`) | - **Interpreter `--simd`**: `rontolisp prog.lisp --simd` runs the same kernels on `jdk.incubator.vector` instead of the scalar `vec.lisp` definitions -- no compilation step, and a large `vec:dot` gets several times faster. The native binary has the incubator module baked in and needs no runtime flag. On a plain `java -jar` the module is absent, so the flag falls back to the scalar reference and prints a note; re-run with `java --add-modules jdk.incubator.vector -jar rontolisp.jar prog.lisp --simd` to get the acceleration there. Without `--simd` the interpreter always runs the scalar reference -- it is the cross-backend oracle. The flag also works in the REPL: `rontolisp --simd` accelerates the `vec:` / `linalg:` kernels the same way. - **JVM `--simd`**: `rontolisp prog.lisp -o Prog.class --simd` routes the kernels to an embedded `jdk.incubator.vector` bridge (a `DoubleVector` for `#d`, a `FloatVector` for `#f`; `vec:matvec` runs that vectorized dot once per matrix row). Running such a class requires the incubator module on the JVM: `java --add-modules jdk.incubator.vector Prog`. Without `--simd` the class runs the scalar reference on any JVM. **Whether the bridge becomes CPU vector instructions is up to the JVM that runs the class.** The Vector API is a normal library that a JVM may or may not compile down to vector instructions, operation by operation; where it does not, it falls back to emulating each lane, which is far slower than the plain scalar loop `--simd` replaced. So `--simd` is not automatically a win on the JVM backend, and the same class can behave very differently on two JVMs. Measure on the JVM you deploy on, with your own data. There is another reason the JVM backend is hard to predict, and it has nothing to do with SIMD. A compiled Lisp numeric loop boxes every intermediate value -- one `Double` per array element read, per product, per running sum, plus a `Long` per loop counter -- so a scalar `vec:` kernel is bound by allocation and dispatch rather than by arithmetic. How much of that boxing a given JIT eliminates (through escape analysis and inlining) varies enormously between JVMs, so the very same scalar loop can be several times faster on one than on another. What `--simd` does here is sidestep the question: it replaces those kernels with primitive `double[]` / `float[]` loops that never box in the first place. - **wasm-GC `--simd` native `v128`**: `rontolisp prog.lisp -o prog.wasm --simd` lowers the `vec:` kernels to WebAssembly fixed-width SIMD (`f64x2.*`, or `f32x4.*` for single-float). A packed float array becomes an `(array (mut v128))` of lane groups -- still an ordinary GC object, still reclaimed by the engine's collector, so memory behaves exactly as it does on scalar wasm-GC. The whole `vec:` API (including `vec:matvec` and `vec:from-list` / `vec:to-list`) keeps working, and the results are unchanged. Composes with `--component` and every `--optimize` level. Run it with `wasmtime run -W gc` as usual -- wasmtime enables the SIMD proposal by default. - **`--no-gc --simd` native `v128`**: `rontolisp prog.lisp -o prog.wasm --no-gc --simd` lowers the same kernels over the packed linear-memory block. **Without `--simd`, `--no-gc` emits plain scalar loops** over the byte-identical block -- a v128-free MVP module that runs on a WebAssembly runtime lacking the SIMD proposal, trading away the vectorized speedup for that portability. `vec:matvec` / `vec:matvec-into` run over a rank-2 packed matrix block (`[rows][cols][data]`, built by a rank-2 `make-array`): the per-row dot is the `f64x2` / `f32x4` dot loop under `--simd` and the scalar loop without it. Only `vec:from-list` / `vec:to-list` (which need Lisp lists) remain unavailable on `--no-gc`; `vec:exp` / `vec:log` / `vec:tanh` / `vec:sin` / `vec:cos` / `vec:tan` / `vec:asin` / `vec:acos` / `vec:atan` / `vec:sinh` / `vec:cosh` / `vec:sign` have no vector instruction, so they run the same per-element loop in both modes, as does `vec:clip` (its bounds are full doubles, so each element is compared widened); `vec:maximum` / `vec:minimum` / `vec:relu` vectorize as comparison-mask selects under `--simd` and fall back to scalar compare-and-select loops without it. On wasm-GC the speedup is large because `--simd` replaces two things at once: the boxing-heavy scalar `vec.lisp` defun *and* the one-element-at-a-time loop. A `vec:dot` over an 8192-element vector, 20000 iterations, runs in ~10.1 s scalar and ~0.10 s with `--simd` under `wasmtime run -W gc`. Reading a lane group out of a GC array costs a bounds check that a `v128.load` from linear memory does not, and no engine hoists it out of the loop, so the same kernel loop is about 1.9x slower on wasm-GC `--simd` than on `--no-gc --simd`. That is the price of letting the collector own your vectors. If a numeric inner loop is your bottleneck and you can live without a garbage collector, compile it with `--no-gc --simd`. Because reductions sum in a different order under SIMD, a reduction over inexact inputs can differ from the left-to-right scalar reference in the last ULP; over the exact doubles typical of tests the results match exactly. The element-wise kernels are always bit-identical. Single-float reductions carry one more caveat. Under `--simd`, an `#f` reduction -- `vec:dot` / `vec:sum` / `vec:matvec` -- accumulates in single precision, in four lanes, on every backend, and widens only the final value. The scalar reference instead reads each element as a double and accumulates in double. So over data that a single-precision accumulator cannot hold, `--simd` can move an `#f` reduction by roughly the single-float epsilon rather than by the last ULP. Every `--simd` backend accumulates the same way, so they agree with one another, and the scalar reference remains the more accurate of the two. `#d` (`double-float`) reductions are unaffected. If a single-float reduction has to be as accurate as the scalar reference, use `#d` for it -- or leave `--simd` off for that computation. ## Accelerating linalg The [`linalg` package](linear-algebra.md) is written over the same packed float arrays, and `--simd` routes thirty-four of its functions to the same kernels: - **accelerated directly**: `add`, `sub`, `mul`, `div`, `sum`, `norm`, `amax`, `amin`, `argmax`, `argmin`, `trace`, `transpose`, `reshape`, `dot`, `outer`, the unary ufuncs `exp`, `log`, `tanh`, `sin`, `cos`, `tan`, `asin`, `acos`, `atan`, `sinh`, `cosh`, `sqrt`, `abs`, `negative`, `sign`, the comparison selects `maximum`, `minimum`, and the two internal convolution helpers behind the Deep Learning from Scratch examples (`linalg::%la-im2col` and `linalg::%la-col2im`, the im2col window unfolding and its adjoint -- index-arithmetic loops rather than lane kernels, bit-identical at both widths) - **accelerated with them**: `mean`, `matmul`, `flatten`, `solve`, `square`, `reciprocal`, `clip` and `relu`, each of which is written in terms of the functions above - **never accelerated**: `emap` (it applies an arbitrary function to each element), `det`, `inv`, `array-equal` and the constructors There is no separate flag and nothing to opt into per function: compile or run with `--simd` and the calls are routed, exactly as they are for `vec`. ```lisp (linalg:norm (linalg:sub #d(4.0 6.0) #d(1.0 2.0))) ; => 5.0 (linalg:emap #'sqrt #d(1.0 4.0 9.0)) ; => #d(1.0 2.0 3.0) ``` Where `vec` insists on packed arrays of one width, `linalg` accepts far more: general boxed arrays such as `#(1 2 3)`, two operands of different widths, plain numbers, arrays of different shapes (broadcast by the numpy rules), and shapes that fit no broadcast (which signal an error). A kernel handles the packed, same-width cases: equal shapes, a scalar operand on either side, two arrays of different-but-broadcastable shapes (the numpy broadcast itself), `transpose`'s axes form (`(linalg:transpose x '(0 3 1 2))`), and the `:axis` forms of `sum` / `amax` / `amin` / `argmax` / `argmin` -- an integer axis, negative axes included, with or without `:keepdims`. **For everything else the portable `linalg.lisp` definition runs instead**, over the very same argument values -- same result, same broadcast, same error message, each argument form still evaluated exactly once. So `--simd` never changes what a linalg program accepts or rejects; it only makes the common case faster. Two shapes fall outside the accelerated set today, and both are worth naming because neural-network code runs into them. **Batched matrix multiplication** -- `linalg:matmul` where either operand has rank 3 or more, which is `torch.bmm` and therefore every attention layer -- runs the portable definition: the flag does not speed it up, and on wasm-GC it makes it slightly *slower*, because under `--simd` a packed array is a block of lane groups whose individual elements cost more to reach. **`linalg:erf`** is an `emap` over a scalar series, and `emap` is never intercepted, so the exact [`torch:gelu`](../reference/functions/torch-gelu.md) gains nothing either (its `:approximate :tanh` form is `mul` / `add` / `tanh` and is accelerated). Both are gaps to be closed rather than deliberate exclusions -- unlike `emap` with a user callback, `det` / `inv`, and `array-equal`, which are excluded by design. The precision rules above carry over, with one exception in linalg's favor: - **Element-wise operations are bit-identical** to the portable definitions at both widths -- `add` / `sub` / `mul` / `div`, whether against another array, against a scalar or across a numpy broadcast, the unary ufuncs `exp` / `log` / `tanh` / `sin` / `cos` / `tan` / `asin` / `acos` / `atan` / `sinh` / `cosh` / `sqrt` / `abs` / `square` / `negative` / `sign` / `reciprocal`, and the comparison selects `maximum` / `minimum` / `clip` / `relu` (a select only copies input bits, so it cannot round) -- and so are `transpose` (the axes form included), `reshape`, `outer`, `trace`, `amax`, `amin`, `argmax` and `argmin`. - **Reductions over the whole array follow the `vec` rule**: `sum`, `mean`, `norm` and the vector and matrix-vector forms of `dot` sum in a different order, and over a single-float array they accumulate in single precision. - **Reductions along an axis are bit-identical.** `(linalg:sum a :axis 0)`, `(linalg:amax a :axis 1)` and the other axis forms fold exactly as the portable definitions do -- in double, in the same order, with the same tie rules -- so unlike the whole-array reductions they cannot differ. - **The full matrix product follows the `vec` rule too.** `(linalg:dot A B)` and `linalg:matmul` over two matrices accumulate in the width of their operands: over `#d` they stay bit-identical to the portable definition, and over `#f` they fold each output cell in single precision -- in the portable definition's own order, but rounding at every step, so the two can differ. This is the ordinary behavior of a single-precision matrix multiply; every mainstream library does the same. It is also what makes `#f` faster than `#d` here rather than twice as slow, because a single-precision accumulator is what lets the kernel run single-precision lanes at all. Which lanes ran cannot move the result -- the lanes go across the output row, not along the axis being summed -- so all three `--simd` backends agree with one another exactly. If a single-float matrix product has to match the portable definition, use `#d` for it, or leave `--simd` off for that computation. `linalg` does not compile under `--no-gc` at all, with or without `--simd`. The `--no-gc` row of the target table above therefore concerns `vec` only. ## Runnable examples The smallest one is [`examples/ml/simd-dot.lisp`](https://github.com/making/rontolisp/blob/develop/examples/ml/simd-dot.lisp): one `vec:dot` over 1024 doubles, four thousand times, and nothing else. Its vector holds `0.0 .. 1023.0`, so the answer is an exact integer that no amount of lane reordering can change -- run it with and without `--simd` and only the elapsed time moves (interpreter 2.59 s -> 2.3 ms; wasm-GC 273 ms -> 2.4 ms). [`examples/ml/simd-gemv.lisp`](https://github.com/making/rontolisp/blob/develop/examples/ml/simd-gemv.lisp) does nothing but the two kernels acceleration exists for -- `vec:matvec` and `vec:dot` -- a hundred times over: project a vector through a 256x256 single-float matrix, rescale it to unit root-mean-square, repeat. That pair is a transformer's projection plus its RMSNorm, and it is where an LLM inference engine spends nearly all of its time. Run it twice: ```bash rontolisp examples/ml/simd-gemv.lisp rontolisp examples/ml/simd-gemv.lisp --simd ``` It prints integer `argmax` indices rather than floats, so the output is identical with and without acceleration -- only the elapsed time changes. On an Apple M4: wasm-GC 467 ms -> 3.9 ms, the interpreter 4.6 s -> 15 ms. [`examples/ml/simd-gemv-nogc.lisp`](https://github.com/making/rontolisp/blob/develop/examples/ml/simd-gemv-nogc.lisp) is the same inner loop compiled with `--no-gc`: a pure-compute reactor module whose host invokes the exported `fingerprint` function and reads back the `argmax` integer. Build it with and without `--simd` and invoke both: ```bash rontolisp examples/ml/simd-gemv-nogc.lisp -o gemv.wasm --no-gc --simd wasmtime run --invoke fingerprint gemv.wasm 100 ``` Both builds print `85` -- the same dominant direction as every other backend -- and the `-into` kernels keep the never-freed `--no-gc` bump heap at exactly three blocks however many steps run. At 20000 steps the scalar module takes ~600 ms and the `--simd` one ~120 ms. The whole engine is [`examples/llama2/llama2.lisp`](https://github.com/making/rontolisp/blob/develop/examples/llama2/llama2.lisp): llama2.c's `run.c` ported to one file -- checkpoint loader, tokenizer, forward pass, sampler -- over the real TinyStories checkpoints, telling the same stories as the C program token for token. Its 15 million weights load through `read-sequence` over packed single-float arrays, and its decode is over a hundred `vec:matvec`s per token; on stories15M, `--simd` takes the JVM from 23 to 87 tokens/s and wasm-GC from 0.4 to 46 (`run.c -O2`: 65). See [its README](https://github.com/making/rontolisp/blob/develop/examples/llama2/README.md). A row must hold at least 128 elements before the interpreter and JVM kernels vectorize it; below that they run the scalar loop, because filling the vector registers would cost more than it saves. The two WASM backends have no such threshold. ## Packages `vec` does not use `cl`; every function is external, referenced as `vec:name`. Put `(in-package :vec)` (or `(defpackage ... (:use :vec))`) in effect to write the exported names unqualified. The related [`linalg` package](linear-algebra.md) offers a broader numpy-style API (shape manipulation, matrix products, exact linear algebra) over the same arrays. --- # FILE: references/guides/tcp-sockets.md # TCP Sockets The `rontolisp` package provides four functions for plain TCP networking, plus encrypted variants for both sides (`tls-connect` and `tls-listen`). They are **not part of Common Lisp**; reference them with the `rontolisp:` qualifier (see [Packages](../reference/packages.md)). A connected socket is a **bidirectional stream handle** in the same handle space as file streams, so the standard stream functions work on it directly: `read-line`, `write-line`, `write-string`, `write-char`, `read-char`, `read-byte`, `write-byte` and `close`. Unlike buffered file output, socket writes are sent immediately (`write-line` flushes per line), and `read-line` returns `nil` once the peer has closed the connection. A socket carries BYTES: `write-string` puts the string's UTF-8 bytes on the wire and `read-char` reads one character back out of them, so `read-byte` and `read-char` can be mixed on the same handle. At end of stream the reads follow their own Common Lisp defaults: `read-char` and `read-byte` signal `end-of-file` unless you pass the eof arguments — `(read-char sock nil :eof)` yields `:eof` — while `read-line` answers `nil`, as it does on a file. The printing functions (`print`, `princ`, `format`) do not take a socket; render with `(format nil ...)` and send the result with `write-line` or `write-string`. | Function | Purpose | |----------|---------| | [`rontolisp:tcp-connect`](../reference/functions/rontolisp-tcp-connect.md) | Open a client connection: `(rontolisp:tcp-connect host port)` | | [`rontolisp:tcp-listen`](../reference/functions/rontolisp-tcp-listen.md) | Bind a listening socket: `(rontolisp:tcp-listen port &optional host)` | | [`rontolisp:tcp-accept`](../reference/functions/rontolisp-tcp-accept.md) | Wait for a client connection: `(rontolisp:tcp-accept listener)` | | [`rontolisp:tcp-local-port`](../reference/functions/rontolisp-tcp-local-port.md) | Read the bound port back (useful after listening on port `0`) | | [`rontolisp:tls-connect`](../reference/functions/rontolisp-tls-connect.md) | Open an **encrypted** client connection: `(rontolisp:tls-connect host port)` | | [`rontolisp:tls-upgrade`](../reference/functions/rontolisp-tls-upgrade.md) | Wrap an **already-connected** stream handle in TLS as a client: `(rontolisp:tls-upgrade stream host)` | | [`rontolisp:tls-listen`](../reference/functions/rontolisp-tls-listen.md) | Bind an **encrypted** listening socket from a PKCS12 keystore: `(rontolisp:tls-listen keystore password port &optional host)` | | [`rontolisp:tls-listen-pem`](../reference/functions/rontolisp-tls-listen-pem.md) | Bind an **encrypted** listening socket from PEM files: `(rontolisp:tls-listen-pem cert-file key-file port &optional host)` | > **Backend support.** The interpreter and JVM-compiled classes use the JDK > socket classes and accept hostnames or IP literals. The WASM backend is > **component-only** (`--component`, over `wasi:sockets@0.3.0`): the tcp > functions compile in Preview 1 (core-module) mode but raise call-time > errors, hosts must be > IPv4 literals, and the component must run with `-W exceptions=y -S tcp=y > -S inherit-network=y` on top of the usual flags (a tcp component always > compiles in exception-handling mode). Combining the tcp functions with > [`rontolisp:http-handler`](http-handler.md) compiles into one component > and runs under `wasmtime serve` — add `-S cli=y` to the flags above > (without it the serve linker reports the `wasi:sockets@0.3.0` > `tcp-socket` resource as missing at instantiation). wasmCloud's > `wash dev` (2.5.2) hosts that component too and provides > `wasi:sockets` 0.3, with one difference: a loopback destination names a > per-workload virtual network, not the machine's real 127.0.0.1 — a > connect to a loopback address only reaches a listener inside the same > wasmCloud workload (such as a service component bound there), while > non-loopback addresses go out over the real network. In the **browser > playground** every tcp function signals an error (the browser sandbox has no > raw TCP), so the runnable example below only works outside the browser. See > the [tcp-connect](../reference/functions/rontolisp-tcp-connect.md) reference > page for the shared limitations (TCP only, no UDP). The TLS *client* > functions > ([`rontolisp:tls-connect`](../reference/functions/rontolisp-tls-connect.md) > and > [`rontolisp:tls-upgrade`](../reference/functions/rontolisp-tls-upgrade.md)) > run on the interpreter, the JVM and the WASM `--component` backend (add > `-S tls=y` to the flags above); the TLS *server* functions > ([`rontolisp:tls-listen`](../reference/functions/rontolisp-tls-listen.md) > and > [`rontolisp:tls-listen-pem`](../reference/functions/rontolisp-tls-listen-pem.md)) > are interpreter/JVM only — the `wasi:tls` proposal defines no server > interface, so they are a permanent compile error on every WASM target. The programs in this guide are complete and self-contained: copy each one into a file and run it with any backend. They use only the `rontolisp:tcp-*` primitives; the [usocket-compatible shim](#the-usocket-compatible-shim) at the end shows how the same programs look through the portability API that existing Common Lisp code expects. ## A first round trip The snippet below is self-contained: it listens on an ephemeral port, connects to itself over the loopback interface, and echoes one line back through the accepted handle: ```lisp (let* ((listener (rontolisp:tcp-listen 0 "127.0.0.1")) (port (rontolisp:tcp-local-port listener)) (sock (rontolisp:tcp-connect "127.0.0.1" port))) (write-line "ping" sock) (let* ((peer (rontolisp:tcp-accept listener)) (line (read-line peer))) (write-line line peer) (let ((reply (read-line sock))) (close peer) (close sock) (close listener) reply))) ; => "ping" ``` ## An echo server A real server binds a fixed port and serves connections in an accept loop. Save the following as `echo-server.lisp`. Each accepted handle is read line by line until `read-line` returns `nil` (the client closed), and every line is written straight back: ```console (let ((listener (rontolisp:tcp-listen 7777))) (if listener (progn (write-line "echo server listening on 127.0.0.1:7777") (do ((n 1 (+ n 1))) (nil) (let ((sock (rontolisp:tcp-accept listener))) (write-line (format nil "client ~a connected" n)) (do ((line (read-line sock) (read-line sock))) ((null line) (close sock) (write-line "client disconnected")) (write-line line sock))))) (write-line "tcp-listen failed (is port 7777 already in use?)"))) ``` The `(if listener ...)` check matters on the WASM component backend, where a failed bind returns `nil` instead of signaling an error (the interpreter and JVM signal). The server loops forever — stop it with `Ctrl-C`. ### Running it On the interpreter: ```bash rontolisp echo-server.lisp ``` Compiled to a JVM class (the class is named after the output file): ```bash rontolisp echo-server.lisp -o EchoServer.class java EchoServer ``` Compiled to a WASM component (wasmtime 46+; note the two `-S` flags that grant network access — without them the component still starts, but `tcp-listen` returns `nil`): ```bash rontolisp echo-server.lisp -o echo-server.wasm --component wasmtime run -W gc=y -W exceptions=y -S tcp=y -S inherit-network=y echo-server.wasm ``` Whichever backend serves, talk to it with any TCP client, for example `nc` (netcat): ```console $ nc 127.0.0.1 7777 hello hello world world ``` ## An echo client The matching client connects to the server, sends every line read from standard input, and prints each reply until stdin ends. Save it as `echo-client.lisp`: ```console (let ((sock (rontolisp:tcp-connect "127.0.0.1" 7777))) (if sock (do ((line (read-line) (read-line))) ((null line) (close sock)) (write-line line sock) (write-line (read-line sock))) (write-line "cannot connect to 127.0.0.1:7777 (is echo-server.lisp running?)"))) ``` Start `echo-server.lisp` first (any backend), then pipe input to the client — the server and the client can each run on a *different* backend: ```bash echo hello | rontolisp echo-client.lisp ``` ## An HTTP server Because a socket handle is a line stream and `read-line` strips one trailing carriage return, HTTP's CRLF-terminated request line and headers read as plain lines (the blank line ending the headers reads as `""`); response header lines get their carriage return back via `code-char 13` before `write-line` appends the newline. That is enough to answer `curl` and browsers. Save the following as `http-hello.lisp` — it serves a small HTML page showing the request line and a running request counter, one connection per request: ```console ;; Appends the carriage return of an HTTP CRLF line ending (write-line then ;; appends the newline). (defun crlf (s) (concatenate 'string s (format nil "~a" (code-char 13)))) ;; Consumes the request headers up to the blank line that ends them. (defun drain-headers (sock) (do ((line (read-line sock) (read-line sock))) ((or (null line) (string= line ""))))) (let ((listener (rontolisp:tcp-listen 8080))) (if listener (progn (write-line "http server listening on http://127.0.0.1:8080/") (do ((n 1 (+ n 1))) (nil) (let* ((sock (rontolisp:tcp-accept listener)) (request (read-line sock))) (if request (let ((body (format nil "

hello from rontolisp

request ~a: ~a

" n request))) (drain-headers sock) (write-line (crlf "HTTP/1.1 200 OK") sock) (write-line (crlf "Content-Type: text/html") sock) ;; + 1: write-line terminates the body with a newline (write-line (crlf (format nil "Content-Length: ~a" (+ (length body) 1))) sock) (write-line (crlf "Connection: close") sock) (write-line (crlf "") sock) (write-line body sock) (write-line (format nil "served request ~a: ~a" n request)))) (close sock)))) (write-line "tcp-listen failed (is port 8080 already in use?)"))) ``` Run it on any backend and open in a browser or with `curl http://127.0.0.1:8080/`. > For real HTTP work there is no need to hand-roll the protocol over a socket: > the *client* side is `rontolisp:fetch` (see the > [HTTP Requests guide](http-fetch.md)), and the *server* side > `rontolisp:http-handler` parses requests and adapts responses for you (see > the [Serving HTTP guide](http-handler.md)). The hand-rolled server above is > here to show the socket primitives, not as the recommended way to serve HTTP. ## A miniature Redis server A larger example: an in-memory key-value server that speaks enough of RESP2 (the Redis serialization protocol) that the real `redis-cli` connects and works, and — like real Redis — also accepts "inline commands" (a plain space-separated line), so `telnet 127.0.0.1 6379` or `nc 127.0.0.1 6379` work too. Both framings arrive as CRLF-terminated lines, which `read-line` reads as plain lines. The store is a hash table with string keys that survives across connections. It supports (case-insensitive) `PING`, `SET`, `GET`, `DEL`, `EXISTS`, `INCR`, `KEYS`, `DBSIZE` and `QUIT`. Save it as `kv-server.lisp`: ```console ;; --- small string helpers --------------------------------------------------- ;; Appends the carriage return of a RESP CRLF line ending (write-line then ;; appends the newline). (defun crlf (s) (concatenate 'string s (format nil "~a" (code-char 13)))) ;; "SET key value" -> ("SET" "key" "value") (defun split-words (s) (cond ((string= s "") nil) (t (let ((p (position #\space s))) (if p (cons (subseq s 0 p) (split-words (subseq s (+ p 1)))) (list s)))))) ;; ("hello" "world") -> "hello world" (defun join-words (ws) (cond ((null ws) "") ((null (cdr ws)) (car ws)) (t (concatenate 'string (car ws) " " (join-words (cdr ws)))))) ;; t when s is a non-empty run of decimal digits (with an optional leading -). (defun integer-string-p (s) (let* ((n (length s)) (start (if (and (> n 0) (char= (char s 0) #\-)) 1 0))) (and (> n start) (do ((i start (+ i 1))) ((or (>= i n) (not (digit-char-p (char s i)))) (>= i n)))))) ;; --- RESP replies ----------------------------------------------------------- (defun reply-simple (s sock) (write-line (crlf (concatenate 'string "+" s)) sock)) (defun reply-error (s sock) (write-line (crlf (concatenate 'string "-ERR " s)) sock)) (defun reply-int (n sock) (write-line (crlf (format nil ":~a" n)) sock)) (defun reply-bulk (s sock) (if s (progn (write-line (crlf (format nil "$~a" (length s))) sock) (write-line (crlf s) sock)) (write-line (crlf "$-1") sock))) (defun reply-array-header (n sock) (write-line (crlf (format nil "*~a" n)) sock)) ;; --- request framing -------------------------------------------------------- ;; Reads one RESP bulk-string element: the "$" header line, then the ;; payload line (the payload must not contain a newline). (defun read-bulk (sock) (let ((header (read-line sock))) (if header (read-line sock) nil))) (defun read-resp-array (count sock acc) (if (<= count 0) (reverse acc) (let ((arg (read-bulk sock))) (if arg (read-resp-array (- count 1) sock (cons arg acc)) nil)))) ;; Reads one command as a list of argument strings: a "*" line starts a ;; RESP2 array (what redis-cli sends); anything else is an inline command ;; (what telnet/nc users type). nil at connection close. (defun read-command (sock) (let ((line (read-line sock))) (cond ((null line) nil) ((string= line "") (read-command sock)) ((char= (char line 0) #\*) (let ((count (subseq line 1))) (if (integer-string-p count) (read-resp-array (parse-integer count) sock nil) (list "!bad-frame")))) (t (split-words line))))) ;; --- commands --------------------------------------------------------------- ;; Handles one command; returns nil after QUIT (closing the session). (defun handle-command (args store sock) (let ((cmd (string-upcase (car args))) (key (cadr args))) (cond ((string= cmd "PING") (if key (reply-bulk key sock) (reply-simple "PONG" sock)) t) ((string= cmd "SET") (if (and key (cddr args)) (progn (setf (gethash key store) (join-words (cddr args))) (reply-simple "OK" sock)) (reply-error "wrong number of arguments for 'set' command" sock)) t) ((string= cmd "GET") (if key (reply-bulk (gethash key store) sock) (reply-error "wrong number of arguments for 'get' command" sock)) t) ((string= cmd "DEL") (let ((removed 0)) (dolist (k (cdr args)) (when (gethash k store) (remhash k store) (incf removed))) (reply-int removed sock)) t) ((string= cmd "EXISTS") (reply-int (if (and key (gethash key store)) 1 0) sock) t) ((string= cmd "INCR") (let ((current (if key (or (gethash key store) "0") "0"))) (cond ((null key) (reply-error "wrong number of arguments for 'incr' command" sock)) ((integer-string-p current) (let ((n (+ (parse-integer current) 1))) (setf (gethash key store) (format nil "~a" n)) (reply-int n sock))) (t (reply-error "value is not an integer or out of range" sock)))) t) ((string= cmd "KEYS") (let ((pattern (or key "*")) (keys nil)) (maphash (lambda (k v) (if (or (string= pattern "*") (string= pattern k)) (push k keys))) store) (reply-array-header (length keys) sock) (dolist (k keys) (reply-bulk k sock))) t) ((string= cmd "DBSIZE") (reply-int (hash-table-count store) sock) t) ((string= cmd "COMMAND") ;; redis-cli asks COMMAND DOCS on connect; an empty array satisfies it. (reply-array-header 0 sock) t) ((string= cmd "QUIT") (reply-simple "OK" sock) nil) (t (reply-error (format nil "unknown command '~a'" (car args)) sock) t)))) ;; --- server loop ------------------------------------------------------------ (let ((store (make-hash-table)) (listener (rontolisp:tcp-listen 6379))) (if listener (progn (write-line "mini-redis listening on 127.0.0.1:6379 (try: redis-cli -p 6379 ping)") (do ((n 1 (+ n 1))) (nil) (let ((sock (rontolisp:tcp-accept listener))) (do ((args (read-command sock) (read-command sock))) ((or (null args) (not (handle-command args store sock))) (close sock)))))) (write-line "tcp-listen failed (is port 6379 already in use? a real redis, perhaps)"))) ``` Run it on any backend, then talk to it with the real `redis-cli`: ```bash redis-cli -p 6379 set greeting hello redis-cli -p 6379 get greeting redis-cli -p 6379 incr counter ``` ## TLS connections [`rontolisp:tls-connect`](../reference/functions/rontolisp-tls-connect.md) is the encrypted counterpart of `tcp-connect`: it performs a TLS handshake after connecting and returns the same kind of stream handle, so `read-line`, `write-line`, `read-byte`, `write-byte` and `close` work unchanged. The server certificate is validated against the JDK default trust store and the hostname is verified; point the `javax.net.ssl.trustStore` system properties at your own trust store to accept self-signed certificates, or pass `:insecure t` to skip verification entirely (development only). See the reference page for details and an HTTPS-by-hand example: ```console (let ((sock (rontolisp:tls-connect "example.com" 443))) ... ; speak any TLS-wrapped protocol over the handle (close sock)) ``` To start TLS **over a connection you already opened** — the shape an HTTP client library needs, since it connects (and possibly issues a proxy `CONNECT`) before starting TLS — use [`rontolisp:tls-upgrade`](../reference/functions/rontolisp-tls-upgrade.md): it takes an existing socket handle plus the server name to verify against and returns a new handle over the same connection. The bundled [`cl+ssl` shim system](asdf-systems.md#built-in-shim-systems) rides it, which is what gives `usocket`+`cl+ssl` client libraries their `https://` path. The *server* side is [`rontolisp:tls-listen`](../reference/functions/rontolisp-tls-listen.md): it takes a PKCS12 keystore file and returns a listener that the plain `rontolisp:tcp-accept` / `rontolisp:tcp-local-port` / `close` work on; each accepted connection completes its handshake on the first read. To serve straight from PEM files (certbot / OpenSSL output) instead of a PKCS12 keystore, use [`rontolisp:tls-listen-pem`](../reference/functions/rontolisp-tls-listen-pem.md). The TLS *client* functions (`tls-connect` / `tls-upgrade`) also run on the WASM `--component` backend, over wasmtime's experimental `wasi:tls@0.3.0-draft` interface — add `-S tls=y` to the socket run flags; failures answer `nil` there, `:insecure` signals (the draft exposes no verification knob), and `tls-upgrade` upgrades the handle **in place** (it answers the same handle, and requires that nothing was written to it yet). The TLS *server* functions are interpreter/JVM only — a permanent compile error on every WASM target, because the `wasi:tls` proposal defines no server interface. Both server programs below need a PKCS12 keystore holding the server key and certificate. Generate a self-signed one for localhost with the JDK `keytool` (or export one from OpenSSL with `openssl pkcs12 -export`): ```bash keytool -genkeypair -alias rontolisp-tls -keyalg EC -dname CN=localhost \ -validity 365 -ext SAN=ip:127.0.0.1,dns:localhost \ -storetype PKCS12 -keystore tls-server.p12 \ -storepass changeit -keypass changeit ``` ### An HTTPS server This is the TLS twin of the HTTP server above: identical once the listener exists, because a `tls-listen` listener hands `tcp-accept` the same kind of stream handle. `tls-listen` never returns `nil` — a missing keystore, a wrong password or a busy port signals an error instead — so there is no `nil` check. Save it as `https-hello.lisp`: ```console ;; Appends the carriage return of an HTTP CRLF line ending (write-line then ;; appends the newline). (defun crlf (s) (concatenate 'string s (format nil "~a" (code-char 13)))) ;; Consumes the request headers up to the blank line that ends them. (defun drain-headers (sock) (do ((line (read-line sock) (read-line sock))) ((or (null line) (string= line ""))))) (let ((listener (rontolisp:tls-listen "tls-server.p12" "changeit" 8443))) (write-line "https server listening on https://127.0.0.1:8443/") (do ((n 1 (+ n 1))) (nil) (let* ((sock (rontolisp:tcp-accept listener)) (request (read-line sock))) (if request (let ((body (format nil "

hello from rontolisp over TLS

request ~a: ~a

" n request))) (drain-headers sock) (write-line (crlf "HTTP/1.1 200 OK") sock) (write-line (crlf "Content-Type: text/html") sock) ;; + 1: write-line terminates the body with a newline (write-line (crlf (format nil "Content-Length: ~a" (+ (length body) 1))) sock) (write-line (crlf "Connection: close") sock) (write-line (crlf "") sock) (write-line body sock) (write-line (format nil "served request ~a: ~a" n request)))) (close sock)))) ``` Run it on the interpreter or the JVM, then (using `-k` because the certificate is self-signed): ```bash curl -k https://127.0.0.1:8443/ ``` ### A TLS Redis server The same is true of the key-value server: swap `tcp-listen` for `tls-listen` and everything else is unchanged. This serves the RESP2 protocol over TLS on port 6380 (like a real Redis with `--tls-port`). Save it as `kv-server-tls.lisp`: ```console ;; --- small string helpers --------------------------------------------------- ;; Appends the carriage return of a RESP CRLF line ending (write-line then ;; appends the newline). (defun crlf (s) (concatenate 'string s (format nil "~a" (code-char 13)))) ;; "SET key value" -> ("SET" "key" "value") (defun split-words (s) (cond ((string= s "") nil) (t (let ((p (position #\space s))) (if p (cons (subseq s 0 p) (split-words (subseq s (+ p 1)))) (list s)))))) ;; ("hello" "world") -> "hello world" (defun join-words (ws) (cond ((null ws) "") ((null (cdr ws)) (car ws)) (t (concatenate 'string (car ws) " " (join-words (cdr ws)))))) ;; t when s is a non-empty run of decimal digits (with an optional leading -). (defun integer-string-p (s) (let* ((n (length s)) (start (if (and (> n 0) (char= (char s 0) #\-)) 1 0))) (and (> n start) (do ((i start (+ i 1))) ((or (>= i n) (not (digit-char-p (char s i)))) (>= i n)))))) ;; --- RESP replies ----------------------------------------------------------- (defun reply-simple (s sock) (write-line (crlf (concatenate 'string "+" s)) sock)) (defun reply-error (s sock) (write-line (crlf (concatenate 'string "-ERR " s)) sock)) (defun reply-int (n sock) (write-line (crlf (format nil ":~a" n)) sock)) (defun reply-bulk (s sock) (if s (progn (write-line (crlf (format nil "$~a" (length s))) sock) (write-line (crlf s) sock)) (write-line (crlf "$-1") sock))) (defun reply-array-header (n sock) (write-line (crlf (format nil "*~a" n)) sock)) ;; --- request framing -------------------------------------------------------- ;; Reads one RESP bulk-string element: the "$" header line, then the ;; payload line (the payload must not contain a newline). (defun read-bulk (sock) (let ((header (read-line sock))) (if header (read-line sock) nil))) (defun read-resp-array (count sock acc) (if (<= count 0) (reverse acc) (let ((arg (read-bulk sock))) (if arg (read-resp-array (- count 1) sock (cons arg acc)) nil)))) ;; Reads one command as a list of argument strings: a "*" line starts a ;; RESP2 array (what redis-cli sends); anything else is an inline command ;; (what telnet/nc users type). nil at connection close. (defun read-command (sock) (let ((line (read-line sock))) (cond ((null line) nil) ((string= line "") (read-command sock)) ((char= (char line 0) #\*) (let ((count (subseq line 1))) (if (integer-string-p count) (read-resp-array (parse-integer count) sock nil) (list "!bad-frame")))) (t (split-words line))))) ;; --- commands --------------------------------------------------------------- ;; Handles one command; returns nil after QUIT (closing the session). (defun handle-command (args store sock) (let ((cmd (string-upcase (car args))) (key (cadr args))) (cond ((string= cmd "PING") (if key (reply-bulk key sock) (reply-simple "PONG" sock)) t) ((string= cmd "SET") (if (and key (cddr args)) (progn (setf (gethash key store) (join-words (cddr args))) (reply-simple "OK" sock)) (reply-error "wrong number of arguments for 'set' command" sock)) t) ((string= cmd "GET") (if key (reply-bulk (gethash key store) sock) (reply-error "wrong number of arguments for 'get' command" sock)) t) ((string= cmd "DEL") (let ((removed 0)) (dolist (k (cdr args)) (when (gethash k store) (remhash k store) (incf removed))) (reply-int removed sock)) t) ((string= cmd "EXISTS") (reply-int (if (and key (gethash key store)) 1 0) sock) t) ((string= cmd "INCR") (let ((current (if key (or (gethash key store) "0") "0"))) (cond ((null key) (reply-error "wrong number of arguments for 'incr' command" sock)) ((integer-string-p current) (let ((n (+ (parse-integer current) 1))) (setf (gethash key store) (format nil "~a" n)) (reply-int n sock))) (t (reply-error "value is not an integer or out of range" sock)))) t) ((string= cmd "KEYS") (let ((pattern (or key "*")) (keys nil)) (maphash (lambda (k v) (if (or (string= pattern "*") (string= pattern k)) (push k keys))) store) (reply-array-header (length keys) sock) (dolist (k keys) (reply-bulk k sock))) t) ((string= cmd "DBSIZE") (reply-int (hash-table-count store) sock) t) ((string= cmd "COMMAND") ;; redis-cli asks COMMAND DOCS on connect; an empty array satisfies it. (reply-array-header 0 sock) t) ((string= cmd "QUIT") (reply-simple "OK" sock) nil) (t (reply-error (format nil "unknown command '~a'" (car args)) sock) t)))) ;; --- server loop ------------------------------------------------------------ (let ((store (make-hash-table)) (listener (rontolisp:tls-listen "tls-server.p12" "changeit" 6380))) (write-line "mini-redis (TLS) listening on 127.0.0.1:6380 (try: redis-cli --tls --insecure -p 6380 ping)") (do ((n 1 (+ n 1))) (nil) (let ((sock (rontolisp:tcp-accept listener))) (do ((args (read-command sock) (read-command sock))) ((or (null args) (not (handle-command args store sock))) (close sock)))))) ``` Run it on the interpreter or the JVM, then talk to it over TLS (`--insecure` because the certificate is self-signed): ```bash redis-cli --tls --insecure -p 6380 set greeting hello redis-cli --tls --insecure -p 6380 get greeting ``` ## The usocket-compatible shim Existing Common Lisp networking code is usually written against the [usocket](https://github.com/usocket/usocket) portability library rather than an implementation's own socket API. rontolisp ships a built-in `usocket` package reproducing its core API over the `rontolisp:tcp-*` built-ins, so such code runs with fewer changes -- Postmodern's cl-postgres socket layer (`socket-connect` with `:element-type '(unsigned-byte 8)` + `socket-stream`) works verbatim: ```lisp (let* ((listener (usocket:socket-listen "127.0.0.1" usocket:*auto-port*)) (port (usocket:get-local-port listener)) (client (usocket:socket-connect "127.0.0.1" port :element-type '(unsigned-byte 8)))) (write-line "hello" (usocket:socket-stream client)) (let* ((server (usocket:socket-accept listener)) (line (read-line (usocket:socket-stream server)))) (usocket:socket-close server) (usocket:socket-close client) (usocket:socket-close listener) line)) ; => "hello" ``` The mapping is direct because a rontolisp socket IS its stream handle: `usocket:socket-stream` is the identity function, `usocket:socket-close` is `close`, and `usocket:socket-listen` flips the host-first argument order onto `rontolisp:tcp-listen`. `usocket:*wildcard-host*` (`"0.0.0.0"`) and `usocket:*auto-port*` (`0`) work as in usocket, the `get-local-*` / `get-peer-*` accessors read ports and addresses back, and the `with-*` convenience macros ([`with-client-socket` / `with-connected-socket` / `with-server-socket` / `with-socket-listener`](../reference/macros/usocket-with-macros.md)) bind and close sockets around a body. The package loads on first use, and it is also the built-in ASDF system `"usocket"`: `(asdf:load-system "usocket")`, `(ql:quickload :usocket)` and a third-party `.asd`'s `:depends-on ("usocket")` all resolve to it without touching the network. The servers earlier in this guide, rewritten against this shim, wrap the accept loop in `with-server-socket` (which closes each connection on every exit) and take the listen failure as a typed `usocket:socket-error`: ```console (handler-case (let ((listener (usocket:socket-listen "127.0.0.1" 7777 :reuse-address t))) (write-line "echo server listening on 127.0.0.1:7777") (do ((n 1 (+ n 1))) (nil) (usocket:with-server-socket (sock (usocket:socket-accept listener)) (let ((stream (usocket:socket-stream sock))) (write-line (format nil "client ~a connected" n)) (do ((line (read-line stream) (read-line stream))) ((null line) (write-line "client disconnected")) (write-line line stream)))))) (usocket:socket-error (e) (declare (ignore e)) (write-line "socket-listen failed (is port 7777 already in use?)"))) ``` Limitations of the shim (deliberate -- rontolisp's socket model is lite): - **TCP only.** `:protocol :datagram` (UDP) signals an error, and `socket-send` / `socket-receive` / `socket-shutdown` do not exist. - **Typed conditions on the interpreter and the JVM.** A failure in `socket-connect`/`socket-listen`/`socket-accept` signals a typed `usocket:socket-error` (message preserved), so `(handler-case (usocket:socket-connect ...) (usocket:socket-error (e) ...))` works there. The subtypes (`connection-refused-error` &c) are defined but the re-signal always uses `socket-error` (catch that). On the WASM component backend a failed connect/accept yields a `nil` handle instead of signaling, so the `handler-case` pattern has nothing to catch there (test the handle for `nil` instead). - **`socket-option` supports `:receive-timeout` only.** `(setf (usocket:socket-option sock :receive-timeout) seconds)` sets a real per-socket read deadline on the interpreter and the JVM (via [`rontolisp:tcp-set-timeout`](../reference/functions/rontolisp-tcp-set-timeout.md)); a timed-out read signals an ordinary catchable `error`, not `usocket:timeout-error`, and the getter reads the set seconds back. On the WASM backends the write SIGNALS instead of installing a timeout that never fires. Every other option signals rather than being silently ignored. - **`wait-for-input` is a `listen`-based poll**: on the interpreter and the JVM the wait is real (`listen` asks the kernel receive buffer, polled every 10 ms until data arrives or `:timeout` elapses; `:ready-only` works). On the WASM backends it returns immediately claiming readiness when nothing is buffered — reads block anyway, so the common wait-then-read loop behaves identically, but a `:timeout` poll cannot be honoured there. Stream sockets only (a listener in the list signals), and wait-list objects do not exist. - **`socket-server` does not exist** (write your own accept loop). - **The `with-*` macros close the socket on every exit** on the interpreter and the JVM (they expand over [`unwind-protect`](../reference/special-forms/unwind-protect.md)); this includes the WASM component backend (every tcp component already runs with `-W exceptions=y`). The compatibility keyword arguments (`:element-type`, `:timeout`, `:nodelay`, `:reuse-address`, ...) are accepted and ignored. - **Backends**: interpreter and JVM are full; WASM is component-only like the tcp built-ins, and the address/peer accessors return real addresses and ports there (a failure returns `nil` instead of signaling). ## See also The [`examples/net/` directory](https://github.com/making/rontolisp/tree/develop/examples/net) ships these programs as ready-to-run files (written against the usocket shim), each with per-backend run instructions in its header comment. For HTTP there is no need to hand-roll the protocol over a socket in either direction: the *client* side is `rontolisp:fetch` (see the [HTTP Requests guide](http-fetch.md)), and for the *server* side `rontolisp:http-handler` parses requests and adapts responses for you (see the [Serving HTTP guide](http-handler.md)). --- # FILE: references/guides/testing.md # Testing (rove) [Rove](https://github.com/fukamachi/rove) — Eitaro Fukamachi's testing framework, the successor of Prove — loads verbatim via `(ql:quickload "rove")` (v0.10.0), and a test suite written in its shape runs with the spec reporter on all four backends: the interpreter, a compiled JVM class, WASM Preview 1 and a WASI 0.3 component. Its dependencies resolve automatically: cl-ppcre and [dissect](https://github.com/Shinmera/dissect) from their real sources, uiop / trivial-gray-streams / bordeaux-threads to the built-in shims. ## Running a suite: `rontolisp test` `rontolisp test TARGET` runs a rove target and **exits with its verdict** — 0 when every test passed, 1 when one did not. It is this tree's version of rove's own `roswell/rove.ros` script, so a CI step, a `make test` or a git hook can read `$?`: ```bash rontolisp test tests/main.lisp # a test file rontolisp test my-app.asd # the system the .asd is named after rontolisp test my-app/tests # an ASDF system designator ``` | Target | What runs | | --- | --- | | `FILE.lisp` | The file is loaded. If its `defpackage` names an ASDF system on the search path, that system is loaded and tested instead (ASDF's package-inferred rule); otherwise the suite of the file's own package is run — unless the file already ran it, which is detected, so its tests run exactly once | | `FILE.asd` | The system the file is named after | | `SYSTEM` | `asdf:load-system` + `asdf:test-system`, then `rove:run` for a system that declares no `:perform (test-op ...)` | The status is **0** when every test passed, **1** when one failed, when the program signalled, or when *no test ran at all* — a suite that stopped registering its tests is a failure rather than a vacuous pass — and **2** when the command line itself was wrong. | Option | Meaning | | --- | --- | | `-r`, `--reporter spec\|dot\|none` | rove's reporter style, `spec` by default | | `--disable-colors`, `--color` | Force the ANSI colors off / on. The default follows the destination: a terminal gets them, a pipe does not | | `--system-path DIRS` | Directories searched for `NAME.asd` (like `PATH`) | | `--dist DISTS` | Dists `ql:quickload` may download from beside quicklisp, e.g. `ultralisp` (see the [Systems guide](asdf-systems.md#adding-a-dist-ultralisp)) | | `-o FILE` | Compile the run instead of performing it (see below) | A plain `rontolisp FILE` is unchanged and keeps Common Lisp semantics: the value of the last top-level form is dropped and the status stays 0, exactly as `sbcl --script` drops it. A program that wants a status of its own says `uiop:quit`. ## Writing tests The full assertion surface works: `deftest`, `testing`, `ok`, `ng`, `signals` (with a user-defined condition class or a built-in one like `'type-error`), `outputs`, `expands`, `pass`, `fail`, `skip`, `failing`, `setup`, `teardown`, `defhook` and `diag`. An assertion whose form signals mid-evaluation is recorded as a failure with its condition — the run continues. ```console $ cat tests/main.lisp (defpackage #:my-app/tests/main (:use #:cl #:rove #:my-app/main)) (in-package #:my-app/tests/main) (deftest add-test (testing "adding two integers" (ok (= (add 1 2) 3)) (ng (= (add 1 2) 4)))) (deftest parse-token-test (testing "invalid tokens" (ok (signals (parse-token "") 'app-error) "Parse error"))) ``` ## The two entry points **System-driven** — `rove:run` takes an ASDF system designator, loads it, and runs every suite it contains. Both system shapes work: a `:package-inferred-system` (the suite is found through the system's package dependencies) and a plain `defsystem` test system (the suite is found through the file-to-package map rove records per `deftest`, keyed on `*load-pathname*`): ```console * (rove:run :my-app/tests) ``` **File-driven** — the README FAQ style: end the test file with `run-suite`, so loading the file runs it: ```console (rove:run-suite *package*) ``` `rove:run-test` (one test symbol) and `rove:run-tests` (a list) work too. Each entry point returns whether everything passed as its first value. For non-interactive output, turn the ANSI colors off first — rove's default is colors ON outside Emacs (`rontolisp test` already does this whenever its output is not a terminal): ```console (setf rove:*enable-colors* nil) ``` ## Running on the four backends A compiled test program is self-contained: the systems named by top-level `asdf:load-system` calls are spliced in at compile time, and rove's own runtime `load-system` of an already-loaded system is a no-op. Point `--system-path` at the directories holding the `.asd` files (the app under test, rove, dissect, cl-ppcre). `rontolisp test -o` compiles the run instead of performing it, and the emitted artifact carries the same exit contract; every compiler flag applies: ```bash SP="path/to/my-app:path/to/rove:path/to/dissect:path/to/cl-ppcre" T="rontolisp test --system-path $SP tests/main.lisp" # 1. Interpreter $T # 2. JVM $T -o Tests.class && java Tests # 3. WASM Preview 1 $T -o tests.wasm && wasmtime run -W gc=y -W exceptions=y tests.wasm # 4. WASI 0.3 component $T -o tests-comp.wasm --component && \ wasmtime run -W gc=y -W exceptions=y tests-comp.wasm ``` Both WASM runs need `-W exceptions=y`: rove records a failing test through `handler-bind`, which puts the module in EH mode. A test program that is its own runner (below) compiles the same way with a plain `rontolisp`, no `test`. ## The exit code `rontolisp test` owns the exit code, and that is where it belongs: a `uiop:quit` written inside a test file kills the process the moment anything *else* loads that file — another suite, the REPL, a system that depends on it. The file owns the tests; the runner owns the exit. Upstream draws the same line: `rove.ros` calls `uiop:quit`, and a `.asd`'s `:perform (test-op ...)` leaves the exit to whoever invoked ASDF. Writing it by hand is right in exactly one place: a one-line runner of your own. `rove:run` returns the passed-p boolean, and `uiop:quit` really ends the process on every backend: ```console (uiop:quit (if (rove:run :my-app/tests) 0 1)) ``` ## Examples that check themselves Three examples in this repository are written this way, and the example harness runs them on every backend — copy whichever shape fits: | Example | Shape | | --- | --- | | [`examples/console/roman.lisp`](https://github.com/making/rontolisp/blob/develop/examples/console/roman.lisp) | A program that prints its demo and then asserts what it printed | | [`examples/cloudflare-workers/httpbin/check.lisp`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin/check.lisp) | A driver: it `load`s the program under test, exercises it and asserts the parsed answers | | [`examples/browser/minesweeper/minesweeper-core-test.lisp`](https://github.com/making/rontolisp/blob/develop/examples/browser/minesweeper/minesweeper-core-test.lisp) | A test file beside a program that cannot run head-less, over the rendering-free core it shares | None of them defines a package or an ASDF system. For a single file, `(use-package :rove)` in `cl-user` plus `run-suite *package*` at the end is the whole of it: ```console (asdf:load-system :rove) (use-package :rove) (setf *enable-colors* nil) (deftest arithmetic (testing "adding two integers" (ok (= (add 1 2) 3)))) (uiop:quit (if (run-suite *package*) 0 1)) ``` ## Limitations - **A raw WASM trap ends the run.** On the interpreter and the JVM a test body that hits `(car 1)` or `(/ 1 0)` becomes a recorded failure; on the WASM backends those compile to raw traps, which no handler can catch. A test that SIGNALS (any `error` call, `check-type`, a bad `aref`) is recorded fine everywhere. - **No backtraces in failure reports** — dissect's stack introspection is the empty no-op interface on every backend, so the `at file:line` / stack lines SBCL prints are absent. - **Symbols in assertion descriptions print package-qualified** (`Expect (= (MY-APP/MAIN:ADD 1 2) 3) ...` where SBCL prints `(= (ADD 1 2) 3)`) — the printer does not yet consult `*package*` accessibility. - **`deftest`'s `:compile-at :run-time` option is interpreter-only** — it routes the body through `compile`, whose eval runtime cannot expand user macros on the compiled backends. - **`:style :none` on a compiled program** needs the program to load `rove/reporter/none` itself — `make-reporter` loads an unknown style's system at run time, which only the interpreter can do. `:spec` (the default) and `:dot` are built in. `rontolisp test -r none -o ...` loads it for you. --- # FILE: references/guides/wasm-browser.md # Running WASM in a Browser Two paths deliver a rontolisp WASM build to a browser: - **Components via `jco transpile`** — turns a component into plain JavaScript modules whose exports become JavaScript functions. - **Reactor modules by hand** — a `--no-wasi` (wasm-GC, full language) or `--no-gc` (scalar-only) core module has no imports either way, so `WebAssembly.instantiate` + `instance.exports` is the whole host side, byte for byte the same JavaScript for both backends. Node and the browser use the same code. ## Running a Component in a Browser (jco) A component is not a wasmtime-only artifact. `jco transpile` turns one into JavaScript, and the result runs in a browser — the exports become plain JavaScript functions. The example used throughout this section is `count-vowels`: one exported function taking a string and returning how many vowels it contains. ```lisp ;; count-vowels.lisp (defun vowelp (c) (or (char= c #\a) (char= c #\e) (char= c #\i) (char= c #\o) (char= c #\u) (char= c #\A) (char= c #\E) (char= c #\I) (char= c #\O) (char= c #\U))) (defun count-vowels (s) (let ((n 0)) (dotimes (i (length s)) (when (vowelp (char s i)) (setq n (+ n 1)))) n)) (rontolisp:wasm-export 'count-vowels :params '(:string) :returns :int) (count-vowels "Hello, World!") ; => 3 ``` It is pure compute — no cons, no I/O — so it stays inside the [`--no-gc` subset](wasm-nogc.md#eligible-subset) and compiles to a component with zero imports. jco camel-cases the component-model export name, so `count-vowels` arrives as `countVowels`. (Verified with jco 1.25.2 on Chrome 149.) The same program driven from a Node host and from a Java host, with the export declared in WIT instead of `wasm-export`, is [`examples/count-vowels`](https://github.com/making/rontolisp/tree/develop/examples/count-vowels). **A `--no-gc --component` needs nothing at all.** Its world has no imports, so jco emits one self-contained ES module — the core WASM base64-inlined inside it, about 90 KB for `count-vowels` — with no `import` statements of its own. The page supplies no shim, no import map and no polyfill: ```bash rontolisp count-vowels.lisp --no-gc --component -o cv.wasm npx @bytecodealliance/jco transpile cv.wasm -o dist ``` ```html ``` **A printing `--no-gc --component` cannot run through jco yet.** Its [print micro-adapter](wasm-nogc.md#compact-component-output---no-gc---component) imports `wasi:cli/stdout@0.3.0` and lifts every export async, so it hits the same jco gaps as the GC component below (jco cannot call an async-lifted export, and its `future` runtime is incomplete) — and the WASI 0.3 shim is Node-only anyway. Keep the program print-free if the component's destination is jco or a browser; the [plain module path](#reactor-modules-by-hand) with a hand-written import object is unaffected. **A wasm-GC `--component` loads and computes, but cannot print there yet.** Chrome supports wasm-GC, JSPI and the canonical ABI, and the component's synchronous exports return correct values. Two gaps are in the way of the rest, both on the JavaScript side (wasmtime runs all of it): - The WASI 0.3 imports it needs have no browser implementation: `@bytecodealliance/preview3-shim` declares only a `node` condition in its package `exports` and pulls in `node:worker_threads`, `node:net`, `node:http`, ... A page must hand-write a stand-in for the nine members jco destructures at module top level — `environment.getEnvironment`, `stdout.writeViaStream`, `stderr.writeViaStream`, `stdin.readViaStream`, `monotonicClock.now`, `systemClock.now`, `preopens.getDirectories`, `types.Descriptor`, `random.getRandomU64` — which for a pure-compute export only have to exist. - Printing then fails inside jco's own generated code, which *references* `FutureReadableEnd` / `FutureWritableEnd` / `FutureEnd` but defines none of them (`ReferenceError: FutureReadableEnd is not defined`). It is reached through `wasi:cli/stdout`'s `write-via-stream`, whose WIT result is a `future`. Separately, jco cannot yet *call* an async export (its 0.3 async ABI gap again), which is what an [`:async t`](wasm-component.md#component-model-function-exports-wasm-export) I/O export is. Node is the weaker host here: Node 22 has no JSPI (`WebAssembly.Suspending is not a constructor`), so it cannot even instantiate a transpiled GC component, while Chrome can. ## Reactor Modules by Hand A reactor module (`--no-wasi` or `--no-gc`) imports nothing, so the whole host side is "instantiate, then call the exports" — and it is the same code in Node and the browser. Here is a complete, copy-paste example end to end. Start with a small kit of three exports: ```lisp ;; mathkit.lisp (defun fact (n) (if (<= n 1) 1 (* n (fact (1- n))))) (defun area (r) (* 3.141592653589793 r r)) (defun in-range (x lo hi) (if (< x lo) nil (if (> x hi) nil t))) (rontolisp:wasm-export 'fact :params '(:int) :returns :int) (rontolisp:wasm-export 'area :params '(:float) :returns :float) (rontolisp:wasm-export 'in-range :params '(:int :int :int) :returns :bool) ``` Compile it with `--no-gc` so it runs on any engine; everything unreachable from the exports is dropped without asking, which here leaves a whole module of ~200 bytes: ```bash rontolisp mathkit.lisp --no-gc -o mathkit.wasm ``` On Node 18+, save this as `run.mjs` and run `node run.mjs`: ```js import { readFile } from 'node:fs/promises'; // Node reads the .wasm from disk. In a browser, use the streaming fetch shown below. const bytes = await readFile(new URL('./mathkit.wasm', import.meta.url)); const { instance } = await WebAssembly.instantiate(bytes); // no import object const ex = instance.exports; console.log(ex.fact(10)); // 3628800 console.log(ex.area(2)); // 12.566370614359172 console.log(Boolean(ex['in-range'](5, 0, 10))); // true (:bool crosses as 0 / 1) console.log(Boolean(ex['in-range'](42, 0, 10))); // false ``` ``` 3628800 12.566370614359172 true false ``` The browser differs only in how the bytes are loaded — `instantiateStreaming` takes a `fetch` directly — so a whole page is: ```html ``` A few boundary details worth knowing: - A hyphenated Lisp name such as `in-range` is not a valid JavaScript identifier, so reach it with bracket access: `ex['in-range'](...)`. - `:int`/`:float` arrive as plain JS numbers; `:bool` crosses as an `i32` (`0`/`1`), so wrap it in `Boolean(...)` for a real JS boolean. - A **`--no-gc`** module runs on **any** WebAssembly engine; a GC **`--no-wasi`** module needs a wasm-GC-capable one (Node 22+, current browsers). The JavaScript above is byte-for-byte identical for both — swap the compile flag and nothing else changes. Proof, not just assertion — recompile the same source with `--no-wasi` and run the unchanged `run.mjs`: ```bash rontolisp mathkit.lisp --no-wasi -o mathkit.wasm node run.mjs ``` ``` 3628800 12.566370614359172 true false ``` Nothing above was `--no-gc`-specific: `mathkit.lisp` never leaves the non-GC subset, so it is one of the (many) programs that compile cleanly under either backend. A program that needs the full language — `cons`, `string-upcase`, `hash-table`s, `defstruct`, ... — simply **requires** `--no-wasi`; it is not a fallback or a lesser path, only the backend a wasm-GC-capable engine (Node 22+, every current browser) runs. ### Passing strings (`:string`) The scalar example above needs no memory because `:int`/`:float`/`:bool` cross the boundary as plain numbers. A `:string` instead passes a `(ptr, len)` pair through the module's exported `memory`: the host writes the argument bytes into memory (at an offset reserved by the exported `__ronto_alloc(size)` bump allocator), passes `(ptr, len)`, then decodes the `(ptr, len)` the export returns. `:string` works under `--no-gc`, so the module still runs on **any** engine — as long as the function stays within the non-GC string subset (see the [eligible subset](wasm-nogc.md#eligible-subset)). A greeting builder is enough to show the protocol: ```lisp ;; greetkit.lisp (defun greet (name) (concatenate 'string "Hello, " name "!")) (rontolisp:wasm-export 'greet :params '(:string) :returns :string) ``` ```bash rontolisp greetkit.lisp --no-gc -o greetkit.wasm ``` ```js import { readFile } from 'node:fs/promises'; const bytes = await readFile(new URL('./greetkit.wasm', import.meta.url)); const { instance } = await WebAssembly.instantiate(bytes); // no import object const ex = instance.exports; const enc = new TextEncoder(), dec = new TextDecoder(); // Copy a JS string into linear memory; return its (ptr, len). function write(str) { const b = enc.encode(str); const ptr = ex.__ronto_alloc(b.length); new Uint8Array(ex.memory.buffer, ptr, b.length).set(b); return [ptr, b.length]; } // Decode a (ptr, len) result. Re-read ex.memory.buffer AFTER the call: a call may grow // memory, which detaches the previous ArrayBuffer. const read = (ptr, len) => dec.decode(new Uint8Array(ex.memory.buffer, ptr, len)); console.log(read(...ex.greet(...write('rontolisp')))); // Hello, rontolisp! ``` ``` Hello, rontolisp! ``` With [`--no-gc --component`](wasm-nogc.md#compact-component-output---no-gc---component) the same `:string` export instead crosses as a typed component-model `string`, and all of the host-side glue above disappears (the canonical ABI does the copying, and a post-return function keeps the heap flat). Richer string functions (`string-upcase`, `subseq`, `string=`, …) are outside the non-GC subset; using one means compiling for the wasm-GC backend (`--no-wasi`) instead — the boundary protocol is identical, only the engine must be wasm-GC capable. The `:s-expr` example below shows that path. ### Passing lists (`:s-expr`) A `:s-expr` carries **any** Lisp value as s-expression *text*: the module parses the input with its embedded reader and prints the result back, over the same `(ptr, len)` / `__ronto_alloc` protocol. That reader/printer/cons machinery is **wasm-GC only**, so `:s-expr` (and the richer string functions above) need `--no-wasi` and a wasm-GC-capable engine (Node 22+, a current browser): ```lisp ;; textkit.lisp (defun shout (s) (string-upcase s)) (defun rev (lst) (reverse lst)) (rontolisp:wasm-export 'shout :params '(:string) :returns :string) ; "hello" -> "HELLO" (rontolisp:wasm-export 'rev :params '(:s-expr) :returns :s-expr) ; a list, reversed ``` ```bash rontolisp textkit.lisp --no-wasi -o textkit.wasm ``` ```js // Same instantiate + write/read helper as above (textkit.wasm needs a wasm-GC engine). console.log(read(...ex.shout(...write('hello')))); // HELLO console.log(read(...ex.rev(...write('("a" "b" "c")')))); // ("c" "b" "a") ``` ``` HELLO ("c" "b" "a") ``` In the browser only the loading line changes (`WebAssembly.instantiateStreaming(fetch(...))`); the `write`/`read`/`memory`/`__ronto_alloc` logic is identical. A function that returns a multi-value `(ptr, len)` shows up in JS as a two-element array, hence `read(...ex.shout(...))`. --- # FILE: references/guides/wasm-component.md # WASI 0.3 Component (`--component`) Add `--component` to emit a WASI 0.3 (Preview 3) **component** instead of a Preview 1 core module. The component prints through `wasi:cli/stdout@0.3.0`: ```bash rontolisp hello.lisp --component -o hello.wasm wasmtime run -W gc=y hello.wasm ``` ``` 3 ``` In WASI 0.3 all byte I/O flows through the built-in component-model `stream` / `future` types and the async canonical ABI. rontolisp keeps the same Preview 1 core module unchanged — it still imports the nine `wasi_snapshot_preview1` functions — and an **adapter** core module implements them over WASI 0.3 (`wasi:cli`, `wasi:filesystem`, `wasi:clocks`, `wasi:random`) using `stream.new`/`stream.read`/`stream.write` and `future.read`. Those built-ins are the **asynchronous** (non-blocking) variants: when one reports BLOCKED, the task parks on a blocking `waitable-set.wait` until the completion event arrives, so the adapter stays straight-line code. The component's `wasi:cli/run@0.3.0` export (an `async func`) is lifted as an async-typed export, from which that blocking wait is legal. All of this sits on the base component-model async ABI, enabled by default in wasmtime 46+ — no gated feature flags remain; only `-W gc=y` (for the wasm-GC core) is needed. The wasmtime invocation does **not** select the output kind. `wasmtime run` is wasmtime's default subcommand and auto-detects a core module vs a component, so `wasmtime run -W gc` runs a Preview 1 `hello.wasm` just as well. Only the `--component` compile flag decides whether a Preview 1 core module or a WASI 0.3 component is produced. (The practical difference shows up on a component-only runtime, which runs the component but not the Preview 1 core module.) ## What Runs Inside a Component What works inside a component, and what each feature needs at run time: - `print`/stdout, stdin (`read`, 0-argument `read-line`, over `wasi:cli/stdin@0.3.0`), and file I/O (`open`, `close`, `write-line`, stream `read-line`, `load`, `with-open-file`) all work. In an async body a pending stdin `read-line`/`read-char` suspends only its own task, like a socket read — a concurrent `rontolisp:wait-for` timer keeps running while the program waits for input. File access requires `--dir` (a relative path resolves against the first preopened directory, an absolute one against the preopened directory whose name is its longest prefix): ```bash cat > fileio.lisp <<'EOF' (with-open-file (out "greeting.txt" :direction :output) (write-line "hello" out)) (with-open-file (in "greeting.txt") (print (read-line in))) EOF rontolisp fileio.lisp --component -o fileio.wasm wasmtime run -W gc=y --dir . fileio.wasm # "hello" ``` - `random` draws real entropy from `wasi:random@0.3.0` (Preview 1 uses the host's `random_get`), so `(random N)` differs each run. `get-universal-time` / `get-internal-real-time` / `get-internal-run-time` read `wasi:clocks@0.3.0` (`system-clock`/`monotonic-clock`), and `uiop:getenv` reads `wasi:cli/environment@0.3.0`. - Outgoing HTTP (`rontolisp:fetch` with the `rontolisp:await` / `rontolisp:futurep` future operations) works in component mode, including true asynchrony: `fetch` sends the request and returns a future (wrapping the in-flight `wasi:http` response handle) immediately, so several requests can overlap before `await` suspends on each. The future operations themselves compile in every mode; only `fetch` is component-only. fetch imports the async `wasi:http@0.3.0` (`wasi:http/types` + `wasi:http/client`) — uniformly WASI 0.3, like the rest of the component. Run a fetch component with `-S http=y` (which makes the host provide `wasi:http`) in addition to the usual flags. Non-fetch components do not import `wasi:http`, so they do not need `-S http`. A transport failure (refused connection, unresolvable host) signals `rontolisp:wit-error` at `await` time on every backend; `nil` comes back only for a request that cannot be started. See the [HTTP fetch guide](http-fetch.md) for the request/response shape. - TCP sockets (`rontolisp:tcp-connect` / `tcp-listen` / `tcp-accept` / `tcp-local-port`) work in component mode over `wasi:sockets@0.3.0` (natively WASI 0.3 — no 0.2 hybrid). A socket is a bidirectional stream handle, so `read-line` / `write-line` / `write-string` / `read-byte` / `write-byte` / `close` work on it directly. Run a socket component with `-W exceptions=y -S tcp=y -S inherit-network=y` in addition to the usual flags (a tcp component always compiles in exception-handling mode); without the `-S` flags the component still starts but every socket operation fails and yields `nil`. Hosts must be IPv4 literals (no hostname resolution yet). `rontolisp:fetch` and the tcp functions can be combined in one component, and tcp works inside a `rontolisp:http-handler` (serve) component. In an async body a pending `tcp-accept` or socket read suspends only its own task — other tasks (a `rontolisp:wait-for` timer, another request) keep running. See the [TCP sockets guide](tcp-sockets.md) for the full API. - The compiled Lisp otherwise behaves identically to the Preview 1 output for the supported features. Serving incoming HTTP (`rontolisp:http-handler`) also compiles to a component, but a different kind (exporting `wasi:http/handler@0.3.0`) run under `wasmtime serve` — see the [HTTP handler guide](http-handler.md). The imported surface follows the program, because [tree shaking](../compiling/wasm.md#optimize-tree-shaking) narrows it: a component that only prints imports `wasi:cli/{types,stdout}` and nothing else — `wasi:cli/stderr` joins only when the program can actually write there ([`warn`](../reference/macros/warn.md), `*error-output*`, or the report an uncaught condition prints) — which is what a `wasm-tools component wit` on it, and the `--emit-wit` output below, will show. Nothing about the flags you run it with changes; a host simply has less to provide. A component built with `--optimize=off` instead declares all of the above whether or not the program uses them, so its imported surface is the same for every program. ## Component-model Function Exports (`wasm-export`) Under `--component`, a [`rontolisp:wasm-export`](wasm-host-boundary.md#exporting-lisp-functions) becomes a **typed component-model export**, callable through the canonical ABI with WAVE syntax (`wasmtime run --invoke 'name(args)'`, no experimental warning) — and it co-exists with the `wasi:cli/run` command entry, so the same component still runs as a command: ```lisp (defun sumsquared (a b) (* (+ a b) (+ a b))) (rontolisp:wasm-export 'sumsquared :params '(:int :int) :returns :int) (print (sumsquared 10 10)) ``` ```bash rontolisp sumsq.lisp --component -o sumsq.wasm wasmtime run -W gc=y --invoke 'sumsquared(2, 3)' sumsq.wasm # 25 (the export's return value, rendered by wasmtime) wasmtime run -W gc=y sumsq.wasm # 400 (the ordinary run entry executes the top-level program) ``` The two commands print different things: `--invoke` calls **only** the named export — the top-level program (the `wasi:cli/run` entry) does not run — and the `25` is wasmtime rendering the export's return value in WAVE syntax, not output from `print`. The plain `run` executes the top-level program instead, so the `400` is the output of `(print (sumsquared 10 10))`. The typed signature (each integer designator under its own WIT name — `:s32` → `s32`, `:u32` → `u32`, … — plus `:float` → `f64`, `:bool` → `bool`, `:string` → `string`, `:s-expr` → `string` carrying the printed s-expression text, omitted `:returns` → no result) is visible to any component host, and `:as` renames the component export just like the core one. A `:string` boundary crosses as a real component-model `string` — no manual pointer handling on either side. The host lowers the argument bytes into linear memory and reads the result back out through the canonical ABI, and the module frees the per-call allocations afterwards (a canonical *post-return* function pops the bump allocator), so a resident instance stays flat across repeated calls: ```lisp ;; greet.lisp (defun greet (s) (concatenate 'string "Hello, " s)) (rontolisp:wasm-export 'greet :params '(:string) :returns :string) ``` ```bash rontolisp greet.lisp --component -o greet.wasm wasmtime run -W gc=y --invoke 'greet("世界")' greet.wasm # "Hello, 世界" ``` By default an export is lifted **synchronously**. Even so, I/O inside it usually works: the asynchronous built-ins complete without blocking whenever the host accepts immediately (stdout does), and only a host that reports BLOCKED forces the blocking wait, which traps in a synchronous task with "cannot block a synchronous task". Declare the export async with **`:async t`** to lift it against an async function type instead — the same async-typed lift as the `run` entry — and remove that residual risk. `wasmtime --invoke` calls an async export exactly the same way: ```lisp ;; status.lisp (rontolisp:async-defun fetch-status (url) (print "fetching") (getf (rontolisp:await (rontolisp:fetch url)) :status)) (rontolisp:wasm-export 'fetch-status :params '(:string) :returns :int :async t) ``` ```bash rontolisp status.lisp --component -o status.wasm wasmtime run -W gc=y -W exceptions=y -S http=y \ --invoke 'fetch-status("https://httpbin.ik.am/status/204")' status.wasm # "fetching" # 204 ``` In the component's WIT-level contract an `:async t` export is an `async func` (for example, jco types it as a Promise-returning function, while a sync export stays a plain function). Sync and async exports mix freely in one component, `:async` composes with every boundary type including `:string`/`:s-expr`, and a program without `:async` exports produces byte-identical output. Current limitations of component exports: - A **sync** (default) export can usually do I/O anyway (the asynchronous built-ins complete without blocking when the host accepts immediately); only a host that reports BLOCKED makes the blocking wait trap with "cannot block a synchronous task". Opt into `:async t` when the export prints, fetches, or otherwise does I/O to remove that residual risk; keep pure-compute exports sync. - `:async` is meaningful only here: Preview 1 / `--no-wasi` core exports ignore it (the host provides I/O directly there), and `--no-gc --component` rejects it (the compact reactor component has no async adapter). - jco (1.25.2) transpiles an `:async t` export and types it as async, but cannot call it yet — its support for the 0.3 async ABI is not implemented upstream (the same gap as calling the transpiled `run`). `wasmtime run --invoke` is the verified path for async exports; sync exports work on both. - The export name must be a lower-kebab-case component-model name (`sum-squared`); for a Lisp name outside that grammar the compiler asks you to rename it with `:as`. - Invoking an export does not run the program's top level first, so an export that reads a `defvar`/`defparameter` global would see it uninitialized (this matches the Preview 1 `--invoke` behavior). The [reactor shape below](#reactor-components---component---no-wasi) removes this: its top level runs at instantiation. For a pure-compute export kit, the compact [`--no-gc --component`](wasm-nogc.md#compact-component-output---no-gc---component) variant emits the same typed exports (minus `:s-expr`) in a component of a few hundred bytes that needs no wasmtime flags at all. ## Reactor Components (`--component --no-wasi`) Add `--no-wasi` to emit a **reactor component**: a component that imports **nothing**. There is no WASI surface at all — `wasm-tools component wit` shows not a single `import` line, at every `--optimize` level — so any component host instantiates it with an empty import object, and its only exports are the lifted `wasm-export` functions (there is no `wasi:cli/run` entry): ```lisp ;; greet-reactor.lisp (defparameter *greeting* "hello, ") (defun greet (name) (concatenate 'string *greeting* name)) (rontolisp:wasm-export 'greet :params '(:string) :returns :string) ``` ```console $ rontolisp greet-reactor.lisp --component --no-wasi -o greet.wasm $ wasm-tools component wit greet.wasm package root:component; world root { export greet: func(p0: string) -> string; } $ wasmtime run -W gc=y --invoke 'greet("world")' greet.wasm "hello, world" ``` Unlike every other component shape, the **top-level forms run at instantiation** (the core module's start section), so the `defparameter` above is already assigned when the first export call arrives — no host cooperation needed. The flip side: a top-level form that traps now prevents instantiation itself. The I/O contract is the Preview 1 reactor's, unchanged: output (`print`, `format t`) is **discarded**, input/time/`random` **trap**, and `with-open-file`/`open` **signal** a catchable error. Anything that would put an import back refuses to compile with `--no-wasi` — `rontolisp:fetch`, `rontolisp:http-handler`, `rontolisp:wait-for` and `rontolisp:wit-import` each report the conflict by name. For a JavaScript embedder this is the component shape that needs no WASI shim at all: `jco transpile greet.wasm --instantiation sync` generates glue whose `ImportObject` type is literally empty (`{}`). The same applies to a `clack:clackup ... :server :rontolisp` application — the reactor build's `handle-request` export (see [the Clack guide](clack.md#a-host-that-calls-you-the-reactor-build)) works under `--component --no-wasi` too, giving a Clack application a typed `handle-request: func(p0: string) -> string` component export. --- # FILE: references/guides/wasm-gc-module.md # wasm-GC Core Module (Default Output) The default output — no flags beyond `-o file.wasm` — is a **WASI Preview 1 core module** over the wasm-GC value model: - **wasm-GC** — Integers are represented as `i31ref` (a value past the fixnum range is boxed as a signed 64-bit struct, and past that as a limb-based big integer, keeping arithmetic exact at any magnitude). Floating-point numbers are boxed in a `float_struct { f64 }`. All values on the stack are typed as `(ref eq)`. This is what supports the full language (cons cells, symbols, closures, hash tables, `eval`, ...), and why the module needs a wasm-GC capable runtime such as wasmtime 14+ (`-W gc`), Node 22+, or a current browser. - **WASI Preview 1** — the module imports the eight `wasi_snapshot_preview1` functions (`fd_write` for stdout, `random_get`, clocks, environment, ...) and exposes the `_start` entry point, so `wasmtime run` executes the program's top level like a command. ```bash echo '(print (+ 1 2))' > hello.lisp rontolisp hello.lisp -o hello.wasm wasmtime run -W gc hello.wasm # 3 ``` An exported function is a **raw core function**: scalars (`:int`/`:float`/`:bool`) cross as plain numbers, so `wasmtime --invoke` and `instance.exports.fact(5)` work directly. The memory-backed `:string` and `:s-expr` designators pass a `(ptr, len)` pair through the module's exported `memory`, together with a `__ronto_alloc(size)` bump allocator the host uses to stage argument bytes — that protocol needs a host that can read and write memory (JavaScript, not `wasmtime --invoke`), and is walked through end to end in the [browser guide's reactor section](wasm-browser.md#reactor-modules-by-hand). Instantiating the module still needs the eight WASI imports satisfied; `wasmtime run` provides them automatically, a browser host can supply no-op stubs for a pure-compute function, or add [`--no-wasi`](#no-wasi-reactor-mode) to drop them entirely. For the full picture of how `wasm-export` behaves in this shape (types carried, `:as` renaming, arity match, void returns), see the [host-boundary guide](wasm-host-boundary.md). ## Value-Model Behavior Notes Two behavioral notes on the wasm-GC value model: - **Parameter limit.** A function (`defun` or `lambda`) may take at most **seven parameters** (the interpreter and JVM backends have no such limit). A fixed-arity `defun` past the limit is bundled automatically: the compiler keeps the first six parameters, packs the rest into a list, and rewrites every direct call site to match — so wide library signatures compile unchanged. Taking such a function's value with `#'name`/`symbol-function` is a compile error (only direct calls know the bundled shape), and a `lambda` or variadic function past the limit still errors — bundle those arguments into a list yourself. The rest list of a variadic function counts as one parameter, so a `&rest` function may declare at most six required parameters while accepting any number of arguments at a direct call site. - **Float printing.** Floats print byte-identically on every backend: the shortest decimal that reads back as the same value (`(print 1.21)` prints `1.21`, `(print (* 1.5 (expt 10.0 12)))` prints `1.5e12`), with `Infinity`, `-Infinity` and `NaN` as those words. `rontolisp:json-stringify` carries the same text. ## Reclaiming the Host's Buffer (the Arena API) The engine collects everything the Lisp side allocates — cons cells, closures, strings — so a wasm-GC module needs no memory discipline *inside*. The one thing the engine cannot see is the buffer the **host** wrote the argument bytes into: that is linear memory, an opaque byte array it never traces, handed out by the `__ronto_alloc` bump allocator, which never frees. A resident host that allocates a fresh input buffer per call therefore grows linear memory without bound. So a module that exports its `memory` also exports a matched pair over the same heap pointer: | export | signature | meaning | | --- | --- | --- | | `__ronto_alloc_mark` | `() -> i32` | snapshot the current bump-heap top | | `__ronto_alloc_reset` | `(i32 mark) -> ()` | restore the top to a saved mark | Snapshot **before** allocating the input, restore **after** reading the result, and a resident instance stays flat no matter how many times it is called or how long each input is: ```js const countVowels = (s) => { const b = enc.encode(s); const mark = ex.__ronto_alloc_mark(); // snapshot BEFORE allocating const ptr = ex.__ronto_alloc(b.length); // a fresh buffer, any length new Uint8Array(ex.memory.buffer, ptr, b.length).set(b); const n = ex['count-vowels'](ptr, b.length); // scalar result, read out here ex.__ronto_alloc_reset(mark); // pop the input buffer return n; }; ``` Two rules, as for any arena: - Only reset to a mark taken **before** everything still live. - A `:string`-**returning** export leaves its result bytes in memory: **decode them before resetting**, or the next allocation overwrites them. One backend-specific guard: on the GC backend the same heap pointer also holds the interned-symbol byte pool (a symbol's identity *is* its offset there), so `__ronto_alloc_reset` never pops below that pool's high-water mark. A call that interns a new symbol (`read`, `intern`, `gensym`) therefore keeps its input buffer; every other call pops all the way back. Nothing to do host-side. The bracket is the same one the [`count-vowels` example](https://github.com/making/rontolisp/tree/develop/examples/count-vowels) walks through on `--no-gc` (from Node and from [Endive](https://endive.run)) — the boundary protocol does not change with the backend, only what you may write inside the function does. Under [`--component`](wasm-component.md) there is no arena API and nothing to bracket: the canonical ABI's `post-return` frees the argument strings for you. ## No-WASI (Reactor) Mode Add `--no-wasi` to emit a Preview 1 module that imports **no** WASI functions, so a host can instantiate it with no import object at all — a "reactor"/library module whose only surface is the exported Lisp functions: ```bash rontolisp fact.lisp --no-wasi -o fact.wasm wasmtime run --invoke fact -W gc fact.wasm 5 # => 120 ``` A reactor is just as easy to drive from JavaScript: there is **no import object**, so the host side is just "instantiate, then call the exports" (`WebAssembly.instantiate(bytes).then(({ instance }) => instance.exports.fact(5))`). A complete, copy-paste runnable Node + browser example is in the [browser guide's reactor section](wasm-browser.md#reactor-modules-by-hand). The WASI import slots are filled with internal stubs so every function index stays fixed (no other codegen changes). What those stubs do follows one rule: **a stub answers when the answer is true of the module, and refuses when answering would mean inventing a value you could not tell from a real one** — though a value the *host* hands in is not an invention, which is how the clock and randomness are served. A reactor really has no output destination, no environment variables and no files, so those are answered; nothing about it makes a byte of input true, so that is not. | what your program does | on `--no-wasi` | | --- | --- | | `print`, `format t`, writes to `*error-output*` | **discarded** (a sink); the call returns normally | | `(uiop:getenv "X")` | `nil` — the environment is empty | | `probe-file`, `directory`, `load` | nothing is found (`nil`, or a catchable error) | | `with-open-file`, `open` | **signals** a catchable error naming WASI | | `(random n)`, `(random 1.0)` | works — a built-in generator, or the host's with `--host-random` | | `rontolisp:random-bytes` | **signals**, unless `--host-random` supplies real entropy | | `rontolisp:fetch` | **compile error**, unless `--host-fetch` routes it at the host's own HTTP client | | `get-universal-time` and the other clocks | the time the host set through `__ronto_set_time`; **signals** until it does | | `(sleep n)` | **signals** — nothing here can make an interval elapse | | `read`, `read-line`, `read-char` (standard input) | **traps** | Everything that signals does so at CALL time, so an `ignore-errors` or `handler-case` around it keeps working and a library whose file-loading or clock-probing branch is dead code still compiles and runs. Only standard input traps, and a trap is not catchable — that is the one place where a `--no-wasi` module still dies rather than reports. Output being a sink is what lets a library that logs while it loads be quickloaded into a reactor at all — the alternative was killing the instance for a log line. If you need the text, return it from the export instead. The clock and randomness are the two services with a choice to make, because both are values the module cannot produce for itself. A core module exports one hook for each — `__ronto_set_time` (nanoseconds since the Unix epoch) and `__ronto_seed_random` — to be called **before `_initialize`**, which is what makes a library that timestamps or draws while it *loads* loadable at all; and `--host-random` routes `random` at a host import instead. Unseeded, the generator repeats one sequence; unset, the clock signals rather than report 1970, and it holds the value you wrote until you write another (so `(sleep n)` signals here — nothing can make an interval elapse). A **reactor component** has neither hook: its top level runs at instantiation, so there is no window in which a host could go first. All of it, with the JavaScript, is in the [clock and randomness guide](clock-and-random.md). Combined with `--component`, the same contract produces a **reactor component** — a component that imports nothing, whose top-level forms run at instantiation — see [the component guide](wasm-component.md#reactor-components---component---no-wasi). **Outgoing HTTP** has the same shape as the clock and randomness: a value only the host can produce. `--host-fetch` routes [`rontolisp:fetch`](../reference/functions/rontolisp-fetch.md) at two injected host imports — `env.fetch(request-json) -> response-head-json` for the request and the reply's head, `env.readResponseBody(ptr, cap) -> i32` for its body, pulled a chunk at a time into a buffer the module passes. Same options, same `(:status :headers :body)` answer as every other backend, `:body` the same asynchronous stream. A JavaScript host implements it with its own `fetch()` behind `WebAssembly.Suspending` (JSPI — the whole wasm stack parks until the promise settles, so the Lisp side stays ordinary synchronous-looking `(await (fetch ...))`) and then must enter every export through `WebAssembly.promising` and serialise calls (or compile [`--reentrant`](wasm-host-boundary.md#overlapping-calls---reentrant) to overlap them) — a re-entered export refuses with a trap instead of corrupting both calls; a synchronous host (node without JSPI, a test stub) just answers directly. The build prints exactly this obligation. The worked example is [`examples/cloudflare-workers/dog-fetcher`](https://github.com/making/rontolisp/tree/develop/examples/cloudflare-workers/dog-fetcher). A `--no-wasi` compile also reads the source with the `:rontolisp-reactor` feature active, which is how a `clack:clackup ... :server :rontolisp` program becomes a **served** reactor here: the handler backend stores the application and the compiler synthesizes a `handle-request` export the host calls per request — see [the Clack guide](clack.md#a-host-that-calls-you-the-reactor-build). A bare [`rontolisp:http-handler`](../reference/functions/rontolisp-http-handler.md) directive lowers to the same transport (a reactor owns no socket, so "serve this handler" can only mean the host-driven envelope), so the one `http-handler` source serves a socket on the interpreter/JVM, `wasi:http` under `--component`, and the `handle-request` export here. Because the module is a reactor (not a WASI command), its top-level initializer is exported as **`_initialize`** rather than `_start`. A host should call `_initialize` once after instantiation to run top-level forms (`defvar`/`defparameter`/`setq` globals that an exported function reads); pure-compute reactors that hold no top-level state can skip it. ### What the build tells you before you run it A refusal reached from a **top-level form** is the exception to everything above: there is no caller to catch the condition, and the message goes to the output sink — so the instance dies inside `_initialize` with a bare `RuntimeError: unreachable` naming nobody. Which primitives your load path can reach is something the build already knows, so it says so — one line per primitive, with the call chain that got there: ```console $ rontolisp app.lisp --no-wasi -o app.wasm .../session/state/cookie.lisp:25:12: warning: GET-UNIVERSAL-TIME is reachable from a top-level form of this --no-wasi module (the top-level (DEFSTRUCT COOKIE-STATE)), so it can run while the module LOADS -- where nothing catches it and the host sees only RuntimeError: unreachable. The module imports no clock: its time is whatever the host writes through the exported __ronto_set_time hook (nanoseconds since the Unix epoch), so call that BEFORE _initialize -- until something does, reading it signals ``` The clock is the line worth having: a program that reads it while loading *is* loadable — on a host that sets it first — so this is a **host obligation**, not a refusal, and nothing but the build can tell you about it in advance. Entropy reads the same way and names `--host-random`. A primitive only an **export** can reach stays quiet, because that one is an ordinary call-time condition your caller can catch. Reachability is static, but it is not blind to the **arguments**: a call carries what the site says about each one — `#'app`, a literal, a `(defvar *app* (make-instance 'ningle:app))` — so a `typecase` branch whose type that value cannot have is not on the load path. That is what keeps every `clack` program quiet about `clackup`'s `(clackup "app.lisp")` file branch, which a reactor never takes. Where the call site says nothing, nothing is ruled out and the line stands. A refusal wrapped in `handler-case` or `ignore-errors` is not reported at all; the program already handles it. --- # FILE: references/guides/wasm-host-boundary.md # WASM Host Boundary (`wasm-export` / `wasm-import`) Two complementary directives declare what crosses the module/host boundary in rontolisp's own type designators. Both work in every WASM output shape (the same source runs on every backend — the directives are no-ops or defun stubs on the interpreter and the JVM). For the typed WIT-driven boundary, see the [WIT contracts guide](wit-contracts.md). ## Exporting Lisp Functions By default a compiled module only exposes its entry point (`_start`). To make an individual Lisp function callable directly from a host (`wasmtime --invoke`, JavaScript, or another module), mark it with the `rontolisp:wasm-export` directive, declaring the WASM-boundary types of its parameters and result: ```lisp (defun fact (n) (if (<= n 1) 1 (* n (fact (- n 1))))) (rontolisp:wasm-export 'fact :params '(:int) :returns :int) ``` ```bash rontolisp fact.lisp -o fact.wasm wasmtime run --invoke fact -W gc fact.wasm 5 ``` ``` 120 ``` The directive itself is the same in every output shape; what changes per shape is the **host contract** of the export — a raw core function on the core-module shapes, a typed component-model export under `--component`. On the interpreter and JVM backends the directive is a no-op (it just returns the named symbol), so the same source runs on every backend. The type designators and their boundary representations are: | Designator | WASM boundary | Notes | | --- | --- | --- | | `:int` | `i32` | full 32-bit signed range | | `:long` | `i64` | full 64-bit signed range on every backend | | `:float` | `f64` | | | `:bool` | `i32` | `0` is `nil`, any non-zero value is `t` | | `:string` | `(ptr, len)` | UTF-8 bytes in linear memory; a component-model `string` under `--component` | | `:s-expr` | `(ptr, len)` | s-expression text (any value except a function); GC value model only | | `:bytes` | `(ptr, len)` argument / `(ptr, cap) -> len` result | an `(unsigned-byte 8)` vector as **raw bytes** — no UTF-8 in either direction; GC core-module shapes only (not `--component`, not `--no-gc`) | `:string` carries a *value* (decoded, allocated per call); `:bytes` carries a *transfer*: the **caller passes the buffer**, the `read(2)` shape. A `:bytes` **result** appends a `(ptr, cap)` pair to the export's parameters — the host reserves `cap` bytes (e.g. with `__ronto_alloc`) and the wrapper copies at most `cap` bytes there — and the single `i32` result is the vector's **full** length, so an undersized buffer is a retry, not a truncation. No per-call allocation is spent on the transfer, which is what keeps a chunked pull loop's memory flat. A side-effecting function can declare a **void** result by omitting `:returns` (or giving it as `nil`, `'()` or `:void`); the wrapper then discards the Lisp return value and has no WASM result. Likewise an omitted, `nil` or `'()` `:params` means no arguments. ```lisp (defun log-it (n) (print n)) (rontolisp:wasm-export 'log-it :params '(:int)) ; (i32) -> () , prints n ``` `:as` renames the export — useful when the host-facing API wants a name that is not an idiomatic Lisp symbol, e.g. camelCase for JavaScript: ```lisp (defun draw-board (w h) (* w h)) (rontolisp:wasm-export 'draw-board :as "drawBoard" :params '(:int :int) :returns :int) ``` Limitations shared by every shape: - Only a top-level `defun` can be exported, the declared parameter count must match its arity, and functions that take or return function values are out of scope. - The exported name defaults to the bare Lisp name (`fact`) and can be renamed with `:as`; how arguments are written depends on the host (`wasmtime --invoke fact module.wasm 5`, `instance.exports.fact(5)`, ...). - The exported function may be a [`rontolisp:async-defun`](../reference/special-forms/rontolisp-async-defun.md): the boundary resolves the future it answers, so the host receives the declared type and never a future. ### Export Modes at a Glance The same directive compiles into four different host contracts depending on the `--no-gc` / `--component` flags: | | GC core module (default / `--no-wasi`) | GC `--component` | `--no-gc` core module | `--no-gc --component` | | --- | --- | --- | --- | --- | | Host requirements | wasm-GC engine (`wasmtime -W gc`, Node 22+, current browsers) | wasmtime 46+ (`-W gc=y`) or a component host with wasm-GC + JSPI (a [browser via jco](wasm-browser.md) loads and computes, but cannot print yet) | **any** WebAssembly engine | any component-model host, **no flags** — including a [browser via jco](wasm-browser.md), with no dependencies at all | | Export shape | raw core function | typed component-model export (WAVE `--invoke`, jco) | raw core function | typed component-model export (WAVE `--invoke`, jco) | | Scalars | `:int`/`:long`/`:float`/`:bool`/void | `:int`/`:long`/`:float`/`:bool`/void | `:int`/`:long`/`:float`/`:bool`/void | `:int`/`:long`/`:float`/`:bool`/void | | `:string` | manual `(ptr,len)` + `__ronto_alloc` | component-model `string` (canonical ABI) | manual `(ptr,len)` + `__ronto_alloc` | component-model `string` (canonical ABI) | | `:s-expr` | manual `(ptr,len)` | component-model `string` (printed text) | not supported | not supported | | `:bytes` | manual `(ptr,len)` / caller-buffered result | not supported (no `list` lift yet) | not supported | not supported | | Function body may use | the full language | the full language | the [non-GC subset](wasm-nogc.md#eligible-subset) | the [non-GC subset](wasm-nogc.md#eligible-subset) | | I/O inside the export | works (real WASI imports; under `--no-wasi` output is discarded, `random` runs on a built-in generator, `getenv`/file lookups answer nothing, the clock is the one the host wrote through `__ronto_set_time` and input traps) | usually works even in a sync export; [`:async t`](wasm-component.md#component-model-function-exports-wasm-export) removes the residual trap risk | `print` only (one `fd_write` import) | `print` only (built-in WASI 0.3 stdout bridge; the exports become async lifts) | | Program top level | runs as `_start` | co-exists as `wasi:cli/run` | `defun` + directives only | `defun` + directives only | | Per-call string memory | host-managed (`__ronto_alloc` + the [arena API](wasm-gc-module.md#reclaiming-the-hosts-buffer-the-arena-api); the Lisp side is the engine's) | freed by the canonical post-return | host-managed (`__ronto_alloc` + the [arena API](wasm-nogc.md#reclaiming-memory-the-arena-api); automatic for scalar returns) | freed by the canonical post-return | | Typical size | ~2 KB ([tree-shaken](../compiling/wasm.md#optimize-tree-shaking); ~100 KB at `--optimize=off`) | ~110 KB | tens of bytes to a few KB | hundreds of bytes to a few KB | Each shape's own guide details how its exports are called, what runs inside them, and what each host must provide: [wasm-GC core module](wasm-gc-module.md), [WASI 0.3 component](wasm-component.md), [--no-gc output and its compact component wrap](wasm-nogc.md). ## Importing Host Functions `rontolisp:wasm-import` is the reverse of `wasm-export`: it declares a function the **host** provides and makes it callable from Lisp under the given name exactly like a top-level `defun` — including `#'name`, `funcall`, `mapcar` and `eval`. `:from` names the import module (default `"env"`), `:as` names the field inside it (default: the Lisp name), and the type designators are the same table as above: ```lisp ; main.lisp (rontolisp:wasm-import 'add :from "host" :params '(:int :int) :returns :int) (defun add10 (n) (add n 10)) (rontolisp:wasm-export 'add10 :params '(:int) :returns :int) ``` In wasmtime, satisfy the imports by preloading another module that exports them — here a host module that is itself written in Lisp, exporting its function under the `:as` alias `add`: ```console $ cat host.lisp (defun host-add (a b) (+ a b)) (rontolisp:wasm-export 'host-add :as "add" :params '(:int :int) :returns :int) $ rontolisp host.lisp -o host.wasm --no-wasi $ rontolisp main.lisp -o main.wasm --no-wasi $ wasmtime run -W gc --preload host=host.wasm --invoke add10 main.wasm 32 42 ``` In a browser (or Node) the import object *is* the module table — one key per `:from` name, one property per `:as` name. This is also the escape hatch for anything the WASM backend does not provide; for example it has no trigonometric built-ins, so borrow JavaScript's: ```lisp (rontolisp:wasm-import 'sin :from "math" :params '(:float) :returns :float) (rontolisp:wasm-import 'cos :from "math" :params '(:float) :returns :float) ``` ```js const imports = { math: { sin: Math.sin, cos: Math.cos } }; const { instance } = await WebAssembly.instantiate(bytes, imports); ``` The [WebGL triangle example](https://github.com/making/rontolisp/tree/develop/examples/browser/webgl-triangle) is the hello world of this pattern: ten imported functions, no exports, and a colored triangle drawn entirely from Lisp. The [WebGL cube example](https://github.com/making/rontolisp/tree/develop/examples/browser/webgl-cube) adds 3D: the perspective and rotation matrices are computed in Lisp every frame. The [WebGL galaxy example](https://github.com/making/rontolisp/tree/develop/examples/browser/webgl-galaxy) is the same idea grown into a complete browser program: the entire WebGL pipeline is driven from Lisp — the GLSL shaders live in the Lisp source, and Lisp compiles, links, buffers and issues every draw call through 32 imported host functions, while JavaScript supplies only one-line bindings over a handle table -- generated from the [WIT](wit-contracts.md#importing-a-wit-interface-wit-import) that declares the boundary. Boundary details beyond the scalar types: - A `:string`/`:s-expr` **argument** reaches the host as a `(ptr, len)` pair into the module's exported `memory` (an `:s-expr` argument is printed to readable text first). - A `:string` **result** must be written into linear memory by the host — reserve the buffer with the exported `__ronto_alloc`, then return the `(ptr, len)` pair (a two-element array in JavaScript). - An `:s-expr` **result** is parsed with the embedded reader, so the host can hand back a whole list structure as text. - A `:bytes` **argument** is an `(unsigned-byte 8)` vector staged as a raw `(ptr, len)` pair — no UTF-8 encode, so arbitrary binary crosses exactly. - A `:bytes` **result** is caller-buffered: the Lisp signature gains one trailing parameter, the `(unsigned-byte 8)` vector to receive into, and the host is called with a trailing `(ptr, cap)` pair — *write up to `cap` bytes at `ptr`, return the full length `n`*. The call answers `n` (an `n` above the buffer's length means "retry with a bigger buffer"), and the wrapper's staging is popped on return, so a pull loop over one reused buffer keeps linear memory flat. - An **asynchronous** host function — a `WebAssembly.Suspending`-wrapped import under JSPI — is declared with `:async t`: the call then returns a future that `rontolisp:await` resolves, the build prints the host's obligations (`Suspending` on the import, `promising` on the exports that can reach it, serialised calls — a re-entered export refuses with a trap instead of corrupting both calls, unless the module was compiled [`--reentrant`](#overlapping-calls---reentrant)), and a call reachable from a top-level form of a `--no-wasi` module is a compile error (`_initialize` cannot suspend). The [reference page](../reference/functions/rontolisp-wasm-import.md) has the full contract. Limitations: - Default (wasm-GC) Preview 1 output only: `--component` and `--no-gc` reject the directive with an error. - On the interpreter and JVM backends the directive defines a stub that signals an error when called, so a shared source still loads everywhere, but actually calling an import needs the WASM host. - Imported functions have the same 10-parameter arity limit as other functions under the wasm-GC value model. - Instantiating the module requires every declared import to be provided: `wasmtime run` needs a `--preload =.wasm` per import module name, and a JavaScript host passes an import object. ## Choosing the Body Boundary (--host-boundary) An HTTP reactor — a `--no-wasi` module whose entry point a host calls, which is what [`clack:clackup`](clack.md) and [`rontolisp:http-handler`](../reference/functions/rontolisp-http-handler.md) compile to there — speaks one JSON envelope in each direction. What `--host-boundary` decides is whether a **body** rides inside that envelope or crosses beside it, and it changes what the **module imports**, so it is a flag of its own rather than a value on `--emit-js-glue`. | | `envelope` (default) | `streaming` | | --- | --- | --- | | Request body | the envelope's `"body"` key | `env.readRequestBody(ptr, cap) -> i32`, a chunk per call | | Response body | the head's `"body"` key | `env.writeResponseBody(ptr, len)`, a chunk per call | | `rontolisp:fetch` reply body (`--host-fetch`) | the reply head's `"body"` key | `env.readResponseBody(ptr, cap) -> i32` | | Host-side state | none | a cursor per reading import | | Binary body | does NOT survive — `ff fe 41` arrives as `ef bf bd ef bf bd 41` | crosses exactly | | Large body | linear memory grows with it | stays flat | | Streamed upstream reply | buffered, then forwarded | forwarded chunk at a time | | Generated host half | `instantiate`, `defaultHost()` and `worker(module)` — the same on both | ```console $ rontolisp worker.lisp -o worker.wasm --no-wasi --host-fetch $ wasm-tools print worker.wasm | grep -oE '\(import "[^"]+" "[^"]+"' (import "env" "fetch" ``` **`envelope` is the default, and it is the one to want.** A body that is a *document* — a Worker that reads one JSON request and answers one JSON reply — pays a copy nobody can measure and gets back a boundary with no host-side state in it, which is where the bugs on this surface have all been. Ask for `--host-boundary=streaming` when one of these is true: - **a body is BINARY** — an image, a file, protobuf, anything already compressed. The envelope carries a body as JSON text, so bytes that are not valid UTF-8 do not survive: `ff fe 41` arrives as the seven bytes `ef bf bd ef bf bd 41`, two replacement characters where two octets were, with the `content-length` beside it still saying three. Nothing reports it. - **a body is LARGE** — the envelope puts it in linear memory whole, so memory grows with the body; the split reads through one reused buffer and stays flat however big it gets. - **you are relaying an upstream reply** — the split forwards it a chunk at a time instead of holding the whole thing first. Neither shape is a subset of the other, and the module sizes land within about 1% of each other either way round, so this is not a size decision either. It is not an ergonomics decision either: `--emit-js-glue` writes the host half of both, so the JavaScript is three lines whichever you pick. **The default moved here, and a rebuild is how you feel it.** Before this, every `--no-wasi` reactor took the bodies out of the envelope; a module rebuilt without the flag now keeps them in it, which is a real regression for the three cases above and nothing at all for everything else. Add `--host-boundary=streaming` and the module is byte-for-byte what it was. `--host-boundary` needs `--no-wasi` and a `.wasm` output, without `--component` or `--no-gc`: those two are in band already (a component's host functions cross the canonical ABI, and `--no-gc` imports nothing at all), and so is a plain WASI command module, whose host is `wasmtime run` and satisfies no `env.*` import. A hand-written reactor — one that spells out its own envelope adapter instead of going through `clack:clackup` — follows the build with the `rontolisp-body-imports` reader feature, which is present exactly where those imports are: ```lisp #+rontolisp-body-imports (rontolisp:wasm-import '%read-request-body :from "env" :as "readRequestBody" :params '() :returns :bytes :async t) ``` ## Generating the Host Glue (--emit-js-glue) Everything above is derived from a declaration, so the JavaScript half can be too. `--emit-js-glue` writes it next to the module (`out.wasm` -> `out.js`): the import object, the `(ptr, len)` staging in both directions, the `__ronto_alloc` bracket around a call, the `WebAssembly.Suspending` wrappers, the `WebAssembly.promising` entry for exactly the exports the build lists, and the one-call-at-a-time queue a module that can suspend needs. ```console $ rontolisp worker.lisp -o worker.wasm --no-wasi --host-fetch --emit-js-glue $ ls worker.* worker.js worker.lisp worker.wasm ``` The generated file asks for the one thing a declaration cannot state: what each host function *does*. `host` is a plain function per import, keyed by import module and field, taking and answering ordinary JavaScript values — never a `(ptr, len)` pair: ```js import { instantiate, suspending } from "./worker.js"; const lisp = instantiate(module, { env: { fetch: suspending(async (request) => hostFetch(request)), readResponseBody: suspending(async () => nextChunk()), readRequestBody: () => take(requestBody), writeResponseBody: (chunk) => chunks.push(chunk), }, }); const reply = await lisp.handleRequest(head); ``` A chunk source must eventually answer `null`, or the module pulls the same octets forever — `take()` above hands the body over once and then reports the end. The glue holds whatever did not fit and drops it at the next call into the module; a host whose source moves *inside* one call (a new upstream reply, say) drops it with `lisp.drop("env.readResponseBody")`, since only that side knows. `suspending()` is how a host says which of its entries answer a promise, and it is per entry because the wrapper is not free: an import that answers *synchronously* through one still parks the stack and returns to the event loop. Mark one and the file switches into its JSPI shape — the marked imports are wrapped, the entry points the build listed are entered through `promising`, and every call rides one promise chain. Mark none and the same file drives a synchronous host, where an entry point answers a value rather than a promise. A callback that answers a promise without being marked is reported by name rather than handing the module a `Promise` where an `i32` was due. Host state belonging to ONE call — what the module pulls during it, what the call leaves behind — is set inside that same critical section, because a suspended call returns to the event loop and the next request would otherwise move it: ```js const reply = await lisp.serially(async (entry) => { requestBody = bytes; chunks = []; return entry.handleRequest(head); }); ``` An import a declaration alone would miss is written too: under `--host-random` the entropy source is *implemented* rather than asked for, since preview1 fixes what `random_get(buf, len)` does. And a `:bytes` result is answered with chunks (`null` ends them), never with the module's buffer: the generated cursor keeps whatever did not fit, so which source the chunks come from — a `ReadableStream`, a `Uint8Array` — is all a host is left to decide. **Where the transport already fixed a host function, the file writes that too.** Two halves of a reactor's boundary are not the program's choice at all, so the generated file exports them: `defaultHost()`, the `env.fetch` half `--host-fetch` fixes in both directions, and `worker(module)`, which maps a `Request` onto the envelope and a `Response` off it. That holds on **either** [boundary](#choosing-the-body-boundary---host-boundary): where a body leaves the envelope, the reader it comes from is the `Request` `worker()` is already holding and the `Response` it is already building, so the body imports are written too. A Worker is then three lines: ```js import module from "./worker.wasm"; import { worker } from "./worker.js"; export default worker(module); ``` Both are defaults, not replacements. `worker(module, options)` takes `host` — import entries laid over the derived ones one at a time — and `remoteAddr`, a `(request, env, ctx) => string` for the envelope's optional client address, which is the one thing a runtime-neutral file may not guess (on Cloudflare it is `(r) => r.headers.get("cf-connecting-ip")`). What is NOT written is an import the program declared itself: `instantiate` still names it, and the sketch at the top of the generated file then says `worker(module, { host })` instead. The flag needs `--no-wasi` and a `.wasm` output: a component is instantiated through its own bindings generator, and a `--no-gc` module imports nothing, so `new WebAssembly.Instance(module, {})` is already the whole of its glue. Nine worked examples on both boundaries — every reactor under [examples/cloudflare-workers](https://github.com/making/rontolisp/tree/develop/examples/cloudflare-workers) but one: `src/worker.js` is generated and checked in, and `src/index.js` is the three lines above. [httpbin](https://github.com/making/rontolisp/tree/develop/examples/cloudflare-workers/httpbin) is the exception, and says so: it declares its `rontolisp:wasm-export` by hand, and only the *synthesized* bridge is recognised as the envelope's own entry point, so no `worker()` is written for it and its host stays hand-written. ## Overlapping Calls (--reentrant) A module that can suspend refuses a second call by default: nothing in it owns its state per call, so every export wrapper carries a re-entry guard and the build's obligation line says *serialise calls*. That is correct, and it costs the whole width of an I/O-bound workload — eight concurrent upstream round trips through one serialised instance take eight round trips. One instance per in-flight call avoids the queue but pays an instantiation per call and a GC heap per instance. `--reentrant` is the opt-in that makes overlap sound on **one** instance: the module then owns its per-call state, the guard is dropped, and a JSPI host may start a call while another is parked. What overlaps is the parked time — one stack still runs at a time — so this buys I/O overlap, never CPU parallelism. What moves, and what a host owes for it: - Every dynamically bound special variable lives in a per-call task record instead of the shared module global, so two overlapped calls binding the same variable each read their own binding back. - Linear-memory staging that must survive a park moves off the scratch stack into recycled park blocks (`__ronto_park_alloc` / `__ronto_park_free`, both exported). Three rules follow: a `:string`/`:s-expr` **export result**'s `(ptr, len)` is a park block the *reader* frees after decoding; a `:string`/`:s-expr` **import result** must be written into a park block, which the module frees; and a `:bytes` receive buffer a host passes into an export must be a park block too. - An arena bracket (`__ronto_alloc_mark` / `__ronto_alloc_reset`) around an entry call is popped *synchronously* the moment the call starts — the arguments are consumed at entry — and the reset never goes below a live park block. [`--emit-js-glue`](#generating-the-host-glue---emit-js-glue) writes all of this and drops the queue, so a generated host needs nothing by hand; the build's obligation lines state the same rules for a hand-written one. The flag requires a program that can suspend (an `:async t` import, or `--host-fetch` with `rontolisp:fetch` used) and a `--no-wasi` core module. It composes with [`--host-boundary=streaming`](#choosing-the-body-boundary---host-boundary): under `--reentrant` every body import leads with an `:int` id — the request envelope names its call with a `"call-id"` key, a fetch reply names its body with `"body-id"` — so each pull and push says what it belongs to instead of sharing one host-side cursor. The generated glue mints the ids and keys its per-call state by them; a body import declared *without* the leading id is refused under the flag. It cannot be combined with `--dynamic`. Reach for it when a workload is I/O-bound *and* cannot afford an instance per request: measured on the envelope Worker shape, eight concurrent 100 ms upstream round trips answer in about 125 ms on one instance, against about 800 ms serialised. --- # FILE: references/guides/wasm-nogc.md # WASM Non-GC Output (`--no-gc`) Every GC-value-model output — even an optimized reactor — needs a **wasm-GC capable** runtime, because every value is a GC heap type (`i31ref`, the float struct, `(ref eq)`). Add `--no-gc` to emit a plain **MVP** module instead: no rec group, no `struct`/`array`/`i31` type, no `eqref` and no import (a plain linear memory is added only when the program uses strings — see [below](#strings) — and the single `fd_write` import only when it [prints](#printing-print--princ--terpri)). A print-free module instantiates with no import object and runs on any MVP-class runtime with **no `-W gc`**: ```bash rontolisp fact.lisp --no-gc -o fact.wasm wasmtime run --invoke fact fact.wasm 5 # => 120, ~108 bytes, no -W gc needed ``` It achieves this by lowering each value directly onto an unboxed wasm scalar, plus a small linear-memory representation for strings — so the eligible subset is a restriction of the language, not a different one. The program shape is also restricted: the top level may contain **only** `defun`s and `rontolisp:wasm-export` directives (a pure-compute reactor — there is no `_start`), and the boundary designators are `:int`, `:long`, `:float`, `:bool`, `:string` (and `:void`/omitted); `:s-expr` is **not** supported — it would need the cons/reader/printer runtime this backend deliberately omits. Numeric vector kernels (the [`vec:` package](simd-acceleration.md)) work under `--no-gc` too, lowered to plain scalar loops by default — so a vector program keeps the "runs on any MVP runtime" property above. Add [`--simd`](../compiling/wasm.md#simd-acceleration---simd) to lower those kernels to native WebAssembly SIMD (`v128`) instead, which then needs a runtime with the SIMD proposal (on by default in wasmtime). ## Eligible subset A function is eligible only if its **entire transitive call graph** stays inside this subset: - numbers and booleans: arithmetic (`+ - * / mod rem 1+ 1- abs min max sqrt`), the integer bitwise operators (`logand logior logxor lognot ash`), comparison and predicates (`= < <= > >= not zerop plusp minusp evenp oddp`); - control and binding: `if`/`when`/`unless`/`cond`/`progn`/`let`/`let*`, recursion and calls to other eligible functions; - iteration and local mutation: `dotimes`/`do`/`do*` and the underlying `while`/`setq`/`return`, with a let/`do`-bound variable freely reassigned; `loop` is eligible only for its non-consing clauses (numeric `for`, `sum`/`count`/`maximize`/`minimize`, `repeat`/`while`/`until`/`do`/ `return`) — its `collect`/`append`/`nconc` and `for ... in`/`on` clauses allocate lists and are not; - float/int conversions: `float truncate floor ceiling round`; - strings and characters: string literals, character literals, `(concatenate 'string ...)`, `length`, `subseq`, `string=`, `char`, `char-code`/`code-char`, `char=` and `princ-to-string` (of integers, floats and strings). There is no separate character type: a character is represented by its code point, so the portable idioms `(char= (char s i) #\x)` and `(char-code (char s i))` behave exactly like the other backends, while a bare `(char s i)` crossing an `:int` boundary shows the code; - printing: `print`, `princ` and `terpri` (without the optional stream argument) — see [below](#printing-print--princ--terpri); - memory reclamation: [`rontolisp:with-arena`](#reclaiming-from-lisp-rontolispwith-arena). Anything else that would allocate a heap object (cons/list, symbols, vectors, hash tables, `eval`/`apply`, I/O, `dolist`/list iteration, a free variable or assignment to a global, a lambda-list keyword such as `&optional`/`&rest`/`&key` — the rest list is a cons) makes the function ineligible. Rather than miscompile silently, that is a **compile error** naming the offending operation, so the boundary stays explicit. ## Numeric model Each value's wasm type is chosen by static type inference: integers use `i64`, floats use `f64`. Types are inferred with a fixpoint over the call graph seeded by the export boundary designators, and where an integer and a float meet (e.g. `(* 3.14 n)`) the integer is promoted to `f64`. Using `i64` makes integer arithmetic exact to 2^63 (the GC backend goes further still, promoting to big integers of any magnitude) — far wider than what an all-`f64` lowering (exact only to 2^53) could offer; for example `a*a - (a-1)*(a+1)` stays exactly `1` even when the intermediates exceed 2^53. Inference also widens automatically: a let/`do`-bound variable takes the join of its initializer and every value assigned to it, so an integer accumulator summed with floats becomes an `f64`: ```lisp (defun sum-squares (n) ; sum of i*i for i in 0..n-1, as a float (let ((acc 0)) ; acc starts as an integer 0 ... (dotimes (i n) (setq acc (+ acc (* (float i) (float i))))) ; ... and widens to f64 here acc)) (sum-squares 5) ; => 30.0 ``` Under `--no-gc` this infers `acc` (and the return value) as `f64` while the loop counter `i` stays `i64`. There is no rational type, so two things differ from full Common Lisp and from the GC backend: `/` is floating-point division (no `1/3` ratios), and a value is false in a boolean context exactly when it is zero (Common Lisp treats only `nil` as false). The **boundary** designators stay host-width — `:int`/`:bool` cross as a 32-bit `i32` (as in the GC backend), so a returned value outside the 32-bit range wraps; the wide `i64` range applies only to the internal computation. When a parameter or result can exceed the 32-bit range, declare it `:long` — it crosses the boundary as `i64` with no `wrap`/`extend`. For the numeric kernels this mode targets (factorials, math/finance functions, validators) the results match the interpreter and the GC backend. ## Strings A string is an `i32` pointer to a `[length][bytes]` header in linear memory, and `(concatenate 'string ...)` bump-allocates a fresh buffer — so building up a string is just an accumulator loop: ```lisp (defun stars (n) ; an n-character run of '*' (let ((out "")) (dotimes (k n) (setq out (concatenate 'string out "*"))) out)) (stars 5) ; => "*****" ``` Slicing and inspection work on the same representation: `length` reads the header, `subseq` copies a slice into a fresh buffer, `string=` compares content byte-wise, `char` indexes a byte, and `princ-to-string` renders an integer — enough for routing/parsing kernels, not just accumulation: ```lisp (defun describe-int (n) (let ((s (princ-to-string n))) (concatenate 'string s " has " (princ-to-string (length s)) " chars"))) (describe-int -42) ; => "-42 has 3 chars" ``` A module that uses strings gains a (growable) linear memory, and exports that `memory` plus a `__ronto_alloc(size)` bump allocator alongside your functions. A `:string` parameter arrives as a `(ptr, len)` pair the host writes into memory, and a `:string` result is returned the same way — so a string-valued export needs a host that can read/write the exported memory (JavaScript, a small Node script, the browser playground) rather than just `wasmtime --invoke`. The [browser guide](wasm-browser.md#passing-strings-string) walks through the JS side, and [`--no-gc --component`](#compact-component-output---no-gc---component) removes the manual protocol entirely. This is what lets the ASCII-art Mandelbrot renderer run with no wasm-GC: [`examples/console/mandelbrot-nogc.lisp`](https://github.com/making/rontolisp/blob/develop/examples/console/mandelbrot-nogc.lisp) keeps the floating-point escape-time loop but returns the rendered grid as one string instead of printing it: ```console $ rontolisp examples/console/mandelbrot-nogc.lisp --no-gc -o mandelbrot.wasm $ node -e '(async () => { const ex = (await WebAssembly.instantiate( require("fs").readFileSync("mandelbrot.wasm"), {})).instance.exports; const [p, n] = ex.mandelbrot(-2.5, 1.0, -1.2, 1.2, 70, 30, 30); process.stdout.write(Buffer.from(new Uint8Array(ex.memory.buffer, p, n)).toString()); })()' ``` The export's type is not written in that Lisp file at all: a checked-in world ([`mandelbrot_component.wit`](https://github.com/making/rontolisp/blob/develop/examples/console/mandelbrot_component.wit)) declares `export mandelbrot: func(x0: f64, ..., max-iter: s32) -> string;`, and [`rontolisp:wit-export`](wit-contracts.md#implementing-a-wit-world-wit-export) says the program implements it. One directive serves both builds: the core module above is byte-identical to the hand-written `wasm-export` it replaced, and the same source compiles as a component where `wasmtime run --invoke 'mandelbrot(-2.5, 1.0, -1.2, 1.2, 70, 30, 30)'` returns the string with no host memory code and no runtime flags. ## Printing (`print` / `princ` / `terpri`) An exported function can print: `print` (readable form plus a trailing newline, so strings come out quoted), `princ` (display form, no newline) and `terpri` (a newline) work inside the eligible subset, with output byte-identical to the interpreter: ```console $ cat show.lisp (defun show (n) (print n) (print (* 1.5 n)) (print "done")) (rontolisp:wasm-export 'show :params '(:int) :returns :void) $ rontolisp show.lisp --no-gc -o show.wasm $ wasmtime run --invoke show show.wasm 4 4 6.0 "done" ``` Floats print through the same digit-extraction printer as the GC backend, including the IEEE edges (`NaN`, `Infinity`/`-Infinity`, `-0.0`; a magnitude ≥ 2^63 uses the WASM backends' `E`-notation shape). Each `print` of a number renders its text into a transient string that is reclaimed immediately, so a print loop does not grow the heap. Two things to know: - **A printing module has one import.** `print`/`princ`/`terpri` write through a single `wasi_snapshot_preview1.fd_write` import — added **only when the program prints**, so a print-free module keeps zero imports and its exact bytes. Any WASI Preview 1 host provides `fd_write` for free (`wasmtime run`, Node's built-in `node:wasi` module), but a printing module no longer instantiates with an empty `{}` import object the way the [Mandelbrot snippet](#strings) does — a raw JavaScript embedder must supply `{ wasi_snapshot_preview1: { fd_write } }` (or use `node:wasi`). Add `--no-wasi` to trade the output for the imports: the `fd_write` import becomes a built-in sink (printed bytes are discarded, nothing traps), the module keeps zero imports, and — under `--component` — a printing program takes the print-free component shape again (one core module, no imports, sync exports, callable through jco on plain Node). - **Booleans print by literal only.** The value model has no runtime boolean type: `(print t)` / `(print nil)` print `t` / `nil`, but a *computed* boolean such as `(print (> a b))` prints its `0`/`1` integer. The optional stream argument and printing a packed float array are compile errors. ## Reclaiming memory (the arena API) `__ronto_alloc` is a bump allocator that never frees, so a **resident** host — one that keeps a single instance alive and calls it in a loop, allocating a fresh input buffer each time — grows its linear memory without bound. Two mechanisms keep it flat: - **Automatic, for scalar returns.** When an export returns a non-memory scalar (`:int`/`:long`/`:float`/`:bool`/`:void`), its wrapper snapshots the heap top on entry and restores it on exit, so everything the *call* allocates (the internal copy of a `:string` argument, plus any `concatenate`/`subseq`/`princ-to-string` scratch) is reclaimed on return. Nothing to do host-side. - **Manual, for the host's own buffer.** The host allocates its input buffer *before* the call, so it sits below the wrapper's auto-reset mark and is left live. To reclaim it too, the string-using module also exports a matched pair over the same heap pointer: | export | signature | meaning | | --- | --- | --- | | `__ronto_alloc_mark` | `() -> i32` | snapshot the current bump-heap top | | `__ronto_alloc_reset` | `(i32 mark) -> ()` | restore the top to a saved mark | Snapshot **before** allocating the input, restore **after** reading the result, and a resident instance stays perfectly flat no matter how many times it is called: ```bash node -e '(async () => { const ex = (await WebAssembly.instantiate( require("fs").readFileSync("count_vowels.wasm"), {})).instance.exports; const enc = new TextEncoder(); const countVowels = (s) => { const b = enc.encode(s); const mark = ex.__ronto_alloc_mark(); // snapshot BEFORE allocating input const ptr = ex.__ronto_alloc(b.length); new Uint8Array(ex.memory.buffer, ptr, b.length).set(b); const n = ex.count_vowels(ptr, b.length); // scalar result read out here ex.__ronto_alloc_reset(mark); // pop the input + wrapper scratch return n; }; const before = ex.memory.buffer.byteLength; for (let i = 0; i < 100000; i++) countVowels("Hello, World! " + i); console.log(before, "->", ex.memory.buffer.byteLength); // 65536 -> 65536 (flat) })()' ``` The arena is a manual stack, not a garbage collector, so two rules apply: - Only reset to a mark taken **before** everything still live — popping to a mark taken *after* data you still need frees that data. - A `:string`-**returning** export does *not* auto-reset (its result is a live heap pointer). **Read the returned bytes out of memory before calling `__ronto_alloc_reset`** — resetting first frees the string and the next allocation overwrites it. The [`count-vowels` example](https://github.com/making/rontolisp/tree/develop/examples/count-vowels) walks through this recipe with both a Node and an [Endive](https://endive.run) (Java) host. The wasm-GC backend exports the same `__ronto_alloc_mark`/`__ronto_alloc_reset` pair with the same host recipe (see the [wasm-GC arena API](wasm-gc-module.md#reclaiming-the-hosts-buffer-the-arena-api)), but only there does the *host's* buffer need reclaiming — the engine handles everything the Lisp side allocates. The automatic scalar-return reset is `--no-gc`-only: it is sound because nothing a `--no-gc` call allocates can outlive it (no cons, closures, hash tables or global `setq` in the subset). ## Reclaiming from Lisp (`rontolisp:with-arena`) Both mechanisms above fire at the **export boundary** — nothing is freed *within* one call. A loop that allocates each iteration (`concatenate 'string` builds a fresh buffer, `vec:zeros`/`vec:ones` a fresh vector) therefore grows the heap for the duration of the call. [`rontolisp:with-arena`](../reference/macros/rontolisp-with-arena.md) names that reclamation boundary in the source: it snapshots the bump heap pointer, runs its body, and pops everything the body allocated — keeping only the body's own value (a string or packed float array result is copied down to the snapshot point): ```lisp (defun train (epochs n) (let ((acc 0.0)) (dotimes (i epochs) (rontolisp:with-arena () ; everything allocated inside ... (setq acc (+ acc (vec:sum (vec:ones n)))) ; ... is popped here )) acc)) ``` With the arena, a hundred thousand iterations stay within the initial linear memory; without it, the same loop grows by one vector per iteration. The escape contract is the same as `__ronto_alloc_reset`'s: **nothing allocated inside the body may be reachable after it, except the body's own value.** On the interpreter, the JVM backend and the default (wasm-GC) output, `with-arena` is observationally a plain `progn` — a real garbage collector already reclaims — so the same source runs on every backend. ## Compact Component Output (`--no-gc --component`) Add `--component` to wrap the same MVP core module as a **WASM component** whose exports become typed component-model exports, callable through the canonical ABI with WAVE syntax. A print-free core module has zero imports, so the wrap needs no WASI adapter, no shared-memory module and no wasm-GC — the whole component stays in the hundreds of bytes for a small program and runs with **no wasmtime flags at all**: ```bash rontolisp fact.lisp --no-gc --component -o fact.wasm wasmtime run --invoke 'fact(5)' fact.wasm # 120 ``` The typed WIT signature carries each designator under its own WIT name (`:s32` → `s32`, `:u32` → `u32`, … up to `:s64`/`:u64`, which only this backend can lift; `:int` and `:long` are the legacy aliases of `:s32` and `:s64`), plus `:float` → `f64`, `:bool` → `bool`, `:string` → `string`, and an omitted `:returns` → no result. The component also transpiles with jco (`jco transpile`, where the 64-bit types surface as JavaScript BigInt) and runs on any component-model host, with no wasm-GC support required. `:long` is valid here, unlike the GC component path — use it when a value can exceed the 32-bit range, matching the backend's internal `i64` arithmetic: ```lisp ;; cube.lisp (defun cube (n) (* n n n)) (rontolisp:wasm-export 'cube :params '(:long) :returns :long) ``` ```bash rontolisp cube.lisp --no-gc --component -o cube.wasm wasmtime run --invoke 'cube(2000000)' cube.wasm # 8000000000000000000 ``` A `:string` boundary crosses as a real component-model `string` — no manual pointer handling on either side. The host lowers the argument bytes into the module's own memory and reads the result back out through the canonical ABI, and the module frees every per-call allocation afterwards (a canonical *post-return* function pops the bump allocator to its base), so a resident instance stays flat across repeated calls: ```bash rontolisp greet.lisp --no-gc --component -o greet.wasm wasmtime run --invoke 'greet("world")' greet.wasm # "Hello, world" ``` [Printing](#printing-print--princ--terpri) works here too: a program that prints gets a built-in **print micro-adapter** — three tiny fixed core modules that implement the core's single `fd_write` import over WASI 0.3 (`wasi:cli/stdout`'s `write-via-stream` plus the async stream/future built-ins), wired in only when the program prints. WASI 0.3 has no synchronous write, so the exports of a printing program become **async lifts** (the WIT world shows them as `async func`) — which is why the component still runs with zero flags: everything it uses is base component-model async, on by default in wasmtime 46+ (the wasmtime floor for a *printing* component; a print-free one has no imports at all and runs on older hosts too). The print output is byte-identical to the interpreter — with the earlier `show.lisp`: ```bash rontolisp show.lisp --no-gc --component -o show.wasm wasmtime run --invoke 'show(4)' show.wasm # 4 # 6.0 # "done" # () ``` Trade-offs against the plain `--no-gc` output, and current limits: - A component needs a component-model-capable host; the raw core module runs on **any** WebAssembly engine through the plain embedding API. Both outputs stay available — pick per host, and note the component is *not* the default for `--no-gc`. (Without `--component`, a `:string` crosses as the manual `(ptr,len)` core ABI instead.) - The component is a pure reactor: there is no `wasi:cli/run` entry (nothing runs at the top level). Printing inside an export works through the micro-adapter above; every other I/O stays outside the `--no-gc` subset as usual. `:async t` is rejected — a printing program's exports are lifted async automatically, and there is nothing else an export could suspend on. - The export name must be a lower-kebab-case component-model name; for a Lisp name outside that grammar the compiler asks you to rename it with `:as`. - Tree shaking composes: the core module is shaken before the wrap. - [`--emit-wit`](wit-contracts.md#emitting-the-wit-world---emit-wit) composes too, and writes a tiny import-free world of just the typed exports (plus the `wasi:cli/stdout@0.3.0` import — and `async func` export signatures — when the program prints). --- # FILE: references/guides/wit-contracts.md # WIT Contracts (`wit-export` / `wit-import`) Two directives let a program's boundary come straight from a `.wit` file someone else wrote (or that a binding generator produced): **`rontolisp:wit-export`** implements a world, and **`rontolisp:wit-import`** calls an interface. Neither adds a new lowering path — each is a typed front-end for the manual [`wasm-export` / `wasm-import`](wasm-host-boundary.md) machinery, plus per-backend implementations that let the same source run everywhere (typed component-model exports under `--component`, provider callbacks on the interpreter and the JVM, byte-identical Preview 1 imports). ## Implementing a WIT World (`wit-export`) **`rontolisp:wit-export`** takes a world someone else wrote, and has the program **implement** it: ```console // wit/greeter.wit package example:greeter; world greeter { /// Greet someone by name. export greet: func(who: string) -> string; } ``` ```console ;;; greet.lisp -- the directive comes last: on the interpreter it sees only the ;;; functions defined so far. (defun greet (who) (concatenate 'string "Hello, " who "!")) (rontolisp:wit-export "wit/greeter.wit" :world greeter) ``` ```bash rontolisp greet.lisp --component -o greet.wasm wasmtime run -W gc=y --invoke 'greet("world")' greet.wasm # "Hello, world!" ``` There is no `:params '(:string) :returns :string` anywhere — the types come from the world. That is the whole point: hand-written boundary types sit next to a `.wit` that is generated separately, and the two drift until `wasmtime --invoke` fails at run time. With `wit-export` **the WIT is the single source of truth**: - The world is the program's export list, so a hand-written `rontolisp:wasm-export` in the same program is a compile error. - Every export must have a matching `defun` of the right arity, every WIT type must be one the boundary carries (every fixed-width integer `s8` … `u64`, plus `f64`, `bool`, `string`), and an `async func` in the world lifts that export with `:async t` (so an export that does I/O is declared async by the WIT instead of being guessed at). Each mismatch is a compile error naming the WIT file and line: `wit/greeter.wit:5: export 'greet' declares 1 parameter(s), but (defun greet ...) takes 2`. - The contract is checked on **every** backend: a plain `rontolisp greet.lisp` run (or a `-o Greet.class` build) verifies the world and exports nothing, so a drift is caught long before a WASM build. The directive is a front-end for the machinery of the previous sections, not a second export path: it lowers into exactly the `rontolisp:wasm-export` directives a hand-written implementation would carry, so **the emitted component is byte-identical** to that one — on the GC path and under [`--no-gc --component`](wasm-nogc.md#compact-component-output---no-gc---component) alike (a world using `s64`/`u64` works on both: the GC backend carries the 64-bit types through its boxed exact integers). Adding [`--emit-wit`](#emitting-the-wit-world---emit-wit) to the build writes out the component's real type, and its export lines come back the way you wrote them, parameter names included — the WIT's names ride through into the component's function type. (A hand-written export names its parameters `p0`, `p1`, ... unless it declares them itself with `:param-names '(who)`.) ```bash rontolisp greet.lisp --component -o greet.wasm --emit-wit # writes greet.wit ``` ```text export greet: func(who: string) -> string; ``` That line is a fixpoint, though, not a verdict: it is derived *from* the world, so it cannot contradict it. The reason to emit anyway is the rest of the file — the `wasi:*` imports and the `wasi:cli/run` export that the world says nothing about, and that a host has to supply. `greet.wit` is 149 lines around that one export. Two differences from the input are deliberate: the `///` doc comments are gone, because a component's type does not store them (`wasm-tools` cannot recover them either), and the emitted world is always `package root:component; world root`. That is what a component's type *is*. ### Exporting an interface Most WIT worlds export an **interface** rather than a bare function — the idiomatic shape keeps the interface definition separate from the world: ```console // wit/adder.wit package docs:adder@0.1.0; interface add { add: func(x: s32, y: s32) -> s32; } world adder { export add; } ``` `wit-export` implements this the same way: it resolves `export add;` to the `add` interface defined in the file and checks each of its functions against the program. The component then genuinely exports the interface, so `wasm-tools component wit` and a host see `docs:adder/add`, not a flattened top-level function: ```console ;;; adder.lisp (defun add (x y) (+ x y)) (rontolisp:wit-export "wit/adder.wit" :world adder) ``` ```bash rontolisp adder.lisp --component -o adder.wasm wasmtime run -W gc=y --invoke 'add(20, 22)' adder.wasm # 42 ``` An inline `export ops: interface { ... }` works the same way, keyed by its plain name, and `--emit-wit` reconstructs the interface — `export docs:adder/add@0.1.0;` plus its `interface` definition — byte-for-byte the way `wasm-tools` prints it. Current limitations: - Only the world's **export** side is bound. `import` items are ignored (a component's WASI imports come from the build, not from the world — [`--emit-wit`](#emitting-the-wit-world---emit-wit) is how you see them), and an inline `import name: func(...)` is rejected rather than silently dropped; the functions a program calls are bound from an interface with [`wit-import`](#importing-a-wit-interface-wit-import) (or declared by hand with `rontolisp:wasm-import`). - A world exports freestanding functions or an interface **defined in the same file** (above); an export naming an interface the file does not define — a bare `wasi:*` reference — is an error, and a `rontolisp:http-handler` program cannot use a world at all (a serve-mode component's only export is `wasi:http/handler@0.3.0`). - `:s-expr` has no WIT spelling, so an export passing an arbitrary s-expression across the boundary still needs a hand-written `rontolisp:wasm-export`. - On the interpreter the directive is evaluated in order and sees only the functions defined so far, so put it at the end of the file. ## Scaffolding an Implementation (`--scaffold-wit`) `--scaffold-wit` is the answer to "someone handed me a `.wit`, now what": it generates the skeleton of an implementation instead of compiling one. ```bash rontolisp --scaffold-wit wit/greeter.wit -o greet.lisp # no -o: print to stdout ``` ```console ;;;; Implementation of the WIT world 'greeter' (wit/greeter.wit). ;;;; ;;;; The world is the contract: the compiler checks every defun below against ;;;; it, so a renamed export, a changed arity or a changed type is a compile ;;;; error rather than a runtime surprise. Fill in the bodies; each one signals ;;;; until you do. ;;; Greet someone by name. ;;; WIT: greet: func(who: string) -> string (defun greet (who) (error "greet is not implemented yet")) (rontolisp:wit-export "wit/greeter.wit" :world greeter) ``` The parameters are named as the WIT names them, each export's WIT signature is carried above its stub as the contract it must satisfy, and the `///` doc comments become `;;;` comments. The stubs signal at **run** time, not compile time, so the generated file compiles unchanged and the exports can be filled in one at a time. A world exporting an interface scaffolds one stub per interface function, so the separated shape above yields the same skeleton. Add `--world NAME` when the `.wit` declares several worlds. ## Emitting the WIT World (`--emit-wit`) Add `--emit-wit` to any `--component` build to also write the component's WIT description next to the `.wasm` output — `-o sumsq.wasm --emit-wit` writes `sumsq.wit`: ```bash rontolisp sumsq.lisp --component -o sumsq.wasm --emit-wit ``` ```text // sumsq.wit (the world; the file also carries the referenced package // definitions, so it is self-contained and parseable on its own) package root:component; world root { import wasi:cli/types@0.3.0; import wasi:cli/stdout@0.3.0; // ... the WASI imports of the build's blob variant ... export wasi:cli/run@0.3.0; export sumsquared: func(p0: s32, p1: s32) -> s32; } ``` The text matches what `wasm-tools component wit sumsq.wasm` prints for the same bytes, so it is exactly the component's real surface — but nothing needs to introspect the binary anymore: hand the `.wit` straight to a binding generator. For example, jco generates TypeScript typings from it without touching the `.wasm`: ```bash npx @bytecodealliance/jco types sumsq.wit -o types/ # types/sumsq.d.ts: export function sumsquared(p0: number, p1: number): number; ``` The world's imports follow the build variant (plain, `rontolisp:fetch`, `rontolisp:tcp-*`, or `rontolisp:http-handler`; with [`--no-gc --component`](wasm-nogc.md#compact-component-output---no-gc---component) the world is import-free, or carries the `wasi:cli/stdout@0.3.0` import — and `async func` exports — when the program prints) and, since [tree shaking](../compiling/wasm.md#optimize-tree-shaking) is on unless you pass `--optimize=off`, the part of that variant the program can actually reach: the world above is the two `wasi:cli` imports it really needs, not the build variant's full fixed surface. An `:async t` export is rendered as `async func`, and a `rontolisp:http-handler` build exports `wasi:http/handler@0.3.0` instead of `run`. `--emit-wit` without `--component` is a compile error — a core module has no WIT-level surface to describe. ### What `--emit-wit` Is For It answers different questions depending on where the export list came from. **A program without a world** — exports written by hand with `rontolisp:wasm-export`, or an `:s-expr` export, which has no WIT spelling at all — has no `.wit` anywhere. `--emit-wit` is the only way to get one, exactly as above. **A program with a world** ([`wit-export`](#implementing-a-wit-world-wit-export)) has already written its exports down. What it has not written down is the component's **imports**, and that is the larger half: `wit-export` reads only the world's `export` items, because a component's WASI surface comes from the build, not from the world. The 6-line `wit/greeter.wit` of the [previous section](#implementing-a-wit-world-wit-export) compiles to a component whose real type is **26 lines** — two `wasi:cli` imports and `export wasi:cli/run@0.3.0` wrapped around the one `greet` you declared, which is as narrow as [tree shaking](../compiling/wasm.md#optimize-tree-shaking) can make it. (Build the same source with `--optimize=off` and you get the build variant's full fixed surface instead — **174 lines**, eleven `wasi:*` imports — which is one more reason to read the emitted world rather than assume it.) Let that same `greet` call `rontolisp:fetch` and the build silently adds four more imports (`wasi:http/types` and `wasi:http/client`, plus the `wasi:filesystem/types` and `wasi:cli/stderr` the HTTP path can reach), for **190 lines**; `rontolisp:tcp-*` pulls in `wasi:sockets` the same way. Short of installing `wasm-tools` and introspecting the binary, `--emit-wit` is the only way to see what you actually built — and it is precisely what a host, or `jco`, needs in order to *supply* those imports. What `--emit-wit` is **not** — for a program that has a world — is a drift check on that program. The export lines are a fixpoint by construction: the world produces the `rontolisp:wasm-export` directives, those produce the component's function types, and those are what is printed back out, over a boundary type set (every fixed-width integer, plus `f64`, `bool`, `string`) that maps one-to-one in both directions. They cannot come out disagreeing with the world you handed in. Re-emitting the `.wit` and diffing it in CI is therefore a regression check on *rontolisp's* type mapping — cheap, and worth keeping — not a check on your source. The thing that catches a drifted program is `wit-export` itself, and it already runs on every backend, including a plain interpreter run. This is transitional: once a world can also declare the imports a program binds, the emitted WIT becomes a genuinely two-sided contract. ## Importing a WIT Interface (`wit-import`) `wit-export` is the export side of a WIT contract. **`rontolisp:wit-import`** is the import side: it declares that the program **calls** a WIT interface, and binds every function that interface declares as an ordinary Lisp function — its name, its lambda list and its types all taken from the `.wit`. It is a compile-time directive that lowers into forms that already exist, and *what* it lowers to depends on the backend. That is the whole point: **one WIT, a different implementation per backend, zero source changes.** ```console // wit/host.wit package example:host@0.1.0; interface math { /// Add two integers on the host. add-ints: func(a: s32, b: s32) -> s32; } ``` ```console ;;; main.lisp -- the directive comes FIRST: it defines the functions the rest of ;;; the file calls. (rontolisp:wit-import "wit/host.wit" :interface "example:host/math@0.1.0") (defun add10 (n) (add-ints n 10)) (rontolisp:wasm-export 'add10 :params '(:int) :returns :int) ``` On Preview 1 WASM each WIT function becomes a [`rontolisp:wasm-import`](wasm-host-boundary.md#importing-host-functions): the import **module** is the interface's bare name (`math`, overridable with `:from`) and the import **field** is the WIT label in camelCase (`addInts` — the JavaScript convention, and what `jco` produces; `:field-style :kebab` keeps the label verbatim). So the host is satisfied exactly as before — here by another Lisp module that exports the function under that field name: ```console ;;; host.lisp (defun host-add (a b) (+ a b)) (rontolisp:wasm-export 'host-add :as "addInts" :params '(:int :int) :returns :int) ``` ```bash rontolisp host.lisp -o host.wasm --no-wasi rontolisp main.lisp -o main.wasm --no-wasi wasmtime run -W gc --preload math=host.wasm --invoke add10 main.wasm 32 # 42 ``` The module is **byte-identical** to the one the hand-written `(rontolisp:wasm-import 'add-ints :from "math" :as "addInts" :params '(:int :int) :returns :int)` produces — the directive is a typed front-end for that machinery, not a second import path — and the [tree shaker](../compiling/wasm.md#optimize-tree-shaking) still shakes out the imports the program never calls, so binding a 29-function interface and using three of them costs nothing. ### Providers: the same source on the interpreter and the JVM There is no WASM host on the interpreter or the JVM, so there each WIT function becomes an ordinary `defun` that dispatches through the interface's **provider**: a Lisp callable taking the bound function's Lisp member name (a string) followed by that function's arguments. [`rontolisp:wit-provide`](../reference/functions/rontolisp-wit-provide.md) binds one — and rontolisp ships **no provider for any interface**. It knows the provider mechanism; it does not know what `wasi:keyvalue` is. Implementing a WIT interface is ordinary Lisp code: ```console ;;; counter.lisp -- wasi:keyvalue, against a store written in Lisp. (rontolisp:wit-import "wit/store.wit" :interface "wasi:keyvalue/store@0.2.0" :package kv) (defvar *rows* (make-hash-table :test #'equal)) (defun my-store (member &rest args) (cond ((string= member "open") 1) ; the bucket handle: any integer ((string= member "bucket-set") (setf (gethash (nth 1 args) *rows*) (nth 2 args)) nil) ((string= member "bucket-get") (gethash (nth 1 args) *rows*)) (t (error 'rontolisp:wit-error :payload (list :other member))))) (rontolisp:wit-provide "wasi:keyvalue/store@0.2.0" #'my-store) (defvar *bucket* (kv:open "counts")) (kv:bucket-set *bucket* "visits" "41") (print (kv:bucket-get *bucket* "visits")) ; "41" ``` `:package kv` synthesizes the `defpackage` that exports the bindings, a WIT `resource` method takes its handle as the first argument (`bucket.get` becomes `(kv:bucket-get b "visits")`), and each binding is an ordinary function, so `#'kv:bucket-get`, `funcall` and `mapcar` work on it. Calling one with no provider bound signals `rontolisp:wit-error` — `No provider is bound for the WIT interface wasi:keyvalue/store@0.2.0 -- bind one with rontolisp:wit-provide` — rather than reaching some default. The payoff is that a provider is *just a function*: swap the hash table above for a real store — Redis, a file, a JDBC connection — and the code calling `(kv:bucket-set b "visits" "41")` does not change. The [`wit/keyvalue` example](https://github.com/making/rontolisp/tree/develop/examples/wit/keyvalue) runs one page-view counter over three of them (a portable Lisp store, a `java.util.LinkedHashMap` one on the JVM, and wasmtime's own `wasi:keyvalue` implementation as a component) with identical output. Compile the same source to WASM instead and the **host** implements the interface: a top-level `rontolisp:wit-provide` is then **dropped** (the host is the provider), rather than being an error, precisely so that one source runs everywhere. A WIT `result` is not a value: the ok arm is the return value, and the error arm signals the `rontolisp:wit-error` condition carrying the mapped `E`, which `handler-case` catches and `rontolisp:wit-error-payload` unpacks. ### Components: the host is the provider (`--component`) Compile the very same source with `--component` and the interface becomes a real component-model **import**: the component declares it in its type, and every bound function is `canon lower`ed into the core module, so the calls go out through the canonical ABI. There is no provider inside the component at all — **the host is the provider**, and any host (or any other component) that exports the interface satisfies it. wasmtime implements `wasi:keyvalue`, so a program written against it runs with no adapter and no rewriting: ```bash rontolisp counter.lisp -o counter.wasm --component wasmtime run -W gc=y -W exceptions=y \ -S keyvalue=y counter.wasm ``` The canonical ABI is what marshals the rich types, so the component boundary carries much more than the Preview 1 one: a `result` (whose error arm arrives as a `rontolisp:wit-error` condition, caught with `handler-case`), an `option`, a `record` (a keyword plist), a `variant`, an `enum`, a `tuple`, a `list`, a `list`, a `string`, a `bool`, and `resource` handles. Everything but `list` crosses **in both directions**, and an argument takes exactly the shape the same type takes as a return value — so a value one call hands you goes straight into the next: ```console ;;; wasi:http/types, imported and called: a variant argument, whose `other` case ;;; carries a string (http:outgoing-request-set-method req :post) (http:outgoing-request-set-method req '(:other . "PATCH")) (http:outgoing-request-method req) ; => (:other . "PATCH") ;;; wasi:sockets/types: an enum argument, then a variant whose case payload is a ;;; record (a keyword plist) carrying a tuple (a positional list) (let ((s (sock:tcp-socket-create :ipv4))) (sock:tcp-socket-bind s '(:ipv4 :port 0 :address (127 0 0 1)))) ``` The one shape that still does not lower is a **`list` argument** (`list` does, as a byte string): an argument is flattened, and a list would have to be written into linear memory as a canonical array instead. It is a compile error naming the WIT line, and `flags` does not cross in either direction yet. One interface a component **cannot** bind is one it already imports for its own WASI surface — and that surface grows with what the program uses (`rontolisp:fetch` pulls in `wasi:http/types` and `wasi:http/client`, the `rontolisp:tcp-*` built-ins pull in `wasi:sockets/types`). A component cannot import the same interface twice, so that is a compile error too: drive the interface through the WIT binding *instead of* the built-in, not alongside it. A component imports **only the functions the program actually calls** (there is no core tree shaker on this path, so unused interface members are dropped from the import itself; `--no-prune` keeps them all), and [`--emit-wit`](#emitting-the-wit-world---emit-wit) writes that pruned interface into the component's world — where `wasm-tools component wit` agrees with it, byte for byte. A component that imports nothing is byte-identical to one built before any of this existed. That is also how components **compose**: a component that imports `wasi:keyvalue/store` plugs into any component that exports it, in any language, with [`wac`](https://github.com/bytecodealliance/wac). The host does not have to be a runtime built-in. ### A served handler with a real store A **served** component ([`rontolisp:http-handler`](http-handler.md) + `--component`) imports user interfaces the same way: its imports are not only the fixed `wasi:http` surface it exports through. That is what lets a handler keep state at all — a `wasi:http` host instantiates the component **afresh for every request**, so a global hash table reads back empty every time, while a store lives outside it: ```bash rontolisp page-hits-server.lisp -o server.wasm --component wasmtime serve -W gc=y -W exceptions=y -S keyvalue=y server.wasm curl http://127.0.0.1:8080/index ``` Whether the counts then *survive* is the host's business, not the component's: wasmtime's built-in key-value provider is an in-memory store it rebuilds per instance (so, under `wasmtime serve`, per request), while a host that links an out-of-process provider keeps them — on wasmCloud (`wash dev`) the same component counts 1, 2, 3. The interfaces a served component may *not* bind are the ones its own surface already imports: `wasi:http/types`, `wasi:http/client`, `wasi:cli/types`, `wasi:cli/stdout`, `wasi:cli/stderr`, `wasi:clocks/*` and `wasi:random/random`. The full example is [`examples/wit/keyvalue`](https://github.com/making/rontolisp/tree/develop/examples/wit/keyvalue). ### Releasing a resource (`-drop`) A handle has to be given back, and **WIT declares no function for giving it back**: releasing a resource is a canonical built-in of the component model, not a member of the interface. So rontolisp names it — **`-drop`**, one argument, the handle — symmetric with the `-new` a constructor binds: ```console (let ((bucket (kv:open ""))) (kv:bucket-set bucket "visits" "41") (print (kv:bucket-get bucket "visits")) (kv:bucket-drop bucket)) ``` It is bound **only when the program names it** (`--no-prune` and `--dynamic` bind every resource's drop instead), which is why a component compiled before drops existed comes out byte-identical — a WIT *function*, by contrast, is bound whether the program calls it or not. On the interpreter and the JVM the drop reaches the interface's provider as the member `"bucket-drop"`, so what it *means* is the provider's decision: forget the handle, close the connection, or answer `nil` because there is nothing to release. On Preview 1 it is a **no-op** — a handle there is an opaque integer the host handed over, and rontolisp will not invent an import for a function the WIT never declared. Under `--component` it becomes `canon resource.drop`, handing the handle back to the host's own table. This is not only about leaks. An interface may make dropping an **obligation**: `wasi:http` requires an `outgoing-body`'s child `output-stream` to be dropped before the body is finished, and traps if it is not. And a drop releases the *reference*, never the thing behind it — the store stays, and the next `kv:open` sees every key still in it. Current limitations: - `--no-gc` rejects the directive with a clear error: its contract is a plain MVP module that imports nothing at all. - On the Preview 1 boundary only the types `rontolisp:wasm-import` can carry cross — the integer scalars up to 32 bits, the float scalars, `bool`, `string`, `list` and resource handles. A `record`, `option`, `result` or `s64` is a compile error naming the WIT file and line, even though `--component`, the interpreter and the JVM all bind it (the `wasi:keyvalue` program above is therefore a component or an interpreter/JVM program, not a Preview 1 one: its `result` arms keep it off that boundary). A core import is a bare host function, with no component type to describe a richer shape with. `stream` and `future` are rejected on every backend. - Under `--component` a **`list` argument** (other than `list`), and `flags` anywhere, is a compile error; a `list` still crosses as a result. - The directive binds an **interface**. A world's `import` items are still not read. - It must appear at top level **before** the code that calls the interface — it is what defines the package and the bindings — which is the opposite of `wit-export`. The [wit-import](../reference/functions/rontolisp-wit-import.md) and [wit-provide](../reference/functions/rontolisp-wit-provide.md) reference pages carry the full option list, the name-mapping rules and the WIT type table. --- # FILE: references/index.md # rontolisp A minimal Common Lisp subset implemented in Java. It supports three execution modes: - **Interpreter** -- Tree-walking evaluation with REPL support - **JVM compiler** -- Compiles Lisp to `.class` bytecode runnable on any JRE - **WASM compiler** -- Compiles Lisp to `.wasm` using wasm-GC, targeting either a WASI Preview 1 core module or a WASI 0.3 (Component Model) component Try it right here -- the examples on these pages run in your browser via the same WebAssembly build as the [playground](https://making.github.io/rontolisp/playground.html): ```lisp (defun fact (n) (if (= n 0) 1 (* n (fact (- n 1))))) (print (fact 10)) ``` Press **Run** above (the rontolisp runtime loads on the first run). Working with an AI coding agent? These same pages are published as an [agent skill](getting-started/agent-skill.md), regenerated on every deploy. --- # FILE: references/reference/data-types.md # Data Types | Type | Example | Description | |------|---------|-------------| | Integer | `42`, `-5`, `1,000`, `#xff`, `#o777`, `#b1010` | 64-bit signed integer that auto-promotes to a big integer on overflow, exact at any magnitude on every backend. `#x`/`#o`/`#b` read hexadecimal/octal/binary literals | | Ratio | `1/3`, `-2/5` | Exact rational number (Common Lisp ratio), always normalized; supported by all three backends | | Double | `3.14`, `-0.5`, `3,000.50`, `1d0`, `6.02e23` | 64-bit floating-point number | | String | `"hello"` | String literal | | Character | `#\a`, `#\Space`, `#\Newline` | Character literal (`#\` plus a glyph or a standard name: `Space`, `Newline`, `Tab`, `Return`, `Page`, `Backspace`, `Nul`, `Rubout`). The WASM backend indexes strings by byte, so non-ASCII characters are out of scope there | | Symbol | `x`, `foo` | Identifier | | Keyword | `:foo`, `:bar` | Self-evaluating symbol starting with `:` | | Nil | `nil` | False / empty list | | T | `t` | True | | Pi | `pi` | The constant π, read as the double `3.141592653589793` | | Fixnum range | `most-positive-fixnum`, `most-negative-fixnum` | Read as self-evaluating integers like `pi`; the value is backend-dependent (a WASM fixnum is an unboxed 31-bit reference, the interpreter and the JVM backend use 64-bit longs) | | Other limits | `char-code-limit`, `array-total-size-limit`, `array-dimension-limit` | Read as self-evaluating integers like the fixnum range; `char-code-limit` is `1114112` (full Unicode code points) on every backend, the array limits are backend-dependent | | Cons | `(1 2 3)`, `(a . 1)` | Linked list built from cons cells; `(a . b)` is dotted-pair notation for a single cell | | Function | `#'car`, `(lambda (x) x)` | Function object obtained via `#'`/`function`/`lambda` | | Array | `#(1 2 3)`, `#2A((1 2) (3 4))` | Fixed-size array of any rank (rank 1 = vector); `#(...)` and `#nA(...)` are self-evaluating array literals | | Hash table | `(make-hash-table)` | Mutable key/value table with structural (`equal`) keys | | Structure | `#S(POINT :X 1 :Y 2)` | An instance of a [`defstruct`](special-forms/defstruct.md) type. `#S(...)` is both how an instance prints and a self-evaluating literal that reads back into one; the `defstruct` must appear in an earlier top-level form | Numeric literals may use `,` as a grouping separator between digits in the integer part, so `1,000` reads as `1000` and `(+ 1,000 100)` evaluates to `1100`. The comma is only treated as a separator when it sits between two digits; it is stripped before parsing and applies to all three backends. This differs from Common Lisp, where `,` is the unquote character (not supported here). Float literals may carry a Common Lisp exponent marker -- a mantissa followed by one of `e`, `s`, `f`, `d`, `l` (case-insensitive), an optional sign, and an exponent, e.g. `1d0`, `1e0`, `1.5d3` (`1500.0`), `-2e-3`, `6.02e23`. This works in all three backends (it is a reader-level feature). **Unlike Common Lisp, rontolisp has a single floating-point type, so every marker reads as the same 64-bit double** -- the single/short/long-float distinction (`1d0` vs `1e0` vs `1f0`) is not preserved, and there is no `*read-default-float-format*`. A marker that is not followed by exponent digits is not a float: `1d` and `1d0x` read as symbols (like `1+`), not numbers. On **every backend**, integer arithmetic never silently wraps: when an operation (`+`, `-`, `*`, `/`, `1+`, `1-`, `abs`, ...) overflows the fixed-width representation, the result is automatically promoted to an arbitrary-precision big integer, and integer literals of any magnitude are read exactly. A big-integer result that fits back in the narrower representation is demoted again, so values keep a single canonical representation. For example, with `(defun fact (n) (if (= n 0) 1 (* n (fact (- n 1)))))`, `(fact 32)` returns the exact `263130836933693530167218012160000000` everywhere. (The WASM compiler promotes in two steps -- its unboxed 31-bit fixnums first box into a signed 64-bit value, then into a limb-based big integer -- but that is invisible to programs.) **All three backends** support Common Lisp ratios (exact rational numbers). `1/3` reads as a ratio literal, and integer division that does not divide evenly returns a ratio instead of truncating: ```console > 1/3 1/3 > (/ 1 2) 1/2 > (+ 1/2 1/3) 5/6 > (/ 1 2.0) 0.5 > (float 1/2) 0.5 ``` Ratio results are always normalized -- reduced by the gcd with the sign on the numerator (`2/4` reads as `1/2`), and demoted to an integer when the denominator reduces to one (`(/ 10 2)` is `5`, `(+ 1/2 1/2)` is `1`). Arithmetic, comparisons (`= < > <= >=`), `eq`/`eql`, `abs`/`min`/`max`/`1+`/`1-`/ `signum`, the predicates (`numberp`, `rationalp`, `zerop`, `plusp`, `minusp`), `truncate`/`floor`/`ceiling`/`round`, `expt` with an integer exponent (`(expt 2 -1)` is `1/2`), and `numerator`/`denominator` all handle ratios; mixing in a float switches to float contagion. Unary `(/ x)` is the reciprocal (`(/ 2)` is `1/2`). Per backend, the components follow the integer representation: the **interpreter and the JVM compiler** use big integers (a ratio of huge numerators/denominators stays exact), while the **WASM compiler** keeps ratio components in the 31-bit fixnum range with no overflow promotion (plain integers promote without bound, and `truncate`/`floor`/`ceiling`/`round`/ `mod`/`rem` over two integers divide exactly at any magnitude -- only a division kept as a fraction is limited: components past 31 bits fold back, and a limb-sized big integer in an uneven `/` traps). The runtime reader emitted for compiled `read`/`load` does not parse ratio literals (a `1/3` token read at runtime is a symbol), and `mod`, `evenp`/`oddp`, `gcd`/`lcm` and `isqrt` remain integer-only. ## Comments, feature conditionals and `*features*` Besides the `;` line comment, the reader supports the Common Lisp `#| ... |#` block comment (nesting, per the standard) and the `#+`/`#-` feature conditionals: `#+expr form` keeps `form` only when the feature expression holds, `#-expr form` only when it does not. A feature expression is a feature name or an `(and ...)`/`(or ...)`/`(not ...)` combination (spelled bare or as keywords, case-insensitive). The active features are `:rontolisp` on every backend plus one backend-identifying feature — `:rontolisp-interpreter`, `:rontolisp-jvm` or `:rontolisp-wasm` — so one source file can select per-backend code, and `:unicode`, the portable spelling of "characters are Unicode code points" (true on every backend, so a library that branches on it takes its UTF-8 path). The interpreter and the JVM also have `:thread-support` (they really spawn threads — see [`rontolisp:make-thread`](functions/rontolisp-make-thread.md)); a WASM compile in reactor mode (`--no-wasi`, or `--no-gc`) additionally has `:rontolisp-reactor` — the module's entry points are exports a host calls, which is how the Clack handler backend picks its transport (see the [Clack guide](../guides/clack.md)) — and a `--component` compile additionally has `:rontolisp-component`, which names the component BOUNDARY rather than a backend: a component's host functions cross the canonical ABI, so [`rontolisp:wasm-import`](../guides/wasm-gc-module.md) is refused there and a source that declares one guards it with `#-rontolisp-component`. (A `--component --no-wasi` build is a reactor too, so it has both.) `*features*` is an ordinary special variable holding that list, on every backend: a program may `push` onto it, `setq` it, and bind it with `let` like any other special. A source may also **announce a feature about itself**: a top-level `(pushnew :my-feature *features*)` — bare or inside an `eval-when`/`progn` — is read by the reader, so a `#+my-feature` in the same file sees it. That is the header idiom real Common Lisp gets from loading a file form at a time, and it behaves identically on all four backends here. Only a **literal** keyword push counts: a push whose value the program computes (`(pushnew (intern name :keyword) *features*)`) is a real run-time push but is invisible to the reader, because deciding it would mean running the program to decide how the program is read. A `.asd` that needs to announce a feature to the files of the systems it defines uses `:rontolisp-features` instead (see the [Systems guide](../guides/asdf-systems.md)). ```lisp #| a block comment #| nesting like Common Lisp |# still commented |# #+rontolisp (print :ok) ; kept: :rontolisp is always active #-(or sbcl ccl) (print :portable) ; kept: neither feature is active #+sbcl (print (uses #.unsupported-syntax)) (print (car *features*)) ; the first feature is always :rontolisp (pushnew :my-feature *features*) #+my-feature (print :announced) ; kept: the reader saw the push above (print (and (member :my-feature *features*) t)) ``` Notes: - Reading happens once, at the frontend: the interpreter reads with `:rontolisp-interpreter`, and compiling to a `.class`/`.wasm` file reads with `:rontolisp-jvm`/`:rontolisp-wasm`, so the set a compiled program's `#+` conditionals were resolved against is fixed at compile time. Files pulled in by the compile-time `load`/`require`/`asdf:load-system` include are read with the same target features. The run-time `*features*` list starts out holding that same set. - A form skipped by a failing `#+`/`#-` guard is skipped at the raw character level without being parsed, so it may use syntax rontolisp does not support (that is the point of guarding it). - `#.` read-time evaluation **is** supported: each `#.` datum is evaluated just before its top-level form runs — against the global environment on the interpreter, and against the compile-time (macro-time) evaluator on the JVM/WASM compile path — and the value is substituted into the form. In `.asd` files a `#.` form is instead skipped with a warning (see the [Systems guide](../guides/asdf-systems.md)); the browser playground's Compile buttons do not support `#.`. - The runtime reader of compiled programs (`read`, `read-from-string`, runtime `load`) does not know block comments or feature conditionals, like backquote — see [Compiled read/load Limitations](../guides/read-load-limitations.md). - `:common-lisp` is deliberately **not** in `*features*`: rontolisp is a subset, not a conforming implementation. ## Source position literals (`rontolisp:current-file`, `rontolisp:current-line`) Two symbols the reader substitutes with the position they stand on, the way `pi` and `array-dimension-limit` are substituted: `rontolisp:current-file` becomes the origin file as a string (or `nil` when there is none — a REPL line, a `read-from-string`), and `rontolisp:current-line` becomes the 1-based line the symbol itself is on. They are ordinary literals afterwards, so they cost nothing at run time and read the same on the interpreter and on every compile backend. A file pulled in by `load` / `require` / `asdf:load-system` names **itself**, not the entry file it was spliced into — which is the point: in a program assembled from many files, a message can say where it really came from. The file is spelled exactly as the frontend saw it (the path given on the command line, or the one `load` resolved), which is also how a read error spells it. ```console $ cat lib.lisp (defun where () (list rontolisp:current-file rontolisp:current-line)) $ cat main.lisp (load "lib.lisp") (print (where)) (print (list rontolisp:current-file rontolisp:current-line)) $ rontolisp main.lisp ("lib.lisp" 2) ("main.lisp" 3) ``` Notes: - Substitution happens at **read** time, so inside a `defmacro` template these name the macro's own definition site, not its call site. A logging macro therefore takes them as arguments at the call site, the way C code passes `__FILE__` / `__LINE__`: ```console (defmacro log-at (file line msg) `(format t "~a:~a: ~a~%" ,file ,line ,msg)) (log-at rontolisp:current-file rontolisp:current-line "started") ; prints e.g. app.lisp:12: started ``` - Only the qualified spellings are recognized (`rontolisp:current-file`, `rontolisp::current-file`, `rl:current-file`). Unlike the rest of the `rontolisp` package these are **not** available unqualified after `(in-package rontolisp)`: reading happens before any `in-package` directive is interpreted. - Being read-time, they are substituted wherever they appear, quoted data included — `'rontolisp:current-line` is the number, not the symbol. This is the same rule `#+`/`#-` and `#.` follow. ## Dotted pairs, association lists and property lists The reader supports Common Lisp dotted-pair notation: `(a . b)` denotes a single cons cell whose car is `a` and whose cdr is `b`, and `(a b . c)` is a list whose final cdr is `c` instead of `nil`. This is how association-list (alist) literals are written: ```lisp (cdr (assoc 'b '((a . 1) (b . 2)))) ; => 2 ``` Dotted tails also work in backquote templates (`` `(a . ,x) `` expands to a `cons` chain), and the runtime reader of compiled programs parses the same notation, so a `read`/`read-from-string` of `"(a . 1)"` behaves identically in all backends. A standalone `.` outside a list is a read error, as in Common Lisp, and `,@` cannot be combined with a dotted tail in a backquote template. A dotted tail in **call position** (e.g. `(+ 1 . 2)`) is an error in all three backends -- a dotted pair is only meaningful as data. The alist function family -- `assoc`, `assoc-if`, `rassoc`, `acons`, `pairlis` and `copy-alist` -- works in all three backends. `assoc` and `rassoc` compare with `eql` by default and accept optional `:test`/`:key` keywords (`:test` a function designator, e.g. `#'equal` for string keys; `:key` a selector applied to each pair's car/cdr before the comparison), like `member`: ```lisp (assoc "b" '(("a" . 1) ("b" . 2)) :test #'equal) ; => ("b" . 2) ``` Property lists (plists) -- flat lists of alternating indicator/value pairs like `(:a 1 :b 2)` -- are the keyword-based cousin of alists. `getf` reads the value for an indicator (two arguments only: no `&optional default`), the `remf` macro removes an indicator/value pair from a plist held in a variable or other `setf` place, and `&key` parameters in lambda lists are parsed from the same shape. `(setf (getf ...))` is not a supported place and there are no symbol plists (`get`/`symbol-plist`); to add or update an entry, rebuild the list, e.g. by prepending with `list*`: ```lisp (let ((p (list :a 1 :b 2))) (remf p :a) (getf (list* :c 3 p) :c)) ; => 3 ``` ## Arrays `make-array`, `aref` and `(setf (aref ...))` work in all three backends. Arrays of **any rank >= 1** are supported; the dimensions argument is an integer (rank 1) or a non-empty list of integers, and `:initial-element` sets every cell (defaulting to nil). Elements are stored row-major with O(1) access (flat rank-independent access via [`row-major-aref`](functions/row-major-aref.md) / [`array-row-major-index`](functions/array-row-major-index.md)), and arrays are compared by identity (`eq`), so two distinct arrays are never `equal`. `length` returns the element count of a vector (rank-1 array); a multidimensional array is not a sequence, so `length` signals an error on it. Unlike the hash-table operators, the array operators are not exposed as first-class function values, so `#'aref` and `#'make-array` are not available (call them directly). Vectors can also be built with [`vector`](functions/vector.md) and read with [`svref`](functions/svref.md), array shapes are inspected with [`array-dimensions`](functions/array-dimensions.md) / [`array-rank`](functions/array-rank.md) / [`array-total-size`](functions/array-total-size.md), and [`coerce`](functions/coerce.md) converts between lists, vectors and strings. For numpy-style vector/matrix math on top of arrays, see the [`linalg` package](../guides/linear-algebra.md). A 2-D array indexed in nested loops: ```lisp (let ((m (make-array (list 2 3) :initial-element 0))) (setf (aref m 1 2) 9) (incf (aref m 1 2)) (aref m 1 2)) ; => 10 ``` The `#(...)` reader syntax denotes a self-evaluating rank-1 vector literal whose elements are read as data (not evaluated), e.g. `#(1 2 3)` or `#(a "b")`. A rank-n array is written `#nA((...) ...)` with its contents as nested lists of depth n (`#2A` for a matrix, `#3A` for a rank-3 array, ...); every list at the same depth must have the same length, so ragged contents are a read error. Arrays print in the same readable syntax across all backends, with `prin1` quoting string elements and `princ` not: ```lisp (print #(1 2 3)) ; #(1 2 3) (princ #(a "b")) ; #(a b) (print #2A((1 2) (3 4))) ; #2A((1 2) (3 4)) (aref #3A(((1 2) (3 4)) ((5 6) (7 8))) 1 0 1) ; => 6 (make-array (list 2 2) :initial-element 0) ; #2A((0 0) (0 0)) ``` ### Packed float arrays (`#d` / `#f`) `#d(...)` and `#f(...)` denote a **packed float array**: a float-typed array whose elements are stored unboxed. `#d(...)` is `double-float` (f64) and `#f(...)` is `single-float` (f32 -- half the memory, double the SIMD lane count). They read like `#(...)`, but every element is coerced to the array's float type, so `#d(1 2 3)` and `#d(1.0 2.0 3.0)` are the same vector and `(array-element-type #d(1.0))` is `double-float` (`single-float` for `#f`). Higher-rank literals use nested lists -- `#d((1.0 2.0) (3.0 4.0))` is a matrix -- and `(make-array n :element-type 'double-float)` (or `'single-float`) builds one at runtime. Scalars stay `double`: reading an element widens it to a `double` (a single-float element is widened f32 -> f64), and storing one narrows it to the array's width (f64 -> f32 for a single-float array). Storing a non-real is a type error (a general array holds any value). Otherwise a packed array behaves like a general array of the same numbers for every operation -- `aref`, `(setf (aref ...))`, `length`, `row-major-aref`, `array-rank`, `array-dimensions` and `coerce` all work on it -- except that it prints with its own `#d(...)` / `#f(...)` reader syntax, so its printed form reads back as a packed array of the same width (preserving the unboxed representation) rather than degrading to a general one. It is simply the unboxed, float-specialized representation the numeric kernels use, so fill pointers, adjustable and displaced arrays are not available on it (those need a general array). The double-float width is the default and what `linalg` produces. For fast vectorized kernels over packed arrays -- and their optional hardware acceleration -- see the [`vec` package](../guides/simd-acceleration.md). A packed array is also a binary I/O buffer: [`read-sequence`](functions/read-sequence.md) / [`write-sequence`](functions/write-sequence.md) move its elements as raw little-endian IEEE-754 in one bulk transfer (any rank, row-major), which is how a weight file or a numpy dump is loaded. ```lisp (aref #d(1.0 2.0 3.0) 1) ; => 2.0 (array-element-type #d(1 2 3)) ; => DOUBLE-FLOAT (array-element-type #f(1.0 2.0)) ; => SINGLE-FLOAT (print #d((1.0 2.0) (3.0 4.0))) ; #d((1.0 2.0) (3.0 4.0)) (coerce #d(1 2 3) 'list) ; => (1.0 2.0 3.0) (let ((v (make-array 3 :element-type 'single-float :initial-element 0.0))) (setf (aref v 0) 5) v) ; => #f(5.0 0.0 0.0) ``` ## Hash tables `make-hash-table`, `gethash`, `(setf (gethash ...))`, `remhash`, `clrhash`, `hash-table-count`, `hash-table-p` and `maphash` work in all three backends. Keys are compared structurally (as if by `equal`): a list key like `(list r c)` matches an equal list, and numbers, symbols, characters and strings match by value. `:test` is accepted for familiarity but does not change this -- an `eql` table also matches structurally-equal aggregate keys. Iteration order (`maphash`) is not guaranteed across backends, so portable code should not depend on it. A table itself prints as SBCL's unreadable tag minus its trailing identity hash -- `#`, the same text on every backend, with no entry content. `:TEST` is always `EQUAL`, the test lookup actually implements and the one `hash-table-test` reports, whatever `:test` the table was made with; `:COUNT` is the live entry count, the same number `hash-table-count` returns. A key is placed by a depth-capped structural hash and then decided by `equal`, so lookup does not depend on the size of the key's printed form and a key whose structure is CYCLIC is usable -- stored and retrieved under the same object: ```lisp (let ((h (make-hash-table))) (setf (gethash 'a h) 1) (princ-to-string h)) ; => "#" ``` They are also usable as first-class function values (`#'gethash`, `#'remhash`, `#'clrhash`, `#'hash-table-count`, `#'hash-table-p`, `#'maphash`, and `#'make-hash-table` in its no-argument form) on all three backends -- passed via fixed-arity wrappers, so `gethash`'s optional default and `make-hash-table`'s keyword arguments are not available through the function value. A typical use -- counting with `incf` on the place: ```lisp (let ((counts (make-hash-table :test 'equal))) (dolist (w '("a" "b" "a")) (incf (gethash w counts 0))) (gethash "a" counts)) ; => 2 ``` --- # FILE: references/reference/function-namespace.md # Function Namespace rontolisp is a **Lisp-2**, following Common Lisp: functions and variables live in separate namespaces. - A bare symbol evaluates as a **variable**. Evaluating `car` alone is an error (`The variable car is unbound` in the interpreter; a compile error in the compilers). - A symbol in **call position** `(f args...)` resolves in the function namespace only. A variable named `car` never shadows the function `car`: `(let ((car 5)) (car (list car 2)))` returns `5`. - A function becomes a **value** through `#'name` (reader syntax for `(function name)`), `#'(lambda ...)`, or `(symbol-function 'name)`. This works for built-in operators (`#'+`, `#'car`, `#'1+`, `#'cadr`), user `defun`s, and lambdas. - `funcall`/`mapcar`/`reduce` also accept a **symbol** naming a function (a function designator): `(funcall 'car '(1 2))` returns `1`. The compilers support this when the symbol is a quoted literal. - `defun` defines into the function namespace and returns the function name. `(setq f (lambda ...))` binds a **variable** to a function value; call it with `(funcall f ...)`, not `(f ...)`. - `#'` of a macro or special operator (e.g. `#'if`, `#'defun`) is an error. Function values can be passed as arguments, returned from functions, and stored in data structures in all three execution modes. **Higher-order functions:** ```lisp (defun apply-twice (f x) (funcall f (funcall f x))) (defun square (x) (* x x)) (print (apply-twice #'square 3)) ; => 81 ``` **Closures (capture by reference):** ```lisp (defun make-counter () (let ((n 0)) (lambda () (setq n (+ n 1)) n))) (setq c (make-counter)) (funcall c) ; => 1 (funcall c) ; => 2 (funcall c) ; => 3 ``` **Lambda as argument:** ```lisp (defun apply-twice (f x) (funcall f (funcall f x))) (print (apply-twice (lambda (x) (+ x 10)) 5)) ; => 25 ``` **Built-in operators as first-class values:** Built-in operators like `+`, `car`, `1+` can be passed to higher-order functions via `#'`: ```lisp (print (reduce #'+ '(1 2 3 4 5) :initial-value 0)) ; => 15 (print (reduce #'* '(1 2 3 4 5) :initial-value 1)) ; => 120 (print (mapcar #'car '((1 2) (3 4) (5 6)))) ; => (1 3 5) (print (mapcar #'1+ '(1 2 3))) ; => (2 3 4) (print (funcall #'+ 3 4)) ; => 7 (setq my-op #'+) (print (funcall my-op 10 20)) ; => 30 (print (funcall (symbol-function 'car) '(9 8))) ; => 9 ``` **Compiler restrictions.** In the JVM/WASM compilers, `#'name` resolves against the functions known at compile time (user `defun`s and built-in operators). `#'reduce` and `#'apply` themselves are not available as values there (the interpreter has them); `#'mapcar`, `#'mapcan`, `#'sort` and `#'funcall` are. `symbol-function` requires a quoted symbol literal argument. In `--dynamic` mode an unresolved `#'name` is deferred to the runtime `eval` environment like any other unresolved reference. In compiled code `apply`/`funcall` dispatch by the actual argument count against a fixed-arity wrapper synthesized for each built-in operator. The naturally variadic operators -- `+`, `-`, `*`, `/`, `list`, `min`, `max` -- have variadic wrappers, so `(funcall #'+ 1 2 3)`, `(apply #'list ...)` and the like accept any argument count. Every other multi-argument built-in keeps a fixed wrapper arity: `#'cons`, `#'append`, `#'gcd` and the comparison chains (`#'<`, `#'=`, ...) are binary, so applying them to a different count is unsupported on the compile path (matching the [Compiled `eval` limitations](../guides/eval-limitations.md)); use a user-defined function or a `lambda` for other arities. The interpreter has no such restriction. ## Redefining a COMMON-LISP function `(defun random ...)`, `(defun length ...)` and the like are **undefined behavior** in Common Lisp (CLHS 11.1.2.1.2), and rontolisp's backends genuinely differ: - the **interpreter** resolves the call through the function cell, so your definition runs; - the **JVM and WASM compilers** recognize the standard operator at the call site and compile it inline, so your definition does not run there. They print a compile-time warning naming the operator and the position of the first such call site, so the divergence is never silent. `#'random` names your definition on every backend, which is why the warning says the definition is still reachable that way. To have one definition everywhere, give it a name of your own or `shadow` the symbol into a package of your own. A `defmethod` on a built-in name is a different case and does work on every backend: it becomes the generic's default method. --- # FILE: references/reference/functions.md # Functions This page is the quick-reference table. **Each function name in the table links to its own page**, which has a fuller description and a runnable example you can evaluate in your browser. Cross-cutting topics have their own homes: the `make-array`/`aref` and hash-table operators are described under [Arrays](data-types.md#arrays) and [Hash tables](data-types.md#hash-tables) on the Data Types page, and each function's deviations from Common Lisp are noted on its own page. ## cl Package Functions The standard Common Lisp functions, in the `cl` package (used by `cl-user`, so they are available unqualified in ordinary programs). Each name links to its own page. | Function | Example | Result | |----------|---------|--------| | `+` | `(+ 1 2 3)`, `(+ 1.5 2.5)` | `6`, `4.0` | | `-` | `(- 10 3)`, `(- 3.5 1.5)` | `7`, `2.0` | | `*` | `(* 3 4)`, `(* 2.0 3.0)` | `12`, `6.0` | | `/` | `(/ 1 2)`, `(/ 10 2)`, `(/ 7.0 2.0)` | `1/2` (exact ratio), `5`, `3.5` | | `mod` | `(mod 10 3)`, `(mod -13 4)` | `1`, `3` (result takes the sign of the divisor) | | `rem` | `(rem 13 4)`, `(rem -13 4)` | `1`, `-1` (result takes the sign of the dividend) | | `=` | `(= 1 1)`, `(= 3 3 3)` | `t` (variadic) | | `eq` | `(eq 'foo 'foo)`, `(eq 1.5 1.5)` | `t`, `nil` (object identity: symbols and small integers compare equal, but floats and ratios are distinct objects, so never `eq`; reference identity for cons cells) | | `eql` | `(eql 1.5 1.5)`, `(eql 3 3.0)` | `t`, `nil` (like `eq`, but numbers of the same type and value are equal — e.g. floats and ratios) | | `equal` | `(equal '(1 2 (3)) '(1 2 (3)))`, `(equal "abc" "abc")` | `t`, `t` (structural equality: cons cells compared recursively by car and cdr, otherwise like `eql`) | | `equalp` | `(equalp "ABC" "abc")` | `t` (like `equal` but strings/characters compare case-insensitively and numbers by value; arrays/hash-tables fall back to `eql`) | | `<` | `(< 1 2)`, `(< 1 2 3)` | `t` (variadic; true when strictly increasing) | | `>` | `(> 2 1)`, `(> 3 2 1)` | `t` (variadic) | | `<=` | `(<= 1 1)` | `t` (variadic) | | `>=` | `(>= 2 1)` | `t` (variadic) | | `print` | `(print 42)` | Prints `42` with a newline | | `prin1` | `(prin1 42)` | Like `print` but without newline | | `princ` | `(princ "hello")` | Prints without quotes and without newline | | `terpri` | `(terpri)` | Prints a newline only | | `fresh-line` | `(fresh-line)` | Prints a newline only if standard output is not already at the start of a line. Returns nil | | `princ-to-string` | `(princ-to-string '(1 "x"))` | `"(1 x)"` -- the string `princ` would print | | `prin1-to-string` | `(prin1-to-string "abc")` | `"\"abc\""` -- the string `prin1` would print (readable form) | | `write` | `(write "hi" :escape nil)` | Prints `hi`; each keyword binds the matching printer control variable around the one print | | `pprint` `pprint-newline` `pprint-indent` `pprint-tab` | `(pprint-newline :mandatory s)` | A newline for `:mandatory` only -- no stream carries a column, so nothing wraps | | `copy-pprint-dispatch` `set-pprint-dispatch` `pprint-dispatch` | `(pprint-dispatch 21 table)` | The pretty-print dispatch table (real entries + lookup; the ordinary printing operators do not consult it) | | `concatenate` | `(concatenate 'string "foo" "bar")` | `"foobar"` (`'string` / `'list` / `'vector` result families; the compilers require a literal quoted designator) | | `string-upcase` | `(string-upcase "abc")` | `"ABC"` (full-Unicode, and length-preserving: `char-upcase` per character) | | `string-downcase` | `(string-downcase "ABC")` | `"abc"` | | `string-capitalize` | `(string-capitalize "hello world")` | `"Hello World"` (first letter of each word) | | `nstring-upcase` `nstring-downcase` `nstring-capitalize` | `(nstring-upcase (copy-seq "abc"))` | `"ABC"` — the destructive spellings: the fold is written back into the argument (a mutable character vector is written in place on every backend; an immutable string is rebuilt on the compile paths) | | `subseq` | `(subseq "hello" 1 3)` | `"el"` (works on strings and lists, e.g. `(subseq '(1 2 3 4) 1 3)` => `(2 3)`; the `end` argument is optional) | | `make-string` | `(make-string 3 :initial-element #\x)` | `"xxx"` -- a fresh string of `n` copies of `:initial-element` (default space); `:element-type` is accepted and ignored | | `make-sequence` | `(make-sequence 'list 3)` | `(nil nil nil)` -- a sequence of the literal quoted result type (string types via `make-string`, `list` via `make-list`, vector types via `make-array`) | | `replace` | `(replace (make-string 5 :initial-element #\a) "XY" :start1 1)` | `"aXYaa"` -- copy `sequence-2` into `sequence-1` (`:start1`/`:end1`/`:start2`/`:end2`); string-aware, mutates an allocated buffer in place | | `fill` | `(fill (list 1 2 3) 7)` | `(7 7 7)` -- store one item into every element between `:start`/`:end`; destructive over a vector or list, string-aware like `replace` | | `string=` | `(string= "abc" "abc")`, `(string= "together" "frog" :start1 1 :end1 3 :start2 2)` | `t` (case-sensitive string equality; `:start1`/`:end1`/`:start2`/`:end2` bound the compared substrings) | | `string<` `string>` `string<=` `string>=` `string/=` | `(string< "abc" "abd")` | `2` -- case-sensitive lexicographic comparison: the mismatch index in `string1` (`end1` when equal), or nil. Same `:start1`/`:end1`/`:start2`/`:end2` keywords | | `string-equal` | `(string-equal "ABC" "abc")` | `t` (case-insensitive, ASCII) | | `string-lessp` `string-greaterp` `string-not-greaterp` `string-not-lessp` `string-not-equal` | `(string-not-greaterp "Abcde" "abcdE")` | `5` -- the case-insensitive counterparts of `string<` `string>` `string<=` `string>=` `string/=` | | `string-trim` | `(string-trim " " " hi ")` | `"hi"` (removes the bag's characters from both ends) | | `string-left-trim` | `(string-left-trim "x" "xxhi")` | `"hi"` | | `string-right-trim` | `(string-right-trim "x" "hixx")` | `"hi"` | | `read-line` | `(read-line)`, `(read-line stream)` | Read one line from stdin (or from an input stream), return as string. `nil` on EOF | | `y-or-n-p` | `(y-or-n-p "Delete ~A?" f)` | Print the optional `format` control plus `" (y or n) "`, read a LINE from stdin, and answer `t` for `y`/`Y`, `nil` for `n`/`N`, re-asking otherwise. Lite: CL reads single characters without echo, and end of input answers `nil` | | `peek-char` | `(peek-char nil s)`, `(peek-char t s)`, `(peek-char #\; s)` | The next character of a stream WITHOUT consuming it. `peek-type` `nil` skips nothing, `t` skips whitespace, a character skips up to that character; the character returned is left in the stream. At EOF, signal `end-of-file`, or return `eof-value` when `eof-error-p` is `nil` | | `read-char-no-hang` | `(read-char-no-hang s)` | One character if one is available without waiting. On a stream handle it is `read-char`; on a [Gray stream](../guides/gray-streams.md) instance it dispatches to `rontolisp:stream-read-char-no-hang` | | `unread-char` | `(unread-char c s)` | Push the character just read back, so the next read returns it again; answers `nil`. One character for one stream, on a [Gray stream](../guides/gray-streams.md) instance and on a stream handle alike | | `open` | `(open "f.txt")`, `(open "f.txt" :output)`, `(open "f.bin" :input '(unsigned-byte 8))` | Open a file and return a stream. The direction must be the literal `:input` (default, read) or `:output` (create/truncate, write); the optional element type must be the literal `'character` (default, text) or `'(unsigned-byte 8)` (binary) | | `close` | `(close stream)` | Close a stream opened by `open`. Returns `t` | | `probe-file` | `(probe-file "f.txt")` | The pathname when the file exists, `nil` otherwise. The only file operation that does not fail on a missing path (`open` signals). `uiop:file-exists-p` is the same operation | | `truename` | `(truename "f.txt")` | The pathname when the file exists, an error otherwise — the signalling twin of `probe-file`, which is what makes `(ignore-errors (truename p))` a portable existence probe | | `directory` | `(directory "src/*.lisp")` | The pathnames matching the pathspec, sorted, keeping its directory prefix and giving each subdirectory a trailing `/`. A wild NAME component matches (`*` any sequence, `?` one character, `*` alone meaning "no type" as in CL); a non-wild one designates itself, so listing a directory is `"src/*.*"`, not `"src/"`. A wild DIRECTORY component walks: `*` one level, `**` the whole subtree | | `pathname-directory` | `(pathname-directory "a/b/c.txt")` | `(:RELATIVE "a" "b")` — the directory component of a namestring as CL's list (`:absolute`/`:relative` plus one string per level), `nil` when there is none. Pure string work; nothing is read | | `pathname-name` | `(pathname-name "d/a.b.c")` | `"a.b"` — the file-name component without its type: everything after the last `/` and before the LAST dot (a dot at position 0 belongs to the name). `nil` when the namestring names no file | | `pathname-type` | `(pathname-type "d/a.b.c")` | `"c"` — the type (extension) without its dot, `nil` when there is none. The other half of the same split | | `pathname-host` | `(pathname-host "d/a.txt")` | always `nil` — a flat namestring carries no host component. The designator is still validated | | `pathname-device` | `(pathname-device #P"d/a.txt")` | always `nil`, for the same reason (and what SBCL answers on Unix) | | `pathname-version` | `(pathname-version #P"d/a.txt")` | always `nil` — there are no file versions here | | `wild-pathname-p` | `(wild-pathname-p "d/*.txt" :name)` | whether the pathname (or just the `:directory`/`:name`/`:type` component named) holds a `*` or `?`. `:host`/`:device`/`:version` are always `nil` | | `enough-namestring` | `(enough-namestring "/a/b/c.lisp" "/a/")` | `"b/c.lisp"` — the shortest namestring that still names the file when merged against the defaults (`*default-pathname-defaults*` by default): the inverse of `merge-pathnames` | | `file-namestring` `directory-namestring` `host-namestring` | `(file-namestring #P"/a/b/c.txt")` | `"c.txt"` — the string-valued components: the name-and-type half, the directory half (they concatenate back to `namestring`), and `""` for the host a rontolisp namestring does not carry | | `translate-pathname` | `(translate-pathname "src/f.lisp" "src/*.lisp" "build/*.fasl")` | `#P"build/f.fasl"` — matches the source against the from-wildcard and substitutes what each `*`/`?` captured into the to-wildcard. A source that does not match signals | | `translate-logical-pathname` | `(translate-logical-pathname "d/a.txt")` | `#P"d/a.txt"` — the identity: every rontolisp pathname is physical, so there is nothing to translate | | `logical-pathname` | `(logical-pathname "SYS:SRC;")` | always signals — no logical host can be defined here, so no argument can name a logical pathname | | `pathname` | `(pathname "d/x")` | `#P"d/x"` — the canonical constructor: a pathname unchanged, a string wrapped into the pathname it designates, anything else signals | | `parse-namestring` | `(parse-namestring "d/a.txt")` | `#P"d/a.txt"` (and the stop position as a second value) — lite: no host parsing, the whole string is the namestring | | `make-pathname` | `(make-pathname :name "b" :defaults "d/a.sql")` | `#P"d/b.sql"` — composes a pathname from `:directory`/`:name`/`:type`, taking every UNSUPPLIED component from `:defaults`. Component-wise, NOT a merge: a supplied component replaces the defaults' one and an explicit `nil` means "no component". A real function on all four backends; literal calls are additionally folded at compile time | | `namestring` | `(namestring #P"/tmp/x")` | `"/tmp/x"` — the namestring a pathname carries; a string (a designator) passes through, anything else signals. `uiop:namestring` and `uiop:native-namestring` are the same function | | `merge-pathnames` | `(merge-pathnames "zoneinfo/" "/opt/lt/")` | Fills the gaps in the first pathname from the second (both spellings accepted): an absolute directory wins, a relative one is appended, an absent one is taken from the defaults. `uiop:merge-pathnames*` is the same merge | | `open-stream-p` | `(open-stream-p stream)` | `t` while the handle names an open stream, `nil` after `close` (exact for sockets on the interpreter/JVM and on `--component`) | | `force-output` | `(force-output stream)` | Flush an output stream (no argument = standard output). Returns nil | | `finish-output` | `(finish-output stream)` | The same operation as `force-output` -- every write here is synchronous once flushed | | `clear-output` | `(clear-output stream)` | Discard an output stream's unwritten buffer. Nothing is buffered that way here, so it validates the designator and returns nil | | `listen` | `(listen stream)` | `t` when input is immediately available without blocking; Preview 1 WASM has no such probe | | `write-line` | `(write-line "hi" stream)`, `(write-line "hi")` | Write the string plus a newline to an output stream (or to standard output). Returns the string | | `read-byte` | `(read-byte stream)`, `(read-byte *standard-input* nil nil)` | Read one byte (0-255) from a binary input stream, or from standard input for the `t`/`nil` designator. At EOF, signal an `end-of-file` condition, or return `eof-value` when `eof-error-p` is `nil` | | `write-byte` | `(write-byte 255 stream)`, `(write-byte 255 *standard-output*)` | Write one raw byte (0-255) to a binary output stream, or to standard output for the `t`/`nil` designator. Returns the byte | | `read-sequence` | `(read-sequence buf stream)`, `(read-sequence buf stream :start 2 :end 4)` | Fill a vector from an input stream -- characters when the buffer is a character vector, raw little-endian elements in bulk when it is a packed float array (any rank) or packed integer vector, else bytes. Returns the fill position. `:start`/`:end` must be literal keywords | | `write-sequence` | `(write-sequence "abcd" s :start 1 :end 3)`, `(write-sequence buf stream)` | Write a sequence to a stream and return it. A string is written as characters (like `write-string`); a packed float array / integer vector as raw little-endian elements in bulk; a vector of bytes (0-255) to a binary output stream. `:start`/`:end` must be literal keywords | | `read` | `(read)`, `(read stream)` | Read one S-expression from stdin (or from an input stream opened by `open`/`with-open-file`) (all three backends). `nil` on EOF | | `read-from-string` | `(read-from-string "(+ 1 2)")` | Parse one datum from a string (all three backends). The optional `eof-error-p`/`eof-value` and `:start`/`:end` arguments are not supported | | `parse-integer` | `(parse-integer "42")`, `(parse-integer "ff" :radix 16)`, `(parse-integer "12x" :junk-allowed t)` | Parse an integer from a string. Supports `:start`/`:end`/`:radix`/`:junk-allowed` on all backends; the stop position is the second value, observable through `multiple-value-bind`. Without `:junk-allowed`, trailing non-whitespace is an error | | `copy-readtable` | `(copy-readtable nil)` | Lite stub: always `nil` -- the reader is not readtable-driven, so there is no readtable object (`*readtable*` exists but is seeded to `nil`) | | `set-dispatch-macro-character` | `(set-dispatch-macro-character #\# #\7 fn)` | Lite stub: accepted and ignored, returns `t` (user dispatch macros cannot extend the reader) | | `readtable-case` | `(readtable-case *readtable*)` | Lite stub: always `:upcase` -- the reader always upcases unescaped symbol names, the standard readtable's mode | | `char` `schar` | `(char "hello" 1)` | `#\e` -- the character at a 0-based string index | | `char-code` | `(char-code #\A)` | `65` -- the code point of a character | | `code-char` | `(code-char 66)` | `#\B` -- the character with a given code point | | `char=` `char<` `char<=` `char>` `char>=` `char/=` `char-equal` | `(char< #\a #\b #\c)` | `t` (variadic comparison by code point; `char/=` = pairwise distinct, `char-equal` = case-insensitive `char=`) | | `char-lessp` `char-greaterp` `char-not-lessp` `char-not-greaterp` `char-not-equal` | `(char-lessp #\a #\B)` | `t` (the case-INSENSITIVE ordering family) | | `char-upcase` `char-downcase` | `(char-upcase #\a)` | `#\A` (full-Unicode case folding on every backend) | | `characterp` | `(characterp #\a)` | `t` | | `alpha-char-p` | `(alpha-char-p #\x)`, `(alpha-char-p #\5)` | `t`, `nil` (ASCII letters in the WASM backend) | | `alphanumericp` | `(alphanumericp #\x)`, `(alphanumericp #\-)` | `t`, `nil` (letter or decimal digit) | | `graphic-char-p` `standard-char-p` | `(graphic-char-p #\Space)`, `(standard-char-p #\Newline)` | `t`, `t` (printing character / the 96 standard characters) | | `make-load-form-saving-slots` | `(make-load-form-saving-slots obj)` | Lite stub: signals (no fasl dumper); exists so `make-load-form` methods compile | | `sxhash` | `(sxhash "ab")` | Structural hash (integers/characters/strings/symbols/conses); stable within a run, not across backends | | `sbit` | `(sbit #*0110 1)` | Bit-vector element read; `(setf (sbit v i) b)` writes | | `bit` | `(bit #*0110 1)` | Bit-array element read; `(setf (bit v i) b)` writes | | `both-case-p` | `(both-case-p #\a)` | True for a cased letter (`lower-case-p` or `upper-case-p`) | | `special-operator-p` | `(special-operator-p 'if)` | `t` for the 25 ANSI special operators, `nil` for everything else | | `macro-function` | `(macro-function 'when)` | The macro expander (real on the interpreter, a signalling stub in compiled output), `nil` for a function or special operator | | `compiled-function-p` | `(compiled-function-p #'car)` | Lite stub: always `nil` | | `function-lambda-expression` | `(function-lambda-expression #'car)` | Lite stub: `(values nil t nil)` (no source recorded) | | `list-all-packages` | `(list-all-packages)` | Every registered package, as the keywords `find-package` answers (the compilers answer from a table baked in at compile time) | | `find-class` | `(find-class 'c)` | The memoized (`eq`-stable) class metaobject; signals unless `errorp` is `nil` | | `allocate-instance` | `(allocate-instance (find-class 'c))` | A fresh instance with every slot unbound; no initforms, no `initialize-instance` | | `class-name` | `(class-name (class-of 42))` | The name symbol of a class metaobject | | `get` | `(get 'sym 'prop)`, `(setf (get 'sym 'prop) v)` | Symbol property lists over one program-global name-keyed store | | `symbol-plist` | `(symbol-plist 'sym)` | The whole property list `get` indexes into, out of the same store; no `(setf symbol-plist)` | | `remprop` | `(remprop 'sym 'prop)` | Drop one property from the same store; `t` when it was there, `nil` when not | | `lower-case-p` `upper-case-p` | `(lower-case-p #\a)`, `(upper-case-p #\A)` | `t`, `t` -- true when up/down-casing changes the character (follows the Unicode case tables) | | `digit-char-p` | `(digit-char-p #\7)`, `(digit-char-p #\f 16)` | `7`, `15` -- the digit weight in the given radix (default 10), or nil | | `digit-char` | `(digit-char 11 16)` | `#\B` -- the character for a weight in the radix (default 10), or nil | | `eval` | `(eval '(+ 1 2))` | Evaluate an expression (all three backends). Returns the result | | `compile` | `(compile nil '(lambda (x) (* x x)))` | Coerce a lambda expression to a function (null lexical environment). In compiled programs, only the definition-time method-construction idiom is supported | | `load` | `(load "bar.lisp")` | Read and evaluate every top-level form in a file in the global environment (all three backends). Returns `t` | | `require` | `(require :util)`, `(require :util "lib/util.lisp")` | Load a module's file (`.lisp` next to the requiring file, or the explicit path) unless already `provide`d. Returns the module name. On the compile path it must be a literal, top-level form | | `provide` | `(provide :util)` | Mark a module as loaded so a later `require` of it is a no-op. Returns the module name. On the compile path it must be a literal, top-level form | | `gensym` | `(gensym)`, `(gensym "tmp")` | `#:g1`, `#:tmp2` -- a fresh symbol for macro temporaries (the counter is program-wide) | | `make-symbol` | `(make-symbol "temp")` | `#:temp` -- a fresh uninterned symbol (the gensym `#:` convention, no counter) | | `copy-symbol` | `(copy-symbol 'foo)` | `#:FOO` -- an uninterned symbol of the same name; the property-list argument is ignored, and the copy inherits `make-symbol`'s identity deviation | | `intern` | `(intern "foo")` | The symbol `foo`. On the interpreter the name is interned into the current package (`in-package` state); `(intern name :keyword)` builds a keyword, any other package argument is an error | | `find-symbol` | `(find-symbol "car")` | `car` when the name is known (cl symbol, keyword, or user definition), else `nil`; a package that does not exist yields `nil` too (compilers: only a literal string can answer `nil`) | | `find-package` | `(find-package :cl)` | `:cl` -- lite: the upcased package name as a keyword (no package objects), `nil` when unknown (the compilers answer a computed designator from a table baked in at compile time) | | `symbol-name` | `(symbol-name 'foo)` | `"FOO"` -- symbols read upcased like CL, so `(symbol-name 'car)` is `"CAR"` too | | `symbol-package` | `(symbol-package :foo)` | `:keyword` -- the same keyword shape `find-package` returns (`:cl` for standard symbols, `:cl-user` otherwise, `nil` for `#:` symbols); the compilers answer `:cl-user` for both `cl` and `cl-user` | | `package-name` | `(package-name (find-package :cl-user))` | `"CL-USER"` -- the name string of a package designator, resolved through `find-package`; an unknown designator signals | | `package-use-list` | `(package-use-list :cl-user)` | `(:CL)` -- the packages a package uses, as `find-package` keywords; an unknown designator signals | | `package-used-by-list` | `(package-used-by-list :cl)` | The inverse: every package whose use list names this one | | `package-shadowing-symbols` | `(package-shadowing-symbols :cl-user)` | Always `nil` (there is no symbol shadowing); the designator is still validated | | `symbol-value` | `(symbol-value '*level*)` | The global variable's value; unbound names signal an error (lexical bindings are invisible) | | `boundp` | `(boundp '*level*)` | `t` when the symbol names a bound global variable (t/nil/keywords are self-bound) | | `fboundp` | `(fboundp 'car)` | `t` for functions, macros and special forms (compilers: a computed argument sees functions only) | | `fmakunbound` | `(fmakunbound 'greet)` | `greet` -- makes the name call-time-undefined again (compilers: late-bound references only) | | `macroexpand-1` | `(macroexpand-1 '(unless c x))` | `(if c nil x)` -- expand the top-level form once (user and built-in macros) | | `macroexpand` | `(macroexpand '(outer 41))` | The full expansion: `macroexpand-1` repeated to a fixpoint | | `null` | `(null nil)` | `t` | | `not` | `(not nil)` | `t` (identical to `null`) | | `atom` | `(atom 1)` | `t` | | `numberp` | `(numberp 42)` | `t` | | `integerp` | `(integerp 42)` | `t` | | `floatp` | `(floatp 3.14)` | `t` | | `rationalp` | `(rationalp 1/2)` | `t` (integers and ratios) | | `numerator` | `(numerator 3/4)` | `3` (an integer is its own numerator) | | `denominator` | `(denominator 3/4)` | `4` (`1` for integers) | | `symbolp` | `(symbolp 'foo)` | `t` | | `stringp` | `(stringp "hello")` | `t` | | `arrayp` | `(arrayp "abc")` | `T` -- a string is an array in CL, like `vectorp` | | `simple-string-p` | `(simple-string-p "hello")` | `t` -- every rontolisp string is "simple" (lite) | | `listp` | `(listp '(1 2))` | `t` | | `consp` | `(consp '(1 2))` | `t` | | `keywordp` | `(keywordp :foo)` | `t` | | `constantp` | `(constantp 5)`, `(constantp 'x)` | `t`, `nil` -- true for self-evaluating objects (numbers, strings, characters, keywords, `t`/`nil`) and `(quote x)` forms (lite); an optional environment argument is accepted and ignored | | `streamp` | `(streamp s)` | `t` if `s` is a stream, else `nil` (lite: streams are integer handles, so equivalent to `integerp`; also backs the `stream` type specifier) | | `cons` | `(cons 1 2)` | `(1 . 2)` | | `car` | `(car (cons 1 2))` | `1` (`(car nil)` is `nil`) | | `cdr` | `(cdr (cons 1 2))` | `2` (`(cdr nil)` is `nil`) | | `caar`..`cddddr` | `(cadr '(1 2 3))` | `2` (compositions of `car`/`cdr`, 2-4 levels) | | `first` | `(first '(1 2 3))` | `1` (same as `car`) | | `rest` | `(rest '(1 2 3))` | `(2 3)` (same as `cdr`) | | `nth` | `(nth 1 '(1 2 3))` | `2` (0-based indexing) | | `second` `third` `fourth` | `(second '(1 2 3))` | `2` | | `list` | `(list 1 2 3)` | `(1 2 3)` | | `nthcdr` | `(nthcdr 2 '(1 2 3))` | `(3)` (skip first n elements) | | `length` | `(length '(1 2 3))`, `(length "abc")`, `(length #(1 2 3))` | `3`, `3`, `3` (lists, strings and vectors; `0` for nil) | | `reverse` | `(reverse '(1 2 3))` | `(3 2 1)` | | `member` | `(member 2 '(1 2 3))` | `(2 3)` (tail whose car is `eql` to the item, or nil; optional `:test`/`:key` keywords, e.g. `(member '(a d) '((a b) (a d)) :test 'equal)` -> `((a d))`) | | `find` | `(find 2 '(1 2 3))` | `2` (first element `eql` to the item, or nil; optional `:test`/`:key` keywords) | | `find-if` | `(find-if #'evenp '(1 3 6 7))` | `6` (first element satisfying the predicate, or nil) | | `find-if-not` | `(find-if-not #'evenp '(2 4 5 6))` | `5` (first element failing the predicate, or nil) | | `member-if` | `(member-if #'oddp '(2 4 5 6))` | `(5 6)` (tail starting at the first element satisfying the predicate, or nil) | | `position` | `(position 3 '(1 2 3))` | `2` (0-based index of the first element `eql` to the item, or nil; optional `:test`/`:key` keywords) | | `position-if` | `(position-if #'evenp '(1 3 6 7))` | `2` (0-based index of the first element satisfying the predicate, or nil) | | `count` | `(count 2 '(1 2 3 2 2))` | `3` (number of elements `eql` to the item; optional `:test`/`:key` keywords) | | `count-if` | `(count-if #'evenp '(1 2 3 4))` | `2` (number of elements satisfying the predicate) | | `count-if-not` | `(count-if-not #'evenp '(1 2 3 4 5))` | `3` (number of elements FAILING the predicate; `:key`/`:start`/`:end`/`:from-end`) | | `assoc` | `(assoc 'b '((a . 1) (b . 2)))` | `(b . 2)` (first pair whose car matches the key, or nil; `eql` compare by default, optional `:test`/`:key` keywords, e.g. `(assoc "b" '(("a" . 1) ("b" . 2)) :test #'equal)`) | | `assoc-if` | `(assoc-if #'oddp '((2 a) (3 b)))` | `(3 b)` (first pair whose car satisfies the predicate, or nil) | | `getf` | `(getf '(:a 1 :b 2) :b)` | `2` (value following the indicator in a property list, or nil; the partner of `remf`. Two arguments only: no `&optional default`) | | `last` | `(last '(1 2 3))`, `(last '(1 2 3) 2)` | `(3)`, `(2 3)` (last cons cell, or the last `n` conses; nil for an empty list) | | `butlast` | `(butlast '(1 2 3))` | `(1 2)` (copy without the last element; nil for an empty or single-element list) | | `remove` | `(remove 2 '(1 2 3 2))` | `(1 3)` (new list without items `eql` to the given one; optional `:test`/`:key` keywords) | | `remove-if` | `(remove-if #'evenp '(1 2 3 4))` | `(1 3)` (new list without items satisfying the predicate) | | `remove-if-not` | `(remove-if-not #'evenp '(1 2 3 4))` | `(2 4)` (new list keeping only items satisfying the predicate) | | `remove-duplicates` | `(remove-duplicates '(1 2 1 3))` | `(2 1 3)` (copy with duplicate elements removed, keeping the last occurrence; `eql` compare by default, optional `:test`/`:key` keywords, `:from-end t` keeps the first occurrence) | | `delete-duplicates` | `(delete-duplicates '(1 2 1 3) :from-end t)` | `(1 2 3)` (`remove-duplicates`' would-be-destructive twin, same rendering and keywords — the standard requires using the result) | | `delete` | `(delete 2 '(1 2 3 2))` | `(1 3)` (destructive `remove`: splices out matching cells in place; optional `:test`/`:key` keywords; use the return value since the head may change) | | `delete-if` | `(delete-if #'evenp '(1 2 3 4))` | `(1 3)` (destructive `remove-if`) | | `delete-if-not` | `(delete-if-not #'evenp '(1 2 3 4))` | `(2 4)` (destructive `remove-if-not`) | | `subst` | `(subst 'x 'a '(a (b a) c))` | `(x (b x) c)` (non-destructive tree substitution; optional `:test`/`:key` keywords) | | `search` | `(search "bc" "abcd")` | `1` (position of one sequence inside another, or nil; `:start1`/`:end1`/`:start2`/`:end2`/`:test`/`:key`/`:from-end`) | | `mismatch` | `(mismatch "apple" "apricot")` | `2` -- the index into the first sequence where the two differ, or nil; same keywords as `search` | | `tree-equal` | `(tree-equal '(1 (2 3)) '(1 (2 3)))` | `t` (same tree shape with leaves matching under `:test` (default `eql`) or `:test-not`) | | `substitute` | `(substitute 0 2 '(1 2 3 2))` | `(1 0 3 0)` (copy with every element `eql` to the old item replaced by the new one; optional `:test`/`:key` keywords) | | `nsubstitute` | `(nsubstitute 0 2 '(1 2 3 2))` | `(1 0 3 0)` (destructive `substitute`: rewrites matching cars in place; optional `:test`/`:key` keywords) | | `substitute-if` | `(substitute-if 0 #'oddp '(1 2 3))` | `(0 2 0)` (copy with every element satisfying the predicate replaced; optional `:key`, no `:test`) | | `substitute-if-not` | `(substitute-if-not 0 #'oddp '(1 2 3))` | `(1 0 3)` (the complement of `substitute-if`) | | `nsubstitute-if` | `(nsubstitute-if 0 #'oddp (list 1 2 3))` | `(0 2 0)` (destructive `substitute-if`; lists only) | | `nsubstitute-if-not` | `(nsubstitute-if-not 0 #'oddp (list 1 2 3))` | `(1 0 3)` (destructive `substitute-if-not`; lists only) | | `get-setf-expansion` | `(get-setf-expansion 'x)` | the five setf-expansion values, consumed with `multiple-value-bind` (lite: variable and accessor places) | | `nconc` | `(nconc (list 1 2) (list 3 4) (list 5))` | `(1 2 3 4 5)` (destructively concatenate any number of lists; returns the first non-`nil` argument) | | `copy-list` | `(copy-list '(1 2 3))` | `(1 2 3)` (shallow copy of a list) | | `copy-tree` | `(copy-tree '(1 (2 3)))` | `(1 (2 3))` (deep copy of a cons tree) | | `nreverse` | `(nreverse '(1 2 3))` | `(3 2 1)` (destructively reverse a list by rewiring each `cdr`; use the return value) | | `make-list` | `(make-list 3 :initial-element 0)` | `(0 0 0)` (list of n cells sharing the one element value; `nil` by default) | | `union` | `(union '(1 2 3) '(2 3 4))` | `(4 1 2 3)` (set union, `eql` compare by default, optional `:test`/`:key` keywords; result order unspecified) | | `intersection` | `(intersection '(1 2 3) '(2 3 4))` | `(3 2)` (set intersection, `eql` compare by default, optional `:test`/`:key` keywords; result order unspecified) | | `set-difference` | `(set-difference '(1 2 3) '(2))` | `(3 1)` (elements of the first list not in the second, `eql` compare by default, optional `:test`/`:key` keywords; result order unspecified) | | `set-exclusive-or` | `(set-exclusive-or '(1 2 3) '(2 3 4))` | `(1 4)` (symmetric difference: the elements of either list with no match in the other; optional `:test`/`:test-not`/`:key` keywords; result order unspecified) | | `adjoin` | `(adjoin 1 '(2 3))` | `(1 2 3)` (prepend the item unless already a member; `eql` compare by default, optional `:test`/`:key` keywords) | | `list*` | `(list* 1 2 '(3 4))`, `(list* 1 2 3)` | `(1 2 3 4)`, `(1 2 . 3)` (cons the leading arguments onto the last one as the tail) | | `acons` | `(acons 'a 1 nil)` | `((a . 1))` (prepend a `(key . value)` pair to an alist) | | `endp` | `(endp nil)`, `(endp '(1))` | `t`, `nil` (end-of-list test; a synonym for `null`, the improper-list error is relaxed) | | `elt` | `(elt '(a b c) 1)` | `b` (0-based element access; lists only, no string indexing) | | `rassoc` | `(rassoc 2 '((a . 1) (b . 2)))` | `(b . 2)` (first pair whose cdr matches the value, or nil; `eql` compare by default, optional `:test`/`:key` keywords) | | `rassoc-if` | `(rassoc-if #'oddp '((a . 2) (b . 3)))` | `(b . 3)` (first pair whose cdr satisfies the predicate, or nil) | | `pairlis` | `(pairlis '(a b) '(1 2))` | `((a . 1) (b . 2))` (pair up a list of keys and a list of values into an alist; an optional third argument is appended as the tail) | | `copy-alist` | `(copy-alist '((a . 1)))` | `((a . 1))` (copy an alist's spine and its pair cells; the keys and values themselves are shared) | | `revappend` | `(revappend '(1 2 3) '(4 5))` | `(3 2 1 4 5)` (reverse the first list and append the second) | | `nreconc` | `(nreconc '(1 2 3) '(4 5))` | `(3 2 1 4 5)` (destructive `revappend`: expands to `(nconc (nreverse x) y)`, reusing the cons cells of the first list) | | `maplist` | `(maplist #'identity '(1 2 3))` | `((1 2 3) (2 3) (3))` (apply to successive tails, collect results; takes any number of lists, stopping at the shortest) | | `mapcon` | `(mapcon (lambda (x) (list (car x))) '(1 2 3))` | `(1 2 3)` (apply to successive tails, concatenate the result lists; takes any number of lists) | | `mapl` | `(mapl #'identity '(1 2 3))` | `(1 2 3)` (apply to successive tails for effect, return the first list; takes any number of lists) | | `sort` | `(sort '(3 1 2) #'<)` | `(1 2 3)` (destructively sort a list with a comparison predicate; not stable) | | `merge` | `(merge 'list (list 1 3) (list 2 4) #'<)` | `(1 2 3 4)` (one sorted sequence from two, stable; the result type is built by `coerce`, so `list`/`vector`/`string`; non-destructive) | | `rplaca` | `(rplaca x val)` | Destructively replace car of cons cell, return the cell | | `rplacd` | `(rplacd x val)` | Destructively replace cdr of cons cell, return the cell | | `1+` | `(1+ 41)` | `42` (same as `(+ x 1)`) | | `1-` | `(1- 43)` | `42` (same as `(- x 1)`) | | `zerop` | `(zerop 0)` | `t` | | `plusp` | `(plusp 3)` | `t` | | `minusp` | `(minusp -3)` | `t` | | `evenp` | `(evenp 4)` | `t` | | `oddp` | `(oddp 3)` | `t` | | `abs` | `(abs -5)`, `(abs -3.14)` | `5`, `3.14` | | `min` | `(min 3 5)`, `(min 5 2 8 1)` | `3`, `1` (variadic) | | `max` | `(max 3 5)`, `(max 5 2 8 1)` | `5`, `8` (variadic) | | `float` | `(float 42)` | `42.0` (convert to double) | | `truncate` | `(truncate 3.7)`, `(truncate -7 2)` | `3`, `-3` (toward zero; with a divisor, the quotient of the division -- the remainder is observable through `multiple-value-bind`) | | `floor` | `(floor 3.7)`, `(floor 7 2)` | `3`, `3` (toward negative infinity; with a divisor, the quotient of the division -- the remainder is observable through `multiple-value-bind`) | | `ceiling` | `(ceiling 3.2)`, `(ceiling 7 2)` | `4`, `4` (toward positive infinity; with a divisor, the quotient of the division) | | `round` | `(round 3.5)`, `(round 2.5)` | `4`, `2` (banker's rounding; an optional divisor rounds the quotient of the division) | | `sqrt` | `(sqrt 16)`, `(sqrt 2)` | `4.0`, `1.4142135623730951` (always a float) | | `isqrt` | `(isqrt 17)` | `4` (integer square root, floor of the real root) | | `expt` | `(expt 2 10)`, `(expt 2.0 3)` | `1024`, `8.0` | | `random` | `(random 100)`, `(random 1.0)` | a value in `[0, 100)` / `[0.0, 1.0)` (the result type follows the limit; `(random 1)` is always `0`). The interpreter and JVM draw from `Math.random`; WASM draws real entropy from the WASI `random_get` host function in Preview 1 mode and `wasi:random@0.3.0` in `--component` mode, so the sequence differs each run | | `make-random-state` | `(make-random-state t)` | always `nil` -- no random-state objects exist; `random` accepts and ignores an optional state argument, so the store-and-pass-back seeding idiom works unchanged | | `get-universal-time` | `(get-universal-time)` | seconds since 1900-01-01 GMT, as an integer on every backend (WASM reads the real host clock in Preview 1, `wasi:clocks@0.3.0` in `--component` mode) | | `encode-universal-time` | `(encode-universal-time 0 0 0 1 1 1970 0)` | `2208988800` -- decoded components to universal time; a missing time zone means GMT, not the local zone | | `decode-universal-time` | `(decode-universal-time 2208988800 0)` | the nine decoded values (second, minute, hour, date, month, year, day-of-week, daylight-p, zone); `daylight-p` is always nil | | `get-internal-real-time` | `(get-internal-real-time)` | elapsed real time in milliseconds (integer on every backend) | | `get-internal-run-time` | `(get-internal-run-time)` | consumed run time in milliseconds (integer on every backend) | | `sleep` | `(sleep 0.5)` | block for a non-negative number of seconds and return `nil` (a real host timer everywhere but WASM Preview 1, which busy-waits on the clock, and `--no-wasi`, which signals) | | `lisp-implementation-type` `lisp-implementation-version` `software-type` `software-version` `machine-type` `machine-version` `machine-instance` `short-site-name` `long-site-name` | `(lisp-implementation-type)` | `"rontolisp"` — the environment enquiry constants: the version is the build's, `software-type` is `"Unix"`, `machine-type` is the ABI targeted (`"JVM"` / `"WASM32"`), and everything rontolisp cannot know is `nil` | | `user-homedir-pathname` | `(user-homedir-pathname)` | The `HOME` directory as a DIRECTORY pathname (trailing separator), or `nil` when the variable is unset | | `invoke-debugger` | `(invoke-debugger c)` | Signals the condition and never returns -- no backend has a debugger to enter | | `compile-file` `compile-file-pathname` `remove-method` | `(compile-file "x.lisp")` | Exist and signal: a rontolisp program is compiled whole (no fasl, no pathname naming one) and a method is not a first-class object | | `exp` | `(exp 0)` | `1.0` (interpreter/JVM use `Math.exp`; WASM uses a software approximation) | | `log` | `(log 1)` | `0.0` (natural log; interpreter/JVM use `Math.log`, WASM a software approximation) | | `sin` `cos` `tan` | `(sin 0)`, `(cos 0)` | `0.0`, `1.0` (interpreter/JVM use `Math.sin`/`cos`/`tan`, WASM a software approximation) | | `asin` `acos` `atan` | `(atan 0)` | `0.0` (all backends -- WASM uses a software approximation) | | `sinh` `cosh` `tanh` | `(tanh 0)` | `0.0` (all backends -- WASM derives all three from its software `exp`) | | `gcd` | `(gcd 12 18)`, `(gcd 24 36 60)` | `6`, `12` (variadic; greatest common divisor, `(gcd)` is `0`) | | `lcm` | `(lcm 4 6)`, `(lcm 2 3 4)` | `12`, `12` (variadic; least common multiple; `0` if any argument is `0`, `(lcm)` is `1`) | | `signum` | `(signum -5)`, `(signum 3.5)` | `-1`, `1.0` (sign, preserving integer/float type) | | `logand` | `(logand 12 10)`, `(logand 12 10 6)` | `8`, `0` (variadic bitwise AND; `(logand)` is `-1`) | | `logior` | `(logior 12 10)`, `(logior 1 2 4 8)` | `14`, `15` (variadic bitwise inclusive OR; `(logior)` is `0`) | | `logxor` | `(logxor 12 10)` | `6` (variadic bitwise exclusive OR; `(logxor)` is `0`) | | `lognot` | `(lognot 5)` | `-6` (bitwise NOT, i.e. ones' complement) | | `logandc1` | `(logandc1 12 10)` | `2` (AND of the complement of the first argument with the second) | | `logandc2` | `(logandc2 12 10)` | `4` (AND of the first argument with the complement of the second) | | `logorc1` | `(logorc1 12 10)` | `-5` (OR of the complement of the first argument with the second) | | `logorc2` | `(logorc2 12 10)` | `-3` (OR of the first argument with the complement of the second) | | `ash` | `(ash 1 4)`, `(ash 255 -4)` | `16`, `15` (arithmetic shift: left for a non-negative count, right otherwise) | | `logtest` | `(logtest 1 3)`, `(logtest 1 2)` | `T`, `NIL` (any bits set in common; `(not (zerop (logand a b)))`) | | `funcall` | `(funcall #'+ 3 4)` | Apply a function to args. Accepts a function value (`#'f`, a lambda) or a symbol naming a function (`(funcall 'car ...)`) | | `mapcar` | `(mapcar #'car '((1 2) (3 4)))` | Apply a function to each element, return new list | | `map` | `(map 'list #'+ '(1 2 3) '(10 20 30))` | `(11 22 33)` (map over sequences -- list/string -- up to the shortest, building a `'list`/`'string` result, or nil for effect) | | `mapc` | `(mapc #'print '(1 2 3))` | Apply a function to each element for effect, return the first list; takes any number of lists, stopping at the shortest | | `mapcan` | `(mapcan (lambda (x) (list x x)) '(1 2))` | `(1 1 2 2)` (apply a function and concatenate the result lists; takes any number of lists, and uses non-destructive `append`) | | `apply` | `(apply #'+ 1 2 '(3 4))` | `10` (apply a function to the leading args plus the spread final list) | | `values` | `(values 1 2 3)`, `(multiple-value-list (values 1 2 3))` | `1`, `(1 2 3)` -- an ordinary context keeps the primary value only; `multiple-value-bind`/`-list`/`-call`/`nth-value` receive all values of a literal `(values ...)` call, the multi-value built-ins (`floor` family, `gethash`, `parse-integer`, `values-list`) and a user function returning `(values ...)` | | `reduce` | `(reduce #'+ '(1 2 3) :initial-value 0)` | Left fold: `(f (f (f init a) b) c)`. Plain form `(reduce f list)` uses the first element as init; the `:initial-value` keyword (literal) supplies an explicit seed | | `every` | `(every #'evenp '(2 4 6))`, `(every #'< '(1 2) '(3 4))` | `t` if the predicate is non-nil for every element (tuple), else `nil`; any number of sequences, stopping at the shortest | | `some` | `(some #'oddp '(2 4 5))`, `(some #'> '(1 5) '(3 4))` | The first non-nil predicate result, or `nil` if every element (tuple) fails; any number of sequences | | `notany` | `(notany #'evenp '(1 3 5))` | `t` if the predicate is nil for every element (tuple), else `nil` (the complement of `some`) | | `notevery` | `(notevery #'evenp '(2 4 5))` | `t` if the predicate is nil for some element (tuple), else `nil` (the complement of `every`) | | `symbol-function` | `(symbol-function 'car)` | Return the function named by a symbol (compilers: the argument must be a quoted symbol literal) | | `identity` | `(identity 42)` | `42` (return the argument unchanged) | | `constantly` | `(mapcar (constantly 7) '(a b c))` | `(7 7 7)` (a function of any arguments answering one fixed value) | | `make-hash-table` | `(make-hash-table)`, `(make-hash-table :test 'equal)` | Create an empty hash table. `:test` is accepted but informational (see the note below); other keywords such as `:size` are ignored | | `gethash` | `(gethash key table)`, `(gethash key table default)` | Return the value stored under `key`, or `default` (nil if omitted) when absent | | `(setf (gethash key table) v)` | `(setf (gethash "a" h) 1)` | Store `v` under `key`; works with `incf`/`decf`/`push` on the place | | `remhash` | `(remhash key table)` | Remove the entry for `key`; returns `t` if one was removed, else `nil` | | `clrhash` | `(clrhash table)` | Remove all entries; returns the table | | `hash-table-count` | `(hash-table-count table)` | The number of entries | | `hash-table-test` | `(hash-table-test table)` | Always `EQUAL`: every backend keys structurally, whatever `:test` was requested | | `hash-table-size` | `(hash-table-size table)` | The entry count (a rontolisp table has no separate capacity) | | `hash-table-rehash-size` | `(hash-table-rehash-size table)` | The standard default `1.5` (growth belongs to the host map) | | `hash-table-rehash-threshold` | `(hash-table-rehash-threshold table)` | The standard default `1.0` | | `hash-table-p` | `(hash-table-p x)` | `t` if `x` is a hash table, else `nil` | | `maphash` | `(maphash (lambda (k v) ...) table)` | Call the function on each key/value pair for effect; returns nil | | `make-array` | `(make-array 5 :initial-element 0)`, `(make-array (list 2 3))` | Create an array of any rank; `:initial-element` sets every cell (nil if omitted). `:element-type` may be computed | | `aref` | `(aref a i)`, `(aref a i j)` | Return the element at the given subscripts | | `(setf (aref a i j) v)` | `(setf (aref a 0 0) 1)` | Store `v` at the subscripts; works with `incf`/`decf`/`push` on the place | | `vector` | `(vector 1 2 3)` | `#(1 2 3)` (a fresh rank-1 array of the arguments) | | `svref` | `(svref (vector 10 20 30) 1)` | `20` (vector element access; also a `setf` place) | | `array-dimensions` | `(array-dimensions (make-array (list 2 3)))` | `(2 3)` (the dimension sizes as a list) | | `array-dimension` | `(array-dimension (make-array (list 2 3)) 1)` | `3` (the size of one axis, 0-based) | | `array-rank` | `(array-rank (vector 1 2))` | `1` (`2` for a rank-2 array, and so on) | | `array-total-size` | `(array-total-size (make-array (list 2 3)))` | `6` (the total element count) | | `row-major-aref` | `(row-major-aref (make-array (list 2 3)) 4)` | The element at a flat row-major index, independent of rank; also a `setf` place | | `array-row-major-index` | `(array-row-major-index (make-array (list 2 3)) 1 1)` | `4` (the flat row-major index of the subscripts) | | `coerce` | `(coerce '(1 2 3) 'vector)`, `(coerce "ab" 'list)` | `#(1 2 3)`, `(#\a #\b)`; the `'list`/`'vector`/`'string` and float families, `t`, and a computed result type | | `fill-pointer` | `(fill-pointer v)` | The fill pointer of a `:fill-pointer` vector (its effective length); also a `setf` place | | `array-has-fill-pointer-p` | `(array-has-fill-pointer-p a)` | `t` if the array has a fill pointer, else `nil` | | `adjustable-array-p` | `(adjustable-array-p a)` | `t` if the array was created `:adjustable`, else `nil` | | `array-element-type` | `(array-element-type a)` | Always `t` (element types are not tracked) | | `vector-push` | `(vector-push x v)` | Store `x` at the fill pointer and return the index, or `nil` when full | | `vector-pop` | `(vector-pop v)` | Decrement the fill pointer and return the element it passed | | `vector-push-extend` | `(vector-push-extend x v &optional ext)` | Like `vector-push` but grows the vector when full | | `subtypep` | `(subtypep 'integer 'number)` | `t` -- the built-in type lattice plus `defclass`/condition hierarchies; a single value, unknown pairs answer `nil`; the compilers fold literal specifiers at compile time | | `mask-field` | `(mask-field (byte 4 4) 255)` | `240` -- the `ldb` field left in its original position | | `scale-float` | `(scale-float 1.5 3)` | `12.0` -- `float × 2^n` with IEEE semantics | | `decode-float` | `(decode-float 6.5)` | `0.8125`, `3`, `1.0` -- significand in [1/2, 1), binary exponent, sign | | `char-name` | `(char-name #\Space)` | `"Space"` -- `nil` for graphic characters | | `fdefinition` | `(fdefinition 'car)` | the function value, like `symbol-function` | | `use-package` | `(use-package :mypkg)` | add packages to a package's use list, so their external symbols are visible unqualified (a literal top-level call is a compile-time directive) | | `export` | `(export '(run))` | make symbols external in a package (a literal top-level call is a compile-time directive) | | `unexport` | `(unexport 'run)` | the inverse of `export`: the symbol stays present but is no longer visible unqualified | | `import` | `(import 'other:sym)` | make another package's symbol accessible unqualified -- the runtime form of `:import-from` (a literal top-level call is a compile-time directive) | | `file-position` | `(file-position s)` | always `nil` (lite: streams do not support repositioning) | | `file-length` | `(file-length s)` | the byte length of the file a file stream is open on; `nil` for any other stream, and `nil` on both WASM backends | | `file-write-date` | `(file-write-date "x.txt")` | the file's modification time as a universal time; `nil` when it cannot be determined (always `nil` on both WASM backends) | | `ensure-directories-exist` | `(ensure-directories-exist "logs/app.log")` | create the pathspec's directory component and return the pathspec (signals on both WASM backends) | | `delete-file` | `(delete-file "notes.txt")` | delete the named file and return `t`; anything that leaves it in place signals, "it was not there" included (signals on both WASM backends, like `ensure-directories-exist` and for the same reason) | | `rename-file` | `(rename-file "notes.txt" "notes.bak")` | rename (move) the file and return the defaulted new name as a pathname; the new name is merged with the old one, so a bare file name keeps the directory. Anything that leaves the file in place signals, "it was not there" included (signals on both WASM backends, like `delete-file`) | | `make-string-output-stream` | `(make-string-output-stream)` | a fresh string output stream -- the explicit form of what `with-output-to-string` builds | | `make-string-input-stream` | `(make-string-input-stream string &optional start end)` | an input stream reading from a string -- the explicit form of what `with-input-from-string` binds | | `get-output-stream-string` | `(get-output-stream-string s)` | everything written to a string output stream so far, CLEARING it (CL's contract) | | `make-synonym-stream` | `(make-synonym-stream '*standard-output*)` | a stream forwarding every operation to the stream the named variable holds AT THAT MOMENT, for any symbol -- so rebinding the variable afterwards redirects it | | `synonym-stream-symbol` | `(synonym-stream-symbol s)` | the symbol a synonym stream forwards to | | `make-broadcast-stream` | `(make-broadcast-stream a b)` | an output stream fanning every write out to each component, in order; with no components, a discarding sink. A stream WITH components is a Gray stream and takes the whole output protocol | | `pathnamep` | `(pathnamep #P"/tmp/x")` | `t` — whether the value is a pathname (the value `#P"..."` denotes); a string is NOT one, and it agrees with `(typep x 'pathname)` | | `input-stream-p` | `(input-stream-p s)` | `t` for any stream handle | | `output-stream-p` | `(output-stream-p s)` | `t` for any stream handle | | `stream-element-type` | `(stream-element-type s)` | always `character` -- every stream is a character stream | | `class-of` | `(class-of 42)` | The value's class metaobject, `eq` to `(find-class 'integer)`; built-ins, CLOS and struct instances alike | | `type-of` | `(type-of 42)` | `integer` -- the type NAME symbol: a struct/CLOS instance answers its structure/class name, agreeing with `(class-name (class-of x))` | | `simple-condition-format-control` | `(simple-condition-format-control c)` | the condition's `:format-control` slot, or `nil` | | `simple-condition-format-arguments` | `(simple-condition-format-arguments c)` | the condition's `:format-arguments` slot, or `nil` | | `type-error-datum` | `(type-error-datum c)` | the `datum` slot of a `type-error` -- the object whose type was wrong | | `type-error-expected-type` | `(type-error-expected-type c)` | the `expected-type` slot of a `type-error` | | `cell-error-name` | `(cell-error-name c)` | the `name` slot of a `cell-error` (`unbound-variable`, `undefined-function`, `unbound-slot`) | | `unbound-slot-instance` | `(unbound-slot-instance c)` | the object whose slot was unbound | | `print-object` | `(print-object obj stream)` | the generic function the printer consults; define a method to control how instances of a type print | | `find-restart` | `(find-restart 'retry c)` | the innermost active restart with that name as a first-class object, or `nil`. Lite: the condition argument is ignored | | `invoke-restart` | `(invoke-restart :reconnect host)` | invoke a restart by name (symbol or keyword) or object, with arguments; a `restart-case` restart transfers control to its clause | | `compute-restarts` | `(compute-restarts)` | every active restart record, innermost first | | `restart-name` | `(restart-name r)` | the name of a restart object | | `muffle-warning` | `(muffle-warning w)` | invoke the `muffle-warning` restart a `warn` establishes, aborting the warning before it prints | | `abort` | `(abort)` | invoke the innermost `abort` restart; an error when none is active | | `continue` | `(continue)` | invoke the innermost `continue` restart (a `cerror`'s); `nil` when none is active | | `use-value` | `(use-value v)` | invoke the innermost `use-value` restart with a value; `nil` when none is active | | `store-value` | `(store-value v)` | invoke the innermost `store-value` restart with a value; `nil` when none is active | ## rontolisp Package Functions The `rontolisp` package provides implementation-specific functions that are **not part of Common Lisp**. Reference them with the `rontolisp:` qualifier (or unqualified after `(in-package rontolisp)`); see [Packages](packages.md) for the package system. Each name below links to its own page. | Function | Example | Result | |----------|---------|--------| | `rontolisp:version` | `(rontolisp:version)` | a property list of build info (`:version`, `:build-timestamp`, `:git-commit`, `:git-branch`) | | `rontolisp:random-bytes` | `(rontolisp:random-bytes 16)` | a vector of cryptographically strong random bytes (`SecureRandom` / WASI `random_get`) | | `rontolisp:make-mutex` | `(rontolisp:make-mutex)` | a fresh mutual-exclusion lock, as an opaque handle (real on the interpreter and the JVM, a no-op on WASM) | | `rontolisp:mutex-acquire` | `(rontolisp:mutex-acquire m)` | block until this thread holds the mutex; returns it (prefer `rontolisp:with-mutex`) | | `rontolisp:mutex-release` | `(rontolisp:mutex-release m)` | release one acquisition of the mutex; returns it | | `rontolisp:make-thread` | `(rontolisp:make-thread fn bindings)` | spawn a virtual thread running the zero-argument function, with optional `(symbol . value)` dynamic bindings established in it; returns an opaque handle (interpreter and JVM; the WASM shims signal) | | `rontolisp:join-thread` | `(rontolisp:join-thread th)` | wait for the thread and yield its function's value; an error it died on is re-signaled here | | `rontolisp:threadp` | `(rontolisp:threadp v)` | `t` if the value is a thread handle | | `rontolisp:thread-alive-p` | `(rontolisp:thread-alive-p th)` | `t` while the thread is still running (`nil` after a join) | | `rontolisp:destroy-thread` | `(rontolisp:destroy-thread th)` | interrupt the thread; returns the handle | | `rontolisp:current-thread` | `(rontolisp:current-thread)` | the calling thread's own handle, `eq`-stable per thread (works for any thread, not only `make-thread` spawns) | | `rontolisp:list-functions` | `(rontolisp:list-functions :cl)` | the function symbols of a package, sorted (defaults to `:cl`) | | `rontolisp:list-macros` | `(rontolisp:list-macros)` | the macro symbols of a package, sorted | | `rontolisp:list-special-forms` | `(rontolisp:list-special-forms)` | the special-form symbols of a package, sorted | | `rontolisp:fetch` | `(rontolisp:fetch "http://example.com/")` | start an HTTP request asynchronously; returns a future | | `rontolisp:futurep` | `(rontolisp:futurep v)` | `t` if the value is a future (as returned by calling an `async-defun` function, `rontolisp:fetch`, `rontolisp:stream-read`, ...) | | `rontolisp:streamp` | `(rontolisp:streamp v)` | `t` if the value is an asynchronous stream (a different predicate from `cl:streamp`, which answers file streams) | | `rontolisp:make-stream` | `(rontolisp:make-stream)` | create a fresh open asynchronous stream; one value owns both the read and the write end | | `rontolisp:stream-read` | `(rontolisp:stream-read s)` | a future settling to the stream's next chunk, or `nil` at end of stream | | `rontolisp:stream-write` | `(rontolisp:stream-write s "chunk")` | append a chunk (never `nil`); returns a future that settles when the stream accepted it | | `rontolisp:stream-close` | `(rontolisp:stream-close s)` | close the write end; buffered chunks stay readable, then reads observe end of stream | | `rontolisp:read-all` | `(rontolisp:read-all s)` | a future settling to the remaining chunks drained into one string (octet chunks -- every HTTP body stream's -- UTF-8 decoded) | | `rontolisp:wait-for` | `(rontolisp:wait-for 100)` | a future settling to `nil` after the given milliseconds; the async counterpart of `cl:sleep` | | `rontolisp:then` | `(rontolisp:then f (lambda (v) (* 2 v)))` | attach a transform to a future as a value; returns a fresh future on the success channel (JavaScript `.then`) | | `rontolisp:then*` | `(rontolisp:then* f #'1+ #'1+)` | variadic chain sugar for `rontolisp:then`; each function receives the previous stage's flattened value | | `rontolisp:catch` | `(rontolisp:catch f (lambda (c) :fallback))` | attach an error fallback to a future as a value (JavaScript `.catch`); distinct from `cl:catch`/`throw` | | `rontolisp:finally` | `(rontolisp:finally f (lambda () (cleanup)))` | run a cleanup thunk on both success and error channels; the original outcome carries through | | `rontolisp:http-handler` | `(rontolisp:http-handler 'handle 8080)` | serve HTTP requests with a handler function taking the Clack environment plist and returning `(status headers body)` (a blocking server; a `wasi:http` component under `--component`) | | `rontolisp:json-parse` | `(rontolisp:json-parse "{\"n\": 1}")` | parse a JSON string (jzon-compatible): objects become hash tables with string keys, arrays vectors | | `rontolisp:json-stringify` | `(rontolisp:json-stringify (vector 1 2))` | serialize a value to a JSON string (hash tables and CLOS instances become objects, lists and vectors arrays) | | `rontolisp:plist-hash-table` | `(rontolisp:plist-hash-table (list :n 1))` | build a hash table from a property list (subset of `alexandria:plist-hash-table`); handy for JSON objects | | `rontolisp:hash-table-plist` | `(rontolisp:hash-table-plist h)` | property list of a hash table's pairs (subset of `alexandria:hash-table-plist`) | | `rontolisp:alist-hash-table` | `(rontolisp:alist-hash-table al)` | build a hash table from an association list (subset of `alexandria:alist-hash-table`) | | `rontolisp:hash-table-alist` | `(rontolisp:hash-table-alist h)` | association list of a hash table's pairs (subset of `alexandria:hash-table-alist`) | | `rontolisp:alist-plist` | `(rontolisp:alist-plist al)` | property list with an association list's keys and values, order preserved (subset of `alexandria:alist-plist`) | | `rontolisp:plist-alist` | `(rontolisp:plist-alist pl)` | association list with a property list's keys and values, order preserved (subset of `alexandria:plist-alist`) | | `rontolisp:tcp-connect` | `(rontolisp:tcp-connect "127.0.0.1" 7777)` | open a blocking TCP connection; returns a bidirectional stream handle | | `rontolisp:tcp-listen` | `(rontolisp:tcp-listen 7777)`, `(rontolisp:tcp-listen 0 "127.0.0.1")` | bind a listening TCP socket and return a listener handle; port `0` picks a free ephemeral port | | `rontolisp:tcp-accept` | `(rontolisp:tcp-accept listener)` | wait for a client connection (blocking); returns a bidirectional stream handle | | `rontolisp:tcp-local-port` | `(rontolisp:tcp-local-port listener)` | the local port a listener or socket is actually bound to | | `rontolisp:tcp-local-address` | `(rontolisp:tcp-local-address listener)` | the local IP address a listener or socket is bound to, as a string | | `rontolisp:tcp-peer-address` | `(rontolisp:tcp-peer-address sock)` | the remote IP address of a connected socket, as a string | | `rontolisp:tcp-peer-port` | `(rontolisp:tcp-peer-port sock)` | the remote port of a connected socket | | `rontolisp:tcp-set-timeout` | `(rontolisp:tcp-set-timeout sock 5000)` | set a read deadline in milliseconds (`nil` clears); a timed-out read signals a catchable error | | `rontolisp:tls-connect` | `(rontolisp:tls-connect "example.com" 443)` | open an encrypted (TLS) client connection; returns the same kind of stream handle as `tcp-connect` | | `rontolisp:tls-listen` | `(rontolisp:tls-listen "server.p12" "changeit" 8443)` | bind an encrypted listening socket from a PKCS12 keystore; accept with `tcp-accept` | | `rontolisp:tls-listen-pem` | `(rontolisp:tls-listen-pem "cert.pem" "key.pem" 8443)` | bind an encrypted listening socket from PEM certificate/key files | | `rontolisp:tls-upgrade` | `(rontolisp:tls-upgrade sock "example.com")` | wrap an already-connected stream handle in TLS as a client; returns a new stream handle | | `rontolisp:wasm-export` | `(rontolisp:wasm-export 'fact :params '(:int) :returns :int)` | mark a `defun` as host-callable when compiling to a WASM core module | | `rontolisp:wasm-import` | `(rontolisp:wasm-import 'add :from "host" :params '(:int :int) :returns :int)` | declare a host function callable from Lisp when compiling to a WASM core module | | `rontolisp:wit-export` | `(rontolisp:wit-export "greeter.wit" :world greeter)` | declare that the program implements a WIT world: its exports are checked against the program's `defun`s, and their types come from the WIT | | `rontolisp:wit-import` | `(rontolisp:wit-import "store.wit" :interface "wasi:keyvalue/store@0.2.0" :package kv)` | declare that the program calls a WIT interface: every function it declares is bound as an ordinary Lisp function (`kv:bucket-get`), against a provider on the interpreter/JVM, a WASM import on Preview 1, and a `canon lower`ed component-model import under `--component`, where the host is the provider | | `rontolisp:wit-provide` | `(rontolisp:wit-provide "wasi:keyvalue/store@0.2.0" #'my-store)` | bind the implementation of a `wit-import`ed interface on the interpreter and JVM backends (inert on WASM, where the host provides it) | The introspection functions (`list-functions` / `list-macros` / `list-special-forms`) are described in detail under [Package introspection](packages.md#package-introspection). `rontolisp:fetch` starts an outgoing HTTP request and returns a future, resolved with `rontolisp:await`; see the [HTTP Requests guide](../guides/http-fetch.md) for a worked overview, and the [fetch](functions/rontolisp-fetch.md), [await](special-forms/rontolisp-await.md) and [futurep](functions/rontolisp-futurep.md) reference pages for options, the result plist, backend support, and limitations. `rontolisp:http-handler` is the incoming counterpart of `fetch` -- it serves HTTP requests with a handler function over the Clack environment plist and `(status headers body)` response list; see the [Serving HTTP guide](../guides/http-handler.md) for a worked example on every backend, and the [http-handler](functions/rontolisp-http-handler.md) reference page for backend support and limitations. `rontolisp:json-parse` and `rontolisp:json-stringify` convert between JSON documents and Lisp values (a lightweight, `com.inuoe.jzon`-compatible subset) -- for example to parse a fetch response body; see the [json-parse](functions/rontolisp-json-parse.md) and [json-stringify](functions/rontolisp-json-stringify.md) reference pages for the value mapping and limitations. The tcp functions (`rontolisp:tcp-connect` / `tcp-listen` / `tcp-accept` / `tcp-local-port` and the [address accessors](functions/rontolisp-tcp-addresses.md)) open plain TCP sockets whose handles work with the standard stream functions (`read-line` / `write-line` / `read-byte` / `write-byte` / `close`); see the [TCP Sockets guide](../guides/tcp-sockets.md) for a worked echo server, and the [tcp-connect](functions/rontolisp-tcp-connect.md), [tcp-listen](functions/rontolisp-tcp-listen.md), [tcp-accept](functions/rontolisp-tcp-accept.md) and [tcp-local-port](functions/rontolisp-tcp-local-port.md) reference pages for backend support and limitations. A [usocket-compatible shim](#usocket-package-functions) is layered over them for portability with existing Common Lisp code. The TLS variants (`rontolisp:tls-connect` / `tls-upgrade` / `tls-listen` / `tls-listen-pem`) wrap the same stream handles in TLS; see the [tls-connect](functions/rontolisp-tls-connect.md), [tls-upgrade](functions/rontolisp-tls-upgrade.md), [tls-listen](functions/rontolisp-tls-listen.md) and [tls-listen-pem](functions/rontolisp-tls-listen-pem.md) reference pages. `rontolisp:wasm-export`, `rontolisp:wasm-import`, `rontolisp:wit-export` and `rontolisp:wit-import` are compile-time directives; the WIT pair take a `.wit` file as the single source of truth for a boundary, so the types are never hand-written. `wit-export` declares that the program **implements** a WIT world (and `--scaffold-wit` generates the implementation's skeleton from it); `wit-import` declares that it **calls** a WIT interface, binding every function the interface declares as an ordinary Lisp function — dispatched on the interpreter and JVM backends to a *provider* ([`rontolisp:wit-provide`](functions/rontolisp-wit-provide.md)), and lowered to `rontolisp:wasm-import` on Preview 1 WASM, where the host is the provider, so one source runs on every backend. rontolisp ships **no provider for any interface**: it knows the provider mechanism, not what any particular interface is, so an implementation of a WIT interface is ordinary Lisp code. A WIT `result`'s error arm signals the `rontolisp:wit-error` condition, whose payload is read with `rontolisp:wit-error-payload`. See their [wasm-export](functions/rontolisp-wasm-export.md), [wasm-import](functions/rontolisp-wasm-import.md), [wit-export](functions/rontolisp-wit-export.md), [wit-import](functions/rontolisp-wit-import.md) and [wit-provide](functions/rontolisp-wit-provide.md) reference pages and the [Compiling to WebAssembly](../compiling/wasm.md) guide. ## linalg Package Functions The `linalg` package provides numpy-style vector and matrix operations over the built-in arrays (the elementwise operations and reductions work for any rank). It is **not part of Common Lisp**; reference its functions with the `linalg:` qualifier (the package does not use `cl`, so most programs stay in `cl-user` and call the qualified names). The package is implemented once in Lisp source and behaves identically on every backend, and its constructors build packed double-float arrays, so it computes in floating point (`det`, `inv` and `solve` run like numpy's). Each name below links to its own page; the [Vectors & Matrices guide](../guides/linear-algebra.md) gives an overview and worked examples. | Function | Example | Result | |----------|---------|--------| | `linalg:zeros` | `(linalg:zeros 3)`, `(linalg:zeros '(2 2))` | `#d(0.0 0.0 0.0)`, `#d((0.0 0.0) (0.0 0.0))` (shape: integer or `(rows cols)` list) | | `linalg:ones` | `(linalg:ones '(2 2))` | `#d((1.0 1.0) (1.0 1.0))` | | `linalg:full` | `(linalg:full '(2 2) 7)` | `#d((7.0 7.0) (7.0 7.0))` | | `linalg:zeros-like` | `(linalg:zeros-like #2A((1 2) (3 4)))` | `#d((0.0 0.0) (0.0 0.0))` (zeros with the input's shape and width) | | `linalg:eye` | `(linalg:eye 2)` | `#d((1.0 0.0) (0.0 1.0))` (the identity matrix) | | `linalg:arange` | `(linalg:arange 5)`, `(linalg:arange 2 10 2)` | `#d(0.0 1.0 2.0 3.0 4.0)`, `#d(2.0 4.0 6.0 8.0)` (stop exclusive; step may be negative) | | `linalg:linspace` | `(linalg:linspace 0 1 5)` | `#d(0.0 0.25 0.5 0.75 1.0)` (n evenly spaced values, inclusive) | | `linalg:from-list` | `(linalg:from-list '((1 2) (3 4)))` | `#d((1.0 2.0) (3.0 4.0))` (a flat list gives a vector) | | `linalg:to-list` | `(linalg:to-list (linalg:eye 2))` | `((1.0 0.0) (0.0 1.0))` | | `linalg:shape` | `(linalg:shape #2A((1 2 3) (4 5 6)))` | `(2 3)` | | `linalg:ndim` | `(linalg:ndim #2A((1 2) (3 4)))` | `2` (the number of dimensions; 0 for a number) | | `linalg:size` | `(linalg:size (linalg:eye 3))` | `9` (the total element count) | | `linalg:reshape` | `(linalg:reshape (linalg:arange 6) '(2 3))` | `#d((0.0 1.0 2.0) (3.0 4.0 5.0))` (row-major; one extent may be `-1` and is inferred) | | `linalg:flatten` | `(linalg:flatten (linalg:eye 2))` | `#d(1.0 0.0 0.0 1.0)` | | `linalg:transpose` | `(linalg:transpose #2A((1 2 3) (4 5 6)))` | `#d((1.0 4.0) (2.0 5.0) (3.0 6.0))` (a vector is returned unchanged) | | `linalg:pad` | `(linalg:pad #(1 2) 1)` | `#d(0.0 1.0 2.0 0.0)` (constant-0 padding; a list gives per-axis `(before after)` pairs) | | `linalg:expand-dims` | `(linalg:expand-dims #(1 2 3) 0)` | `#d((1.0 2.0 3.0))` (a new extent-1 axis; numpy's `expand_dims` / torch's `unsqueeze`) | | `linalg:squeeze` | `(linalg:squeeze #2A((1 2 3)))` | `#d(1.0 2.0 3.0)` (drops extent-1 axes; `:axis` picks which) | | `linalg:concatenate` | `(linalg:concatenate (list #(1 2) #(3)))` | `#d(1.0 2.0 3.0)` (join a LIST of arrays along an existing `:axis`) | | `linalg:stack` | `(linalg:stack (list #(1 2) #(3 4)))` | `#d((1.0 2.0) (3.0 4.0))` (join along a NEW `:axis`) | | `linalg:slice` | `(linalg:slice #(0 1 2 3 4 5) '((nil nil 2)))` | `#d(0.0 2.0 4.0)` (basic numpy slicing; one `nil` / `(start end [step])` spec per axis) | | `linalg:triu` | `(linalg:triu (linalg:ones '(3 3)) :k 1)` | `#d((0.0 1.0 1.0) (0.0 0.0 1.0) (0.0 0.0 0.0))` (upper triangle; the causal mask) | | `linalg:tril` | `(linalg:tril #2A((1 2) (3 4)))` | `#d((1.0 0.0) (3.0 4.0))` (lower triangle) | | `linalg:add` | `(linalg:add #(1 2 3) 10)` | `#d(11.0 12.0 13.0)` (elementwise; a scalar operand broadcasts) | | `linalg:sub` | `(linalg:sub #(5 5) 1)` | `#d(4.0 4.0)` | | `linalg:mul` | `(linalg:mul m1 m2)` | The Hadamard (elementwise) product -- not the matrix product | | `linalg:div` | `(linalg:div #(1 2 3) 2)` | `#d(0.5 1.0 1.5)` (a packed double-float array) | | `linalg:+` | `(linalg:+ #(1 2) #(3 4) #(10 10))` | `#d(14.0 16.0)` (n-ary `add`; the CL operator spelling) | | `linalg:-` | `(linalg:- #(10 10) 1 2)` | `#d(7.0 7.0)` (n-ary `sub`; one argument negates) | | `linalg:*` | `(linalg:* #(1 2) #(3 4))` | `#d(3.0 8.0)` (n-ary `mul`, Hadamard -- not the matrix product) | | `linalg:/` | `(linalg:/ #(1 2 3) 2)` | `#d(0.5 1.0 1.5)` (n-ary `div`; one argument gives the reciprocal) | | `linalg:emap` | `(linalg:emap (lambda (x) (* x x)) (linalg:arange 4))` | `#d(0.0 1.0 4.0 9.0)` (apply a function to every element) | | `linalg:exp` | `(linalg:exp (linalg:zeros 3))` | `#d(1.0 1.0 1.0)` (elementwise `e^x`) | | `linalg:log` | `(linalg:log #(1 1 1))` | `#d(0.0 0.0 0.0)` (elementwise natural log) | | `linalg:tanh` | `(linalg:tanh (linalg:zeros 3))` | `#d(0.0 0.0 0.0)` (elementwise hyperbolic tangent) | | `linalg:sin` | `(linalg:sin (linalg:zeros 3))` | `#d(0.0 0.0 0.0)` (elementwise sine) | | `linalg:cos` | `(linalg:cos (linalg:zeros 3))` | `#d(1.0 1.0 1.0)` (elementwise cosine) | | `linalg:tan` | `(linalg:tan (linalg:zeros 3))` | `#d(0.0 0.0 0.0)` (elementwise tangent) | | `linalg:asin` | `(linalg:asin (linalg:zeros 3))` | `#d(0.0 0.0 0.0)` (elementwise arc sine) | | `linalg:acos` | `(linalg:acos (linalg:ones 3))` | `#d(0.0 0.0 0.0)` (elementwise arc cosine) | | `linalg:atan` | `(linalg:atan (linalg:zeros 3))` | `#d(0.0 0.0 0.0)` (elementwise arc tangent) | | `linalg:sinh` | `(linalg:sinh (linalg:zeros 3))` | `#d(0.0 0.0 0.0)` (elementwise hyperbolic sine) | | `linalg:cosh` | `(linalg:cosh (linalg:zeros 3))` | `#d(1.0 1.0 1.0)` (elementwise hyperbolic cosine) | | `linalg:sqrt` | `(linalg:sqrt #(4 9 16))` | `#d(2.0 3.0 4.0)` (elementwise square root) | | `linalg:abs` | `(linalg:abs #(-3 2 -1))` | `#d(3.0 2.0 1.0)` (elementwise absolute value) | | `linalg:square` | `(linalg:square #(1 2 3))` | `#d(1.0 4.0 9.0)` (elementwise `x * x`) | | `linalg:negative` | `(linalg:negative #(1 -2 3))` | `#d(-1.0 2.0 -3.0)` (elementwise negation) | | `linalg:sign` | `(linalg:sign #(-5 0 7))` | `#d(-1.0 0.0 1.0)` (elementwise sign) | | `linalg:reciprocal` | `(linalg:reciprocal #(2 4 8))` | `#d(0.5 0.25 0.125)` (elementwise `1 / x`, in float) | | `linalg:power` | `(linalg:power #(1 2 3) 2)` | `#d(1.0 4.0 9.0)` (elementwise `a ** b`; either operand may be a scalar) | | `linalg:maximum` | `(linalg:maximum #(1 5 3) #(4 2 3))` | `#d(4.0 5.0 3.0)` (elementwise larger; either operand may be a scalar) | | `linalg:minimum` | `(linalg:minimum #(1 5 3) 4)` | `#d(1.0 4.0 3.0)` (elementwise smaller; either operand may be a scalar) | | `linalg:clip` | `(linalg:clip #(-2 0 3) -1.0 1.0)` | `#d(-1.0 0.0 1.0)` (elementwise `min(max(x, lo), hi)`) | | `linalg:relu` | `(linalg:relu #(-2 0 3))` | `#d(0.0 0.0 3.0)` (elementwise `max(x, 0.0)`) | | `linalg:erf` | `(linalg:erf #(0 1))` | `#d(0.0 0.842700792949715)` (elementwise Gauss error function) | | `linalg:softmax` | `(linalg:softmax #(1 1 1 1))` | `#d(0.25 0.25 0.25 0.25)` (max-subtracted softmax; `:axis` normalizes per slice) | | `linalg:log-softmax` | `(linalg:log-softmax #(0 0))` | `#d(-0.6931471805599453 -0.6931471805599453)` (the stable log of `softmax`) | | `linalg:dot` | `(linalg:dot v1 v2)` | numpy-style dispatch: vec.vec scalar, mat.vec / vec.mat vector, mat.mat matrix product | | `linalg:matmul` | `(linalg:matmul #2A((1 2) (3 4)) #2A((5 6) (7 8)))` | `#d((19.0 22.0) (43.0 50.0))` (the matrix product; rank >= 3 stacks on the last two axes) | | `linalg:outer` | `(linalg:outer #(1 2) #(3 4 5))` | `#d((3.0 4.0 5.0) (6.0 8.0 10.0))` (the outer product) | | `linalg:sum` | `(linalg:sum #2A((1 2) (3 4)))` | `10` (a reduction follows the element type; `:axis` / `:keepdims` keywords) | | `linalg:mean` | `(linalg:mean #(1 2 3 4))` | `5/2` (a reduction follows the element type; `:axis` / `:keepdims` keywords) | | `linalg:var` | `(linalg:var #(1 2 3 4))` | `1.25` (variance; `:axis` / `:keepdims` / `:ddof` keywords) | | `linalg:std` | `(linalg:std #(2 4 4 4 5 5 7 9))` | `2.0` (the square root of `linalg:var`, same keywords) | | `linalg:amax` | `(linalg:amax #2A((1 9) (3 4)))` | `9` (the largest element; `:axis` / `:keepdims` keywords) | | `linalg:amin` | `(linalg:amin #(5 2 8))` | `2` (the smallest element; `:axis` / `:keepdims` keywords) | | `linalg:argmax` | `(linalg:argmax #(1 9 3))` | `1` (first index on ties; `:axis` gives per-slice indices) | | `linalg:argmin` | `(linalg:argmin #(5 2 8))` | `1` (first index on ties; `:axis` gives per-slice indices) | | `linalg:norm` | `(linalg:norm #(3 4))` | `5.0` (the Euclidean / Frobenius norm) | | `linalg:trace` | `(linalg:trace #2A((1 2) (3 4)))` | `5` (square matrices only) | | `linalg:diff` | `(linalg:diff #(1 2 4 7 0))` | `#d(1.0 2.0 3.0 -7.0)` (the `:n`-th discrete difference along `:axis`; defaults 1 and the last axis) | | `linalg:gradient` | `(linalg:gradient #(0 1 4 9 16))` | `#d(1.0 2.0 4.0 6.0 7.0)` (central differences, same length as the input; optional scalar spacing or coordinate vector) | | `linalg:det` | `(linalg:det #2A((1 2) (3 4)))` | `-2.0` (floating point; a singular matrix may give a small epsilon) | | `linalg:inv` | `(linalg:inv #2A((4 0) (2 4)))` | `#d((0.25 0.0) (-0.125 0.25))` (signals an error for a singular matrix) | | `linalg:solve` | `(linalg:solve a b)` | The solution of `a . x = b` (`b` a vector or matrix) | | `linalg:array-equal` | `(linalg:array-equal (linalg:eye 2) #2A((1 0) (0 1)))` | `t` (same shape and numerically equal elements; arrays themselves are only `eq`-comparable) | | `linalg:equal` | `(linalg:equal #(1 5 3) #(2 5 1))` | `#d(0.0 1.0 0.0)` (elementwise `=` as a 0/1 mask; broadcasts) | | `linalg:greater` | `(linalg:greater #(1 5 3) 2)` | `#d(0.0 1.0 1.0)` (elementwise `>` as a 0/1 mask) | | `linalg:greater-equal` | `(linalg:greater-equal #(1 5 3) #(1 6 2))` | `#d(1.0 0.0 1.0)` (elementwise `>=` as a 0/1 mask) | | `linalg:less` | `(linalg:less #(1 5 3) 3)` | `#d(1.0 0.0 0.0)` (elementwise `<` as a 0/1 mask) | | `linalg:less-equal` | `(linalg:less-equal #(1 5 3) 3)` | `#d(1.0 0.0 1.0)` (elementwise `<=` as a 0/1 mask) | | `linalg:where` | `(linalg:where #(1 0 1) 10 20)` | `#d(10.0 20.0 10.0)` (elementwise select on a non-zero mask; broadcasts) | | `linalg:take-rows` | `(linalg:take-rows #2A((1 2 3) (4 5 6) (7 8 9)) #(2 0))` | `#d((7.0 8.0 9.0) (1.0 2.0 3.0))` (the axis-0 slices selected by an index vector) | | `linalg:row` | `(linalg:row #2A((1 2 3) (4 5 6) (7 8 9)) 1)` | `#d(4.0 5.0 6.0)` (one axis-0 slice, axis dropped -- numpy's `x[i]`) | | `linalg:gather` | `(linalg:gather #2A((10 11 12) (20 21 22)) #(2 0))` | `#d(12.0 20.0)` (the per-row elements `a[i, idx[i]]` of a matrix) | | `linalg:one-hot` | `(linalg:one-hot #(1 0 2) 3)` | `#d((0.0 1.0 0.0) (1.0 0.0 0.0) (0.0 0.0 1.0))` (row i holds 1.0 in column `indices[i]`) | | `linalg:seed` | `(linalg:seed 42)` | `42` (resets the shared random generator; seeded draws are identical on every backend) | | `linalg:rand` | `(linalg:rand 4)` | Uniform `[0, 1)` draws with the given shape | | `linalg:randn` | `(linalg:randn '(2 2))` | Standard-normal draws (Irwin-Hall; tails clip at +/- 6 sigma) | | `linalg:uniform` | `(linalg:uniform 10 20 4)` | Uniform draws in `[lo, hi)` with the given shape | | `linalg:choice` | `(linalg:choice 60000 4)` | 4 uniform indices in `[0, 60000)`, with replacement (a packed double vector) | | `linalg:permutation` | `(linalg:permutation 10)` | The integers `0..9` in a Fisher-Yates shuffle (a packed double vector) | ## torch Package Functions The `torch` package is the PyTorch-style differentiable layer over `linalg` (see the [Neural Networks guide](../guides/neural-networks.md)): a tensor that records how it was computed, and a `torch:backward` that walks that history to fill in gradients. It is **not part of Common Lisp**; reference its functions with the `torch:` qualifier (the package does not use `cl`). Every operation accepts tensors, numbers, arrays or lists as operands, computes through the `linalg` kernels (so `--simd` accelerates torch programs for free), and a tensor prints as `#` -- read the values back with `torch:data` / `torch:item` / `torch:grad`. The middle of the table is the `nn`-style module layer: a module owns its parameters in a fields plist, composes, and is run with `torch:forward`. The last part is what turns a model into a training run: the optimizers, whose `torch:step` updates every parameter in place, and the batching / padding / mask helpers, which are plain functions rather than a `Dataset`/`DataLoader` hierarchy. The one macro, `torch:no-grad`, is on the [Macros page](macros/torch-no-grad.md). | Function | Example | Result | |----------|---------|--------| | `torch:tensor` | `(torch:tensor '(1 2) :requires-grad t)` | a leaf tensor over packed data (`:element-type 'single-float` for `#f`) | | `torch:tensorp` | `(torch:tensorp x)` | `T` for a tensor, `NIL` otherwise | | `torch:data` | `(torch:data tn)` | the linalg array (or number, for a scalar tensor) | | `torch:grad` | `(torch:grad tn)` | the accumulated gradient, or `NIL` before backward | | `torch:shape` | `(torch:shape tn)` | the dims list; `NIL` for a scalar tensor | | `torch:item` | `(torch:item tn)` | the number in a one-element tensor | | `torch:detach` | `(torch:detach tn)` | a leaf sharing the data, cut off from the tape | | `torch:zero-grad` | `(torch:zero-grad tn)` | clears the gradient slot; returns the tensor | | `torch:requires-grad-p` | `(torch:requires-grad-p tn)` | whether the tensor participates in autograd | | `torch:backward` | `(torch:backward loss)` | reverse-mode autograd from a scalar tensor (accumulates into `torch:grad`) | | `torch:add` | `(torch:add a b)` | differentiable elementwise `+` with broadcasting | | `torch:sub` | `(torch:sub a b)` | differentiable elementwise `-` | | `torch:mul` | `(torch:mul a b)` | differentiable elementwise (Hadamard) `*` | | `torch:div` | `(torch:div a b)` | differentiable elementwise `/` | | `torch:neg` | `(torch:neg a)` | differentiable negation | | `torch:power` | `(torch:power a 2)` | differentiable elementwise `a ** b` | | `torch:exp` | `(torch:exp a)` | differentiable `e^x` | | `torch:log` | `(torch:log a)` | differentiable natural log | | `torch:sqrt` | `(torch:sqrt a)` | differentiable square root | | `torch:tanh` | `(torch:tanh a)` | differentiable hyperbolic tangent | | `torch:relu` | `(torch:relu a)` | differentiable `max(x, 0.0)` | | `torch:erf` | `(torch:erf a)` | differentiable Gauss error function | | `torch:gelu` | `(torch:gelu a)` | differentiable GELU (`:approximate :none` / `:tanh`) | | `torch:matmul` | `(torch:matmul a b)` | differentiable matrix product (batched at rank >= 3) | | `torch:sum` | `(torch:sum a :axis 0)` | differentiable sum (whole tensor or along an axis) | | `torch:mean` | `(torch:mean a)` | differentiable mean | | `torch:var` | `(torch:var a :ddof 1)` | differentiable variance (`(n - ddof)` divisor) | | `torch:std` | `(torch:std a)` | differentiable standard deviation | | `torch:amax` | `(torch:amax a :axis 0)` | differentiable maximum (gradient split among ties) | | `torch:argmax` | `(torch:argmax a)` | index of the largest element (non-differentiable, raw value) | | `torch:topk` | `(torch:topk a 5)` | the `k` largest values along an axis, largest first (`:indices t` for their positions) | | `torch:multinomial` | `(torch:multinomial probs)` | indices drawn per row from the seeded generator (`:num-samples`, `:replacement`) | | `torch:softmax` | `(torch:softmax a :axis 1)` | differentiable max-subtracted softmax | | `torch:log-softmax` | `(torch:log-softmax a :axis 1)` | differentiable log-softmax (cross-entropy half) | | `torch:masked-fill` | `(torch:masked-fill a mask v)` | differentiable fill of `v` where the mask is non-zero | | `torch:gather` | `(torch:gather a idx)` | differentiable per-row element pick of a matrix | | `torch:index-select` | `(torch:index-select a idx)` | differentiable row selection (the embedding lookup; repeats accumulate) | | `torch:reshape` | `(torch:reshape a '(2 3))` | differentiable row-major reshape | | `torch:view` | `(torch:view a '(2 3))` | `torch:reshape` under PyTorch's other name | | `torch:transpose` | `(torch:transpose a '(1 0 2))` | differentiable transpose / axes permutation | | `torch:unsqueeze` | `(torch:unsqueeze a 0)` | differentiable extent-1 axis insertion | | `torch:squeeze` | `(torch:squeeze a)` | differentiable extent-1 axis removal | | `torch:cat` | `(torch:cat (list a b) :axis 1)` | differentiable concatenation along an existing axis | | `torch:stack` | `(torch:stack (list a b))` | differentiable join along a new axis | | `torch:slice` | `(torch:slice a '(nil (0 2)))` | differentiable numpy basic slicing | | `torch:set-data` | `(torch:set-data tn v)` | replaces a tensor's data in place (the parameter update) | | `torch:module` | `(torch:module :k fields fn)` | a user layer: a kind, a fields plist and a forward function | | `torch:modulep` | `(torch:modulep x)` | `T` for a module, `NIL` otherwise | | `torch:module-kind` | `(torch:module-kind m)` | the module's kind keyword | | `torch:field` | `(torch:field m :weight)` | the value of a module's named field (signals when absent) | | `torch:fields` | `(torch:fields m)` | the whole fields plist, as a fresh list -- the module walk | | `torch:set-field` | `(torch:set-field m :weight p)` | sets a module's named field; returns the module | | `torch:forward` | `(torch:forward m x)` | runs a module's (or a plain function's) forward pass | | `torch:parameter` | `(torch:parameter '(1.0))` | a leaf tensor with `requires-grad` -- a trainable parameter | | `torch:parameters` | `(torch:parameters m)` | every parameter reachable from a module, deduplicated | | `torch:train` | `(torch:train m)` | puts the module and its submodules into training mode | | `torch:eval` | `(torch:eval m)` | puts the module and its submodules into evaluation mode | | `torch:training-p` | `(torch:training-p m)` | whether the module is in training mode | | `torch:linear` | `(torch:linear 4 8)` | a fully connected layer (`:weight`, `:bias`) | | `torch:embedding` | `(torch:embedding 100 8)` | an embedding table (`:weight`), indices of any shape | | `torch:sequential` | `(torch:sequential a #'torch:relu b)` | a chain of layers and/or plain functions | | `torch:layer-norm` | `(torch:layer-norm 8)` | layer normalization over the last axis (`ddof` 0) | | `torch:dropout` | `(torch:dropout 0.1)` | inverted dropout; the identity in evaluation mode | | `torch:mse-loss` | `(torch:mse-loss y target)` | mean squared error (`:reduction :mean` / `:sum` / `:none`) | | `torch:cross-entropy-loss` | `(torch:cross-entropy-loss logits target)` | cross entropy over logits; target is class indices (`:ignore-index` skips padding) or a probability distribution | | `torch:optimizer` | `(torch:optimizer :k ps fields fn)` | a user optimizer: a kind, parameters, a fields plist and a step function | | `torch:optimizerp` | `(torch:optimizerp x)` | `T` for an optimizer, `NIL` otherwise | | `torch:optimizer-kind` | `(torch:optimizer-kind o)` | the optimizer's kind keyword | | `torch:optimizer-params` | `(torch:optimizer-params o)` | the parameter tensors the optimizer updates | | `torch:step` | `(torch:step o)` | applies the update rule to every parameter (in place, off the tape) | | `torch:step-count` | `(torch:step-count o)` | how many times `torch:step` has run (Adam's `t`) | | `torch:sgd` | `(torch:sgd model :lr 0.1)` | SGD, optionally with `:momentum` / `:weight-decay` | | `torch:adam` | `(torch:adam model :lr 0.001)` | Adam (`:betas`, `:eps`, `:weight-decay`), bias-corrected from the first step | | `torch:adamw` | `(torch:adamw model :lr 0.001)` | AdamW: the same rule with DECOUPLED `:weight-decay` (default `0.01`) | | `torch:clip-grad-norm` | `(torch:clip-grad-norm model 1.0)` | scales every gradient in place when their total L2 norm exceeds the bound; returns that norm | | `torch:pad-sequence` | `(torch:pad-sequence seqs)` | variable-length sequences as one padded batch-first tensor | | `torch:shuffled-batches` | `(torch:shuffled-batches n 32)` | mini-batches from the seeded generator (`:shuffle`, `:drop-last`) | | `torch:padding-mask` | `(torch:padding-mask tokens)` | `(batch 1 length)` mask of the padding positions (a raw array) | | `torch:subsequent-mask` | `(torch:subsequent-mask 8)` | `(1 n n)` causal mask, `1.0` above the diagonal (a raw array) | ## java Package Functions The `java` package drives arbitrary Java APIs by reflection. It is **JVM-only** — it works on the interpreter (`java -jar rontolisp.jar`) and in JVM-compiled classes (the compiler embeds a reflection bridge into the generated `.class`), but not on the WASM backend, and the GraalVM native binary carries no reflection metadata to interpret it — and **not part of Common Lisp**; reference its functions with the `java:` qualifier. Each name below links to its own page; the [Java interop guide](../guides/java-interop.md) covers marshalling, overload resolution and limitations. | Function | Example | Result | |----------|---------|--------| | `java:new` | `(java:new "java.lang.StringBuilder" "ab")` | a host object (`#`) | | `java:call` | `(java:call obj "size")` | the marshalled instance-method result | | `java:static` | `(java:static "java.lang.Math" "max" 3 7)` | the marshalled static-method result | | `java:field` | `(java:field "java.lang.Integer" "MAX_VALUE")` | the marshalled field value | | `java:proxy` | `(java:proxy "java.lang.Runnable" (lambda (m) ...))` | an interface instance backed by the callable | ## asdf Package Functions The `asdf` package is a limited, API-compatible subset of ASDF for loading multi-file systems from `.asd` definitions. It is **not part of Common Lisp**; reference its symbols with the `asdf:` qualifier. Each name below links to its own page; the [Systems guide](../guides/asdf-systems.md) gives a full project layout and the search-path details. | Function | Example | Result | |----------|---------|--------| | `asdf:defsystem` | `(asdf:defsystem :my-lib :components ((:file "main")))` | define a system (name, `:depends-on`, `:serial`, `:components`) for a later `load-system` | | `asdf:load-system` | `(asdf:load-system :my-lib)` | load a system: its dependency systems first, then its component files in order (a literal, top-level form on the compile path) | | `asdf:test-system` | `(asdf:test-system "my-app")` | load the system, follow its `:in-order-to` test-op chain, then run its recorded `:perform (test-op ...)` body — the standard `.asd` test entry point | | `asdf:find-system` | `(asdf:find-system :my-lib nil)` | the system's metaobject, a real `asdf:system` CLOS instance memoized per name (`eq` across calls); nil for an unknown name when `error-p` is nil | | `asdf:registered-systems` | `(asdf:registered-systems)` | the downcased names of every registered system, in registration order | | `asdf:system-relative-pathname` | `(asdf:system-relative-pathname :my-lib "data/tlds.dat")` | the namestring of a path resolved against the system's source directory (folded to a literal on the compile path) | | `asdf:component-pathname` | `(asdf:component-pathname (asdf:find-system :my-lib))` | a system's source directory with a trailing `/`, or a source-file child's resolved path; accepts the metaobject or a name designator | | `asdf:component-name` | `(asdf:component-name (asdf:find-system :my-lib))` | reader: the component's downcase-canonical name | | `asdf:component-version` | `(asdf:component-version (asdf:find-system :my-lib))` | reader: the declared `:version` string, or nil when the `.asd` declared none as a plain string (a computed spelling is never evaluated) | | `asdf:component-children` | `(asdf:component-children (asdf:find-system :my-lib))` | reader: a system's component files in load order, one `asdf:cl-source-file` per file | | `asdf:component-sideway-dependencies` | `(asdf:component-sideway-dependencies (asdf:find-system :my-lib))` | reader: the system's `:depends-on` names (package-inferred sub-system names included) | | `asdf:component-parent` | `(asdf:component-parent child)` | reader: the parent component — the system for a source file, nil for a system | | `asdf:component-system` | `(asdf:component-system child)` | the system a component belongs to (walks `component-parent` up) | ## uiop Package Functions The `uiop` package is ASDF's portability layer — the spelling implementation-independent libraries already use for the operations Common Lisp never standardized. It is **not part of Common Lisp**; reference its symbols with the `uiop:` qualifier, never unqualified. It is 15 sub-packages and 429 exports, so it has a page of its own: **[The uiop Package](uiop.md)** — the sub-package layout, what is implemented, and what an unimplemented member signals. ## ql and ql-dist Package Functions The `ql` package is a limited, API-compatible subset of Quicklisp: `quickload` downloads a system from the real Quicklisp distribution into a local cache and then loads it through the `asdf` subset (`quicklisp` is a built-in nickname). `ql-dist` holds the distribution machinery, of which the one member a program writes is `install-dist`: it adds another Quicklisp-format distribution — [Ultralisp](https://ultralisp.org/), or any distinfo URL — to the dists `quickload` searches, in installation order. Neither package is **part of Common Lisp**; reference their symbols with the `ql:` / `ql-dist:` qualifier. The names below link to their own pages; the [Systems guide](../guides/asdf-systems.md#downloading-with-quickload) covers the cache layout and limitations. | Function | Example | Result | |----------|---------|--------| | `ql:quickload` | `(ql:quickload "split-sequence")` | download a system (and its dependencies) from the installed dists, cache it under `~/.rontolisp/`, and load it; returns the list of loaded system names | | `ql-dist:install-dist` | `(ql-dist:install-dist "ultralisp")` | add a Quicklisp-format distribution (a known name or a distinfo URL) to the dists `quickload` searches; returns the dist name | | `ql:update-dist` | `(ql:update-dist "ultralisp")` | drop a dist's cached indexes so the next `quickload` sees its newest releases; returns the dist name | ## usocket Package Functions The `usocket` package is a compatibility shim over the `rontolisp:tcp-*` built-ins reproducing the [usocket](https://github.com/usocket/usocket) API, so existing Common Lisp networking code (such as Postmodern's cl-postgres socket layer) runs with fewer changes. It is **not part of Common Lisp**; reference its symbols with the `usocket:` qualifier. A socket IS its stream handle here, so `socket-stream` is the identity function and the standard stream functions work on sockets directly. The package is loaded on first use and is also the built-in ASDF system `"usocket"` (satisfying `asdf:load-system`, `ql:quickload` and `:depends-on ("usocket")` without a download). TCP only -- UDP (`socket-send` / `socket-receive`), `wait-for-input`, `socket-server` and the condition hierarchy (`usocket:socket-error` under `handler-case`) are not supported. The variables `usocket:*wildcard-host*` (`"0.0.0.0"`) and `usocket:*auto-port*` (`0`) are provided. See the [TCP Sockets guide](../guides/tcp-sockets.md#the-usocket-compatible-shim) for a worked overview and the full limitation list. | Function | Example | Result | |----------|---------|--------| | `usocket:socket-connect` | `(usocket:socket-connect "localhost" 5432 :element-type '(unsigned-byte 8))` | open a blocking TCP connection; `:protocol :datagram` signals, the other options are accepted and ignored | | `usocket:socket-listen` | `(usocket:socket-listen usocket:*wildcard-host* usocket:*auto-port*)` | bind a listening TCP socket (host first, usocket-style) | | `usocket:socket-accept` | `(usocket:socket-accept listener)` | wait for a client connection (blocking) | | `usocket:socket-stream` | `(read-line (usocket:socket-stream sock))` | the stream of a socket (the identity function in this shim) | | `usocket:socket-close` | `(usocket:socket-close sock)` | close a socket or listener | | `usocket:get-local-port` | `(usocket:get-local-port listener)` | the locally bound port (read an ephemeral port back) | | `usocket:get-local-address` | `(usocket:get-local-address listener)` | the locally bound IP address, as a string | | `usocket:get-peer-address` | `(usocket:get-peer-address sock)` | the remote IP address of a connected socket | | `usocket:get-peer-port` | `(usocket:get-peer-port sock)` | the remote port of a connected socket | | `usocket:get-local-name` | `(usocket:get-local-name sock)` | local address and port as `(values address port)` | | `usocket:get-peer-name` | `(usocket:get-peer-name sock)` | remote address and port as `(values address port)` | | `usocket:host-to-hostname` | `(usocket:host-to-hostname #(192 168 0 1))` | a host designator (string, vector quad, host-byte-order integer or `nil`) as a hostname/dotted-quad string | | `usocket:get-host-by-name` | `(usocket:get-host-by-name "example.com")` | lite: renders the designator through `host-to-hostname` instead of resolving it — no backend has a name-resolution primitive, and the socket call the address reaches resolves it for real | The `with-*` convenience macros (`usocket:with-client-socket` / `with-connected-socket` / `with-server-socket` / `with-socket-listener`) are listed on the [macros page](macros.md) and described on their [reference page](macros/usocket-with-macros.md); on the interpreter and the JVM they close the socket on every exit (they expand over [`unwind-protect`](special-forms/unwind-protect.md)), on the WASM component backend on normal exit only. --- # FILE: references/reference/functions/1minus.md # 1- `(1- number)` Returns `number` minus one, equivalent to `(- number 1)`. The result preserves the argument's numeric type, so an integer stays an integer, a float stays a float, and a ratio stays a ratio. ```lisp (1- 43) ; => 42 ``` --- # FILE: references/reference/functions/1plus.md # 1+ `(1+ number)` Returns `number` plus one, equivalent to `(+ number 1)`. The result preserves the argument's numeric type, so an integer stays an integer, a float stays a float, and a ratio stays a ratio. ```lisp (1+ 41) ; => 42 ``` --- # FILE: references/reference/functions/abort.md # abort `(abort [condition])` Invokes the innermost active `abort` restart. rontolisp establishes no `abort` restart of its own (there is no debugger REPL), so this reaches only a restart your program established under that name — and signals an error when none is active, the CL contract. ```lisp (restart-case (progn (abort) :not-reached) (abort () :aborted)) ; => :ABORTED ``` --- # FILE: references/reference/functions/abs.md # abs `(abs number)` Returns the absolute value of `number`, preserving its numeric type: an integer yields an integer, a float yields a float, and a ratio yields a ratio. ```lisp (abs -5) ; => 5 ``` ```lisp (abs -3.14) ; => 3.14 ``` --- # FILE: references/reference/functions/acons.md # acons `(acons key datum alist)` Returns a new alist with the pair `(key . datum)` prepended to `alist`. It is shorthand for `(cons (cons key datum) alist)` and does not modify the original list. The fresh pair shadows any earlier entry for the same key in `assoc` lookups. ```lisp (acons 'b 2 '((a . 1))) ; => ((B . 2) (A . 1)) ``` --- # FILE: references/reference/functions/adjoin.md # adjoin `(adjoin item list &key test key)` Returns `list` unchanged if `item` is already a member; otherwise returns a new list with `item` prepended. The comparison is `eql` by default; the optional `:test` keyword takes a function designator to use a different comparison, and the optional `:key` keyword takes a selector function applied to both the item and the list elements (the prepended item stays unkeyed). It is the non-destructive way to add an element to a set without creating duplicates. ```lisp (adjoin 1 '(2 3)) ; => (1 2 3) ``` ```lisp (adjoin "a" '("a" "b") :test #'string=) ; => ("a" "b") ``` --- # FILE: references/reference/functions/adjust-array.md # adjust-array `(adjust-array array new-dimensions &key initial-element fill-pointer)` Resizes `array` to `new-dimensions` (an integer for a vector, or a list of the same rank as `array`), preserving the elements at the subscripts valid in both shapes -- resizing a matrix keeps `(i, j)` at `(i, j)`, not at the same flat position. New cells are set to `:initial-element` (default nil). An array created [`:adjustable`](make-array.md) is adjusted in place and returned itself (`eq` to the argument), so every reference sees the new shape; otherwise a fresh array is returned and the original is left untouched -- use the return value either way. Without an explicit `:fill-pointer` the array's own [fill pointer](fill-pointer.md) carries over (an error if it no longer fits); `t` sets it to the new size, an integer to that position. Adjusting a displaced view or passing `:displaced-to` is not supported and signals an error. ```lisp (defparameter *v* (make-array 3 :adjustable t :initial-element 1)) (eq (adjust-array *v* 5 :initial-element 9) *v*) ; => T *v* ; => #(1 1 1 9 9) (adjust-array (make-array '(2 2) :initial-element 5) '(2 3) :initial-element 0) ; => #2A((5 5 0) (5 5 0)) ``` --- # FILE: references/reference/functions/adjustable-array-p.md # adjustable-array-p `(adjustable-array-p array)` Returns `t` when the array was created with [`make-array`](make-array.md) `:adjustable`, otherwise nil. The flag is reported verbatim; [`vector-push-extend`](vector-push-extend.md) grows any vector with a fill pointer regardless of it. ```lisp (adjustable-array-p (make-array 2 :adjustable t)) ; => T (adjustable-array-p (make-array 2)) ; => NIL ``` --- # FILE: references/reference/functions/allocate-instance.md # allocate-instance `(allocate-instance class &rest initargs)` Returns a fresh instance of `class` -- a class metaobject (the [`find-class`](find-class.md) / [`class-of`](class-of.md) answer) or a class name symbol -- with EVERY slot unbound: no `:initform` runs and `initialize-instance` is not called. This is the low-level allocation step object mappers build on (allocate, then fill each slot with `(setf (slot-value ...))`); reading a slot before it is written signals `unbound-slot`. The `initargs` are accepted and ignored, as in Common Lisp (they are only seen by methods on `allocate-instance`, which the static subset does not support). Only `defclass` / `define-condition` classes can be allocated; a built-in class or a `defstruct` class signals an error. ```lisp (defclass point () ((x :initarg :x :initform 0) (y :initarg :y))) (let ((p (allocate-instance (find-class 'point)))) (setf (slot-value p 'x) 10) (list (slot-boundp p 'y) (slot-value p 'x))) ; => (NIL 10) ``` --- # FILE: references/reference/functions/alpha-char-p.md # alpha-char-p `(alpha-char-p character)` Returns `t` if `character` is an alphabetic letter and `nil` otherwise (for example a digit yields `nil`). In the WASM backend the test recognizes the ASCII letters `a`-`z` and `A`-`Z` only. ```lisp (alpha-char-p #\x) ; => T ``` --- # FILE: references/reference/functions/alphanumericp.md # alphanumericp `(alphanumericp character)` Returns true if `character` is a letter or a decimal digit and `nil` otherwise. For a digit the returned value is its weight (like `digit-char-p`); for a letter it is `t` — both are true. In the WASM backend the letter test recognizes the ASCII letters `a`-`z` and `A`-`Z` only. ```lisp (alphanumericp #\x) ; => T ``` ```lisp (alphanumericp #\-) ; => NIL ``` --- # FILE: references/reference/functions/apply.md # apply `(apply function &rest args)` Calls `function` with the leading arguments prepended to the elements of the final argument, which must be a list, and returns its result. As with `funcall`, `function` may be a function value (`#'f`, a `lambda`) or a quoted symbol designator. `(apply #'+ 1 2 '(3 4))` is equivalent to `(funcall #'+ 1 2 3 4)`; passing only the spread list as in `(apply #'+ '(1 2 3))` works too. ```lisp (apply #'+ 1 2 '(3 4)) ; => 10 ``` --- # FILE: references/reference/functions/aref.md # aref `(aref array &rest subscripts)` Returns the element of `array` at the given 0-based subscripts, one per dimension (one for a rank-1 vector, two for a rank-2 array, and so on). A string is a rank-1 character array, so `(aref s i)` reads like [`char`](char.md) (writing a string element goes through the `schar`/`char` setf place instead). Flat rank-independent access is available via [`row-major-aref`](row-major-aref.md). To modify an element, use `aref` as a `setf` place: `(setf (aref array i j) value)`, which also works with `incf`/`decf`/`push`. `#'aref` is a first-class function value, so it can be passed to `mapcar`/`funcall` like any other function. ```lisp (let ((a (make-array 3 :initial-element 0))) (setf (aref a 1) 9) (aref a 1)) ; => 9 ``` --- # FILE: references/reference/functions/array-dimension.md # array-dimension `(array-dimension array axis-number)` Returns the size of the dimension of `array` at the 0-based `axis-number`. For a rank-1 vector the only valid axis is 0; for a rank-n array the axes are 0 to n-1. Use [`array-dimensions`](array-dimensions.md) to get all sizes at once. ```lisp (array-dimension (make-array '(2 3)) 0) ; => 2 (array-dimension (make-array '(2 3)) 1) ; => 3 ``` --- # FILE: references/reference/functions/array-dimensions.md # array-dimensions `(array-dimensions array)` Returns a list of the sizes of each dimension of `array`. A rank-1 vector yields a one-element list and a rank-2 array yields a two-element list (only ranks 1 and 2 are supported). See [`array-dimension`](array-dimension.md) for the size of a single axis and [`array-rank`](array-rank.md) for the number of dimensions. ```lisp (array-dimensions (make-array '(2 3))) ; => (2 3) (array-dimensions (vector 1 2)) ; => (2) ``` --- # FILE: references/reference/functions/array-displacement.md # array-displacement `(array-displacement array)` Returns two values: the array `array` was displaced to with [`make-array`](make-array.md) `:displaced-to`, and the `:displaced-index-offset` it was created with. For a non-displaced array the values are nil and 0. The second value is only observable through a multiple-value consumer such as `multiple-value-bind`. ```lisp (defparameter *base* (make-array 5 :initial-element 0)) (defparameter *view* (make-array 2 :displaced-to *base* :displaced-index-offset 3)) (multiple-value-bind (target offset) (array-displacement *view*) (list (eq target *base*) offset)) ; => (T 3) (array-displacement *base*) ; => NIL ``` --- # FILE: references/reference/functions/array-element-type.md # array-element-type `(array-element-type array)` Returns the array's element type. A packed float array answers `double-float` or `single-float`, and a packed unsigned-integer vector ([`make-array`](make-array.md) with a literal `:element-type '(unsigned-byte 8|16|32)` on a rank-1 array, or a [`concatenate`](concatenate.md) whose result type spells that element type) answers its real `(unsigned-byte n)` specifier. A string — a vector of characters — answers `character`. For every other general array the answer is the symbol `t` (other element types are accepted but not tracked). Provided for compatibility with portable code such as cl-utilities' `copy-array`. ```lisp (array-element-type "abc") ; => CHARACTER (array-element-type (make-array 3)) ; => T (array-element-type (make-array 3 :element-type '(unsigned-byte 8))) ; => (UNSIGNED-BYTE 8) ``` --- # FILE: references/reference/functions/array-has-fill-pointer-p.md # array-has-fill-pointer-p `(array-has-fill-pointer-p array)` Returns `t` when the array has a fill pointer (it was created with [`make-array`](make-array.md) `:fill-pointer`), otherwise nil. Only rank-1 arrays (vectors) can have one. ```lisp (array-has-fill-pointer-p (make-array 3 :fill-pointer 0)) ; => T (array-has-fill-pointer-p (make-array 3)) ; => NIL ``` --- # FILE: references/reference/functions/array-rank.md # array-rank `(array-rank array)` Returns the number of dimensions of `array`: 1 for a vector, 2 for a two-dimensional array, and so on for higher ranks. It equals the length of the list returned by [`array-dimensions`](array-dimensions.md). ```lisp (array-rank (vector 1)) ; => 1 (array-rank (make-array '(2 3))) ; => 2 ``` --- # FILE: references/reference/functions/array-row-major-index.md # array-row-major-index `(array-row-major-index array &rest subscripts)` Returns the 0-based flat row-major index of the element of `array` at the given subscripts (one per dimension): the fold `((s0 * d1 + s1) * d2 + s2) ...` over the dimension sizes. The result is a valid index for [`row-major-aref`](row-major-aref.md). ```lisp (array-row-major-index (make-array (list 2 3)) 1 1) ; => 4 ``` --- # FILE: references/reference/functions/array-total-size.md # array-total-size `(array-total-size array)` Returns the total number of elements in `array` -- the product of its dimension sizes over every dimension. For a rank-1 vector this is simply its length; for a rank-2 array it is rows times columns. See also [`array-dimensions`](array-dimensions.md). ```lisp (array-total-size (vector 1 2 3)) ; => 3 (array-total-size (make-array '(2 3))) ; => 6 ``` --- # FILE: references/reference/functions/arrayp.md # arrayp `(arrayp object)` Returns `t` when the object is an array and `nil` otherwise. A string **is** an array in Common Lisp (a rank-1 array of characters), so `(arrayp "abc")` is true; [`vectorp`](vectorp.md) answers the same for rank-1 arrays and strings. ```lisp (list (arrayp (vector 1 2)) (arrayp "abc") (arrayp '(1 2))) ; => (T T NIL) ``` --- # FILE: references/reference/functions/asdf-component-children.md # asdf:component-children `(asdf:component-children parent)` Reader: a parent component's children in load order. A system's children are its component files, **one `asdf:cl-source-file` per file** — a `:module` contributes its path prefix to the file names rather than a nested instance — and a package-inferred sub-system has exactly one child, real ASDF's shape. Each child's [`asdf:component-pathname`](asdf-component-pathname.md) is the resolved source path and its [`asdf:component-parent`](asdf-component-parent.md) is the system. ```lisp (asdf:defsystem :demo-ch :components ((:file "one") (:module "m" :components ((:file "two"))))) (mapcar (lambda (c) (asdf:component-name c)) (asdf:component-children (asdf:find-system :demo-ch))) ; => ("one" "m/two") ``` ## Backend support Works on all four backends. --- # FILE: references/reference/functions/asdf-component-name.md # asdf:component-name `(asdf:component-name component)` Reader: the component's downcase-canonical name. For a system that is the system name; for a source file it is the file's path relative to the system, minus the `.lisp` extension. ```lisp (asdf:defsystem :demo-cn :components ((:file "main"))) (asdf:component-name (asdf:find-system :demo-cn)) ; => "demo-cn" ``` ## Backend support Works on all four backends, on any component object ([`asdf:find-system`](asdf-find-system.md)'s systems and their [`asdf:component-children`](asdf-component-children.md)). --- # FILE: references/reference/functions/asdf-component-parent.md # asdf:component-parent `(asdf:component-parent component)` Reader: the component's parent component — the system for a source file, `nil` for a system itself. ```lisp (asdf:defsystem :demo-par :components ((:file "main"))) (asdf:component-parent (asdf:find-system :demo-par)) ; => NIL ``` ## Backend support Works on all four backends. [`asdf:component-system`](asdf-component-system.md) walks it up to the system. --- # FILE: references/reference/functions/asdf-component-pathname.md # asdf:component-pathname `(asdf:component-pathname component)` Returns the component's pathname as a namestring: for a **system**, the directory it was found in with a trailing `/`; for a **source file** (an entry of [`asdf:component-children`](asdf-component-children.md)), the resolved path of the file itself. `component` may be the metaobject [`asdf:find-system`](asdf-find-system.md) answers or a plain name designator (a string, keyword or symbol) — the designator names a system, which must be registered (loaded, or currently loading). This is how a library locates data files bundled beside its own sources: local-time finds its `zoneinfo/` repository with `(asdf:component-pathname (asdf:find-system :local-time nil))`. [`asdf:system-relative-pathname`](asdf-system-relative-pathname.md) composes this with a relative name in one call. ```console (asdf:load-system "my-lib") (print (asdf:component-pathname (asdf:find-system "my-lib"))) ``` ## Backend support Works on all four backends. The interpreter answers from its system registry at run time; the compile paths fold the call to a literal namestring when the system name is a literal (a literal `find-system` around it included), which is the shape every library uses — anything else reads the registry the compiled program carries. --- # FILE: references/reference/functions/asdf-component-sideway-dependencies.md # asdf:component-sideway-dependencies `(asdf:component-sideway-dependencies system)` Reader: the system's `:depends-on` names, in order. For a package-inferred system these are the names derived from the component file's own `defpackage`, sub-system names included — which is what rove's `package-inferred-system-component-names` filters by the primary's prefix. ```lisp (asdf:defsystem :demo-deps :depends-on ("demo-base") :components ((:file "main"))) (asdf:defsystem :demo-base :components ((:file "base"))) (asdf:component-sideway-dependencies (asdf:find-system :demo-deps)) ; => ("demo-base") ``` ## Backend support Works on all four backends. --- # FILE: references/reference/functions/asdf-component-system.md # asdf:component-system `(asdf:component-system component)` Walks [`asdf:component-parent`](asdf-component-parent.md) up to the system a component belongs to; a system answers itself. ```lisp (asdf:defsystem :demo-cs :components ((:file "main"))) (let ((sys (asdf:find-system :demo-cs))) (eq (asdf:component-system (car (asdf:component-children sys))) sys)) ; => T ``` ## Backend support Works on all four backends. --- # FILE: references/reference/functions/asdf-component-version.md # asdf:component-version `(asdf:component-version component)` Reader: the `:version` string the system's `defsystem` declared, or `nil` when it declared none. Only a plain string literal is recorded — a `.asd` is parsed as **data**, so ASDF's `(:read-file-form "version.sexp")` indirection (and any other computed spelling) is never evaluated and answers `nil`. A component file has no version of its own. ```lisp (asdf:defsystem :demo-cv :version "0.9.15" :components ((:file "main"))) (asdf:component-version (asdf:find-system :demo-cv)) ; => "0.9.15" ``` ## Backend support Works on all four backends, on any component object ([`asdf:find-system`](asdf-find-system.md)'s systems and their [`asdf:component-children`](asdf-component-children.md)). --- # FILE: references/reference/functions/asdf-defsystem.md # asdf:defsystem `(asdf:defsystem name &rest options)` Defines a **system** — a named collection of source files with load-order constraints — for a later [`asdf:load-system`](asdf-load-system.md), and returns the system name as a symbol. This is a limited, API-compatible subset of ASDF's `defsystem`: the options are plain data and are never evaluated. `name` is a literal designator (a string `"my-lib"`, a keyword `:my-lib`, or a symbol). Supported options: - Metadata — `:description`, `:long-description`, `:version`, `:author`, `:maintainer`, `:license` (also `:licence`), `:homepage`, `:bug-tracker`, `:source-control`, `:mailto` — accepted for `.asd` compatibility and ignored, except that a `:version` written as a plain string is read back by [`asdf:component-version`](asdf-component-version.md) (a computed spelling such as `(:read-file-form "version.sexp")` is never evaluated and answers nil). - `:depends-on (system...)` — systems loaded before this one, located through the same search path as `load-system`. - `:defsystem-depends-on (system...)` — systems real ASDF loads while the `.asd` is being **read**, so they are located the same way and loaded before `:depends-on`. They are not dependencies of the system (they never appear in [`asdf:component-sideway-dependencies`](asdf-component-sideway-dependencies.md)). A built-in shim system named here also **announces its features** to this system: `:defsystem-depends-on ("trivial-features")` is what puts `:unix` and `:little-endian` in force while this system's own clauses and component files are read. A third-party system announces nothing — it would have to run, and a `.asd` is data here. - `:serial t` — each component implicitly depends on the previous one. - `:pathname "dir"` — a directory prefix for every component of the system, so the components can be named bare (`(:file "main")` under `:pathname "src"` is `src/main.lisp`). A literal namestring only; an empty string adds no directory level. A `:module` prefix and a component-level `:pathname` nest inside it. - `:components (component...)` — the source files: `(:file "name" [:depends-on ("other"...)] [:pathname "file.lisp"])` names `name.lisp`; `(:module "dir" [:serial t] [:depends-on (...)] [:pathname "other-dir"] :components (...))` prefixes its children with `dir/`; `(:static-file "name")` is accepted and contributes no source. A component may also carry `:if-feature expr`, which keeps its place in the load order but contributes no source when the feature expression does not hold. Components are loaded in a stable topological order of their `:depends-on` constraints. - `:class :package-inferred-system` — the system has no `:components` at all and its graph is derived from the sources: a sub-system name is a file path under the system's directory (`my-lib/util/text` is `util/text.lisp`, below `:pathname` when the system has one), and that file's leading `defpackage` names its dependencies. See the [Systems guide](../../guides/asdf-systems.md#what-is-and-is-not-supported). No other `:class` is supported. The test-op wiring options `:in-order-to` and `:perform` are tolerated and ignored (there is no `test-op`/`operate` machinery to drive). Any other option (`:around-compile`, a computed `:pathname`, ...) or component type is an error naming the unsupported clause. Normally the form lives in a `NAME.asd` file next to the sources; a `.asd` file is parsed as **data**, so it may contain only `defsystem` forms (bare or `asdf:`-qualified), `in-package`/`defpackage` forms (skipped), `register-system-packages` forms (which record "this package lives in that system", read when a package-inferred system turns a `defpackage` dependency into a system name) and top-level `defparameter`s of pure literal values. An inline top-level `(asdf:defsystem ...)` in a program also registers the system. Component paths resolve against the directory of the `.asd` file (or of the defining source). ```console ;; my-lib.asd (defsystem :my-lib :description "A small example system" :version "0.1.0" :serial t :components ((:file "package") (:file "main"))) ``` The system loads `package.lisp` then `main.lisp` (`:serial t`), both from the directory of `my-lib.asd`. See the [Systems guide](../../guides/asdf-systems.md) for a complete project walkthrough. --- # FILE: references/reference/functions/asdf-find-system.md # asdf:find-system `(asdf:find-system name &optional (error-p t))` Returns the **system metaobject** for the named system: a CLOS instance of [`asdf:system`](asdf-component-name.md) (or of `asdf:package-inferred-system` for a `:class :package-inferred-system` one). The instance is memoized per name — repeated calls answer the **same object** (`eq`), like real ASDF — and passing a component object back in answers it unchanged. `name` is a string, keyword or symbol designator (a symbol downcases, a string stays verbatim). A name that is not registered signals an error, unless `error-p` is nil — then the answer is `nil`, the probe shape libraries use (`(asdf:find-system name nil)` guarding a `load-system`). Registered means: defined by a prior [`asdf:defsystem`](asdf-defsystem.md) or a loaded `.asd`, derived as a package-inferred sub-system, or one of the built-in shim systems. `find-system` never searches the filesystem itself. The component model behind the instance is real ASDF's: the classes `asdf:component`, `asdf:child-component` / `asdf:parent-component`, `asdf:module`, `asdf:system`, `asdf:package-inferred-system`, `asdf:source-file`, `asdf:cl-source-file` and `asdf:static-file` are real CLOS classes on every backend, so `typep`, `typecase` and `defmethod` specializers over them all work. ```lisp (asdf:defsystem :demo :components ((:file "main"))) (asdf:component-name (asdf:find-system :demo)) ; => "demo" ``` ```lisp (asdf:defsystem :demo2 :components ((:file "main"))) (eq (asdf:find-system :demo2) (asdf:find-system "demo2")) ; => T ``` ## Backend support Works on all four backends. The interpreter answers from its live system registry; a compiled program carries the registry that was spliced at compile time, so `find-system` knows exactly the systems the program loaded (plus every `defsystem` the `.asd`s declared). A literal `(asdf:system-source-directory (asdf:find-system 'lib nil))` still folds to a literal namestring at compile time, so the bundled-data-file idiom needs no runtime registry. --- # FILE: references/reference/functions/asdf-load-system.md # asdf:load-system `(asdf:load-system name &rest options)` Loads the named system: its `:depends-on` systems first (recursively), then its component files in their `:depends-on`/`:serial` order, each evaluated like [`load`](load.md). Returns the system name as a symbol. Loading the same system twice is a no-op (like [`require`](require.md)). The system definition comes from a prior [`asdf:defsystem`](asdf-defsystem.md), or from `NAME.asd` located by searching, in order: the directory of the loading file, the directories given with `--system-path`, and the directories in the `RONTOLISP_SOURCE_REGISTRY` environment variable (both accept several directories joined with the platform path separator, like `PATH`). For a secondary system name such as `"lib/tests"` the file searched is the primary system's (`lib.asd`). Keyword options (`:verbose nil`, `:force t`, ...) are accepted and **ignored** on every backend: there is no `operate` machinery for them to drive and loading a system twice is already a no-op. They must still be `:keyword value` pairs, so a stray second system name is an error rather than a silently dropped load. On the interpreter, `load-system` is an ordinary runtime function, so a computed name works (the name may also be the metaobject [`asdf:find-system`](asdf-find-system.md) answers). On the compile path (JVM/WASM), a **literal, top-level** `(asdf:load-system NAME)` is expanded at compile time: the dependency systems and component files are spliced into the program exactly like the compile-time [`load`](load.md)/[`require`](require.md) include, so the compilers see the definitions natively on every backend. A `load-system` nested inside another form or with a computed argument compiles to a runtime call that answers `nil` when the system was already spliced — the "load if missing" shape libraries guard with a [`find-system`](asdf-find-system.md) probe — and signals for any system the compiled program does not carry (nothing can be loaded at run time). ```console ;; my-lib.asd (defsystem :my-lib :components ((:file "main" :depends-on ("package")) (:file "package"))) ;; run.lisp (asdf:load-system :my-lib) (my-lib:greet) ``` `package.lisp` loads before `main.lisp` (the `:depends-on` constraint), then the program calls into the loaded system. See the [Systems guide](../../guides/asdf-systems.md) for the full project layout and the search-path details. --- # FILE: references/reference/functions/asdf-registered-systems.md # asdf:registered-systems `(asdf:registered-systems)` Returns the downcased names of every registered system, in registration order: systems declared by an [`asdf:defsystem`](asdf-defsystem.md) or a parsed `.asd`, package-inferred sub-systems that have been derived, and built-in shim systems that have been loaded. ```lisp (asdf:defsystem :demo-a :components ((:file "main"))) (asdf:defsystem :demo-b :components ((:file "main"))) (asdf:registered-systems) ; => ("demo-a" "demo-b") ``` ## Backend support Works on all four backends. The interpreter answers its live registry; a compiled program answers the registry baked at compile time. --- # FILE: references/reference/functions/asdf-system-relative-pathname.md # asdf:system-relative-pathname `(asdf:system-relative-pathname system relative)` Returns the namestring of `relative` resolved against the source directory of the named `system` — the one-call form of merging a relative path onto `(asdf:system-source-directory system)`. This is how a library names a data file it bundles next to its `.asd`. `system` is a string, keyword or symbol designator (or a value returned by [`asdf:find-system`](asdf-load-system.md)); a system that is not registered is an error. `relative` takes either spelling — a namestring, or the pathname `#P"data/list.dat"` denotes. The answer stays a namestring (the ASDF locators are compile-time facts here, not pathname producers). On the compile path (JVM/WASM) the call is folded to that literal namestring while the program is being built, so a `with-open-file` over the result can be inlined into the artifact and the compiled program needs neither the system registry nor the file at run time. ```console $ cat my-lib.asd (defsystem :my-lib :components ((:file "main"))) $ cat main.lisp (print (asdf:system-relative-pathname :my-lib "data/tlds.dat")) $ rontolisp run.lisp --system-path . "/home/me/my-lib/data/tlds.dat" ``` --- # FILE: references/reference/functions/asdf-test-system.md # asdf:test-system `(asdf:test-system name)` Loads the named system, follows its `:in-order-to ((test-op (test-op ...)))` chain (loading and testing each chained system the same way), then runs the system's recorded `:perform (test-op (o c) ...)` body with the operation parameter bound to `nil` (there is no `operate` machinery) and the component parameter bound to the system's metaobject ([`asdf:find-system`](asdf-find-system.md)'s answer). Returns `t`. A system with no test-op wiring is a no-op, like real ASDF's default `perform`. This is the standard entry point fukamachi-style `.asd`s ship: ```console (defsystem "my-app" :components ((:file "main")) :in-order-to ((test-op (test-op "my-app/tests")))) (defsystem "my-app/tests" :depends-on ("my-app" "rove") :components ((:file "tests/main")) :perform (test-op (op c) (symbol-call :rove :run c))) ;; run the tests: (asdf:test-system "my-app") ``` The body is recorded as data when the `.asd` is parsed; its bare symbols resolve the way `asdf-user` would resolve them (`symbol-call` is `uiop:symbol-call`, `component-name` is `asdf:component-name`). A `:perform` with a method qualifier (`test-op :after (o c)`) or a `#.` reader macro in its body stays tolerated-and-ignored, as all test-op wiring used to be. ## Backend support Works on all four backends. On the compile paths a **literal, top-level** `(asdf:test-system NAME)` splices the system *and* its test-op chain at compile time (a plain `load-system` never pulls the tests system in), then runs the recorded bodies at run time; a nested/computed call can only reach systems the program already spliced. --- # FILE: references/reference/functions/ash.md # ash `(ash integer count)` Arithmetic shift of `integer` by `count` bit positions: left (toward more significant bits) when `count` is non-negative, right (with sign extension) when `count` is negative. The result is an exact integer of any magnitude on every backend. ```lisp (ash 1 4) ; => 16 ``` ```lisp (ash 255 -4) ; => 15 ``` --- # FILE: references/reference/functions/asin-acos-atan.md # asin acos atan `(asin number)` `(acos number)` `(atan number)` The inverse trigonometric functions, each returning an angle in radians as a float. `asin` is the arcsine, `acos` the arccosine, and `atan` the arctangent. Only the one-argument `atan` is supported -- there is no two-argument `(atan y x)` form. All three work on every backend: the interpreter and JVM use `Math.asin`/`Math.acos`/`Math.atan`, while the WASM backend computes `atan` with a software approximation (argument folding plus a Taylor series, ~1e-15 relative error) and derives `asin`/`acos` from it, so a WASM result can differ from the interpreter's and the JVM's in the last digit or two. `(asin 1)` is exactly `pi/2`, `(acos 1)` exactly `0.0`, and an `asin`/`acos` argument outside `[-1, 1]` returns `NaN`. ```lisp (atan 0) ; => 0.0 ``` --- # FILE: references/reference/functions/assoc-if.md # assoc-if `(assoc-if predicate alist)` Searches an association list and returns the first pair whose car satisfies `predicate`, or `nil` if none does. This is the predicate-based counterpart of `assoc`. ```lisp (assoc-if #'oddp '((2 a) (3 b))) ; => (3 B) ``` --- # FILE: references/reference/functions/assoc.md # assoc `(assoc key alist &key test key)` Searches an association list (a list of `(key . value)` pairs) and returns the first pair whose car matches `key`, or `nil` if none matches. The comparison is `eql` by default; the optional `:test` keyword takes a function designator to use a different comparison (e.g. `#'equal` for string keys), and the optional `:key` keyword takes a selector function applied to each pair's car before the comparison. The returned pair shares structure with the alist. Use `rassoc` to search by value instead of by key. ```lisp (assoc 'b '((a . 1) (b . 2))) ; => (B . 2) ``` ```lisp (assoc "b" '(("a" . 1) ("b" . 2)) :test #'equal) ; => ("b" . 2) ``` ```lisp (assoc 2 '((1 . a) (2 . b)) :key (lambda (k) (+ k 1))) ; => (1 . A) ``` --- # FILE: references/reference/functions/atom.md # atom `(atom object)` Returns `t` if `object` is not a cons cell, otherwise `nil`. Everything that is not a cons -- numbers, symbols, strings, characters, and `nil` -- is an atom, so `(atom nil)` is `t`. It is the exact complement of `consp`. Works in all three backends. ```lisp (atom 1) ; => T ``` ```lisp (atom '(1 2)) ; => NIL ``` --- # FILE: references/reference/functions/bit.md # bit `(bit bit-array index)` Reads the bit at `index` of a bit array (a `#*` literal or a `make-array` result with `:element-type 'bit`, represented as the general vector holding 0/1). `(setf (bit bit-array index) bit)` writes it. The non-simple twin of [`sbit`](sbit.md); both behave identically here. ```lisp (bit #*0110 1) ; => 1 ``` --- # FILE: references/reference/functions/both-case-p.md # both-case-p `(both-case-p character)` Returns true if the character is a cased letter (it has both an upper- and a lowercase form): `lower-case-p` or `upper-case-p`. ```lisp (both-case-p #\a) ; => T ``` ```lisp (both-case-p #\5) ; => NIL ``` --- # FILE: references/reference/functions/boundp.md # boundp `(boundp symbol)` Returns `t` when `symbol` names a bound **global** variable (`defvar`/`defparameter`/top-level `setq`), nil otherwise. Like Common Lisp's dynamic-only `boundp`, lexical bindings (`let`, function parameters) are invisible. `t`, `nil` and keywords are self-evaluating constants, so `boundp` of any of them is `t`. On the compiled backends a `boundp` of a LITERAL symbol is answered at compile time and costs nothing: a compiled program cannot make a global appear at run time, so the definitions before the call decide the answer. A computed symbol (`(boundp (intern name))`) is resolved against the embedded eval runtime's global environment instead, and pulls that runtime into the output like `eval` does — as [`symbol-value`](symbol-value.md) and [`fboundp`](fboundp.md) do whatever their argument is. A program compiled with `--dynamic`, or one that calls `eval`/`load`, keeps the run-time check throughout. ```lisp (defvar *level* 7) (boundp '*level*) ; => T ``` ```lisp (boundp '*undefined-var*) ; => NIL ``` ```lisp (boundp :key) ; => T ``` ```lisp (let ((x 1)) (boundp 'x)) ; => NIL ``` --- # FILE: references/reference/functions/butlast.md # butlast `(butlast list)` Returns a fresh copy of `list` with its last element removed; the original is not modified. An empty or single-element list yields `nil`. Unlike full Common Lisp, rontolisp's `butlast` takes only a list -- the optional count argument is not supported. ```lisp (butlast '(1 2 3 4)) ; => (1 2 3) ``` --- # FILE: references/reference/functions/byte-position.md # byte-position `(byte-position bytespec)` Returns the position (starting bit offset, 0 = least significant bit) of the byte specifier `bytespec` built by [`byte`](byte.md). ```lisp (byte-position (byte 8 3)) ; => 3 ``` --- # FILE: references/reference/functions/byte-size.md # byte-size `(byte-size bytespec)` Returns the size (bit count) of the byte specifier `bytespec` built by [`byte`](byte.md). ```lisp (byte-size (byte 8 3)) ; => 8 ``` --- # FILE: references/reference/functions/byte.md # byte `(byte size position)` Builds a byte specifier naming a field of `size` bits starting at bit `position` (0 = least significant bit), for use with [`ldb`](ldb.md) and [`dpb`](dpb.md). The specifier is an ordinary object whose parts are read back with [`byte-size`](byte-size.md) and [`byte-position`](byte-position.md). ```lisp (byte-size (byte 8 3)) ; => 8 ``` --- # FILE: references/reference/functions/car-cdr-compositions.md # caar cddddr `(cadr list)`, `(caddr list)`, ... -- every `c{a,d}+r` name from two to four levels deep. Each composite accessor applies a sequence of `car`/`cdr` operations read right to left: `cadr` is `(car (cdr x))`, `caddr` is `(car (cdr (cdr x)))`, `cddr` is `(cdr (cdr x))`, and so on through all 2-, 3-, and 4-level combinations up to `cddddr`. Like `car`/`cdr`, applying one to `nil` along the way yields `nil` instead of erroring. ```lisp (caddr '(1 2 3 4)) ; => 3 ``` ```lisp (cddr '(1 2 3 4)) ; => (3 4) ``` --- # FILE: references/reference/functions/car.md # car `(car list)` Returns the car (first element) of a cons cell. As a special case `(car nil)` is `nil` rather than an error, so traversing to the end of a list is safe. Use `first` as a more readable synonym when working with lists. ```lisp (car '(1 2 3)) ; => 1 ``` ```lisp (car nil) ; => NIL ``` --- # FILE: references/reference/functions/case-char-p.md # lower-case-p, upper-case-p `(lower-case-p character)` `(upper-case-p character)` `lower-case-p` returns `t` if `character` is a lowercase letter, `upper-case-p` if it is an uppercase letter, and `nil` otherwise. A character counts as lowercase exactly when upcasing it changes it (and uppercase when downcasing changes it), so both follow the platform's Unicode case tables. Available on all backends except `--no-gc`. ```lisp (list (lower-case-p #\a) (upper-case-p #\A) (lower-case-p #\5)) ; => (T T NIL) ``` --- # FILE: references/reference/functions/cdr.md # cdr `(cdr list)` Returns the cdr of a cons cell -- the rest of a list after its first element. As a special case `(cdr nil)` is `nil` rather than an error. Use `rest` as a more readable synonym when working with lists. ```lisp (cdr '(1 2 3)) ; => (2 3) ``` ```lisp (cdr nil) ; => NIL ``` --- # FILE: references/reference/functions/ceiling.md # ceiling `(ceiling number &optional divisor)` Rounds `number` (or `number/divisor` when a divisor is given) toward positive infinity to an integer. In an ordinary (single-value) context the result is the quotient only; the remainder is the second value, observable through [`multiple-value-bind`](../macros/multiple-value-bind.md) and the other multiple-value consumers. ```lisp (ceiling 3.2) ; => 4 ``` ```lisp (multiple-value-bind (q r) (ceiling 7 2) (list q r)) ; => (4 -1) ``` --- # FILE: references/reference/functions/cell-error-name.md # cell-error-name `(cell-error-name condition)` The `name` slot of a `cell-error` condition -- the name of the cell that could not be accessed. Every `cell-error` subtype carries it, `unbound-variable`, `undefined-function` and [`unbound-slot`](../macros/slot-boundp.md) included. ```lisp (defclass ce-box () ((v))) (handler-case (slot-value (make-instance 'ce-box) 'v) (unbound-slot (e) (cell-error-name e))) ; => V ``` --- # FILE: references/reference/functions/char-case.md # char-upcase char-downcase `(char-upcase character)` -- `(char-downcase character)` Return the upper- or lower-case form of a single character; a character that has no case (such as a digit or punctuation) is returned unchanged. `char-upcase` maps lowercase letters to uppercase and `char-downcase` does the reverse. The WASM backend folds case using ASCII rules only. ```lisp (char-upcase #\a) ; => #\A ``` --- # FILE: references/reference/functions/char-code.md # char-code `(char-code character)` Returns the integer code point of `character`. For ASCII characters this is the familiar value (for example `#\A` is `65`). It is the inverse of `code-char`. ```lisp (char-code #\A) ; => 65 ``` --- # FILE: references/reference/functions/char-compare-ci.md # char-lessp char-greaterp char-not-lessp char-not-greaterp char-not-equal `(char-lessp character &rest characters)` -- `(char-greaterp character &rest characters)` -- `(char-not-lessp character &rest characters)` -- `(char-not-greaterp character &rest characters)` -- `(char-not-equal character &rest characters)` The case-INSENSITIVE character ordering family, the counterparts of `char<` / `char>` / `char>=` / `char<=` / `char/=`: each argument is downcased before its code point is compared. `char-lessp` is true when the arguments are in strictly increasing order, `char-greaterp` in strictly decreasing order, `char-not-lessp` in non-increasing order, `char-not-greaterp` in non-decreasing order, and `char-not-equal` when ALL arguments are pairwise distinct. ```lisp (list (char-lessp #\a #\B) (char-lessp #\B #\a) (char-greaterp #\b #\A) (char-not-equal #\a #\b #\A)) ; => (T NIL T NIL) ``` --- # FILE: references/reference/functions/char-compare.md # char= char< char<= char> char>= char/= char-equal `(char= character &rest characters)` -- `(char< character &rest characters)` -- `(char<= character &rest characters)` -- `(char> character &rest characters)` -- `(char>= character &rest characters)` -- `(char/= character &rest characters)` -- `(char-equal character &rest characters)` Compare characters by their code points and return `t` or `nil`. All are variadic: `char=` is true when every argument is the same character, `char<`/`char<=` when the arguments are in strictly increasing / non-decreasing order, `char>`/`char>=` when they are in strictly decreasing / non-increasing order, `char/=` when ALL arguments are pairwise distinct, and `char-equal` is the case-insensitive `char=`. ```lisp (char< #\a #\b #\c) ; => T ``` --- # FILE: references/reference/functions/char-name.md # char-name `(char-name character)` The name of a non-graphic character (`"Space"`, `"Newline"`, `"Tab"`, ...), a `"U+XXXX"` form for other non-printing code points, or nil for a graphic character. ```lisp (char-name #\Space) ; => "Space" ``` ```lisp (char-name #\a) ; => NIL ``` --- # FILE: references/reference/functions/char.md # char schar `(char string index)` -- `(schar string index)` Returns the character at the 0-based `index` of `string`. `char` and `schar` behave identically here; in Common Lisp `schar` is the simple-string variant, but rontolisp treats them the same. The WASM backend indexes strings by byte, so indexing is well-defined for ASCII text only. Both are also `setf` places: `(setf (schar s i) c)` / `(setf (char s i) c)` replaces the character at `i` and returns `c`. The interpreter mutates the string in place; the compiled backends rebuild the string and rebind it, so the string expression must be a **variable** there, and an alias made before the write still sees the old content. ```lisp (char "hello" 1) ; => #\e ``` --- # FILE: references/reference/functions/characterp.md # characterp `(characterp object)` Returns `t` if `object` is a character, otherwise `nil`. Character literals are written with the `#\` prefix, e.g. `#\a`. A one-character string is not a character, so `(characterp "a")` is `nil`. Works in all three backends. ```lisp (characterp #\a) ; => T ``` ```lisp (characterp "a") ; => NIL ``` --- # FILE: references/reference/functions/class-name.md # class-name `(class-name class)` Returns the name symbol of a class metaobject -- what [`find-class`](find-class.md) and [`class-of`](class-of.md) answer. Signals when the argument is not a class metaobject. ```lisp (class-name (class-of "hello")) ; => STRING ``` --- # FILE: references/reference/functions/class-of.md # class-of `(class-of object)` Returns the class METAOBJECT of any value -- the same memoized `standard-class` instance [`find-class`](find-class.md) answers, so `(eq (class-of x) (find-class 'name))` holds. A CLOS instance answers its class, a `defstruct` instance its structure type (as a `standard-class` instance too -- there is no `structure-class`), and every other value a slot-less built-in class named `integer`, `string`, `cons`, ..., with `t` for values outside that set (arrays included). Read the name with [`class-name`](class-name.md). ```lisp (defclass point () ((x :initarg :x))) (list (class-name (class-of 42)) (class-name (class-of (make-instance 'point))) (eq (class-of 42) (find-class 'integer))) ; => (INTEGER POINT T) ``` --- # FILE: references/reference/functions/clear-output.md # clear-output `(clear-output &optional stream)` Discards whatever the designated output stream has buffered but not yet written, and returns nil. No backend buffers output in a way a program could throw away -- a write reaches the underlying sink as it is made -- so this validates its stream designator and does nothing else. It exists because the [Gray protocol](../../guides/gray-streams.md) names `stream-clear-output`, which a portable stream class implements: on a Gray instance the call reaches that method. ```lisp (progn (princ "kept") (clear-output)) ; => NIL ``` --- # FILE: references/reference/functions/close.md # close `(close stream)` Closes a stream previously returned by `open` and returns `t`. After closing, the stream handle is no longer valid and must not be used for further `read`/`read-line`/`write-line` calls. Works in all three backends. In most code you should prefer the `with-open-file` macro, which arranges the matching `close` for you even when the body exits early. ```console (let ((s (open "out.txt" :output))) (write-line "done" s) (close s)) ``` Here the stream is opened for output, written to, and then closed; the `close` call flushes and releases the file descriptor. --- # FILE: references/reference/functions/clrhash.md # clrhash `(clrhash table)` Removes all entries from `table`, leaving it empty, and returns the now-empty table. The same table object is reused, so existing references to it remain valid. ```lisp (let ((h (make-hash-table))) (setf (gethash 'a h) 1) (clrhash h) (hash-table-count h)) ; => 0 ``` --- # FILE: references/reference/functions/code-char.md # code-char `(code-char code)` Returns the character whose code point is the integer `code` (for example `66` yields `#\B`). It is the inverse of `char-code`. ```lisp (code-char 66) ; => #\B ``` --- # FILE: references/reference/functions/coerce.md # coerce `(coerce object result-type)` Converts `object` to the given sequence or float type. `result-type` may be `'list`, `'vector`, `'string` (and their `simple-`/`base-` spellings), a compound spec such as `'(vector t)` or `'(string 8)`, a float type (`'float`, `'single-float`, `'double-float`, `'short-float`, `'long-float` -- all the one double representation), or `t` (the identity). A COMPUTED result type is accepted too and dispatches at runtime over exactly those families, so an expression like `(coerce seq type)` with `type` in a variable behaves the same as the literal form. A `'string` result requires a sequence of characters, and a value already of the requested type is returned unchanged. `coerce` is not a first-class function value (`#'coerce` is unavailable), so call it directly. A vector `result-type` spelling an `(unsigned-byte 8)`, `(unsigned-byte 16)` or `(unsigned-byte 32)` element type -- `'(vector (unsigned-byte 8))`, `'(simple-array (unsigned-byte 32) (*))` -- builds a specialized vector of that element type, the same representation [`make-array`](make-array.md) and [`concatenate`](concatenate.md) produce, so `array-element-type` reports it and `typep` against the matching `simple-array` specifier answers true. Elements are stored masked to the element width. Any other element type builds a general vector, whose element type is `t`. This is the spelling a lookup table usually takes, and a table whose elements are all literals is built at compile time on the compiled backends. ```lisp (coerce '(1 2 3) 'vector) ; => #(1 2 3) (coerce (vector 1 2 3) 'list) ; => (1 2 3) (coerce "ab" 'list) ; => (#\a #\b) (coerce '(#\a #\b) 'string) ; => "ab" ``` ```lisp (coerce '(1 2 260) '(vector (unsigned-byte 8))) ; => #(1 2 4) (array-element-type (coerce '(1) '(simple-array (unsigned-byte 32) (*)))) ; => (UNSIGNED-BYTE 32) ``` ```lisp (coerce 1/4 'double-float) ; => 0.25 ``` ```lisp (defun convert (seq type) (coerce seq type)) (convert (vector 1 2) 'list) ; => (1 2) ``` --- # FILE: references/reference/functions/compile-file.md # compile-file `(compile-file input-file &key ...)` `(compile-file-pathname input-file &key ...)` `(remove-method generic-function method)` The three standard names that exist so a program mentioning them loads, and signal an error when they are actually called. `compile-file` and `compile-file-pathname` have nothing to name. rontolisp has no file compiler: a program is compiled **whole** in one pass and a `load`ed file is spliced into it, so no fasl is ever produced and no pathname names one -- which is also why `*compile-file-pathname*` and `*compile-file-truename*` are permanently nil. Both signal rather than answering a fabricated pathname for a file that will never exist. To compile, run the compiler: `rontolisp prog.lisp -o Prog.class`, `-o prog.wasm`. `remove-method` has no method to remove. A method here is a registry row plus a generated function, never a first-class object, and there is no `find-method` to obtain one from -- so no caller can name the method it means. ```console $ rontolisp -e '(compile-file "x.lisp")' Unhandled condition: compile-file is not supported (no file compiler: a program is compiled whole) ``` ## Backend support All three behave identically on all four backends. --- # FILE: references/reference/functions/compile.md # compile `(compile name definition)` Coerces a literal `(lambda ...)` definition to a function, evaluated in the null lexical environment. With `name` nil, the function itself is returned; with a symbol `name`, the function is also installed as the global definition of that name and the name is returned, as in Common Lisp. rontolisp's `compile` does not generate native code -- it exists so definition-time code construction idioms load: in particular, a no-argument definition whose body defines methods over class metaobjects (the `(funcall (compile nil `(lambda () ,code)))` idiom of object mappers, where `code` closes over values computed from the class) is executed as definition-time method construction. In a compiled program (JVM / WASM output), that method-construction idiom is expanded at compile time and spliced into the program, and the run-time re-execution of the same call is a no-op; any other run-time `compile` call signals an error, because a compiled program carries no compiler. Use [`eval`](eval.md) for run-time evaluation of plain expressions. ```lisp (list (funcall (compile nil '(lambda (x) (* x x))) 7) (compile 'my-inc '(lambda (x) (+ x 1))) (my-inc 41)) ; => (49 MY-INC 42) ``` --- # FILE: references/reference/functions/compiled-function-p.md # compiled-function-p `(compiled-function-p object)` Lite stub: always returns `nil` (no distinction between compiled and interpreted function objects is recorded). ```lisp (compiled-function-p #'car) ; => NIL ``` --- # FILE: references/reference/functions/compute-restarts.md # compute-restarts `(compute-restarts [condition])` Returns a list of every active restart record, innermost first (clauses of one [`restart-case`](../macros/restart-case.md) in their written order). Each element answers to [`restart-name`](restart-name.md) and can be passed to [`invoke-restart`](invoke-restart.md). Lite: the optional `condition` argument is accepted and ignored. ```lisp (restart-case (restart-case (mapcar (function restart-name) (compute-restarts)) (aaa () nil) (bbb () nil)) (ccc () nil)) ; => (AAA BBB CCC) ``` --- # FILE: references/reference/functions/concatenate.md # concatenate `(concatenate result-type &rest sequences)` Joins its sequence arguments into one new sequence of `result-type`. Three result families are supported: `'string` (also `'simple-string` / `'base-string`), `'list` (also `'cons`), and `'vector` (also `'simple-vector` / `'array` / `'bit-vector`, and compound specs such as `'(vector (unsigned-byte 8))`). Every family walks any mix of sequence arguments: the `'string` family too takes any character sequence, `nil` -- the empty list -- included. A `result-type` naming a user `deftype` resolves through its registered expansion to one of the families. With no sequences given it returns the empty sequence of that type, and the result is always fresh -- no argument is shared with it. In the compiled backends `result-type` must be written as a literal quoted designator; the interpreter also accepts a computed one. A vector `result-type` spelling an `(unsigned-byte 8)`, `(unsigned-byte 16)` or `(unsigned-byte 32)` element type -- `'(vector (unsigned-byte 8))`, `'(simple-array (unsigned-byte 8) (*))` -- builds a specialized vector of that element type, the same representation [`make-array`](make-array.md) produces, so `array-element-type` reports it and `typep` against the matching `simple-array` specifier answers true. Elements are stored masked to the element width. Any other element type builds a general vector, whose element type is `t`. ```lisp (concatenate 'string "foo" "bar") ; => "foobar" (concatenate 'string "a" '(#\b #\c) nil "d") ; => "abcd" (concatenate 'list '(1 2) "ab" #(3)) ; => (1 2 #\a #\b 3) (concatenate 'vector '(1 2) #(3)) ; => #(1 2 3) (concatenate '(vector (unsigned-byte 8)) #(1) #(2 3)) ; => #(1 2 3) (array-element-type (concatenate '(vector (unsigned-byte 8)) #(1))) ; => (UNSIGNED-BYTE 8) (array-element-type (concatenate 'vector #(1))) ; => T (progn (deftype octet-vector () '(simple-array (unsigned-byte 8) (*))) (concatenate 'octet-vector #(1) #(2 3))) ; => #(1 2 3) ``` --- # FILE: references/reference/functions/cons.md # cons `(cons object1 object2)` Allocates and returns a fresh cons cell whose car is `object1` and whose cdr is `object2`. When the cdr is a list the result is a longer list; otherwise it is a dotted pair. This is the fundamental list-building primitive. ```lisp (cons 1 '(2 3)) ; => (1 2 3) ``` ```lisp (cons 1 2) ; => (1 . 2) ``` --- # FILE: references/reference/functions/consp.md # consp `(consp object)` Returns `t` if `object` is a cons cell, otherwise `nil`. The empty list `nil` is not a cons cell, so `(consp nil)` is `nil` -- this is where `consp` differs from `listp`. It is the exact complement of `atom`. Works in all three backends. ```lisp (consp '(1 2)) ; => T ``` ```lisp (consp nil) ; => NIL ``` --- # FILE: references/reference/functions/constantly.md # constantly `(constantly value)` Returns a function of any number of arguments that always answers `value`. The idiomatic "accept everything" argument -- `uiop:collect-sub*directories` takes two of them. ```lisp (mapcar (constantly 7) '(a b c)) ; => (7 7 7) ``` ## Backend support All four backends -- one definition in rontolisp source, so unlike `complement` (which expands into a wrapping lambda) it is a real function and `#'constantly` works. --- # FILE: references/reference/functions/constantp.md # constantp `(constantp form &optional environment)` Returns `t` if `form` is a constant object and `nil` otherwise. This is a lite implementation: it recognizes self-evaluating objects (numbers, strings, characters, keywords, `t`, and `nil`) and `(quote x)` forms. Anything else -- including a plain symbol or a function-call form -- yields `nil`. False negatives are harmless (a consumer just defers the work to runtime). The optional `environment` argument is accepted and ignored (a macro's `&environment` parameter is bound to nil). Available on all backends except `--no-gc`. ```lisp (list (constantp 5) (constantp 'x) (constantp '(quote y))) ; => (T NIL T) ``` --- # FILE: references/reference/functions/continue.md # continue `(continue [condition])` Invokes the innermost active `continue` restart — the one a [`cerror`](../macros/cerror.md) establishes — and returns `nil` when none is active (unlike [`abort`](abort.md), this is not an error). Calling it from a [`handler-bind`](../macros/handler-bind.md) handler makes the interrupted `cerror` return `nil` and execution resume past it. ```lisp (handler-bind ((error (lambda (c) (continue)))) (list :after (cerror "Carry on." "recoverable problem"))) ; => (:AFTER NIL) ``` --- # FILE: references/reference/functions/copy-alist.md # copy-alist `(copy-alist alist)` Returns a copy of an association list: both the list spine and each `(key . value)` pair cell are fresh cons cells, so mutating a pair in the copy (e.g. with `rplacd` or `setf` of `cdr`) leaves the original alist intact. The keys and values themselves are shared, not copied. ```lisp (copy-alist '((a . 1) (b . 2))) ; => ((A . 1) (B . 2)) ``` ```lisp (let* ((orig '((a . 1))) (copy (copy-alist orig))) (rplacd (assoc 'a copy) 99) (cdr (assoc 'a orig))) ; => 1 ``` --- # FILE: references/reference/functions/copy-list.md # copy-list `(copy-list list)` Returns a shallow copy of `list`: the top-level cons cells are freshly allocated, but the elements themselves are shared with the original. This lets you destructively modify the copy's structure without affecting the source. Nested sublists are not copied. ```lisp (copy-list '(1 2 3)) ; => (1 2 3) ``` --- # FILE: references/reference/functions/copy-readtable.md # copy-readtable `(copy-readtable &optional from to)` Lite stub: a no-op returning `nil` — the reader is not readtable-driven, so there is no readtable object to copy (the `*readtable*` variable exists but is seeded to `nil`). The arguments are still evaluated. Exists so the common library header idiom `(defparameter *my-readtable* (copy-readtable nil))` loads. ```lisp (copy-readtable nil) ; => NIL ``` --- # FILE: references/reference/functions/copy-seq.md # copy-seq `(copy-seq sequence)` Returns a fresh copy of `sequence` — a list or a string — equivalent to `(subseq sequence 0)`. The copy shares no cons cells with the original, so destructive operations on one do not affect the other. ```lisp (let ((original '(1 2 3))) (eq (copy-seq original) original)) ; => NIL ``` ```lisp (copy-seq "xyz") ; => "xyz" ``` --- # FILE: references/reference/functions/copy-symbol.md # copy-symbol `(copy-symbol symbol &optional copy-properties)` Returns a fresh uninterned symbol with the same name as `symbol`. `copy-properties` is accepted and ignored: there is no `(setf symbol-plist)` to carry a property list across with. The copy inherits [`make-symbol`](make-symbol.md)'s identity deviation exactly. A symbol here IS its spelling and there is no intern table, so two uninterned symbols of one name are `eq` -- a copy is not distinguishable from any other copy of the same symbol. Code that only needs a name nobody else uses should reach for [`gensym`](gensym.md), which does give a fresh one every call. ```lisp (symbol-name (copy-symbol 'foo)) ; => "FOO" ``` ## Backend support Works on all four backends: one definition in rontolisp source over `make-symbol`. --- # FILE: references/reference/functions/copy-tree.md # copy-tree `(copy-tree tree)` Returns a deep copy of a cons tree: every cons cell is fresh, while non-cons leaves (numbers, symbols, strings, ...) are shared with the original. Compare `copy-list`, which copies only the top-level spine. ```lisp (copy-tree '(1 (2 3) . 4)) ; => (1 (2 3) . 4) ``` ```lisp (let* ((orig (list (list 1 2))) (copy (copy-tree orig))) (setf (car (car copy)) 99) orig) ; => ((1 2)) ``` --- # FILE: references/reference/functions/count-if-not.md # count-if-not `(count-if-not predicate sequence &key key start end from-end)` Returns the number of elements of `sequence` that do **not** satisfy `predicate` -- the complement of `count-if`. The sequence may be a list, a vector or a string. `:key` selects what the predicate sees, `:start`/`:end` bound the scanned region, and `:from-end` is accepted but changes nothing (it only reorders the predicate calls, which cannot change a count). ```lisp (count-if-not #'evenp '(1 2 3 4 5)) ; => 3 ``` ```lisp (count-if-not #'alpha-char-p "ab1c2") ; => 2 ``` ```lisp (count-if-not #'oddp '((1) (2) (3)) :key #'car) ; => 1 ``` --- # FILE: references/reference/functions/count-if.md # count-if `(count-if predicate sequence)` Returns the number of elements in `sequence` that satisfy `predicate`. The sequence may be a list or a string (whose elements are characters). This is the predicate-based counterpart of `count`. ```lisp (count-if #'evenp '(1 2 3 4)) ; => 2 ``` ```lisp (count-if #'digit-char-p "a1b2") ; => 2 ``` --- # FILE: references/reference/functions/count.md # count `(count item sequence &key test key)` Returns the number of elements in `sequence` that match `item`. The comparison is `eql` by default; the optional `:test` keyword takes a function designator to use a different comparison, and the optional `:key` keyword takes a selector function applied to each element before the comparison. The sequence may be a list or a string (whose elements are characters). Use `count-if` to count by a predicate. ```lisp (count 2 '(1 2 3 2 2)) ; => 3 ``` ```lisp (count #\a "banana") ; => 3 ``` ```lisp (count "a" '("a" "b" "a") :test #'string=) ; => 2 ``` --- # FILE: references/reference/functions/decode-float.md # decode-float `(decode-float float)` Returns three values: the significand as a float in [1/2, 1), the binary exponent, and the sign (`1.0` or `-1.0`), such that `significand * 2^exponent * sign` is the original number. Zero decodes as `0.0`, `0` and its sign. The decomposition scales by two, which is exact in binary floating point, so every backend returns bit-identical values. [`scale-float`](scale-float.md) is the reverse step. ```lisp (multiple-value-list (decode-float 6.5)) ; => (0.8125 3 1.0) ``` --- # FILE: references/reference/functions/decode-universal-time.md # decode-universal-time `(decode-universal-time universal-time &optional time-zone)` Splits a universal time into the nine decoded values `second`, `minute`, `hour`, `date`, `month`, `year`, `day-of-week` (0 = Monday), `daylight-p` and `time-zone`. Like [`encode-universal-time`](encode-universal-time.md), a missing zone means GMT; `daylight-p` is always nil (no backend knows a daylight-saving rule). ```lisp (multiple-value-list (decode-universal-time 2208988800 0)) ; => (0 0 0 1 1 1970 3 NIL 0) ``` --- # FILE: references/reference/functions/delete-duplicates.md # delete-duplicates `(delete-duplicates sequence &key test key from-end)` Returns the sequence with duplicate elements removed — `remove-duplicates`' would-be-destructive twin, sharing its rendering (the standard requires callers to use the RESULT, so the non-destructive scan is conforming, like `sort` via `stable-sort`). By default the last occurrence of each element survives; `:from-end t` keeps the FIRST occurrence instead. The comparison is `eql` by default; `:test` takes a comparison function designator and `:key` a selector applied to both sides. `:from-end` must be a literal `t` or `nil`. ```lisp (delete-duplicates '(1 2 1 3 2)) ; => (1 3 2) ``` ```lisp (delete-duplicates '(1 2 1 3 2) :from-end t) ; => (1 2 3) ``` ```lisp (delete-duplicates '((1 . :a) (1 . :b) (2 . :c)) :key #'car :from-end t) ; => ((1 . :A) (2 . :C)) ``` --- # FILE: references/reference/functions/delete-file.md # delete-file `(delete-file pathname)` Deletes the named file and returns `t`. Anything that leaves the file in place is an error -- including "it was not there to begin with", which Common Lisp also makes a `file-error`. Probe first with [`probe-file`](probe-file.md) when a missing file should be tolerated, or wrap the call in `ignore-errors`. **Both WASM backends signal at call time.** No WASI unlink call is imported there, and unlike [`file-write-date`](file-write-date.md) this operation has no "cannot be determined" answer in its contract: either the file is gone afterwards or it is not, so answering anything but an error would be a lie. This is the same deal [`ensure-directories-exist`](ensure-directories-exist.md) makes, for the same reason -- a program merely CONTAINING the call still compiles there; only executing it signals. ```console (with-open-file (out "notes.txt" :direction :output) (write-line "draft" out)) (delete-file "notes.txt") ; => T (probe-file "notes.txt") ; => NIL (delete-file "notes.txt") ; signals: DELETE-FILE: cannot delete notes.txt ``` --- # FILE: references/reference/functions/delete-if-not.md # delete-if-not `(delete-if-not predicate list)` The destructive counterpart of `remove-if-not`: returns `list` keeping only the elements that satisfy `predicate`, splicing out the rest in place. Because the head may change, use the return value rather than the original variable. ```lisp (delete-if-not #'evenp '(1 2 3 4)) ; => (2 4) ``` --- # FILE: references/reference/functions/delete-if.md # delete-if `(delete-if predicate list)` The destructive counterpart of `remove-if`: returns `list` with every element satisfying `predicate` spliced out in place. Because the head may change, use the return value rather than the original variable. ```lisp (delete-if #'evenp '(1 2 3 4)) ; => (1 3) ``` --- # FILE: references/reference/functions/delete.md # delete `(delete item list &key test key)` The destructive counterpart of `remove`: returns `list` with every element matching `item` spliced out, modifying the cons cells in place rather than copying. The comparison is `eql` by default; the optional `:test` keyword takes a function designator to use a different comparison, and the optional `:key` keyword takes a selector function applied to each element before the comparison. Because the head of the list may change, always use the return value rather than relying on the original variable. ```lisp (delete 2 '(1 2 3 2)) ; => (1 3) ``` ```lisp (delete "b" (list "a" "b" "c") :test #'string=) ; => ("a" "c") ``` --- # FILE: references/reference/functions/denominator.md # denominator `(denominator rational)` Returns the denominator of a rational number in its reduced form. For an integer the denominator is always `1`. ```lisp (denominator 3/4) ; => 4 ``` ```lisp (denominator 5) ; => 1 ``` --- # FILE: references/reference/functions/digit-char-p.md # digit-char-p `(digit-char-p character &optional radix)` Returns the integer weight of `character` as a digit in the given `radix` (default 10), or `nil` if it is not a valid digit in that radix. For example `#\7` is `7`, and `#\f` is `15` in radix 16. Letters are accepted as digits for radices above 10, case-insensitively. ```lisp (digit-char-p #\7) ; => 7 ``` --- # FILE: references/reference/functions/digit-char.md # digit-char `(digit-char weight &optional radix)` Returns the character denoting `weight` in `radix` (10 by default), in upper case, or `nil` when the weight is not a non-negative integer below the radix. It is the inverse of [`digit-char-p`](digit-char-p.md). ```lisp (list (digit-char 7) (digit-char 11 16) (digit-char 12)) ; => (#\7 #\B NIL) ``` --- # FILE: references/reference/functions/directory.md # directory `(directory pathspec)` The pathnames matching `pathspec`, sorted with `string<` so the same program prints the same answer on every backend whatever order the host hands entries back in. `nil` when nothing matches -- it never signals. A **wild name component** lists the directory and keeps what the pattern matches. `*` stands for any sequence of characters, `?` for exactly one, and the answers keep the pathspec's own directory prefix so each is directly openable, with a subdirectory carrying a trailing `/`: ```lisp (directory "no-such-directory/*.*") ; => NIL ``` Given a directory holding `a.txt`, `b.txt` and the subdirectories `sub/` and `empty/`: | pathspec | answer | |---|---| | `"d/*.*"` | `(#P"d/a.txt" #P"d/b.txt" #P"d/empty/" #P"d/sub/")` — everything | | `"d/*.txt"` | `(#P"d/a.txt" #P"d/b.txt")` | | `"d/?.txt"` | `(#P"d/a.txt" #P"d/b.txt")` | | `"d/*"` | `(#P"d/empty/" #P"d/sub/")` — a wild name with NO type, so only untyped entries | | `"d/a*"` | `NIL` — same rule: `a.txt` has a type | A **non-wild pathspec designates itself**, as in Common Lisp: `"d/a.txt"` answers `(#P"d/a.txt")` when the file exists, and a directory answers itself in directory form -- both `"d"` and `"d/"` give `(#P"d/")`. **Listing a directory is `"d/*.*"`, not `"d/"`.** A **wild directory component** expands the prefix before the name is matched. `*` walks exactly one level and `**` walks the whole subtree -- and, as in Common Lisp, `**` matches ZERO levels as well as many, so the base directory's own files come back too: | pathspec | answer | |---|---| | `"d/*/*.lisp"` | the `.lisp` files one level below `d/` | | `"d/**/*.lisp"` | every `.lisp` file in `d/` and anywhere below it | | `"d/**/"` | `d/` itself and every directory below it | Every expectation above is the same answer SBCL gives for the same tree. ## Backend support All four backends, through one primitive each: the source-loader abstraction on the interpreter (so a host without a filesystem, such as the browser playground, answers `nil` rather than failing), `java.io.File.list` on the JVM, and WASI `fd_readdir` on both WASM backends -- Preview 1 binds the real host function, `--component` an adapter over `wasi:filesystem`'s `read-directory`. A WASM module resolves the path against its preopened directories -- a relative path against the first one, an absolute path against the preopened directory whose name is its longest prefix -- so run it with `--dir`; without one nothing matches. The `.` and `..` self/parent entries are never returned on any backend. Everything else in the family -- `uiop:directory-files`, `uiop:subdirectories`, `uiop:collect-sub*directories` and `uiop:directory-exists-p` -- is defined in terms of this one function. --- # FILE: references/reference/functions/div.md # / `(/ number &rest numbers)` Divides the first argument by the rest, left to right; with one argument returns its reciprocal. Integer division is exact: when the result is not a whole number it is returned as a reduced ratio rather than truncated, and an evenly dividing pair yields an integer. If any argument is a float the result is a float. Dividing by zero signals an error. ```lisp (/ 1 2) ; => 1/2 ``` ```lisp (/ 10 2) ; => 5 ``` --- # FILE: references/reference/functions/dpb.md # dpb `(dpb newbyte bytespec integer)` Deposit byte: returns a copy of `integer` with the field named by the byte specifier `bytespec` (see [`byte`](byte.md)) replaced by the low `size` bits of `newbyte`; the other bits are unchanged. `integer` may be any magnitude on every backend. ```lisp (dpb 0 (byte 4 0) 255) ; => 240 ``` --- # FILE: references/reference/functions/elt.md # elt `(elt sequence index)` Returns the element at zero-based `index` of `sequence`: a character for a string, the element for a list (the same traversal as `nth` with the arguments swapped) or for a vector. It is also a `setf` place -- see [`setf`](../macros/setf.md) for what each sequence kind does on a write. ```lisp (elt '(a b c) 1) ; => B ``` ```lisp (elt "abcd" 1) ; => #\b ``` ```lisp (elt (vector 10 20 30) 2) ; => 30 ``` --- # FILE: references/reference/functions/encode-universal-time.md # encode-universal-time `(encode-universal-time second minute hour date month year &optional time-zone)` Returns the universal time -- seconds since the Common Lisp epoch of 1900-01-01 00:00:00 GMT -- denoted by the decoded components. `time-zone` is the offset west of GMT in hours; **a missing (or nil) zone means GMT, not the machine's local zone**, because no backend-portable local-zone source exists (the WASM targets expose no timezone at all). The calendar arithmetic is exact for any year: it runs the era-based proleptic-Gregorian algorithm over integers, so every backend answers identically. ```lisp (encode-universal-time 0 0 0 1 1 1970 0) ; => 2208988800 ``` The inverse is [`decode-universal-time`](decode-universal-time.md), and [`get-universal-time`](get-universal-time.md) reads the clock in the same units. --- # FILE: references/reference/functions/endp.md # endp `(endp list)` The end-of-list test: returns `t` when `list` is `nil` (the empty list) and `nil` when it is a cons cell. It is the canonical way to detect the end while cdr-ing down a list. In rontolisp it behaves as a synonym for `null` -- the strict improper-list type check of standard Common Lisp is relaxed. ```lisp (endp '(1)) ; => NIL ``` ```lisp (endp nil) ; => T ``` --- # FILE: references/reference/functions/enough-namestring.md # enough-namestring `(enough-namestring pathname &optional defaults)` The shortest namestring that still names the same file once merged against `defaults` (which itself defaults to `*default-pathname-defaults*`, initially `#P""`). The value is a STRING, not a pathname. It is the inverse of [`merge-pathnames`](merge-pathnames.md): the merge prefixes a relative namestring with the defaults' DIRECTORY, so the shortest namestring is the path with that directory prefix removed. When the path does not start with the prefix nothing can be dropped and the whole namestring is the answer. ```lisp (list (enough-namestring "/a/b/c.lisp" "/a/") (enough-namestring "/a/b/c.lisp" "/x/") (namestring (merge-pathnames (enough-namestring "/a/b/c.lisp" "/a/") "/a/"))) ; => ("b/c.lisp" "/a/b/c.lisp" "/a/b/c.lisp") ``` `*default-pathname-defaults*` is a genuine dynamic variable on every backend, so binding it around a block of path work works: ```lisp (let ((*default-pathname-defaults* #P"/a/b/")) (enough-namestring "/a/b/c.lisp")) ; => "c.lisp" ``` ## Backend support All four backends -- one definition in rontolisp source over primitives every backend has. --- # FILE: references/reference/functions/ensure-directories-exist.md # ensure-directories-exist `(ensure-directories-exist pathspec)` Creates the directory component of `pathspec`, including every missing parent, and returns `pathspec`. The directory component is everything up to and including the last `/`, so `"logs/app.log"` creates `logs/` and leaves the file alone; a namestring that already ends in `/` *is* the directory; a namestring with no `/` names a file in the working directory and creates nothing. An existing directory is not an error. Lite: Common Lisp returns `(values pathspec created)` and this returns the pathspec only — a second value would not survive the function boundary on the compiled backends, so promising one would be misleading. **Both WASM backends signal at call time.** No WASI directory-creation call is imported there, and unlike [`file-write-date`](file-write-date.md) this operation has no "cannot be determined" answer in its contract: either the directory exists afterwards or it does not, so answering anything but an error would be a lie. ```console (ensure-directories-exist "logs/2026/app.log") (with-open-file (out "logs/2026/app.log" :direction :output) (write-line "started" out)) ``` --- # FILE: references/reference/functions/environment-enquiry.md # lisp-implementation-type lisp-implementation-version software-type software-version machine-type machine-version machine-instance short-site-name long-site-name `(lisp-implementation-type)` -- `(lisp-implementation-version)` -- `(software-type)` -- `(software-version)` -- `(machine-type)` -- `(machine-version)` -- `(machine-instance)` -- `(short-site-name)` -- `(long-site-name)` The environment enquiry functions. Each answers a string or `nil`, and every answer here is a constant: nothing is read from the host, so a User-Agent or a banner a library composes out of them is the same string on every backend except for `machine-type`. | function | answer | | --- | --- | | `lisp-implementation-type` | `"rontolisp"` | | `lisp-implementation-version` | the project version -- the same string `rontolisp --version` and `(getf (rontolisp:version) :version)` report | | `software-type` | `"Unix"` -- the same claim `uiop:os-unix-p` and `uiop:operating-system` make: every backend presents the POSIX-shaped file model | | `software-version` | `nil` | | `machine-type` | the ABI the running artifact targets: `"JVM"` on the interpreter and the JVM backend, `"WASM32"` on both WASM backends | | `machine-version` | `nil` | | `machine-instance` | `nil` | | `short-site-name` | `nil` | | `long-site-name` | `nil` | `machine-type` deliberately names the ABI rather than the host processor: a class file and a wasm module are both CPU-independent, which is why [`uiop:architecture`](../uiop/os.md) answers `:jvm` / `:wasm32` the same way. Everything rontolisp cannot know answers `nil`, which is what Common Lisp prescribes when no appropriate and relevant result can be supplied -- a fabricated host name or version would be an answer rather than the absence of one. ```lisp (list (lisp-implementation-type) (software-type) (machine-type) (machine-version)) ; => ("rontolisp" "Unix" "JVM" NIL) ``` The version is the build's own, so it is shown rather than asserted: ```lisp (format nil "dexador/1.0 (~A ~A); ~A" (lisp-implementation-type) (lisp-implementation-version) (software-type)) ``` ## Backend support All four backends -- one definition in rontolisp source, spliced into the program when it is referenced. Only `machine-type` differs between backends. --- # FILE: references/reference/functions/eq.md # eq `(eq x y)` Tests object identity, returning `t` or `nil`. Symbols and small integers with the same value are the same object and so compare `eq`, but floats and ratios are distinct boxed objects and are never `eq` even when numerically equal; cons cells compare by reference. Use `eql` or `equal` to compare numbers or structure by value. Works in all three backends. ```lisp (eq 'foo 'foo) ; => T ``` ```lisp (eq 1.5 1.5) ; => NIL ``` --- # FILE: references/reference/functions/eql.md # eql `(eql x y)` Like `eq`, but additionally returns `t` for two numbers of the same type and value -- so equal floats and equal ratios compare `eql` even though they are not `eq`. Numbers of different types are not `eql` (`(eql 3 3.0)` is `nil`). It does not descend into cons cells; use `equal` for structural comparison. Works in all three backends. ```lisp (eql 1.5 1.5) ; => T ``` ```lisp (eql 3 3.0) ; => NIL ``` --- # FILE: references/reference/functions/equal.md # equal `(equal x y)` Structural equality: cons cells are compared recursively by their car and cdr, and strings are compared character by character; for everything else it behaves like `eql`. Returns `t` or `nil`. Works in all three backends. ```lisp (equal '(1 2 (3)) '(1 2 (3))) ; => T ``` ```lisp (equal "abc" "abc") ; => T ``` --- # FILE: references/reference/functions/equalp.md # equalp `(equalp x y)` Like [`equal`](equal.md), but strings and characters are compared case-insensitively and numbers by numeric value (`=`). Cons cells are compared recursively. Arrays are compared element by element with `equalp` when their dimensions match. Lite: hash tables and structures fall back to `eql`. ```lisp (equalp #(1 "A") #(1 "a")) ; => T ``` --- # FILE: references/reference/functions/eval.md # eval `(eval form)` Evaluates a form -- typically a list produced by `read`, `read-from-string`, or quoting -- in the global environment and returns its result. Works in all three backends: the interpreter evaluates directly, while the JVM and WASM compilers emit a small runtime tree-walking interpreter that shares the compiled value representation. Top-level globals defined by the compiled program are mirrored into the eval environment, so an eval'd expression can see them. ```lisp (eval '(+ 1 2)) ; => 3 ``` Quoting the form keeps it unevaluated until `eval` runs it; `(eval (read-from-string "(* 6 7)"))` returns `42`. --- # FILE: references/reference/functions/evenp.md # evenp `(evenp integer)` Returns `t` if `integer` is even, else `nil`. The argument must be an integer. ```lisp (evenp 4) ; => T ``` --- # FILE: references/reference/functions/every.md # every `(every predicate &rest sequences)` Applies `predicate` to one element of each sequence at a time and returns `t` if every call is non-nil, or `nil` as soon as one fails (testing stops at the first failure). Each sequence may be a list or a string (whose elements are characters). With more than one sequence the predicate receives one argument per sequence and the walk stops as soon as the shortest one runs out. An empty sequence yields `t`. ```lisp (every #'evenp '(2 4 6)) ; => T ``` ```lisp (every #'digit-char-p "123") ; => T ``` ```lisp (every #'< '(1 2) '(3 4)) ; => T ``` --- # FILE: references/reference/functions/exp.md # exp `(exp number)` Returns e raised to the power of `number` as a float. The interpreter and JVM backends compute it with `Math.exp`; the WASM backend uses a software approximation, so its result may differ slightly in the least significant digits. ```lisp (exp 0) ; => 1.0 ``` --- # FILE: references/reference/functions/export.md # export `(export symbols &optional package)` Makes `symbols` (a symbol or a list of them) **external** in `package` (the current package by default), so they are visible unqualified through a [`use-package`](use-package.md) and spell with one colon rather than two. Returns `t`. It is the runtime form of the [`defpackage`](../special-forms/defpackage.md) `:export` clause; [`unexport`](unexport.md) is the inverse. Packages are resolved at read/compile time here (see [Packages](../packages.md)), so a literal top-level call is consumed at compile time like `in-package` and takes effect for the forms that follow it — which is what makes it work on every backend. A runtime-computed call (a symbol list built at run time) works on the interpreter only. Exporting changes only *accessibility*, so it may come before or after the definitions it publishes — the everyday shape of a Common Lisp file, which defines its functions and exports them at the end, works: ```lisp (defpackage #:greeter2 (:use #:cl)) (in-package #:greeter2) (export '(hello)) (defun hello () "hi") (in-package #:cl-user) (greeter2:hello) ; => "hi" ``` ```lisp (defpackage #:greeter3 (:use #:cl)) (defun greeter3::hi () "hi") (export '(greeter3::hi) :greeter3) (greeter3:hi) ; => "hi" ``` A reference written *before* the `export` is still an error, as in Common Lisp — the symbol is not external yet at that point. One deviation: a symbol exported *after* it was first named prints with the double colon (`greeter3::hi`), because the qualifier is part of the stored symbol here rather than recomputed at print time. Both spellings name the same symbol. --- # FILE: references/reference/functions/expt.md # expt `(expt base power)` Returns `base` raised to `power`. With integer arguments the result is an exact integer (`(expt 2 10)` is `1024`), and a negative integer power gives the reciprocal (`(expt 2 -1)` is `1/2`); if either argument is a float -- or the power is a ratio -- the result is a float, and a fractional power (`(expt 2 0.5)`, `(expt 10000.0 0.75)`) works on every backend. The dispatch is on the run-time values, so a power computed at run time behaves like a literal one. On the WASM backends a fractional power is `exp(y * log(x))` over the software `exp`/`log` ([Math Function Backends](../../guides/math-backends.md)), so its low-order digits differ from `Math.pow`'s. Works in all three backends. ```lisp (expt 2 10) ; => 1024 ``` ```lisp (expt 2.0 3) ; => 8.0 ``` ```lisp (expt 4 1/2) ; => 2.0 ``` --- # FILE: references/reference/functions/fboundp.md # fboundp `(fboundp symbol)` Returns `t` when `symbol` names something callable or expandable: a function (built-in or `defun`), a macro (built-in or `defmacro`), a special form, or a `car`/`cdr` composition like `cadr`. This matches Common Lisp, where `fboundp` is true of macros and special operators too. On the compiled backends a **literal** quoted argument is decided at compile time with full knowledge (macros and special forms included); a computed argument is checked at runtime against the function registries, which only know real functions — so `(fboundp (intern "cond"))` is nil in compiled code but `t` in the interpreter, and `defmacro` macros are likewise compile-time-only there. A name retired by [`fmakunbound`](fmakunbound.md) answers `nil` again, at a literal call site too. ```lisp (fboundp 'car) ; => T ``` ```lisp (fboundp 'cond) ; => T ``` ```lisp (defun greet (n) n) (fboundp 'greet) ; => T ``` ```lisp (fboundp 'no-such-fn) ; => NIL ``` --- # FILE: references/reference/functions/fdefinition.md # fdefinition `(fdefinition symbol)` The function value of a symbol, like [`symbol-function`](symbol-function.md) (setf-function names are not supported). A quoted symbol literal (`(fdefinition 'car)`) resolves at compile time in the compilers; a runtime-computed symbol resolves late through the compiled name registry when the result is called, with the same deviations as [`symbol-function`](symbol-function.md). ```lisp (funcall (fdefinition 'car) '(1 2 3)) ; => 1 ``` `fdefinition` is the same `setf` place as [`symbol-function`](symbol-function.md): `(setf (fdefinition 'name) fn)` installs `fn` as the symbol's global function definition. --- # FILE: references/reference/functions/file-length.md # file-length `(file-length stream)` The byte length of the file a **file** stream is open on, or `nil` when it cannot be determined. Every other stream answers `nil`: a string stream, a socket, one of the standard streams, and a handle that has already been closed. An output stream is flushed first, so the answer counts what has been written rather than what happens to have reached the disk. **Both WASM backends always answer `nil`**, file streams included: no WASI `filestat` call is imported there. `nil` is exactly Common Lisp's answer for "the length cannot be determined", so a portable caller takes its unknown-length fallback rather than failing. The interpreter and the JVM answer for real. ```lisp (with-input-from-string (s "abc") (file-length s)) ; => NIL ``` ```console (with-open-file (in "data.txt") (print (file-length in))) ``` --- # FILE: references/reference/functions/file-position.md # file-position `(file-position stream [position])` Lite: always returns nil — streams do not support repositioning, so portable callers (which guard this with `ignore-errors`) take their non-seeking fallback path. ```lisp (with-input-from-string (s "abc") (file-position s)) ; => NIL ``` --- # FILE: references/reference/functions/file-write-date.md # file-write-date `(file-write-date pathname)` The file's last-modification time as a [universal time](get-universal-time.md) (seconds since 1900-01-01 GMT), or `nil` when it cannot be determined — which is what a missing or unreadable file answers. Like [`probe-file`](probe-file.md) it never signals, so it can be used as a probe. The path is interpreted exactly as `open` interprets it. **Both WASM backends always answer `nil`**: no WASI `filestat` call is imported there, and `nil` is precisely Common Lisp's answer for "the time cannot be determined", so a portable caller's unknown-time fallback runs rather than the program failing. The interpreter and the JVM answer for real. ```console (let ((stamp (file-write-date "config.lisp"))) (if stamp (print (decode-universal-time stamp)) (print "unknown"))) ``` --- # FILE: references/reference/functions/fill-pointer.md # fill-pointer `(fill-pointer vector)` Returns the fill pointer of a vector created with [`make-array`](make-array.md) `:fill-pointer`. The fill pointer is the vector's effective length: `length` and printing stop at it, while [`aref`](aref.md) can still reach the full backing storage. It is a `setf` place, so `(setf (fill-pointer v) n)` moves it to any position between 0 and the vector's total size. Signals an error when the array has no fill pointer (test with [`array-has-fill-pointer-p`](array-has-fill-pointer-p.md) first). ```lisp (defparameter *v* (make-array 5 :fill-pointer 2 :initial-element 0)) (fill-pointer *v*) ; => 2 (setf (fill-pointer *v*) 4) ; => 4 (length *v*) ; => 4 ``` --- # FILE: references/reference/functions/fill.md # fill `(fill sequence item &key start end)` Stores `item` into every element of `sequence` between `:start` (default 0) and `:end` (default the length) and returns the sequence. Destructive, as in CL: a vector -- a general one, a packed `(unsigned-byte 8|16|32)` or float vector, or a string allocated by [`make-string`](make-string.md) or [`make-array`](make-array.md) `:element-type 'character` -- is written in place, and so is a list. A string LITERAL is supported too, but on the compiled backends it is an immutable value, so `fill` returns a fresh string instead of mutating it (the interpreter mutates in place) -- the same deviation as [`replace`](replace.md). Available on all backends except `--no-gc`. ```lisp (fill (make-array 5 :element-type '(unsigned-byte 8) :initial-element 9) 0 :start 1 :end 4) ; => #(9 0 0 0 9) (fill (list 1 2 3) 7) ; => (7 7 7) ``` --- # FILE: references/reference/functions/find-class.md # find-class `(find-class symbol &optional (errorp t) environment)` Returns the class metaobject named by `symbol` -- a `standard-class` instance whose slots the [`class-name`](class-name.md) / `closer-mop` readers (`class-slots`, `slot-definition-name`, ...) consume. The answer is memoized, so two calls for the same class return the same (`eq`) object -- the same object [`class-of`](class-of.md) answers for an instance of that class. When no class is named `symbol`, an error is signaled unless `errorp` is `nil`, in which case `nil` is returned; `environment` is ignored. The known classes are every `defclass` / `define-condition` / `defstruct` in the program, the built-in condition hierarchy, and the built-in classes (`integer`, `string`, ..., `t`). On the compiled backends the class set is fixed at compile time; classes built from runtime data do not exist. ```lisp (defclass point () ((x :initarg :x))) (list (eq (find-class 'point) (find-class 'point)) (find-class 'no-such-class nil)) ; => (T NIL) ``` `(setf (find-class alias) class)` registers `class` under a second name: after it, `find-class`, `make-instance`, `typep`, `subtypep` and a `handler-case` clause all resolve the alias to the very same class (the metaobject is `eq`). Only this aliasing shape is supported -- the value must be a literal `(find-class 'target)` naming an already defined class -- and only at top level, because the compiled backends build their class table at compile time. ```lisp (defclass shape () ((n :initarg :n :reader shape-n))) (setf (find-class ') (find-class 'shape)) (list (eq (find-class ') (find-class 'shape)) (shape-n (make-instance ' :n 7))) ; => (T 7) ``` --- # FILE: references/reference/functions/find-if-not.md # find-if-not `(find-if-not predicate sequence)` Returns the first element of `sequence` that does **not** satisfy `predicate`, or `nil` if every element satisfies it. The sequence may be a list or a string (whose elements are characters). It returns the element itself. This is the complement of `find-if`. ```lisp (find-if-not #'evenp '(2 4 5 6)) ; => 5 ``` ```lisp (find-if-not #'digit-char-p "12a3") ; => #\a ``` --- # FILE: references/reference/functions/find-if.md # find-if `(find-if predicate sequence)` Returns the first element of `sequence` that satisfies `predicate`, or `nil` if none does. The sequence may be a list or a string (whose elements are characters). It returns the element itself, not its index or tail. Use `find-if-not` for the complementary search. ```lisp (find-if #'evenp '(1 3 6 7)) ; => 6 ``` ```lisp (find-if #'digit-char-p "ab3c") ; => #\3 ``` --- # FILE: references/reference/functions/find-package.md # find-package `(find-package designator)` Lite: rontolisp has no package objects, so the returned "package" is the upcased canonical package name as a keyword, and `nil` for an unknown package. `designator` is a keyword, a string, a symbol, or `nil` (which designates the package named `"NIL"`, so it answers `nil`); package names are case-sensitive, exactly as in Common Lisp. A literal designator is folded at compile time. A computed one is answered from the live registry on the interpreter, and on the compiled backends from a table of the program's packages baked in at compile time — so a package created after compilation is invisible there. ```lisp (list (find-package :cl) (find-package "nope")) ; => (:CL NIL) ``` --- # FILE: references/reference/functions/find-restart.md # find-restart `(find-restart identifier [condition])` Returns the innermost active restart named `identifier` (symbol or keyword) as a first-class restart object, or `nil` when none is active; a restart object passes through unchanged. The object can be passed to [`invoke-restart`](invoke-restart.md) and read with [`restart-name`](restart-name.md). Lite: the optional `condition` argument is accepted and ignored (restarts are not associated with conditions). ```lisp (restart-case (let ((r (find-restart 'retry))) (list (null r) (restart-name r))) (retry () nil)) ; => (NIL RETRY) ``` --- # FILE: references/reference/functions/find-symbol.md # find-symbol `(find-symbol string [package])` Like [`intern`](intern.md) but never creates: returns the symbol when the name is already known to the image, nil otherwise. "Known" means a `cl` symbol (function, macro, or special form), a keyword, or a user definition. With a `package` designator the name is looked up in that package instead of the current one; a package that does not exist provides no symbol, so the answer is `nil` rather than the `package-error` Common Lisp signals — that keeps probes for optional systems (`(find-symbol "TIMESTAMP" :simple-date)`) working the same way on every backend. Deviations from Common Lisp: on the compiled backends (JVM/WASM) only a **literal** string can answer `nil` — the check is folded at compile time against the compile-time view (cl symbols plus the program's own `defun`s), so runtime-defined variables and macros are not visible there (the interpreter checks the live image, including global variables and `defmacro` macros). A computed name is interned instead, so it always yields a symbol, and its status is read off the spelling that lowering builds (`:external` for a qualified one, `:internal` for a bare one) rather than from the image. A second value reports the ANSI accessibility status of the name in that package — `:external`, `:inherited`, `:internal`, or `nil` when the package does not provide it, so the two values are `nil` together: ```lisp (multiple-value-list (find-symbol "CAR" 'common-lisp)) ; => (CAR :EXTERNAL) ``` ```lisp (multiple-value-list (find-symbol "CAR")) ; => (CAR :INHERITED) ``` ```lisp (find-symbol "car") ; => NIL ``` ```lisp (find-symbol "cond") ; => NIL ``` ```lisp (find-symbol "no-such-name") ; => NIL ``` ```lisp (defun greet (n) n) (find-symbol "greet") ; => NIL ``` ```lisp (find-symbol "TIMESTAMP" :simple-date) ; => NIL ``` --- # FILE: references/reference/functions/find.md # find `(find item sequence &key test key)` Returns the first element of `sequence` that matches `item`, or `nil` if no element matches. The comparison is `eql` by default; the optional `:test` keyword takes a function designator to use a different comparison, and the optional `:key` keyword takes a selector function applied to each element before the comparison (the returned element is the original one, not the keyed value). The sequence may be a list or a string; the elements of a string are characters. Unlike `member`, which returns the matching tail, `find` returns the element itself. Use `position` to obtain the index instead. ```lisp (find 2 '(1 2 3)) ; => 2 ``` ```lisp (find #\l "hello") ; => #\l ``` ```lisp (find "b" '("a" "b" "c") :test #'string=) ; => "b" ``` ```lisp (find 4 '((1 2) (3 4)) :key #'cadr) ; => (3 4) ``` --- # FILE: references/reference/functions/finish-output.md # finish-output `(finish-output &optional stream)` The same operation as [`force-output`](force-output.md): flushes the designated output stream and returns nil. Common Lisp distinguishes the two (`finish-output` waits for the sink to accept the data), but every rontolisp write is synchronous once flushed, so both names name one behavior. ```lisp (progn (princ "buffered") (finish-output)) ; => NIL ``` --- # FILE: references/reference/functions/first.md # first `(first list)` Returns the first element of a list. It is an exact synonym for `car`, including the `(first nil)` is `nil` behavior, but reads more naturally when the argument is meant as a list rather than a raw cons cell. ```lisp (first '(10 20 30)) ; => 10 ``` --- # FILE: references/reference/functions/float.md # float `(float number &optional prototype)` Converts `number` to a floating-point value (a double). Integers and ratios are turned into their nearest float; a value that is already a float is returned unchanged. The optional `prototype` selects the float subtype in Common Lisp; rontolisp has a single float representation, so it is evaluated and ignored. ```lisp (float 42) ; => 42.0 ``` ```lisp (float 1/2) ; => 0.5 ``` --- # FILE: references/reference/functions/floatp.md # floatp `(floatp object)` Returns `t` if `object` is a floating-point number, otherwise `nil`. Integers and ratios are not floats, so `(floatp 3)` and `(floatp 1/2)` are both `nil`. Works in all three backends. ```lisp (floatp 3.14) ; => T ``` ```lisp (floatp 3) ; => NIL ``` --- # FILE: references/reference/functions/floor.md # floor `(floor number &optional divisor)` Rounds `number` (or `number/divisor` when a divisor is given) toward negative infinity to an integer. In an ordinary (single-value) context the result is the quotient only; the remainder is the second value, observable through [`multiple-value-bind`](../macros/multiple-value-bind.md) and the other multiple-value consumers. ```lisp (floor 3.7) ; => 3 ``` ```lisp (floor -3.7) ; => -4 ``` ```lisp (multiple-value-bind (q r) (floor 7 2) (list q r)) ; => (3 1) ``` --- # FILE: references/reference/functions/fmakunbound.md # fmakunbound `(fmakunbound symbol)` Makes `symbol` name no function again, and returns the symbol. An unknown name is a no-op. On the interpreter the global function binding (and any `defmacro` macro of the same name) is removed outright, so a later call signals `The function X is undefined`. On the compiled backends the name is retired only for **late-bound** references — [`fboundp`](fboundp.md), `funcall`/`#'name`/`eval` through the symbol — because a call site the compiler already bound directly cannot be undone. Built-in macros and special forms are part of the language, not of the image's function namespace, so they are not affected. ```lisp (defun greet (n) n) (list (fboundp 'greet) (fmakunbound 'greet) (fboundp 'greet)) ; => (T GREET NIL) ``` --- # FILE: references/reference/functions/force-output.md # force-output `(force-output &optional stream)` Flushes the designated output stream's buffered bytes to its underlying sink and returns nil. With no argument (or `nil`/`t`) it flushes standard output. `finish-output` is the same operation here: once flushed, every rontolisp write is synchronous, so there is nothing further to wait for. Socket writes never buffer on any backend, which is why flushing a socket is a no-op that still costs nothing to write. ```lisp (progn (princ "no newline yet") (force-output)) ; => NIL ``` --- # FILE: references/reference/functions/fresh-line.md # fresh-line `(fresh-line)` Writes a newline to standard output only if output is not already at the start of a fresh line, so it never produces a blank line where the cursor is already at column zero. Returns nil. Use it to guarantee subsequent output begins on its own line without risking a double newline. ```lisp (princ "a") (fresh-line) (princ "b") ``` ``` a b ``` --- # FILE: references/reference/functions/funcall.md # funcall `(funcall function &rest args)` Calls `function` with the given arguments and returns its result. The `function` argument may be a function value -- `#'name`, a `lambda`, or the result of `symbol-function` -- or a quoted symbol that names a function (`(funcall 'car x)`); the interpreter resolves the symbol at runtime, while the compilers rewrite a literal `(quote name)` in function position into `(function name)`. ```lisp (funcall #'+ 3 4) ; => 7 ``` --- # FILE: references/reference/functions/function-lambda-expression.md # function-lambda-expression `(function-lambda-expression function)` Lite stub: returns `(values nil t nil)` -- no source expression is recorded for function objects. ```lisp (function-lambda-expression #'car) ; => NIL ``` --- # FILE: references/reference/functions/functionp.md # functionp `(functionp object)` Returns `t` if `object` is a function value — the result of `(lambda ...)`, `#'name` or `symbol-function` — and `nil` for any other object. In Lisp-2 a bare symbol is never a function, so `(functionp 'car)` is `nil` while `(functionp #'car)` is `t`. ```lisp (functionp #'car) ; => T ``` ```lisp (functionp (lambda (x) x)) ; => T ``` ```lisp (functionp 'car) ; => NIL ``` --- # FILE: references/reference/functions/gcd.md # gcd `(gcd &rest integers)` Returns the greatest common divisor of its integer arguments. It is variadic; with no arguments it returns `0`, and with one argument it returns that integer's absolute value. The result is always a non-negative integer. ```lisp (gcd 12 18) ; => 6 ``` ```lisp (gcd 24 36 60) ; => 12 ``` --- # FILE: references/reference/functions/ge.md # >= `(>= &rest numbers)` Returns `t` if its arguments are in non-increasing order (each greater than or equal to the next), else `nil`. It is variadic and compares by numeric value across integer, ratio and float types. With a single argument it returns `t`. ```lisp (>= 2 1 1) ; => T ``` --- # FILE: references/reference/functions/gensym.md # gensym `(gensym [prefix])` Returns a fresh symbol named `#:`, where `prefix` defaults to `g` and `n` is a program-wide counter starting at 1. rontolisp has no uninterned symbols: the result is an ordinary symbol whose uniqueness rests on the `#:` prefix (which no user-written name normally carries) and the ever-increasing counter. Its main use is generating capture-safe temporaries inside [`defmacro`](../special-forms/defmacro.md) bodies, replacing the older `__`-prefixed naming convention. Deviations from Common Lisp: the prefix must be a **literal** string on the compilation path (JVM/WASM), so the symbol text is known at compile time — a computed prefix is a compile error (the interpreter accepts any string). There is no `*gensym-counter*` variable, and because the symbol is interned like any other, `read`ing the same printed name twice yields `eq` symbols. ```lisp (list (gensym) (gensym)) ; => (#:g1 #:g2) ``` ```lisp (gensym "tmp") ; => #:tmp3 ``` ```lisp (eq (gensym) (gensym)) ; => NIL ``` A macro temporary generated with `gensym` cannot collide with the caller's variables: ```lisp (defmacro swap! (a b) (let ((tmp (gensym))) `(let ((,tmp ,a)) (setq ,a ,b) (setq ,b ,tmp)))) (setq tmp 1) (setq other 2) (swap! tmp other) (list tmp other) ; => (2 1) ``` --- # FILE: references/reference/functions/get-internal-real-time.md # get-internal-real-time `(get-internal-real-time)` Returns the elapsed real (wall-clock) time in milliseconds, suitable for timing operations by taking the difference of two readings. Every backend returns an integer. The absolute value is only meaningful relative to another reading, not as a calendar time. ```lisp (get-internal-real-time) ``` A typical use is `(- (get-internal-real-time) start)` to measure how many milliseconds a computation took. Because it reflects the current clock, the value is non-deterministic — except on a `--no-wasi` module, whose clock only moves when its host writes it, so two readings inside one call are equal (see the [clock and randomness guide](../../guides/clock-and-random.md#setting-the-clock----ronto-set-time)). --- # FILE: references/reference/functions/get-internal-run-time.md # get-internal-run-time `(get-internal-run-time)` Returns the consumed run (CPU) time in milliseconds, suitable for measuring processing time by differencing two readings. Every backend returns an integer. As with `get-internal-real-time`, only the difference between two readings is meaningful, not the absolute value. ```lisp (get-internal-run-time) ``` Bracket a computation with two calls and subtract to obtain the run time it consumed. The value depends on prior execution, so it is non-deterministic. A `--no-wasi` module has only the one clock its host set, so this reads the same value as `get-internal-real-time` there and a bracket measures zero (see the [clock and randomness guide](../../guides/clock-and-random.md#setting-the-clock----ronto-set-time)). --- # FILE: references/reference/functions/get-output-stream-string.md # get-output-stream-string `(get-output-stream-string stream)` Returns everything written to a `make-string-output-stream` stream so far, and **clears** it: the next call answers only what was written after this one. That is Common Lisp's contract, and it is what lets one accumulator stream be reused for a sequence of tokens. ```lisp (let ((s (make-string-output-stream))) (write-string "ab" s) (let ((first (get-output-stream-string s))) (write-string "cd" s) (list first (get-output-stream-string s) (get-output-stream-string s)))) ; => ("ab" "cd" "") ``` --- # FILE: references/reference/functions/get-setf-expansion.md # get-setf-expansion `(get-setf-expansion place &optional environment)` Returns the five setf-expansion values for `place`: a list of temporary variables, the list of value forms they bind, a list holding one store variable, the writer form, and the reader form. Lite: a variable place expands to a `setq` writer with no temporaries; an accessor form `(f args...)` binds one temporary per argument and writes through `setf`. The `environment` argument is accepted and ignored (a macro's `&environment` parameter is bound to nil). Consume the values with `multiple-value-bind`, as in the portable `incf`-style macro idiom. ```lisp (multiple-value-bind (vars vals stores writer reader) (get-setf-expansion 'x) (list vars vals (length stores) reader)) ; => (NIL NIL 1 X) ``` --- # FILE: references/reference/functions/get-universal-time.md # get-universal-time `(get-universal-time)` Returns the current time as the number of seconds since the Common Lisp epoch of 1900-01-01 GMT (Unix time plus 2208988800). Every backend returns an integer; the WASM backend reads the clock from the host (the real host clock in Preview 1, `wasi:clocks@0.3.0` in `--component` mode). A `--no-wasi` module imports no clock and reports the time its host handed over — see the [clock and randomness guide](../../guides/clock-and-random.md#setting-the-clock----ronto-set-time). ```lisp (get-universal-time) ``` Each call yields the wall-clock time at the moment of the call, so the result is non-deterministic; subtracting two readings gives an elapsed number of seconds. --- # FILE: references/reference/functions/get.md # get `(get symbol indicator &optional default)` Reads the symbol's property list entry for `indicator`, or `default` when absent; `(setf (get symbol indicator) value)` writes it. Symbols have no identity cells to hang plists on (they compare by name), so the store is one program-global name-keyed table. ```lisp (setf (get 'my-sym 'color) :red) (get 'my-sym 'color) ; => :RED ``` ```lisp (get 'my-sym 'absent :fallback) ; => :FALLBACK ``` --- # FILE: references/reference/functions/getf.md # getf `(getf plist indicator &optional default)` Returns the value following `indicator` in a property list (a flat list of alternating indicators and values). If the indicator is absent, `default` is returned -- `nil` when it is omitted. `getf` is a function, so `default` is evaluated whether or not the indicator is found. An indicator that IS present but whose value is `nil` yields `nil`, not the default. It is the partner of `remf`. `(setf (getf ...) value)` is not supported; use `remf` to delete a property. ```lisp (getf '(:a 1 :b 2) :b) ; => 2 ``` ```lisp (getf '(:a 1) :on-delete :restrict) ; => :RESTRICT ``` --- # FILE: references/reference/functions/gethash.md # gethash `(gethash key table &optional default)` Looks up `key` in `table` and returns the associated value, or `default` (nil when omitted) if the key is absent. Keys are compared structurally as if by `equal`, so an equal list, string, or number key matches. To store a value, use `gethash` as a `setf` place: `(setf (gethash key table) value)`, which also cooperates with `incf`/`decf`/`push`. Under [`multiple-value-bind`](../macros/multiple-value-bind.md) (and the other multiple-value consumers) a `gethash` call also supplies a second present-p value distinguishing a stored nil from a missing key. ```lisp (let ((h (make-hash-table))) (setf (gethash 'a h) 1) (gethash 'a h)) ; => 1 ``` ```lisp (let ((h (make-hash-table))) (setf (gethash 'a h) nil) (multiple-value-bind (v present-p) (gethash 'a h) (list v present-p))) ; => (NIL T) ``` --- # FILE: references/reference/functions/graphic-char-p.md # graphic-char-p standard-char-p `(graphic-char-p character)` -- `(standard-char-p character)` Character predicates. `graphic-char-p` is true for a printing character -- code points 32 to 126 and everything from 160 up -- so `#\Space` counts and `#\Newline` does not. `standard-char-p` is true for the 96 standard characters: the printing ASCII range plus `#\Newline`, which is the one character the two predicates disagree on. ```lisp (list (graphic-char-p #\a) (graphic-char-p #\Space) (graphic-char-p #\Newline) (standard-char-p #\Newline) (standard-char-p #\Tab)) ; => (T T NIL T NIL) ``` --- # FILE: references/reference/functions/gt.md # > `(> &rest numbers)` Returns `t` if its arguments are in strictly decreasing order, else `nil`. It is variadic; each adjacent pair is compared by numeric value, mixing integers, ratios and floats. With a single argument it returns `t`. ```lisp (> 3 2 1) ; => T ``` --- # FILE: references/reference/functions/hash-table-count.md # hash-table-count `(hash-table-count table)` Returns the number of entries currently stored in `table` as an integer (0 for an empty table). The count drops as entries are removed with `remhash` or `clrhash`. ```lisp (let ((h (make-hash-table))) (setf (gethash 'a h) 1) (setf (gethash 'b h) 2) (hash-table-count h)) ; => 2 ``` --- # FILE: references/reference/functions/hash-table-p.md # hash-table-p `(hash-table-p object)` Returns `t` if `object` is a hash table (one made by `make-hash-table`), and `nil` for anything else. A hash table is not a cons, so `consp` returns nil on it. ```lisp (hash-table-p (make-hash-table)) ; => T ``` --- # FILE: references/reference/functions/hash-table-rehash-size.md # hash-table-rehash-size `(hash-table-rehash-size hash-table)` Returns the standard default growth factor `1.5`. rontolisp tables do not expose a growth knob (the host map grows on its own), so the value is a constant reported for the benefit of portable code that reads it before rebuilding a table. ```lisp (hash-table-rehash-size (make-hash-table)) ; => 1.5 ``` --- # FILE: references/reference/functions/hash-table-rehash-threshold.md # hash-table-rehash-threshold `(hash-table-rehash-threshold hash-table)` Returns the standard default threshold `1.0`, the companion constant of [`hash-table-rehash-size`](hash-table-rehash-size.md). ```lisp (hash-table-rehash-threshold (make-hash-table)) ; => 1.0 ``` --- # FILE: references/reference/functions/hash-table-size.md # hash-table-size `(hash-table-size hash-table)` Returns the table's size. A rontolisp table has no capacity of its own -- growth belongs to the host map -- so the size **is** the entry count, the same value [`hash-table-count`](hash-table-count.md) returns. It exists so a portable table-copying utility can read the triple it expects. ```lisp (let ((h (make-hash-table))) (setf (gethash 'a h) 1) (hash-table-size h)) ; => 1 ``` --- # FILE: references/reference/functions/hash-table-test.md # hash-table-test `(hash-table-test hash-table)` Returns the test the table's lookups implement -- always the symbol `equal`. rontolisp keys every table structurally on every backend, whatever `:test` was passed to [`make-hash-table`](make-hash-table.md), so reporting the requested test would describe behavior that does not exist here. ```lisp (hash-table-test (make-hash-table)) ; => EQUAL ``` --- # FILE: references/reference/functions/identity.md # identity `(identity object)` Returns `object` unchanged. It is useful as a default or placeholder function argument to higher-order operators such as `mapcar`, `find-if`, or `sort`, where a no-op transform or key is wanted. ```lisp (identity 42) ; => 42 ``` --- # FILE: references/reference/functions/import.md # import `(import symbols &optional package)` Makes `symbols` (a symbol or a list of them) accessible **unqualified** in `package` (the current package by default): a later bare `name` resolves to the imported symbol rather than to a fresh symbol of the importing package. Returns `t`. It is the runtime form of the [`defpackage`](../special-forms/defpackage.md) `:import-from` clause. The argument keeps its package qualifier — that is what says where the symbol comes from — and an unqualified symbol is already the current package's own, so importing it does nothing. An unknown package signals (`No such package: NOSUCH`). Packages are resolved at read/compile time here (see [Packages](../packages.md)), so a literal top-level call is consumed at compile time like `in-package` and takes effect for the forms that follow it — which is what makes it work on every backend. A runtime-computed call (a symbol built at run time) works on the interpreter only. ```lisp (defpackage #:importer (:use #:cl) (:export #:shout)) (in-package #:importer) (defun shout () "HI") (in-package #:cl-user) (import 'importer:shout) (shout) ; => "HI" ``` --- # FILE: references/reference/functions/input-stream-p.md # input-stream-p `(input-stream-p stream)` Lite: `t` for any stream handle (every rontolisp stream answers both directions) and for the standard-output designator `t`, nil otherwise. A [Gray stream](../../guides/gray-streams.md) instance is exact instead: it answers `t` only when its class descends from `rontolisp:fundamental-input-stream`. ```lisp (with-input-from-string (s "x") (input-stream-p s)) ; => T ``` --- # FILE: references/reference/functions/integer-length.md # integer-length `(integer-length integer)` Number of bits needed to represent the two's-complement magnitude of `integer`, excluding the sign. For a non-negative `integer` this is the position of the highest set bit plus one; for a negative `integer` it is the length of the ones' complement, so `(integer-length -1)` is `0` and `(integer-length -5)` is `3`. `integer` may be any magnitude on every backend. ```lisp (integer-length 255) ; => 8 ``` ```lisp (integer-length -5) ; => 3 ``` --- # FILE: references/reference/functions/integerp.md # integerp `(integerp object)` Returns `t` if `object` is an integer, otherwise `nil`. Floats and ratios are not integers, so `(integerp 3.0)` and `(integerp 1/2)` are both `nil`. Works in all three backends. ```lisp (integerp 42) ; => T ``` ```lisp (integerp 3.0) ; => NIL ``` --- # FILE: references/reference/functions/intern.md # intern `(intern string)` Returns the symbol named by `string` (no case folding). rontolisp symbols compare by name — there is no separate intern table — so the result is `eq` to any symbol with the same name, including quoted literals. On the interpreter the name is interned into the **current package** (Common Lisp's `*package*` semantics): an accessible symbol keeps its home spelling, and an unknown name becomes a symbol of the package selected by `in-package` — which is what lets a macro-time `(intern (concatenate ...))` name the same function as a literal `defun` in that file. A string that already carries a package qualifier (`"LIB:WIDGET"`) names that symbol rather than a fresh symbol of the whole string, so a canonical spelling produced at run time — the type name [`type-of`](type-of.md) reads off a class, say — round-trips. `(intern name :keyword)` builds a keyword, and `(intern name package)` accepts any package designator — a keyword, a string, or a package value held in a variable; a package that does not exist signals an error. Like `find-symbol`, a second value reports the accessibility status of the name (`nil` for a name the package does not already provide). Deviation from Common Lisp: on the compiled backends a package-qualified `intern` always yields the single-colon external spelling, so an unexported symbol interned this way is not `eq` to its double-colon literal (calling it as a function still works). ```lisp (intern "hello") ; => hello ``` ```lisp (eq (intern "foo") 'foo) ; => NIL ``` ```lisp (defvar *level* 7) (symbol-value (intern "*LEVEL*")) ; => 7 ``` ```lisp (defpackage :evt (:use :cl) (:export :fire)) (in-package :evt) (defun fire (x) (list :fired x)) (in-package :cl-user) (funcall (intern "FIRE" :evt) 7) ; => (:FIRED 7) ``` --- # FILE: references/reference/functions/intersection.md # intersection `(intersection list1 list2 &key test key)` Returns a list of the elements that appear in **both** `list1` and `list2`, treating them as sets. The comparison is `eql` by default; the optional `:test` keyword takes a function designator to use a different comparison, and the optional `:key` keyword takes a selector function applied to both compared elements. The order of elements in the result is unspecified. ```lisp (intersection '(1 2 3) '(2 3 4)) ; => (3 2) ``` ```lisp (intersection '("a" "b") '("b" "c") :test #'string=) ; => ("b") ``` --- # FILE: references/reference/functions/invoke-debugger.md # invoke-debugger `(invoke-debugger condition)` Signals `condition` and never returns. No backend has an interactive debugger to enter, so what "entering the debugger and being told to abort" amounts to here is the condition reaching whatever handler is established outside the caller -- and the standard-error report plus a non-zero exit when none is. ```lisp (handler-case (invoke-debugger (make-condition 'simple-error :format-control "boom")) (error (e) (princ-to-string e))) ; => "boom" ``` ## Backend support Works on all four backends: one definition in rontolisp source over `error`. --- # FILE: references/reference/functions/invoke-restart.md # invoke-restart `(invoke-restart restart arg...)` Invokes a restart with the given arguments. `restart` is a restart **name** (symbol or keyword — the innermost active restart with that name wins) or a restart **object** from [`find-restart`](find-restart.md)/[`compute-restarts`](compute-restarts.md). For a [`restart-case`](../macros/restart-case.md) restart, control transfers non-locally to the establishing frame and the clause body runs with `arg...`; for a [`restart-bind`](../macros/restart-bind.md) restart, the bound function is called at the invocation point and its value returned. Signals an error when no matching restart is active. ```lisp (handler-bind ((error (lambda (c) (invoke-restart :use-value 7)))) (restart-case (error "no value") (:use-value (v) (list :used v)))) ; => (:USED 7) ``` --- # FILE: references/reference/functions/isqrt.md # isqrt `(isqrt natural)` Returns the integer square root of a non-negative integer: the largest integer whose square does not exceed the argument (the floor of the real square root). Unlike `sqrt`, the result is an integer rather than a float. Works in all three backends. ```lisp (isqrt 17) ; => 4 ``` --- # FILE: references/reference/functions/java-call.md # java:call `(java:call object "methodName" args...)` Invokes an instance method on a `java` object by reflection, choosing the overload whose parameters best match the arguments, and returns the marshalled result (a `void` method returns `nil`). Part of the JVM-only `java` interop package — available on the interpreter and in JVM-compiled classes, not on the WASM backend. See the [Java interop guide](../../guides/java-interop.md). ```lisp (let ((lst (java:new "java.util.ArrayList"))) (java:call lst "add" 7) (java:call lst "size")) ; => 1 ``` A `java.util.ArrayList` is created, one element is added, and `size` returns the element count. --- # FILE: references/reference/functions/java-field.md # java:field `(java:field class-or-object "fieldName")` Reads a field by reflection: with a class-name string it reads a static field (such as a constant), and with a `java` object it reads that instance's field. Returns the marshalled value. Part of the JVM-only `java` interop package — available on the interpreter and in JVM-compiled classes, not on the WASM backend. See the [Java interop guide](../../guides/java-interop.md). ```lisp (java:field "java.lang.Integer" "MAX_VALUE") ; => 2147483647 ``` The static constant `Integer.MAX_VALUE` is read and marshalled to a rontolisp integer. --- # FILE: references/reference/functions/java-new.md # java:new `(java:new "fully.qualified.ClassName" args...)` Constructs a host (Java) object by reflection, choosing the constructor whose parameters best match the arguments, and returns an opaque `java` object that prints as `#>`. Part of the JVM-only `java` interop package — available on the interpreter and in JVM-compiled classes, not on the WASM backend, and it needs the class to be present and reflectable at runtime. See the [Java interop guide](../../guides/java-interop.md). ```lisp (java:call (java:new "java.lang.StringBuilder" "ab") "length") ; => 2 ``` A `java.lang.StringBuilder` is constructed from the string `"ab"`, then its `length` method returns `2`. --- # FILE: references/reference/functions/java-proxy.md # java:proxy `(java:proxy "fully.qualified.Interface" callable)` Creates a host instance of the given interface backed by a rontolisp callable. Every interface method is dispatched to the callable as `(callable "method-name" arg...)` — so the callable's **first argument is the name of the invoked method** (a string) and the remaining arguments are the method's arguments. The callable's return value is marshalled back to the method's return type (a `void` method ignores it). This is how a rontolisp lambda becomes a Java listener or comparator. For a single-method (SAM) interface the method name is always the same, so it is conventionally ignored (the `method` parameter in the examples below). Part of the JVM-only `java` interop package — available on the interpreter and in JVM-compiled classes, not on the WASM backend. See the [Java interop guide](../../guides/java-interop.md). ```lisp (java:call (java:proxy "java.util.function.Supplier" (lambda (method) 42)) "get") ; => 42 ``` A `java.util.function.Supplier` is implemented by the lambda; calling its `get` method runs the lambda and returns `42`. ## Functional interfaces Any interface works, including the `java.util.function` family. The lambda's first parameter receives the method name; the rest receive the method arguments, so match the arity to the interface's single abstract method (SAM): | Interface | SAM | Lambda shape | |-----------|-----|--------------| | `Supplier` | `get()` | `(lambda (method) ...)` | | `Function` | `apply(x)` | `(lambda (method x) ...)` | | `Consumer` | `accept(x)` | `(lambda (method x) ...)` | | `Predicate` | `test(x)` | `(lambda (method x) ...)` | | `BiFunction` | `apply(a, b)` | `(lambda (method a b) ...)` | | `BinaryOperator` | `apply(a, b)` | `(lambda (method a b) ...)` | | `Comparator` | `compare(a, b)` | `(lambda (method a b) ...)` | ```lisp ;; Function: apply(x) -> x + 1 (java:call (java:proxy "java.util.function.Function" (lambda (method x) (+ x 1))) "apply" 41) ; => 42 ``` ```lisp ;; BiFunction: apply(a, b) -> a * b (java:call (java:proxy "java.util.function.BiFunction" (lambda (method a b) (* a b))) "apply" 6 7) ; => 42 ``` ```lisp ;; Predicate: test(x) -> even? (java:call (java:proxy "java.util.function.Predicate" (lambda (method x) (evenp x))) "test" 4) ; => T ``` The proxy also works when the JDK itself invokes the SAM method. For example `HashMap.merge` calls the supplied `BiFunction` to combine the old and new value: ```lisp (let ((m (java:new "java.util.HashMap")) (mul (java:proxy "java.util.function.BiFunction" (lambda (method a b) (* a b))))) (java:call m "put" "x" 10) (java:call m "merge" "x" 5 mul) (java:call m "get" "x")) ; => 50 ``` ## Default methods A dynamic proxy routes **every** method call to the callable, including default methods such as `BiFunction.andThen` or `Predicate.and`. Calling one dispatches to the lambda as `(callable "andThen" ...)` rather than running the interface's built-in default implementation, so combinators like `(f.andThen g)` are not available — call the single abstract method (`apply`/`test`/`accept`/`get`/ `compare`) instead. --- # FILE: references/reference/functions/java-static.md # java:static `(java:static "fully.qualified.ClassName" "methodName" args...)` Invokes a static method by reflection, choosing the overload whose parameters best match the arguments, and returns the marshalled result. Part of the JVM-only `java` interop package — available on the interpreter and in JVM-compiled classes, not on the WASM backend. See the [Java interop guide](../../guides/java-interop.md). ```lisp (java:static "java.lang.Math" "max" 3 7) ; => 7 ``` `Math.max` is overloaded for `int`/`long`/`float`/`double`; the integer arguments select the `int` overload, so the result is the integer `7`. --- # FILE: references/reference/functions/keywordp.md # keywordp `(keywordp object)` Returns `t` if `object` is a keyword -- a symbol written with a leading colon, such as `:foo` -- otherwise `nil`. An ordinary symbol is not a keyword, so `(keywordp 'foo)` is `nil`. Works in all three backends. ```lisp (keywordp :foo) ; => T ``` ```lisp (keywordp 'foo) ; => NIL ``` --- # FILE: references/reference/functions/last.md # last `(last list &optional n)` Returns the last cons cell of `list` -- the one-element list containing the final element. For an empty list it returns `nil`. With the optional count `n` it returns the last `n` conses instead: an `n` larger than the list yields the whole list, and `n` of 0 yields the terminating atom (`nil` for a proper list, the dotted tail otherwise). ```lisp (last '(1 2 3)) ; => (3) (last '(1 2 3) 2) ; => (2 3) (last '(1 2 3) 0) ; => NIL (last '(1 2 3) 5) ; => (1 2 3) ``` --- # FILE: references/reference/functions/lcm.md # lcm `(lcm &rest integers)` Returns the least common multiple of its integer arguments. It is variadic; with no arguments it returns `1`, and the result is `0` if any argument is `0`. The result is always a non-negative integer. ```lisp (lcm 4 6) ; => 12 ``` ```lisp (lcm 2 3 4) ; => 12 ``` --- # FILE: references/reference/functions/ldb.md # ldb `(ldb bytespec integer)` Load byte: extracts the field of `integer` named by the byte specifier `bytespec` (see [`byte`](byte.md)) and returns it right-justified, so the field's least significant bit becomes bit 0. `integer` may be any magnitude on every backend. ```lisp (ldb (byte 4 4) 255) ; => 15 ``` --- # FILE: references/reference/functions/le.md # <= `(<= &rest numbers)` Returns `t` if its arguments are in non-decreasing order (each less than or equal to the next), else `nil`. It is variadic and compares by numeric value across integer, ratio and float types. With a single argument it returns `t`. ```lisp (<= 1 1 2) ; => T ``` --- # FILE: references/reference/functions/length.md # length `(length sequence)` Returns the number of elements in a sequence. It works on lists, strings, and rank-1 vectors; `(length nil)` is `0`. A rank-2 array is not a sequence, so calling `length` on one signals an error. ```lisp (length '(a b c d)) ; => 4 ``` ```lisp (length "hello") ; => 5 ``` --- # FILE: references/reference/functions/linalg-abs.md # linalg:abs `(linalg:abs array)` Returns a fresh array of the same shape with the absolute value of every element (numpy's `np.abs`) -- equivalent to `(linalg:emap #'abs array)`, but as a named function it is accelerated under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg). ```lisp (linalg:abs #(-3 2 -1)) ; => #d(3.0 2.0 1.0) ``` --- # FILE: references/reference/functions/linalg-acos.md # linalg:acos `(linalg:acos array)` Returns a fresh array of the same shape with the arc cosine applied to every element (numpy's `np.arccos`) -- equivalent to `(linalg:emap #'acos array)`, but as a named function it is accelerated under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg). Like [`acos`](asin-acos-atan.md) itself, the WASM backends compute it with a software approximation whose low-order digits can differ from the interpreter's and the JVM's. ```lisp (linalg:acos (linalg:ones 3)) ; => #d(0.0 0.0 0.0) ``` --- # FILE: references/reference/functions/linalg-add.md # linalg:add `(linalg:add a b)` Adds `a` and `b` elementwise, returning a fresh array. The operands broadcast by numpy's rules: a scalar is broadcast over the other operand's shape, and two arrays of different shapes align their trailing axes -- each aligned pair of extents must be equal or contain a 1 (a missing leading axis counts as 1), and the axis of extent 1 stretches over the other operand's extent; anything else signals a shape-mismatch error. The result keeps the first array operand's element type. The other elementwise operators are [`linalg:sub`](linalg-sub.md), [`linalg:mul`](linalg-mul.md) and [`linalg:div`](linalg-div.md). ```lisp (linalg:add #(1 2 3) 10) ; => #d(11.0 12.0 13.0) (linalg:add #(1 2) #(3 4)) ; => #d(4.0 6.0) (linalg:add #2A((1 2) (3 4)) #2A((100) (200))) ; => #d((101.0 102.0) (203.0 204.0)) ``` --- # FILE: references/reference/functions/linalg-amax.md # linalg:amax `(linalg:amax array &key axis keepdims)` Returns the largest element of a vector or matrix. An empty array signals an error. For the *index* of the largest element, use [`linalg:argmax`](linalg-argmax.md); the counterpart for the smallest element is [`linalg:amin`](linalg-amin.md). With an integer `:axis` (negative counts from the end) it reduces along that axis instead, following the axis and `:keepdims` rules of [`linalg:sum`](linalg-sum.md). The reduction is a strict-comparison fold: the first element wins ties and a `NaN` never replaces the seed. An empty array or axis signals an error. ```lisp (linalg:amax #2A((1 9) (3 4))) ; => 9 (linalg:amax #2A((1 9) (3 4)) :axis 0) ; => #d(3.0 9.0) (linalg:amax #2A((1 9) (3 4)) :axis 1 :keepdims t) ; => #d((9.0) (4.0)) ``` --- # FILE: references/reference/functions/linalg-amin.md # linalg:amin `(linalg:amin array &key axis keepdims)` Returns the smallest element of a vector or matrix. An empty array signals an error. For the *index* of the smallest element, use [`linalg:argmin`](linalg-argmin.md); the counterpart for the largest element is [`linalg:amax`](linalg-amax.md). With an integer `:axis` (negative counts from the end) it reduces along that axis instead, following the axis and `:keepdims` rules of [`linalg:sum`](linalg-sum.md). The reduction is a strict-comparison fold: the first element wins ties and a `NaN` never replaces the seed. An empty array or axis signals an error. ```lisp (linalg:amin #(5 2 8)) ; => 2 (linalg:amin #2A((1 9) (3 4)) :axis 0) ; => #d(1.0 4.0) ``` --- # FILE: references/reference/functions/linalg-arange.md # linalg:arange `(linalg:arange stop &key element-type)` / `(linalg:arange start stop &optional step &key element-type)` Creates the vector of numbers from `start` (default 0) up to but excluding `stop`, advancing by `step` (default 1; may be negative). With one argument it counts from 0, like numpy's `arange`. Double-float by default; pass `:element-type 'single-float` for a packed single-float (`#f`) result. For a fixed element count with both endpoints included, use [`linalg:linspace`](linalg-linspace.md) instead. ```lisp (linalg:arange 5) ; => #d(0.0 1.0 2.0 3.0 4.0) (linalg:arange 2 10 2) ; => #d(2.0 4.0 6.0 8.0) (linalg:arange 5 0 -1) ; => #d(5.0 4.0 3.0 2.0 1.0) (linalg:arange 0 4 :element-type 'single-float) ; => #f(0.0 1.0 2.0 3.0) ``` --- # FILE: references/reference/functions/linalg-argmax.md # linalg:argmax `(linalg:argmax array &key axis)` Returns the zero-based index of the largest element of a vector, taking the first index on ties. Without an axis it accepts vectors only; an empty vector signals an error. For the value itself, use [`linalg:amax`](linalg-amax.md); the counterpart is [`linalg:argmin`](linalg-argmin.md). With an integer `:axis` (negative counts from the end) it returns the per-slice indices along that axis, the axis dropped from the result -- the classification idiom `(linalg:argmax logits :axis 1)`. A rank >= 2 result is a packed *double* array of index values (linalg arrays have no integer width, and `(= 3.0 3)` holds for comparisons); a vector reduces to the integer index itself. ```lisp (linalg:argmax #(1 9 3)) ; => 1 (linalg:argmax #2A((1 9 3) (7 5 6)) :axis 1) ; => #d(1.0 0.0) ``` --- # FILE: references/reference/functions/linalg-argmin.md # linalg:argmin `(linalg:argmin array &key axis)` Returns the zero-based index of the smallest element of a vector, taking the first index on ties. Without an axis it accepts vectors only; an empty vector signals an error. For the value itself, use [`linalg:amin`](linalg-amin.md); the counterpart is [`linalg:argmax`](linalg-argmax.md). With an integer `:axis` (negative counts from the end) it returns the per-slice indices along that axis, the axis dropped from the result. A rank >= 2 result is a packed *double* array of index values (linalg arrays have no integer width, and `(= 3.0 3)` holds for comparisons); a vector reduces to the integer index itself. ```lisp (linalg:argmin #(5 2 8)) ; => 1 (linalg:argmin #2A((1 9 3) (7 5 6)) :axis 1) ; => #d(0.0 1.0) ``` --- # FILE: references/reference/functions/linalg-array-equal.md # linalg:array-equal `(linalg:array-equal a b)` Returns `t` when `a` and `b` have the same shape and numerically equal elements, `nil` otherwise. Numbers are compared with `=`, so `1` and `1.0` compare equal (like numpy's `array_equal`). This is the way to compare linalg results: arrays themselves compare by identity only (`eq`), so two separately built arrays are never `equal`. ```lisp (linalg:array-equal (linalg:eye 2) #2A((1 0) (0 1))) ; => T (linalg:array-equal #(1 2) #(1 2 3)) ; => NIL ``` --- # FILE: references/reference/functions/linalg-asin.md # linalg:asin `(linalg:asin array)` Returns a fresh array of the same shape with the arc sine applied to every element (numpy's `np.arcsin`) -- equivalent to `(linalg:emap #'asin array)`, but as a named function it is accelerated under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg). Like [`asin`](asin-acos-atan.md) itself, the WASM backends compute it with a software approximation whose low-order digits can differ from the interpreter's and the JVM's. ```lisp (linalg:asin (linalg:zeros 3)) ; => #d(0.0 0.0 0.0) ``` --- # FILE: references/reference/functions/linalg-atan.md # linalg:atan `(linalg:atan array)` Returns a fresh array of the same shape with the arc tangent applied to every element (numpy's `np.arctan`) -- equivalent to `(linalg:emap #'atan array)`, but as a named function it is accelerated under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg). Like [`atan`](asin-acos-atan.md) itself, the WASM backends compute it with a software approximation whose low-order digits can differ from the interpreter's and the JVM's. ```lisp (linalg:atan (linalg:zeros 3)) ; => #d(0.0 0.0 0.0) ``` --- # FILE: references/reference/functions/linalg-choice.md # linalg:choice `(linalg:choice n size)` Returns `size` uniform indices in `[0, n)`, drawn *with* replacement (numpy's `np.random.choice` default for an integer argument), as a packed double vector of integer values -- the mini-batch sampling idiom, typically fed to [`linalg:take-rows`](linalg-take-rows.md). Seed with [`linalg:seed`](linalg-seed.md) for a backend-identical sequence; for indices without replacement, use [`linalg:permutation`](linalg-permutation.md). ```lisp (linalg:seed 42) ; => 42 (linalg:choice 60000 4) ; => #d(26833.0 11120.0 29256.0 22347.0) ``` --- # FILE: references/reference/functions/linalg-clip.md # linalg:clip `(linalg:clip a lo hi)` Returns a fresh array with every element limited to the `[lo, hi]` interval (numpy's `np.clip` with scalar bounds), defined as the composition `(linalg:minimum (linalg:maximum a lo) hi)`. Two consequences of that definition: a `NaN` element becomes `lo` (the first comparison is false, so the bound wins), and inverted bounds (`lo > hi`) send every element to `hi`. Rides the [`linalg:maximum`](linalg-maximum.md) / [`linalg:minimum`](linalg-minimum.md) kernels under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg). ```lisp (linalg:clip #d(-2.0 0.5 3.0) -1.0 1.0) ; => #d(-1.0 0.5 1.0) ``` --- # FILE: references/reference/functions/linalg-concatenate.md # linalg:concatenate `(linalg:concatenate arrays &key axis)` Joins the arrays in the list `arrays` along an **existing** axis (numpy's `np.concatenate`, torch's `cat`). `:axis` defaults to 0 and a negative value counts from the end. Every input must have the same rank and the same extents on every axis but that one; the joined axis's extent is their sum. The result is a fresh array with the first input's element width. To join along a **new** axis instead, use [`linalg:stack`](linalg-stack.md). ```lisp (linalg:concatenate (list #(1 2) #(3))) ; => #d(1.0 2.0 3.0) (linalg:concatenate (list #2A((1 2)) #2A((3 4)))) ; => #d((1.0 2.0) (3.0 4.0)) (linalg:concatenate (list #2A((1 2)) #2A((3 4))) :axis 1) ; => #d((1.0 2.0 3.0 4.0)) ``` --- # FILE: references/reference/functions/linalg-cos.md # linalg:cos `(linalg:cos array)` Returns a fresh array of the same shape with the cosine applied to every element (numpy's `np.cos`) -- equivalent to `(linalg:emap #'cos array)`, but as a named function it is accelerated under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg). Like [`cos`](sin-cos-tan.md) itself, the WASM backends compute it with a software approximation whose low-order digits can differ from the interpreter's and the JVM's. ```lisp (linalg:cos (linalg:zeros 3)) ; => #d(1.0 1.0 1.0) ``` --- # FILE: references/reference/functions/linalg-cosh.md # linalg:cosh `(linalg:cosh array)` Returns a fresh array of the same shape with the hyperbolic cosine applied to every element (numpy's `np.cosh`) -- equivalent to `(linalg:emap #'cosh array)`, but as a named function it is accelerated under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg). Like [`cosh`](sinh-cosh-tanh.md) itself, the WASM backends compute it with a software approximation whose low-order digits can differ from the interpreter's and the JVM's. ```lisp (linalg:cosh (linalg:zeros 3)) ; => #d(1.0 1.0 1.0) ``` --- # FILE: references/reference/functions/linalg-det.md # linalg:det `(linalg:det matrix)` Returns the determinant of a square matrix, computed by Gaussian elimination with partial pivoting in floating point. A non-square argument signals an error. A zero determinant is the condition under which [`linalg:inv`](linalg-inv.md) and [`linalg:solve`](linalg-solve.md) fail; note that because the computation is floating point, a nearly singular matrix may yield a small epsilon rather than exactly `0` (the example below is exact only because its elimination is roundoff-free). ```lisp (linalg:det #2A((1 2) (3 4))) ; => -2.0 (linalg:det #2A((1 2) (2 4))) ; => 0.0 ``` --- # FILE: references/reference/functions/linalg-diff.md # linalg:diff `(linalg:diff array &key n axis)` Returns the `:n`-th discrete difference along `:axis` (numpy's `np.diff`; `:axis` defaults to the last axis, `-1`, and a negative axis counts from the end): each output element is `a[..., i+1, ...] - a[..., i, ...]`, applied `:n` times (default 1), so each step shortens that axis by one. It works for any rank -- a matrix differences within each row by default, down each column with `:axis 0` -- and the result is a fresh packed array of the input's width (`#f` stays `#f`). `:n 0` returns a packed copy. For a derivative estimate that keeps the input length, use [`linalg:gradient`](linalg-gradient.md). ```lisp (linalg:diff #(1 2 4 7 0)) ; => #d(1.0 2.0 3.0 -7.0) ``` ```lisp (linalg:diff #(1 2 4 7 0) :n 2) ; => #d(1.0 1.0 -10.0) ``` ```lisp (linalg:diff #2A((1 3 6) (0 5 6))) ; => #d((2.0 3.0) (5.0 1.0)) ``` ```lisp (linalg:diff #2A((1 3 6) (0 5 6)) :axis 0) ; => #d((-1.0 2.0 0.0)) ``` --- # FILE: references/reference/functions/linalg-div.md # linalg:div `(linalg:div a b)` Divides `a` by `b` elementwise, returning a fresh packed float array. The operands broadcast by numpy's rules, exactly as [`linalg:add`](linalg-add.md) describes: a scalar broadcasts over the other operand's shape, and two arrays of different shapes broadcast along their trailing axes when each aligned extent pair is equal or contains a 1. ```lisp (linalg:div #(1 2 3) 2) ; => #d(0.5 1.0 1.5) ``` --- # FILE: references/reference/functions/linalg-dot.md # linalg:dot `(linalg:dot a b)` The numpy-style dot product, dispatching on the operand ranks: vector . vector gives a scalar (the inner product), matrix . vector and vector . matrix give a vector, and matrix . matrix gives the matrix product. A scalar operand multiplies elementwise, like [`linalg:mul`](linalg-mul.md). Mismatched inner dimensions signal an error. Both operands must be of rank <= 2: numpy's `np.dot` contracts a rank-n operand against the *second-to-last* axis of the other, which is not what a stacked matrix product means, so that shape signals an error pointing at [`linalg:matmul`](linalg-matmul.md) instead of returning a silently wrong answer. When only the matrix product is intended, `linalg:matmul` additionally rejects scalar operands and stacks rank >= 3. ```lisp (linalg:dot #(1 2 3) #(4 5 6)) ; => 32 (linalg:dot #2A((1 2) (3 4)) #(1 1)) ; => #d(3.0 7.0) (linalg:dot #(1 1) #2A((1 2) (3 4))) ; => #d(4.0 6.0) ``` --- # FILE: references/reference/functions/linalg-emap.md # linalg:emap `(linalg:emap function array)` Returns a fresh array of the same shape with `function` applied to every element -- the array analogue of `mapcar`. It works on vectors and matrices alike, and accepts any function value: a `lambda`, a `#'`-quoted built-in, or a `#'`-quoted linalg function. The binary elementwise operators [`linalg:add`](linalg-add.md), [`linalg:sub`](linalg-sub.md), [`linalg:mul`](linalg-mul.md) and [`linalg:div`](linalg-div.md) cover the common two-operand cases. ```lisp (linalg:emap (lambda (x) (* x x)) (linalg:arange 4)) ; => #d(0.0 1.0 4.0 9.0) ``` --- # FILE: references/reference/functions/linalg-equal.md # linalg:equal `(linalg:equal a b)` Returns the elementwise numeric equality of `a` and `b` as a `0.0`/`1.0` mask (numpy's `==`, which produces a boolean array); either operand may be a scalar, and arrays broadcast by numpy's rules exactly as [`linalg:add`](linalg-add.md) describes. Multiply by the mask where numpy would boolean-index. For a single boolean answer over the whole array, use [`linalg:array-equal`](linalg-array-equal.md); the ordering comparisons are [`linalg:greater`](linalg-greater.md), [`linalg:greater-equal`](linalg-greater-equal.md), [`linalg:less`](linalg-less.md) and [`linalg:less-equal`](linalg-less-equal.md). ```lisp (linalg:equal #(1 5 3) #(2 5 1)) ; => #d(0.0 1.0 0.0) ``` --- # FILE: references/reference/functions/linalg-erf.md # linalg:erf `(linalg:erf a)` Elementwise Gauss error function (`scipy.special.erf`), an odd function rising from `-1` to `1`: ```text erf(x) = 2 / sqrt(pi) * integral from 0 to x of e^(-t^2) dt ``` Accurate to the last few ulps of a double over the whole range: it sums the all-positive-term series `A&S 7.1.6` rather than the alternating Maclaurin series, whose cancellation loses every significant digit by `|x| ~ 3`, and returns exactly `+-1` beyond `|x| = 6`, where the difference is below a double's resolution. There is no `erfc` member: `(linalg:sub 1.0 (linalg:erf a))` is it, and the far tail where a dedicated `erfc` would be more accurate is also where `erf` itself is already `1`. The differentiable counterpart is [`torch:erf`](torch-erf.md), and `x * (1 + erf(x / sqrt(2))) / 2` is [`torch:gelu`](torch-gelu.md). ```lisp (linalg:erf #(0.0 1.0 -1.0)) ; => #d(0.0 0.842700792949715 -0.842700792949715) ``` --- # FILE: references/reference/functions/linalg-exp.md # linalg:exp `(linalg:exp array)` Returns a fresh array of the same shape with `e^x` applied to every element (numpy's `np.exp`) -- equivalent to `(linalg:emap #'exp array)`, but as a named function it is accelerated under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg). Like [`exp`](exp.md) itself, the WASM backends compute it with a software approximation whose low-order digits can differ from the interpreter's and the JVM's. ```lisp (linalg:exp (linalg:zeros 3)) ; => #d(1.0 1.0 1.0) ``` --- # FILE: references/reference/functions/linalg-expand-dims.md # linalg:expand-dims `(linalg:expand-dims array axis)` Returns a copy of `array` with a new axis of extent 1 inserted at `axis` (numpy's `np.expand_dims`, torch's `unsqueeze`). A negative `axis` counts from the end of the *result*, so `-1` appends the new axis. The row-major element order is unchanged -- only the shape is -- and the element width follows the input. The inverse is [`linalg:squeeze`](linalg-squeeze.md). ```lisp (linalg:expand-dims #(1 2 3) 0) ; => #d((1.0 2.0 3.0)) (linalg:expand-dims #(1 2 3) -1) ; => #d((1.0) (2.0) (3.0)) ``` --- # FILE: references/reference/functions/linalg-eye.md # linalg:eye `(linalg:eye n &key element-type)` Creates the `n`-by-`n` identity matrix: ones on the main diagonal, zeros everywhere else. Multiplying by it with [`linalg:matmul`](linalg-matmul.md) leaves a matrix unchanged, and it is a convenient reference operand for [`linalg:array-equal`](linalg-array-equal.md). Double-float by default; pass `:element-type 'single-float` for a packed single-float (`#f`) result. ```lisp (linalg:eye 3) ; => #d((1.0 0.0 0.0) (0.0 1.0 0.0) (0.0 0.0 1.0)) (linalg:eye 2 :element-type 'single-float) ; => #f((1.0 0.0) (0.0 1.0)) ``` --- # FILE: references/reference/functions/linalg-flatten.md # linalg:flatten `(linalg:flatten array)` Returns the elements of an array as a fresh rank-1 vector, in row-major order. It is equivalent to [`linalg:reshape`](linalg-reshape.md) with the total [`linalg:size`](linalg-size.md) as the target shape; a vector input yields a fresh copy of itself. ```lisp (linalg:flatten #2A((1 2) (3 4))) ; => #d(1.0 2.0 3.0 4.0) ``` --- # FILE: references/reference/functions/linalg-from-list.md # linalg:from-list `(linalg:from-list list &key element-type)` Converts a list into a linalg array: a flat list becomes a rank-1 vector, and a list of equal-length row lists becomes a rank-2 matrix. This is the usual way to write array literals in linalg code. The inverse conversion is [`linalg:to-list`](linalg-to-list.md). Double-float by default; pass `:element-type 'single-float` for a packed single-float (`#f`) result. ```lisp (linalg:from-list '(1 2 3)) ; => #d(1.0 2.0 3.0) (linalg:from-list '((1 2) (3 4))) ; => #d((1.0 2.0) (3.0 4.0)) (linalg:from-list '(1 2) :element-type 'single-float) ; => #f(1.0 2.0) ``` --- # FILE: references/reference/functions/linalg-full.md # linalg:full `(linalg:full shape value &key element-type)` Creates an array with every element set to `value`. `shape` is an integer for a rank-1 vector or a list `(rows cols)` for a rank-2 matrix. [`linalg:zeros`](linalg-zeros.md) and [`linalg:ones`](linalg-ones.md) are the special cases for 0 and 1. Double-float by default; pass `:element-type 'single-float` for a packed single-float (`#f`) result. ```lisp (linalg:full '(2 2) 7) ; => #d((7.0 7.0) (7.0 7.0)) (linalg:full 2 0.5 :element-type 'single-float) ; => #f(0.5 0.5) ``` --- # FILE: references/reference/functions/linalg-gather.md # linalg:gather `(linalg:gather matrix indices)` Returns the per-row elements `a[i, idx[i]]` of a matrix (numpy's `y[np.arange(n), t]` fancy-indexing idiom) as a vector of the input's width -- the pick-the-target-class-probability step of a cross-entropy loss. Index values are truncated to integers. It signals an error unless the input is a matrix and the index length matches its rows. For selecting whole rows instead, use [`linalg:take-rows`](linalg-take-rows.md). ```lisp (linalg:gather #2A((10 11 12) (20 21 22)) #(2 0)) ; => #d(12.0 20.0) ``` --- # FILE: references/reference/functions/linalg-gradient.md # linalg:gradient `(linalg:gradient samples &optional spacing)` Returns the numerical derivative of a vector of samples (numpy's `np.gradient`): second-order central differences at interior points and first-order one-sided differences at the two ends, so the result has the same length as the input (unlike [`linalg:diff`](linalg-diff.md)). `spacing` is either a uniform sample spacing (a number, default 1) or a coordinate vector of the same length for non-uniformly spaced samples (numpy's second-order interior formula, exact for quadratics). Vectors only, with at least 2 samples; the result preserves the input's width. ```lisp (linalg:gradient #(0 1 4 9 16)) ; => #d(1.0 2.0 4.0 6.0 7.0) ``` ```lisp (linalg:gradient #(0 1 4 9 16) 2) ; => #d(0.5 1.0 2.0 3.0 3.5) ``` ```lisp (linalg:gradient #(0 1 9) #(0 1 3)) ; => #d(1.0 2.0 4.0) ``` --- # FILE: references/reference/functions/linalg-greater-equal.md # linalg:greater-equal `(linalg:greater-equal a b)` Returns the elementwise `a >= b` comparison as a `0.0`/`1.0` mask (numpy's `>=`, which produces a boolean array); either operand may be a scalar, and arrays broadcast by numpy's rules exactly as [`linalg:add`](linalg-add.md) describes. Multiply by the mask where numpy would boolean-index. The siblings are [`linalg:greater`](linalg-greater.md), [`linalg:less`](linalg-less.md), [`linalg:less-equal`](linalg-less-equal.md) and [`linalg:equal`](linalg-equal.md). ```lisp (linalg:greater-equal #(1 5 3) #(1 6 2)) ; => #d(1.0 0.0 1.0) ``` --- # FILE: references/reference/functions/linalg-greater.md # linalg:greater `(linalg:greater a b)` Returns the elementwise `a > b` comparison as a `0.0`/`1.0` mask (numpy's `>`, which produces a boolean array); either operand may be a scalar, and arrays broadcast by numpy's rules exactly as [`linalg:add`](linalg-add.md) describes. Multiply by the mask where numpy would boolean-index -- `(linalg:greater x 0)` is the relu-gradient mask, and a thresholded [`linalg:rand`](linalg-rand.md) gives a dropout mask. The siblings are [`linalg:greater-equal`](linalg-greater-equal.md), [`linalg:less`](linalg-less.md), [`linalg:less-equal`](linalg-less-equal.md) and [`linalg:equal`](linalg-equal.md). ```lisp (linalg:greater #(1 5 3) 2) ; => #d(0.0 1.0 1.0) ``` --- # FILE: references/reference/functions/linalg-inv.md # linalg:inv `(linalg:inv matrix)` Returns the inverse of a square matrix, computed by Gauss-Jordan elimination on the augmented matrix `[a | I]`. The result is a packed double-float array -- linalg computes in floating point (speed over exactness), so a general inverse carries the usual rounding. To solve a linear system it is usually clearer to call [`linalg:solve`](linalg-solve.md) directly. ```lisp (linalg:inv #2A((4 0) (2 4))) ; => #d((0.25 0.0) (-0.125 0.25)) ``` A singular matrix (one whose [`linalg:det`](linalg-det.md) is 0) has no inverse and signals an error: ```console > (linalg:inv #2A((1 2) (2 4))) Error: linalg: inv of a singular matrix ``` --- # FILE: references/reference/functions/linalg-less-equal.md # linalg:less-equal `(linalg:less-equal a b)` Returns the elementwise `a <= b` comparison as a `0.0`/`1.0` mask (numpy's `<=`, which produces a boolean array); either operand may be a scalar, and arrays broadcast by numpy's rules exactly as [`linalg:add`](linalg-add.md) describes. Multiply by the mask where numpy would boolean-index. The siblings are [`linalg:less`](linalg-less.md), [`linalg:greater`](linalg-greater.md), [`linalg:greater-equal`](linalg-greater-equal.md) and [`linalg:equal`](linalg-equal.md). ```lisp (linalg:less-equal #(1 5 3) 3) ; => #d(1.0 0.0 1.0) ``` --- # FILE: references/reference/functions/linalg-less.md # linalg:less `(linalg:less a b)` Returns the elementwise `a < b` comparison as a `0.0`/`1.0` mask (numpy's `<`, which produces a boolean array); either operand may be a scalar, and arrays broadcast by numpy's rules exactly as [`linalg:add`](linalg-add.md) describes. Multiply by the mask where numpy would boolean-index. The siblings are [`linalg:less-equal`](linalg-less-equal.md), [`linalg:greater`](linalg-greater.md), [`linalg:greater-equal`](linalg-greater-equal.md) and [`linalg:equal`](linalg-equal.md). ```lisp (linalg:less #(1 5 3) 3) ; => #d(1.0 0.0 0.0) ``` --- # FILE: references/reference/functions/linalg-linspace.md # linalg:linspace `(linalg:linspace start stop n &key element-type)` Creates the packed vector of `n` evenly spaced numbers from `start` to `stop`, both endpoints included. Double-float by default; pass `:element-type 'single-float` for a packed single-float (`#f`) result. For a half-open integer range driven by a step size, use [`linalg:arange`](linalg-arange.md). ```lisp (linalg:linspace 0 1 5) ; => #d(0.0 0.25 0.5 0.75 1.0) (linalg:linspace 0 1 3 :element-type 'single-float) ; => #f(0.0 0.5 1.0) ``` --- # FILE: references/reference/functions/linalg-log-softmax.md # linalg:log-softmax `(linalg:log-softmax array &key axis)` Returns the logarithm of [`linalg:softmax`](linalg-softmax.md), computed as `(x - max) - log(sum(exp(x - max)))` rather than as the log of the softmax, so an exactly-zero weight gives `-infinity` instead of a `NaN`. The `:axis` rules are `linalg:softmax`'s. This is the numerically stable half of a cross-entropy loss. ```lisp (linalg:log-softmax #(0 0)) ; => #d(-0.6931471805599453 -0.6931471805599453) (linalg:log-softmax #2A((0 0) (1 1)) :axis 1) ; => #d((-0.6931471805599453 -0.6931471805599453) (-0.6931471805599453 -0.6931471805599453)) ``` --- # FILE: references/reference/functions/linalg-log.md # linalg:log `(linalg:log array)` Returns a fresh array of the same shape with the natural logarithm applied to every element (numpy's `np.log`) -- equivalent to `(linalg:emap #'log array)`, but as a named function it is accelerated under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg). Like [`log`](log.md) itself, the WASM backends compute it with a software approximation whose low-order digits can differ from the interpreter's and the JVM's. ```lisp (linalg:log #(1 1 1)) ; => #d(0.0 0.0 0.0) ``` --- # FILE: references/reference/functions/linalg-matmul.md # linalg:matmul `(linalg:matmul a b)` The matrix product of `a` and `b` (numpy's `np.matmul`, the `@` operator). At rank <= 2 it behaves like [`linalg:dot`](linalg-dot.md) -- matrix . vector included -- but signals an error when either operand is a scalar, catching the mistake of writing a matrix product where an elementwise [`linalg:mul`](linalg-mul.md) would silently apply. The inner dimensions must agree; a mismatch signals an error. At rank >= 3 on either side it is the **stacked** matrix product (torch's `bmm` / `matmul`): the last two axes are the matrix and every leading axis broadcasts by the numpy rules, so a `(batch heads n d)` query times a `(batch heads d n)` key gives `(batch heads n n)` attention scores. A rank-1 operand is promoted for the product -- a row on the left, a column on the right -- and its axis is dropped again from the result, exactly as in numpy. ```lisp (linalg:matmul #2A((1 2) (3 4)) #2A((5 6) (7 8))) ; => #d((19.0 22.0) (43.0 50.0)) (linalg:shape (linalg:matmul (linalg:zeros '(2 3 4)) (linalg:zeros '(2 4 5)))) ; => (2 3 5) (linalg:matmul (linalg:reshape (linalg:arange 8) '(2 2 2)) #2A((1 0) (0 1))) ; => #d(((0.0 1.0) (2.0 3.0)) ((4.0 5.0) (6.0 7.0))) ``` --- # FILE: references/reference/functions/linalg-maximum.md # linalg:maximum `(linalg:maximum a b)` Returns a fresh array with the element-wise larger of `a` and `b` (numpy's `np.maximum`); the operands broadcast by numpy's rules, exactly as [`linalg:add`](linalg-add.md) describes (a scalar over the other operand's shape, arrays of different shapes along their trailing axes). It is defined by the strict comparison `(if (> x y) x y)`, not by an IEEE min/max primitive: the second operand wins whenever the comparison is false, which covers ties (a `-0.0` element against `0.0` takes the second operand) and unordered `NaN` comparisons (`(linalg:maximum nan-array b)` takes `b`'s elements, the reverse keeps the `NaN`s). The same rule on every backend. As a named function it is accelerated under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg). ```lisp (linalg:maximum #d(1.0 5.0 3.0) #d(4.0 2.0 3.0)) ; => #d(4.0 5.0 3.0) (linalg:maximum #d(1.0 5.0 3.0) 2.5) ; => #d(2.5 5.0 3.0) ``` --- # FILE: references/reference/functions/linalg-mean.md # linalg:mean `(linalg:mean array &key axis keepdims)` Returns the arithmetic mean of every element: the [`linalg:sum`](linalg-sum.md) divided by the [`linalg:size`](linalg-size.md). Like a reduction in numpy, the result follows the element type: a packed double-float array (anything built by a linalg constructor) gives a double, while a plain integer array gives an exact rational. With an integer `:axis` (negative counts from the end) it averages along that axis instead, following exactly the axis and `:keepdims` rules of [`linalg:sum`](linalg-sum.md): the axis is dropped from the result (kept with extent 1 under a non-nil `:keepdims`), and a vector without `:keepdims` reduces to the scalar itself. ```lisp (linalg:mean #(1 2 3 4)) ; => 5/2 (linalg:mean #2A((1 2 3) (4 5 6)) :axis 0) ; => #d(2.5 3.5 4.5) ``` --- # FILE: references/reference/functions/linalg-minimum.md # linalg:minimum `(linalg:minimum a b)` Returns a fresh array with the element-wise smaller of `a` and `b` (numpy's `np.minimum`); the operands broadcast by numpy's rules, exactly as [`linalg:add`](linalg-add.md) describes (a scalar over the other operand's shape, arrays of different shapes along their trailing axes). The mirror of [`linalg:maximum`](linalg-maximum.md): defined by `(if (< x y) x y)`, so the second operand wins whenever the comparison is false (ties and `NaN` included). As a named function it is accelerated under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg). ```lisp (linalg:minimum #d(1.0 5.0 3.0) #d(4.0 2.0 3.0)) ; => #d(1.0 2.0 3.0) (linalg:minimum 4.0 #d(1.0 5.0 3.0)) ; => #d(1.0 4.0 3.0) ``` --- # FILE: references/reference/functions/linalg-minus.md # linalg:- `(linalg:- a &rest arrays)` Subtracts the remaining arguments from `a` elementwise, left to right, and returns a fresh array. It is the CL operator spelling of [`linalg:sub`](linalg-sub.md), broadcasting by the same numpy rules. With a single argument it negates, exactly like CL `-`. ```lisp (linalg:- #(5 5) 1) ; => #d(4.0 4.0) (linalg:- #(10 10) 1 2) ; => #d(7.0 7.0) (linalg:- #(5 5)) ; => #d(-5.0 -5.0) ``` --- # FILE: references/reference/functions/linalg-mul.md # linalg:mul `(linalg:mul a b)` Multiplies `a` and `b` elementwise (the Hadamard product), returning a fresh array -- this is NOT the matrix product; for that use [`linalg:matmul`](linalg-matmul.md) or [`linalg:dot`](linalg-dot.md). The operands broadcast by numpy's rules, exactly as [`linalg:add`](linalg-add.md) describes: a scalar broadcasts over the other operand's shape, and two arrays of different shapes broadcast along their trailing axes when each aligned extent pair is equal or contains a 1. ```lisp (linalg:mul 2 #2A((1 2) (3 4))) ; => #d((2.0 4.0) (6.0 8.0)) (linalg:mul #2A((1 2) (3 4)) #2A((5 6) (7 8))) ; => #d((5.0 12.0) (21.0 32.0)) (linalg:mul #2A((1 2) (3 4)) #(10 20)) ; => #d((10.0 40.0) (30.0 80.0)) ``` --- # FILE: references/reference/functions/linalg-ndim.md # linalg:ndim `(linalg:ndim a)` Returns the number of dimensions of `a` (numpy's `np.ndim`): 0 for a plain number, 1 for a vector, 2 for a matrix, and so on. It is the linalg spelling of `array-rank`, extended to accept scalars. For the dimension sizes themselves, use [`linalg:shape`](linalg-shape.md); for the total element count, [`linalg:size`](linalg-size.md). ```lisp (linalg:ndim 3.0) ; => 0 (linalg:ndim #(1 2 3)) ; => 1 (linalg:ndim #2A((1 2) (3 4))) ; => 2 ``` --- # FILE: references/reference/functions/linalg-negative.md # linalg:negative `(linalg:negative array)` Returns a fresh array of the same shape with every element negated (numpy's `np.negative`) -- equivalent to `(linalg:emap (lambda (x) (- x)) array)`, but as a named function it is accelerated under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg). ```lisp (linalg:negative #(1 -2 3)) ; => #d(-1.0 2.0 -3.0) ``` --- # FILE: references/reference/functions/linalg-norm.md # linalg:norm `(linalg:norm array)` Returns the Euclidean (L2) norm of a vector, or the Frobenius norm of a matrix: the square root of the sum of the squared elements. Because `sqrt` returns a float, the result is a float even for integer inputs. ```lisp (linalg:norm #(3 4)) ; => 5.0 ``` --- # FILE: references/reference/functions/linalg-one-hot.md # linalg:one-hot `(linalg:one-hot indices n &key element-type)` Returns the `(length indices)` x `n` one-hot matrix: row `i` holds `1.0` in column `indices[i]` (truncated to an integer) and `0.0` elsewhere -- the label-encoding step for a classification loss. Double by default; pass `:element-type 'single-float` for a packed `#f` result. [`linalg:gather`](linalg-gather.md) goes the other way, picking one element per row by index. ```lisp (linalg:one-hot #(1 0 2) 3) ; => #d((0.0 1.0 0.0) (1.0 0.0 0.0) (0.0 0.0 1.0)) ``` --- # FILE: references/reference/functions/linalg-ones.md # linalg:ones `(linalg:ones shape &key element-type)` Creates an array with every element set to 1. `shape` is an integer for a rank-1 vector or a list `(rows cols)` for a rank-2 matrix, like [`linalg:zeros`](linalg-zeros.md). For an arbitrary fill value use [`linalg:full`](linalg-full.md). Double-float by default; pass `:element-type 'single-float` for a packed single-float (`#f`) result. ```lisp (linalg:ones '(2 2)) ; => #d((1.0 1.0) (1.0 1.0)) (linalg:ones 2 :element-type 'single-float) ; => #f(1.0 1.0) ``` --- # FILE: references/reference/functions/linalg-outer.md # linalg:outer `(linalg:outer u v)` The outer product of two vectors: element `(i j)` of the resulting matrix is the product of `u`'s element `i` and `v`'s element `j`. Like numpy, both inputs are flattened first, so matrices are accepted and treated as their row-major element sequence. For the inner product, use [`linalg:dot`](linalg-dot.md). ```lisp (linalg:outer #(1 2) #(3 4 5)) ; => #d((3.0 4.0 5.0) (6.0 8.0 10.0)) ``` --- # FILE: references/reference/functions/linalg-pad.md # linalg:pad `(linalg:pad array pads)` Zero padding (numpy's `np.pad` in its default constant mode): returns a fresh array with `array` copied into the interior and `0.0` everywhere else. `pads` is a list of `(before after)` pairs, one per axis -- or a single non-negative integer applied to both sides of every axis. The result keeps the input's element width (a `#f` array pads to `#f`). ```lisp (linalg:pad #(1 2) 1) ; => #d(0.0 1.0 2.0 0.0) (linalg:pad #2A((1 2) (3 4)) '((0 0) (1 1))) ; => #d((0.0 1.0 2.0 0.0) (0.0 3.0 4.0 0.0)) ``` --- # FILE: references/reference/functions/linalg-permutation.md # linalg:permutation `(linalg:permutation n)` Returns the integers `0..n-1` in a Fisher-Yates shuffle (numpy's `np.random.permutation` of an integer), as a packed double vector of integer values -- the epoch-shuffling idiom, typically fed to [`linalg:take-rows`](linalg-take-rows.md). Seed with [`linalg:seed`](linalg-seed.md) for a backend-identical shuffle; for indices *with* replacement, use [`linalg:choice`](linalg-choice.md). ```lisp (linalg:seed 9) ; => 9 (linalg:permutation 10) ; => #d(4.0 5.0 6.0 2.0 9.0 7.0 1.0 0.0 8.0 3.0) ``` --- # FILE: references/reference/functions/linalg-plus.md # linalg:+ `(linalg:+ &rest arrays)` Sums its arguments elementwise, left to right, and returns a fresh array. It is the CL operator spelling of [`linalg:add`](linalg-add.md): every fold step broadcasts by the same numpy rules, so scalars and shape-compatible arrays mix freely. With no argument it returns `0`, and with one argument it returns that argument unchanged. ```lisp (linalg:+ #(1 2 3) 10) ; => #d(11.0 12.0 13.0) (linalg:+ #(1 2) #(3 4) #(10 10)) ; => #d(14.0 16.0) (linalg:+) ; => 0 ``` --- # FILE: references/reference/functions/linalg-power.md # linalg:power `(linalg:power a b)` Elementwise `a` raised to `b` (numpy's `np.power`, the `**` operator). Either operand may be a scalar and two arrays broadcast by the numpy rules, exactly like [`linalg:mul`](linalg-mul.md). Both operands go through the same float element model as the rest of `linalg`, so a fractional exponent is the ordinary float power. ```lisp (linalg:power #(1 2 3) 2) ; => #d(1.0 4.0 9.0) (linalg:power 2 #(1 2 3)) ; => #d(2.0 4.0 8.0) ``` --- # FILE: references/reference/functions/linalg-rand.md # linalg:rand `(linalg:rand shape &key element-type)` Returns an array of uniform draws in `[0, 1)` (numpy's `np.random.rand`, but taking a shape designator like [`linalg:zeros`](linalg-zeros.md): an integer for a vector, a list for a matrix). Double-float by default; pass `:element-type 'single-float` for a packed `#f` result. Draws come from the shared generator, so a program that calls [`linalg:seed`](linalg-seed.md) first gets the same values on every backend. ```lisp (linalg:seed 42) ; => 42 (linalg:emap (lambda (x) (truncate (* 1024 x))) (linalg:rand 4)) ; => #d(457.0 189.0 499.0 381.0) ``` --- # FILE: references/reference/functions/linalg-randn.md # linalg:randn `(linalg:randn shape &key element-type)` Returns an array of standard-normal draws (numpy's `np.random.randn`, but taking a shape designator like [`linalg:zeros`](linalg-zeros.md); double by default, `:element-type 'single-float` for `#f`). The Gaussians are built by Irwin-Hall -- the sum of 12 uniforms minus 6 -- rather than Box-Muller, so a sequence seeded with [`linalg:seed`](linalg-seed.md) stays bit-identical across backends (WASM's `log`/`cos` are polynomial approximations); the tails clip at +/- 6 sigma, which is fine for weight initialization but not a distribution-exact `np.random.randn`. ```lisp (linalg:seed 42) ; => 42 (linalg:emap (lambda (x) (truncate (* 1024 x))) (linalg:randn 4)) ; => #d(164.0 -469.0 -1782.0 -1292.0) ``` --- # FILE: references/reference/functions/linalg-reciprocal.md # linalg:reciprocal `(linalg:reciprocal array)` Returns a fresh array of the same shape with `1 / x` for every element, in float (numpy's `np.reciprocal` over floats) -- `(linalg:div 1 array)` under a numpy-parity name, so it rides [`linalg:div`](linalg-div.md)'s [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg) kernels. A zero element yields infinity, as in numpy's float semantics. ```lisp (linalg:reciprocal #(2 4 8)) ; => #d(0.5 0.25 0.125) ``` --- # FILE: references/reference/functions/linalg-relu.md # linalg:relu `(linalg:relu a)` Returns a fresh array with every element replaced by `max(x, 0.0)` -- the rectified linear unit, the most common neural-network activation. Defined as `(linalg:maximum a 0.0)`, so a `-0.0` or `NaN` element becomes `0.0` (the strict comparison's false arm). Rides the [`linalg:maximum`](linalg-maximum.md) kernel under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg). ```lisp (linalg:relu #d(-2.0 -0.0 3.0)) ; => #d(0.0 0.0 3.0) ``` --- # FILE: references/reference/functions/linalg-reshape.md # linalg:reshape `(linalg:reshape array shape)` Returns a fresh array with the given shape and the same elements in row-major order. `shape` is an integer for a vector or a list `(rows cols)` for a matrix, and its total size must match the input's [`linalg:size`](linalg-size.md) -- a mismatch signals an error. One extent may be `-1` and is inferred from the element count (numpy's rule); a bare `-1` shape flattens, and more than one `-1` signals an error. [`linalg:flatten`](linalg-flatten.md) is the common special case of reshaping to a vector. ```lisp (linalg:reshape (linalg:arange 6) '(2 3)) ; => #d((0.0 1.0 2.0) (3.0 4.0 5.0)) (linalg:shape (linalg:reshape (linalg:arange 12) '(3 -1))) ; => (3 4) ``` --- # FILE: references/reference/functions/linalg-row.md # linalg:row `(linalg:row array index)` Returns the axis-0 slice `index` of `array` with axis 0 **dropped** (numpy's `x[i]` integer indexing), as a fresh array of the input's width: a matrix yields the row vector, a rank-4 batch yields the rank-3 sample. The index value is truncated to an integer. This is the one-slice sibling of [`linalg:take-rows`](linalg-take-rows.md), which keeps axis 0 (numpy's `x[[i]]`, so a matrix stays a `(1 n)` matrix). Use `linalg:row` wherever numpy writes `x[i]` -- feeding one image of a batch to a forward pass, for instance -- and `aref` to read a single element. `array` must have rank >= 2; on a vector `linalg:row` signals an error, since `(aref v i)` already returns the element. ```lisp (linalg:row #2A((1 2 3) (4 5 6) (7 8 9)) 1) ; => #d(4.0 5.0 6.0) ``` --- # FILE: references/reference/functions/linalg-seed.md # linalg:seed `(linalg:seed n)` Resets the shared linalg random generator deterministically from a non-negative integer seed and returns `n`. The generator is a Wichmann-Hill combination whose draws are exact integer arithmetic plus IEEE double operations, so a seeded [`linalg:rand`](linalg-rand.md) / [`linalg:randn`](linalg-randn.md) / [`linalg:uniform`](linalg-uniform.md) / [`linalg:choice`](linalg-choice.md) / [`linalg:permutation`](linalg-permutation.md) sequence is bit-identical on every backend (interpreter, JVM and WASM). ```lisp (linalg:seed 42) ; => 42 ``` --- # FILE: references/reference/functions/linalg-shape.md # linalg:shape `(linalg:shape array)` Returns the dimension sizes of an array as a list: `(n)` for a rank-1 vector, `(rows cols)` for a rank-2 matrix. It is the linalg spelling of `array-dimensions`. For the number of dimensions, use [`linalg:ndim`](linalg-ndim.md); for the total element count, [`linalg:size`](linalg-size.md). ```lisp (linalg:shape #2A((1 2 3) (4 5 6))) ; => (2 3) (linalg:shape #(1 2 3)) ; => (3) ``` --- # FILE: references/reference/functions/linalg-sign.md # linalg:sign `(linalg:sign array)` Returns a fresh array of the same shape with the sign of every element as `-1.0` / `0.0` / `1.0` (numpy's `np.sign`) -- equivalent to `(linalg:emap #'signum array)`, but as a named function it is accelerated under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg). It follows [`signum`](signum.md)'s own edges on each backend, so a `-0.0` element (which the interpreter and JVM keep as `-0.0` and WASM maps to `0.0`) is best avoided in cross-backend output. ```lisp (linalg:sign #(-5 0 7)) ; => #d(-1.0 0.0 1.0) ``` --- # FILE: references/reference/functions/linalg-sin.md # linalg:sin `(linalg:sin array)` Returns a fresh array of the same shape with the sine applied to every element (numpy's `np.sin`) -- equivalent to `(linalg:emap #'sin array)`, but as a named function it is accelerated under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg). Like [`sin`](sin-cos-tan.md) itself, the WASM backends compute it with a software approximation whose low-order digits can differ from the interpreter's and the JVM's. ```lisp (linalg:sin (linalg:zeros 3)) ; => #d(0.0 0.0 0.0) ``` --- # FILE: references/reference/functions/linalg-sinh.md # linalg:sinh `(linalg:sinh array)` Returns a fresh array of the same shape with the hyperbolic sine applied to every element (numpy's `np.sinh`) -- equivalent to `(linalg:emap #'sinh array)`, but as a named function it is accelerated under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg). Like [`sinh`](sinh-cosh-tanh.md) itself, the WASM backends compute it with a software approximation whose low-order digits can differ from the interpreter's and the JVM's. ```lisp (linalg:sinh (linalg:zeros 3)) ; => #d(0.0 0.0 0.0) ``` --- # FILE: references/reference/functions/linalg-size.md # linalg:size `(linalg:size array)` Returns the total number of elements in an array -- the product of its dimensions, like `array-total-size`. For the per-dimension sizes, use [`linalg:shape`](linalg-shape.md). ```lisp (linalg:size #2A((1 2 3) (4 5 6))) ; => 6 ``` --- # FILE: references/reference/functions/linalg-slash.md # linalg:/ `(linalg:/ a &rest arrays)` Divides `a` by the remaining arguments elementwise, left to right, and returns a fresh array. It is the CL operator spelling of [`linalg:div`](linalg-div.md), broadcasting by the same numpy rules. With a single argument it returns the reciprocal, exactly like CL `/`. ```lisp (linalg:/ #(1 2 3) 2) ; => #d(0.5 1.0 1.5) (linalg:/ #(12 12) 2 3) ; => #d(2.0 2.0) (linalg:/ #(1.0 2.0 4.0)) ; => #d(1.0 0.5 0.25) ``` --- # FILE: references/reference/functions/linalg-slice.md # linalg:slice `(linalg:slice array specs)` Basic numpy slicing -- `x[i0:j0:k0, i1:j1, ...]` -- spelled as a list with one spec per axis. A spec is either `nil` (leave the axis whole) or `(start end)` / `(start end step)`. A negative index counts from the end, `nil` in the `start` or `end` position means "from the beginning" / "to the end", a negative `step` walks the axis backwards, and a **missing trailing spec** leaves that axis whole. Every axis is kept, exactly as numpy's `x[:, 0:3]` keeps both; to take one slice and *drop* its axis, use [`linalg:row`](linalg-row.md). The result is a fresh array with the input's element width. ```lisp (linalg:slice #2A((0 1 2) (3 4 5)) '(nil (0 2))) ; => #d((0.0 1.0) (3.0 4.0)) (linalg:slice #2A((0 1 2) (3 4 5)) '((1 2))) ; => #d((3.0 4.0 5.0)) (linalg:slice #(0 1 2 3 4 5) '((nil nil 2))) ; => #d(0.0 2.0 4.0) (linalg:slice #(0 1 2 3 4 5) '((-2 nil))) ; => #d(4.0 5.0) ``` --- # FILE: references/reference/functions/linalg-softmax.md # linalg:softmax `(linalg:softmax array &key axis)` Returns the softmax of `array`: `exp(x - max)` normalized to sum to 1. With no `:axis` the whole array is one distribution (scipy's `softmax` default); with an integer `:axis` (negative counting from the end) each slice along that axis is normalized on its own, which is the attention-weight form -- torch's `softmax(x, dim)`. The maximum is subtracted first, so a large logit cannot overflow, and an element of `-infinity` (a masked position, see [`linalg:where`](linalg-where.md)) comes out as exactly `0.0`. Like [`linalg:relu`](linalg-relu.md), `softmax` is not in numpy proper -- it lives here because it is the array-level primitive an activation layer needs. The logarithm is [`linalg:log-softmax`](linalg-log-softmax.md). ```lisp (linalg:softmax #(1 1 1 1)) ; => #d(0.25 0.25 0.25 0.25) (linalg:softmax #2A((0 0) (1 1)) :axis 1) ; => #d((0.5 0.5) (0.5 0.5)) ``` --- # FILE: references/reference/functions/linalg-solve.md # linalg:solve `(linalg:solve a b)` Solves the linear system `a . x = b` for `x`, where `a` is a square matrix and `b` is a vector (giving a vector solution) or a matrix (giving a matrix solution, one column system at a time). The result is a packed double-float array, since the implementation applies [`linalg:inv`](linalg-inv.md) via [`linalg:dot`](linalg-dot.md) in floating point. A singular `a` signals an error. ```lisp (linalg:solve #2A((4 0) (2 4)) #(8 8)) ; => #d(2.0 1.0) ``` --- # FILE: references/reference/functions/linalg-sqrt.md # linalg:sqrt `(linalg:sqrt array)` Returns a fresh array of the same shape with the square root of every element (numpy's `np.sqrt`) -- equivalent to `(linalg:emap #'sqrt array)`, but as a named function it is accelerated under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg). ```lisp (linalg:sqrt #(4 9 16)) ; => #d(2.0 3.0 4.0) ``` --- # FILE: references/reference/functions/linalg-square.md # linalg:square `(linalg:square array)` Returns a fresh array of the same shape with every element multiplied by itself (numpy's `np.square`) -- `(linalg:mul array array)` under a numpy-parity name, so it rides [`linalg:mul`](linalg-mul.md)'s [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg) kernels. A plain number squares to a plain number. ```lisp (linalg:square #(1 2 3)) ; => #d(1.0 4.0 9.0) ``` --- # FILE: references/reference/functions/linalg-squeeze.md # linalg:squeeze `(linalg:squeeze array &key axis)` Returns a copy of `array` with extent-1 axes removed (numpy's `np.squeeze`). With no `:axis` every such axis goes; with an integer `:axis` -- or a list of them, negative counting from the end -- only those, and an axis whose extent is not 1 signals an error. Squeezing away *every* axis returns the single element itself, because `linalg` has no rank-0 arrays (a plain number is what rank 0 means here). The inverse is [`linalg:expand-dims`](linalg-expand-dims.md). ```lisp (linalg:squeeze #2A((1 2 3))) ; => #d(1.0 2.0 3.0) (linalg:squeeze (linalg:expand-dims #(1 2) 0) :axis 0) ; => #d(1.0 2.0) ``` --- # FILE: references/reference/functions/linalg-stack.md # linalg:stack `(linalg:stack arrays &key axis)` Joins the arrays in the list `arrays` along a **new** axis (numpy's `np.stack`). Every input must have exactly the same shape; the result has one more axis, of extent `(length arrays)`, inserted at `:axis` (default 0, negative counting from the end of the *result*, so `-1` appends it). The result is a fresh array with the first input's element width. This is how a list of per-sample arrays becomes one batch array; to join along an axis that already exists, use [`linalg:concatenate`](linalg-concatenate.md). ```lisp (linalg:stack (list #(1 2) #(3 4))) ; => #d((1.0 2.0) (3.0 4.0)) (linalg:stack (list #(1 2) #(3 4)) :axis 1) ; => #d((1.0 3.0) (2.0 4.0)) ``` --- # FILE: references/reference/functions/linalg-star.md # linalg:* `(linalg:* &rest arrays)` Multiplies its arguments elementwise (the Hadamard product, **not** the matrix product -- that is [`linalg:matmul`](linalg-matmul.md)), left to right, and returns a fresh array. It is the CL operator spelling of [`linalg:mul`](linalg-mul.md), broadcasting by the same numpy rules. With no argument it returns `1`, and with one argument it returns that argument unchanged. ```lisp (linalg:* #(1 2) #(3 4)) ; => #d(3.0 8.0) (linalg:* #(1 2 3) 2 10) ; => #d(20.0 40.0 60.0) (linalg:*) ; => 1 ``` --- # FILE: references/reference/functions/linalg-std.md # linalg:std `(linalg:std array &key axis keepdims ddof)` Returns the standard deviation: the square root of [`linalg:var`](linalg-var.md), with the same `:axis`, `:keepdims` and `:ddof` rules (numpy's `np.std`). Together with [`linalg:mean`](linalg-mean.md) along the same axis this is the LayerNorm normalizer. ```lisp (linalg:std #(2 4 4 4 5 5 7 9)) ; => 2.0 (linalg:std #2A((0 1 2) (3 4 5)) :axis 0) ; => #d(1.5 1.5 1.5) ``` --- # FILE: references/reference/functions/linalg-sub.md # linalg:sub `(linalg:sub a b)` Subtracts `b` from `a` elementwise, returning a fresh array. The operands broadcast by numpy's rules, exactly as [`linalg:add`](linalg-add.md) describes: a scalar broadcasts over the other operand's shape, and two arrays of different shapes broadcast along their trailing axes when each aligned extent pair is equal or contains a 1. See also [`linalg:mul`](linalg-mul.md) and [`linalg:div`](linalg-div.md). ```lisp (linalg:sub #(5 5) 1) ; => #d(4.0 4.0) ``` --- # FILE: references/reference/functions/linalg-sum.md # linalg:sum `(linalg:sum array &key axis keepdims)` Returns the sum of every element of a vector or matrix. Like a reduction in numpy, the result follows the element type: a packed double-float array (anything built by a linalg constructor) gives a double, while a plain integer array gives an integer. For the average, use [`linalg:mean`](linalg-mean.md). With an integer `:axis` (negative counts from the end, numpy's rule) it instead sums along that axis: the axis is dropped from the result -- kept with extent 1 under a non-nil `:keepdims` -- and a vector without `:keepdims` reduces to the scalar itself. A non-nil `:keepdims` with no axis wraps the full sum in an all-ones-shape array, as in numpy. ```lisp (linalg:sum #2A((1 2) (3 4))) ; => 10 (linalg:sum #2A((1 2 3) (4 5 6)) :axis 0) ; => #d(5.0 7.0 9.0) (linalg:sum #2A((1 2 3) (4 5 6)) :axis 1 :keepdims t) ; => #d((6.0) (15.0)) ``` --- # FILE: references/reference/functions/linalg-take-rows.md # linalg:take-rows `(linalg:take-rows array indices)` Returns the axis-0 slices of `array` selected by the index vector `indices` (numpy's `x[mask]` / `np.take(a, idx, axis=0)`), as a fresh array of the input's width. Whole slabs are copied row-major, so any rank >= 1 works; index values are truncated to integers and the same index may appear more than once. With [`linalg:choice`](linalg-choice.md) or [`linalg:permutation`](linalg-permutation.md) supplying the indices this is the mini-batch extraction idiom; the per-row element pick is [`linalg:gather`](linalg-gather.md). Axis 0 survives even for a single index (`#(2)` yields a `(1 n)` matrix, numpy's `x[[2]]`); to take one slice and drop the axis, use [`linalg:row`](linalg-row.md). ```lisp (linalg:take-rows #2A((1 2 3) (4 5 6) (7 8 9)) #(2 0)) ; => #d((7.0 8.0 9.0) (1.0 2.0 3.0)) ``` --- # FILE: references/reference/functions/linalg-tan.md # linalg:tan `(linalg:tan array)` Returns a fresh array of the same shape with the tangent applied to every element (numpy's `np.tan`) -- equivalent to `(linalg:emap #'tan array)`, but as a named function it is accelerated under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg). Like [`tan`](sin-cos-tan.md) itself, the WASM backends compute it with a software approximation whose low-order digits can differ from the interpreter's and the JVM's (and the divergence is amplified near the poles, where the cosine crosses zero). ```lisp (linalg:tan (linalg:zeros 3)) ; => #d(0.0 0.0 0.0) ``` --- # FILE: references/reference/functions/linalg-tanh.md # linalg:tanh `(linalg:tanh array)` Returns a fresh array of the same shape with the hyperbolic tangent applied to every element (numpy's `np.tanh`) -- equivalent to `(linalg:emap #'tanh array)`, but as a named function it is accelerated under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg), which makes it the go-to activation function for neural-network code over packed arrays. Like [`tanh`](sinh-cosh-tanh.md) itself, the WASM backends compute it with a software approximation whose low-order digits can differ from the interpreter's and the JVM's. ```lisp (linalg:tanh (linalg:zeros 3)) ; => #d(0.0 0.0 0.0) ``` --- # FILE: references/reference/functions/linalg-to-list.md # linalg:to-list `(linalg:to-list array)` Converts a linalg array back into a list: a vector becomes a flat list, and a matrix becomes a list of row lists. It is the inverse of [`linalg:from-list`](linalg-from-list.md), useful for handing array contents to list functions like `mapcar` or `reduce`. ```lisp (linalg:to-list (linalg:from-list '((1 2) (3 4)))) ; => ((1.0 2.0) (3.0 4.0)) ``` --- # FILE: references/reference/functions/linalg-trace.md # linalg:trace `(linalg:trace matrix)` Returns the trace of a square matrix: the sum of the elements on the main diagonal. A non-square (or rank-1) argument signals an error. See also [`linalg:det`](linalg-det.md) for the other classic square-matrix scalar. ```lisp (linalg:trace #2A((1 2) (3 4))) ; => 5 ``` --- # FILE: references/reference/functions/linalg-transpose.md # linalg:transpose `(linalg:transpose array &optional axes)` Returns the transpose of a matrix: element `(i j)` of the result is element `(j i)` of the input. Like numpy, a rank-1 vector is returned unchanged -- there is no distinct row/column vector representation. To turn a vector into a genuine 1-row or 1-column matrix, use [`linalg:reshape`](linalg-reshape.md). With an `axes` list (numpy's `x.transpose(0, 3, 1, 2)`), returns the rank-n axis permutation instead: axis `k` of the result is axis `(nth k axes)` of the input, so the result's shape is the input's shape reindexed by `axes`. The list must name each of the input's axes exactly once. ```lisp (linalg:transpose #2A((1 2 3) (4 5 6))) ; => #d((1.0 4.0) (2.0 5.0) (3.0 6.0)) (linalg:transpose #(1 2 3)) ; => #(1 2 3) (linalg:shape (linalg:transpose (linalg:zeros '(2 3 4)) '(1 0 2))) ; => (3 2 4) ``` --- # FILE: references/reference/functions/linalg-tril.md # linalg:tril `(linalg:tril array &key k)` Returns the lower triangle of `array`: a copy with everything **above** the `k`-th diagonal set to zero (numpy's `np.tril`). The [`linalg:triu`](linalg-triu.md) rules with the comparison flipped -- `:k` defaults to 0, rank must be at least 2, and a stack of matrices is masked on its last two axes. ```lisp (linalg:tril #2A((1 2 3) (4 5 6) (7 8 9))) ; => #d((1.0 0.0 0.0) (4.0 5.0 0.0) (7.0 8.0 9.0)) (linalg:tril #2A((1 2 3) (4 5 6) (7 8 9)) :k -1) ; => #d((0.0 0.0 0.0) (4.0 0.0 0.0) (7.0 8.0 0.0)) ``` --- # FILE: references/reference/functions/linalg-triu.md # linalg:triu `(linalg:triu array &key k)` Returns the upper triangle of `array`: a copy with everything **below** the `k`-th diagonal set to zero (numpy's `np.triu`). `:k` defaults to 0, which keeps the main diagonal; a positive `:k` moves the boundary up and to the right, a negative one down and to the left. Rank must be at least 2, and for a stack of matrices the last two axes are the matrix. Applied to an all-ones matrix with `:k 1` this is the causal ("subsequent") attention mask. The mirror is [`linalg:tril`](linalg-tril.md). ```lisp (linalg:triu #2A((1 2 3) (4 5 6) (7 8 9))) ; => #d((1.0 2.0 3.0) (0.0 5.0 6.0) (0.0 0.0 9.0)) (linalg:triu (linalg:ones '(3 3)) :k 1) ; => #d((0.0 1.0 1.0) (0.0 0.0 1.0) (0.0 0.0 0.0)) ``` --- # FILE: references/reference/functions/linalg-uniform.md # linalg:uniform `(linalg:uniform lo hi shape &key element-type)` Returns an array of uniform draws in `[lo, hi)` (numpy's `np.random.uniform`, but with a required shape designator like [`linalg:zeros`](linalg-zeros.md); double by default, `:element-type 'single-float` for `#f`). Each element is `lo + (hi - lo) * u` for a `[0, 1)` draw `u` from the shared generator, so a sequence seeded with [`linalg:seed`](linalg-seed.md) is the same on every backend. ```lisp (linalg:seed 7) ; => 7 (linalg:emap (lambda (x) (truncate x)) (linalg:uniform 10 20 4)) ; => #d(15.0 15.0 18.0 12.0) ``` --- # FILE: references/reference/functions/linalg-var.md # linalg:var `(linalg:var array &key axis keepdims ddof)` Returns the variance -- the mean of the squared deviations from the mean -- of every element (no `:axis`) or along one axis, following the same `:axis` / `:keepdims` rules as [`linalg:sum`](linalg-sum.md). The divisor is `n - ddof`: the default `:ddof 0` is numpy's `np.var` and torch's `unbiased=False`, while `:ddof 1` gives the sample variance (Bessel's correction). The square root is [`linalg:std`](linalg-std.md). ```lisp (linalg:var #(1 2 3 4)) ; => 1.25 (linalg:var #(1 2 3 4) :ddof 1) ; => 1.6666666666666667 (linalg:var #2A((0 1 2) (3 4 5)) :axis 1) ; => #d(0.6666666666666666 0.6666666666666666) ``` --- # FILE: references/reference/functions/linalg-where.md # linalg:where `(linalg:where mask x y)` Elementwise selection (numpy's `np.where`): the element of `x` wherever `mask` is **non-zero**, the element of `y` wherever it is zero. The 0.0/1.0 masks that [`linalg:greater`](linalg-greater.md) and its siblings return therefore select directly, with no multiply-by-mask detour -- which matters because multiplying turns an infinite operand into a `NaN` while selecting does not, so a `-infinity` attention mask survives into [`linalg:softmax`](linalg-softmax.md) as a weight of exactly zero. All three arguments may be scalars or arrays and broadcast together by the numpy rules; the result keeps `x`'s element width when `x` is an array, otherwise `y`'s. ```lisp (linalg:where (linalg:greater #(1 5 3) 2) #(1 5 3) 0) ; => #d(0.0 5.0 3.0) (linalg:where #(1 0 1) 10 20) ; => #d(10.0 20.0 10.0) ``` --- # FILE: references/reference/functions/linalg-zeros-like.md # linalg:zeros-like `(linalg:zeros-like array)` Creates a zero-filled array with the input's shape *and* element width (numpy's `np.zeros_like`): a packed single-float (`#f`) input gives a `#f` result, anything else a packed double-float one. Unlike [`linalg:zeros`](linalg-zeros.md), which takes a shape designator, it takes the array whose shape is wanted -- the gradient-accumulator idiom. ```lisp (linalg:zeros-like #2A((1 2) (3 4))) ; => #d((0.0 0.0) (0.0 0.0)) ``` --- # FILE: references/reference/functions/linalg-zeros.md # linalg:zeros `(linalg:zeros shape &key element-type)` Creates a zero-filled array. `shape` is an integer for a rank-1 vector of that length, or a list of two integers `(rows cols)` for a rank-2 matrix -- the same shape convention used by [`linalg:ones`](linalg-ones.md) and [`linalg:full`](linalg-full.md). See the [linalg guide](../../guides/linear-algebra.md) for an overview of the package. Double-float by default; pass `:element-type 'single-float` for a packed single-float (`#f`) result. ```lisp (linalg:zeros 3) ; => #d(0.0 0.0 0.0) (linalg:zeros '(2 2)) ; => #d((0.0 0.0) (0.0 0.0)) (linalg:zeros 2 :element-type 'single-float) ; => #f(0.0 0.0) ``` --- # FILE: references/reference/functions/list*.md # list* `(list* object &rest more)` Conses the leading arguments onto the last argument, which becomes the tail. When the final argument is a list the result is a proper list extended at the front; when it is a non-list the result is a dotted list. Calling it with a single argument just returns that argument. ```lisp (list* 1 2 '(3 4)) ; => (1 2 3 4) ``` ```lisp (list* 1 2 3) ; => (1 2 . 3) ``` --- # FILE: references/reference/functions/list-all-packages.md # list-all-packages `(list-all-packages)` Every registered package, as the keywords [`find-package`](find-package.md) answers (rontolisp has no package objects). The list covers the built-in packages, the `keyword` pseudo-package and every [`defpackage`](../special-forms/defpackage.md) in the program. The interpreter reads its live registry. The compiled backends have no registry at run time and answer from a table baked in at compile time, so a package a compiled program creates later is invisible there. ```lisp (defpackage #:listed (:use #:cl)) (car (member :listed (list-all-packages))) ; => :LISTED ``` --- # FILE: references/reference/functions/list.md # list `(list &rest objects)` Returns a freshly allocated proper list of its evaluated arguments, in order. With no arguments it returns `nil` (the empty list). Each argument becomes one element, so nested calls build nested lists. ```lisp (list 1 2 3) ; => (1 2 3) ``` ```lisp (list) ; => NIL ``` --- # FILE: references/reference/functions/listen.md # listen `(listen &optional stream)` Returns `t` when a character or byte is immediately available on the designated input stream (standard input with no argument), `nil` otherwise. It never blocks: this is the `available()`/`ready()` question, which is what a protocol implementation asks to detect unexpected data -- cl-postgres uses it to spot an oversized SSL response. On the interpreter and the JVM a socket answers from the kernel receive buffer, so the answer is exact. On the WASM `--component` backend the answer is exact for a socket too, but for a different reason: it reports whether the socket's already-read chunk still holds unconsumed bytes -- bytes still waiting host-side are not observable there without blocking. Preview 1 WASM has no non-blocking probe at all and rejects `listen` at compile time. ```console (listen) ; => NIL, with no pending standard input ``` --- # FILE: references/reference/functions/listp.md # listp `(listp object)` Returns `t` if `object` is a list -- that is, either a cons cell or the empty list `nil` -- otherwise `nil`. Because `nil` counts as a list, `(listp nil)` is `t`, which is where `listp` differs from `consp`. Works in all three backends. ```lisp (listp '(1 2)) ; => T ``` ```lisp (listp nil) ; => T ``` --- # FILE: references/reference/functions/load.md # load `(load filename &key verbose print if-does-not-exist external-format)` Reads a file and evaluates every top-level form in it in the global environment, then returns `t`. `:if-does-not-exist` is real: a false value answers `nil` instead of signalling when the file is not there, which is what makes `(load "optional-config.lisp" :if-does-not-exist nil)` work. The other three are accepted and ignored -- `load` produces no progress output, so `:verbose` and `:print` have nothing to do, and every backend reads UTF-8, so there is no second `:external-format` to select. Every option value is evaluated, in the order it was written, whether or not it is used. Definitions such as `defun` and `setq` in the loaded file remain available to subsequent code. A relative `filename` resolves against the directory of the file doing the load (the entry file for a top-level `load`), so a program can be run from any working directory and still find a `(load "sibling.lisp")`. In compiled output the loaded definitions live in the runtime `eval` interpreter's global environment, so they are reached through `eval` (e.g. `(load "lib.lisp")` then `(eval '(square 5))`). Works in all three backends; the WASM `load` reads the file with WASI `path_open`, so the module must be run with a directory granted (e.g. `wasmtime run -W gc --dir . prog.wasm`). ```console (load "lib.lisp") (eval '(square 5)) (load "optional.lisp" :if-does-not-exist nil) ``` After loading a file that defines `square`, the definition is invoked through `eval`. The WASM backend needs `--dir` because it resolves the path against the preopened directories: a relative path against the first one, an absolute path against the preopened directory whose name is its longest prefix. `load` is deliberately **not** idempotent: loading the same file twice evaluates it twice, matching Common Lisp. For load-once module semantics, see [`require`](require.md) / [`provide`](provide.md). --- # FILE: references/reference/functions/log.md # log `(log number)` Returns the natural logarithm (base e) of `number` as a float. Only the one-argument form is supported -- there is no `(log number base)` form for an arbitrary base. The interpreter and JVM backends compute it with `Math.log`; the WASM backend uses a software approximation (exponent extraction plus a polynomial series), so its result may differ slightly in the least significant digits. The IEEE edges match everywhere: `(log 0.0)` is `-Infinity`, a negative argument gives `NaN`. ```lisp (log 1) ; => 0.0 ``` --- # FILE: references/reference/functions/logand.md # logand `(logand &rest integers)` Variadic bitwise AND of its integer arguments. With no arguments it returns `-1` (the identity, all bits set). The operation is exact for arbitrarily large integers on every backend. ```lisp (logand 12 10) ; => 8 ``` --- # FILE: references/reference/functions/logandc1.md # logandc1 `(logandc1 integer1 integer2)` Bitwise AND of the complement of `integer1` with `integer2`, i.e. `(logand (lognot integer1) integer2)`. The operation is exact for arbitrarily large integers on every backend. ```lisp (logandc1 12 10) ; => 2 ``` --- # FILE: references/reference/functions/logandc2.md # logandc2 `(logandc2 integer1 integer2)` Bitwise AND of `integer1` with the complement of `integer2`, i.e. `(logand integer1 (lognot integer2))`. The operation is exact for arbitrarily large integers on every backend. ```lisp (logandc2 12 10) ; => 4 ``` --- # FILE: references/reference/functions/logbitp.md # logbitp `(logbitp index integer)` Tests bit `index` (0 = least significant bit) of the two's-complement `integer`, returning `t` when it is set and `nil` otherwise. A negative `integer` has infinitely many high one-bits, so `(logbitp index -1)` is `t` for every `index`. `integer` may be any magnitude on every backend. ```lisp (logbitp 2 5) ; => T ``` ```lisp (logbitp 1 5) ; => NIL ``` --- # FILE: references/reference/functions/logical-pathname.md # logical-pathname `(logical-pathname pathspec)` Always signals. Common Lisp requires an error unless the argument is a logical pathname or a logical-pathname namestring, and rontolisp can define no logical host, so no argument can satisfy that. Answering a physical pathname instead would claim a translation table exists. ```console > (logical-pathname "SYS:SRC;") LOGICAL-PATHNAME: "SYS:SRC;" does not name a logical pathname (rontolisp defines no logical hosts) ``` Use [`pathname`](pathname.md) to build the physical pathname a namestring names, and [`translate-logical-pathname`](translate-logical-pathname.md) -- the identity here -- where portable code normalizes before opening. ## Backend support All four backends -- one definition in rontolisp source over primitives every backend has. --- # FILE: references/reference/functions/logior.md # logior `(logior &rest integers)` Variadic bitwise inclusive OR of its integer arguments. With no arguments it returns `0` (the identity). The operation is exact for arbitrarily large integers on every backend. ```lisp (logior 1 2 4 8) ; => 15 ``` --- # FILE: references/reference/functions/lognot.md # lognot `(lognot integer)` Returns the bitwise NOT (ones' complement) of `integer`, equivalent to `(- (+ integer 1))`. The operation is exact for arbitrarily large integers on every backend. ```lisp (lognot 5) ; => -6 ``` --- # FILE: references/reference/functions/logorc1.md # logorc1 `(logorc1 integer1 integer2)` Bitwise inclusive OR of the complement of `integer1` with `integer2`, i.e. `(logior (lognot integer1) integer2)`. The operation is exact for arbitrarily large integers on every backend. ```lisp (logorc1 12 10) ; => -5 ``` --- # FILE: references/reference/functions/logorc2.md # logorc2 `(logorc2 integer1 integer2)` Bitwise inclusive OR of `integer1` with the complement of `integer2`, i.e. `(logior integer1 (lognot integer2))`. The operation is exact for arbitrarily large integers on every backend. ```lisp (logorc2 12 10) ; => -3 ``` --- # FILE: references/reference/functions/logtest.md # logtest `(logtest integer-1 integer-2)` Tests whether `integer-1` and `integer-2` have any one-bits in common: equivalent to `(not (zerop (logand integer-1 integer-2)))`. Returns `t` when they share a set bit and `nil` otherwise. Both arguments may be any magnitude on every backend. ```lisp (logtest 1 3) ; => T ``` ```lisp (logtest 1 2) ; => NIL ``` --- # FILE: references/reference/functions/logxor.md # logxor `(logxor &rest integers)` Variadic bitwise exclusive OR of its integer arguments. With no arguments it returns `0` (the identity). The operation is exact for arbitrarily large integers on every backend. ```lisp (logxor 12 10) ; => 6 ``` --- # FILE: references/reference/functions/lt.md # < `(< &rest numbers)` Returns `t` if its arguments are in strictly increasing order, else `nil`. It is variadic; each adjacent pair is compared by numeric value, mixing integers, ratios and floats. With a single argument it returns `t`. ```lisp (< 1 2 3) ; => T ``` --- # FILE: references/reference/functions/macro-function.md # macro-function `(macro-function symbol &optional environment)` The macro expander of `symbol`, or `nil` when the name is a function, one of the 25 [special operators](special-operator-p.md), or unknown. It answers non-nil for a user macro defined with [`defmacro`](../special-forms/defmacro.md), for every built-in macro (the names `rontolisp:list-macros` reports), and for the Common Lisp macros rontolisp implements as special forms of its own (`defun`, `handler-case`, `dolist`, ...) -- together, every name a caller may not `apply`. The `environment` argument is accepted and ignored: `macrolet` bodies are expanded away before any body runs, so the global answer is the only one there is. ```lisp (defmacro greet (x) `(list :hello ,x)) (list (and (macro-function 'greet) t) (and (macro-function 'when) t) (macro-function 'car) (macro-function 'if)) ; => (T T NIL NIL) ``` On the interpreter the value is the real expander -- a single-step expansion callable as `(funcall expander form environment)`: ```lisp (funcall (macro-function 'when) '(when t 1) nil) ; => (IF T 1 NIL) ``` A COMPILED program has no macro table left (macros are fully expanded before the backends see the program), so there the value is a stub: the predicate above is exact on all four backends, but calling it signals `macro-function: a compiled program cannot expand a macro at run time`. The `setf` place is supported for one shape only: `(setf (macro-function 'new) (macro-function 'existing))` gives an existing `defmacro`-defined macro a second name sharing its expander, so both names expand identically from then on. Anything else -- an arbitrary expander function, or a name that is not a user macro -- signals an error, because there is no macro function object to store. ```lisp (defmacro greet2 (x) `(list :hello ,x)) (setf (macro-function 'hi) (macro-function 'greet2)) (hi "world") ; => (:HELLO "world") ``` Lite: a non-symbol argument answers `nil` where Common Lisp signals a type error. --- # FILE: references/reference/functions/macroexpand-1.md # macroexpand-1 `(macroexpand-1 form)` Expands `form` once when its operator is a user macro (defined with [`defmacro`](../special-forms/defmacro.md)) or a built-in macro (the names `rontolisp:list-macros` reports), and returns the form unchanged otherwise. Only the top-level operator is expanded; subforms are left alone. The second value is Common Lisp's `expanded-p` flag: `(multiple-value-list (macroexpand-1 '(unless c x)))` is `((IF C NIL X) T)`. The environment argument is accepted and ignored (there is no lexical macro environment to consult). On the compilation path only a LITERAL quoted argument expands: the CLI folds the call to its expansion at compile time. A computed argument reaches a compiled program, which has no macro table left, so it answers the form unchanged with `expanded-p` nil — unless the form IS a macro call, which signals `macroexpand-1: a compiled program cannot expand a macro at run time`. That is the same answer [`macro-function`](macro-function.md)'s stub gives there, and it is what makes the usual "expand until it stops expanding" loop terminate on every backend. ```lisp (macroexpand-1 '(unless c x)) ; => (IF C NIL X) ``` ```lisp (defmacro my-when (test &body body) `(if ,test (progn ,@body) nil)) (macroexpand-1 '(my-when (> 2 1) 'a 'b)) ; => (IF (> 2 1) (PROGN (QUOTE A) (QUOTE B)) NIL) ``` A non-macro form is returned as-is: ```lisp (macroexpand-1 '(+ 1 2)) ; => (+ 1 2) ``` --- # FILE: references/reference/functions/macroexpand.md # macroexpand `(macroexpand form)` Repeats [`macroexpand-1`](macroexpand-1.md) on the top-level form until it stops expanding, and returns the result plus the same `expanded-p` second value. Like `macroexpand-1`, only the operator position is expanded — macro calls in subforms stay unexpanded — the environment argument is ignored, and on the compilation path only a literal quoted argument is folded to its expansion (a computed one answers the form unchanged, or signals when the form is a macro call). ```lisp (defmacro inner (x) `(+ ,x 1)) (defmacro outer (x) `(inner ,x)) (macroexpand '(outer 41)) ; => (+ 41 1) ``` Subforms are not walked: ```lisp (macroexpand '(when a (when b c))) ; => (IF A (WHEN B C) NIL) ``` --- # FILE: references/reference/functions/make-array.md # make-array `(make-array dimensions &key initial-element initial-contents element-type fill-pointer adjustable displaced-to displaced-index-offset)` Creates and returns a new array. `dimensions` is an integer for a rank-1 vector, or a non-empty list of integers for an array of any rank. `:initial-element` sets every cell to the given value, defaulting to nil. Elements are stored row-major with O(1) access via `aref`, and arrays are compared by identity (`eq`), so two distinct arrays are never `equal`. `make-array` and `aref` are not first-class function values -- `#'make-array` is unavailable, so call it directly. `:fill-pointer` (rank-1 only) gives the vector a [fill pointer](fill-pointer.md): an integer sets it to that position, `t` to the vector size. The fill pointer is the effective length -- `length` and printing stop at it, while `aref` still reaches the full storage -- and is what [`vector-push`](vector-push.md)/[`vector-pop`](vector-pop.md)/[`vector-push-extend`](vector-push-extend.md) operate on. `:adjustable` marks the array adjustable, reported verbatim by [`adjustable-array-p`](adjustable-array-p.md); an adjustable array is resized in place by [`adjust-array`](adjust-array.md). `:initial-contents` fills the array from a list (row-major; on the compiled backends rank-1 only). `:element-type 'double-float`/`'single-float` (with no fill pointer/adjustability/displacement) selects the packed float representation, and `:element-type 'character` under the same conditions builds a **string** (a rank-1 character array IS a string, the [`make-string`](make-string.md) result shape). `:element-type 'character` **with** `:fill-pointer`/`:adjustable` builds a fill-pointered mutable string on every backend: `vector-push-extend` of characters grows it, `replace` and `(setf (char ...))` write into it in place, and it prints, compares (`string=`/`equal`, `equal` hash keys) and passes `stringp` as a string. `:element-type 'character` with `:initial-contents` copies the contents (a string, a mutable string, or a character list) into a fresh simple string. `:element-type '(unsigned-byte 8)`, `'(unsigned-byte 16)` or `'(unsigned-byte 32)` on a rank-1 array (again with no fill pointer/adjustability/displacement, and written as a literal) selects a packed unsigned-integer vector: a store masks the value to the element width (two's-complement truncation) and a read returns it widened unsigned, a non-integer store is an error, and [`array-element-type`](array-element-type.md) reports the real `(unsigned-byte n)` specifier. [`subseq`](subseq.md) and `copy-seq` of a packed vector stay packed at the same width. A zero-parameter [`deftype`](../macros/deftype.md) name is resolved before any of these checks, so an alias selects exactly what its expansion would. Any other element type is accepted but ignored (element types are not otherwise tracked; [`array-element-type`](array-element-type.md) returns `t` for general arrays). `:displaced-to` builds a view over another array's storage instead of allocating one: element `i` (row-major) of the view reads and writes element `i + offset` of the target, where `:displaced-index-offset` defaults to 0, so changes are visible in both directions. The view has its own dimensions (they may differ in rank from the target's, e.g. a vector view over a matrix row), must fit inside the target, and is inspected with [`array-displacement`](array-displacement.md). A displaced view cannot be combined with `:fill-pointer`, `:adjustable` or `:initial-element`, and cannot itself be adjusted. ```lisp (let ((a (make-array 3 :initial-element 0))) (aref a 0)) ; => 0 (length (make-array 5 :fill-pointer 2 :initial-element 0)) ; => 2 (let* ((base (make-array 4 :initial-element 1)) (view (make-array 2 :displaced-to base :displaced-index-offset 1))) (setf (aref view 0) 9) (aref base 1)) ; => 9 (let ((bytes (make-array 3 :element-type '(unsigned-byte 8)))) (setf (aref bytes 0) 300) ; stores 300 mod 256 (aref bytes 0)) ; => 44 (progn (deftype octet () '(unsigned-byte 8)) (array-element-type (make-array 3 :element-type 'octet))) ; => (UNSIGNED-BYTE 8) ``` --- # FILE: references/reference/functions/make-broadcast-stream.md # make-broadcast-stream `(make-broadcast-stream &rest streams)` An output stream that fans every write out to each component stream, in the order given. With no components it is the discarding sink instead: writes to it are dropped (the CL idiom for a null output stream). ```lisp (let ((a (make-string-output-stream)) (b (make-string-output-stream))) (let ((s (make-broadcast-stream a b))) (format s "sync ~A" 42)) (list (get-output-stream-string a) (get-output-stream-string b))) ; => ("sync 42" "sync 42") ``` ```lisp (let ((s (make-broadcast-stream))) (write-string "discarded" s) :done) ; => :DONE ``` A broadcast stream WITH components is a [Gray stream](../../guides/gray-streams.md), so it takes the whole output protocol: [`format`](../macros/format.md), [`princ`](princ.md), [`prin1`](prin1.md), [`print`](print.md), [`write-string`](write-string.md), `write-char`, [`terpri`](terpri.md), [`fresh-line`](fresh-line.md), [`write-line`](write-line.md), [`force-output`](force-output.md), [`finish-output`](finish-output.md), [`clear-output`](clear-output.md) and [`close`](close.md). `fresh-line` always writes a newline on one: a broadcast stream tracks no column, so it cannot tell whether it is already at the start of a line. --- # FILE: references/reference/functions/make-hash-table.md # make-hash-table `(make-hash-table &key test size)` Creates and returns a new, empty hash table. Keys are compared structurally, as if by `equal`, so list, string, number, symbol and character keys match by value. The `:test` keyword is accepted for familiarity but is informational only -- it does not change the comparison -- and `:size` and other keywords are ignored. Store entries with `(setf (gethash key table) value)` and read them with `gethash`. ```lisp (let ((h (make-hash-table :test 'equal))) (setf (gethash "x" h) 1) (gethash "x" h)) ; => 1 ``` --- # FILE: references/reference/functions/make-list.md # make-list `(make-list size &key initial-element)` Returns a freshly allocated proper list of `size` elements, every one of which is `initial-element` (`nil` by default). A `size` of `0` yields the empty list. The element form is evaluated ONCE and every cell shares that one value, as Common Lisp specifies -- so a mutable element is the same object in every cell. Any other keyword is an error. ```lisp (make-list 3) ; => (NIL NIL NIL) ``` ```lisp (make-list 3 :initial-element 0) ; => (0 0 0) ``` --- # FILE: references/reference/functions/make-load-form-saving-slots.md # make-load-form-saving-slots `(make-load-form-saving-slots object &key slot-names environment)` Lite stub: rontolisp has no fasl dumper, so calling this standard function signals an error. It exists so a library's `make-load-form` methods (which only run when an implementation dumps compiled files) still compile; such call sites are dead at run time. ```console > (make-load-form-saving-slots (make-instance 'point)) Error: make-load-form-saving-slots is not supported (no fasl dumper) ``` --- # FILE: references/reference/functions/make-pathname.md # make-pathname `(make-pathname &key directory name type defaults)` Builds a pathname from its components: `:directory` (Common Lisp's list -- `:absolute` or `:relative` followed by one component per level -- or a directory namestring), `:name` (the file name without its type) and `:type` (the extension without its dot). A directory component is a string or one of the keywords Common Lisp names the special levels by: `:up` / `:back` (`..`), `:wild` (`*`, one level) and `:wild-inferiors` (`**`, any number of levels). `:wild` is also what `:name` and `:type` take for their `*`: ```lisp (namestring (make-pathname :directory (list :absolute "a" :wild-inferiors) :name :wild :type "lisp")) ; => "/a/**/*.lisp" ``` That is the pathspec [`directory`](directory.md) walks a whole subtree with and the from-wildcard [`translate-pathname`](translate-pathname.md) rewrites against. `:host`, `:device`, `:version` and `:case` are accepted and dropped, as is any other keyword: a namestring models no such component, and a portability layer's call still works. `:defaults` supplies every component the call did NOT, **component-wise -- this is not a merge**. A supplied component REPLACES the defaults' one instead of combining with it, and an explicitly supplied `nil` means "no component" rather than "take the default". That is Common Lisp's rule, so a supplied `:directory` does not nest under the defaults' directory: | Call | Result | |------|--------| | `(make-pathname :name "b" :defaults "d/a.sql")` | `#P"d/b.sql"` | | `(make-pathname :name "b" :type nil :defaults "d/a.sql")` | `#P"d/b"` | | `(make-pathname :type "txt" :defaults "d/a.sql")` | `#P"d/a.txt"` | | `(make-pathname :directory (list :relative "m") :name "b" :defaults "d/a.sql")` | `#P"m/b.sql"` | | `(make-pathname :directory (list :absolute "u" "s") :name "b" :type "c")` | `#P"/u/s/b.c"` | Naming a sibling file is what this is for: [`pathname-name`](pathname-name.md) and [`pathname-type`](pathname-type.md) take a namestring apart by the same rule this puts one together by. To combine two paths instead of replacing components, use [`merge-pathnames`](merge-pathnames.md). ```lisp (make-pathname :name "20260101.down" :defaults "db/20260101.up.sql") ; => #P"db/20260101.down.sql" ``` ## Backend support All four backends, as a real run-time function -- one definition in rontolisp source. On the compiled backends a call whose keywords and values are all literals is additionally folded to a literal pathname while the program is being built (which is what lets an [`asdf:system-relative-pathname`](asdf-system-relative-pathname.md) result become a constant in the artifact); every other call -- a computed `:defaults` or `:name`, say -- runs the function. Both renderings implement the same rule. --- # FILE: references/reference/functions/make-random-state.md # make-random-state `(make-random-state &optional state)` Always returns `nil`: rontolisp has no random-state objects. [`random`](random.md) accepts (and ignores) an optional random-state argument and draws from the backend's own entropy source, so the common seeding idiom — store `(make-random-state t)` in a variable and pass it back to `random` — works unchanged (uuid's `*uuid-random-state*` is the driving consumer). The argument (`nil`, `t`, or a state) is accepted and ignored. ```lisp (make-random-state t) ; => NIL ``` --- # FILE: references/reference/functions/make-sequence.md # make-sequence `(make-sequence result-type size &key initial-element)` Creates a sequence of the given type and size. The result type must be a literal quoted specifier: a string type (`string`, `simple-string`, `base-string`, `simple-base-string`) builds a string like [`make-string`](make-string.md), `list` builds a list like [`make-list`](make-list.md), and a vector type (`vector`, `simple-vector`) builds an array like [`make-array`](make-array.md). A non-literal result type is an error, and the keyword arguments are forwarded to the underlying constructor (so `:initial-element` follows its support there). ```lisp (length (make-sequence 'simple-string 5)) ; => 5 ``` ```lisp (make-sequence 'list 3) ; => (NIL NIL NIL) ``` --- # FILE: references/reference/functions/make-string-input-stream.md # make-string-input-stream `(make-string-input-stream string &optional start end)` Returns a character input stream that reads from `string`, so `read-char`, `read-line`, `peek-char` and `read` consume it like any other input stream. It is the explicit form of the stream `with-input-from-string` binds, and is what you need when the stream has to outlive one expression -- when it is stored, or handed to a function that takes a stream. `start` and `end` bound the portion read, in characters. ```lisp (let ((s (make-string-input-stream "one two"))) (list (read-line s) (read-line s) (read-line s nil :eof))) ; => ("one" "two" :EOF) ``` --- # FILE: references/reference/functions/make-string-output-stream.md # make-string-output-stream `(make-string-output-stream)` Returns a fresh string output stream: an output stream that accumulates everything written to it, to be read back with `get-output-stream-string`. It is the explicit form of what `with-output-to-string` builds, and is what a `defstruct` slot `:initform` needs when the stream has to outlive one expression. CL's `:element-type` keyword argument is accepted and ignored -- every rontolisp stream is a character stream. ```lisp (let ((s (make-string-output-stream))) (write-string "ab" s) (princ 12 s) (get-output-stream-string s)) ; => "ab12" ``` --- # FILE: references/reference/functions/make-string.md # make-string `(make-string size &key initial-element element-type)` Returns a fresh string of `size` characters, each equal to `:initial-element` (default a space; the standard leaves the fill character implementation-defined). `:element-type` is accepted and ignored -- rontolisp has a single string representation. The result is a MUTABLE buffer on every backend: [`replace`](replace.md), `(setf (char ...))` and `(setf (subseq ...))` write into it in place, and the write is visible through every reference to it. Available on all backends except `--no-gc`. ```lisp (make-string 3 :initial-element #\x) ; => "xxx" ``` ```lisp (let ((buf (make-string 5))) (replace buf "ab") (replace buf "cde" :start1 2) buf) ; => "abcde" ``` --- # FILE: references/reference/functions/make-symbol.md # make-symbol `(make-symbol string)` Returns a fresh uninterned symbol named `#:` — the same `#:` convention [`gensym`](gensym.md) uses. rontolisp has no intern table (symbols compare by name), so "uninterned" is represented by the `#:` name prefix: the result is never `eq` to a plain symbol with the same spelling, but two `make-symbol` calls with the same string DO yield `eq` symbols (a deviation from Common Lisp, where every `make-symbol` result is a distinct object). For guaranteed-unique temporaries use `gensym`, which appends a counter. ```lisp (make-symbol "temp") ; => #:temp ``` ```lisp (eq (make-symbol "foo") 'foo) ; => NIL ``` ```lisp (symbolp (make-symbol "temp")) ; => T ``` --- # FILE: references/reference/functions/make-synonym-stream.md # make-synonym-stream `(make-synonym-stream symbol)` Returns a stream that forwards every operation to the stream the *special variable* `symbol` holds **at the time of that operation** -- so rebinding the variable afterwards redirects a synonym stream that was constructed earlier. The usual shape is a `defvar` whose default output follows whatever the standard stream currently is. The result is a stream *value*, not a designator: it is true, [`streamp`](streamp.md) / [`input-stream-p`](input-stream-p.md) / [`output-stream-p`](output-stream-p.md) answer `t` for it, [`synonym-stream-symbol`](synonym-stream-symbol.md) reads the symbol back, and [`close`](close.md) closes the synonym (which is nothing to do) and answers `t`. A Gray stream may sit on either side of it: a synonym stream handed to a [Gray](../../guides/gray-streams.md) output stream is written through, and a synonym whose variable holds a Gray stream reaches it. ```lisp (defvar *report-output* (make-synonym-stream '*standard-output*)) (write-string "hello" *report-output*) ; => "hello" ``` --- # FILE: references/reference/functions/map-into.md # map-into `(map-into result-sequence function &rest sequences)` Applies `function` to successive elements of the given `sequences` and stores the results destructively into `result-sequence` -- which may be a list or a vector -- then returns `result-sequence`. The function is called with one element from each sequence in parallel, and iteration stops at the end of the shortest sequence, the result sequence included; any remaining elements of `result-sequence` are left unchanged. With no source sequences the function is called with no arguments to fill each element. A result with a fill pointer is filled up to that fill pointer. ```lisp (map-into (list 0 0 0 0) #'+ '(1 2 3) '(10 20 30 40)) ; => (11 22 33 0) ``` ```lisp (map-into (make-array 3) #'* #(2 3 4) #(5 6 7)) ; => #(10 18 28) ``` --- # FILE: references/reference/functions/map.md # map `(map result-type function &rest sequences)` Applies `function` to successive elements of the given `sequences` -- each of which may be a list or a string -- and builds a result of the requested type. When several sequences are given, the function is called with one element from each and iteration stops at the end of the shortest sequence. `result-type` must be written as a literal designator (resolved statically, like `concatenate`): `'list` collects the results into a list, `'string` builds a string from the character results, and `nil` calls the function only for its side effects and returns nil. ```lisp (map 'list #'+ '(1 2 3) '(10 20 30)) ; => (11 22 33) ``` ```lisp (map 'string #'char-upcase "abc") ; => "ABC" ``` --- # FILE: references/reference/functions/mapc.md # mapc `(mapc function list &rest more-lists)` Applies `function` to successive elements of the given lists for its side effects, discarding the results, and returns the first list. Use it instead of `mapcar` when you only care about the effect (such as printing). With a single list, the function receives one element per call. When several lists are supplied, the function is called with one element from each list in parallel, and iteration stops at the end of the shortest list. Each argument must be a list (`nil`, the empty list, is accepted); passing a non-list such as a string signals an error rather than silently doing nothing. Use `map` to map over a string or vector. ```lisp (mapc #'print '(1 2 3)) ``` ``` 1 2 3 ``` ```lisp (mapc (lambda (a b) (print (list a b))) '(1 2) '(3 4)) ``` ``` (1 3) (2 4) ``` --- # FILE: references/reference/functions/mapcan.md # mapcan `(mapcan function list &rest more-lists)` Applies `function` to successive elements of the given lists -- each call should return a list -- and concatenates the results into a single list. rontolisp joins the pieces with non-destructive `append` (it does not splice with `nconc` as standard Common Lisp does). It is a convenient way to map and filter at once: return `nil` for elements to drop. When several lists are supplied, the function is called with one element from each list in parallel, and iteration stops at the end of the shortest list. Each argument must be a list (`nil`, the empty list, is accepted); passing a non-list such as a string signals an error rather than silently returning `nil`. Use `map` to map over a string or vector. ```lisp (mapcan (lambda (x) (list x x)) '(1 2)) ; => (1 1 2 2) ``` ```lisp (mapcan #'list '(1 2) '(3 4)) ; => (1 3 2 4) ``` --- # FILE: references/reference/functions/mapcar.md # mapcar `(mapcar function list &rest more-lists)` Applies `function` to successive elements of the given lists and returns a new list of the results. With a single list, the function receives one element per call. When several lists are supplied, the function is called with one element from each list in parallel, and iteration stops at the end of the shortest list. Each argument must be a list (`nil`, the empty list, is accepted); passing a non-list such as a string signals an error rather than silently returning `nil`. Use `map` to map over a string or vector. ```lisp (mapcar #'car '((1 2) (3 4))) ; => (1 3) ``` ```lisp (mapcar #'+ '(1 2 3 4) '(10 20 30 40)) ; => (11 22 33 44) ``` --- # FILE: references/reference/functions/mapcon.md # mapcon `(mapcon function list &rest more-lists)` Like `maplist`, `function` is applied to successive tails of the given lists, but the resulting lists are concatenated into one (the tail-walking counterpart of `mapcan`). The pieces are joined with `append`. When several lists are supplied, the function is called with one tail from each list in parallel, and iteration stops at the end of the shortest list. Each argument must be a list (`nil`, the empty list, is accepted); passing a non-list such as a string signals an error rather than silently returning `nil`. Use `map` to map over a string or vector. ```lisp (mapcon (lambda (x) (list (car x))) '(1 2 3)) ; => (1 2 3) ``` ```lisp (mapcon #'list '(1 2) '(3 4)) ; => ((1 2) (3 4) (2) (4)) ``` --- # FILE: references/reference/functions/maphash.md # maphash `(maphash function table)` Calls `function` once for each entry in `table`, passing the key and value as its two arguments, for side effects only; the results are discarded and `maphash` returns nil. Iteration order is unspecified and may differ across backends, so portable code should not rely on it. ```lisp (let ((h (make-hash-table))) (setf (gethash 'a h) 1) (maphash (lambda (k v) (print (list k v))) h)) ``` ``` (A 1) ``` --- # FILE: references/reference/functions/mapl.md # mapl `(mapl function list &rest more-lists)` Like [`maplist`](maplist.md), but `function` is applied to the successive cdrs (tails) of the given lists for its side effects only, and the first list is returned rather than a list of the results. When several lists are supplied, the function is called with one tail from each list in parallel, and iteration stops at the end of the shortest list. Each argument must be a list (`nil`, the empty list, is accepted); passing a non-list such as a string signals an error. ```lisp (mapl #'identity '(1 2 3)) ; => (1 2 3) ``` ```lisp (mapl (lambda (a b) (print (list a b))) '(1 2) '(3 4)) ``` ``` ((1 2) (3 4)) ((2) (4)) ``` --- # FILE: references/reference/functions/maplist.md # maplist `(maplist function list &rest more-lists)` Like `mapcar`, but `function` is applied to successive cdrs (tails) of the given lists rather than to their elements: first the whole list, then its rest, and so on down to the last single-element tail. Returns a new list of the results. When several lists are supplied, the function is called with one tail from each list in parallel, and iteration stops at the end of the shortest list. Each argument must be a list (`nil`, the empty list, is accepted); passing a non-list such as a string signals an error rather than silently returning `nil`. Use `map` to map over a string or vector. ```lisp (maplist #'identity '(1 2 3)) ; => ((1 2 3) (2 3) (3)) ``` ```lisp (maplist #'list '(1 2) '(3 4)) ; => (((1 2) (3 4)) ((2) (4))) ``` --- # FILE: references/reference/functions/mask-field.md # mask-field `(mask-field bytespec integer)` The bits of `integer` selected by the [`byte`](byte.md) specifier, left in their original position (unlike [`ldb`](ldb.md), which shifts them down to bit 0). ```lisp (mask-field (byte 4 4) 255) ; => 240 ``` --- # FILE: references/reference/functions/max.md # max `(max &rest numbers)` Returns the largest of its arguments. It is variadic and requires at least one argument. Comparison is by numeric value, mixing integers, ratios and floats freely. ```lisp (max 5 2 8 1) ; => 8 ``` --- # FILE: references/reference/functions/member-if.md # member-if `(member-if predicate list)` Searches `list` for the first element that satisfies `predicate` and returns the sublist (tail) starting at that element, or `nil` if none does. The returned tail shares structure with the original list. Use `member` to search by item value instead of by predicate. ```lisp (member-if #'oddp '(2 4 5 6)) ; => (5 6) ``` --- # FILE: references/reference/functions/member.md # member `(member item list &key test key)` Searches `list` for the first element matching `item` and returns the sublist (tail) starting at that element, or `nil` if none matches. By default the comparison is `eql`; the optional `:test` keyword takes a function designator to use a different comparison, and the optional `:key` keyword takes a selector function applied to each element before the comparison. The result shares structure with the original list rather than being a copy. ```lisp (member 2 '(1 2 3)) ; => (2 3) ``` ```lisp (member '(a d) '((a b) (a d)) :test 'equal) ; => ((A D)) ``` ```lisp (member 3 '((1 2) (3 4) (5 6)) :key #'car) ; => ((3 4) (5 6)) ``` --- # FILE: references/reference/functions/merge-pathnames.md # merge-pathnames `(merge-pathnames pathname &optional defaults)` Fills the gaps in `pathname` from `defaults` and returns the merged pathname. Both arguments take either spelling (a pathname or a namestring), and the rule works on the two parts a namestring has: the *directory* (everything through the last `/`) and the *file* (what follows it). The directory of `pathname` wins when it is absolute, is appended to `defaults`' directory when it is relative, and is taken from `defaults` when `pathname` has none; the file of `pathname` wins unless it is empty. Omitting `defaults` merges against the working directory, which leaves `pathname` unchanged. This is how a library names a file relative to a directory it computed earlier -- a data file next to its own sources, say. `uiop:merge-pathnames*` is the ASDF/UIOP spelling of the same merge. ```lisp (merge-pathnames "zoneinfo/" "/opt/local-time/") ; => #P"/opt/local-time/zoneinfo/" ``` ## Backend support Works on all four backends: one definition in rontolisp source, spliced into the program when it is referenced. --- # FILE: references/reference/functions/merge.md # merge `(merge result-type sequence-1 sequence-2 predicate &key key)` Merges two already-sorted sequences into one sorted sequence of `result-type`. `predicate` is the same ordering predicate `sort` takes, and `:key` selects what it sees. The merge is stable: when neither element precedes the other, the one from `sequence-1` comes first. `result-type` may be a run-time value, but it is limited to the sequence families `coerce` builds -- `list`, `vector` and `string`. Unlike Common Lisp's, this `merge` does not destroy its arguments. ```lisp (merge 'list (list 1 3 5) (list 2 4 6) #'<) ; => (1 2 3 4 5 6) ``` ```lisp (merge 'string "ac" "bd" #'char<) ; => "abcd" ``` ```lisp (merge 'list (list '(1 a)) (list '(1 b) '(2 c)) #'< :key #'car) ; => ((1 A) (1 B) (2 C)) ``` --- # FILE: references/reference/functions/min.md # min `(min &rest numbers)` Returns the smallest of its arguments. It is variadic and requires at least one argument. Comparison is by numeric value, mixing integers, ratios and floats freely. ```lisp (min 5 2 8 1) ; => 1 ``` --- # FILE: references/reference/functions/minus.md # - `(- number &rest numbers)` With one argument, returns its negation. With several, subtracts the rest from the first, left to right. The result type follows the usual numeric contagion: integers stay integers, but any float argument makes the result a float, and ratios are kept exact. Integer results promote to big integers on overflow, on every backend. ```lisp (- 10 3) ; => 7 ``` ```lisp (- 5) ; => -5 ``` --- # FILE: references/reference/functions/minusp.md # minusp `(minusp number)` Returns `t` if `number` is strictly less than zero, else `nil`. It works for any real numeric type. ```lisp (minusp -3) ; => T ``` --- # FILE: references/reference/functions/mismatch.md # mismatch `(mismatch sequence1 sequence2 &key test key start1 end1 start2 end2 from-end)` Returns the index **into `sequence1`** of the first position where the two (bounded) sequences differ, or `nil` when they match element for element. `:test` compares elements (`eql` by default) and `:key` selects the compared value; `:start1`/`:end1`/`:start2`/`:end2` bound each sequence. Lite: `:from-end` is accepted but the scan still runs forward, so the returned index is the forward one. ```lisp (list (mismatch "apple" "apricot") (mismatch '(1 2 3) '(1 2 3))) ; => (2 NIL) ``` --- # FILE: references/reference/functions/mod.md # mod `(mod number divisor)` Returns the remainder of `number` divided by `divisor` using floored division, so the result always takes the sign of the divisor. This makes it the companion of `floor`. Use `rem` instead when you want the result to follow the sign of the dividend. ```lisp (mod 10 3) ; => 1 ``` ```lisp (mod -13 4) ; => 3 ``` --- # FILE: references/reference/functions/muffle-warning.md # muffle-warning `(muffle-warning [condition])` Invokes the `muffle-warning` restart that every [`warn`](../macros/warn.md) establishes in a restart-system program, aborting the pending warning **before it is printed** — `warn` then returns `nil` silently. Meant to be called from a [`handler-bind`](../macros/handler-bind.md) handler on `warning`. Signals an error when no `muffle-warning` restart is active (i.e. outside a `warn`). ```lisp (handler-bind ((warning (lambda (w) (muffle-warning)))) (list :done (warn "nothing to see"))) ; => (:DONE NIL) ``` --- # FILE: references/reference/functions/mul.md # * `(* &rest numbers)` Returns the product of its arguments, or `1` with no arguments. The result is an integer when all arguments are integers, exact when ratios are involved, and a float if any argument is a float. Integer results promote to big integers on overflow, on every backend. ```lisp (* 3 4) ; => 12 ``` ```lisp (* 2.0 3.0) ; => 6.0 ``` --- # FILE: references/reference/functions/namestring-components.md # file-namestring directory-namestring host-namestring `(file-namestring pathname)` -- `(directory-namestring pathname)` -- `(host-namestring pathname)` The string-valued components of a namestring. `file-namestring` is the name-and-type part -- everything after the last `/`, or the whole namestring when there is none -- and `directory-namestring` is what comes before it, up to and including that `/`. The two are exact complements: concatenating them always gives [`namestring`](namestring.md) back. `host-namestring` is always `""`, because a rontolisp namestring carries no host syntax; [`pathname-host`](pathname-host.md) is the `nil`-answering spelling of the same absence, and Common Lisp requires a string here. All three accept either spelling of a pathname designator and signal on anything else, exactly like `namestring`. ```lisp (list (file-namestring #P"/a/b/c.txt") (directory-namestring #P"/a/b/c.txt") (host-namestring #P"/a/b/c.txt")) ; => ("c.txt" "/a/b/" "") ``` A namestring that names a directory has no file half, and one without a `/` has no directory half: ```lisp (list (file-namestring "/a/b/") (directory-namestring "a.txt")) ; => ("" "") ``` The leading dot of a dotfile belongs to the name, the same rule [`pathname-name`](pathname-name.md) follows: `(file-namestring "/a/.bashrc")` is `".bashrc"`. ## Backend support All four backends -- one definition in rontolisp source, spliced into the program when it is referenced. --- # FILE: references/reference/functions/namestring.md # namestring `(namestring pathname)` The namestring of a pathname designator: a pathname value unwraps to the namestring it carries, a string is already one and passes through, and anything else signals. Portable code calls it where a pathname object has to be turned into a string before printing, concatenating or handing it outside Lisp. `uiop:namestring` names this same function -- real UIOP re-exports Common Lisp's, and so does this -- and so does `uiop:native-namestring` (a rontolisp namestring already is the host spelling). ```lisp (namestring #P"/tmp/data.json") ; => "/tmp/data.json" ``` `(namestring "/tmp/data.json")` is the string itself. ## Backend support All four backends -- one definition in rontolisp source, spliced into the program when it is referenced. --- # FILE: references/reference/functions/nconc.md # nconc `(nconc &rest lists)` Destructively concatenates its list arguments by setting the last cdr of each non-empty list to point at the following argument, then returns the first non-`nil` argument. No new cons cells are allocated, so the argument lists are modified in place. `(nconc)` returns `nil`, `(nconc x)` returns `x`, and `nil` arguments are skipped. The last argument may be any object (it is spliced onto the tail of the preceding list without being copied). It is also never traversed, so `(nconc x x)` — the usual way to build a circular list — links `x` onto itself and returns instead of chasing the cycle it just created. ```lisp (nconc (list 1 2) (list 3 4) (list 5)) ; => (1 2 3 4 5) ``` --- # FILE: references/reference/functions/ne.md # /= `(/= number...)` Returns `t` when every argument is numerically different from every other (pairwise, as in Common Lisp), `nil` otherwise. Each argument is evaluated once. ```lisp (/= 1 2) ; => T ``` ```lisp (/= 1 2 1) ; => NIL ``` --- # FILE: references/reference/functions/not.md # not `(not object)` Logical negation: returns `t` if `object` is `nil` (false), otherwise `nil`. Because `nil` is the only false value, any non-nil argument yields `nil`. It is identical in behavior to `null`; choose `not` for boolean contexts and `null` for empty-list tests. Works in all three backends. ```lisp (not nil) ; => T ``` ```lisp (not 42) ; => NIL ``` --- # FILE: references/reference/functions/notany.md # notany `(notany predicate &rest sequences)` Returns `t` if `predicate` is nil for every element (tuple) of the sequences, and `nil` if any satisfies it -- the complement of `some`. Each sequence may be a list or a string (whose elements are characters). With more than one sequence the predicate receives one argument per sequence and the walk stops as soon as the shortest one runs out. An empty sequence yields `t`. ```lisp (notany #'evenp '(1 3 5)) ; => T ``` ```lisp (notany #'digit-char-p "abc") ; => T ``` ```lisp (notany #'> '(1 2) '(3 4)) ; => T ``` --- # FILE: references/reference/functions/notevery.md # notevery `(notevery predicate &rest sequences)` Returns `t` if `predicate` is nil for at least one element (tuple) of the sequences, and `nil` if every one satisfies it -- the complement of `every`. Each sequence may be a list or a string (whose elements are characters). With more than one sequence the predicate receives one argument per sequence and the walk stops as soon as the shortest one runs out. An empty sequence yields `nil`. ```lisp (notevery #'evenp '(2 4 5)) ; => T ``` ```lisp (notevery #'digit-char-p "12a") ; => T ``` ```lisp (notevery #'< '(1 2) '(3 0)) ; => T ``` --- # FILE: references/reference/functions/nreconc.md # nreconc `(nreconc list tail)` The destructive version of `revappend`: it reverses `list` in place and attaches `tail`, reusing `list`'s cons cells (it expands to `(nconc (nreverse list) tail)`). Because the input list is rewired, pass a fresh list and use the return value rather than the original variable. ```lisp (nreconc (list 1 2 3) '(4 5)) ; => (3 2 1 4 5) ``` --- # FILE: references/reference/functions/nreverse.md # nreverse `(nreverse sequence)` Reverses `sequence` destructively by rewiring each cons cell's `cdr`, then returns the new head. Because the original cells are reused and the head changes, you must use the return value -- the variable you passed in no longer points at the full reversed list. When you need to keep the original, use `reverse` instead. ```lisp (nreverse (list 1 2 3)) ; => (3 2 1) ``` --- # FILE: references/reference/functions/nstring-case.md # nstring-upcase nstring-downcase nstring-capitalize `(nstring-upcase string)` -- `(nstring-downcase string)` -- `(nstring-capitalize string)` The destructive spellings of [`string-upcase`](string-upcase.md), [`string-downcase`](string-downcase.md) and [`string-capitalize`](string-capitalize.md): the folded characters are written back into the argument, and the string is returned. The fold is the non-destructive sibling's, so the returned value is the same on every backend. The write is real for a **mutable character vector** -- what [`make-string`](make-string.md) and `(make-array n :element-type 'character)` build: the very same object comes back, and a caller holding its own reference sees the change. ```lisp (let ((s (make-string 3 :initial-element #\a))) (list (eq s (nstring-upcase s)) s)) ; => (T "AAA") ``` For an **immutable** string the compiled backends rebuild rather than write, which is the deviation every indexed write (`(setf (aref s i) c)`, [`replace`](replace.md), [`fill`](fill.md)) has there -- so the caller's own reference is left alone while the interpreter's is folded. Portable code uses the returned value, which is correct on all four backends. ```lisp (nstring-upcase (copy-seq "hello world")) ; => "HELLO WORLD" ``` The whole string is folded: like their non-destructive siblings these take no `:start` / `:end`. Each is a first-class function value, so `#'nstring-upcase` can be passed to `funcall`, `mapcar` or `intern`. ## Backend support All four backends -- one definition in rontolisp source, spliced into the program when it is referenced. --- # FILE: references/reference/functions/nsubstitute-if-not.md # nsubstitute-if-not `(nsubstitute-if-not new predicate list &key key)` The destructive variant of [`substitute-if-not`](substitute-if-not.md): rewrites the `car` of every cons whose element the predicate *rejects* and returns the (possibly mutated) original list. Lists only; see [`nsubstitute-if`](nsubstitute-if.md) for the shared cons-reuse semantics. ```lisp (nsubstitute-if-not 0 #'oddp (list 1 2 3 4 5)) ; => (1 0 3 0 5) ``` --- # FILE: references/reference/functions/nsubstitute-if.md # nsubstitute-if `(nsubstitute-if new predicate list &key key)` The destructive variant of [`substitute-if`](substitute-if.md): rewrites the `car` of every cons whose element satisfies the predicate and returns the (possibly mutated) original list. The cons cells are reused, so any other reference to the list observes the change. Lists only — unlike `substitute-if` it does not rebuild strings or vectors. ```lisp (nsubstitute-if 0 #'oddp (list 1 2 3 4 5)) ; => (0 2 0 4 0) ``` ```lisp (let* ((a (list 1 2 3)) (b a)) (nsubstitute-if 0 #'oddp a) b) ; => (0 2 0) ``` --- # FILE: references/reference/functions/nsubstitute.md # nsubstitute `(nsubstitute new old list &key test key)` The destructive counterpart of `substitute`: rewrites the cars of `list` in place, replacing every element matching `old` with `new`. The comparison is `eql` by default; the optional `:test` keyword takes a function designator to use a different comparison, and the optional `:key` keyword takes a selector function applied to each element before the comparison. The list structure is reused, so the modification is visible through the original variable. ```lisp (nsubstitute 0 2 '(1 2 3 2)) ; => (1 0 3 0) ``` ```lisp (nsubstitute 'x 2 (list '(1) '(2)) :key #'car) ; => ((1) X) ``` --- # FILE: references/reference/functions/nth.md # nth `(nth n list)` Returns the element at zero-based index `n` of `list`. If `n` is greater than or equal to the list length the result is `nil`. Note the argument order: the index comes first, then the list. ```lisp (nth 2 '(a b c d)) ; => C ``` ```lisp (nth 10 '(a b c)) ; => NIL ``` --- # FILE: references/reference/functions/nthcdr.md # nthcdr `(nthcdr n list)` Applies `cdr` to `list` `n` times and returns the resulting tail -- i.e. the sublist with the first `n` elements skipped. If `n` reaches or exceeds the list length the result is `nil`. `(nthcdr 0 list)` returns the list unchanged. ```lisp (nthcdr 2 '(a b c d)) ; => (C D) ``` --- # FILE: references/reference/functions/null.md # null `(null object)` Returns `t` if `object` is the empty list `nil`, otherwise `nil`. Since `nil` is the only false value, this also serves as the logical-false test. It is identical in behavior to `not`. Works in all three backends. ```lisp (null nil) ; => T ``` ```lisp (null '(1 2)) ; => NIL ``` --- # FILE: references/reference/functions/numberp.md # numberp `(numberp object)` Returns `t` if `object` is a number -- an integer, a float, or a ratio -- otherwise `nil`. Works in all three backends. ```lisp (numberp 42) ; => T ``` ```lisp (numberp "42") ; => NIL ``` --- # FILE: references/reference/functions/numeq.md # = `(= &rest numbers)` Returns `t` if all of its arguments are numerically equal, else `nil`. It is variadic and compares by numeric value across types, so an integer and an equal float or ratio compare equal (unlike `eql`). With a single argument it returns `t`. ```lisp (= 3 3 3) ; => T ``` --- # FILE: references/reference/functions/numerator.md # numerator `(numerator rational)` Returns the numerator of a rational number in its reduced form. For an integer, which is its own numerator, it returns the integer unchanged. ```lisp (numerator 3/4) ; => 3 ``` ```lisp (numerator 5) ; => 5 ``` --- # FILE: references/reference/functions/oddp.md # oddp `(oddp integer)` Returns `t` if `integer` is odd, else `nil`. The argument must be an integer. ```lisp (oddp 3) ; => T ``` --- # FILE: references/reference/functions/open-stream-p.md # open-stream-p `(open-stream-p stream)` Returns `t` while the stream handle names an open stream and `nil` after it has been closed -- the question the close-if-open idiom asks so it neither double-closes nor leaks. The interpreter and the JVM answer from the stream table (a [`close`](close.md) removes the entry) and additionally report `nil` for a socket closed from the other side. On the WASM `--component` backend a socket answers exactly the same way (the socket table is Lisp state there). Any other stream designator answers `t` when it is non-nil: Preview 1 keeps no per-descriptor open/closed record. ```lisp (with-input-from-string (s "x") (open-stream-p s)) ; => T ``` The close case touches a file, so it is shown statically: ```console (let ((s (open "f.txt" :direction :input))) (open-stream-p s) ; => T (close s) (open-stream-p s)) ; => NIL ``` --- # FILE: references/reference/functions/open.md # open `(open filename &optional direction element-type)` Opens a file and returns a stream. The optional direction is `:input` (the default -- open for reading) or `:output` (create or truncate, open for writing). The keyword-argument shape `(open filename :direction :output :if-exists :append)` opens for writing WITHOUT truncating, so every write lands at the end of an existing file; the other `:if-exists`/`:if-does-not-exist`/`:external-format` values are accepted only where they name the native behavior already (`:supersede`, `:create`/`:error`, `:utf-8`/`:default`). The optional element type selects the stream kind: `'character` (the default) opens a text stream for `read`/`read-line`/`write-line`, and `'(unsigned-byte 8)` -- or the unsized `'unsigned-byte` -- opens a binary stream for `read-byte`/`write-byte`/`read-sequence`/`write-sequence`. In the keyword shape an option value may be COMPUTED (`(open path :direction dir :element-type type)`): it is read when the call runs and dispatched onto the matching literal shape, which is what lets a portable wrapper take its options as arguments. A literal value is resolved at compile time exactly as before, and a value outside the supported set signals an error when the call runs. The returned stream is an opaque handle (an index into a stream table on the interpreter/JVM, the WASI file descriptor on WASM) valid only within the producing run, and should be passed to the matching read/write functions and then `close`. On WASM the path is resolved against the preopened directories -- a relative path against the first one, an absolute path against the preopened directory whose name is its longest prefix -- so run with `--dir`. Prefer `with-open-file`, which closes the stream automatically. ```console (let ((s (open "data.txt"))) (print (read-line s)) (close s)) ``` This opens `data.txt` for input, reads its first line, and closes the stream. Passing `:output` instead would create or truncate the file for writing; `(open "data.bin" :input '(unsigned-byte 8))` opens the same kind of handle in binary mode. --- # FILE: references/reference/functions/output-stream-p.md # output-stream-p `(output-stream-p stream)` Lite: `t` for any stream handle (every rontolisp stream answers both directions) and for the standard-output designator `t`, nil otherwise. A [Gray stream](../../guides/gray-streams.md) instance is exact instead: it answers `t` only when its class descends from `rontolisp:fundamental-output-stream`. ```lisp (with-output-to-string (s) (princ (output-stream-p s) s)) ; => "T" ``` --- # FILE: references/reference/functions/package-name.md # package-name `(package-name package-designator)` The name of the designated package as a string. The designator is resolved through [`find-package`](find-package.md) first, so a nickname answers the canonical name; an unknown designator signals an error. A "package" value in rontolisp is its canonical upcased name as a keyword, so the name string is that keyword's string. ```lisp (package-name (find-package :cl-user)) ; => "CL-USER" ``` --- # FILE: references/reference/functions/package-shadowing-symbols.md # package-shadowing-symbols `(package-shadowing-symbols package)` Always `nil`: rontolisp has no symbol shadowing. The [`defpackage`](../special-forms/defpackage.md) `:shadow` clause records names for *resolution* and mints no shadowing symbol, and the runtime `shadow` / `shadowing-import` do not exist. The designator is still validated, so an unknown package signals like [`package-name`](package-name.md). ```lisp (package-shadowing-symbols :cl-user) ; => NIL ``` --- # FILE: references/reference/functions/package-use-list.md # package-use-list `(package-use-list package)` The packages `package` uses, as the keywords [`find-package`](find-package.md) answers (rontolisp has no package objects). The argument is any package designator — a keyword, a string, a symbol, or a package value; an unknown one signals. [`package-used-by-list`](package-used-by-list.md) is the inverse. The interpreter reads its live registry. The compiled backends have no registry at run time and answer from a table baked in at compile time, so a package a compiled program creates later is invisible there. ```lisp (defpackage #:uses-cl (:use #:cl)) (package-use-list '#:uses-cl) ; => (:CL) ``` --- # FILE: references/reference/functions/package-used-by-list.md # package-used-by-list `(package-used-by-list package)` Every package whose use list names `package`, as the keywords [`find-package`](find-package.md) answers — the inverse of [`package-use-list`](package-use-list.md). An unknown designator signals. The interpreter reads its live registry; the compiled backends answer from the same compile-time table `package-use-list` uses. ```lisp (defpackage #:provider (:use #:cl) (:export #:thing)) (defpackage #:consumer (:use #:cl #:provider)) (package-used-by-list '#:provider) ; => (:CONSUMER) ``` --- # FILE: references/reference/functions/pairlis.md # pairlis `(pairlis keys data &optional alist)` Pairs up a list of keys and a list of values into an association list, preserving key order, and appends the optional existing `alist` as the tail. Pairing stops at the end of the shorter list. ```lisp (pairlis '(a b) '(1 2)) ; => ((A . 1) (B . 2)) ``` ```lisp (pairlis '(a b) '(1 2) '((c . 3))) ; => ((A . 1) (B . 2) (C . 3)) ``` --- # FILE: references/reference/functions/parse-integer.md # parse-integer `(parse-integer string &key start end radix junk-allowed)` Parses an integer from a string, skipping surrounding whitespace. `:start`/`:end` bound the parsed region, `:radix` selects the base (default 10) and `:junk-allowed`, when non-nil, stops at the first non-digit and returns the integer parsed so far (or `nil` if none). Without `:junk-allowed`, any trailing non-whitespace character is an error. The second return value is the position where parsing stopped, ready to feed back into `:start`. The full keyword set and both values work identically on every backend; usable as a first-class value (`#'parse-integer`). ```lisp (parse-integer "ff" :radix 16) ; => 255 ``` ```lisp (multiple-value-bind (n pos) (parse-integer "42x" :junk-allowed t) (list n pos)) ; => (42 2) ``` `(parse-integer "42")` returns `42`, and `(parse-integer "x9x" :start 1 :end 2)` returns `9` by parsing only the bounded region. --- # FILE: references/reference/functions/parse-namestring.md # parse-namestring `(parse-namestring thing &optional host defaults)` Parses a namestring into a pathname, returning it and the position parsing stopped at as a second value. Lite: a rontolisp namestring has no host component to parse against, so the whole string is the namestring, the second value is its length, and `host`/`defaults` are accepted and ignored. A pathname argument answers itself. ```lisp (parse-namestring "d/a.txt") ; => #P"d/a.txt" ``` `(multiple-value-list (parse-namestring "d/a.txt"))` is `(#P"d/a.txt" 7)`. ## Backend support All four backends -- one definition in rontolisp source, spliced into the program when it is referenced. --- # FILE: references/reference/functions/pathname-device.md # pathname-device `(pathname-device pathname)` The device component of a pathname -- always `nil` here, for the reason [`pathname-host`](pathname-host.md) is: a flat namestring carries no device. That is also what SBCL answers on Unix. ```lisp (pathname-device #P"d/a.txt") ; => NIL ``` Portable code tests the answer against both `nil` and `:unspecific` before using it, so a `nil` device simply means "there is no device component to account for". ## Backend support All four backends -- one definition in rontolisp source over primitives every backend has. --- # FILE: references/reference/functions/pathname-directory.md # pathname-directory `(pathname-directory pathname)` The directory component of a namestring, as Common Lisp's list: `:absolute` or `:relative` followed by one component per directory level. A namestring with no directory part answers `nil`. It is component-for-component the INVERSE of what [`make-pathname`](make-pathname.md) builds, so the special levels come back as the keywords they were given as: `..` is `:up`, `*` is `:wild` and `**` is `:wild-inferiors`. Takes a pathname or a namestring; the split is pure string work on the namestring -- nothing is read from the filesystem and a nonexistent path answers just the same. It pairs with [`directory`](directory.md): a walk uses it to decide what to do with each entry it was handed. ```lisp (pathname-directory "/usr/share/zoneinfo/Asia/Tokyo") ; => (:ABSOLUTE "usr" "share" "zoneinfo" "Asia") ``` `(pathname-directory "a/b/c.txt")` is `(:RELATIVE "a" "b")`, `(pathname-directory "c.txt")` is `NIL`, `(pathname-directory "/")` is `(:ABSOLUTE)`, and `(pathname-directory "/a/**/x.lisp")` is `(:ABSOLUTE "a" :WILD-INFERIORS)`. ## Backend support All four backends -- one definition in rontolisp source over primitives every backend has. --- # FILE: references/reference/functions/pathname-host.md # pathname-host `(pathname-host pathname)` The host component of a pathname -- always `nil` here. A rontolisp namestring is a flat, Unix-shaped path with no host syntax, so the component is not present, and `nil` is what Common Lisp prescribes for a component that is not there. The argument is still validated as a pathname designator, exactly as [`namestring`](namestring.md) validates it, so a non-designator signals rather than answering `nil`. [`pathname-device`](pathname-device.md) and [`pathname-version`](pathname-version.md) are the two siblings with the same answer for the same reason. ```lisp (pathname-host "d/a.txt") ; => NIL ``` ## Backend support All four backends -- one definition in rontolisp source over primitives every backend has. --- # FILE: references/reference/functions/pathname-name.md # pathname-name `(pathname-name pathname)` The file-name component of a namestring, without its type: everything after the last `/` and before the LAST dot. A dot at position 0 is part of the name rather than a type separator, and a namestring that names no file -- one ending in `/` -- answers `nil`. Takes a pathname or a namestring; the split is pure string work on the namestring -- nothing is read from the filesystem and a nonexistent path answers just the same. It splits by exactly the rule [`pathname-type`](pathname-type.md) uses for the other half, and the one [`make-pathname`](make-pathname.md) defaults `:defaults` with, so the three can never disagree. ```lisp (pathname-name "db/migrations/20260101.up.sql") ; => "20260101.up" ``` `(pathname-name "d/a")` is `"a"`, `(pathname-name "d/.a")` is `".a"`, `(pathname-name "d/a.b.c")` is `"a.b"`, and `(pathname-name "d/")` is `NIL`. A name that is exactly `*` answers `:WILD` -- the keyword [`make-pathname`](make-pathname.md) builds it from, so decomposition is the inverse of construction here too. ## Backend support All four backends -- one definition in rontolisp source over primitives every backend has. --- # FILE: references/reference/functions/pathname-type.md # pathname-type `(pathname-type pathname)` The type (extension) component of a namestring, without its dot -- what follows the LAST dot of the file part -- or `nil` when there is none. A dot at position 0 does not separate a type, so a dotfile has a name and no type. Takes a pathname or a namestring; the split is pure string work on the namestring -- nothing is read from the filesystem and a nonexistent path answers just the same. It is the other half of the split [`pathname-name`](pathname-name.md) takes, by the same rule. ```lisp (pathname-type "db/migrations/20260101.up.sql") ; => "sql" ``` `(pathname-type "d/a")` is `NIL`, `(pathname-type "d/.a")` is `NIL`, and `(pathname-type "d/a.b.c")` is `"c"`. A type that is exactly `*` answers `:WILD`, the keyword [`make-pathname`](make-pathname.md) builds it from. ## Backend support All four backends -- one definition in rontolisp source over primitives every backend has. --- # FILE: references/reference/functions/pathname-version.md # pathname-version `(pathname-version pathname)` The version component of a pathname -- always `nil` here, for the reason [`pathname-host`](pathname-host.md) is: rontolisp has no file versions, so no namestring can carry one. ```lisp (pathname-version #P"d/a.txt") ; => NIL ``` Deviation: SBCL answers `:newest` for a pathname it PARSED from a namestring and `nil` for one [`make-pathname`](make-pathname.md) built. `nil` -- "the component is not present" -- is the one answer that is true of every pathname here. ## Backend support All four backends -- one definition in rontolisp source over primitives every backend has. --- # FILE: references/reference/functions/pathname.md # pathname `(pathname pathspec)` The pathname the designator names: a pathname is returned unchanged, a string yields a fresh pathname carrying it as the namestring, and anything else signals. This is the canonical constructor the whole family funnels through -- `#P"..."` in source denotes the same value this builds at run time. ```lisp (pathname "d/notes.txt") ; => #P"d/notes.txt" ``` `(pathname #P"d/notes.txt")` is the argument itself; `(pathnamep (pathname "x"))` is `T`. ## Backend support All four backends -- one definition in rontolisp source, spliced into the program when it is referenced. --- # FILE: references/reference/functions/pathnamep.md # pathnamep `(pathnamep object)` Whether `object` is a pathname -- the distinct value `#P"..."` denotes, an object carrying its namestring. A string is NOT one (it merely DESIGNATES a pathname, as in standard Common Lisp), and neither is any other value. It agrees with `(typep object 'pathname)`, as Common Lisp requires the two to. The producers -- `pathname`, `make-pathname`, `merge-pathnames`, `probe-file`, `truename`, `directory` and the `uiop:` directory walkers -- all answer pathnames, and every path-taking operator accepts a pathname and a namestring alike, so the predicate is what lets a library tell a FILE from TEXT: `(typecase in (pathname (open in)) (t ...))` opens a `#P"..."` argument and parses a string argument as content. ```lisp (pathnamep #P"/tmp/data.json") ; => T ``` `(pathnamep "/tmp/data.json")` and `(pathnamep 42)` are `NIL`. --- # FILE: references/reference/functions/peek-char.md # peek-char `(peek-char &optional peek-type stream eof-error-p eof-value)` Returns the next character of `stream` (default: standard input) **without consuming it**, so the following `read-char` returns the same character. `peek-type` selects what to skip first: `nil` (the default) skips nothing, `t` skips whitespace, and a character skips input up to that character. In every case the character that is returned is left in the stream. At end of input it signals an `end-of-file` condition unless `eof-error-p` is `nil`, in which case it returns `eof-value` (default `nil`). ```lisp (with-input-from-string (s " ab") (list (peek-char t s) (read-char s) (peek-char nil s) (read-char s))) ; => (#\a #\a #\b #\b) ``` --- # FILE: references/reference/functions/plus.md # + `(+ &rest numbers)` Returns the sum of its arguments, or `0` with no arguments. The result is an integer when all arguments are integers, a ratio when ratios are involved, and a float if any argument is a float (contagion). Integer results promote to big integers on overflow, on every backend. ```lisp (+ 1 2 3) ; => 6 ``` ```lisp (+ 1.5 2.5) ; => 4.0 ``` --- # FILE: references/reference/functions/plusp.md # plusp `(plusp number)` Returns `t` if `number` is strictly greater than zero, else `nil`. It works for any real numeric type. ```lisp (plusp 3) ; => T ``` --- # FILE: references/reference/functions/position-if-not.md # position-if-not `(position-if-not predicate sequence &key key start end from-end)` Returns the 0-based index of the first element of `sequence` that does NOT satisfy `predicate`, or `nil` if every element does. The complement of `position-if`, with the same keyword arguments. ```lisp (position-if-not #'evenp '(2 4 5 6)) ; => 2 ``` ```lisp (position-if-not #'digit-char-p "42a7") ; => 2 ``` --- # FILE: references/reference/functions/position-if.md # position-if `(position-if predicate sequence &key key start end from-end)` Returns the 0-based index of the first element of `sequence` that satisfies `predicate`, or `nil` if none does. The sequence may be a list or a string (whose elements are characters). It returns the integer position rather than the element itself (compare `find-if`). `:key` selects what the predicate sees, `:start`/`:end` bound the scanned region, and with `:from-end` true the last satisfying element's index is returned. ```lisp (position-if #'evenp '(1 3 6 7)) ; => 2 ``` ```lisp (position-if #'digit-char-p "ab3c") ; => 2 ``` --- # FILE: references/reference/functions/position.md # position `(position item sequence &key test test-not key start end from-end)` Returns the 0-based index of the first element of `sequence` that matches `item`, or `nil` if no element matches. The comparison is `eql` by default; `:test` takes a function designator to use a different comparison and `:test-not` its negation, while `:key` takes a selector function applied to each element before the comparison. `:start`/`:end` bound the scanned region (the returned index still counts from the start of the whole sequence), and with `:from-end` true the LAST match wins. The sequence may be a list or a string; the elements of a string are characters. Unlike `find`, which returns the element, `position` returns its integer position. The full keyword set also works through the first-class value, e.g. `(apply #'position item seq other-keys)`. ```lisp (position 3 '(1 2 3)) ; => 2 ``` ```lisp (position #\space "hello world") ; => 5 ``` ```lisp (position "b" '("a" "b" "c") :test #'string=) ; => 1 ``` ```lisp (position #\, "a,b,c" :start 2) ; => 3 ``` ```lisp (position 2 '(1 2 3 2 4) :from-end t) ; => 3 ``` --- # FILE: references/reference/functions/pprint-dispatch.md # copy-pprint-dispatch set-pprint-dispatch pprint-dispatch `(copy-pprint-dispatch &optional table)` -- `(set-pprint-dispatch type-specifier function &optional priority table)` -- `(pprint-dispatch object &optional table)` The pretty-print dispatch table: a set of `(type-specifier function priority)` entries saying how to print a value of a given type. `copy-pprint-dispatch` answers a fresh copy of `*print-pprint-dispatch*` (or of the empty initial table when given `nil`), `set-pprint-dispatch` adds an entry -- or removes one when `function` is `nil` -- and `pprint-dispatch` answers the highest-priority entry function matching an object, plus a second value saying whether one was found. An entry function is called as `(funcall function stream object)`. The ordinary printing operators (`print`, `princ`, `prin1`, `~A`, `~S`) do **not** consult the table: an entry takes effect where the program calls the entry function itself. ```lisp (let ((table (copy-pprint-dispatch))) (set-pprint-dispatch 'integer (lambda (stream x) (princ (* 2 x) stream)) 0 table) (with-output-to-string (out) (funcall (pprint-dispatch 21 table) out 21))) ; => "42" ``` --- # FILE: references/reference/functions/pprint.md # pprint pprint-newline pprint-indent pprint-tab `(pprint object &optional stream)` -- `(pprint-newline kind &optional stream)` -- `(pprint-indent relative-to n &optional stream)` -- `(pprint-tab kind colnum colinc &optional stream)` `pprint` writes a fresh line and then `object` in its readable form, returning no values. The other three are the pretty printer's layout operators. **A rontolisp stream carries no column**, so only `(pprint-newline :mandatory)` does anything: it writes a newline when `*print-pretty*` is true. The three conditional kinds (`:linear`, `:fill`, `:miser`), `pprint-indent` and `pprint-tab` are accepted and do nothing, and `*print-right-margin*` / `*print-miser-width*` / `*print-lines*` are inert for the same reason -- text comes out as if the line were wide enough. ```lisp (with-output-to-string (s) (princ "a" s) (pprint-newline :fill s) (princ "b" s)) ; => "ab" ``` --- # FILE: references/reference/functions/prin1-to-string.md # prin1-to-string `(prin1-to-string object)` Returns, as a string, the text that `prin1` would write for `object` -- the readable form in which strings keep their surrounding quotes (with every embedded `"` and `\` preceded by a `\`) and characters use `#\` syntax. Nothing is printed; the rendering is captured and returned, so `(read-from-string (prin1-to-string s))` gives `s` back. ```lisp (prin1-to-string "abc") ; => "\"abc\"" ``` --- # FILE: references/reference/functions/prin1.md # prin1 `(prin1 object &optional stream)` Writes `object` to standard output in its readable form, exactly like `print` but without the trailing newline. Strings are printed with surrounding quotes and characters in `#\` syntax, so the output could be read back by `read`. Inside a string every embedded `"` and `\` is preceded by a `\`; a newline is printed literally. Binding `*print-case*` converts the case of every symbol printed ([Reader Case](../../guides/reader-case.md)). With the optional stream argument the output goes to that stream instead of standard output. Returns `object`. ```lisp (prin1 "hello") ``` ``` "hello" ``` ```lisp (prin1 "{\"hello\":\"aaa\"}") ``` ``` "{\"hello\":\"aaa\"}" ``` --- # FILE: references/reference/functions/princ-to-string.md # princ-to-string `(princ-to-string object)` Returns, as a string, the text that `princ` would write for `object` -- the human-readable form with no quotes on strings and no `#\` prefix on characters. Nothing is printed; the rendering is captured and returned. Useful for building text out of arbitrary values. ```lisp (princ-to-string '(1 "x")) ; => "(1 x)" ``` --- # FILE: references/reference/functions/princ.md # princ `(princ object &optional stream)` Writes `object` to standard output in human-readable form, with no surrounding quotes on strings and no `#\` prefix on characters, and without a trailing newline. This is the form meant for display rather than for reading back. A symbol prints as its [`symbol-name`](symbol-name.md) alone: a keyword's leading `:`, a gensym's `#:` and a package qualifier (`quri:uri` prints as `URI`) are all where the symbol lives rather than part of its name, and none of them is printed (`prin1`/`print` keep them). A condition object prints its [`:report`](../macros/define-condition.md) (`prin1` keeps the `#<...>` instance syntax). Binding `*print-case*` converts the case of every symbol printed ([Reader Case](../../guides/reader-case.md)). With the optional stream argument the output goes to that stream instead of standard output. Returns `object`. ```lisp (princ "hello") (princ :ready) ``` ``` helloREADY ``` --- # FILE: references/reference/functions/print-object.md # print-object `(print-object object stream)` The generic function the printer consults. Defining a [`defmethod`](../special-forms/defmethod.md) on it for a [`defclass`](../special-forms/defclass.md) class or a [`defstruct`](../special-forms/defstruct.md) type makes `print`, `princ`, `prin1`, `princ-to-string`, `prin1-to-string` and [`format`](../macros/format.md)'s `~A`/`~S` render instances of that type through the method instead of the built-in `#S(...)` / `#<...>` syntax. The method writes to the stream it is given and its return value is ignored; [`print-unreadable-object`](../macros/print-unreadable-object.md) is the usual body. The PRINTER only routes through the generic for a type some method specializes on: a type no method covers keeps the built-in rendering, and a program that defines no `print-object` method prints exactly as it did before. A DIRECT `(print-object object stream)` call is a different matter and always works, with or without a user method on the object -- Common Lisp supplies a system method for every object, and so does rontolisp. It writes the object's raw rendering (`*print-escape*` picking the `prin1` or the `princ` spelling) and returns the object, so defining a method on one class never loses the printer for the rest, and `(call-next-method)` out of the least specific method reaches it. ```lisp (with-output-to-string (s) (print-object 42 s)) ; => "42" ``` One limit on the direct call: the system method renders raw, so a nested instance inside the value it is handed does not get its own method consulted. Printing through `print`/`princ`/`prin1` does walk into it, as below. A [`defstruct`](../special-forms/defstruct.md) `(:print-object fn)` / `(:print-function fn)` option is exactly a method on this generic, so the two are interchangeable and a later `defmethod` on the same type replaces the option's method. The one built-in rendering that is not `#S(...)`/`#<...>` is a CONDITION's: `princ`/`princ-to-string`/`~A` write its [`:report`](../macros/define-condition.md) instead. A `print-object` method on a condition class wins over that report, in both escape modes. `*print-escape*` is bound around the call — `t` for `prin1`/`print`/`~S`, `nil` for `princ`/`~A` — so a portable method that branches on it (the Common Lisp idiom for rendering readably or bare) behaves the same way here. `*print-readably*` is always `nil`. ```lisp (defstruct po-uri text) (defmethod print-object ((u po-uri) stream) (if (and (null *print-readably*) (null *print-escape*)) (write-string (po-uri-text u) stream) (format stream "#" (po-uri-text u)))) (list (princ-to-string (make-po-uri :text "/x")) (prin1-to-string (make-po-uri :text "/x"))) ; => ("/x" "#") ``` The method is consulted wherever the instance SITS, not only when the printing operator is handed it directly: an element of a printed list or vector — at any depth, and in a dotted tail — goes through the method too. ```lisp (defstruct po-node value) (defmethod print-object ((n po-node) stream) (print-unreadable-object (n stream :type t) (princ (po-node-value n) stream))) (list (princ-to-string (make-po-node :value 42)) (princ-to-string (list (make-po-node :value 7) (vector (make-po-node :value 8))))) ; => ("#" "(# #(#))") ``` Lite: the containers walked that way are the list and the general one-dimensional vector. A value stored in a STRUCTURE or class slot, a hash table, an array of rank other than one or a specialized float vector is still rendered by the container's own printer, so a method on its type does not apply there. --- # FILE: references/reference/functions/print.md # print `(print object &optional stream)` Writes `object` to standard output in its readable (`prin1`) form -- strings are surrounded by quotes, with every embedded `"` and `\` preceded by a `\`, and characters use `#\` syntax -- followed by a trailing newline, then returns `object`. With the optional stream argument (a file stream or a `with-output-to-string` string stream) the output goes to that stream instead of standard output. Use it for quick, machine-readable output that another `read` could parse back. ```lisp (print "hello") ``` ``` "hello" ``` --- # FILE: references/reference/functions/probe-file.md # probe-file `(probe-file pathname)` Answers whether a file exists: the pathname when it does, `nil` when it does not. This is the only file operation that asks the question without opening anything, and the only one that does not fail on a missing path -- `open` (and therefore `with-open-file`) signals an error on every backend. Catching that error is a workable substitute, but a heavier one: `handler-case` puts the WASM backends into exception mode, and `--no-gc` rejects catching entirely, so a plain probe is the portable spelling. Works on all four backends. [`truename`](truename.md) is the signalling twin of this function. The "truename" answered on success is a pathname carrying the argument namestring: no backend resolves symbolic links or makes the path absolute. The argument takes either spelling (a pathname or a namestring). The path is interpreted exactly as `open` interprets it -- relative to the process working directory on the interpreter and the JVM, and to the preopened directories on WASM (a relative path against the first one, an absolute path against the preopened directory whose name is its longest prefix; run with `--dir`). A directory counts as existing. `uiop:file-exists-p` is the same operation under its ASDF/UIOP name. ```console (if (probe-file "config.lisp") (load "config.lisp") (print "no config")) ``` Here the file is loaded only when it is there; without `probe-file` the missing-file case would abort the program rather than take the `else` branch. --- # FILE: references/reference/functions/provide.md # provide `(provide module-name)` Marks a module as loaded, so a later [`require`](require.md) of the same name returns without loading any file. Returns the module name as a symbol; `module-name` is a designator (keyword, symbol, or string), and providing an already-provided name is a no-op. A file loaded by `require` is expected to call `provide` itself, conventionally as its first form (which also lets mutually requiring files terminate). The backend split mirrors `require`: an ordinary runtime function on the interpreter, and a literal, top-level compile-time directive on the JVM/WASM compile path (a nested or computed `provide` is a compile error). The Common Lisp `*modules*` variable is not available. ```lisp (provide :my-module) ; => MY-MODULE ``` ```lisp (provide :my-module) (require :my-module) ; => MY-MODULE ``` The `require` returns immediately because the module was already provided in the same program — no `my-module.lisp` file is looked up. --- # FILE: references/reference/functions/ql-dist-install-dist.md # ql-dist:install-dist `(ql-dist:install-dist name-or-url &rest options)` Installs a Quicklisp-format distribution beside the Quicklisp dist that [`ql:quickload`](ql-quickload.md) downloads from. The argument is a known dist name — `"quicklisp"` or `"ultralisp"` — or the URL of a distinfo (`"http://dist.ultralisp.org/"`, the URL [Ultralisp](https://ultralisp.org/) itself tells you to install, serves its distinfo directly). The return value is the installed dist's name as a string; installing the same dist twice is a no-op. Keyword options (`:prompt nil`, ...) are accepted and ignored — nothing here prompts before downloading. Only Quicklisp is installed by default, so Ultralisp is **opt-in**: this call, or the CLI `--dist` option / the `RONTOLISP_DISTS` environment variable for invocations with nowhere to put a form. The dists are searched **in the order they were installed**, per system: `ql:quickload` takes each system (and each dependency) from the first dist that lists it, so an added dist supplies the names Quicklisp does not have without changing where anything else comes from. A dist's indexes are downloaded only when a lookup actually reaches it. Each dist caches under `~/.rontolisp//` (`RONTOLISP_DIST_HOME` overrides the base; `RONTOLISP_QUICKLISP_HOME` still overrides the quicklisp one). Like `ql:quickload`, it takes effect at **interpret time or compile time** (on the Java side, not inside the compiled program). On the compile path (JVM/WASM) a **literal, top-level** call configures the dists the `quickload` forms below it download from while the program is being spliced, and is then consumed; a call nested inside another form, or with a computed argument, is a compile error. On the interpreter it is an ordinary runtime function, so a computed URL works. ```console $ rontolisp > (ql-dist:install-dist "http://dist.ultralisp.org/" :prompt nil) "ultralisp" > (ql:quickload "split-sequence") (split-sequence) ``` See the [Systems guide](../../guides/asdf-systems.md#adding-a-dist-ultralisp) for the search order, the cache layout and [`ql:update-dist`](ql-update-dist.md), which refreshes a dist's index. --- # FILE: references/reference/functions/ql-quickload.md # ql:quickload `(ql:quickload name-or-names &rest options)` Downloads a system (and its dependencies) from the real [Quicklisp](https://www.quicklisp.org/) distribution into a local cache, then loads it exactly like [`asdf:load-system`](asdf-load-system.md). The argument is a single system-name designator (string, keyword or symbol) or a list of them; the return value is the list of loaded system names as symbols. `quicklisp` is a nickname for the `ql` package. Keyword options (`:silent t`, ...) are accepted and ignored, like [`asdf:load-system`](asdf-load-system.md)'s. The download uses the Quicklisp dist metadata: `quicklisp.txt` (the distinfo) points at `systems.txt` (dependency resolution) and `releases.txt` (the tarball URL per project). Each release tarball is fetched, extracted and cached under `~/.rontolisp/quicklisp/` (override the location with the `RONTOLISP_QUICKLISP_HOME` environment variable), so a second `quickload` of the same system does no network I/O. Quicklisp is the only dist installed by default; [`ql-dist:install-dist`](ql-dist-install-dist.md) (or the CLI `--dist`) adds another Quicklisp-format distribution such as [Ultralisp](https://ultralisp.org/), and each system — and each dependency — is then taken from the first installed dist that lists it. After the sources are present, the extracted `.asd` directories are added to the system search path and loading proceeds through the `asdf` subset — so `ql:quickload` is `asdf:load-system` with an auto-download step in front of it. It is subject to the same limitations: many libraries use Common Lisp features rontolisp does not implement (the full CLOS protocol, the condition system) and will not load even once downloaded. The download happens at **interpret time or compile time** (on the Java side, not inside the compiled program). On the interpreter, `quickload` is an ordinary runtime function, so a computed name works. On the compile path (JVM/WASM), a **literal, top-level** `(ql:quickload NAME)` is downloaded and its component files are spliced into the program at compile time — so the compiled program has the sources baked in and never fetches at runtime (the WASM `fetch` limitation does not apply). A `quickload` nested inside another form or with a computed argument is a compile error. ```console $ rontolisp > (ql:quickload "split-sequence") (split-sequence) > (split-sequence:split-sequence #\, "a,b,c") ("a" "b" "c") ``` The first call downloads and caches `split-sequence`; the second call into the loaded system runs it. See the [Systems guide](../../guides/asdf-systems.md) for the cache layout and the list of libraries that actually load. --- # FILE: references/reference/functions/ql-update-dist.md # ql:update-dist `(ql:update-dist name &rest options)` Drops a dist's cached indexes, so the next [`ql:quickload`](ql-quickload.md) re-reads that distribution's `systems.txt` and `releases.txt` and sees the releases published since the cache was written. The argument is an installed dist's name (`"quicklisp"`, `"ultralisp"`, ...) as a string, keyword or symbol, and the return value is that name; naming a dist that is not installed is an error. Keyword options are accepted and ignored. Without it a dist index is cached forever, which is what makes a repeated `quickload` free — but a distribution rebuilt every few minutes ([Ultralisp](https://ultralisp.org/)) publishes releases the cached index cannot name. Already-extracted sources are kept: a release directory is named after its version, so a newer release extracts beside the old one rather than replacing it. Same timing as `ql:quickload`: interpret time, or compile time for a **literal, top-level** call (which is then consumed — the compiled program downloads nothing at run time). A nested or computed call is a compile error on the JVM/WASM backends. ```console $ rontolisp > (ql:update-dist "ultralisp") "ultralisp" > (ql:quickload "split-sequence") (split-sequence) ``` See the [Systems guide](../../guides/asdf-systems.md#adding-a-dist-ultralisp) and [`ql-dist:install-dist`](ql-dist-install-dist.md). --- # FILE: references/reference/functions/random.md # random `(random limit &optional random-state)` Returns a random number in the half-open interval `[0, limit)`. The result type follows the limit: an integer limit yields an integer, a float limit yields a float (so `(random 1)` is always `0`). The interpreter and JVM draw from `Math.random`; WASM draws real entropy from the WASI host (`random_get` in Preview 1, `wasi:random` under `--component`), so the sequence differs on every run. A `--no-wasi` module has no host to draw from and carries its own generator instead -- see the [clock and randomness guide](../../guides/clock-and-random.md). The optional random-state argument is accepted and ignored (evaluated for effect): no random-state objects exist here — [`make-random-state`](make-random-state.md) answers `nil` — and the backend's own entropy always draws. ```lisp (random 1) ; => 0 ``` --- # FILE: references/reference/functions/rassoc-if.md # rassoc-if `(rassoc-if predicate alist)` Searches an association list and returns the first pair whose cdr satisfies `predicate`, or `nil` if none does. It is the mirror of `assoc-if`, which tests each pair's car, and the predicate form of `rassoc`. Each pair is tested with `(funcall predicate (cdr pair))`; non-cons elements of the list are skipped. The returned pair shares structure with the alist. ```lisp (rassoc-if #'oddp '((a . 2) (b . 3))) ; => (B . 3) ``` ```lisp (rassoc-if #'consp '((1 . 2) (3 4 . 5))) ; => (3 4 . 5) ``` --- # FILE: references/reference/functions/rassoc.md # rassoc `(rassoc value alist &key test key)` Searches an association list and returns the first pair whose cdr matches `value`, or `nil` if none matches. It is the mirror of `assoc`, which searches by car. The comparison is `eql` by default; the optional `:test` keyword takes a function designator to use a different comparison, and the optional `:key` keyword takes a selector function applied to each pair's cdr before the comparison. The returned pair shares structure with the alist. ```lisp (rassoc 2 '((a . 1) (b . 2))) ; => (B . 2) ``` ```lisp (rassoc "x" '((a . "w") (b . "x")) :test #'equal) ; => (B . "x") ``` ```lisp (rassoc 2 '((a . 1) (b . 3)) :key (lambda (v) (- v 1))) ; => (B . 3) ``` --- # FILE: references/reference/functions/rationalp.md # rationalp `(rationalp object)` Returns `t` if `object` is a rational number -- an integer or a ratio -- otherwise `nil`. Floats are not rational, so `(rationalp 3.14)` is `nil`. Works in all three backends. ```lisp (rationalp 1/2) ; => T ``` ```lisp (rationalp 3.14) ; => NIL ``` --- # FILE: references/reference/functions/read-byte.md # read-byte `(read-byte stream &optional eof-error-p eof-value)` Reads one byte from a binary input stream -- a stream opened with `:element-type '(unsigned-byte 8)` -- and returns it as an integer between 0 and 255. At end of file it signals an `end-of-file` condition by default (catchable as `end-of-file`, or as `error`); passing `nil` as `eof-error-p` makes it return `eof-value` (default `nil`) instead. Works in all four backends. Bytes pass through raw: values such as 0 (NUL), 10 (LF) and 34 (`"`) are not interpreted. `stream` takes the same designators every other stream operation takes: `t` is the process standard input, and `nil` means the current `*standard-input*` -- which holds `t` unless you bind it. `(read-byte *standard-input*)` therefore reads raw octets from standard input, which is how a byte-oriented filter reads its input. Because it touches the filesystem, `read-byte` is shown here statically rather than as a runnable example: ```console (with-open-file (in "data.bin" :element-type '(unsigned-byte 8)) (read-byte in) ; => 137 (read-byte in nil nil)) ; => nil at end of file (read-byte *standard-input* nil nil) ; => the next octet of stdin, nil at EOF ``` The first call returns the next byte of `data.bin`; the second form reads until end of file without signalling, returning `nil` when the bytes are exhausted -- the usual loop-termination test when reading a whole file. Do not mix `read-byte` with `read-line` / `read-char` on the same stream: the character reads buffer ahead, so bytes a following `read-byte` owes you may already have been consumed. --- # FILE: references/reference/functions/read-char-no-hang.md # read-char-no-hang `(read-char-no-hang &optional stream eof-error-p eof-value)` Reads one character from `stream` (default: standard input) if one is available without waiting, and returns it. On a stream HANDLE -- a file, a string input stream, a socket -- rontolisp answers exactly what `read-char` answers: no source it can open reports "a character would block" separately from "read one", and CL allows an implementation to say so. On a [Gray stream](../../guides/gray-streams.md) instance it dispatches to `rontolisp:stream-read-char-no-hang`, which is the generic a class with a genuinely non-blocking source overrides; that generic's own default is `stream-read-char`. At end of input it signals an `end-of-file` condition unless `eof-error-p` is `nil`, in which case it returns `eof-value` (default `nil`). ```lisp (with-input-from-string (s "hi") (list (read-char-no-hang s) (read-char-no-hang s) (read-char-no-hang s nil :end))) ; => (#\h #\i :END) ``` --- # FILE: references/reference/functions/read-char.md # read-char `(read-char &optional stream eof-error-p eof-value)` Reads one character from `stream` (default: standard input) and returns it. The stream may be a file stream opened by `open`/`with-open-file`, a string input stream from `with-input-from-string`, or a [TCP or TLS socket handle](../../guides/tcp-sockets.md) -- on a socket the character is assembled from the wire's UTF-8 bytes, so `read-char` and `read-byte` can be mixed on one connection. At end of input it signals an `end-of-file` condition unless `eof-error-p` is `nil`, in which case it returns `eof-value` (default `nil`). Because the condition is the registered `end-of-file` class, the usual CL lexer shape -- a read loop wrapped in `(handler-case ... (end-of-file (e) ...))` -- terminates as written. On the interpreter and the JVM backend a character is a UTF-16 code unit, matching the rest of the string representation; on the WASM backend strings are byte-indexed (like `char`/`schar`), so a character read is a byte read. ```lisp (with-input-from-string (s "hi") (let* ((c1 (read-char s)) (c2 (read-char s)) (c3 (read-char s nil :end))) (list c1 c2 c3))) ; => (#\h #\i :END) ``` --- # FILE: references/reference/functions/read-from-string.md # read-from-string `(read-from-string string)` Parses and returns one datum from the given string. It reuses the same reader as [`read`](read.md), so on the compiled backends it accepts the same frontend-parity syntax (`#S(...)`, `#(...)`, `#\a`, ratios, radix integers, ... -- with `#.`, `#+`/`#-` and reader labels signaling; the WASM reader's numbers carry integers of any magnitude but no float exponents), and `(read-from-string (prin1-to-string x))` round-trips. The interpreter evaluates a `#.` datum in place (binding `*read-eval*` to `nil` makes it signal, per the standard). The optional `eof-error-p`/`eof-value` and the `:start`/`:end` keyword arguments are not supported -- only the single string argument is accepted. Works in all three backends and is usable as a first-class value (`#'read-from-string`). ```lisp (read-from-string "(+ 1 2)") ; => (+ 1 2) ``` The result is the parsed list `(+ 1 2)` as data, not its evaluation; pass it to `eval` if you want the value `3`. Symbols are read with the reader's [upcasing](../../guides/reader-case.md), identically on every backend: your symbols and the standard names alike read upcased (there is no fold to a lowercase spelling). ```lisp (read-from-string "foo") ; => FOO ``` --- # FILE: references/reference/functions/read-line.md # read-line `(read-line &optional stream)` Reads one line of text and returns it as a string with the trailing newline removed (a CRLF line ending also loses its carriage return, like Java's `BufferedReader.readLine` -- so CRLF-terminated input such as HTTP over a [`rontolisp:tcp-connect`](rontolisp-tcp-connect.md) socket reads as plain lines). With no argument it reads from standard input; given a stream opened by `open` or `with-open-file` it reads the next line from that stream. At end of input it returns `nil` rather than signalling an error. Works in all three backends; unlike `read`, it returns the raw line without parsing it as an S-expression. ```console (print (read-line)) ``` Typing `hello world` on standard input makes `read-line` return the string `"hello world"`. When the input is exhausted it returns `nil`, which is the usual loop-termination test when reading a file line by line. --- # FILE: references/reference/functions/read-sequence.md # read-sequence `(read-sequence sequence stream &key start end)` Fills `sequence` -- a one-dimensional array created with `make-array` -- with elements read from `stream`, and returns the index of the first element that was not filled (the fill position). Reading starts at index `:start` (default 0) and stops before index `:end` (default the array length) or at end of file, whichever comes first. The `:start`/`:end` keywords must be literal; their values may be arbitrary expressions. The BUFFER decides which element is read: a character vector -- what `(make-array n :element-type 'character)` and `make-string` build -- is filled with characters from a text stream, and any other array is filled with bytes from a stream opened with `:element-type '(unsigned-byte 8)`. The element type may itself be computed, as in `(make-array n :element-type (stream-element-type s))`. ```lisp (with-input-from-string (s "abcdef") (let ((buf (make-array 4 :element-type 'character))) (list (read-sequence buf s) buf))) ; => (4 "abcd") ``` Because it touches the filesystem, the binary form is shown here statically rather than as a runnable example: ```console (let ((buf (make-array 8))) (with-open-file (in "data.bin" :element-type '(unsigned-byte 8)) (read-sequence buf in)) ; => 4 when data.bin has 4 bytes (aref buf 0)) ; => the first byte ``` A return value smaller than the array length means the input ended early; elements at and beyond the fill position keep their previous values. ## Packed buffers: raw binary elements in bulk When the buffer is a **packed** array -- a packed float array of any rank (`:element-type 'single-float` / `'double-float`, `#f(...)` / `#d(...)`) or a packed integer vector (`:element-type '(unsigned-byte 8)`, `16` or `32`) -- `read-sequence` reads its elements as **raw little-endian binary** from a binary stream, in one bulk transfer instead of a byte-at-a-time loop: a single-float is the 4 bytes of its IEEE-754 encoding, a double-float 8 bytes, an `(unsigned-byte 16)` 2 bytes, and so on. A rank-2 or rank-3 packed float array is filled in row-major order (`:start`/`:end` count elements, `:end` defaults to the total size). This is how a program loads a weight matrix, a numpy `.npy` payload or any C-struct dump: a `make-array` and one `read-sequence`, on every backend, at memcpy speed -- a llama2 checkpoint's 15 million floats load in about 0.2 s. A trailing partial element at end of file is not stored and not counted. ```console (with-open-file (in "weights.bin" :element-type '(unsigned-byte 8)) (let ((w (make-array '(288 288) :element-type 'single-float :initial-element 0.0))) (read-sequence w in))) ; => 82944 -- 288*288 little-endian float32s, row-major ``` Only a general (boxed) array is filled through the `read-byte` loop above, so a program that wants integers larger than 255 as elements should use a packed `(unsigned-byte 16|32)` vector -- and a general vector still receives one byte per element. --- # FILE: references/reference/functions/read.md # read `(read &optional stream)` Reads and parses a single S-expression. With no argument it reads from standard input; given a stream opened by `open` or `with-open-file` it reads from that stream. Blank and comment-only lines are skipped, the datum must fit on a single line, and EOF returns `nil`. The compiled backends emit a runtime reader with frontend parity: lists (dotted pairs included), `'`, `#'`, strings, symbols, numbers (ratios and `#x`/`#o`/`#b` radix integers included), `#\` character literals, `#(...)`/`#nA(...)` arrays, `#*` bit vectors, `#f(`/`#d(` packed float arrays, `#S(...)` structure literals and `#|...|#` block comments. `#.`, `#+`/`#-` and `#n=`/`#n#` need an evaluator or the feature set at read time, so the compiled reader signals a catchable error on them (the interpreter still resolves them; binding `*read-eval*` to `nil` there makes `#.` signal instead, per the standard). The WASM reader's numbers are narrower (64-bit integers, decimal floats without exponents, static error messages) -- see [Compiled read/load Limitations](../../guides/read-load-limitations.md). ```console (print (read)) ``` Reading the line `(+ 1 2)` from standard input parses it into the list `(+ 1 2)`, which `print` then echoes back. At end of input `read` returns `nil`. Symbols read at run time follow the reader's [upcasing](../../guides/reader-case.md) -- your symbols and the standard names alike upcase (there is no fold) -- identically on every backend. --- # FILE: references/reference/functions/readtable-case.md # readtable-case `(readtable-case readtable)` Lite stub: always returns `:upcase` -- the reader is not readtable-driven and always upcases unescaped symbol names, which is exactly the standard readtable's `:upcase` mode. The argument is evaluated but ignored (the `*readtable*` variable exists but is seeded to `nil`). Exists so library code that branches on the readtable case, like s-sql's `from-sql-name`, takes the standard-mode branch. ```lisp (readtable-case *readtable*) ; => :UPCASE ``` --- # FILE: references/reference/functions/reduce.md # reduce `(reduce function sequence &key initial-value from-end key start end)` Combines the elements of `sequence` with a binary `function`, left-associatively: `(reduce #'f '(a b c))` computes `(f (f a b) c)`. The sequence may be a list or a string, whose characters are folded. With `:initial-value` the seed is supplied explicitly and folded in first: `(f (f (f init a) b) c)`; otherwise the first element of the sequence is the seed. With `:from-end t` the elements are combined right-associatively with the accumulator on the right: `(reduce #'f '(a b c) :from-end t)` computes `(f a (f b c))`, and with `:initial-value i`, `(f a (f b (f c i)))`. `:key` names a one-argument function applied to each sequence element (but not the initial value) before folding. `:start`/`:end` restrict the fold to a subsequence (an `:end` of `nil` means the whole sequence). Keywords may appear in any order; the keyword names are read at compile time (their values may be runtime expressions). An empty sequence returns the initial value, or calls `function` with no arguments when none was given. ```lisp (reduce #'+ '(1 2 3) :initial-value 0) ; => 6 ``` ```lisp (reduce (lambda (acc c) (if (char= c #\a) (+ acc 1) acc)) "banana" :initial-value 0) ; => 3 ``` ```lisp (reduce #'cons '((1) (2) (3)) :from-end t :key #'car :initial-value nil) ; => (1 2 3) ``` --- # FILE: references/reference/functions/rem.md # rem `(rem number divisor)` Returns the remainder of `number` divided by `divisor` using truncated division, so the result always takes the sign of the dividend. It is the companion of `truncate`. Use `mod` instead when you want the result to follow the sign of the divisor. ```lisp (rem 13 4) ; => 1 ``` ```lisp (rem -13 4) ; => -1 ``` --- # FILE: references/reference/functions/remhash.md # remhash `(remhash key table)` Removes the entry for `key` from `table`. Returns `t` if an entry was present and removed, or `nil` if the key was not found. Keys are matched structurally, the same way `gethash` looks them up. ```lisp (let ((h (make-hash-table))) (setf (gethash 'a h) 1) (remhash 'a h)) ; => T ``` --- # FILE: references/reference/functions/remove-duplicates.md # remove-duplicates `(remove-duplicates sequence &key test key from-end)` Returns a new sequence with duplicate elements removed, keeping the last occurrence of each (so the order of the surviving elements follows their last appearance); `:from-end t` keeps the FIRST occurrence instead and must be a literal `t` or `nil`. The comparison is `eql` by default; the optional `:test` keyword takes a function designator to use a different comparison, and the optional `:key` keyword takes a selector function applied to each element before the comparison. The sequence may be a list or a string; a string yields a new string. The original sequence is not modified. See also [`delete-duplicates`](delete-duplicates.md), which shares this rendering. ```lisp (remove-duplicates '(1 2 1 3)) ; => (2 1 3) ``` ```lisp (remove-duplicates "banana") ; => "bna" ``` ```lisp (remove-duplicates '("a" "b" "a" "c") :test #'string=) ; => ("b" "a" "c") ``` --- # FILE: references/reference/functions/remove-if-not.md # remove-if-not `(remove-if-not predicate sequence &key key)` Returns a new sequence keeping only the elements of `sequence` that satisfy `predicate` (those failing it are removed). The sequence may be a list or a string; a string yields a new string. It is the complement of `remove-if`. The original sequence is not modified. With `:key`, the predicate sees the keyed value while the kept elements are the originals. ```lisp (remove-if-not #'evenp '(1 2 3 4)) ; => (2 4) ``` ```lisp (remove-if-not #'digit-char-p "a1b2") ; => "12" ``` ```lisp (remove-if-not #'evenp '((1 a) (2 b) (3 c)) :key #'car) ; => ((2 B)) ``` --- # FILE: references/reference/functions/remove-if.md # remove-if `(remove-if predicate sequence &key key)` Returns a new sequence containing the elements of `sequence` that do **not** satisfy `predicate` (the satisfying elements are removed). The sequence may be a list or a string; a string yields a new string. The original sequence is not modified; use `delete-if` for the destructive version (lists only). With `:key`, the predicate sees the keyed value while the kept elements are the originals. ```lisp (remove-if #'evenp '(1 2 3 4)) ; => (1 3) ``` ```lisp (remove-if #'digit-char-p "a1b2") ; => "ab" ``` ```lisp (remove-if #'evenp '((1 a) (2 b) (3 c)) :key #'car) ; => ((1 A) (3 C)) ``` --- # FILE: references/reference/functions/remove.md # remove `(remove item sequence &key test key)` Returns a new sequence containing the elements of `sequence` with every element matching `item` omitted. The comparison is `eql` by default; the optional `:test` keyword takes a function designator to use a different comparison, and the optional `:key` keyword takes a selector function applied to each element before the comparison (the kept elements are the original ones). The sequence may be a list or a string; a string yields a new string. The original sequence is not modified; use `delete` for the destructive version (lists only). ```lisp (remove 2 '(1 2 3 2)) ; => (1 3) ``` ```lisp (remove #\l "hello") ; => "heo" ``` ```lisp (remove 1 '((1 a) (2 b) (1 c)) :key #'car) ; => ((2 B)) ``` --- # FILE: references/reference/functions/remprop.md # remprop `(remprop symbol indicator)` Removes the `indicator` property from the symbol's property list, returning true when it was there and `nil` when it was not. The partner of [`get`](get.md) / `(setf (get ...))` and [`symbol-plist`](symbol-plist.md): all four read the one program-global, name-keyed store (symbols have no identity cells to hang plists on). Common Lisp only promises a generalized boolean here; this returns `t`, where some implementations return the plist tail. ```lisp (setf (get 'my-node 'color) :red) (setf (get 'my-node 'size) 3) (list (remprop 'my-node 'color) (symbol-plist 'my-node) (remprop 'my-node 'color)) ; => (T (SIZE 3) NIL) ``` --- # FILE: references/reference/functions/rename-file.md # rename-file `(rename-file file new-name)` Renames (moves) `file` to `new-name` and returns the defaulted new name as a pathname. `new-name` is merged with `file` the way [`merge-pathnames`](merge-pathnames.md) merges, so a bare file name keeps the original directory. Anything that leaves the file where it was signals -- "it was not there" included, exactly like [`delete-file`](delete-file.md). ```console > (rename-file "notes.txt" "notes.bak") #P"notes.bak" > (rename-file "db/2026.up.sql" "2026.down.sql") #P"db/2026.down.sql" ``` Lite deviation: Common Lisp returns `(values defaulted-new-name old-truename new-truename)` and this returns the defaulted new name only -- the same rule [`ensure-directories-exist`](ensure-directories-exist.md) follows, because a secondary value would not survive the function boundary on the compiled backends. ## Backend support Interpreter and JVM rename for real. Both WASM backends signal at CALL time: the WASI import set here carries no rename call, and "the file is at the new name afterwards" has no honest non-answer -- the same divergence [`delete-file`](delete-file.md) and [`ensure-directories-exist`](ensure-directories-exist.md) have. --- # FILE: references/reference/functions/replace.md # replace `(replace sequence-1 sequence-2 &key start1 end1 start2 end2)` Copies the elements of `sequence-2` (bounded by `:start2`/`:end2`) into `sequence-1` (bounded by `:start1`/`:end1`) and returns the result. The number of elements copied is the smaller of the two bounded lengths. A vector `sequence-1` -- including a string allocated by [`make-string`](make-string.md) or [`make-array`](make-array.md) `:element-type 'character` -- is mutated in place and returned, as in CL, so the "allocate a buffer, write into it, return it" idiom works on every backend. A list `sequence-1` is likewise rewritten in place, through its cons cells. A string LITERAL is supported as `sequence-1` too (the case cl-who needs), but on the compiled backends it is an immutable value, so `replace` returns a fresh string instead of mutating it (the interpreter mutates it in place). Available on all backends except `--no-gc`. ```lisp (replace (make-string 5 :initial-element #\a) "XY" :start1 1) ; => "aXYaa" ``` --- # FILE: references/reference/functions/require.md # require `(require module-name [pathname])` Loads the file for a module unless the module was already registered by [`provide`](provide.md), then returns the module name as a symbol. `module-name` is a designator: a keyword (`:util`), a symbol (`'util`), or a string (`"util"`). Without an explicit `pathname` the file `.lisp` is resolved relative to the requiring file, exactly like [`load`](load.md); an explicit second argument overrides that mapping. The required file is expected to `(provide )` itself, conventionally as its first form — that is what marks the module, so a second `require` of the same name is consumed without loading. This makes `require` the tool for the diamond dependency (`a.lisp` and `b.lisp` both requiring `utils.lisp` loads it once), where `load` would evaluate the file twice. On the interpreter, `require` is an ordinary runtime function. On the compile path (JVM/WASM), a **literal, top-level** `(require ...)` is expanded at compile time: the module file is spliced into the program like the compile-time `load` include, so the compilers see its definitions natively. Unlike `load`, a `require` nested inside another form or with a computed argument is a compile error — it cannot be deferred to the compiled runtime reader. The Common Lisp `*modules*` variable is not available. ```console ;; util.lisp (provide :util) (defun u-sq (x) (* x x)) ;; main.lisp (require :util) (require :util) ; already provided: no second load (print (u-sq 7)) ; prints 49 ``` The first `require` loads `util.lisp`, whose `provide` marks the module; the second is a no-op returning `util`. --- # FILE: references/reference/functions/rest.md # rest `(rest list)` Returns the list with its first element removed -- everything after the head. It is an exact synonym for `cdr`, including the `(rest nil)` is `nil` behavior, and pairs with `first` for readable list traversal. ```lisp (rest '(10 20 30)) ; => (20 30) ``` --- # FILE: references/reference/functions/restart-name.md # restart-name `(restart-name restart)` The name (a symbol or keyword) of a restart object obtained from [`find-restart`](find-restart.md) or [`compute-restarts`](compute-restarts.md). ```lisp (restart-case (restart-name (find-restart :reconnect)) (:reconnect () nil)) ; => :RECONNECT ``` --- # FILE: references/reference/functions/revappend.md # revappend `(revappend list tail)` Returns a new list consisting of the elements of `list` in reverse order followed by `tail`. It is equivalent to `(append (reverse list) tail)` but done in one pass. `list` is copied (not modified); `tail` is shared with the result, becoming its final segment. ```lisp (revappend '(1 2 3) '(4 5)) ; => (3 2 1 4 5) ``` --- # FILE: references/reference/functions/reverse.md # reverse `(reverse sequence)` Returns a new sequence with the elements of `sequence` in reverse order, leaving the original untouched. The sequence may be a list or a string; a string reverses to a new string. This is the non-destructive counterpart to `nreverse`. The empty list reverses to `nil`. ```lisp (reverse '(1 2 3)) ; => (3 2 1) ``` ```lisp (reverse "abc") ; => "cba" ``` --- # FILE: references/reference/functions/rontolisp-alist-hash-table.md # rontolisp:alist-hash-table `(rontolisp:alist-hash-table alist &rest hash-table-initargs)` Builds a hash table from an association list — each `(key . value)` cons becomes an entry, and the first occurrence of a key wins — passing any trailing arguments on to `make-hash-table`. A lightweight subset of `alexandria:alist-hash-table`, so a program can switch to alexandria unchanged. It pairs with [`rontolisp:json-stringify`](rontolisp-json-stringify.md) for turning an alist (like a [`rontolisp:query-params`](rontolisp-query-params.md) result or the request headers) into a JSON object. ```lisp (rontolisp:json-stringify (rontolisp:alist-hash-table '(("n" . 1)))) ; => "{\"n\":1}" ``` The default hash-table test is `eql`, like `alexandria:alist-hash-table`; pass `:test 'equal` for string keys that should dedup by content: ```lisp (hash-table-count (rontolisp:alist-hash-table '(("a" . 1) ("a" . 2)) :test 'equal)) ; => 1 ``` The inverse is [`rontolisp:hash-table-alist`](rontolisp-hash-table-alist.md). ## Backend support Works on every backend and in every WASM mode (Preview 1 included): the function is written in rontolisp itself (part of the prelude) and is compiled into the program when used. --- # FILE: references/reference/functions/rontolisp-alist-plist.md # rontolisp:alist-plist `(rontolisp:alist-plist alist)` Returns a property list holding the same keys and values as the association list `alist`, in the same order — the inverse of [`rontolisp:plist-alist`](rontolisp-plist-alist.md). A lightweight subset of `alexandria:alist-plist`, so a program can switch to alexandria unchanged. ```lisp (rontolisp:alist-plist '((:a . 1) (:b . 2))) ; => (:A 1 :B 2) ``` Unlike [`rontolisp:hash-table-plist`](rontolisp-hash-table-plist.md) there is no hash table in between, so the order is the input's — deterministic on every backend — and duplicate keys are kept rather than collapsed. An empty list returns `nil`. ## Backend support Works on every backend and in every WASM mode (Preview 1 included): the function is written in rontolisp itself (part of the prelude) and is compiled into the program when used. --- # FILE: references/reference/functions/rontolisp-catch.md # rontolisp:catch `(rontolisp:catch future handler)` Returns a fresh future that, on the input's error settlement, invokes `handler` on the condition and settles to the handler's return value; on successful settlement the value passes through unchanged. If the handler itself signals, the returned future carries THAT condition. ```lisp (rontolisp:async-defun boom () (error "nope")) (rontolisp:await (rontolisp:catch (boom) (lambda (c) (declare (ignore c)) :fallback))) ; => :FALLBACK ``` Use `rontolisp:catch` when the future crosses a boundary as a value and you want a JavaScript `.catch`-style single-handler fallback. Type-dispatch already exists lexically as `(handler-case (rontolisp:await f) (my-err (c) ...))` -- write that instead when the future is right there in the body. A user who wants typed dispatch inside a catch handler writes it explicitly: ```console (rontolisp:catch f (lambda (c) (handler-case (signal c) (my-err (e) ...) (error (e) ...)))) ``` A non-future first argument is a `type-error`. ### Name collision with `cl:catch` Common Lisp's [`catch`](../special-forms/catch.md) / [`throw`](../special-forms/throw.md) is a tag-based non-local exit special form. This operator is `rontolisp:catch`, in a different package: qualified names never collide (`cl:catch` still names the CL special form). A user in `cl-user` (or in a package that `:use`s both) needs the explicit `rontolisp:` / `rl:` prefix to get this operator, and the explicit `cl:` prefix (or an unqualified bare name in `cl-user`) to get the tag-based special form. Inside `(in-package :rontolisp)` a bare `catch` is neither: the name belongs to `cl`, which that package does not `:use`, so it is an "Undefined symbol: CATCH (use CL:CATCH)" error until you qualify it. ## Backend support Same as [`rontolisp:then`](rontolisp-then.md): interpreter, JVM, WASM `--component`. Preview 1 WASM supports the success passthrough only (the error path there needs the futured error-at-await contract that the component backend provides). `--no-gc` rejects at compile time. --- # FILE: references/reference/functions/rontolisp-current-thread.md # rontolisp:current-thread `(rontolisp:current-thread)` Returns the calling thread's own **opaque thread handle**. It works for any thread — the main thread and served requests included, not only [`rontolisp:make-thread`](rontolisp-make-thread.md) spawns — and it is `eq`-stable: repeated calls from one thread return the same handle, so it can key an `eq` hash table. That property is what the `bt2:current-thread` shim (and through it `dbi`'s per-thread connection cache) relies on. ```lisp (let ((h (rontolisp:current-thread))) (list (rontolisp:threadp h) (eq h (rontolisp:current-thread)) (rontolisp:thread-alive-p h))) ; => (T T T) ``` Threads are real on the interpreter and the JVM backend. Both WASM backends are single-threaded by construction and do not compile this function; the `bordeaux-threads`/`bt2` shim's `current-thread` signals a clear error there at call time. ## Limitations - The handle a spawned function sees for itself is its own cached one, not the handle its spawner got from `make-thread` — only the [`rontolisp:threadp`](rontolisp-threadp.md) / [`rontolisp:thread-alive-p`](rontolisp-thread-alive-p.md) answers are portable on a handle either way. - Passing your own handle to [`rontolisp:join-thread`](rontolisp-join-thread.md) blocks forever (joining yourself does upstream too). - These primitives have no function value: `#'rontolisp:current-thread` is an error. --- # FILE: references/reference/functions/rontolisp-destroy-thread.md # rontolisp:destroy-thread `(rontolisp:destroy-thread thread)` Interrupts the thread behind the handle and returns the handle. A thread blocked in a waiting operation unblocks with an error there; like Java's `Thread.interrupt`, a body that never blocks may run to completion anyway — this is a request, not a kill. ```lisp (let ((th (rontolisp:make-thread (lambda () 1)))) (rontolisp:join-thread th) (rontolisp:threadp (rontolisp:destroy-thread th))) ; => T ``` ## Limitations - Delivery is asynchronous: `thread-alive-p` may still answer `t` for a moment after this returns. - A value that is not a thread handle is an error. - Interpreter and JVM backend only, like [`rontolisp:make-thread`](rontolisp-make-thread.md) itself. --- # FILE: references/reference/functions/rontolisp-fetch.md # rontolisp:fetch `(rontolisp:fetch url &optional options)` Starts an outgoing HTTP request, modeled on the JavaScript `fetch` API, and immediately returns a **future** while the request runs asynchronously. A future is an opaque value (it prints as `#` and satisfies [`rontolisp:futurep`](rontolisp-futurep.md)); pass it to [`rontolisp:await`](../special-forms/rontolisp-await.md) to suspend until the response arrives and obtain the result property list `(:status :headers :body )`; the `:body` stream is drained with [`rontolisp:read-all`](rontolisp-read-all.md). ```lisp (let ((p (rontolisp:fetch "https://httpbin.ik.am/get"))) (getf (rontolisp:await p) :status)) ; => 200 ``` Because the request is already in flight when `fetch` returns, several requests can overlap: ```console (let ((p1 (rontolisp:fetch "https://httpbin.ik.am/status/200")) (p2 (rontolisp:fetch "https://httpbin.ik.am/status/201"))) ; both requests running (list (rontolisp:await p1) (rontolisp:await p2))) ``` ## Options The optional second argument is an options property list. Recognized keys: - `:method` — the HTTP method as a string (default `"GET"`). Supported methods are `GET`, `HEAD`, `POST`, `PUT`, `DELETE`, `OPTIONS` and `PATCH`, matched case-insensitively; any other method is an error. - `:headers` — request headers, an alist of `(name . value)` string pairs. - `:body` — the request body as a string (omit for no body). Every request carries `User-Agent: rontolisp/ ()` — the version and abbreviated commit [`rontolisp:version`](rontolisp-version.md) reports, the commit omitted when the build had no git repository — unless `:headers` already names that field (matched case-insensitively, so your own spelling and value win). The browser playground and a `--host-fetch` reactor leave the field to the host, which owns it there. The options are validated when `fetch` is called (like the JavaScript `fetch`, which throws synchronously on invalid arguments). ```console ;; GET with request headers (an alist of (name . value) string pairs) (rontolisp:fetch "https://httpbin.ik.am/get" '(:headers (("Accept" . "application/json")))) ;; POST with a request body (rontolisp:fetch "https://httpbin.ik.am/post" '(:method "POST" :headers (("Content-Type" . "application/json")) :body "{\"name\":\"rontolisp\"}")) ``` ## Result `fetch` itself returns the future. Awaiting it yields the property list `(:status :headers :body )`, where `:headers` is an alist of `(name . value)` response-header pairs and `:body` is an **asynchronous stream** of the body's octet chunks (`(unsigned-byte 8)` vectors, the bytes as they arrive) — drain it to one decoded string with [`rontolisp:read-all`](rontolisp-read-all.md), take the chunks one at a time with [`rontolisp:stream-read`](rontolisp-stream-read.md), or answer the stream itself as a served response body to relay the reply byte-exact: ```console (let ((res (rontolisp:await (rontolisp:fetch "https://httpbin.ik.am/get")))) (print (getf res :status)) ; => 200 (print (rontolisp:await (rontolisp:read-all (getf res :body)))) ; => "{...}" (print (getf res :headers))) ; => (("content-type" . "application/json") ...) ``` `:body` is that stream on every backend; on the JVM (whose client takes the whole reply at once) it holds one chunk. A JSON response body parses into Lisp values with [`rontolisp:json-parse`](rontolisp-json-parse.md), and [`rontolisp:json-stringify`](rontolisp-json-stringify.md) builds a JSON request `:body` from an s-expression. ## Backend support - **Interpreter** and **JVM**: use the JDK `java.net.http.HttpClient`; the request runs on a background thread from the moment `fetch` returns. - **WASM**: component-only, over the async `wasi:http@0.3.0` — fetch is ordinary Lisp glue calling the wit-imported `wasi:http/client@0.3.0`, so the component is uniformly WASI 0.3. The future wraps the in-flight async `client.send` subtask, so multiple requests genuinely overlap. Compile with `--component` and run with `wasmtime run -W gc=y -W exceptions=y -S http=y` (wasmtime 46+; `-S http=y` makes the host provide `wasi:http`). fetch remains a compile error in Preview 1 (core-module) mode, which has no host `wasi:http`; the generic future operations (`await`, `then`, `futurep`) compile in every mode. fetch also works inside a [`rontolisp:http-handler`](rontolisp-http-handler.md) serve component (a proxy-style handler): run it with `wasmtime serve -W gc=y -W exceptions=y` — the serve host provides `wasi:http/client` by default, no `-S http=y` needed. - **`--no-wasi` reactor with `--host-fetch`**: the same source compiles on a reactor (which imports no WASI), lowered onto two injected host imports — `env.fetch(request-json) -> response-head-json` and `env.readResponseBody(ptr, cap) -> i32` — over the host's own HTTP client (a Cloudflare Worker's `fetch` behind JSPI, or any synchronous implementation). The result plist is the same, `:body` the same asynchronous stream: the head arrives with the call and the body is pulled a chunk at a time as the drain asks for it, so a large or binary reply never becomes a JSON string. The future is settled at creation (the host call blocked the stack until the headers): requests never overlap, and a transport failure before the head signals at the `fetch` call rather than at `await` — one during the body signals at the drain, as on every other backend. Without the flag, `--no-wasi` keeps the compile error. - **Browser playground**: truly asynchronous. The interpreter runs in a Web Worker; `fetch` hands the request to the page's main thread, which runs the real browser `fetch()` (subject to CORS) while the program continues, so requests overlap, and `await` blocks the worker until the response arrives. When cross-origin isolation is unavailable (`SharedArrayBuffer` disabled) the playground falls back to a synchronous request per fetch — programs behave the same, requests simply do not overlap. ## Limitations - The method must be one of `GET`, `HEAD`, `POST`, `PUT`, `DELETE`, `OPTIONS`, `PATCH`. An unsupported `:method` is an error: the interpreter and JVM reject it at `fetch` time; the WASM backend resolves the method statically and rejects a statically-known unsupported `:method` at compile time (a method computed at runtime cannot be checked there and is treated as GET, while a runtime-computed `:body` is sent normally). - A failed request (for example a refused connection) surfaces when the future is awaited — the same timing as a JavaScript `await` rejection: every backend signals an error there (on WASM it is a `rontolisp:wit-error` condition, catchable with `handler-case`). A request that cannot even be *started* (for example a malformed URL, or an unsupported runtime-computed method on the interpreter/JVM) makes `fetch` itself error or, on WASM, return `nil` instead of a future — and awaiting `nil` yields `nil`. --- # FILE: references/reference/functions/rontolisp-finally.md # rontolisp:finally `(rontolisp:finally future thunk)` Returns a fresh future carrying the input's original settlement (either value or condition) and runs the zero-argument `thunk` exactly once on whichever outcome the future produces. The thunk's return value is discarded; a condition raised inside the thunk **replaces** the pending outcome (matches `unwind-protect`). ```lisp (defvar *cleanup-log* nil) (rontolisp:async-defun produce () 5) (let ((v (rontolisp:await (rontolisp:finally (produce) (lambda () (push :done *cleanup-log*)))))) (list v (reverse *cleanup-log*))) ; => (5 (:DONE)) ``` Use it to run a cleanup step (release a resource, log a metric, decrement a counter) that must fire on both the success and the error channels of a future you receive from a callee. A non-future first argument is a `type-error`. ## Backend support Same as [`rontolisp:then`](rontolisp-then.md): interpreter, JVM, WASM `--component`. Preview 1 WASM supports the success shape (the error arm would need the futured error-at-await contract that the component backend provides). `--no-gc` rejects at compile time. --- # FILE: references/reference/functions/rontolisp-futurep.md # rontolisp:futurep `(rontolisp:futurep value)` Returns `t` if `value` is a future — as returned by calling an [`rontolisp:async-defun`](../special-forms/rontolisp-async-defun.md) function, [`rontolisp:fetch`](rontolisp-fetch.md) or [`rontolisp:stream-read`](rontolisp-stream-read.md) — and `nil` otherwise. ```lisp (rontolisp:async-defun f () 1) (rontolisp:futurep (f)) ; => T (rontolisp:futurep 42) ; => NIL ``` A future is an opaque value: it has no reader syntax and prints as `#`. Its settled value is obtained with [`rontolisp:await`](../special-forms/rontolisp-await.md). ```lisp (f) ; => # ``` --- # FILE: references/reference/functions/rontolisp-hash-table-alist.md # rontolisp:hash-table-alist `(rontolisp:hash-table-alist table)` Returns an association list of the hash table's key/value pairs — the inverse of [`rontolisp:alist-hash-table`](rontolisp-alist-hash-table.md). A lightweight subset of `alexandria:hash-table-alist`. ```lisp (rontolisp:hash-table-alist (rontolisp:alist-hash-table '(("k" . 7)))) ; => (("k" . 7)) ``` The pair order follows the table's iteration order (backend-specific, like `maphash`), so it is well defined for a single-entry table. ## Backend support Works on every backend and in every WASM mode (Preview 1 included): the function is written in rontolisp itself (part of the prelude) and is compiled into the program when used. --- # FILE: references/reference/functions/rontolisp-hash-table-plist.md # rontolisp:hash-table-plist `(rontolisp:hash-table-plist table)` Returns a property list of the hash table's key/value pairs — the inverse of [`rontolisp:plist-hash-table`](rontolisp-plist-hash-table.md). A lightweight subset of `alexandria:hash-table-plist`. ```lisp (rontolisp:hash-table-plist (rontolisp:plist-hash-table (list :a 1))) ; => (:A 1) ``` The pair order follows the table's iteration order (backend-specific, like `maphash`), so it is well defined for a single-entry table; for a [`rontolisp:json-parse`](rontolisp-json-parse.md) object the keys are strings, which `getf` cannot look up (it compares with `eq`), so read those with `gethash` instead. ## Backend support Works on every backend and in every WASM mode (Preview 1 included): the function is written in rontolisp itself (part of the prelude) and is compiled into the program when used. --- # FILE: references/reference/functions/rontolisp-http-handler.md # rontolisp:http-handler `(rontolisp:http-handler handler &optional port &key raw-body)` Serves HTTP requests with a Lisp handler function. `handler` is a quoted symbol naming a one-argument function (like [`rontolisp:wasm-export`](rontolisp-wasm-export.md)). The handler receives the Clack environment property list and returns the Clack response list `(status headers body)` — a Clack application is a valid handler as is (see [Clack Web Applications](../../guides/clack.md)): - **environment** — a property list with exactly these keys, always all present: `:request-method` (an upcased interned keyword, `:GET` / `:POST` / ..., so `(eq m :POST)` works), `:script-name` (always `""`), `:path-info` (the percent-decoded path), `:query-string` (the raw text after the first `?`, or `nil` — parse it with [`rontolisp:query-param`](rontolisp-query-param.md) / [`rontolisp:query-params`](rontolisp-query-params.md)), `:server-name`, `:server-port` (an integer), `:server-protocol` (a keyword, e.g. `:HTTP/1.1`), `:request-uri` (the raw request target verbatim, still encoded, query included), `:url-scheme` (`"http"`/`"https"`), `:remote-addr` / `:remote-port` (the real peer on the interpreter/JVM; `nil` on the WASI component), `:headers` (an `equal` hash table keyed by lowercased names, repeated headers joined with `", "`, never `nil` — `(gethash "content-type" (getf env :headers))`), `:content-type` and `:content-length` (string / integer, or `nil`), and `:raw-body`. - **`:raw-body`** — by default (`:raw-body :stream`) an asynchronous stream; a handler that reads it drains it with `(rontolisp:await (rontolisp:read-all (getf env :raw-body)))` and must be an [`rontolisp:async-defun`](../special-forms/rontolisp-async-defun.md). With the directive argument `(rontolisp:http-handler 'handle 8080 :raw-body :buffered)` the body is instead read in full and handed over as a synchronous in-memory bivalent stream readable with `read-line`/`read-char` and `read-byte`/`read-sequence`, with a real `file-position` — what a Clack application (lack-request, http-body) needs; a bodiless request then gets `:raw-body nil`. - **response** — the positional list `(status headers body)`. `status` is a required integer (a non-integer car signals an error). `headers` is a keyword plist (`'(:content-type "text/plain")`) or a dotted alist (so a [`rontolisp:fetch`](rontolisp-fetch.md) result's `:headers` passes straight through); repeated names each become their own header line, `content-length` / `transfer-encoding` are dropped (the server computes them), and `nil` is fine. `body` is a list of strings (joined), `nil`/omitted (an empty body — the two-element `(status headers)` form is valid), an `(unsigned-byte 8)` vector, or a rontolisp stream (e.g. a proxied fetch body) drained by the server; a **bare string signals an error** (a rontolisp pathname is its namestring, and in Clack a pathname body means "serve this file"). A function response is supported in Clack's delayed form only — `(lambda (responder) ... (funcall responder (list 200 nil (list "later"))))` — and the streaming-writer form is refused. On the **interpreter** and **JVM** backends `http-handler` starts a blocking embedded HTTP server on `port` (default `8080`, one virtual thread per request) and serves until the process is stopped (Ctrl-C). Compiled to a **WASI component** (`--component`) it instead exports `wasi:http/handler@0.3.0`, so the module runs as a serverless HTTP component under `wasmtime serve` (the `port` argument is ignored — the host owns the socket). ```console (defun handle (env) (list 200 '(:content-type "text/plain") (list (format nil "Hello from rontolisp!~%~a ~a~%" (getf env :request-method) (getf env :path-info))))) (rontolisp:http-handler 'handle 8080) ``` Run it on the interpreter, then talk to it with `curl`: ```console $ rontolisp app.lisp $ curl http://127.0.0.1:8080/hello Hello from rontolisp! GET /hello ``` Compile it to a JVM class (the class implements the embedded server's handler interface, so the rontolisp executable JAR must be on the classpath when running it — this is the one step that needs the JAR instead of the native binary): ```console $ rontolisp app.lisp -o App.class $ java -cp rontolisp-0.1.0-SNAPSHOT-exec.jar:. App $ curl http://127.0.0.1:8080/hello Hello from rontolisp! GET /hello ``` Or compile it to a WASI HTTP component and serve it with `wasmtime serve`: ```console $ rontolisp app.lisp -o app.wasm --component $ wasmtime serve -W gc=y -W exceptions=y app.wasm $ curl http://127.0.0.1:8080/hello Hello from rontolisp! GET /hello ``` ## Backend support `http-handler` runs on the **interpreter** backend (a blocking server), the **JVM** backend (the same blocking server; the compiled class needs the rontolisp executable JAR, `rontolisp-0.1.0-SNAPSHOT-exec.jar`, on the classpath) and the **WASI component** backend (`--component`, a `wasi:http/handler@0.3.0` component for `wasmtime serve`). Request and response headers are marshalled on every backend, the WASI component included: the handler reads `:headers` (an `equal` hash table keyed by lowercased names) and the response's `headers` element is written back. Inside a served handler `random`, the time built-ins and `print` (to the host's stdout) work — they are bridged to `wasi:random` / `wasi:clocks` / `wasi:cli`, which every `wasi:http` host provides; `uiop:getenv` reads the host environment through the component's own `wasi:cli/environment@0.3.0` import (`wasmtime serve --env NAME=value` or `-S inherit-env=y`), and file streams are unavailable. [`rontolisp:fetch`](rontolisp-fetch.md) also works inside a served handler — serve and serve+fetch are one component shape, whose `wasi:http/client@0.3.0` import `wasmtime serve` provides by default — so proxy-style handlers run on every backend with the same serve command. A handler that awaits (fetch inside serve, say) is an asynchronous function and must be defined with [`rontolisp:async-defun`](../special-forms/rontolisp-async-defun.md) rather than `defun`: `rontolisp:await` is legal only inside asynchronous bodies. A handler may also call a WIT interface of its own with [`rontolisp:wit-import`](rontolisp-wit-import.md), which the served component imports alongside its fixed `wasi:http` surface. That is how a served handler keeps **state**: a `wasi:http` host instantiates the component afresh for every request, so a global hash table reads back empty every time, while a `wasi:keyvalue` store lives outside it. The serve component targets the async `wasi:http@0.3.0` (`service` world); its handler is a callback async lift over the base component-model async ABI, default-on in wasmtime 46+, so `wasmtime serve` needs no gated feature flags. wasmCloud hosts it too: the released `wash` (2.5.2) runs it with `wash dev`, given `dev.wasm_proposals: [gc, exception-handling, component-model-async]`. So does **Spin**, from the [canary build](https://github.com/spinframework/spin/releases/tag/canary) (4.1.0-pre0) on, with a plain `spin.toml` and no flags; released Spin 4.0.2 cannot, because its wasmtime 44 speaks the `wasi:http@0.3.0-rc-2026-03-15` snapshot instead of the released 0.3.0. jco does not implement the 0.3 async ABI. See the [Serving HTTP guide](../../guides/http-handler.md) for the full example and the per-runtime commands. --- # FILE: references/reference/functions/rontolisp-join-thread.md # rontolisp:join-thread `(rontolisp:join-thread thread)` Blocks until the thread's function returns, waits for the thread itself to die, and yields the function's value. If the thread died signalling an error, the error is re-signaled in the joining thread, so a `handler-case` around the join dispatches by condition type exactly like a same-thread signal. ```lisp (rontolisp:join-thread (rontolisp:make-thread (lambda () (+ 40 2)))) ; => 42 ``` After a join, [`rontolisp:thread-alive-p`](rontolisp-thread-alive-p.md) on the same handle answers `nil`. ## Limitations - Interpreter and JVM backend only, like [`rontolisp:make-thread`](rontolisp-make-thread.md) itself. - A value that is not a thread handle is an error. - There is no timed join. --- # FILE: references/reference/functions/rontolisp-json-parse.md # rontolisp:json-parse `(rontolisp:json-parse string)` Parses a JSON document string into Lisp values, following the defaults of the [`com.inuoe.jzon`](../../guides/asdf-systems.md) library: a JSON object becomes a hash table with string keys, an array a vector, and `true`/`false`/`null` become `t`, `nil` and the symbol `null`. `rontolisp:json-parse` is a lightweight subset of jzon, so a program can start here and later switch to jzon without changing shape — with a single deliberate exception, the wide-integer rule [noted below](#the-one-incompatibility-with-jzon). Switch to `com.inuoe.jzon` when you outgrow the subset: for its richer features (pretty-printing, a streaming writer, a `:replacer`, custom serialization), or to make the JSON code portable to other Common Lisp implementations — `com.inuoe.jzon` is a standard library, while `rontolisp:json-*` runs only on rontolisp. ```lisp (gethash "name" (rontolisp:json-parse "{\"name\": \"rontolisp\", \"n\": 2}")) ; => "rontolisp" (gethash "b" (gethash "a" (rontolisp:json-parse "{\"a\": {\"b\": [1, true, null]}}"))) ; => #(1 T NULL) ``` ## Value mapping | JSON | Lisp | |------|------| | object | hash table with string keys (`equal` test) | | array | vector | | string | string (`\uXXXX` escapes and surrogate pairs are decoded) | | number | integer, or float when it has a fraction, an exponent or more than 18 digits | | `true` | `t` | | `false` | `nil` | | `null` | the symbol `null` | ```lisp (rontolisp:json-parse "[1, 2.5, \"x\", false, null]") ; => #(1 2.5 "x" NIL NULL) (rontolisp:json-parse "1e3") ; => 1000.0 (rontolisp:json-parse "\"a\\u3042b\"") ; => "aあb" ``` ### The one incompatibility with jzon Integers wider than 18 digits become floats on every backend -- a shared library rule that keeps the parse identical across all backends. jzon instead keeps them as exact integers of any width, so this is the single point where `rontolisp:json-parse` and `jzon:parse` disagree — a 13-digit millisecond timestamp parses exactly on both, but a 19-digit integer parses as a float here and as an exact integer under jzon. Everything else round-trips identically. ```lisp (rontolisp:json-parse "1234567890123") ; => 1234567890123 (floatp (rontolisp:json-parse "1234567890123456789")) ; => T ``` ## Errors Invalid JSON and trailing characters after the value signal an error when `json-parse` is called: ```console > (rontolisp:json-parse "{\"a\": ") Error: json-parse: unexpected end of input > (rontolisp:json-parse "1 2") Error: json-parse: unexpected trailing characters ``` ## Limitations - A JSON object always parses to a hash table, so `{}` (an empty hash table) is distinct from `false`/`nil`, from an empty array `#()`, and from the `null` symbol — unlike JavaScript, the four are never conflated. - On the WASM backends a float with magnitude 2³¹ or larger parses correctly but cannot be *printed* (`print`/`princ-to-string` trap); see the [WASM guide](../../compiling/wasm.md). ## Backend support Works on every backend and in every WASM mode (Preview 1 included): the parser is written in rontolisp itself and is compiled into the program when used. The typical use is parsing a [`rontolisp:fetch`](rontolisp-fetch.md) response body: ```console (print (gethash "url" (rontolisp:json-parse (getf (rontolisp:await (rontolisp:fetch "https://httpbin.ik.am/get")) :body)))) ; "https://httpbin.ik.am/get" ``` The inverse operation is [`rontolisp:json-stringify`](rontolisp-json-stringify.md). --- # FILE: references/reference/functions/rontolisp-json-stringify.md # rontolisp:json-stringify `(rontolisp:json-stringify value)` Serializes a Lisp value into a JSON document string, following the defaults of the [`com.inuoe.jzon`](../../guides/asdf-systems.md) library and inverting [`rontolisp:json-parse`](rontolisp-json-parse.md): a hash table becomes an object, a vector or list an array, and `nil`, `t` and the symbol `null` become `false`, `true` and `null`. It is a lightweight subset of jzon, so a program can switch to jzon without changing shape. Switch to `com.inuoe.jzon` when you outgrow the subset: for its richer features (pretty-printing, a streaming writer, a `:replacer`, custom serialization), or to make the JSON code portable to other Common Lisp implementations — `com.inuoe.jzon` is a standard library, while `rontolisp:json-*` runs only on rontolisp. ```lisp (rontolisp:json-stringify (vector 1 2 3)) ; => "[1,2,3]" (rontolisp:json-stringify (list 1 (list 2 3) nil)) ; => "[1,[2,3],false]" (let ((h (make-hash-table :test 'equal))) (setf (gethash "name" h) "rontolisp") (rontolisp:json-stringify h)) ; => "{\"name\":\"rontolisp\"}" ``` ## Value mapping | Lisp | JSON | |------|------| | `nil` | `false` | | `t` | `true` | | the symbol `null` | `null` | | integer, float | number | | ratio | number (converted with `float`) | | string | string (quote, backslash and control characters are escaped) | | vector, list | array | | hash table | object (a symbol key is down-cased unless it has a lower-case letter) | | CLOS instance (`standard-object`) | object (each slot name → its value, in definition order) | | keyword, symbol, character | string | Anything else (functions, streams, multidimensional arrays) signals an error. A hash table and a CLOS instance both serialize as objects, so there are two ways to build one — a hash table (often via [`rontolisp:plist-hash-table`](rontolisp-plist-hash-table.md)) for dynamic keys, and a class for a fixed shape. A slot may itself hold a hash table (a nested object), a list or vector (an array), or another instance: ```lisp (defclass response () ((status :initarg :status) (body :initarg :body))) (let ((h (make-hash-table :test 'equal))) (setf (gethash "content-type" h) "text/plain") (rontolisp:json-stringify (make-instance 'response :status 200 :body h))) ; => "{\"status\":200,\"body\":{\"content-type\":\"text/plain\"}}" ``` ```lisp (rontolisp:json-stringify :key) ; => "\"KEY\"" (rontolisp:json-stringify 3/2) ; => "1.5" (rontolisp:json-stringify "a\"b") ; => "\"a\\\"b\"" ``` A value parsed from JSON round-trips structurally: ```lisp (rontolisp:json-stringify (rontolisp:json-parse "{\"deep\": {\"list\": [{\"k\": \"v\"}, 2.5, true]}}")) ; => "{\"deep\":{\"list\":[{\"k\":\"v\"},2.5,true]}}" ``` ## Limitations - `nil` serializes as `false` and the empty list is `nil`, so use `#()` (an empty vector) for an empty array and an empty hash table for an empty object `{}`. - A list is always an array — build a hash table for a JSON object (jzon dropped alist/plist detection, and so does this subset). [`rontolisp:plist-hash-table`](rontolisp-plist-hash-table.md) turns a keyword property list, and [`rontolisp:alist-hash-table`](rontolisp-alist-hash-table.md) an association list, into that hash table. - Hash-table key order in the output is backend-specific (unspecified), like `maphash`. - Non-ASCII characters are emitted verbatim (never `\uXXXX`-escaped), which is valid JSON. - On the WASM backends a float with magnitude 2³¹ or larger cannot be serialized (the float formatter traps); see the [WASM guide](../../compiling/wasm.md). ## Backend support Works on every backend and in every WASM mode (Preview 1 included), like [`rontolisp:json-parse`](rontolisp-json-parse.md): the serializer is written in rontolisp itself and is compiled into the program when used. --- # FILE: references/reference/functions/rontolisp-list-functions.md # rontolisp:list-functions `(rontolisp:list-functions &optional package)` Returns the function symbols of a package, sorted alphabetically. The optional package designator is a keyword, a bare symbol, a quoted symbol, or a string (`:cl`, `cl`, `'cl`, `"cl"`) and defaults to `:cl`. A name is listed as a function exactly when it is usable as a function value via `#'name`. For `:cl-user` it lists the user-defined `defun`s. An unknown package is an error. See [Package introspection](../packages.md#package-introspection) for details. ```lisp (rontolisp:list-functions :rontolisp) ; => (AWAIT CATCH FETCH FINALLY HTTP-HANDLER JSON-PARSE JSON-STRINGIFY LIST-FUNCTIONS LIST-MACROS LIST-SPECIAL-FORMS MAKE-MUTEX MUTEX-ACQUIRE MUTEX-RELEASE QUERY-PARAM QUERY-PARAMS RANDOM-BYTES TCP-ACCEPT TCP-CONNECT TCP-LISTEN TCP-LOCAL-ADDRESS TCP-LOCAL-PORT TCP-PEER-ADDRESS TCP-PEER-PORT TCP-SET-TIMEOUT THEN THEN* TLS-CONNECT TLS-LISTEN TLS-LISTEN-PEM TLS-UPGRADE URL-DECODE URL-ENCODE URL-PATH URL-QUERY VERSION WIT-ERROR-PAYLOAD WIT-PROVIDE) ``` --- # FILE: references/reference/functions/rontolisp-list-macros.md # rontolisp:list-macros `(rontolisp:list-macros &optional package)` Returns the macro symbols of a package, sorted alphabetically -- the operators that have no function value. The optional package designator defaults to `:cl` and may be a keyword, a bare symbol, a quoted symbol, or a string. An unknown package is an error. See [Package introspection](../packages.md#package-introspection) for details. ```lisp (rontolisp:list-macros) ; => (AND ASSERT BLOCK CASE CCASE CERROR CHANGE-CLASS CHECK-TYPE COMPLEMENT COMPLEX COND CTYPECASE DECF DECLAIM DECLARE DEFINE-COMPILER-MACRO DEFINE-CONDITION DEFINE-MODIFY-MACRO DEFINE-SETF-EXPANDER DEFSETF DEFTYPE DESTRUCTURING-BIND DO DO* DO-EXTERNAL-SYMBOLS DO-SYMBOLS DOCUMENTATION DOLIST DOTIMES ECASE ERROR ETYPECASE EVAL-WHEN FLET FORMAT HANDLER-BIND HANDLER-CASE IGNORE-ERRORS INCF LABELS LET* LOAD-TIME-VALUE LOCALLY LOOP MACROLET MAKE-CONDITION MAKE-INSTANCE MAKE-SEQUENCE MULTIPLE-VALUE-BIND MULTIPLE-VALUE-CALL MULTIPLE-VALUE-LIST MULTIPLE-VALUE-PROG1 MULTIPLE-VALUE-SETQ NTH-VALUE OR POP PPRINT-LOGICAL-BLOCK PRINT-UNREADABLE-OBJECT PROCLAIM PROG PROG* PROG1 PROG2 PSETF PSETQ PUSH PUSHNEW REMF RESTART-BIND RESTART-CASE RETURN-FROM ROTATEF SETF SHIFTF SIGNAL SLOT-BOUNDP SLOT-EXISTS-P SLOT-MAKUNBOUND SLOT-VALUE SYMBOL-MACROLET THE TIME TYPECASE TYPEP UNLESS WARN WHEN WITH-ACCESSORS WITH-COMPILATION-UNIT WITH-INPUT-FROM-STRING WITH-OPEN-FILE WITH-OPEN-STREAM WITH-OUTPUT-TO-STRING WITH-PACKAGE-ITERATOR WITH-SIMPLE-RESTART WITH-SLOTS WITH-STANDARD-IO-SYNTAX WRITE-CHAR) ``` --- # FILE: references/reference/functions/rontolisp-list-special-forms.md # rontolisp:list-special-forms `(rontolisp:list-special-forms &optional package)` Returns the special-form symbols of a package, sorted alphabetically -- the operators evaluated specially that have no function value. The optional package designator defaults to `:cl` and may be a keyword, a bare symbol, a quoted symbol, or a string. An unknown package is an error. See [Package introspection](../packages.md#package-introspection) for details. ```lisp (rontolisp:list-special-forms) ; => (CATCH DEFCLASS DEFCONSTANT DEFGENERIC DEFMACRO DEFMETHOD DEFPACKAGE DEFPARAMETER DEFSTRUCT DEFUN DEFVAR FUNCTION GO IF IN-PACKAGE LAMBDA LET PROGN PROGV QUOTE RETURN SETQ TAGBODY THROW UNWIND-PROTECT WHILE) ``` --- # FILE: references/reference/functions/rontolisp-make-mutex.md # rontolisp:make-mutex `(rontolisp:make-mutex)` Returns a fresh mutual-exclusion lock, as an **opaque handle**. Pass it to [`rontolisp:with-mutex`](../macros/rontolisp-with-mutex.md) (or to [`rontolisp:mutex-acquire`](rontolisp-mutex-acquire.md) / [`rontolisp:mutex-release`](rontolisp-mutex-release.md)) and to nothing else: what the handle actually is differs per backend, so printing one, comparing two with `<`, or doing arithmetic on it is not portable. Comparing a handle with `eq`/`eql` to itself does work. rontolisp really runs concurrent code — [`rontolisp:http-handler`](rontolisp-http-handler.md) puts one virtual thread per request on the interpreter and the JVM backend, and [`rontolisp:make-thread`](rontolisp-make-thread.md) lets your own code spawn one — which is what a lock is for. On both WASM backends there is only ever one thread, so the primitives are no-ops there; the same source runs everywhere. ```lisp (let ((m (rontolisp:make-mutex))) (rontolisp:with-mutex (m) :guarded)) ; => :GUARDED ``` The lock is **reentrant**: the thread holding it may acquire it again, and must release it as many times as it acquired it. ## Limitations - The handle is opaque and backend-dependent — do not print or order it. - Macros and these primitives have no function value: `#'rontolisp:make-mutex` is an error. --- # FILE: references/reference/functions/rontolisp-make-stream.md # rontolisp:make-stream `(rontolisp:make-stream)` Creates a fresh open asynchronous stream. One value owns both the read and the write end: producers append chunks with [`rontolisp:stream-write`](rontolisp-stream-write.md) and finish with [`rontolisp:stream-close`](rontolisp-stream-close.md); consumers take chunks with [`rontolisp:stream-read`](rontolisp-stream-read.md) (each read yields a future) or drain the string chunks in one go with [`rontolisp:read-all`](rontolisp-read-all.md). ```lisp (let ((s (rontolisp:make-stream))) (rontolisp:stream-write s "hello ") (rontolisp:stream-write s "world") (rontolisp:stream-close s) (rontolisp:await (rontolisp:read-all s))) ; => "hello world" ``` ## Backend support Guest-created streams (`rontolisp:make-stream` / `rontolisp:stream-write`) exist on the interpreter and the JVM backend today; the WASM backends reject them at compile time (a `--component` program's streams come from `rontolisp:fetch` / `rontolisp:http-handler` bodies). --- # FILE: references/reference/functions/rontolisp-make-thread.md # rontolisp:make-thread `(rontolisp:make-thread function &optional bindings)` Spawns a new (virtual) thread running the zero-argument `function` and returns an **opaque thread handle** immediately. Pass the handle to [`rontolisp:join-thread`](rontolisp-join-thread.md), [`rontolisp:thread-alive-p`](rontolisp-thread-alive-p.md) or [`rontolisp:destroy-thread`](rontolisp-destroy-thread.md), and test for one with [`rontolisp:threadp`](rontolisp-threadp.md); like a mutex handle, what it actually is differs per backend, so printing or ordering one is not portable. `bindings` is an alist of `(symbol . value)` pairs, each established as a thread-scoped dynamic binding in the **new** thread before `function` runs. The spawned thread inherits no dynamic bindings from its spawner: without an entry here it reads every special variable's global value. Binding `*standard-output*` this way routes the new thread's print family into a stream of your choice — the shape the `bordeaux-threads`/`bt2` libraries (and Clack's handler) use. ```lisp (defvar *cap* (make-string-output-stream)) (rontolisp:join-thread (rontolisp:make-thread (lambda () (princ "from the thread")) (list (cons '*standard-output* *cap*)))) (get-output-stream-string *cap*) ; => "from the thread" ``` Threads are real on the interpreter and the JVM backend. Both WASM backends are single-threaded by construction and do not compile this function; the `bordeaux-threads`/`bt2` shim's `make-thread` signals a clear error there at call time. ## Limitations - WASM: not available (see above) — a Clack app there runs with `:use-thread nil`. - The bindings' values are used as given; unlike upstream `bordeaux-threads`, there is no form evaluation in the new thread (the `bt2:make-thread` shim accepts `quote` forms and self-evaluating values in `:initial-bindings` and signals on anything else). - These primitives have no function value: `#'rontolisp:make-thread` is an error. --- # FILE: references/reference/functions/rontolisp-mutex-acquire.md # rontolisp:mutex-acquire `(rontolisp:mutex-acquire mutex)` Blocks until the calling thread holds `mutex` (created by [`rontolisp:make-mutex`](rontolisp-make-mutex.md)), then returns the mutex. Prefer [`rontolisp:with-mutex`](../macros/rontolisp-with-mutex.md), which releases the lock even when the body exits by signalling an error; a bare `mutex-acquire` whose matching [`rontolisp:mutex-release`](rontolisp-mutex-release.md) is skipped leaves the lock held forever. The lock is reentrant, so a thread that already holds it acquires it again immediately and must release it once per acquisition. On both WASM backends there is only one thread, so this is a no-op that returns its argument. ```lisp (let ((m (rontolisp:make-mutex))) (rontolisp:mutex-acquire m) (unwind-protect :critical (rontolisp:mutex-release m))) ; => :CRITICAL ``` ## Limitations - There is no non-blocking or timed acquisition. - A value that is not a mutex handle is an error. --- # FILE: references/reference/functions/rontolisp-mutex-release.md # rontolisp:mutex-release `(rontolisp:mutex-release mutex)` Releases one acquisition of `mutex` and returns it. Releasing a mutex the calling thread does not hold is an error on the interpreter and the JVM backend (and unnoticed on WASM, where the primitives are no-ops). Because the lock is reentrant, a thread that acquired it twice must release it twice before another thread can take it. [`rontolisp:with-mutex`](../macros/rontolisp-with-mutex.md) pairs the acquire and the release for you, including on a non-local exit; reach for the bare primitives only when the two cannot sit in one lexical block. ```lisp (let ((m (rontolisp:make-mutex))) (eq (rontolisp:mutex-release (rontolisp:mutex-acquire m)) m)) ; => T ``` ## Limitations - A value that is not a mutex handle is an error. - On the WASM backends nothing is checked: releasing an unheld lock is silently accepted. --- # FILE: references/reference/functions/rontolisp-plist-alist.md # rontolisp:plist-alist `(rontolisp:plist-alist plist)` Returns an association list holding the same keys and values as the property list `plist` — the odd elements are keys, the even elements values — in the same order. The inverse of [`rontolisp:alist-plist`](rontolisp-alist-plist.md), and a lightweight subset of `alexandria:plist-alist`, so a program can switch to alexandria unchanged. ```lisp (rontolisp:plist-alist '(:a 1 :b 2)) ; => ((:A . 1) (:B . 2)) ``` Unlike [`rontolisp:plist-hash-table`](rontolisp-plist-hash-table.md) there is no hash table in between, so the order is the input's — deterministic on every backend — and duplicate keys are kept rather than collapsed: ```lisp (rontolisp:plist-alist '(:a 1 :a 9)) ; => ((:A . 1) (:A . 9)) ``` ## Backend support Works on every backend and in every WASM mode (Preview 1 included): the function is written in rontolisp itself (part of the prelude) and is compiled into the program when used. --- # FILE: references/reference/functions/rontolisp-plist-hash-table.md # rontolisp:plist-hash-table `(rontolisp:plist-hash-table plist &rest hash-table-initargs)` Builds a hash table from a property list — the odd elements are keys, the even elements values — passing any trailing arguments on to `make-hash-table`. A lightweight subset of `alexandria:plist-hash-table`, so a program can switch to alexandria unchanged. It pairs with [`rontolisp:json-stringify`](rontolisp-json-stringify.md) for building JSON objects: keyword keys are down-cased, so `:name` becomes `"name"`. ```lisp (rontolisp:json-stringify (rontolisp:plist-hash-table (list :name "rontolisp"))) ; => "{\"name\":\"rontolisp\"}" ``` Objects with several keys work the same way (the key order in the JSON output is backend-specific, like `maphash`); the table is a real hash table, so its values read back with `gethash`: ```lisp (gethash :ok (rontolisp:plist-hash-table (list :name "x" :ok t))) ; => T ``` The default hash-table test is `eql`, like `alexandria:plist-hash-table`; pass `:test 'equal` (or any `make-hash-table` argument) to change it: ```lisp (gethash "k" (rontolisp:plist-hash-table (list "k" 9) :test 'equal)) ; => 9 ``` The inverse is [`rontolisp:hash-table-plist`](rontolisp-hash-table-plist.md). ## Backend support Works on every backend and in every WASM mode (Preview 1 included): the function is written in rontolisp itself (part of the prelude) and is compiled into the program when used. --- # FILE: references/reference/functions/rontolisp-query-param.md # rontolisp:query-param `(rontolisp:query-param query name)` Returns the url-decoded value of the first `name` match in a query string, or `nil` when the name does not appear. `query` may be `nil` (the result is then `nil` too), so the one-liner `(rontolisp:query-param (getf env :query-string) "name")` works unchanged for requests without a query string inside an [`rontolisp:http-handler`](rontolisp-http-handler.md) handler. ```lisp (rontolisp:query-param "a=1&name=ronto%20lisp" "name") ; => "ronto lisp" (rontolisp:query-param "q=1&q=2" "q") ; => "1" (rontolisp:query-param "a=1" "missing") ; => NIL (rontolisp:query-param nil "a") ; => NIL ``` ## Backend support Works on every backend and in every WASM mode (Preview 1 included): the library is written in rontolisp itself and is compiled into the program when used. To read all parameters at once use [`rontolisp:query-params`](rontolisp-query-params.md). --- # FILE: references/reference/functions/rontolisp-query-params.md # rontolisp:query-params `(rontolisp:query-params query)` Parses a query string such as `"a=1&b=two&flag"` into an alist of `(key . value)` string pairs. Keys and values are url-decoded with [`rontolisp:url-decode`](rontolisp-url-decode.md); a key without `=` gets the value `""`; duplicate keys are preserved in order; empty segments are skipped. `nil` (a request without a query string) yields `nil`, so `(rontolisp:query-params (getf env :query-string))` is always safe inside an [`rontolisp:http-handler`](rontolisp-http-handler.md) handler. ```lisp (rontolisp:query-params "a=1&b=two&flag") ; => (("a" . "1") ("b" . "two") ("flag" . "")) (rontolisp:query-params "q=%E3%81%82&q=2") ; => (("q" . "あ") ("q" . "2")) (rontolisp:query-params nil) ; => NIL ``` The alist prints readably and works with `assoc`: ```lisp (cdr (assoc "b" (rontolisp:query-params "a=1&b=two") :test #'string=)) ; => "two" ``` ## Backend support Works on every backend and in every WASM mode (Preview 1 included): the library is written in rontolisp itself and is compiled into the program when used. For the common "value of one name" lookup use [`rontolisp:query-param`](rontolisp-query-param.md). --- # FILE: references/reference/functions/rontolisp-random-bytes.md # rontolisp:random-bytes `(rontolisp:random-bytes count)` Returns a vector of `count` cryptographically strong random bytes (each an integer 0-255). Unlike [`random`](random.md), which is an ordinary pseudo-random generator, every byte comes from the platform's cryptographic entropy source: `java.security.SecureRandom` on the interpreter and the JVM, the WASI `random_get` host function on both WASM backends (`wasi:random` under `--component`). That is what makes it suitable for nonces, salts and session identifiers. A `--no-wasi` module has no entropy source, so the call signals there unless the build passes `--host-random` ([clock and randomness guide](../../guides/clock-and-random.md)). ```lisp (length (rontolisp:random-bytes 16)) ; => 16 ``` The values differ on every call, so only the length is shown here. --- # FILE: references/reference/functions/rontolisp-read-all.md # rontolisp:read-all `(rontolisp:read-all stream)` Returns a future settling to the remaining chunks of an asynchronous stream drained into **one string**: string chunks (a guest-created stream) are concatenated, and octet chunks -- the `(unsigned-byte 8)` vectors every HTTP body stream answers, a fetched reply's `:body` and a served request's `:raw-body` -- are joined and decoded as UTF-8, so a document-shaped consumer reads text off a byte stream. A stream mixing the two kinds is an error. The future settles once the stream reaches end of stream, so the producer side must eventually call [`rontolisp:stream-close`](rontolisp-stream-close.md). ```lisp (let ((s (rontolisp:make-stream))) (rontolisp:stream-write s "hello ") (rontolisp:stream-write s "world") (rontolisp:stream-close s) (rontolisp:await (rontolisp:read-all s))) ; => "hello world" ``` It is the idiomatic way to drain a [`rontolisp:fetch`](rontolisp-fetch.md) response body: ```console (let ((r (rontolisp:await (rontolisp:fetch "https://example.com")))) (rontolisp:await (rontolisp:read-all (getf r :body)))) ``` To take the chunks one at a time instead, use [`rontolisp:stream-read`](rontolisp-stream-read.md); to forward a body without reading it, answer the stream itself as a response body -- the transport drains it byte-exact, nothing decodes on the way through. A **string** passes straight through (the future settles to the string itself): a body that has already fully arrived is its own drained value, so the one drain spelling above works whatever shape `:body` took. ## Backend support Asynchronous streams exist on the interpreter, the JVM backend and -- for the request/response body streams `rontolisp:fetch` / `rontolisp:http-handler` produce -- the `--component` WASM backend. A Preview 1 WASM module can hold a stream value only when a host-backed body gives it one; where none can exist, `rontolisp:streamp` answers `nil` and `rontolisp:stream-read` / `rontolisp:stream-close` signal an error when called. --- # FILE: references/reference/functions/rontolisp-stream-close.md # rontolisp:stream-close `(rontolisp:stream-close stream)` Closes the write end of an asynchronous stream and returns `nil`. Buffered chunks stay readable; once they are drained, [`rontolisp:stream-read`](rontolisp-stream-read.md) observes end of stream (`nil`). Closing an already-closed stream is a no-op. A [`rontolisp:stream-write`](rontolisp-stream-write.md) after the close signals an error. ```lisp (let ((s (rontolisp:make-stream))) (rontolisp:stream-write s "x") (rontolisp:stream-close s) (rontolisp:stream-close s)) ; => NIL ``` ## Backend support Asynchronous streams exist on the interpreter, the JVM backend and -- for the request/response body streams `rontolisp:fetch` / `rontolisp:http-handler` produce -- the `--component` WASM backend. A Preview 1 WASM module can hold a stream value only when a host-backed body gives it one; where none can exist, `rontolisp:streamp` answers `nil` and `rontolisp:stream-read` / `rontolisp:stream-close` signal an error when called. --- # FILE: references/reference/functions/rontolisp-stream-read.md # rontolisp:stream-read `(rontolisp:stream-read stream)` Returns a future settling to the stream's next chunk, or `nil` once the stream is closed and drained (end of stream). Chunks are never `nil`, so a `nil` result always means end of stream. A read on an open, empty stream stays pending until a write arrives — that is the suspension an awaiting asynchronous function parks on. A chunk is whatever the producer wrote: a string for a guest-created stream, and an `(unsigned-byte 8)` vector for every HTTP body stream (a fetched reply's `:body`, a served request's `:raw-body`) — the octets exactly as they came off the wire, so a body relayed as a response body crosses byte-exact. [`rontolisp:read-all`](rontolisp-read-all.md) is the drain that decodes them to text. ```lisp (let ((s (rontolisp:make-stream))) (rontolisp:stream-write s "a") (rontolisp:stream-close s) (print (rontolisp:await (rontolisp:stream-read s))) (print (rontolisp:await (rontolisp:stream-read s)))) ``` ``` "a" NIL ``` To drain all remaining chunks into one string in one await, use [`rontolisp:read-all`](rontolisp-read-all.md) instead. ## Backend support Asynchronous streams exist on the interpreter, the JVM backend and -- for the request/response body streams `rontolisp:fetch` / `rontolisp:http-handler` produce -- the `--component` WASM backend. A Preview 1 WASM module can hold a stream value only when a host-backed body gives it one; where none can exist, `rontolisp:streamp` answers `nil` and `rontolisp:stream-read` / `rontolisp:stream-close` signal an error when called. --- # FILE: references/reference/functions/rontolisp-stream-write.md # rontolisp:stream-write `(rontolisp:stream-write stream chunk)` Appends `chunk` (which must not be `nil`) to an asynchronous stream and returns a future that settles when the stream accepted it, so a producer can flow-control by [`rontolisp:await`](../special-forms/rontolisp-await.md)ing each write. ```lisp (let ((s (rontolisp:make-stream))) (rontolisp:await (rontolisp:stream-write s "chunk")) (rontolisp:stream-close s) (rontolisp:await (rontolisp:stream-read s))) ; => "chunk" ``` Writing to a stream whose write end was closed with [`rontolisp:stream-close`](rontolisp-stream-close.md) signals an error: ```console > (let ((s (rontolisp:make-stream))) (rontolisp:stream-close s) (rontolisp:stream-write s "x")) stream-write: the stream is closed ``` ## Backend support Guest-created streams (`rontolisp:make-stream` / `rontolisp:stream-write`) exist on the interpreter and the JVM backend today; the WASM backends reject them at compile time (a `--component` program's streams come from `rontolisp:fetch` / `rontolisp:http-handler` bodies). --- # FILE: references/reference/functions/rontolisp-streamp.md # rontolisp:streamp `(rontolisp:streamp value)` Returns `t` if `value` is an *asynchronous* stream — as returned by [`rontolisp:make-stream`](rontolisp-make-stream.md) or found in a [`rontolisp:fetch`](rontolisp-fetch.md) response body — and `nil` otherwise. ```lisp (rontolisp:streamp (rontolisp:make-stream)) ; => T (rontolisp:streamp 42) ; => NIL ``` This is a different symbol from `cl:streamp`, the file-stream predicate: each answers `nil` for the other's streams. ```lisp (streamp (rontolisp:make-stream)) ; => NIL ``` An asynchronous stream is an opaque value: it has no reader syntax and prints as `#`. ## Backend support Asynchronous streams exist on the interpreter, the JVM backend and -- for the request/response body streams `rontolisp:fetch` / `rontolisp:http-handler` produce -- the `--component` WASM backend. A Preview 1 WASM module can hold a stream value only when a host-backed body gives it one; where none can exist, `rontolisp:streamp` answers `nil` and `rontolisp:stream-read` / `rontolisp:stream-close` signal an error when called. --- # FILE: references/reference/functions/rontolisp-tcp-accept.md # rontolisp:tcp-accept `(rontolisp:tcp-accept listener)` Blocks until a client connects to the given listener handle (from [`rontolisp:tcp-listen`](rontolisp-tcp-listen.md)) and returns a **bidirectional stream handle** for the accepted connection — the same kind of handle [`rontolisp:tcp-connect`](rontolisp-tcp-connect.md) returns, usable with `read-line`, `write-line`, `write-string`, `read-byte`, `write-byte` and `close`. The example is self-contained: because the client connects *before* the accept, the connection waits in the listen backlog and the single-threaded program never blocks for long: ```lisp (let* ((listener (rontolisp:tcp-listen 0 "127.0.0.1")) (port (rontolisp:tcp-local-port listener)) (client (rontolisp:tcp-connect "127.0.0.1" port)) (server (rontolisp:tcp-accept listener))) (write-byte 65 client) (let ((b (read-byte server))) (close server) (close client) (close listener) b)) ; => 65 ``` ## Backend support - **Interpreter** and **JVM**: `java.net.ServerSocket.accept()`; accepting on a closed listener signals an error. - **WASM**: component-only. The accept is a cooperatively blocking read of one `tcp-socket` handle from the `wasi:sockets@0.3.0` accept stream; in an async body a pending accept suspends only its own task, so other tasks (a `rontolisp:wait-for` timer, another request) keep running. Returns `nil` if accepting fails. Call-time error in Preview 1 (core-module) mode. - **Browser playground**: not supported. ## Limitations - Blocks indefinitely until a client connects; there is no timeout parameter. - One connection is served per call — accept again for the next client (see the server loop under [`rontolisp:tcp-listen`](rontolisp-tcp-listen.md)). --- # FILE: references/reference/functions/rontolisp-tcp-addresses.md # rontolisp:tcp-local-address rontolisp:tcp-peer-address rontolisp:tcp-peer-port `(rontolisp:tcp-local-address handle)` -- `(rontolisp:tcp-peer-address handle)` -- `(rontolisp:tcp-peer-port handle)` Address introspection for TCP handles. `tcp-local-address` returns the local (bound) IP address of a listener or socket handle as a string; `tcp-peer-address` and `tcp-peer-port` return the remote IP address (a string) and remote port (an integer) of a connected socket handle. Together with [`rontolisp:tcp-local-port`](rontolisp-tcp-local-port.md) they back the `usocket:get-local-*` / `usocket:get-peer-*` accessors of the [usocket shim](usocket-accessors.md). ```lisp (let* ((listener (rontolisp:tcp-listen 0 "127.0.0.1")) (port (rontolisp:tcp-local-port listener)) (client (rontolisp:tcp-connect "127.0.0.1" port)) (server (rontolisp:tcp-accept listener)) (peer (rontolisp:tcp-peer-address client))) (close server) (close client) (close listener) peer) ; => "127.0.0.1" ``` The peer accessors reject a listener handle (a listener has no peer): ```console $ rontolisp > (setq l (rontolisp:tcp-listen 0 "127.0.0.1")) > (rontolisp:tcp-peer-address l) Error: tcp-peer-address expects a connected socket handle ``` ## Backend support - **Interpreter** and **JVM**: `getLocalAddress()` / `getInetAddress()` / `getPort()` on the underlying `java.net.Socket` / `ServerSocket`. A handle that is not the right kind of socket signals an error (interpreter) or fails with a cast error (JVM). - **WASM**: component mode only -- all three return real addresses and ports, exactly like the interpreter/JVM. On failure or a wrong kind of handle they return `nil` instead of signaling (so a spliced usocket program still runs there). Call-time error in Preview 1 (core-module) mode, like every tcp built-in. - **Browser playground**: not supported. --- # FILE: references/reference/functions/rontolisp-tcp-connect.md # rontolisp:tcp-connect `(rontolisp:tcp-connect host port)` Opens a blocking TCP connection to `host`/`port` and returns a **bidirectional stream handle**. The handle lives in the same handle space as file streams, so the standard stream functions work on it directly: [`read-line`](read-line.md), [`write-line`](write-line.md), [`write-string`](write-string.md), [`write-char`](../macros/write-char.md), [`read-char`](read-char.md), [`read-byte`](read-byte.md), [`write-byte`](write-byte.md) and [`close`](close.md). Unlike buffered file output, socket writes are sent immediately (`write-line` flushes per line). `read-line` returns `nil` once the peer has closed the connection. The example below is self-contained: it listens on an ephemeral port with [`rontolisp:tcp-listen`](rontolisp-tcp-listen.md), connects to itself over the loopback interface, and echoes one line through [`rontolisp:tcp-accept`](rontolisp-tcp-accept.md): ```lisp (let* ((listener (rontolisp:tcp-listen 0 "127.0.0.1")) (port (rontolisp:tcp-local-port listener)) (sock (rontolisp:tcp-connect "127.0.0.1" port))) (write-line "ping" sock) (let* ((peer (rontolisp:tcp-accept listener)) (line (read-line peer))) (write-line line peer) (let ((reply (read-line sock))) (close peer) (close sock) (close listener) reply))) ; => "ping" ``` A typical client connects to a fixed host and port and exchanges lines until the server closes: ```console (let ((sock (rontolisp:tcp-connect "127.0.0.1" 7777))) (write-line "hello" sock) (print (read-line sock)) ; the server's reply (close sock)) ``` ## Backend support - **Interpreter** and **JVM**: use the JDK `java.net.Socket`; `host` may be a hostname or an IP literal. A failed connection (for example a refused port) signals an error. - **WASM**: component-only, over `wasi:sockets@0.3.0` (natively WASI 0.3 — unlike `rontolisp:fetch`, no 0.2 hybrid is needed). `host` must be an **IPv4 literal** such as `"127.0.0.1"` (hostname resolution via `wasi:sockets/ip-name-lookup` is not wired yet). Compile with `--component` and run with `wasmtime run -W gc=y -W exceptions=y -S tcp=y -S inherit-network=y` (wasmtime 46+). A failed connection returns `nil` instead of a handle (the same nil-on-failure convention as `rontolisp:fetch`); without the `-S` flags the component still starts, but every socket operation fails and yields `nil`. The tcp built-ins compile in Preview 1 (core-module) mode but every call raises a call-time error naming the backends that work. - **Browser playground**: not supported — the browser sandbox provides no raw TCP sockets, so every tcp function signals an error. ## Limitations - TCP only (no UDP yet): the connection is plain text. For an encrypted client connection use [`rontolisp:tls-connect`](rontolisp-tls-connect.md) (interpreter/JVM only). - The WASM component backend accepts IPv4 literals only. - `read` (the s-expression reader) does not work on socket handles; read lines or bytes and parse them explicitly (e.g. with [`read-from-string`](read-from-string.md)). --- # FILE: references/reference/functions/rontolisp-tcp-listen.md # rontolisp:tcp-listen `(rontolisp:tcp-listen port &optional host)` Binds a listening TCP socket on `port` and returns a **listener handle** for [`rontolisp:tcp-accept`](rontolisp-tcp-accept.md) and [`close`](close.md). Port `0` picks a free ephemeral port — read the actual port back with [`rontolisp:tcp-local-port`](rontolisp-tcp-local-port.md). Without `host` the listener binds all interfaces; pass an address string (for example `"127.0.0.1"`) to bind only one. ```lisp (let* ((listener (rontolisp:tcp-listen 0 "127.0.0.1")) (port (rontolisp:tcp-local-port listener))) (close listener) (> port 0)) ; => T ``` A server accepts connections in a loop; each accepted handle is a bidirectional stream (see [`rontolisp:tcp-accept`](rontolisp-tcp-accept.md)): ```console (let ((listener (rontolisp:tcp-listen 7777))) (do ((n 1 (+ n 1))) (nil) (let ((sock (rontolisp:tcp-accept listener))) (do ((line (read-line sock) (read-line sock))) ((null line) (close sock)) (write-line line sock))))) ; echo every client forever ``` ## Backend support - **Interpreter** and **JVM**: use the JDK `java.net.ServerSocket` (with `SO_REUSEADDR` on the interpreter); a failed bind (for example a port already in use) signals an error. - **WASM**: component-only, over `wasi:sockets@0.3.0`. `host` must be an IPv4 literal. Compile with `--component` and run with `wasmtime run -W gc=y -W exceptions=y -S tcp=y -S inherit-network=y` (wasmtime 46+); a failed bind returns `nil`. Call-time error in Preview 1 (core-module) mode. - **Browser playground**: not supported (no raw TCP in the browser sandbox). ## Limitations - The listener handle only supports `rontolisp:tcp-accept`, `rontolisp:tcp-local-port` and `close` — it is not a byte stream itself. - See [`rontolisp:tcp-connect`](rontolisp-tcp-connect.md) for the shared socket limitations (TCP only, IPv4 literals on WASM). The listener serves plain TCP; for a TLS listener use [`rontolisp:tls-listen`](rontolisp-tls-listen.md) (interpreter/JVM only). --- # FILE: references/reference/functions/rontolisp-tcp-local-port.md # rontolisp:tcp-local-port `(rontolisp:tcp-local-port handle)` Returns the local TCP port bound to a listener or socket handle as an integer. Its main use is reading the actual port back after listening on port `0` (an ephemeral port picked by the operating system), which is how a test or a self-contained example avoids hard-coding a port: ```lisp (let* ((listener (rontolisp:tcp-listen 0 "127.0.0.1")) (port (rontolisp:tcp-local-port listener))) (close listener) (> port 0)) ; => T ``` It also works on a connected socket handle, where it reports the local (client-side) port of the connection: ```console (let ((sock (rontolisp:tcp-connect "127.0.0.1" 7777))) (print (rontolisp:tcp-local-port sock)) ; the ephemeral client port (close sock)) ``` ## Backend support - **Interpreter** and **JVM**: `getLocalPort()` on the underlying `java.net.ServerSocket` / `Socket`. A handle that is not a socket or listener signals an error (interpreter) or fails with a cast error (JVM). - **WASM**: component-only, via `wasi:sockets`' `get-local-address`; returns `nil` for a handle that is not a socket or listener. Call-time error in Preview 1 (core-module) mode, like every tcp built-in. - **Browser playground**: not supported. --- # FILE: references/reference/functions/rontolisp-tcp-set-timeout.md # rontolisp:tcp-set-timeout `(rontolisp:tcp-set-timeout handle milliseconds)` Sets the read deadline of a connected socket handle: every subsequent blocking read on the handle (`read-line`, `read-char`, `read-byte`, ...) signals an error after `milliseconds` without data instead of waiting forever. `milliseconds` is a non-negative integer (the [`rontolisp:wait-for`](rontolisp-wait-for.md) convention), and `nil` clears the deadline. Returns the `milliseconds` argument. Listener handles are not accepted (the deadline is a read deadline). ```lisp (let* ((listener (rontolisp:tcp-listen 0 "127.0.0.1")) (port (rontolisp:tcp-local-port listener)) (sock (rontolisp:tcp-connect "127.0.0.1" port))) (rontolisp:tcp-set-timeout sock 200) (prog1 (handler-case (progn (read-line sock) :read) (error (e) :timed-out)) ; nothing is ever written -> the deadline fires (close sock) (close listener))) ; => :TIMED-OUT ``` The timeout error is a plain catchable `error` whose message names the read that timed out; it is not a distinct condition class. The deadline lives on the raw socket, so it keeps governing a connection later upgraded with [`rontolisp:tls-upgrade`](rontolisp-tls-upgrade.md). This is the primitive behind the usocket shim's `(setf (usocket:socket-option sock :receive-timeout) seconds)` (see the [TCP Sockets guide](../../guides/tcp-sockets.md#the-usocket-compatible-shim)). ## Backend support - **Interpreter** and **JVM**: real, via `Socket.setSoTimeout`. - **WASM component**: SIGNALS at call time — `wasi:sockets@0.3.0` exposes no receive-timeout knob, and a timeout that silently never fires is the failure mode a client sets it to avoid. Catch it (or do not set a read timeout) on this backend. Call-time error in Preview 1 (core-module) mode, like every tcp built-in. - **Browser playground**: not supported. --- # FILE: references/reference/functions/rontolisp-then-star.md # rontolisp:then* `(rontolisp:then* future &rest functions)` Variadic chain sugar for `rontolisp:then`: threads the value through each `function` in order without the parenthesis nesting a manual chain would require. Each function receives the previous stage's (auto-flattened via `await`) settled value; a stage returning a future is flattened on the next stage's read. With no callbacks the operator returns the input future unchanged (documented degenerate identity). ```lisp (rontolisp:async-defun produce () 40) (rontolisp:await (rontolisp:then* (produce) #'1+ #'1+)) ; => 42 ``` A non-future first argument is a `type-error`. Note on the name: this is `then` + `*` (the CL convention for a variadic sibling of a two-argument operator), not `thenCompose`/`thenApply` from Java's `CompletableFuture`; because `await` flattens on read, the "compose" and "apply" distinctions collapse into the same shape here. ## Backend support Same as [`rontolisp:then`](rontolisp-then.md): interpreter, JVM, WASM `--component`, and the success-only shape on Preview 1 WASM. `--no-gc` rejects at compile time. --- # FILE: references/reference/functions/rontolisp-then.md # rontolisp:then `(rontolisp:then future function)` Attaches a transform to a future as a value: returns a **fresh** future that, on the input's successful settlement, invokes `function` with the settled value and settles to the function's return value. If `function` returns a future, `await` on the returned future flattens it (users never observe `future>`). On upstream error the callback is skipped and the condition propagates through the returned future unchanged. Use it to compose asynchronous work when the future crosses a boundary as a value -- the plain caller does not have to be an `rontolisp:async-defun` just because its callee is: ```lisp (rontolisp:async-defun some-future-producer () 21) (defun caller () (rontolisp:then (some-future-producer) (lambda (v) (* 2 v)))) (rontolisp:await (caller)) ; => 42 ``` A non-future first argument is a `type-error` -- there is no JavaScript-style auto-coercion to a resolved promise. ## Backend support Supported on the interpreter, the JVM backend and WASM `--component`. Preview 1 WASM supports the success-only shape (the degenerate synchronous semantics of the async surface there: an errored body signals at the call rather than at await, so the error-propagation contract falls back to the enclosing `handler-case`). `--no-gc` rejects the whole async surface at compile time. --- # FILE: references/reference/functions/rontolisp-thread-alive-p.md # rontolisp:thread-alive-p `(rontolisp:thread-alive-p thread)` Returns `t` while the thread behind the handle is still running, `nil` once it has died. After a [`rontolisp:join-thread`](rontolisp-join-thread.md) the answer is reliably `nil` (the join waits for the thread's teardown, not just its value). ```lisp (let ((th (rontolisp:make-thread (lambda () 1)))) (rontolisp:join-thread th) (rontolisp:thread-alive-p th)) ; => NIL ``` ## Limitations - A value that is not a thread handle is an error (use [`rontolisp:threadp`](rontolisp-threadp.md) first when unsure). - Interpreter and JVM backend only, like [`rontolisp:make-thread`](rontolisp-make-thread.md) itself. --- # FILE: references/reference/functions/rontolisp-threadp.md # rontolisp:threadp `(rontolisp:threadp value)` Returns `t` when `value` is a thread handle (as returned by [`rontolisp:make-thread`](rontolisp-make-thread.md)), else `nil`. This is the one thread operation that accepts any value. ```lisp (list (rontolisp:threadp (rontolisp:make-thread (lambda () 1))) (rontolisp:threadp 42)) ; => (T NIL) ``` On the WASM backends no thread handle can exist, so the `bt2:threadp` shim constantly answers `nil` there — which is how Clack's `stop` takes its non-threaded branch. --- # FILE: references/reference/functions/rontolisp-tls-connect.md # rontolisp:tls-connect `(rontolisp:tls-connect host port)` `(rontolisp:tls-connect host port :insecure value)` Opens a blocking TCP connection to `host`/`port`, performs a **TLS handshake**, and returns a **bidirectional stream handle** — the encrypted counterpart of [`rontolisp:tcp-connect`](rontolisp-tcp-connect.md). The handle lives in the same handle space as file streams, so the standard stream functions work on it directly: [`read-line`](read-line.md), [`write-line`](write-line.md), [`read-byte`](read-byte.md), [`write-byte`](write-byte.md) and [`close`](close.md). As with plain sockets, writes are sent immediately and `read-line` returns `nil` once the peer has closed the connection. The server certificate is validated against the JDK default trust store and the hostname is verified (HTTPS-style endpoint identification), so connecting to a server with an untrusted or mismatching certificate signals an error. To trust a self-signed certificate, point the standard `javax.net.ssl.trustStore` / `javax.net.ssl.trustStorePassword` system properties at your own trust store; they are re-read on every call. Passing `:insecure` with a non-`nil` `value` **disables** both checks — the certificate chain is accepted unconditionally and the hostname is not verified. This is intended for development against a self-signed server; never use it for real endpoints, since it removes all protection against man-in-the-middle attacks. `:insecure nil` is the same as omitting the option (verification on). The example below speaks HTTP/1.1 over TLS by hand (the request lines end with CRLF, so the carriage return is appended explicitly; `read-line` strips it from the response). For real HTTPS requests prefer [`rontolisp:fetch`](rontolisp-fetch.md) — `tls-connect` is for arbitrary TLS-wrapped protocols: ```console (let ((sock (rontolisp:tls-connect "example.com" 443)) (cr (princ-to-string (code-char 13)))) (write-line (concatenate 'string "GET / HTTP/1.1" cr) sock) (write-line (concatenate 'string "Host: example.com" cr) sock) (write-line (concatenate 'string "Connection: close" cr) sock) (write-line cr sock) (print (read-line sock)) ; "HTTP/1.1 200 OK" (close sock)) ``` ## Backend support - **Interpreter** and **JVM**: use the JDK TLS stack (`SSLSocket`); `host` may be a hostname or an IP literal. A failed connection or handshake (refused port, untrusted certificate, hostname mismatch) signals an error. - **WASM `--component`** (WASI 0.3): supported, over wasmtime's `wasi:tls@0.3.0-draft` interface — add `-S tls=y` to the usual socket run flags (`-W exceptions=y -S tcp=y -S inherit-network=y`). Like `tcp-connect` there, `host` must be an **IPv4 literal** (or `localhost`) — and it doubles as the name the certificate is verified against, so for a real-world host prefer `tcp-connect` to its address plus [`rontolisp:tls-upgrade`](rontolisp-tls-upgrade.md) with the DNS name. Failures follow the WASM error convention and return `nil` instead of signaling. Certificates are verified against the trust anchors compiled into the host (wasmtime bundles the Mozilla root store; the trust-store system properties and `:insecure` have no effect there — a non-`nil` `:insecure` value **signals** rather than silently verifying). The interface is an explicitly experimental draft, so a wasmtime update may need a matching rontolisp update. - **WASM Preview 1**: not supported — a **compile error** (no `wasi:tls` host API exists for Preview 1). - **Browser playground**: not supported — the browser sandbox provides no raw TCP sockets, so `tls-connect` signals an error. ## Limitations - `:insecure` is an all-or-nothing opt-out (no per-certificate pinning); to trust specific additional certificates while keeping verification on, use the trust-store system properties instead. For the *server* side of TLS see [`rontolisp:tls-listen`](rontolisp-tls-listen.md). - `read` (the s-expression reader) does not work on socket handles; read lines or bytes and parse them explicitly (e.g. with [`read-from-string`](read-from-string.md)). --- # FILE: references/reference/functions/rontolisp-tls-listen-pem.md # rontolisp:tls-listen-pem `(rontolisp:tls-listen-pem cert-file key-file port &optional host)` Binds a listening **TLS** socket serving a certificate and private key read from **PEM files** — the certbot / OpenSSL-friendly counterpart of [`rontolisp:tls-listen`](rontolisp-tls-listen.md) (which takes a PKCS12 keystore). `cert-file` is a PEM certificate chain (leaf certificate first) and `key-file` is the matching **unencrypted PKCS#8 private key** (`-----BEGIN PRIVATE KEY-----`, RSA/EC/DSA/EdDSA). Everything else matches `tls-listen`: the listener handle works with [`rontolisp:tcp-accept`](rontolisp-tcp-accept.md), [`rontolisp:tcp-local-port`](rontolisp-tcp-local-port.md) and [`close`](close.md), and an accepted connection handshakes on its first read or write. Generate a self-signed cert and key for local development with OpenSSL (this writes an unencrypted PKCS#8 key thanks to `-nodes`): ```bash openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:prime256v1 -nodes \ -keyout key.pem -out cert.pem -days 365 -subj /CN=localhost \ -addext subjectAltName=IP:127.0.0.1,DNS:localhost ``` The example below is a one-connection TLS echo server; test it with `openssl s_client -connect 127.0.0.1:8443` (type a line, it comes back): ```console (let* ((listener (rontolisp:tls-listen-pem "cert.pem" "key.pem" 8443)) (sock (rontolisp:tcp-accept listener)) ; blocks for a client (line (read-line sock))) ; the handshake happens here (write-line line sock) (close sock) (close listener)) ``` ## Backend support - **Interpreter**: reads the PEM files at run time, so `cert-file` and `key-file` may be computed at run time. A missing file, a malformed certificate, an encrypted or unsupported key, or a busy port signals an error. - **JVM**: the certificate and key are parsed **at compile time** and the resulting keystore is embedded in the compiled class, so `cert-file` and `key-file` must be **string literals** (a computed path is a compile error); relative paths resolve against the source file's directory. The compiled program needs no PEM files at run time. - **WASM**: not supported — `tls-listen-pem` is a **compile error** in both Preview 1 and `--component` mode. The `wasi:tls` proposal is **client-only by design** (there is no server-side TLS interface in any draft), so a TLS *server* has no WASM path — only the client side ([`rontolisp:tls-connect`](rontolisp-tls-connect.md) / [`rontolisp:tls-upgrade`](rontolisp-tls-upgrade.md)) runs under `--component`. - **Browser playground**: not supported — the browser sandbox provides no raw TCP sockets, so `tls-listen-pem` signals an error. ## Limitations - The private key must be an **unencrypted PKCS#8** key (`-----BEGIN PRIVATE KEY-----`). Encrypted keys and legacy PKCS#1 (`-----BEGIN RSA PRIVATE KEY-----`) or SEC1 (`-----BEGIN EC PRIVATE KEY-----`) formats are not read; convert with `openssl pkcs8 -topk8 -nocrypt -in old.pem -out key.pem`. - No client-certificate authentication (mutual TLS) options. - On the JVM backend the paths are compile-time literals (see Backend support); for a keystore chosen at run time on the JVM, use [`rontolisp:tls-listen`](rontolisp-tls-listen.md) with a PKCS12 file instead. --- # FILE: references/reference/functions/rontolisp-tls-listen.md # rontolisp:tls-listen `(rontolisp:tls-listen keystore password port &optional host)` Binds a listening **TLS** socket — the encrypted counterpart of [`rontolisp:tcp-listen`](rontolisp-tcp-listen.md). `keystore` is the path to a **PKCS12 keystore file** holding the server private key and certificate, and `password` unlocks it. The returned listener handle works with the plain TCP functions unchanged: accept connections with [`rontolisp:tcp-accept`](rontolisp-tcp-accept.md), read the bound port back with [`rontolisp:tcp-local-port`](rontolisp-tcp-local-port.md) (useful after listening on port `0`) and shut down with [`close`](close.md). An accepted connection is a normal bidirectional stream handle; its TLS handshake completes on the first read or write. Generate a self-signed keystore for local development with the JDK `keytool` (or export one from openssl with `openssl pkcs12 -export`): ```bash keytool -genkeypair -alias my-server -keyalg EC -dname CN=localhost \ -validity 365 -ext SAN=ip:127.0.0.1,dns:localhost \ -storetype PKCS12 -keystore tls-server.p12 \ -storepass changeit -keypass changeit ``` The example below is a one-connection TLS echo server; test it with `openssl s_client -connect 127.0.0.1:8443` (type a line, it comes back): ```console (let* ((listener (rontolisp:tls-listen "tls-server.p12" "changeit" 8443)) (sock (rontolisp:tcp-accept listener)) ; blocks for a client (line (read-line sock))) ; the handshake happens here (write-line line sock) (close sock) (close listener)) ``` Complete servers live in the `examples/` directory: [`https-hello.lisp`](https://github.com/making/rontolisp/blob/develop/examples/net/https-hello.lisp) (an HTTPS server `curl -k` understands) and [`kv-server-tls.lisp`](https://github.com/making/rontolisp/blob/develop/examples/net/kv-server-tls.lisp) (a mini-Redis served over TLS that the real `redis-cli --tls` talks to). ## Backend support - **Interpreter** and **JVM**: use the JDK TLS stack. Unlike `tcp-listen` on the WASM backend, failures never yield `nil`: a missing keystore, a wrong password or a busy port signals an error on both backends. - **WASM**: not supported — `tls-listen` is a **compile error** in both Preview 1 and `--component` mode. The `wasi:tls` proposal is **client-only by design** (there is no server-side TLS interface in any draft), so a TLS *server* has no WASM path — only the client side ([`rontolisp:tls-connect`](rontolisp-tls-connect.md) / [`rontolisp:tls-upgrade`](rontolisp-tls-upgrade.md)) runs under `--component`. - **Browser playground**: not supported — the browser sandbox provides no raw TCP sockets, so `tls-listen` signals an error. ## Limitations - The keystore must be PKCS12 (`keytool`'s default). To serve a certificate and key straight from **PEM files** (certbot / OpenSSL output), use [`rontolisp:tls-listen-pem`](rontolisp-tls-listen-pem.md) instead. - No client-certificate authentication (mutual TLS) options. - The handshake of an accepted connection is lazy: a client that fails certificate validation surfaces as an error on the server's first read or write of that connection, not at `rontolisp:tcp-accept` time. --- # FILE: references/reference/functions/rontolisp-tls-upgrade.md # rontolisp:tls-upgrade `(rontolisp:tls-upgrade stream host)` `(rontolisp:tls-upgrade stream host :insecure value)` Wraps an **already-connected** TCP stream handle in **TLS** as a client: performs the handshake over the existing connection and returns a **new** stream handle carrying the encrypted stream. Where [`rontolisp:tls-connect`](rontolisp-tls-connect.md) opens a fresh connection, `tls-upgrade` takes the handle an earlier [`rontolisp:tcp-connect`](rontolisp-tcp-connect.md) (or `usocket:socket-connect`) answered — the shape an HTTP client library needs, since it connects first (possibly issuing a proxy `CONNECT`) and only then starts TLS. The returned handle works with the standard stream functions ([`read-line`](read-line.md), [`write-line`](write-line.md), [`read-byte`](read-byte.md), [`write-byte`](write-byte.md), [`close`](close.md)); closing it also closes the underlying connection. The server certificate is validated against the JDK default trust store and `host` is verified against it (HTTPS-style endpoint identification; `host` is also sent as the SNI server name). To trust a self-signed certificate, point the standard `javax.net.ssl.trustStore` / `javax.net.ssl.trustStorePassword` system properties at your own trust store; they are re-read on every call. Passing `:insecure` with a non-`nil` `value` disables both checks — development only, exactly like `tls-connect`'s option. This is the primitive behind the bundled [`cl+ssl` shim system](../../guides/asdf-systems.md#built-in-shim-systems): `cl+ssl:make-ssl-client-stream` — the call every CL HTTP client (dexador, drakma, ...) makes for an `https://` URL — upgrades the stream it is handed through `tls-upgrade`. The example speaks HTTPS by hand over an upgraded plain connection (for real HTTPS requests prefer [`rontolisp:fetch`](rontolisp-fetch.md)): ```console (let* ((sock (rontolisp:tcp-connect "example.com" 443)) (tls (rontolisp:tls-upgrade sock "example.com")) (cr (princ-to-string (code-char 13)))) (write-line (concatenate 'string "HEAD / HTTP/1.1" cr) tls) (write-line (concatenate 'string "Host: example.com" cr) tls) (write-line (concatenate 'string "Connection: close" cr) tls) (write-line cr tls) (print (read-line tls)) ; "HTTP/1.1 200 OK" (close tls)) ``` ## Backend support - **Interpreter** and **JVM**: use the JDK TLS stack (`SSLSocketFactory.createSocket(socket, host, port, true)`); a failed handshake (untrusted certificate, hostname mismatch, a peer that does not speak TLS) signals an error. - **WASM `--component`** (WASI 0.3): supported, over wasmtime's `wasi:tls@0.3.0-draft` interface — add `-S tls=y` to the usual socket run flags. On this backend the upgrade is the *natural* primitive (the host's `connector` transforms wrap the socket's own streams), with three divergences: the answer is the **same** handle it was given (upgraded in place, not a new handle); the handle must not have been **written to** yet (the transform has to be interposed before the socket's send side is committed — a handle with prior writes answers `nil`); and failures return `nil` instead of signaling (the WASM error convention). Certificates are verified against the trust anchors compiled into the host (wasmtime bundles the Mozilla root store; the trust-store system properties and `:insecure` have no effect there — a non-`nil` `:insecure` value **signals** rather than silently verifying). The interface is an explicitly experimental draft, so a wasmtime update may need a matching rontolisp update. - **WASM Preview 1**: not supported — a **compile error** (no `wasi:tls` host API exists for Preview 1). - **Browser playground**: not supported — the browser sandbox provides no raw TCP sockets, so `tls-upgrade` signals an error. ## Limitations - `stream` must be a **connected socket handle** (from `tcp-connect` or `tcp-accept`); a listener or file-stream handle signals an error. The original handle still names the raw connection underneath — after the upgrade, read and write through the new handle only. - Client certificates are not supported (there is no way to present a client identity); the `cl+ssl` shim signals on its `:key`/`:certificate`/`:password` options for this reason rather than silently connecting unauthenticated. - `:insecure` is an all-or-nothing opt-out, like `tls-connect`'s. --- # FILE: references/reference/functions/rontolisp-url-decode.md # rontolisp:url-decode `(rontolisp:url-decode string)` Decodes a percent-encoded (URL-encoded) string: each `%XX` escape becomes a byte and the byte sequence is decoded as UTF-8 (multi-byte escapes reassemble into one character), and `+` becomes a space — the query-string convention. The inverse is [`rontolisp:url-encode`](rontolisp-url-encode.md). ```lisp (rontolisp:url-decode "Will+it+work%3F") ; => "Will it work?" (rontolisp:url-decode "%E3%81%82%E3%81%84") ; => "あい" (rontolisp:url-decode "plain") ; => "plain" ``` An invalid escape (`%` not followed by two hex digits, or bytes that are not valid UTF-8) signals an error: ```console > (rontolisp:url-decode "%2") Error: url-decode: unterminated percent escape ``` ## Backend support Works on every backend and in every WASM mode (Preview 1 included): the library is written in rontolisp itself and is compiled into the program when used. [`rontolisp:query-params`](rontolisp-query-params.md) and [`rontolisp:query-param`](rontolisp-query-param.md) decode keys and values with it automatically. --- # FILE: references/reference/functions/rontolisp-url-encode.md # rontolisp:url-encode `(rontolisp:url-encode string)` Encodes a string for embedding in a URL: RFC 3986 unreserved characters (letters, digits, `-`, `.`, `_`, `~`) pass through unchanged and every other character becomes the percent-encoded form of its UTF-8 bytes (a space becomes `%20`, not `+`). The inverse is [`rontolisp:url-decode`](rontolisp-url-decode.md). ```lisp (rontolisp:url-encode "a b/c~d") ; => "a%20b%2Fc~d" (rontolisp:url-encode "あ") ; => "%E3%81%82" (rontolisp:url-decode (rontolisp:url-encode "日本語 text?&=")) ; => "日本語 text?&=" ``` The typical use is building a [`rontolisp:fetch`](rontolisp-fetch.md) URL from runtime values: ```lisp (concatenate 'string "https://httpbin.ik.am/get?q=" (rontolisp:url-encode "ronto lisp")) ; => "https://httpbin.ik.am/get?q=ronto%20lisp" ``` ## Backend support Works on every backend and in every WASM mode (Preview 1 included): the library is written in rontolisp itself and is compiled into the program when used. --- # FILE: references/reference/functions/rontolisp-url-path.md # rontolisp:url-path `(rontolisp:url-path string)` Returns the part of a URL or request-target string before the first `?` (the whole string when there is no `?`). The counterpart [`rontolisp:url-query`](rontolisp-url-query.md) returns the part after it. ```lisp (rontolisp:url-path "/get?a=1") ; => "/get" (rontolisp:url-path "/get") ; => "/get" (rontolisp:url-path "https://example.com/a/b?x=1") ; => "https://example.com/a/b" ``` Inside an [`rontolisp:http-handler`](rontolisp-http-handler.md) handler the environment plist's `:path-info` already carries the path only, so this helper is mainly for splitting URL strings on the [`rontolisp:fetch`](rontolisp-fetch.md) (client) side. ## Backend support Works on every backend and in every WASM mode (Preview 1 included): the library is written in rontolisp itself and is compiled into the program when used. --- # FILE: references/reference/functions/rontolisp-url-query.md # rontolisp:url-query `(rontolisp:url-query string)` Returns the raw query-string part of a URL or request-target string: the text after the first `?` (possibly empty), or `nil` when there is no `?`. The counterpart [`rontolisp:url-path`](rontolisp-url-path.md) returns the part before it. The result is not decoded — pass it to [`rontolisp:query-params`](rontolisp-query-params.md) or [`rontolisp:query-param`](rontolisp-query-param.md). ```lisp (rontolisp:url-query "/get?a=1&b=2") ; => "a=1&b=2" (rontolisp:url-query "/get") ; => NIL (rontolisp:query-param (rontolisp:url-query "https://example.com/s?q=lisp") "q") ; => "lisp" ``` ## Backend support Works on every backend and in every WASM mode (Preview 1 included): the library is written in rontolisp itself and is compiled into the program when used. --- # FILE: references/reference/functions/rontolisp-version.md # rontolisp:version `(rontolisp:version)` Returns build and version information about the running rontolisp, as a property list with the keys `:version`, `:build-timestamp`, `:git-commit` and `:git-branch`. It is the same information printed by `rontolisp --version`. The values (timestamp and git revision) depend on the build, so the result varies between builds and is not supported inside the compiled runtime `eval`/`load`. ```lisp (getf (rontolisp:version) :version) ``` The call returns a plist such as `(:version "0.1.0-SNAPSHOT" :build-timestamp "..." :git-commit "..." :git-branch "...")`; use `getf` to read a single field. --- # FILE: references/reference/functions/rontolisp-wait-for.md # rontolisp:wait-for `(rontolisp:wait-for milliseconds)` Returns a future that settles to `nil` after the given number of milliseconds (a non-negative integer). The timer starts immediately, so awaiting it delays the *awaiting* code only -- other async bodies keep running, which makes `wait-for` the async counterpart of `cl:sleep` (which blocks and takes seconds). ```lisp (rontolisp:await (rontolisp:wait-for 100)) ; => NIL ``` Timers run concurrently: two futures started together settle in delay order, not start order, and awaiting both takes about the longer delay, not the sum. ```lisp (rontolisp:async-defun delayed (ms tag) (rontolisp:await (rontolisp:wait-for ms)) tag) (let ((slow (delayed 200 "slow")) (fast (delayed 20 "fast"))) (list (rontolisp:await fast) (rontolisp:await slow))) ; => ("fast" "slow") ``` ## Backend support `rontolisp:wait-for` exists on the interpreter, the JVM backend and WASM `--component` (where it lowers to the host timer, `wasi:clocks/monotonic-clock@0.3.0`'s `wait-for`, as a pending future the event loop settles -- timers genuinely overlap there too). Preview 1 WASM rejects it at compile time (no host timer). --- # FILE: references/reference/functions/rontolisp-wasm-export.md # rontolisp:wasm-export `(rontolisp:wasm-export 'name :as "alias" :params '(type...) :param-names '(name...) :returns type :async t)` Marks a top-level `defun` as host-callable when compiling to a WebAssembly core module, declaring the WASM-boundary types of its parameters and result. It is a compile-time directive, not an ordinary function: on the **interpreter** and **JVM** backends it is a no-op that simply returns the named symbol, so the same source runs on every backend. See [Compiling to WebAssembly](../../compiling/wasm.md) for the full guide. ```lisp (defun fact (n) (if (<= n 1) 1 (* n (fact (- n 1))))) (rontolisp:wasm-export 'fact :params '(:int) :returns :int) ; => FACT ``` ## Arguments - A quoted symbol naming the top-level `defun` to export. It resolves in the current [package](../packages.md) like a `defun` name. - `:as` — the WASM export name, as a string (e.g. `"factorial"`, or a camelCase name for a JavaScript-facing API). Defaults to the bare Lisp name (`fact`, without any package qualifier). - `:params` — a list of boundary type designators, one per parameter. Omitted, `nil` or `'()` means no arguments. - `:param-names` — the parameter names of the **component-model** signature, one per `:params` entry, as symbols or strings. Each must be a component-model label (lower-kebab-case words). Defaults to `p0`, `p1`, ... — the names a host or a binding generator sees in the component's type, and therefore the names [`--emit-wit`](../../guides/wit-contracts.md#emitting-the-wit-world---emit-wit) prints. It is ignored outside `--component` (a core WASM parameter has no name), and a program that implements a WIT world with [`rontolisp:wit-export`](rontolisp-wit-export.md) gets these from the world instead of declaring them. - `:returns` — the result boundary type designator. Omitted, `nil`, `'()` or `:void` declares a void result (the Lisp return value is discarded). - `:async` — `t` lifts the export as an **async** component-model function under `--component`, so I/O inside it (`print`, `rontolisp:fetch`, ...) works instead of trapping. Defaults to `nil` (a synchronous, pure-compute lift). Meaningful only under `--component`: Preview 1 / `--no-wasi` core exports ignore it, and `--no-gc --component` rejects it. The type designators and their boundary representations are: | Designator | WIT type | WASM boundary | Notes | | --- | --- | --- | --- | | `:s8` `:s16` `:s32` | `s8` `s16` `s32` | `i32` | `:int` is a permanent alias of `:s32` | | `:u8` `:u16` `:u32` | `u8` `u16` `u32` | `i32` | | | `:s64` `:u64` | `s64` `u64` | `i64` | `:long` is a permanent alias of `:s64`; a `:u64` value of 2^63 or more traps (it has no exact representation in the signed 64-bit integers every backend computes with) | | `:float` | `f64` | `f64` | rontolisp has no single-precision float, so `f32` is not a boundary type | | `:bool` | `bool` | `i32` | `0` is `nil`, any non-zero value is `t` | | `:string` | `string` | `(ptr, len)` | UTF-8 bytes in linear memory | | `:s-expr` | `string` | `(ptr, len)` | s-expression text in linear memory (any value except a function); no WIT type of its own | | `:bytes` | — | `(ptr, len)` argument / `(ptr, cap) -> len` result | an `(unsigned-byte 8)` vector as raw bytes, no UTF-8 in either direction; GC core-module shapes only | A `:bytes` **result** is caller-buffered (the `read(2)` shape): the export's core signature gains a trailing `(ptr, cap)` pair the host passes — reserve `cap` bytes with the exported `__ronto_alloc` — the wrapper copies at most `cap` bytes there, and the single `i32` result is the vector's **full** length, so an undersized buffer is a retry, not a truncation. **The boundary carries the value exactly, or the call traps.** A value the declared type cannot state — a negative returned through `:u32`, `300` through `:u8`, anything past the 32-bit range through `:s32` — stops the call instead of arriving silently wrapped. Nothing is masked, which is also what keeps a component behaving the same under `wasmtime` and under stricter binding generators such as `jco`. The representable range is the declared type's own, on every backend. With the default (GC) backend an incoming integer arrives as an exact integer (a fixnum when it fits, a boxed 64-bit integer past that), so a `:u32` argument of `3000000000` reaches the Lisp code as the exact integer `3000000000`; integer arithmetic inside the Lisp code is exact at any magnitude (`(+ x 1)` on a `:u32` argument of `1073741823` returns `1073741824` exactly), and only a result the declared type cannot state traps at the boundary. On the non-GC backend (`--no-gc`) integers are computed as `i64`, crossing the same way. ## Limitations - Under `--component`, an export becomes a **typed component-model export** callable with WAVE syntax (`wasmtime run --invoke 'name(args)'`): the whole fixed-width integer family (`:long` included), `:float`/`:bool`/void, `:string` and `:s-expr` as component-model `string` (`--no-gc` has no `:s-expr`). A sync (default) export must be pure-compute — I/O inside it traps; declare `:async t` when the export prints or fetches. Under `--no-gc --component`, `:async` is rejected but printing still works, through a built-in WASI 0.3 stdout micro-adapter wired in only when the program prints — every export of a printing program is then lifted `async` automatically. The export name must be lower-kebab-case (rename with `:as` otherwise), and adding `--emit-wit` writes the component's WIT world (with every export's typed signature) next to the `.wasm`. See [Component-model function exports](../../guides/wasm-component.md#component-model-function-exports-wasm-export) and [Compact component output](../../guides/wasm-nogc.md#compact-component-output---no-gc---component). On the interpreter and JVM the directive just returns the named symbol. - Only a top-level `defun` can be exported; the declared parameter count must match its arity, and functions that take or return function values are out of scope. - Outside `--component`, the exported function is pure-compute: reading, time and file access (from the function or from a top-level form) are unsupported. Under `--no-wasi` each of those has its own defined answer rather than a bare trap — output is discarded, `getenv` and file lookups answer nothing, the clock reports what a host wrote through `__ronto_set_time` (and signals until one does), `rontolisp:random-bytes` signals a catchable error, `random` runs on a built-in generator, and only standard input traps; see [No-WASI (reactor) mode](../../guides/wasm-gc-module.md#no-wasi-reactor-mode). One more exception: under `--no-gc`, `print`/`princ`/`terpri` work through a single `fd_write` import that is added only when the program prints (see [Printing](../../guides/wasm-nogc.md#printing-print--princ--terpri)). - The non-GC backend (`--no-gc`) supports `:int`/`:long`/`:float`/`:bool`/`:string` but not `:s-expr`, which needs the cons/reader/printer runtime, and not `:bytes`, which needs arrays. - `:bytes` is a GC core-module (Preview 1 / `--no-wasi`) boundary type: `--component` rejects it (there is no `list` lift yet), so it has no WIT spelling. --- # FILE: references/reference/functions/rontolisp-wasm-import.md # rontolisp:wasm-import `(rontolisp:wasm-import 'name :from "module" :as "field" :params '(type...) :returns type [:async t])` Declares a function the WASM host provides (JavaScript in a browser, or another module preloaded into wasmtime) and makes it callable from Lisp under `name` exactly like a top-level `defun` — including `#'name`, `funcall`, `mapcar` and `eval`. It is a compile-time directive, not an ordinary function: on the **interpreter** and **JVM** backends it defines a stub that signals an error when called (there is no host to call), so the same source still loads on every backend. See the [WASM host boundary guide](../../guides/wasm-host-boundary.md) for the full guide and the [WebGL galaxy example](https://github.com/making/rontolisp/tree/develop/examples/browser/webgl-galaxy) for a complete browser program. ```lisp (rontolisp:wasm-import 'draw-pixel :from "gl" :as "drawPixel" :params '(:int :int :int) :returns :void) ; => DRAW-PIXEL ``` ## Arguments - A quoted symbol naming the Lisp-visible function. It resolves in the current [package](../packages.md) like a `defun` name, so a directive after `(in-package mylib)` defines `mylib:name`. - `:from` — the import module name (the import-object key on the JavaScript side, or the `--preload` name in wasmtime). Defaults to `"env"`. - `:as` — the import field name (the property inside that module object). Defaults to the bare Lisp name (without any package qualifier). - `:params` — a list of boundary type designators, one per parameter. Omitted, `nil` or `'()` means no arguments. - `:returns` — the result boundary type designator. Omitted, `nil`, `'()` or `:void` declares a void result (Lisp receives `nil`). The type designators are shared with [`rontolisp:wasm-export`](rontolisp-wasm-export.md): | Designator | WASM boundary | Notes | | --- | --- | --- | | `:int` | `i32` | 31-bit signed range (the internal `i31ref`) | | `:float` | `f64` | an int or ratio argument is converted like the arithmetic built-ins | | `:bool` | `i32` | `nil` crosses as `0`, anything else as `1`; a non-zero result reads back as `t` | | `:string` | `(ptr, len)` | UTF-8 bytes in linear memory | | `:s-expr` | `(ptr, len)` | the argument is printed to readable text; a result is parsed by the embedded reader | | `:bytes` | `(ptr, len)` argument / `(ptr, cap) -> len` result | an `(unsigned-byte 8)` vector as raw bytes — no UTF-8 in either direction | A `:string` result must be written into linear memory by the host (reserve the buffer with the exported `__ronto_alloc`) and returned as a `(ptr, len)` pair (a two-element array from JavaScript). A `:bytes` **result** is caller-buffered (the `read(2)` shape): the Lisp signature gains one trailing parameter — the `(unsigned-byte 8)` vector to receive into — and the host function is called with a trailing `(ptr, cap)` pair: *write up to `cap` bytes at `ptr` and return the value's full length*. The Lisp call answers that full length, so a result longer than the buffer is a retry with a bigger buffer, never a silent truncation; the wrapper's staging is popped on return, so a pull loop over one reused buffer keeps linear memory flat. ```lisp (rontolisp:wasm-import 'read-chunk :from "env" :as "readChunk" :params '() :returns :bytes) ; => READ-CHUNK ;; (read-chunk buf) => the chunk's full length; up to (length buf) bytes ;; of buf are overwritten. On the JS side: readChunk(ptr, cap) -> n. ``` ## `:async t` — a host function that may suspend `:async t` declares that the host may implement the function **asynchronously** — on a JavaScript host, a `WebAssembly.Suspending`-wrapped function (JSPI). The call then returns a **future** that [`rontolisp:await`](../special-forms/rontolisp-await.md) resolves, so the source says at the call site that the boundary is asynchronous — the same reading as an `async func` member of a [`rontolisp:wit-import`](rontolisp-wit-import.md), which lowers to exactly this option on this backend. (The word deliberately matches [`rontolisp:wasm-export`](rontolisp-wasm-export.md)'s `:async`: WIT spells both directions `async func`, and the directive carries the direction.) ```lisp (rontolisp:wasm-import 'host-fetch :from "env" :as "fetch" :params '(:string) :returns :string :async t) ; => HOST-FETCH ``` - On this backend the future is **settled at creation**: the host call blocks the wasm stack — synchronously, or suspended through JSPI — so the value is ready when the call returns and `await` never actually suspends. The option buys one source that reads the same everywhere, not concurrency. - The build prints what the host now owes: wrap the import in `WebAssembly.Suspending`, enter every export that can reach it through `WebAssembly.promising` (the build lists them), and serialise calls — a suspended module can be re-entered, and a re-entered export **refuses with a trap** instead of silently corrupting both calls (every export wrapper of a module that can suspend carries a re-entry guard, unless it was compiled [`--reentrant`](../../guides/wasm-host-boundary.md#overlapping-calls---reentrant), which lets a JSPI host overlap calls instead). A host that answers synchronously is equally valid; the call returns an already-settled future either way. `--emit-js-glue` WRITES that half rather than describing it (the [host boundary guide](../../guides/wasm-host-boundary.md#generating-the-host-glue---emit-js-glue)): the host is then left with what its functions do, and says which of them suspend. - Under `--no-wasi`, a call reachable from a top-level form is a **compile error**: `_initialize` runs on a stack no `promising` entered, so a suspension there traps naming nobody. Move the call behind an export, or drop `:async t` if the host answers synchronously. ## Limitations - Applies to the default (wasm-GC) Preview 1 core module only; `--component` and `--no-gc` reject the directive with an error. On the interpreter and JVM the declared name signals an error when called. - The directive must appear at top level, before use like a `defun`. - Instantiating the compiled module requires the host to provide every declared import; `wasmtime run` needs a `--preload =.wasm` for each import module name, and a JavaScript host passes an import object. - At most 10 parameters (the general WASM-backend arity limit). --- # FILE: references/reference/functions/rontolisp-wit-export.md # rontolisp:wit-export `(rontolisp:wit-export "world.wit" :world name)` Declares that the program **implements a WIT world**. The world's `export` items are checked against the program's top-level `defun`s at compile time and lowered into the [`rontolisp:wasm-export`](rontolisp-wasm-export.md) directives they stand for, so the boundary types are never written by hand and the `.wit` file and the compiled component cannot drift apart. The WIT is the single source of truth: the world is the program's export list (a hand-written `rontolisp:wasm-export` alongside it is an error), and the emitted component is byte-identical to the one those hand-written directives would have produced. It is a compile-time directive, not an ordinary function: on the **interpreter** and **JVM** backends it runs the same contract check and then returns `nil`, so the same source runs on every backend. See [Implementing a WIT World](../../guides/wit-contracts.md#implementing-a-wit-world-wit-export) for the full guide. Because the directive reads a `.wit` file from disk, the example is shown statically: ```console // wit/greeter.wit package example:greeter; world greeter { /// Greet someone by name. export greet: func(who: string) -> string; } ``` ```console ;;; greet.lisp -- the directive comes last: on the interpreter it sees only the ;;; functions defined so far. (defun greet (who) (concatenate 'string "Hello, " who "!")) (rontolisp:wit-export "wit/greeter.wit" :world greeter) ``` ```bash rontolisp greet.lisp --component -o greet.wasm wasmtime run -W gc=y --invoke 'greet("world")' greet.wasm # "Hello, world!" ``` ## Arguments - The WIT file path, as a string. A relative path resolves against the directory of the source file that names it, like [`load`](load.md). - `:world` — the world to implement, as a bare symbol (spelled the way WIT spells it) or a string. It may be omitted when the file declares exactly one world; when the file declares several, one must be named. Everything else comes from the world: `rontolisp:wasm-export`'s `:params`, `:param-names`, `:returns` and `:async` are all filled in from it, so the `defun`s carry no boundary types at all. ## Supported WIT types | WIT type | Boundary type | Lisp value | | --- | --- | --- | | `s8` `s16` `s32` | `:s8` `:s16` `:s32` | an integer | | `u8` `u16` `u32` | `:u8` `:u16` `:u32` | an integer | | `s64` `u64` | `:s64` `:u64` | an integer; a `u64` value of 2^63 or more traps at the boundary | | `f64` | `:float` | a float | | `bool` | `:bool` | `t` or `nil` | | `string` | `:string` | a string | | (no result) | `:void` | the function's value is discarded | The whole fixed-width integer family crosses, so the canonical component-model tutorial world compiles unedited: ```console // wit/adder.wit package docs:adder@0.1.0; interface add { add: func(x: u32, y: u32) -> u32; } world adder { export add; } ``` Each type carries its own range exactly, or the call traps — a negative returned through `u32` is refused rather than delivered as `4294967295`. The component model has no integer subtyping, so this is not cosmetic: a component that lifted `u32` as `s32` would be rejected against its own world by `wasm-tools component targets`, by `jco`, and by any `bindgen`-based host. An `async func` in the world lifts the export with `:async t`, so blocking is always legal inside it: I/O inside a sync export usually works too (the asynchronous built-ins complete without blocking when the host accepts immediately), but a host that reports BLOCKED would make it trap, and the async lift removes that residual risk — the WIT states which exports are async rather than leaving it to be guessed. Every other WIT type (`record`, `list`, `option`, `result`, resources, ...) is a compile error at the export boundary today; the error names the rontolisp representation the type is settled to have, once marshalling it lands. ## What it checks Every violation is a compile error naming the WIT file and the line of the offending export: - an export the world declares with no matching `defun` — `wit/greeter.wit:5: export 'greet' has no matching (defun greet ...) in the program` - an arity mismatch — `wit/greeter.wit:5: export 'greet' declares 1 parameter(s), but (defun greet ...) takes 2` (an exported function takes required parameters only: `&optional` / `&rest` / `&key` are rejected) - a WIT type the export boundary does not carry (the whole fixed-width integer family plus `f64` / `bool` / `string` crosses; a `record`, `list`, ... does not yet) - an `async func` under `--no-gc --component`, whose adapter-free reactor has no async machinery - an export name that is not a component-model label (lower-kebab-case words), a duplicate export, or the reserved name `run` (the component's `wasi:cli/run` entry point) - a world with no exports, a `:world` the file does not declare, or an omitted `:world` when the file declares several Because the world *is* the export list, so is mixing it with the hand-written form: a `rontolisp:wasm-export` in a program that also has a `rontolisp:wit-export`, and a `rontolisp:http-handler` together with a world (a serve-mode component exports only `wasi:http/handler@0.3.0`). ## Limitations - Only the world's **export** side is a contract. `import` items are ignored (a component's WASI imports come from the build, not from the world), and an inline `import name: func(...)` is rejected rather than silently dropped — the functions a program calls are bound from an interface with [`rontolisp:wit-import`](rontolisp-wit-import.md), or declared by hand with [`rontolisp:wasm-import`](rontolisp-wasm-import.md) (both Preview 1 only). The component you get therefore has a much larger type than the world you wrote: the 6-line world above compiles to a 149-line component type (ten `wasi:*` imports plus `export wasi:cli/run`), and calling `rontolisp:fetch` inside `greet` silently adds five more. `--emit-wit` is how you see it. - A world exports freestanding functions or an **interface defined in the same file**: `export add;` referencing an in-file `interface add { ... }`, or an inline `export ops: interface { ... }`, is implemented member by member and produces a real `docs:adder/add` instance export (see [Exporting an interface](../../guides/wit-contracts.md#exporting-an-interface)). An export naming an interface the file does not define (a bare `wasi:*` reference) is still an error. - `:s-expr` has no WIT spelling, so an export carrying an arbitrary s-expression still needs a hand-written [`rontolisp:wasm-export`](rontolisp-wasm-export.md) — and therefore a program without a world. - On the **interpreter** the directive is an ordinary form evaluated in order, so it sees only the functions defined **so far**: put it at the end of the file. (The compile path collects every top-level `defun` first, so there the position does not matter.) - Adding `--emit-wit` writes the component's real type back out, and its export lines reproduce the world handed in, parameter names included — but by construction, not by coincidence: the world *is* what those lines are derived from, so they cannot disagree with it. Emitting is worth it for the import side, not as a check on your program (that is `wit-export`'s own job, on every backend). Two deliberate differences from the input file: the `///` doc comments are gone (a component's type does not store them), and the emitted world is always `package root:component; world root`. --- # FILE: references/reference/functions/rontolisp-wit-import.md # rontolisp:wit-import `(rontolisp:wit-import "kv.wit" :interface "wasi:keyvalue/store@0.2.0" :package kv)` Declares that the program **calls a WIT interface**. Every function the interface declares is bound as an ordinary Lisp function, with its name, its lambda list and its types taken from the `.wit` file — the mirror image of [`rontolisp:wit-export`](rontolisp-wit-export.md), and, like it, a compile-time directive that **lowers into forms that already exist** rather than a new call path. What it lowers to depends on the backend, and that is the whole point: **one WIT, a different implementation per backend, zero source changes**. On the **interpreter** and the **JVM** each binding becomes a `defun` dispatching through a *provider* — an ordinary Lisp callable you bind with [`rontolisp:wit-provide`](rontolisp-wit-provide.md); on **Preview 1 WASM** it becomes a [`rontolisp:wasm-import`](rontolisp-wasm-import.md); and under **`--component`** the interface becomes a real component-model **import**, whose functions are `canon lower`ed into the module — so the provider is the *host*, and the component composes with anyone who exports that interface. See [Importing a WIT Interface](../../guides/wit-contracts.md#importing-a-wit-interface-wit-import) for the full guide. Because the directive reads a `.wit` file from disk, the example is shown statically: ```console // wit/store.wit -- an excerpt of the real wasi:keyvalue/store@0.2.0 package wasi:keyvalue@0.2.0; interface store { variant error { no-such-store, access-denied, other(string), } resource bucket { get: func(key: string) -> result>, error>; set: func(key: string, value: list) -> result<_, error>; delete: func(key: string) -> result<_, error>; exists: func(key: string) -> result; } open: func(identifier: string) -> result; } ``` ```console ;;; counter.lisp -- the directive comes FIRST: it defines the kv package and the ;;; functions the rest of the file calls. (rontolisp:wit-import "wit/store.wit" :interface "wasi:keyvalue/store@0.2.0" :package kv) ;;; What those functions CALL is a provider -- ordinary Lisp code, and yours to ;;; write. rontolisp ships none: it knows the mechanism, not the interface. (defvar *rows* (make-hash-table :test #'equal)) (defun my-store (member &rest args) (cond ((string= member "open") 1) ; the bucket handle: any integer ((string= member "bucket-set") (setf (gethash (nth 1 args) *rows*) (nth 2 args)) nil) ((string= member "bucket-get") (gethash (nth 1 args) *rows*)) ((string= member "bucket-exists") (if (gethash (nth 1 args) *rows*) t nil)) (t (error 'rontolisp:wit-error :payload (list :other member))))) (rontolisp:wit-provide "wasi:keyvalue/store@0.2.0" #'my-store) (defvar *bucket* (kv:open "counts")) (kv:bucket-set *bucket* "visits" "41") (print (kv:bucket-get *bucket* "visits")) (print (kv:bucket-exists *bucket* "missing")) ``` ```bash rontolisp counter.lisp # the provider bound above # "41" # nil rontolisp counter.lisp -o Counter.class && java Counter # "41" # nil ``` Nothing is bound by hand: `kv:bucket-get`, and its `(self key)` lambda list, come from the WIT. What those functions *call* is the one thing a `.wit` cannot say — so it is a **provider**, and rontolisp ships **none, for any interface**. It knows the provider mechanism; it does not know what `wasi:keyvalue` is. An implementation of a WIT interface is therefore ordinary Lisp code, like `my-store` above, and swapping that hash table for a real store is one line the program never sees. [`examples/wit/keyvalue`](https://github.com/making/rontolisp/tree/develop/examples/wit/keyvalue) is exactly that: one page-view counter, with a portable in-memory Lisp store behind it on the interpreter, a `java.util.LinkedHashMap` one on the JVM, and — compiled with `--component` — **wasmtime's own `wasi:keyvalue` implementation**, a host that has never heard of the program. The output is identical all three ways. ## Arguments - The WIT file path, as a string. A relative path resolves against the directory of the source file that names it, like [`load`](load.md). - `:interface` — the interface to bind (required). Written as the fully-qualified id (`"wasi:keyvalue/store@0.2.0"`), the id without its version (`"wasi:keyvalue/store"`), or the bare interface name (`store`) when the file defines it only once. A string or a bare symbol. - `:package` — the Lisp package the bindings land in (`kv:open`, `kv:bucket-get`). A `defpackage` exporting them is synthesized, so no `defpackage` is written by hand. Omitted, the names land in the current package. - `:from` — the Preview 1 WASM import module name. Defaults to the interface's bare name (`store`). Ignored on the other backends (a component imports the interface under its fully-qualified id, which is not renameable). - `:field-style` — how a WIT label is spelled as a Preview 1 import **field**: `:camel` (the default — `create-shader` becomes `createShader`, the JavaScript convention and what `jco` produces) or `:kebab` (the label verbatim). Ignored on the other backends. ## What gets bound | WIT | Lisp function | Call | | --- | --- | --- | | `open: func(identifier: string) -> ...` | `kv:open` | `(kv:open "counts")` | | `resource bucket` method `get: func(key: string) -> ...` | `kv:bucket-get` | `(kv:bucket-get b "visits")` | | `resource bucket` `constructor(...)` | `kv:bucket-new` | `(kv:bucket-new ...)` | | `resource bucket` `static func from-name` | `kv:bucket-from-name` | `(kv:bucket-from-name "x")` | | `resource bucket` — its **release** (WIT declares no function for it) | `kv:bucket-drop` | `(kv:bucket-drop b)` | A resource member is prefixed with the resource, so two resources may declare the same method without colliding in the flat Lisp-2 function namespace, and a **method takes the handle as its first argument** (`self`, which WIT leaves implicit). The other parameters are named exactly as the WIT names them, and a resource itself is an opaque integer handle. Each binding is an **ordinary function**, so `#'kv:bucket-get`, `funcall`, `mapcar` and `eval` work on it with no extra wiring. The last row is the odd one out: a `.wit` never declares a function that releases a resource, because the component model makes releasing one a canonical built-in rather than a member of the interface. So there is nothing to bind — and rontolisp names it anyway, as **`-drop`**, symmetric with the `-new` a constructor binds. Both are rontolisp spellings of something WIT does not name as a function. See [Releasing a resource](#releasing-a-resource-resource-drop). ## How it lowers | Backend | The directive becomes | | --- | --- | | interpreter | one `defun` per WIT function, dispatching through the interface's provider | | JVM (`-o Prog.class`) | the same `defun`s, compiled | | Preview 1 WASM (`-o prog.wasm`) | one [`rontolisp:wasm-import`](rontolisp-wasm-import.md) per WIT function | | `--component` | a component-model **instance import** of the interface, each function `canon lower`ed into the core module | | `--no-gc` | a compile error (its MVP module imports nothing) | On Preview 1 the module is **byte-identical** to the hand-written equivalent, and [tree shaking](../../compiling/wasm.md#optimize-tree-shaking) still shakes out the imports the program never calls: ```console ;;; What (rontolisp:wit-import "wit/host.wit" :interface "example:host/math@0.1.0") ;;; lowers to on Preview 1 WASM, for `add-ints: func(a: s32, b: s32) -> s32`: (rontolisp:wasm-import 'add-ints :from "math" :as "addInts" :params '(:int :int) :returns :int) ``` Under `--component` the interface becomes an instance import of the component, and each bound function a `canon lower`ed core import. A component **only imports the functions the program actually calls** (the component path has no core tree shaker, so unused interface members are dropped from the import instead; `--no-prune` keeps them all), and [`--emit-wit`](../../guides/wit-contracts.md#emitting-the-wit-world---emit-wit) writes that pruned interface into the component's world — where `wasm-tools component wit` agrees with it, byte for byte. An import-free component is unchanged. ```bash rontolisp counter.lisp -o counter.wasm --component wasmtime run -W gc=y -W exceptions=y \ -S keyvalue=y counter.wasm # the HOST is the provider ``` That holds for a **served** component too ([`rontolisp:http-handler`](rontolisp-http-handler.md) + `--component`): its imports are no longer only the fixed `wasi:http` surface, so a handler's state can live in a real store instead of a process-local hash table — which is the only way a served component can keep state at all, since a `wasi:http` host instantiates it afresh for every request. ```bash rontolisp page-hits-server.lisp -o server.wasm --component wasmtime serve -W gc=y -W exceptions=y -S keyvalue=y server.wasm ``` Whether the state then *survives* is the host's business, not the component's: wasmtime's built-in key-value provider is an in-memory store it rebuilds per instance (so, under `wasmtime serve`, per request), while a host that links an out-of-process provider — wasmCloud, say — keeps it. The component is the same either way. ## Releasing a resource (`-drop`) A handle you were given has to be given back, and **WIT declares no function for that**: releasing a resource is a canonical built-in of the component model, not a member of the interface, so there is nothing in the `.wit` for a binder to find. rontolisp binds it as **`-drop`** — one argument, the handle: ```console ;;; The handle kv:open handed over -- give it back when the work is done. (let ((bucket (kv:open ""))) (kv:bucket-set bucket "visits" "41") (print (kv:bucket-get bucket "visits")) (kv:bucket-drop bucket)) ``` It is bound **only when the program names it**, which is why a program compiled before drops existed comes out byte-identical: nothing in it mentions a `-drop` name. A WIT *function*, by contrast, is bound whether the program calls it or not (Preview 1 binds the whole interface). `--no-prune` and `--dynamic` bind every resource's drop instead — and so does the interpreter, which compiles nothing whose bytes have to stay the same. | Backend | `(kv:bucket-drop b)` becomes | | --- | --- | | interpreter, JVM | a call into the interface's provider, with the member name `"bucket-drop"` and the handle as its only argument | | Preview 1 WASM (`-o prog.wasm`) | a **no-op**. A handle there is an opaque integer the host handed over and the guest holds nothing; importing a release function the WIT never declared would be inventing one | | `--component` | `canon resource.drop` — the handle goes back to the host's own table | | `--no-gc` | it rejects `rontolisp:wit-import` itself | Two things follow, and both matter more than tidiness: - **An interface may make dropping an obligation, not a courtesy.** `wasi:http` says an `outgoing-body`'s child `output-stream` must be dropped before the body is finished — otherwise the call **traps**. A program that cannot drop cannot send a request body at all. - **Dropping a handle releases the *reference*, not the thing behind it.** The store — or the file, or the socket — stays exactly where it was; the next `kv:open` sees every key still in it. What a drop *means* is the provider's decision, never the language's (below). ## Providers On the interpreter and the JVM there is no host, so the call goes to a **provider**: an ordinary Lisp callable taking the bound function's Lisp member name (a **string** — `"open"`, `"bucket-get"`, and `"bucket-drop"` for a resource's release) followed by that function's arguments. [`rontolisp:wit-provide`](rontolisp-wit-provide.md) binds one: ```console (rontolisp:wit-provide "wasi:keyvalue/store@0.2.0" #'my-store) ``` - **rontolisp ships no provider for any interface.** Binding an interface's functions is the language's business; implementing them is yours. A new host interface therefore costs a `.wit` file and one `rontolisp:wit-provide`, not core code. - Calling a bound function with **no provider bound** for its interface signals [`rontolisp:wit-error`](rontolisp-wit-provide.md#the-wit-error-condition): `No provider is bound for the WIT interface wasi:keyvalue/store@0.2.0 -- bind one with rontolisp:wit-provide`. - `rontolisp:wit-provide` **replaces** the interface's provider, so a fake store can be swapped for the real one — one line, in a file of its own if you like, and no call site changes. - On the **WASM** backends the host supplies the imports, so a top-level `rontolisp:wit-provide` is **dropped** (inert), not an error — one source runs everywhere. ## Errors (`result`) A WIT `result` is not a value: the **ok arm is the function's return value**, and the **error arm signals `rontolisp:wit-error`** carrying the mapped `E` as its payload (the settled mapping, on every backend). The provider is what signals it; a caller handles it with `handler-case` and reads the payload with [`rontolisp:wit-error-payload`](rontolisp-wit-provide.md#the-wit-error-condition). That is the RETURN direction — the only one a `result` had until now. A `result` you **pass in** cannot signal anything (an argument has to *say* which arm it is), so it keeps its arm: the envelope cons `(:ok . V)` / `(:error . E)`, which is exactly the shape a returned `result` has before the ok arm is unwrapped. The asymmetry is one-way and deliberate: **unwrapped on the way out, wrapped on the way in** — so a value one call returns can be passed straight into the next. ## Supported WIT types The boundary is three-tiered. On the **interpreter and the JVM** the call is an ordinary Lisp call, so every representation crosses — the table is the contract the provider is written against, not a marshaller. On the **Preview 1 WASM** boundary only the flat set `rontolisp:wasm-import` can carry crosses: a core import is a bare host function, with no component type to describe a richer shape with. Under **`--component`** the canonical ABI marshals the rich types, so a `record`, `variant`, `enum`, `option`, `tuple` or `result` crosses **in both directions** — as an argument as well as a result. Two do not: `flags` (in neither direction), and a `list` **as an argument** (`list` crosses, as a byte string). Anything unsupported is a compile error naming the WIT file and line. | WIT type | Lisp value | Preview 1 | `--component` | | --- | --- | --- | --- | | `s8` `s16` `s32` `u8` `u16` `u32` | an integer | `:int` | yes | | `s64` `u64` | an integer | no | yes | | `f32` `f64` | a float | `:float` | yes | | `bool` | `t` / `nil` | `:bool` | yes | | `string` | a string | `:string` | yes | | `char` | a character | no | yes | | `list` | a string of raw bytes (one per char) | `:string` | yes | | `list` | a proper list | no | result only | | `tuple<...>` | a proper list, positional | no | yes | | `option` | the value, or `nil` | no | yes | | `result` | returned: the ok value, the error arm signals `rontolisp:wit-error`; passed: the `(:ok . V)` / `(:error . E)` envelope | no | yes | | `record` | a keyword plist | no | yes | | `enum` | a keyword | no | yes | | `variant` | a keyword, or `(keyword . payload)` | no | yes | | `flags` | a list of keywords | no | no | | `resource`, `borrow`, `own` | an opaque integer handle | `:int` | yes | | `stream`, `future` | — | no | no | `stream` and `future` have no rontolisp value on any backend (they need language-level async), so they are rejected everywhere. ### Rich values as arguments An argument takes **exactly the shape the same type takes as a return value**, so a value one call hands you can be passed straight into the next: ```console ;;; a variant: the case keyword, or (keyword . payload) when the case carries one (http:outgoing-request-set-method req :post) (http:outgoing-request-set-method req '(:other . "PATCH")) (http:outgoing-request-method req) ; => (:other . "PATCH") ;;; an enum: a keyword (sock:tcp-socket-create :ipv4) ;;; a record is a keyword plist, a tuple a positional list -- here both, inside a ;;; variant case's payload (sock:tcp-socket-bind s '(:ipv4 :port 0 :address (127 0 0 1))) ;;; a result ARGUMENT is the (:ok . V) / (:error . E) envelope -- the same shape a ;;; result RESULT has before the ok arm is unwrapped. A payload-less arm may also ;;; be written as the bare keyword. (cli:exit '(:error)) (cli:exit :ok) ``` A keyword that names no case of the variant is a **type error**: on the WASM backends it traps, exactly as every other type error does there (`(+ 1 "a")` included); on the interpreter and the JVM it simply reaches the provider, which decides what to make of it. ## Limitations - `--no-gc` rejects the directive with a clear error: its contract is a plain MVP module that imports nothing at all. - On the Preview 1 boundary only the flat set above crosses; a `record`, `option`, `result` or `s64` is a compile error naming the WIT file and line — `wit/store.wit:12: 'bucket-get': the WIT type of the result does not cross the Preview 1 WASM import boundary, which carries the flat set (...)` — even though `--component`, the interpreter and the JVM all bind it. The `wasi:keyvalue` example above is therefore a component (or an interpreter/JVM) program, not a Preview 1 one: its `result` arms keep it off that boundary. - Under `--component` a **`list` argument** (other than `list`) is a compile error, though the same type crosses as a *result*: an argument is flattened, and a list would have to be written into linear memory as a canonical array instead. `flags` does not cross in either direction yet. - Under `--component` the interface must not be one the component **already imports for its own WASI surface** (which grows with what the program uses: `rontolisp:fetch` adds `wasi:http/types` and `wasi:http/client`, the `rontolisp:tcp-*` built-ins add `wasi:sockets/types`). A component cannot import the same interface twice, so this is a compile error naming it — drive the interface through the WIT binding *instead of* the built-in, not alongside it. - The directive must appear at **top level, before the code that calls the interface** (the opposite of [`wit-export`](rontolisp-wit-export.md), which must come last): it is what defines the package and the bindings. A directive below its call sites is a `No such package: kv` error on every backend. - Prefer `:package`. Without it the bindings land in the current package, where a WIT label that collides with a `cl` name (`open`, `close`, `delete`, ...) resolves inconsistently across backends — the interpreter takes the binding, the JVM backend takes the `cl` function. - Only an **interface** can be bound. A world's `import` items are not read (a component's WASI imports come from the build, not from the world). - A resource handle is opaque — only whoever handed it out may read anything into the integer — and rontolisp never releases one on its own. It is released by [`-drop`](#releasing-a-resource-resource-drop) and by nothing else: `cl:close` does not apply to a WIT resource, whose handles are the provider's (or the host's) private numbering, not a slot in a rontolisp stream table. - On the WASM backend the 10-parameter arity limit applies to a binding like any other function, counting a method's leading `self`. - Instantiating the compiled Preview 1 module requires the host to provide every import that survives tree shaking: `wasmtime run` needs a `--preload =.wasm`, and a JavaScript host passes an import object. --- # FILE: references/reference/functions/rontolisp-wit-provide.md # rontolisp:wit-provide `(rontolisp:wit-provide "wasi:keyvalue/store@0.2.0" #'my-store)` Binds the **implementation** of a WIT interface brought in with [`rontolisp:wit-import`](rontolisp-wit-import.md). On the interpreter and the JVM backend there is no WASM host to call, so every imported function dispatches through the interface's *provider*; this is what supplies one. It returns the interface id and **replaces** any provider already bound for that interface. rontolisp ships **no provider for any interface**: it knows the provider mechanism, not what `wasi:keyvalue` — or any other interface — *is*. An implementation of a WIT interface is ordinary Lisp code, and this is how you hand it in. A provider is an ordinary Lisp callable taking the bound function's **Lisp member name** (a string — `"open"`, `"bucket-get"`, the name the binding is spelled with, not the raw WIT label) followed by that function's arguments. Here is a complete one — a store in a hash table, which is all a store has to be: ```lisp (defvar *rows* (make-hash-table :test #'equal)) (defun my-store (member &rest args) ; ("bucket-set" bucket "visits" "41") (cond ((string= member "open") 1) ; the bucket handle: any integer ((string= member "bucket-set") (setf (gethash (nth 1 args) *rows*) (nth 2 args)) nil) ((string= member "bucket-get") (gethash (nth 1 args) *rows*)) ((string= member "bucket-exists") (if (gethash (nth 1 args) *rows*) t nil)) ((string= member "bucket-drop") nil) ; the handle is gone; the rows stay (t (error 'rontolisp:wit-error :payload (list :other member))))) (rontolisp:wit-provide "wasi:keyvalue/store@0.2.0" #'my-store) ; => "wasi:keyvalue/store@0.2.0" ``` With that in the source, the `wasi:keyvalue` program of [`wit-import`](rontolisp-wit-import.md) talks to `*rows*` — same `(kv:bucket-get b "visits")` call sites, same `.wit`, and nothing in the program that knows where the pairs live. ## Arguments - The interface id, as a string: the fully-qualified id from the `.wit` (`"wasi:keyvalue/store@0.2.0"`). A `rontolisp:wit-import` dispatches on that canonical id whichever of the accepted spellings its `:interface` was given (`"wasi:keyvalue/store"` and the bare `store` name the same interface), so this one key binds the provider for all of them. - The provider: any Lisp callable of `(member &rest args)` — a `#'name` function, a `lambda`, or anything else `funcall` accepts. ## The members a provider is asked for One per function the interface declares, spelled as [`wit-import` binds it](rontolisp-wit-import.md#what-gets-bound): `"open"`, `"bucket-get"`, `"bucket-new"` for a constructor — plus, for each resource, **`"-drop"`**, whose only argument is the handle. That last one is a member no `.wit` declares: releasing a resource is a canonical built-in of the component model rather than a function of the interface, so rontolisp names it [`-drop`](rontolisp-wit-import.md#releasing-a-resource-resource-drop) and dispatches it to the provider like any other member. **What a drop *means* is the provider's decision, and only the provider's.** The core knows that the program is done with a handle; it does not know what the handle stood for. So a store that keeps its rows in a hash table forgets the handle and keeps the rows (the `my-store` above answers `nil`, and that is a complete implementation); one holding a JDBC connection or an open file closes it; a provider whose handles cost nothing has nothing to release and simply answers `nil` too. What a drop must **not** do is destroy the thing the handle referred to: a handle is a *reference* to a store, never the store itself, so a later `(kv:open "counts")` must still find every key that was written through the dropped one. Leave the member out and a drop falls into whatever fallback the provider has — in `my-store` above, the `rontolisp:wit-error` clause. So a provider with nothing to release should still answer `nil` for it rather than say nothing at all. ## rontolisp ships no providers The core knows the provider **mechanism**. It does not know what `wasi:keyvalue` is, and it ships no implementation of it — or of any other interface. That is deliberate: a new host interface should cost a `.wit` file, not core code. Calling a bound function before any provider is bound for its interface signals [`rontolisp:wit-error`](#the-wit-error-condition) rather than reaching some default: ```console $ rontolisp counter.lisp No provider is bound for the WIT interface wasi:keyvalue/store@0.2.0 -- bind one with rontolisp:wit-provide ``` Because a provider is *just a function*, a fake and the real thing are interchangeable and the program cannot tell them apart. The [`wit/keyvalue` example](https://github.com/making/rontolisp/tree/develop/examples/wit/keyvalue) is a page-view counter written against `wasi:keyvalue/store` with two stores behind it: a portable in-memory Lisp one, and — on the JVM — one backed by a real `java.util.LinkedHashMap` through [`java:` interop](../../guides/java-interop.md), bound afterwards so that it replaces the first. The store changes; the `(kv:bucket-set b "/index" "3")` call sites do not, and the counter's output is identical either way. A real deployment swaps the map for Redis or a JDBC connection the same way, in one line. Compiling to Preview 1 WASM instead makes the **host** the provider, again with no change to the program. ## The `wit-error` condition `rontolisp:wit-error` is the condition a WIT `result`'s **error arm** signals: the ok arm is the function's return value, the error arm is a condition, on every backend. The provider signals it with the mapped `E` as its payload (above: the `error` variant of `wasi:keyvalue`, a tagged list), and the caller reads the payload back with `(rontolisp:wit-error-payload e)`: ```console ;;; The caller of an imported function, in the same program as the wit-import. (handler-case (kv:bucket-delete *bucket* "visits") (rontolisp:wit-error (e) (print (rontolisp:wit-error-payload e)))) ; (:other "read-only store") ``` It is an ordinary condition class: `handler-case`, `ignore-errors` and `unwind-protect` all work on it, and it is a subclass of `error`, so a bare `(handler-case ... (error (e) ...))` catches it too. ## Backends | Backend | Effect | | --- | --- | | interpreter | binds the provider; imported functions dispatch to it | | JVM (`-o Prog.class`) | the same | | Preview 1 WASM (`-o prog.wasm`) | a top-level form is **dropped** — the WASM host is the provider | | `--component` | the same: dropped, the host (or a composed component) is the provider | | `--no-gc` | it rejects `rontolisp:wit-import` itself | Dropping the form rather than rejecting it is what lets **one source run on every backend**: the `rontolisp:wit-provide` that backs the program on the interpreter is simply inert once the host takes over. ## Limitations - The interface id is matched as a **string**, so a provider bound under `"wasi:keyvalue/store"` does not serve calls dispatching to `"wasi:keyvalue/store@0.2.0"`. Spell it as the `wit-import` `:interface` does. - A provider is global and unscoped: the last `rontolisp:wit-provide` for an interface wins, for the rest of the program. - Nothing is marshalled or type-checked at the boundary on these backends — the provider is handed the Lisp values as they are, and its return value is handed back as it is. The [WIT type table](rontolisp-wit-import.md#supported-wit-types) is the contract to write it against. - On a `rontolisp:wasm-import` declared by hand there is no provider to bind: `rontolisp:wit-provide` serves the interfaces `rontolisp:wit-import` binds. --- # FILE: references/reference/functions/round.md # round `(round number &optional divisor)` Rounds `number` (or `number/divisor` when a divisor is given) to the nearest integer, using banker's rounding: a value exactly halfway between two integers rounds to the even one. In an ordinary (single-value) context the result is the quotient only; the remainder is the second value, observable through [`multiple-value-bind`](../macros/multiple-value-bind.md) and the other multiple-value consumers. ```lisp (round 3.5) ; => 4 ``` ```lisp (round 2.5) ; => 2 ``` ```lisp (multiple-value-bind (q r) (round 7 2) (list q r)) ; => (4 -1) ``` --- # FILE: references/reference/functions/row-major-aref.md # row-major-aref `(row-major-aref array index)` Returns the element of `array` at the given 0-based flat row-major `index`, independent of the array's rank -- element `(i, j)` of a 2x3 array is at flat index `i * 3 + j`. Use [`array-row-major-index`](array-row-major-index.md) to compute the flat index of a set of subscripts. To modify an element, use `row-major-aref` as a `setf` place: `(setf (row-major-aref array k) value)`. Like `aref`, it is not exposed as a first-class function value, so call it directly. ```lisp (let ((m (make-array (list 2 3) :initial-element 0))) (setf (row-major-aref m 4) 9) (aref m 1 1)) ; => 9 ``` --- # FILE: references/reference/functions/rplaca.md # rplaca `(rplaca cons object)` Destructively replaces the car of `cons` with `object`, modifying the cons cell in place. Returns the modified cons cell itself (not the new car), so any other reference to the same cell sees the change. This is the primitive that `setf` of `car` expands to. ```lisp (let ((c (cons 1 2))) (rplaca c 99) c) ; => (99 . 2) ``` --- # FILE: references/reference/functions/rplacd.md # rplacd `(rplacd cons object)` Destructively replaces the cdr of `cons` with `object`, modifying the cons cell in place. Returns the modified cons cell itself, so the original reference now sees the new tail. This is the primitive that `setf` of `cdr` expands to, and the building block for splicing operations like `nconc`. ```lisp (let ((c (cons 1 2))) (rplacd c 99) c) ; => (1 . 99) ``` --- # FILE: references/reference/functions/sbit.md # sbit `(sbit bit-array index)` Reads the bit at `index` of a bit vector (a `#*` literal or a `make-array` result with `:element-type 'bit`, represented as the general vector holding 0/1). `(setf (sbit bit-array index) bit)` writes it. ```lisp (sbit #*0110 1) ; => 1 ``` --- # FILE: references/reference/functions/scale-float.md # scale-float `(scale-float float integer)` Returns `float × 2^integer` with exact IEEE 754 semantics (including the subnormal range). ```lisp (scale-float 1.5 3) ; => 12.0 ``` --- # FILE: references/reference/functions/search.md # search `(search sequence-1 sequence-2 &key start1 end1 start2 end2 test key from-end)` Returns the position in `sequence-2` where `sequence-1` first occurs as a subsequence, or nil when it does not. With `:from-end` the LAST occurrence's start position is returned. `:start1`/`:end1` bound the pattern, `:start2`/`:end2` bound the searched sequence, elements compare with `:test` (default `eql`) after `:key`. Works on strings and lists (a simple O(n*m) scan over `elt`). ```lisp (search "bc" "abcd") ; => 1 ``` ```lisp (search "x" "abcd") ; => NIL ``` ```lisp (search "ab" "ab-ab" :from-end t) ; => 3 ``` --- # FILE: references/reference/functions/second-third-fourth.md # second third fourth `(second list)`, `(third list)`, `(fourth list)` Ordinal accessors for the 2nd, 3rd, and 4th elements of a list, equivalent to `cadr`, `caddr`, and `cadddr` respectively. Each returns `nil` when the list is too short to have that element, since the underlying `car`/`cdr` walk bottoms out at `nil`. ```lisp (third '(a b c d)) ; => C ``` ```lisp (fourth '(a b)) ; => NIL ``` --- # FILE: references/reference/functions/set-difference.md # set-difference `(set-difference list1 list2 &key test key)` Returns a list of the elements of `list1` that do **not** appear in `list2`, treating both as sets. The comparison is `eql` by default; the optional `:test` keyword takes a function designator to use a different comparison, and the optional `:key` keyword takes a selector function applied to both compared elements. The order of elements in the result is unspecified. ```lisp (set-difference '(1 2 3) '(2)) ; => (3 1) ``` ```lisp (set-difference '("a" "b") '("b" "c") :test #'string=) ; => ("a") ``` --- # FILE: references/reference/functions/set-dispatch-macro-character.md # set-dispatch-macro-character `(set-dispatch-macro-character disp-char sub-char function &optional readtable)` Lite stub: accepted and ignored, returning `t` — user dispatch macros cannot extend the Java-side reader. ```lisp (set-dispatch-macro-character #\# #\7 (lambda (s c n) nil)) ; => T ``` --- # FILE: references/reference/functions/set-exclusive-or.md # set-exclusive-or `(set-exclusive-or list1 list2 &key test test-not key)` Returns the symmetric difference of the two lists: the elements of either list that have no match in the other. The comparison is `eql` by default; `:test` takes a function designator, `:test-not` matches where the given function answers false, and `:key` is a selector applied to both compared elements. The comparison is always made with the `list1` element first, in both directions. The result lists the `list1`-only elements in order, then the `list2`-only ones (Common Lisp leaves the order unspecified). ```lisp (set-exclusive-or '(1 2 3) '(2 3 4)) ; => (1 4) ``` ```lisp (set-exclusive-or '("a" "b") '("B" "c") :test #'string-equal) ; => ("a" "c") ``` ```lisp (set-exclusive-or '((1 a) (2 b)) '((2 x)) :key #'car) ; => ((1 A)) ``` --- # FILE: references/reference/functions/signum.md # signum `(signum number)` Returns `-1`, `0`, or `1` indicating the sign of `number`, preserving its numeric type. An integer or ratio argument yields an integer result, while a float argument yields a float result (e.g. `1.0`, `0.0`, `-1.0`). ```lisp (signum -5) ; => -1 ``` ```lisp (signum 3.5) ; => 1.0 ``` --- # FILE: references/reference/functions/simple-condition-format-arguments.md # simple-condition-format-arguments `(simple-condition-format-arguments condition)` The `:format-arguments` slot of a condition instance (nil when the condition has no such slot). ```lisp (simple-condition-format-arguments (make-condition 'simple-error :format-control "boom ~a" :format-arguments '(1))) ; => (1) ``` --- # FILE: references/reference/functions/simple-condition-format-control.md # simple-condition-format-control `(simple-condition-format-control condition)` The `:format-control` slot of a condition instance (nil when the condition has no such slot). With [`simple-condition-format-arguments`](simple-condition-format-arguments.md) it lets a `:report` function re-render a wrapped condition's message. ```lisp (simple-condition-format-control (make-condition 'simple-error :format-control "boom ~a")) ; => "boom ~a" ``` --- # FILE: references/reference/functions/simple-string-p.md # simple-string-p `(simple-string-p object)` Returns true when `object` is a string. Lite: every rontolisp string answers true (there is no separate simple-string representation), so the portable "coerce unless `simple-string-p`" idiom keeps the string unchanged instead of copying. ```lisp (simple-string-p "abc") ; => T ``` ```lisp (simple-string-p 42) ; => NIL ``` --- # FILE: references/reference/functions/sin-cos-tan.md # sin cos tan `(sin radians)` `(cos radians)` `(tan radians)` The three basic trigonometric functions, each taking an angle in radians and returning a float. `sin` is the sine, `cos` the cosine, and `tan` the tangent. The interpreter and JVM backends compute them with `Math.sin`/`Math.cos`/`Math.tan`; the WASM backend uses a software approximation (Cody-Waite argument reduction over quarter-turn quadrants plus Taylor polynomials), so its result may differ slightly in the least significant digits, and for very large arguments (beyond about `2^30`) it progressively loses precision where the JVM stays exact. The zero and quadrant anchors are exact everywhere: `(sin 0)` is `0.0`, `(cos 0)` is `1.0`, `(sin (/ pi 2))` is `1.0`, `(cos pi)` is `-1.0`. `NaN` and infinite arguments give `NaN` on every backend. ```lisp (cos 0) ; => 1.0 ``` --- # FILE: references/reference/functions/sinh-cosh-tanh.md # sinh cosh tanh `(sinh number)` `(cosh number)` `(tanh number)` The hyperbolic functions, each returning a float. `sinh` is the hyperbolic sine, `cosh` the hyperbolic cosine, and `tanh` the hyperbolic tangent. All three work on every backend: the interpreter and JVM use `Math.sinh`/`Math.cosh`/`Math.tanh`, while the WASM backend derives all three from its software `exp` approximation, so its results may differ slightly in the least significant digits (~1e-7 relative for `|x|` up to ~20, degrading for larger arguments like the software `exp` itself; `sinh` switches to a Taylor series below `|x| = 0.25`, so tiny arguments stay accurate). `(sinh 0)` is exactly `0.0` and `(cosh 0)` exactly `1.0` everywhere. ```lisp (tanh 0) ; => 0.0 ``` --- # FILE: references/reference/functions/sleep.md # sleep `(sleep seconds)` Blocks for `seconds` — any non-negative real, so sub-second waits are written as `0.5` — and returns `nil`. A zero or negative duration returns immediately. Durations are rounded to whole milliseconds, the resolution every backend shares with [`get-internal-real-time`](get-internal-real-time.md). The interpreter and the JVM park the thread. `--component` waits on the real host timer (`wasi:clocks`), forcing it through the module scheduler: the wait costs no CPU and other pending tasks still progress. **WASM Preview 1 is the one backend that busy-waits**, looping on the clock until the deadline — its imports include a clock but no timer to wait on, so burning the interval is the only way to let it pass. The wait is honest there, but it costs a core and blocks the instance for its duration. A `--no-wasi` module **signals** instead of waiting: it imports no timer, and its clock only moves when its host writes it, so no interval can elapse while the call is running (see the [clock and randomness guide](../../guides/clock-and-random.md)). ```lisp (sleep 0) ; => NIL ``` ```console (let ((start (get-internal-real-time))) (sleep 0.5) (print (>= (- (get-internal-real-time) start) 500))) ``` --- # FILE: references/reference/functions/some.md # some `(some predicate &rest sequences)` Applies `predicate` to one element of each sequence at a time and returns the first non-nil result, stopping as soon as one is found; if every element fails it returns `nil`. Each sequence may be a list or a string (whose elements are characters). Note the return value is the predicate's result, not necessarily `t`. With more than one sequence the predicate receives one argument per sequence and the walk stops as soon as the shortest one runs out. ```lisp (some #'oddp '(2 4 5)) ; => T ``` ```lisp (some #'digit-char-p "abc1") ; => 1 ``` ```lisp (some #'> '(1 5) '(3 4)) ; => T ``` --- # FILE: references/reference/functions/sort.md # sort `(sort sequence predicate)` Sorts `sequence` using `predicate`, a two-argument comparison function that returns non-nil when its first argument should precede its second. A list is sorted destructively: its cons cells are rearranged in place, so use the return value rather than the original variable. A string sorts as a sequence of its characters and returns a new string (the original string is not modified). The sort is not stable, so the relative order of elements considered equal by `predicate` is unspecified. ```lisp (sort (list 3 1 2) #'<) ; => (1 2 3) ``` ```lisp (sort "cab" #'char<) ; => "abc" ``` --- # FILE: references/reference/functions/special-operator-p.md # special-operator-p `(special-operator-p symbol)` True when the symbol names one of the 25 ANSI special operators -- `block`, `catch`, `eval-when`, `flet`, `function`, `go`, `if`, `labels`, `let`, `let*`, `load-time-value`, `locally`, `macrolet`, `multiple-value-call`, `multiple-value-prog1`, `progn`, `progv`, `quote`, `return-from`, `setq`, `symbol-macrolet`, `tagbody`, `the`, `throw`, `unwind-protect` -- and `nil` for everything else. `nil` includes the Common Lisp MACROS that rontolisp happens to implement as special forms of its own (`defun`, `handler-case`, `dolist`, ...). The question a caller asks is "may I `apply` this name", and those names answer it through [`macro-function`](macro-function.md) instead, exactly as in Common Lisp. ```lisp (list (special-operator-p 'if) (special-operator-p 'defun) (special-operator-p 'car)) ; => (T NIL NIL) ``` Lite: a non-symbol argument answers `nil` where Common Lisp signals a type error. --- # FILE: references/reference/functions/sqrt.md # sqrt `(sqrt number)` Returns the square root of `number` as a float, even when the argument is a perfect square (`(sqrt 16)` is `4.0`, not `4`). The result is always a double-precision float. Works in all three backends (interpreter, JVM, WASM). ```lisp (sqrt 2) ; => 1.4142135623730951 ``` --- # FILE: references/reference/functions/stable-sort.md # stable-sort `(stable-sort sequence predicate &key key)` Sorts `sequence` like [`sort`](sort.md), but preserves the relative order of elements that `predicate` considers equal (neither `(predicate a b)` nor `(predicate b a)` is true). The optional `:key` function is applied to each element before comparison. Unlike Common Lisp's destructive `stable-sort`, the result is always a fresh list — the argument is not modified, and a string or vector argument also comes back as a list of its elements. ```lisp (stable-sort '((1 . b) (0 . a) (1 . a)) #'< :key #'car) ; => ((0 . A) (1 . B) (1 . A)) ``` ```lisp (stable-sort '(3 1 2) #'<) ; => (1 2 3) ``` --- # FILE: references/reference/functions/store-value.md # store-value `(store-value value [condition])` Invokes the innermost active `store-value` restart with `value`, and returns `nil` when none is active — the sibling of [`use-value`](use-value.md) (CL pairs the two: `use-value` supplies a one-off replacement, `store-value` asks the signaler to also store it). ```lisp (handler-bind ((error (lambda (c) (store-value 7)))) (restart-case (error "no value") (store-value (v) (list :stored v)))) ; => (:STORED 7) ``` --- # FILE: references/reference/functions/stream-element-type.md # stream-element-type `(stream-element-type stream)` Always the symbol `character`: every rontolisp stream is a character stream (there are no binary element types). ```lisp (with-input-from-string (s "x") (stream-element-type s)) ; => CHARACTER ``` --- # FILE: references/reference/functions/streamp.md # streamp `(streamp object)` Returns `t` if `object` is a stream and `nil` otherwise. Streams are opaque integer handles across all backends, so this is a lite test equivalent to `integerp`; the standard-output designator `t` (what `*standard-output*` is bound to) also counts as a stream. The `stream` type specifier used by `check-type`/`typecase` is backed by the same test. Its subtype names resolve too: `synonym-stream` has an exact test (a synonym stream is the one stream kind that is a value rather than a handle), while `file-stream` is lite in the same direction `streamp` is -- it is true of every handle stream, because nothing in a handle tells a file's from a string stream's. `readtable` is the type of the opaque `nil` token `*readtable*` holds. Available on all backends except `--no-gc`. ```lisp (with-output-to-string (s) (princ (streamp s) s)) ; => "T" ``` --- # FILE: references/reference/functions/string-capitalize.md # string-capitalize `(string-capitalize string-designator)` Returns a new string in which the first letter of each word is uppercased and the remaining letters of each word are lowercased, where words are runs of alphanumeric characters separated by other characters. The original string is unchanged. The argument is a [string designator](string.md), so a symbol, a keyword or a character is also accepted -- a symbol's name is used and a keyword's leading colon is dropped, so `(string-capitalize :foo-bar)` returns `"Foo-Bar"` and `(string-capitalize nil)` returns `"Nil"`. Anything that is not one of those three types is an error. As with the other case operators the fold is full-Unicode and identical on every backend, and a word constituent is any Unicode letter or digit, so `(string-capitalize "élan vital")` returns `"Élan Vital"`. ```lisp (string-capitalize "hello world") ; => "Hello World" ``` --- # FILE: references/reference/functions/string-compare.md # string< string> string<= string>= string/= string-lessp string-greaterp string-not-greaterp string-not-lessp string-not-equal `(string< string1 string2 &key start1 end1 start2 end2)` -- `(string> ...)` -- `(string<= ...)` -- `(string>= ...)` -- `(string/= ...)` -- `(string-lessp ...)` -- `(string-greaterp ...)` -- `(string-not-greaterp ...)` -- `(string-not-lessp ...)` -- `(string-not-equal ...)` Compare two strings lexicographically and return the mismatch index (a true value) when the relation holds, `nil` otherwise. The index is the position **in `string1`** of the first differing character; when the compared substrings are equal it is `end1`, which is what `string<=` / `string>=` / `string-not-greaterp` / `string-not-lessp` return for equal strings. The first five are case-sensitive; `string-lessp`, `string-greaterp`, `string-not-greaterp`, `string-not-lessp` and `string-not-equal` are the case-insensitive counterparts of `string<`, `string>`, `string<=`, `string>=` and `string/=`. Each argument is coerced with [`string`](string.md), so a symbol or character designator is accepted and anything else is an error. `:start1`/`:end1`/`:start2`/`:end2` bound the substrings actually compared, and the returned index stays absolute in `string1`. ```lisp (list (string< "aaaa" "aaab") (string>= "aaaaa" "aaaa") (string-not-greaterp "Abcde" "abcdE") (string-lessp "012AAAA789" "01aaab6" :start1 3 :end1 7 :start2 2 :end2 6)) ; => (3 4 5 6) ``` --- # FILE: references/reference/functions/string-downcase.md # string-downcase `(string-downcase string-designator)` Returns a new string with every uppercase letter converted to lowercase; the original string is unchanged. The argument is a [string designator](string.md), so a symbol, a keyword or a character is also accepted -- a symbol's name is used and a keyword's leading colon is dropped, so `(string-downcase :FOO)` returns `"foo"` and `(string-downcase #\A)` returns `"a"`. Anything that is not one of those three types is an error. Case conversion is full-Unicode and identical on every backend: each character is folded with `char-downcase`, so `(string-downcase "ÉΛΩ")` returns `"éλω"`. Because the fold is per character, the result always has the same length as the argument and no context-sensitive rule applies (a Greek final sigma is not special-cased). ```lisp (string-downcase "ABC") ; => "abc" ``` --- # FILE: references/reference/functions/string-eq.md # string= `(string= string1 string2 &key start1 end1 start2 end2)` Compares two strings character by character and returns `t` when they are exactly equal, `nil` otherwise. The comparison is case-sensitive, so `"abc"` and `"ABC"` are not equal; use `string-equal` for a case-insensitive test. `:start1`/`:end1`/`:start2`/`:end2` bound the substrings actually compared. ```lisp (list (string= "abc" "abc") (string= "together" "frog" :start1 1 :end1 3 :start2 2)) ; => (T T) ``` --- # FILE: references/reference/functions/string-equal.md # string-equal `(string-equal string1 string2 &key start1 end1 start2 end2)` Compares two strings character by character ignoring case and returns `t` when they match, `nil` otherwise. Case folding follows ASCII rules, so `"ABC"` and `"abc"` are equal. Use `string=` for a case-sensitive comparison. `:start1`/`:end1`/`:start2`/`:end2` bound the substrings actually compared. ```lisp (list (string-equal "ABC" "abc") (string-equal "TOGETHER" "frog" :start1 1 :end1 3 :start2 2)) ; => (T T) ``` --- # FILE: references/reference/functions/string-left-trim.md # string-left-trim `(string-left-trim character-bag string)` Returns a new string with leading characters that appear in `character-bag` removed from the front only; the right end is left intact. `character-bag` is any sequence of characters -- a string, a list, or a vector -- whose members form the set to strip, and trimming stops at the first character not in the bag. `string` is a [string designator](string.md), so `(string-left-trim "F" '|FOO|)` returns `"OO"`. ```lisp (string-left-trim "x" "xxhi") ; => "hi" ``` --- # FILE: references/reference/functions/string-right-trim.md # string-right-trim `(string-right-trim character-bag string)` Returns a new string with trailing characters that appear in `character-bag` removed from the end only; the left end is left intact. `character-bag` is any sequence of characters -- a string, a list, or a vector -- whose members form the set to strip, and trimming stops at the last character not in the bag. `string` is a [string designator](string.md), so `(string-right-trim "O" '|FOO|)` returns `"F"`. ```lisp (string-right-trim "x" "hixx") ; => "hi" ``` --- # FILE: references/reference/functions/string-trim.md # string-trim `(string-trim character-bag string)` Returns a new string with all leading and trailing characters that appear in `character-bag` removed; interior characters are untouched. `character-bag` is any sequence of characters -- a string, a list, or a vector -- whose members form the set to strip. Trimming stops at the first character on each end that is not in the bag. `string` is a [string designator](string.md), so a symbol or a character is accepted and its name is trimmed: `(string-trim "*" '*foo*)` returns `"FOO"`. `character-bag` is NOT a designator -- it is a sequence -- so a lone character there is an error, not a one-character bag. ```lisp (string-trim " " " hi ") ; => "hi" ``` ```lisp (string-trim (list #\Space #\Tab) " hi ") ; => "hi" ``` --- # FILE: references/reference/functions/string-upcase.md # string-upcase `(string-upcase string-designator)` Returns a new string with every lowercase letter converted to uppercase; the original string is unchanged. The argument is a [string designator](string.md), so a symbol, a keyword or a character is also accepted -- a symbol's name is used and a keyword's leading colon is dropped, so `(string-upcase :foo)` returns `"FOO"` and `(string-upcase #\a)` returns `"A"`. Anything that is not one of those three types is an error. Case conversion is full-Unicode and identical on every backend: each character is folded with `char-upcase`, so `(string-upcase "éλω")` returns `"ÉΛΩ"`. Because the fold is per character, the result always has the same length as the argument -- there is no multi-character special casing (`(string-upcase "straße")` returns `"STRAßE"`, not `"STRASSE"`). ```lisp (string-upcase "abc") ; => "ABC" ``` --- # FILE: references/reference/functions/string.md # string `(string x)` Coerces a *string designator* to a string. A string is returned unchanged, a symbol yields its [`symbol-name`](symbol-name.md) (a keyword's leading `:` and a gensym's `#:` are package markers and are stripped), and a character yields a one-character string. `t` and `nil` coerce like symbols (`"T"` / `"NIL"`). Symbols read upcased like Common Lisp, so `(string 'foo)` is `"FOO"` and `(string 'car)` is `"CAR"`. A non-designator argument signals an error on every backend. `string` is the single coercion every string-designator position routes through -- the [`string-trim`](string-trim.md) family's trimmed value, the case operators, [`string=`](string-eq.md) and the ordering predicates -- so anything it accepted silently would become a wrong answer there instead of a type error. ```lisp (string 'foo) ; => "FOO" ``` ```lisp (string #\a) ; => "a" ``` ```lisp (string "already") ; => "already" ``` --- # FILE: references/reference/functions/stringp.md # stringp `(stringp object)` Returns `t` if `object` is a string, otherwise `nil`. Symbols are not strings, so `(stringp 'hello)` is `nil`. Works in all three backends. ```lisp (stringp "hello") ; => T ``` ```lisp (stringp 'hello) ; => NIL ``` --- # FILE: references/reference/functions/subseq.md # subseq `(subseq sequence start &optional end)` Returns a fresh subsequence of `sequence` (a string or a list) covering the half-open range from index `start` up to but not including `end`, using 0-based indexing. When `end` is omitted the subsequence runs to the end of the sequence. The result has the same type as the input -- a string for a string, a list for a list. ```lisp (subseq "hello" 1 3) ; => "el" ``` --- # FILE: references/reference/functions/subst.md # subst `(subst new old tree &key test key)` Non-destructive tree substitution: returns a copy of `tree` with every subtree or leaf matching `old` replaced by `new`. A match is decided by `(funcall test old (funcall key subtree))`; `:test` defaults to `eql` (so by default only atoms match) and `:key` defaults to the subtree itself. Unchanged subtrees are shared with the original, not copied. ```lisp (subst 'x 'a '(a (b a) c)) ; => (X (B X) C) ``` ```lisp (subst 9 '(m) '(f (m) g) :test #'equal) ; => (F 9 G) ``` --- # FILE: references/reference/functions/substitute-if-not.md # substitute-if-not `(substitute-if-not new predicate sequence &key key)` The complement of [`substitute-if`](substitute-if.md): returns a new sequence in which every element the predicate *rejects* is replaced by `new`. Takes the same optional `:key` selector, keeps the sequence kind, and does not modify the original; the destructive version is [`nsubstitute-if-not`](nsubstitute-if-not.md). ```lisp (substitute-if-not 0 #'oddp '(1 2 3 4 5)) ; => (1 0 3 0 5) ``` ```lisp (substitute-if-not 'keep #'stringp '("a" 1 "b")) ; => ("a" KEEP "b") ``` --- # FILE: references/reference/functions/substitute-if.md # substitute-if `(substitute-if new predicate sequence &key key)` Returns a new sequence in which every element satisfying `predicate` is replaced by `new`; all other elements are kept unchanged. It is [`substitute`](substitute.md) with the `eql` comparison replaced by a predicate call, so it takes no `:test` — the predicate *is* the test. The optional `:key` keyword takes a selector function applied to each element before the predicate sees it (the replacement value is `new` itself, unkeyed). The sequence may be a list, a string or a vector, and the result keeps that kind. The original sequence is not modified; use [`nsubstitute-if`](nsubstitute-if.md) for the destructive version (lists only). ```lisp (substitute-if 0 #'oddp '(1 2 3 4 5)) ; => (0 2 0 4 0) ``` ```lisp (substitute-if #\- (lambda (c) (member c '(#\. #\/) :test 'char=)) "lack/mw.backtrace") ; => "lack-mw-backtrace" ``` ```lisp (substitute-if 0 #'oddp '((1) (2) (3)) :key #'car) ; => (0 (2) 0) ``` --- # FILE: references/reference/functions/substitute.md # substitute `(substitute new old sequence &key test key)` Returns a new sequence in which every element matching `old` is replaced by `new`; all other elements are kept unchanged. The comparison is `eql` by default; the optional `:test` keyword takes a function designator to use a different comparison, and the optional `:key` keyword takes a selector function applied to each element before the comparison (the replacement value is `new` itself, unkeyed). The sequence may be a list or a string; a string yields a new string (with `new` a character). The original sequence is not modified; use `nsubstitute` for the destructive version (lists only). ```lisp (substitute 0 2 '(1 2 3 2)) ; => (1 0 3 0) ``` ```lisp (substitute #\o #\a "banana") ; => "bonono" ``` ```lisp (substitute "X" "b" '("a" "b" "c") :test #'string=) ; => ("a" "X" "c") ``` --- # FILE: references/reference/functions/subtypep.md # subtypep `(subtypep type1 type2)` Whether `type1` names a subtype of `type2`, answering over the built-in type lattice (e.g. `integer` ⊂ `rational` ⊂ `real` ⊂ `number`, `string` ⊂ `vector` ⊂ `array`/`sequence`) plus the class registry's ancestor sets (`defclass`/`define-condition` hierarchies). Lite: a single primary value — an unknown pair answers nil. The float and character type names collapse to the one runtime representation, so `(subtypep 'short-float 'single-float)` is `t`. Either argument may be a class metaobject instead of a type name: what [`find-class`](find-class.md) and [`class-of`](class-of.md) answer designates its own class, so a metaobject compares exactly like the name spelling. Both arguments may also be computed at run time. On the JVM and WASM compilers a literal (quoted) pair is folded into a constant at compile time; anything else is answered at run time over the same lattice. ```lisp (subtypep 'integer 'number) ; => T ``` ```lisp (subtypep 'type-error 'error) ; => T ``` ```lisp (defclass animal () ()) (defclass dog (animal) ()) (list (subtypep (find-class 'dog) (find-class 'animal)) (subtypep (find-class 'animal) (find-class 'dog))) ; => (T NIL) ``` --- # FILE: references/reference/functions/svref.md # svref `(svref vector index)` Returns the element of a rank-1 array at the given 0-based `index`. It behaves like [`aref`](aref.md) restricted to exactly one subscript, and is likewise a `setf` place: `(setf (svref v i) value)` replaces an element. `#'svref` is a first-class function value, so it can be passed to `mapcar`/`funcall` like any other function. ```lisp (svref (vector 10 20 30) 1) ; => 20 (let ((v (vector 1 2 3))) (setf (svref v 0) 99) (svref v 0)) ; => 99 ``` --- # FILE: references/reference/functions/sxhash.md # sxhash `(sxhash object)` Returns a non-negative integer hash of the object, hashing integers, characters, strings, symbols, and conses by structural content (anything else hashes to 0). Values are stable within a run but NOT specified across backends. ```lisp (= (sxhash "ab") (sxhash "ab")) ; => T ``` --- # FILE: references/reference/functions/symbol-function.md # symbol-function `(symbol-function symbol)` Returns the function value bound to `symbol` in the function namespace -- the same value `#'name` denotes. The result can be passed to `funcall`/`apply` or stored. Because rontolisp is a Lisp-2, this looks only in the function namespace, never at a variable of the same name. A quoted symbol literal (`(symbol-function 'car)`) resolves at compile time in the compilers; a runtime-computed symbol resolves late through the compiled name registry when the result is called -- with two deviations there: `functionp` of the result answers `nil`, and an undefined name signals at the call rather than at `symbol-function` itself. ```lisp (funcall (symbol-function 'car) '(1 2 3)) ; => 1 ``` `symbol-function` is also a `setf` place: `(setf (symbol-function 'name) fn)` installs `fn` as the symbol's global function definition -- defining an alias for an existing function, or replacing one. In the compilers a call site the compiler already bound directly keeps the original function ([`fmakunbound`](fmakunbound.md)'s divergence); a name bound ONLY this way is fully late-bound, and calling it before the assignment signals `The function NAME is undefined`. ```lisp (defun double (x) (* x 2)) (setf (symbol-function 'twice) #'double) (twice 21) ; => 42 ``` --- # FILE: references/reference/functions/symbol-name.md # symbol-name `(symbol-name symbol)` Returns the symbol's name as a string. The reader upcases unescaped symbols like Common Lisp (see the [reader case guide](../../guides/reader-case.md)), so a symbol reports its upcased name — `(symbol-name 'foo)` is `"FOO"` and `(symbol-name 'car)` is `"CAR"`, the Common Lisp answer. A keyword's leading `:`, a [`gensym`](gensym.md)/[`make-symbol`](make-symbol.md) result's `#:` prefix and a package qualifier are all where the symbol lives rather than part of its name, and are stripped — the same text [`princ`](princ.md) prints (`prin1` keeps them). `nil` and `t` are the symbols `NIL` and `T`, so they name themselves upcased like any other symbol. On the compiled backends (JVM/WASM) `symbol-name` shares the `princ-to-string` machinery, so a non-symbol argument yields its display text instead of signaling an error (the interpreter signals). The reader supports the Common Lisp escape syntaxes for symbol names: a backslash makes the next character part of the name verbatim, and a `|...|` multiple escape makes everything between the pipes part of the name — whitespace and terminating characters included — so `'|when used|` is one symbol named `"when used"` and `'|Foo|` keeps its mixed case. ```lisp (symbol-name 'foo) ; => "FOO" ``` ```lisp (symbol-name :bar) ; => "BAR" ``` ```lisp (symbol-name 'car) ; => "CAR" ``` ```lisp (intern (symbol-name 'round-trip)) ; => ROUND-TRIP ``` ```lisp (symbol-name '|when used|) ; => "when used" ``` --- # FILE: references/reference/functions/symbol-package.md # symbol-package `(symbol-package symbol)` Lite: returns the same keyword shape [`find-package`](find-package.md) returns, so the two are `eq`-comparable: `:keyword` for a keyword, the qualifier of a package-qualified symbol, `:cl` for a standard symbol, `:cl-user` otherwise, and `nil` for an uninterned (`#:`) symbol. The compiled backends have no package registry at run time and cannot tell `cl` from `cl-user`: they answer `:cl-user` for both. ```lisp (symbol-package :foo) ; => :KEYWORD ``` --- # FILE: references/reference/functions/symbol-plist.md # symbol-plist `(symbol-plist symbol)` Returns the symbol's whole property list — the indicator/value pairs [`get`](get.md) indexes into — or `nil` when it has none. Symbols have no identity cells to hang plists on (they compare by name), so the list comes out of the same program-global name-keyed store `get` writes to. There is no `(setf symbol-plist)`. ```lisp (symbol-plist 'no-props) ; => NIL ``` ```lisp (setf (get 'my-sym 'color) :red) (symbol-plist 'my-sym) ; => (COLOR :RED) ``` --- # FILE: references/reference/functions/symbol-value.md # symbol-value `(symbol-value symbol)` Returns the value of the **global** variable named by `symbol`; an unbound name signals an error (a trap on WASM). Like Common Lisp's dynamic-only `symbol-value`, lexical bindings are invisible. `t`, `nil` and keywords evaluate to themselves. Use [`boundp`](boundp.md) to test first, and [`intern`](intern.md) to build the name at runtime. ```lisp (defvar *level* 7) (symbol-value '*level*) ; => 7 ``` ```lisp (symbol-value (intern "*LEVEL*")) ; => 7 ``` ```lisp (symbol-value :key) ; => :KEY ``` An unbound variable signals an error: ```console > (symbol-value '*nope*) The variable *nope* is unbound ``` --- # FILE: references/reference/functions/symbolp.md # symbolp `(symbolp object)` Returns `t` if `object` is a symbol, otherwise `nil`. Note that `nil` and keywords are symbols, so `(symbolp nil)` and `(symbolp :foo)` are both `t`. Quoted symbols and string literals share a runtime representation but are distinguished by a leading quote character, so strings are not symbols. Works in all three backends. ```lisp (symbolp 'foo) ; => T ``` ```lisp (symbolp "foo") ; => NIL ``` --- # FILE: references/reference/functions/synonym-stream-symbol.md # synonym-stream-symbol `(synonym-stream-symbol stream)` Returns the symbol a synonym stream forwards to, i.e. the argument [`make-synonym-stream`](make-synonym-stream.md) was given. Signals when `stream` is not a synonym stream. ```lisp (synonym-stream-symbol (make-synonym-stream '*standard-output*)) ; => *STANDARD-OUTPUT* ``` --- # FILE: references/reference/functions/terpri.md # terpri `(terpri &optional stream)` Writes a single newline unconditionally and returns nil. With no argument it writes to standard output; with the optional stream argument (a file stream or a `with-output-to-string` string stream) it writes to that stream. The name is short for "terminate print"; use it to end a line built up with `princ` or `prin1`. ```lisp (princ "a") (terpri) (princ "b") ``` ``` a b ``` --- # FILE: references/reference/functions/torch-adam.md # torch:adam `(torch:adam params &key lr betas eps)` Returns an Adam optimizer (PyTorch's `torch.optim.Adam`) over `params`, a module or a list of parameter tensors. `:lr` defaults to `0.001`, `:betas` to `(0.9 0.999)` (PyTorch's `(beta1, beta2)` tuple, as a two-element list) and `:eps` to `1.0e-8`. Per element, [`torch:step`](torch-step.md) computes ```text m <- beta1 * m + (1 - beta1) * grad v <- beta2 * v + (1 - beta2) * grad^2 param <- param - lr * (m / (1 - beta1^t)) / (sqrt(v / (1 - beta2^t)) + eps) ``` `t` is the optimizer's own [`torch:step-count`](torch-step-count.md), which is `1` during the first step -- so the first update is fully bias-corrected and has magnitude `lr`, not `lr * (1 - beta1)`. ```lisp (defparameter *w* (torch:parameter '(1.0))) (defparameter *opt* (torch:adam (list *w*) :lr 0.125)) (torch:backward (torch:sum (torch:mul *w* *w*))) (torch:step *opt*) (torch:step-count *opt*) ; => 1 (< (abs (- (torch:item *w*) 0.875)) 1.0e-8) ; => T ``` --- # FILE: references/reference/functions/torch-adamw.md # torch:adamw `(torch:adamw params &key lr betas eps weight-decay)` Returns an AdamW optimizer (PyTorch's `torch.optim.AdamW`) over `params`, a module or a list of parameter tensors. [`torch:adam`](torch-adam.md)'s rule with DECOUPLED weight decay: the parameter shrinks on its own before the Adam step, instead of the decay entering the gradient and being rescaled by the adaptive denominator. Per element, [`torch:step`](torch-step.md) computes ```text param <- param - lr * weight-decay * param m <- beta1 * m + (1 - beta1) * grad v <- beta2 * v + (1 - beta2) * grad^2 param <- param - lr * (m / (1 - beta1^t)) / (sqrt(v / (1 - beta2^t)) + eps) ``` `:lr` defaults to `0.001`, `:betas` to `(0.9 0.999)`, `:eps` to `1.0e-8` and `:weight-decay` to `0.01` -- PyTorch's default, against `torch:adam`'s `0`. This is the rule a transformer is trained with, and a parameter that must NOT decay (a bias, a LayerNorm gain, an embedding table) belongs in a SECOND optimizer built with `:weight-decay 0.0`: two optimizers over disjoint parameter lists are what `torch.optim`'s parameter GROUPS express here. ```lisp (defparameter *w* (torch:parameter '(1.0))) (defparameter *opt* (torch:adamw (list *w*) :lr 0.1 :weight-decay 0.5)) (torch:backward (torch:sum (torch:mul *w* *w*))) (torch:step *opt*) (< (abs (- (torch:item *w*) 0.85)) 1.0e-8) ; => T ``` --- # FILE: references/reference/functions/torch-add.md # torch:add `(torch:add a b)` Differentiable elementwise `a + b` with numpy-style broadcasting (`linalg:add`); either operand may be a tensor, a number, an array or a list. The backward pass sums the gradient over every broadcast axis, so a `(d)` bias added to a `(b s d)` activation gets the `(d)` gradient it should. ```lisp (torch:data (torch:add (torch:tensor '((1.0 2.0) (3.0 4.0))) (torch:tensor '(10.0 20.0)))) ; => #d((11.0 22.0) (13.0 24.0)) (torch:data (torch:add (torch:tensor '(1.0 2.0)) 0.5)) ; => #d(1.5 2.5) ``` --- # FILE: references/reference/functions/torch-amax.md # torch:amax `(torch:amax a &key axis keepdims)` Differentiable maximum, of every element or along an axis (`linalg:amax`'s rules). The gradient flows to every element equal to the maximum, split evenly among ties (PyTorch's `amax` rule). ```lisp (torch:item (torch:amax (torch:tensor '(1.0 5.0 3.0)))) ; => 5.0 (torch:data (torch:amax (torch:tensor '((1.0 4.0) (3.0 2.0))) :axis 1)) ; => #d(4.0 3.0) ``` --- # FILE: references/reference/functions/torch-argmax.md # torch:argmax `(torch:argmax a &key axis)` Non-differentiable: returns the index of the largest element (`linalg:argmax`) as a raw value, not a tensor -- the integer index for a vector, the per-slice index array with `:axis`. The indices feed [`torch:gather`](torch-gather.md) / [`torch:index-select`](torch-index-select.md), and greedy decoding reads its result directly. ```lisp (torch:argmax (torch:tensor '(1.0 5.0 3.0))) ; => 1 (torch:argmax (torch:tensor '((1.0 4.0) (3.0 2.0))) :axis 1) ; => #d(1.0 0.0) ``` --- # FILE: references/reference/functions/torch-backward.md # torch:backward `(torch:backward tensor)` Runs reverse-mode automatic differentiation from a scalar (one-element) tensor: seeds its gradient with `1.0`, walks the recorded tape in reverse topological order, and accumulates each operation's input gradients into its parents -- so a tensor reached over more than one path (a residual connection, a reused embedding row) collects the sum. Read the results with [`torch:grad`](torch-grad.md); returns `nil`. A tensor with more than one element signals. Gradients are retained on intermediate tensors too, and repeated backward calls keep accumulating -- clear parameters with [`torch:zero-grad`](torch-zero-grad.md) between training steps. ```lisp (defparameter *w* (torch:tensor '(1.0 2.0) :requires-grad t)) (defparameter *loss* (torch:sum (torch:mul *w* *w*))) (torch:backward *loss*) (torch:grad *w*) ; => #d(2.0 4.0) ``` --- # FILE: references/reference/functions/torch-cat.md # torch:cat `(torch:cat tensors &key axis)` Differentiable concatenation of the list `tensors` along an existing axis (`linalg:concatenate`, `torch.cat`; default 0, negative counts from the end). The backward pass slices the gradient back into each input's extent along that axis. ```lisp (torch:data (torch:cat (list (torch:tensor '(1.0 2.0)) (torch:tensor '(3.0))))) ; => #d(1.0 2.0 3.0) ``` --- # FILE: references/reference/functions/torch-clip-grad-norm.md # torch:clip-grad-norm `(torch:clip-grad-norm params max-norm)` Gradient-norm clipping (PyTorch's `torch.nn.utils.clip_grad_norm_`) over `params`: a module, an optimizer's parameter list, or a plain list of tensors. Returns the TOTAL L2 norm of every gradient, taken over all of them at once as if they were one long vector -- the norm as MEASURED, before any clipping, so a training loop can log it. When that norm exceeds `max-norm`, every gradient is scaled IN PLACE by `max-norm / (norm + 1e-6)`, PyTorch's denominator; otherwise nothing is touched. A parameter no gradient reached is skipped. Call it between [`torch:backward`](torch-backward.md) and [`torch:step`](torch-step.md): it rewrites the gradients the optimizer is about to read, and touches no tape. ```lisp (defparameter *w* (torch:parameter '(3.0 4.0))) (torch:backward (torch:sum (torch:mul *w* *w*))) ; grad = (6 8), norm 10 (< (abs (- (torch:clip-grad-norm (list *w*) 1.0) 10.0)) 1.0e-9) ; => T (< (abs (- (aref (torch:grad *w*) 0) 0.6)) 1.0e-6) ; => T ``` --- # FILE: references/reference/functions/torch-cross-entropy-loss.md # torch:cross-entropy-loss `(torch:cross-entropy-loss logits targets &key ignore-index reduction)` Returns the cross entropy over raw **logits** as a scalar tensor (PyTorch's `nn.CrossEntropyLoss`). The logits have shape `(... num-classes)` -- the leading axes are flattened, so `(batch seq vocab)` works directly. It is computed from `-log-softmax`, which is the numerically stable form; do **not** pass softmax outputs. The target is read one of two ways: - **class indices** of the matching leading shape -- a number, a list, an index vector or a tensor. The loss is `-log-softmax` picked at the target class. - **class probabilities** -- a tensor or array of the logits' own shape, PyTorch's soft-label form. The loss is `-sum(target * log-softmax(logits))` per position, and the gradient flows into the target too when it requires one. A LIST is always class indices, so the probability spelling needs a tensor or an array. `:ignore-index k` drops every position whose class-index target is `k` from both the sum and the mean's denominator, which is what keeps padding positions from contributing; like PyTorch it does not apply to probability targets. `:reduction :sum` adds instead of averaging and `:reduction :none` returns the per-position tensor. ```lisp (torch:item (torch:cross-entropy-loss (torch:tensor '((0.0 0.0))) #(0))) ; => 0.6931471805599453 (torch:item (torch:cross-entropy-loss (torch:tensor '((0.0 0.0) (0.0 0.0))) #(0 1) :ignore-index 1)) ; => 0.6931471805599453 (torch:item (torch:cross-entropy-loss (torch:tensor '((0.0 0.0))) (torch:tensor '((0.5 0.5))))) ; => 0.6931471805599453 ``` --- # FILE: references/reference/functions/torch-data.md # torch:data `(torch:data tensor)` Returns the tensor's data: a linalg array (packed float, any rank), or a number for a scalar tensor. This is the raw array the `linalg` functions accept, so a value leaves the differentiable layer through this reader (use [`torch:detach`](torch-detach.md) to stay a tensor while leaving the tape). ```lisp (torch:data (torch:tensor '((1.0 2.0) (3.0 4.0)))) ; => #d((1.0 2.0) (3.0 4.0)) (torch:data (torch:sum (torch:tensor '(1.0 2.0)))) ; => 3.0 ``` --- # FILE: references/reference/functions/torch-detach.md # torch:detach `(torch:detach tensor)` Returns a new leaf tensor sharing the tensor's data but cut off from the autograd tape: no `requires-grad`, no recorded history, so nothing computed from it flows gradients back. The whole-block spelling is the [`torch:no-grad`](../macros/torch-no-grad.md) macro. ```lisp (defparameter *w* (torch:tensor '(1.0 2.0) :requires-grad t)) (defparameter *y* (torch:mul *w* 3.0)) (torch:requires-grad-p *y*) ; => T (torch:requires-grad-p (torch:detach *y*)) ; => NIL ``` --- # FILE: references/reference/functions/torch-div.md # torch:div `(torch:div a b)` Differentiable elementwise `a / b` with numpy-style broadcasting (`linalg:div`): the numerator's gradient is `g / b`, the denominator's `-g * a / b^2`. ```lisp (torch:data (torch:div (torch:tensor '(6.0 9.0)) (torch:tensor '(2.0 3.0)))) ; => #d(3.0 3.0) ``` --- # FILE: references/reference/functions/torch-dropout.md # torch:dropout `(torch:dropout p)` Returns a dropout layer (PyTorch's `nn.Dropout`) with drop probability p, in the single field `:p`. In **training** mode it zeroes each element with probability p and scales the survivors by `1 / (1 - p)` (inverted dropout, so the expectation is unchanged); in **evaluation** mode ([`torch:eval`](torch-eval.md)) it is the identity, and so is `p` 0. The mask comes from the seeded [`linalg:seed`](linalg-seed.md) generator, so a seeded training run reproduces on every backend. ```lisp (defparameter *drop* (torch:dropout 0.5)) (torch:data (torch:forward (torch:eval *drop*) (torch:tensor '(1.0 2.0)))) ; => #d(1.0 2.0) (torch:data (torch:forward (torch:dropout 0) (torch:tensor '(1.0 2.0)))) ; => #d(1.0 2.0) ``` --- # FILE: references/reference/functions/torch-embedding.md # torch:embedding `(torch:embedding num-embeddings embedding-dim)` Returns an embedding table (PyTorch's `nn.Embedding`): the single field `:weight`, a `(num-embeddings embedding-dim)` parameter drawn from the standard normal like PyTorch's default. The forward takes integer indices of **any** shape and returns them with the embedding axis appended; a row selected twice accumulates both gradients ([`torch:index-select`](torch-index-select.md)'s adjoint). ```lisp (defparameter *emb* (torch:embedding 4 2)) (torch:set-field *emb* :weight (torch:parameter '((0.0 1.0) (2.0 3.0) (4.0 5.0) (6.0 7.0)))) (torch:data (torch:forward *emb* #(2 0))) ; => #d((4.0 5.0) (0.0 1.0)) (torch:shape (torch:forward *emb* #2A((1 2) (3 0)))) ; => (2 2 2) ``` --- # FILE: references/reference/functions/torch-erf.md # torch:erf `(torch:erf a)` Differentiable elementwise Gauss error function (PyTorch's `torch.erf`, over [`linalg:erf`](linalg-erf.md)). The adjoint is the Gaussian `2 / sqrt(pi) * e^(-x^2)`, so the gradient is exact rather than an approximation of the forward approximation. ```lisp (defparameter *x* (torch:tensor '(0.0) :requires-grad t)) (torch:backward (torch:sum (torch:erf *x*))) (< (abs (- (torch:item (torch:tensor (torch:grad *x*))) 1.1283791670955126)) 1.0e-12) ; => T ``` --- # FILE: references/reference/functions/torch-eval.md # torch:eval `(torch:eval module)` Puts the module and every submodule into **evaluation** mode (PyTorch's `nn.Module.eval`) and returns the module: [`torch:dropout`](torch-dropout.md) becomes the identity. Inference additionally wants [`torch:no-grad`](../macros/torch-no-grad.md), which is a separate, orthogonal switch. ```lisp (defparameter *drop* (torch:dropout 0.5)) (torch:training-p (torch:eval *drop*)) ; => NIL (torch:data (torch:forward *drop* (torch:tensor '(1.0 2.0)))) ; => #d(1.0 2.0) ``` --- # FILE: references/reference/functions/torch-exp.md # torch:exp `(torch:exp a)` Differentiable elementwise `e^x` (`linalg:exp`); the backward pass reuses the forward result (`d/dx e^x = e^x`). ```lisp (torch:data (torch:exp (torch:tensor '(0.0 1.0)))) ; => #d(1.0 2.718281828459045) ``` --- # FILE: references/reference/functions/torch-field.md # torch:field `(torch:field module name)` Returns the value of the module's named field, `name` being the field's keyword: a parameter, a buffer, a submodule, a list of submodules or a plain hyper-parameter. Signals when the module has no such field, so a misspelled name is loud rather than silently `NIL`. This is how a layer's forward reads its own parameters (see [`torch:module`](torch-module.md)). ```lisp (torch:shape (torch:field (torch:linear 3 2) :weight)) ; => (3 2) (torch:shape (torch:field (torch:linear 3 2) :bias)) ; => (2) ``` --- # FILE: references/reference/functions/torch-fields.md # torch:fields `(torch:fields module)` The whole fields plist of a module or an optimizer, as a FRESH list: the field names in registration order, each followed by its value. [`torch:field`](torch-field.md) reads ONE field by name; this is what makes a module tree WALKABLE from outside the package. `nn.Module.apply` and `nn.Module.named_parameters` have no counterpart here because a walk is written over this plist plus [`torch:module-kind`](torch-module-kind.md), which says what each layer IS -- more precise than PyTorch's dotted parameter names, where selecting the LayerNorm parameters by testing for `'ln'` in the name also selects a layer someone called `blend`. The spine is fresh, so consing onto the result cannot corrupt the module; the VALUES are the live parameters and submodules, and the way to replace one is still [`torch:set-field`](torch-set-field.md). ```lisp (defparameter *layer* (torch:linear 3 2)) (do ((p (torch:fields *layer*) (cddr p)) (acc nil (cons (car p) acc))) ((null p) (reverse acc))) ; => (:WEIGHT :BIAS) (eq (nth 1 (torch:fields *layer*)) (torch:field *layer* :weight)) ; => T ``` --- # FILE: references/reference/functions/torch-forward.md # torch:forward `(torch:forward module &rest args)` Runs a module's forward pass -- `(funcall its forward-fn module args...)` -- and returns the output tensor. A plain **function** is also accepted and simply applied, so a stateless step (an activation, a reshape) can sit in a [`torch:sequential`](torch-sequential.md) without a wrapper layer existing for it; that is why the package has no activation-module type. ```lisp (defparameter *lin* (torch:linear 2 2)) (torch:set-field *lin* :weight (torch:parameter '((1.0 0.0) (0.0 -1.0)))) (torch:set-field *lin* :bias (torch:parameter '(0.0 0.0))) (torch:data (torch:forward *lin* (torch:tensor '((2.0 3.0))))) ; => #d((2.0 -3.0)) (torch:data (torch:forward (function torch:relu) (torch:tensor '(-1.0 2.0)))) ; => #d(0.0 2.0) ``` --- # FILE: references/reference/functions/torch-gather.md # torch:gather `(torch:gather a idx)` Differentiable per-row selection of a matrix: element `a[i, idx[i]]` for each row `i`, as a vector (`linalg:gather` -- the "pick the target logit" idiom of a cross-entropy loss). `idx` may be an index vector, a list or a tensor. The backward pass scatters the gradient back to the picked cells. ```lisp (torch:data (torch:gather (torch:tensor '((1.0 2.0 3.0) (4.0 5.0 6.0))) #(2 0))) ; => #d(3.0 4.0) ``` --- # FILE: references/reference/functions/torch-gelu.md # torch:gelu `(torch:gelu a &key approximate)` The Gaussian error linear unit (PyTorch's `nn.GELU` / `torch.nn.functional.gelu`), composed from torch operations and therefore differentiable with no adjoint of its own. `:approximate` selects the formulation: | `:approximate` | formula | PyTorch | | --- | --- | --- | | `:none` (default) | `x * (1 + erf(x / sqrt(2))) / 2` | `approximate='none'` | | `:tanh` | `x * (1 + tanh(sqrt(2/pi) * (x + 0.044715 x^3))) / 2` | `approximate='tanh'` | The default is the EXACT `x * P(X <= x)` for a standard normal `X`, over [`torch:erf`](torch-erf.md); the `:tanh` form is the GPT/BERT formulation and agrees with it to about `1e-3`. Unlike [`torch:relu`](torch-relu.md) it is smooth everywhere and passes a small negative gradient, which is why a transformer feed-forward block uses it. Accuracy is not the only axis: under [`--simd`](../../guides/simd-acceleration.md#accelerating-linalg) the `:tanh` form is accelerated (it is `mul` / `add` / `tanh`) and the default is not, because [`linalg:erf`](linalg-erf.md) is not among the intercepted kernels. ```lisp (torch:data (torch:gelu (torch:tensor '(-1.0 0.0 1.0)))) ; => #d(-0.15865525393145702 0.0 0.841344746068543) ``` --- # FILE: references/reference/functions/torch-grad.md # torch:grad `(torch:grad tensor)` Returns the gradient [`torch:backward`](torch-backward.md) accumulated into the tensor -- a raw linalg value of the data's shape -- or `nil` before any backward pass has reached it. Gradients accumulate (`+=`) across backward calls; [`torch:zero-grad`](torch-zero-grad.md) clears the slot. ```lisp (defparameter *w* (torch:tensor '(1.0 2.0) :requires-grad t)) (torch:grad *w*) ; => NIL (torch:backward (torch:sum (torch:mul *w* *w*))) (torch:grad *w*) ; => #d(2.0 4.0) ``` --- # FILE: references/reference/functions/torch-index-select.md # torch:index-select `(torch:index-select a idx)` Differentiable axis-0 slice selection (`linalg:take-rows`) -- the embedding lookup: row `idx[i]` of the table for each `i`, any rank >= 1, and the same index may repeat. The backward pass scatter-adds each output slab's gradient back into its source row, so a row selected twice accumulates both contributions (the shared-embedding case). ```lisp (torch:data (torch:index-select (torch:tensor '((1.0 2.0) (3.0 4.0))) #(1 0 1))) ; => #d((3.0 4.0) (1.0 2.0) (3.0 4.0)) ``` --- # FILE: references/reference/functions/torch-item.md # torch:item `(torch:item tensor)` Returns the single element of a scalar (or one-element) tensor as a number -- the way a loss value leaves the graph for printing or logging. A tensor with more than one element signals. ```lisp (torch:item (torch:sum (torch:tensor '(1.0 2.0 3.0)))) ; => 6.0 (torch:item (torch:tensor '(7.0))) ; => 7.0 ``` --- # FILE: references/reference/functions/torch-layer-norm.md # torch:layer-norm `(torch:layer-norm d-model &key eps)` Returns a layer-normalization layer over the last axis (PyTorch's `nn.LayerNorm`): fields `:weight` (a `(d-model)` parameter of ones), `:bias` (a `(d-model)` parameter of zeros) and the `:eps` hyper-parameter, `1.0e-5` by default. The forward is `(x - mean) / sqrt(var + eps) * weight + bias` with the **biased** variance (`ddof` 0, PyTorch's `unbiased=False`). The whole expression is composed from `torch` operations, so the normalization itself is differentiable -- the gradient flows through the mean and the variance too, not just through the affine parameters. ```lisp (torch:data (torch:forward (torch:layer-norm 2 :eps 0.0) (torch:tensor '((1.0 3.0))))) ; => #d((-1.0 1.0)) ``` --- # FILE: references/reference/functions/torch-linear.md # torch:linear `(torch:linear in-features out-features &key bias)` Returns a fully connected layer (PyTorch's `nn.Linear`): the field `:weight` is an `(in-features out-features)` parameter and `:bias` an `(out-features)` parameter, or `nil` under `:bias nil`. The forward is `x . W (+ b)`, so the bias broadcasts over every leading axis and an input of rank 3 is transformed batch-wise. Both parameters are drawn from PyTorch's default `U(-1/sqrt(in-features), 1/sqrt(in-features))` using the seeded [`linalg:seed`](linalg-seed.md) generator, so a seeded run reproduces on every backend. The weight is stored `(in out)` -- not PyTorch's transposed `(out in)` -- so the forward is a plain [`torch:matmul`](torch-matmul.md). ```lisp (defparameter *lin* (torch:linear 3 2)) (torch:set-field *lin* :weight (torch:parameter '((1.0 0.0) (0.0 1.0) (1.0 1.0)))) (torch:set-field *lin* :bias (torch:parameter '(0.5 -0.5))) (torch:data (torch:forward *lin* (torch:tensor '((1.0 2.0 3.0))))) ; => #d((4.5 4.5)) (torch:shape (torch:forward *lin* (torch:tensor (linalg:ones '(2 4 3))))) ; => (2 4 2) ``` --- # FILE: references/reference/functions/torch-log-softmax.md # torch:log-softmax `(torch:log-softmax a &key axis)` Differentiable log-softmax (`linalg:log-softmax`, the numerically stable half of a cross-entropy loss): computed as `(x - max) - log(sum(exp(x - max)))`, never as `log` of [`torch:softmax`](torch-softmax.md). The backward pass is `g - softmax(x) * sum(g)`. ```lisp (torch:data (torch:log-softmax (torch:tensor '(0.0 0.0)))) ; => #d(-0.6931471805599453 -0.6931471805599453) ``` --- # FILE: references/reference/functions/torch-log.md # torch:log `(torch:log a)` Differentiable elementwise natural logarithm (`linalg:log`); the gradient is `g / x`. ```lisp (torch:data (torch:log (torch:tensor '(1.0 2.718281828459045)))) ; => #d(0.0 1.0) ``` --- # FILE: references/reference/functions/torch-masked-fill.md # torch:masked-fill `(torch:masked-fill a mask value)` Differentiable masked fill (`torch.masked_fill` over `linalg:where`): the scalar `value` where `mask` is non-zero, `a`'s element where it is zero. `mask` (a 0/1 array, a comparison mask, or a tensor) and `value` are constants -- no gradient flows to them; `a`'s gradient is zero at the filled positions. Filling attention scores with `-infinity` before [`torch:softmax`](torch-softmax.md) is the masked-attention idiom. ```lisp (torch:data (torch:masked-fill (torch:tensor '((1.0 2.0) (3.0 4.0))) #2A((0 1) (0 0)) -1.0)) ; => #d((1.0 -1.0) (3.0 4.0)) ``` --- # FILE: references/reference/functions/torch-matmul.md # torch:matmul `(torch:matmul a b)` Differentiable matrix product with `torch.matmul`'s rank rules: two vectors give the dot product (a scalar tensor), a matrix and a vector the usual products, and rank >= 3 on either side the batched product (`linalg:matmul`: the last two axes are the matrix, leading axes broadcast). Gradients flow to both operands -- `g . b^T` and `a^T . g` in the matrix case -- with batch axes summed back like every broadcasting adjoint. ```lisp (torch:data (torch:matmul (torch:tensor '((1.0 2.0) (3.0 4.0))) (torch:tensor '((5.0 6.0) (7.0 8.0))))) ; => #d((19.0 22.0) (43.0 50.0)) (torch:item (torch:matmul (torch:tensor '(1.0 2.0)) (torch:tensor '(3.0 4.0)))) ; => 11.0 ``` --- # FILE: references/reference/functions/torch-mean.md # torch:mean `(torch:mean a &key axis keepdims)` Differentiable arithmetic mean (`linalg:mean`, same `:axis` / `:keepdims` rules as [`torch:sum`](torch-sum.md)); the backward pass is the sum adjoint divided by the reduced element count. The mean of a squared [`torch:sub`](torch-sub.md) is the MSE loss. ```lisp (torch:item (torch:mean (torch:tensor '(1.0 2.0 3.0)))) ; => 2.0 ``` --- # FILE: references/reference/functions/torch-module-kind.md # torch:module-kind `(torch:module-kind module)` Returns the module's kind keyword -- the one given to [`torch:module`](torch-module.md), `:linear` / `:embedding` / `:sequential` / `:layer-norm` / `:dropout` for the built-in layers. ```lisp (torch:module-kind (torch:linear 2 2)) ; => :LINEAR (torch:module-kind (torch:sequential)) ; => :SEQUENTIAL ``` --- # FILE: references/reference/functions/torch-module.md # torch:module `(torch:module kind fields forward-fn)` Returns a new module -- the parameter-owning, composable object of the `torch` package. `kind` is a keyword naming the layer, `fields` a plist of **keyword**/value pairs holding every parameter, buffer, submodule and hyper-parameter, and `forward-fn` is what [`torch:forward`](torch-forward.md) applies, as `(funcall forward-fn module args...)`. The fields plist is the module's parameter registration: [`torch:parameters`](torch-parameters.md) walks it, so a layer's forward must read its parameters back with [`torch:field`](torch-field.md) rather than from a closed-over variable. A field holding a tensor without `requires-grad` is a buffer and is skipped by the walk. The built-in layers ([`torch:linear`](torch-linear.md) and friends) are ordinary callers of this function. ```lisp (defparameter *scale* (torch:module :scale (list :gain (torch:parameter '(2.0 3.0))) (lambda (self x) (torch:mul x (torch:field self :gain))))) (torch:data (torch:forward *scale* (torch:tensor '(1.0 10.0)))) ; => #d(2.0 30.0) (length (torch:parameters *scale*)) ; => 1 ``` --- # FILE: references/reference/functions/torch-modulep.md # torch:modulep `(torch:modulep x)` Returns `T` when x is a `torch` module (the fixed-layout record [`torch:module`](torch-module.md) builds), `NIL` otherwise. A tensor is not a module. ```lisp (torch:modulep (torch:linear 2 2)) ; => T (torch:modulep (torch:tensor 1.0)) ; => NIL ``` --- # FILE: references/reference/functions/torch-mse-loss.md # torch:mse-loss `(torch:mse-loss input target &key reduction)` Returns the mean squared error between input and target as a scalar tensor (PyTorch's `nn.MSELoss`). `:reduction :sum` adds instead of averaging and `:reduction :none` returns the per-element tensor. The target is a constant unless it is itself a tensor requiring gradients; either argument may be a number, a list or an array. ```lisp (torch:item (torch:mse-loss (torch:tensor '(1.0 2.0)) '(0.0 0.0))) ; => 2.5 (torch:item (torch:mse-loss (torch:tensor '(1.0 2.0)) '(0.0 0.0) :reduction :sum)) ; => 5.0 ``` --- # FILE: references/reference/functions/torch-mul.md # torch:mul `(torch:mul a b)` Differentiable elementwise (Hadamard) `a * b` with numpy-style broadcasting (`linalg:mul`); the matrix product is [`torch:matmul`](torch-matmul.md). Each operand's gradient is the incoming gradient times the other operand. ```lisp (torch:data (torch:mul (torch:tensor '(1.0 2.0 3.0)) (torch:tensor '(4.0 5.0 6.0)))) ; => #d(4.0 10.0 18.0) (torch:data (torch:mul (torch:tensor '(1.0 2.0)) 2)) ; => #d(2.0 4.0) ``` --- # FILE: references/reference/functions/torch-multinomial.md # torch:multinomial `(torch:multinomial probs &key num-samples replacement)` `num-samples` indices drawn from each row of `probs`, whose LAST axis holds the weights (PyTorch's `torch.multinomial`). The weights need not sum to `1` -- each row is normalized -- and must be non-negative. A rank-1 input answers a `(num-samples)` index array, a rank-n input its own shape with the last axis replaced by `num-samples`. Non-differentiable, and a RAW linalg array rather than a tensor. Without `:replacement t` an index already drawn cannot be drawn again within the same row, which is PyTorch's default; `num-samples` must then not exceed the number of weights. The draw comes from the SEEDED [`linalg:seed`](linalg-seed.md) generator, so a sampling run reproduces on every backend. ```lisp (linalg:seed 3) (torch:multinomial (linalg:from-list '((0.0 1.0 0.0) (0.0 0.0 1.0)))) ; => #d((1.0) (2.0)) ``` --- # FILE: references/reference/functions/torch-neg.md # torch:neg `(torch:neg a)` Differentiable elementwise negation (`linalg:negative`); the gradient is negated on the way back. ```lisp (torch:data (torch:neg (torch:tensor '(1.0 -2.0)))) ; => #d(-1.0 2.0) ``` --- # FILE: references/reference/functions/torch-optimizer-kind.md # torch:optimizer-kind `(torch:optimizer-kind optimizer)` Returns the optimizer's kind keyword, as given to [`torch:optimizer`](torch-optimizer.md) -- `:sgd` for [`torch:sgd`](torch-sgd.md), `:adam` for [`torch:adam`](torch-adam.md). Signals unless the argument is an optimizer. ```lisp (torch:optimizer-kind (torch:sgd nil)) ; => :SGD (torch:optimizer-kind (torch:adam nil)) ; => :ADAM ``` --- # FILE: references/reference/functions/torch-optimizer-params.md # torch:optimizer-params `(torch:optimizer-params optimizer)` Returns the list of parameter tensors the optimizer updates -- what its step function walks. When the optimizer was built over a module, this is the [`torch:parameters`](torch-parameters.md) walk taken once, at construction. ```lisp (defparameter *net* (torch:linear 2 3)) (defparameter *opt* (torch:sgd *net* :lr 0.1)) (length (torch:optimizer-params *opt*)) ; => 2 (torch:shape (car (torch:optimizer-params *opt*))) ; => (2 3) ``` --- # FILE: references/reference/functions/torch-optimizer.md # torch:optimizer `(torch:optimizer kind params fields step-fn)` Returns a fresh optimizer: `kind` is a keyword naming the rule, `params` a module (whose [`torch:parameters`](torch-parameters.md) are walked) or a plain list of parameter tensors, `fields` a plist of `KEYWORD`/value hyper-parameters and state buffers, and `step-fn` a function called as `(funcall step-fn optimizer)` by [`torch:step`](torch-step.md). The step counter starts at `0`. This is how a user-written rule is spelled -- [`torch:sgd`](torch-sgd.md) and [`torch:adam`](torch-adam.md) are ordinary callers of it. The step function reads its hyper-parameters back with [`torch:field`](torch-field.md) and its parameters with [`torch:optimizer-params`](torch-optimizer-params.md), so everything the rule needs lives in the record rather than in a closure. ```lisp (defun scaled-step (self) (dolist (p (torch:optimizer-params self)) (unless (null (torch:grad p)) (torch:set-data p (linalg:sub (torch:data p) (linalg:mul (torch:field self :lr) (torch:grad p))))))) (defparameter *p* (torch:parameter '(1.0 2.0))) (defparameter *opt* (torch:optimizer :my-sgd (list *p*) (list :lr 0.5) (function scaled-step))) (torch:backward (torch:sum (torch:mul *p* *p*))) (torch:step *opt*) (torch:data *p*) ; => #d(0.0 0.0) (torch:optimizer-kind *opt*) ; => :MY-SGD ``` --- # FILE: references/reference/functions/torch-optimizerp.md # torch:optimizerp `(torch:optimizerp x)` Returns `T` when `x` is a torch optimizer -- the fixed-layout record [`torch:optimizer`](torch-optimizer.md) builds -- and `NIL` for anything else, including a tensor or a module. ```lisp (torch:optimizerp (torch:sgd nil)) ; => T (torch:optimizerp (torch:tensor '(1.0))) ; => NIL (torch:optimizerp 42) ; => NIL ``` --- # FILE: references/reference/functions/torch-pad-sequence.md # torch:pad-sequence `(torch:pad-sequence sequences &key padding-value)` Returns a list of variable-length sequences (lists, index vectors or tensors) as one padded rank-2 tensor, **batch first**: `(batch longest)`, every row filled up to the longest one with `padding-value` (`0` by default). This is `torch.nn.utils.rnn.pad_sequence` with `batch_first=True`. The result is a constant tensor of token indices, ready for [`torch:embedding`](torch-embedding.md) and for [`torch:padding-mask`](torch-padding-mask.md). ```lisp (torch:data (torch:pad-sequence '((1 2 3) (4 5) (6)))) ; => #d((1.0 2.0 3.0) (4.0 5.0 0.0) (6.0 0.0 0.0)) (torch:shape (torch:pad-sequence '((1 2) (3)) :padding-value 9)) ; => (2 2) ``` --- # FILE: references/reference/functions/torch-padding-mask.md # torch:padding-mask `(torch:padding-mask tokens &key pad-id)` Returns the padding mask of a `(batch length)` token matrix: `1.0` at every position holding `pad-id` (`0` by default) and `0.0` elsewhere, with a query axis inserted -- `(batch 1 length)` -- so it broadcasts over an attention score's `(batch query-length key-length)`. The result is a **raw linalg array**, not a tensor: a mask is a constant, and [`torch:masked-fill`](torch-masked-fill.md) takes it as one. Combine it with [`torch:subsequent-mask`](torch-subsequent-mask.md) using `linalg:add` or `linalg:maximum` -- every non-zero counts as masked. ```lisp (defparameter *tokens* (torch:pad-sequence '((1 2 3) (4 5)))) (torch:padding-mask *tokens*) ; => #d(((0.0 0.0 0.0)) ((0.0 0.0 1.0))) (linalg:shape (torch:padding-mask *tokens*)) ; => (2 1 3) ``` --- # FILE: references/reference/functions/torch-parameter.md # torch:parameter `(torch:parameter x &key element-type)` Returns a leaf tensor with `:requires-grad t` -- the spelling that marks a value as a **trainable parameter** of a module. Identical to `(torch:tensor x :requires-grad t)`; the separate name is what makes a module's fields readable at a glance. A field holding a tensor *without* `requires-grad` is a buffer instead, and [`torch:parameters`](torch-parameters.md) skips it. ```lisp (torch:requires-grad-p (torch:parameter '(1.0 2.0))) ; => T (torch:data (torch:parameter '(1.0 2.0))) ; => #d(1.0 2.0) ``` --- # FILE: references/reference/functions/torch-parameters.md # torch:parameters `(torch:parameters module)` Returns every parameter reachable from the module, in registration order and deduplicated by identity: its own parameter fields, then those of its submodules and of any **list** of submodules it holds, recursively. A weight shared by two layers appears once. This is the list a training loop (and an optimizer) is built over -- reaching a parameter needs no declaration beyond putting it in the fields plist. ```lisp (defparameter *net* (torch:sequential (torch:linear 4 8) (function torch:relu) (torch:linear 8 2))) (length (torch:parameters *net*)) ; => 4 (torch:shape (car (torch:parameters *net*))) ; => (4 8) ``` --- # FILE: references/reference/functions/torch-power.md # torch:power `(torch:power a b)` Differentiable elementwise `a ** b` (`linalg:power`); either operand may be a scalar and both are differentiable. The base's gradient is `g * b * a^(b-1)`; the exponent's -- computed only when the exponent tracks gradients -- is `g * a^b * ln a`, which is only meaningful for a positive base. ```lisp (torch:data (torch:power (torch:tensor '(2.0 3.0)) 2)) ; => #d(4.0 9.0) ``` --- # FILE: references/reference/functions/torch-relu.md # torch:relu `(torch:relu a)` Differentiable elementwise `max(x, 0.0)` (`linalg:relu`); the gradient passes where `x > 0` and is `0` elsewhere (`0` at exactly `x = 0`, like PyTorch). ```lisp (torch:data (torch:relu (torch:tensor '(-1.0 0.0 2.0)))) ; => #d(0.0 0.0 2.0) ``` --- # FILE: references/reference/functions/torch-requires-grad-p.md # torch:requires-grad-p `(torch:requires-grad-p tensor)` Returns whether the tensor participates in autograd: a leaf created with `:requires-grad t`, or any result recorded on the tape (computed -- outside [`torch:no-grad`](../macros/torch-no-grad.md) -- from something that participates). ```lisp (defparameter *w* (torch:tensor '(1.0) :requires-grad t)) (torch:requires-grad-p *w*) ; => T (torch:requires-grad-p (torch:mul *w* 2.0)) ; => T (torch:requires-grad-p (torch:tensor '(1.0))) ; => NIL ``` --- # FILE: references/reference/functions/torch-reshape.md # torch:reshape `(torch:reshape a shape)` Differentiable reshape (row-major, `linalg:reshape`'s rules: sizes must agree, one extent may be `-1` and is inferred); the backward pass reshapes the gradient back to the input's shape. ```lisp (torch:data (torch:reshape (torch:tensor '(1.0 2.0 3.0 4.0)) '(2 2))) ; => #d((1.0 2.0) (3.0 4.0)) ``` --- # FILE: references/reference/functions/torch-sequential.md # torch:sequential `(torch:sequential &rest layers)` Returns a chain of layers (PyTorch's `nn.Sequential`): the forward threads its argument through each element in order. An element may be a module **or a plain function**, so an activation goes in as `(function torch:relu)` -- there is no separate activation-module type. The elements live in the single field `:layers`, and [`torch:parameters`](torch-parameters.md) walks that list, so every nested parameter is reachable. A list of modules is itself a valid field value everywhere in this package, so a stack of N identical blocks needs no `ModuleList` type: hold the list in a field of your own [`torch:module`](torch-module.md) and the walk finds it. ```lisp (defparameter *net* (torch:sequential (torch:linear 4 8) (function torch:relu) (torch:linear 8 2))) (torch:shape (torch:forward *net* (torch:tensor (linalg:zeros '(3 4))))) ; => (3 2) (length (torch:parameters *net*)) ; => 4 ``` --- # FILE: references/reference/functions/torch-set-data.md # torch:set-data `(torch:set-data tensor value)` Replaces the tensor's data **in place** with value (a linalg array or a number) and returns the tensor. This is the parameter update of a training loop: it writes into the very tensor a module's fields already point at, so the layer keeps using it. The tape is untouched, so call it inside [`torch:no-grad`](../macros/torch-no-grad.md), like `torch.no_grad()` around an optimizer step. ```lisp (defparameter *p* (torch:parameter '(1.0 2.0))) (torch:no-grad (torch:set-data *p* (linalg:mul 2.0 (torch:data *p*)))) (torch:data *p*) ; => #d(2.0 4.0) (torch:requires-grad-p *p*) ; => T ``` --- # FILE: references/reference/functions/torch-set-field.md # torch:set-field `(torch:set-field module name value)` Sets the module's named field (adding it when it is new) and returns the module. Replacing a parameter this way re-binds a layer to a given set of weights -- which is what makes a layer's output reproducible in a test or an example. ```lisp (defparameter *lin* (torch:linear 3 2)) (torch:set-field *lin* :weight (torch:parameter '((1.0 0.0) (0.0 1.0) (1.0 1.0)))) (torch:set-field *lin* :bias (torch:parameter '(0.5 -0.5))) (torch:data (torch:forward *lin* (torch:tensor '((1.0 2.0 3.0))))) ; => #d((4.5 4.5)) ``` --- # FILE: references/reference/functions/torch-sgd.md # torch:sgd `(torch:sgd params &key lr momentum weight-decay)` Returns a stochastic gradient descent optimizer (PyTorch's `torch.optim.SGD`) over `params`, a module or a list of parameter tensors. `:lr` defaults to `0.01`, `:momentum` and `:weight-decay` to `0`. Per element, [`torch:step`](torch-step.md) computes ```text g <- grad + weight-decay * param buf <- momentum * buf + g ; only when momentum is non-zero param <- param - lr * (momentum non-zero ? buf : g) ``` The momentum buffer starts at zero, which is PyTorch's clone-on-first-step. The hyper-parameters are ordinary fields, so a learning-rate schedule is `(torch:set-field opt :lr new)`. ```lisp (defparameter *p* (torch:parameter '(1.0 2.0))) (defparameter *opt* (torch:sgd (list *p*) :lr 0.125)) (torch:backward (torch:sum (torch:mul *p* *p*))) (torch:step *opt*) (torch:data *p*) ; => #d(0.75 1.5) (torch:field *opt* :lr) ; => 0.125 ``` --- # FILE: references/reference/functions/torch-shape.md # torch:shape `(torch:shape tensor)` Returns the dims list of the tensor's data (the linalg `shape`), or `nil` for a scalar tensor (rank 0). ```lisp (torch:shape (torch:tensor '((1 2 3) (4 5 6)))) ; => (2 3) (torch:shape (torch:tensor 2.5)) ; => NIL ``` --- # FILE: references/reference/functions/torch-shuffled-batches.md # torch:shuffled-batches `(torch:shuffled-batches data batch-size &key shuffle drop-last)` Cuts `data` into mini-batches and returns them as a list of lists, each of `batch-size` elements except possibly the last -- dropped under `:drop-last t`, like a `DataLoader`'s `drop_last`. `data` is a **list** of examples, or a non-negative **integer** `n` standing for the index list `0..n-1`, which is the spelling that batches several parallel arrays at once (the caller selects the same rows out of each). The order comes from the seeded [`linalg:seed`](linalg-seed.md) generator, so an epoch reproduces on every backend; `:shuffle nil` keeps `data`'s own order, so an evaluation pass uses the same function. ```lisp (linalg:seed 1) (torch:shuffled-batches 7 3) ; => ((6 0 5) (1 4 3) (2)) (torch:shuffled-batches '(a b c d e) 2 :shuffle nil) ; => ((A B) (C D) (E)) (torch:shuffled-batches '(a b c d e) 2 :shuffle nil :drop-last t) ; => ((A B) (C D)) ``` --- # FILE: references/reference/functions/torch-slice.md # torch:slice `(torch:slice a specs)` Differentiable numpy basic slicing (`linalg:slice`: one spec per axis -- `nil` leaves the axis whole, `(start end)` / `(start end step)` selects along it, negative indexing and steps included). The backward pass scatters the gradient back into zeros at the positions the slice read from. ```lisp (torch:data (torch:slice (torch:tensor '((0.0 1.0 2.0) (3.0 4.0 5.0))) '(nil (0 2)))) ; => #d((0.0 1.0) (3.0 4.0)) ``` --- # FILE: references/reference/functions/torch-softmax.md # torch:softmax `(torch:softmax a &key axis)` Differentiable max-subtracted softmax (`linalg:softmax`): with no `:axis` the whole tensor is one distribution, with an integer `:axis` one distribution per slice -- torch's `softmax(x, dim)`, the attention-weight form. The backward pass is `s * (g - sum(g * s))` over each distribution. Masked positions filled with `-infinity` by [`torch:masked-fill`](torch-masked-fill.md) come out as exactly `0.0`. ```lisp (torch:data (torch:softmax (torch:tensor '(1.0 1.0 1.0 1.0)))) ; => #d(0.25 0.25 0.25 0.25) (torch:data (torch:softmax (torch:tensor '((0.0 0.0) (1.0 1.0))) :axis 1)) ; => #d((0.5 0.5) (0.5 0.5)) ``` --- # FILE: references/reference/functions/torch-sqrt.md # torch:sqrt `(torch:sqrt a)` Differentiable elementwise square root (`linalg:sqrt`); the gradient is `g / (2 sqrt x)`. ```lisp (torch:data (torch:sqrt (torch:tensor '(4.0 9.0)))) ; => #d(2.0 3.0) ``` --- # FILE: references/reference/functions/torch-squeeze.md # torch:squeeze `(torch:squeeze a &key axis)` Differentiable extent-1 axis removal (`linalg:squeeze`): all of them with no `:axis`, else only the named axis (or list of axes). Squeezing every axis away yields the scalar tensor. ```lisp (torch:shape (torch:squeeze (torch:tensor '((1.0 2.0 3.0))))) ; => (3) (torch:data (torch:squeeze (torch:tensor '((7.0))))) ; => 7.0 ``` --- # FILE: references/reference/functions/torch-stack.md # torch:stack `(torch:stack tensors &key axis)` Differentiable join of the list `tensors` along a new axis (`linalg:stack`): equal shapes, result rank + 1, the new axis at `:axis` (negative counts from the end of the result). The backward pass slices the gradient at each input's index and drops the axis again. ```lisp (torch:data (torch:stack (list (torch:tensor '(1.0 2.0)) (torch:tensor '(3.0 4.0))))) ; => #d((1.0 2.0) (3.0 4.0)) ``` --- # FILE: references/reference/functions/torch-std.md # torch:std `(torch:std a &key axis keepdims ddof)` Differentiable standard deviation: [`torch:sqrt`](torch-sqrt.md) of [`torch:var`](torch-var.md). The mean and std along one axis are the two statistics LayerNorm needs. ```lisp (torch:item (torch:std (torch:tensor '(2.0 4.0 4.0 4.0 5.0 5.0 7.0 9.0)))) ; => 2.0 ``` --- # FILE: references/reference/functions/torch-step-count.md # torch:step-count `(torch:step-count optimizer)` Returns how many times [`torch:step`](torch-step.md) has run on this optimizer: `0` before the first step, and the `t` of Adam's bias correction during the step itself. The counter belongs to the optimizer, not to any parameter, so two optimizers over the same parameters keep separate schedules. ```lisp (defparameter *opt* (torch:sgd (list (torch:parameter '(1.0))) :lr 0.1)) (torch:step-count *opt*) ; => 0 (torch:step *opt*) (torch:step *opt*) (torch:step-count *opt*) ; => 2 ``` --- # FILE: references/reference/functions/torch-step.md # torch:step `(torch:step optimizer)` Applies the optimizer's rule to every parameter and returns the optimizer (PyTorch's `optimizer.step()`). The step counter is incremented **first**, so a bias correction reading [`torch:step-count`](torch-step-count.md) sees `1` during the first step. The update writes each parameter's data in place with no torch operation, so it records nothing on the tape: unlike a hand-written update built from [`torch:set-data`](torch-set-data.md), it needs no [`torch:no-grad`](../macros/torch-no-grad.md) around it. A parameter whose gradient is still `NIL` is skipped. ```lisp (defparameter *p* (torch:parameter '(4.0))) (defparameter *opt* (torch:sgd (list *p*) :lr 0.5)) (torch:backward (torch:sum (torch:mul *p* *p*))) (torch:step *opt*) (torch:data *p*) ; => #d(0.0) (torch:step-count *opt*) ; => 1 ``` --- # FILE: references/reference/functions/torch-sub.md # torch:sub `(torch:sub a b)` Differentiable elementwise `a - b` with numpy-style broadcasting (`linalg:sub`); the second operand's gradient is negated (and unbroadcast, like [`torch:add`](torch-add.md)). ```lisp (torch:data (torch:sub (torch:tensor '(5.0 7.0)) (torch:tensor '(1.0 2.0)))) ; => #d(4.0 5.0) ``` --- # FILE: references/reference/functions/torch-subsequent-mask.md # torch:subsequent-mask `(torch:subsequent-mask sequence-length)` Returns the causal (look-ahead) mask of a sequence: `1.0` strictly **above** the diagonal, shaped `(1 sequence-length sequence-length)` so it broadcasts over the batch. Position `i` may not attend to any `j > i`. Like [`torch:padding-mask`](torch-padding-mask.md) it is a raw linalg array. Filling the masked scores with `-infinity` before [`torch:softmax`](torch-softmax.md) is the masked-attention idiom, and the masked weight comes out as exactly `0.0`. ```lisp (torch:subsequent-mask 3) ; => #d(((0.0 1.0 1.0) (0.0 0.0 1.0) (0.0 0.0 0.0))) (defparameter *scores* (torch:tensor (linalg:ones '(1 2 2)))) (torch:data (torch:softmax (torch:masked-fill *scores* (torch:subsequent-mask 2) (/ -1.0 0.0)) :axis -1)) ; => #d(((1.0 0.0) (0.5 0.5))) ``` --- # FILE: references/reference/functions/torch-sum.md # torch:sum `(torch:sum a &key axis keepdims)` Differentiable sum, of every element (no `:axis`) or along an axis, following `linalg:sum`'s `:axis` / `:keepdims` rules. The backward pass broadcasts the gradient back over the reduced extent. ```lisp (torch:item (torch:sum (torch:tensor '(1.0 2.0 3.0)))) ; => 6.0 (torch:data (torch:sum (torch:tensor '((1.0 2.0) (3.0 4.0))) :axis 0)) ; => #d(4.0 6.0) ``` --- # FILE: references/reference/functions/torch-tanh.md # torch:tanh `(torch:tanh a)` Differentiable elementwise hyperbolic tangent (`linalg:tanh`) -- the classic activation; the gradient is `g * (1 - tanh^2 x)`, computed from the forward result. ```lisp (torch:data (torch:tanh (torch:tensor '(0.0)))) ; => #d(0.0) ``` --- # FILE: references/reference/functions/torch-tensor.md # torch:tensor `(torch:tensor x &key requires-grad element-type)` Returns a fresh leaf tensor -- the differentiable value of the `torch` package -- from a number (a rank-0 scalar tensor), a list (flat, or a list of equal-length rows), an array, a linalg array, or another tensor (whose data is copied). `:requires-grad t` marks it as a parameter whose gradient [`torch:backward`](torch-backward.md) should fill in; `:element-type 'single-float` builds packed single-float (`#f`) data. A tensor prints as `#`, with ` :REQUIRES-GRAD T` appended for a parameter -- the same text on every backend, since only the data is shown and never the backward closure it may carry. Read the values themselves back with [`torch:data`](torch-data.md), [`torch:item`](torch-item.md) and [`torch:grad`](torch-grad.md). ```lisp (print (torch:tensor '(1 2 3))) (print (torch:tensor '(1.0) :requires-grad t)) ``` ``` # # ``` ```lisp (torch:data (torch:tensor '(1 2 3))) ; => #d(1.0 2.0 3.0) (torch:data (torch:tensor 2.5)) ; => 2.5 (torch:data (torch:tensor #(1 2) :element-type 'single-float)) ; => #f(1.0 2.0) (torch:requires-grad-p (torch:tensor '(1.0) :requires-grad t)) ; => T ``` --- # FILE: references/reference/functions/torch-tensorp.md # torch:tensorp `(torch:tensorp x)` Returns whether `x` is a torch tensor (the value [`torch:tensor`](torch-tensor.md) and every torch operation build); anything else -- a linalg array included -- answers `nil`. ```lisp (torch:tensorp (torch:tensor '(1 2))) ; => T (torch:tensorp #(1 2)) ; => NIL ``` --- # FILE: references/reference/functions/torch-topk.md # torch:topk `(torch:topk a k &key axis indices)` The `k` largest elements along `axis` (`-1`, the last axis, by default), ORDERED LARGEST FIRST, as a RAW linalg array shaped like `a` with that axis narrowed to `k`. Non-differentiable, like [`torch:argmax`](torch-argmax.md). PyTorch's `torch.topk` returns the values and their indices as a pair; this returns ONE of them -- the values, or under `:indices t` the positions they came from -- because every function in this package is single-valued. Ties keep the LOWEST index, so a run is reproducible on every backend, where `torch.topk`'s tie order is not specified at all. The top-`k` step of a sampling loop is this plus [`torch:masked-fill`](torch-masked-fill.md): everything below the row's `k`-th largest logit becomes `-infinity`, so the softmax gives it weight exactly `0`. ```lisp (torch:topk (linalg:from-list '((1.0 5.0 3.0) (9.0 2.0 8.0))) 2) ; => #d((5.0 3.0) (9.0 8.0)) (torch:topk (linalg:from-list '((1.0 5.0 3.0) (9.0 2.0 8.0))) 2 :indices t) ; => #d((1.0 2.0) (0.0 2.0)) ``` --- # FILE: references/reference/functions/torch-train.md # torch:train `(torch:train module &optional mode)` Puts the module and every submodule into **training** mode (PyTorch's `nn.Module.train`) and returns the module; an explicit `nil` mode is the same as [`torch:eval`](torch-eval.md). Only [`torch:dropout`](torch-dropout.md) reads the flag today. ```lisp (defparameter *net* (torch:sequential (torch:dropout 0.5))) (torch:eval *net*) (torch:training-p (torch:train *net*)) ; => T (torch:training-p (car (torch:field *net* :layers))) ; => T ``` --- # FILE: references/reference/functions/torch-training-p.md # torch:training-p `(torch:training-p module)` Returns `T` when the module is in training mode, `NIL` in evaluation mode. A module starts in training mode; [`torch:train`](torch-train.md) and [`torch:eval`](torch-eval.md) switch it, recursively through submodules. ```lisp (torch:training-p (torch:dropout 0.5)) ; => T (torch:training-p (torch:eval (torch:dropout 0.5))) ; => NIL ``` --- # FILE: references/reference/functions/torch-transpose.md # torch:transpose `(torch:transpose a &optional axes)` Differentiable transpose: with no `axes` the matrix transpose (a vector passes through, like `linalg:transpose`); with an axes list the rank-n permutation (`out-dims[k] = dims[axes[k]]`, a negative axis counting from the end). The backward pass applies the inverse permutation to the gradient. ```lisp (torch:data (torch:transpose (torch:tensor '((1.0 2.0) (3.0 4.0))))) ; => #d((1.0 3.0) (2.0 4.0)) ``` --- # FILE: references/reference/functions/torch-unsqueeze.md # torch:unsqueeze `(torch:unsqueeze a axis)` Differentiable extent-1 axis insertion (`linalg:expand-dims`, torch's `unsqueeze`): a negative axis counts from the end of the result, so `-1` appends. Row-major order is unchanged, so the backward pass is a reshape. ```lisp (torch:shape (torch:unsqueeze (torch:tensor '(1.0 2.0 3.0)) 0)) ; => (1 3) (torch:shape (torch:unsqueeze (torch:tensor '(1.0 2.0 3.0)) -1)) ; => (3 1) ``` --- # FILE: references/reference/functions/torch-var.md # torch:var `(torch:var a &key axis keepdims ddof)` Differentiable variance with the `(n - ddof)` divisor (`linalg:var`'s rules: the default `:ddof 0` is torch's `unbiased=False`, `:ddof 1` the sample variance). It is composed from [`torch:mean`](torch-mean.md), [`torch:sub`](torch-sub.md), [`torch:mul`](torch-mul.md) and [`torch:sum`](torch-sum.md), so its backward pass comes from the tape. ```lisp (torch:item (torch:var (torch:tensor '(1.0 2.0 3.0 4.0)))) ; => 1.25 (torch:item (torch:var (torch:tensor '(1.0 2.0 3.0 4.0)) :ddof 1)) ; => 1.6666666666666667 ``` --- # FILE: references/reference/functions/torch-view.md # torch:view `(torch:view a shape)` PyTorch's other reshape spelling. rontolisp arrays are always contiguous and every linalg result is a fresh copy, so `view` is exactly [`torch:reshape`](torch-reshape.md) here -- it does not alias storage. ```lisp (torch:shape (torch:view (torch:tensor '(1.0 2.0 3.0 4.0 5.0 6.0)) '(2 3))) ; => (2 3) ``` --- # FILE: references/reference/functions/torch-zero-grad.md # torch:zero-grad `(torch:zero-grad tensor-or-module)` Clears accumulated gradients (back to `nil`) and returns the argument: for a tensor its own gradient, for a **module** the gradient of every parameter [`torch:parameters`](torch-parameters.md) reaches. Because [`torch:backward`](torch-backward.md) accumulates (`+=`), a training loop calls this on its model between steps. ```lisp (defparameter *w* (torch:tensor '(1.0 2.0) :requires-grad t)) (torch:backward (torch:sum (torch:mul *w* *w*))) (torch:grad *w*) ; => #d(2.0 4.0) (torch:grad (torch:zero-grad *w*)) ; => NIL (defparameter *lin* (torch:linear 2 2)) (torch:backward (torch:sum (torch:forward *lin* (torch:tensor '((1.0 2.0)))))) (torch:zero-grad *lin*) (torch:grad (torch:field *lin* :bias)) ; => NIL ``` --- # FILE: references/reference/functions/translate-logical-pathname.md # translate-logical-pathname `(translate-logical-pathname pathname &key)` The pathname itself. Every rontolisp pathname is PHYSICAL -- there are no logical hosts and no `logical-pathname-translations` table to consult -- so the translation is the identity, which is what Common Lisp prescribes for a physical pathname argument. Portable code that normalizes a path through this before opening it therefore works unchanged. ```lisp (namestring (translate-logical-pathname "d/a.txt")) ; => "d/a.txt" ``` [`logical-pathname`](logical-pathname.md) is the other half of that decision: it always signals, because nothing here can be a logical pathname. ## Backend support All four backends -- one definition in rontolisp source over primitives every backend has. --- # FILE: references/reference/functions/translate-pathname.md # translate-pathname `(translate-pathname source from-wildcard to-wildcard &key)` Matches `source` against `from-wildcard`, then substitutes the pieces its wildcards captured into `to-wildcard`, left to right. The wildcards are the ones the rest of the pathname family understands: `*` (any run of characters), `?` (one character) and `**/` (zero or more whole directory levels). A `source` that does not match `from-wildcard` signals, as it does in Common Lisp. ```lisp (list (namestring (translate-pathname "src/foo.lisp" "src/*.lisp" "build/*.fasl")) (namestring (translate-pathname "a/b.c" "*/*.*" "x/*-y.*"))) ; => ("build/foo.fasl" "x/a-y.b") ``` A `**/` is ONE wildcard, separator included: it captures the whole run of directory levels it consumed, and a `**/` in `to-wildcard` writes that run back verbatim. Because it matches zero levels as well as many, a source with no intervening directory still translates: ```lisp (list (namestring (translate-pathname "/a/b/d/c.lisp" "/a/**/*.lisp" "/x/**/*.fasl")) (namestring (translate-pathname "/a/c.lisp" "/a/**/*.lisp" "/x/**/*.fasl"))) ; => ("/x/b/d/c.fasl" "/x/c.fasl") ``` Lite: matching runs over the FLAT namestring and captures are substituted POSITIONALLY rather than component by component, so a plain `*` may span a `/` where a structured implementation would stop at a directory boundary, and a `to-wildcard` holding fewer wildcards than `from-wildcard` consumes the first of them rather than the matching component. ## Backend support All four backends -- one definition in rontolisp source over primitives every backend has. --- # FILE: references/reference/functions/tree-equal.md # tree-equal `(tree-equal tree-1 tree-2 &key test test-not)` Returns `t` when the two cons trees have the same shape and every pair of corresponding leaves matches. A cons only ever matches a cons, so a tree and a leaf in the same position differ. Leaves compare with `:test` (default `eql`) or, with `:test-not`, match exactly where that function answers false. Note that two strings with the same characters are already `eql` here, unlike in most Common Lisps. ```lisp (tree-equal '(1 (2 3)) '(1 (2 3))) ; => T ``` ```lisp (tree-equal '(1 (2)) '(1 2)) ; => NIL ``` ```lisp (tree-equal '("a" ("b")) '("A" ("B")) :test #'string-equal) ; => T ``` --- # FILE: references/reference/functions/truename.md # truename `(truename pathname)` Returns the pathname when the file exists and signals an error when it does not. The signal is the point: `(ignore-errors (truename path))` is the Common Lisp idiom for "this path if it is there, `nil` otherwise", and libraries use it to probe for an optional file or directory. rontolisp resolves no symbolic links and makes nothing absolute, so the value on success is a pathname carrying the argument namestring. When you want the answer without the condition, use [`probe-file`](probe-file.md), which asks the same question and returns `nil` instead of signalling. ```lisp (ignore-errors (truename "definitely-missing.txt")) ; => NIL ``` ## Backend support Works on all four backends: one definition in rontolisp source over `probe-file`, spliced into the program when it is referenced. --- # FILE: references/reference/functions/truncate.md # truncate `(truncate number &optional divisor)` Rounds `number` (or `number/divisor` when a divisor is given) toward zero to an integer, discarding any fractional part. In an ordinary (single-value) context the result is the quotient only; the remainder is the second value, observable through [`multiple-value-bind`](../macros/multiple-value-bind.md) and the other multiple-value consumers. ```lisp (truncate 3.7) ; => 3 ``` ```lisp (multiple-value-bind (q r) (truncate -7 2) (list q r)) ; => (-3 -1) ``` --- # FILE: references/reference/functions/type-error-datum.md # type-error-datum `(type-error-datum condition)` The `datum` slot of a `type-error` condition -- the object whose type was wrong. Its companion is [`type-error-expected-type`](type-error-expected-type.md). ```lisp (handler-case (error 'type-error :datum 3 :expected-type 'string) (type-error (e) (type-error-datum e))) ; => 3 ``` --- # FILE: references/reference/functions/type-error-expected-type.md # type-error-expected-type `(type-error-expected-type condition)` The `expected-type` slot of a `type-error` condition -- the type specifier the datum failed. See [`type-error-datum`](type-error-datum.md). ```lisp (handler-case (error 'type-error :datum 3 :expected-type 'string) (type-error (e) (type-error-expected-type e))) ; => STRING ``` --- # FILE: references/reference/functions/type-of.md # type-of `(type-of object)` The type name of a value as a symbol: a `defstruct`/CLOS instance answers its structure/class NAME, any other value answers a built-in type-name symbol (`integer`, `string`, `cons`, ...), falling back to `t`. It is the name-only view of what [`class-of`](class-of.md) answers as a class metaobject: `(type-of x)` and `(class-name (class-of x))` agree. A class defined in another package answers its package-qualified name — one colon when the package exports it, two when it does not — whatever package the caller is in. ```lisp (type-of 42) ; => INTEGER ``` ```lisp (defpackage :gfx (:use :cl) (:export :sprite)) (in-package :gfx) (defclass sprite () ()) (defclass hidden () ()) (defpackage :game (:use :cl)) (in-package :game) (list (type-of (make-instance 'gfx:sprite)) (type-of (make-instance 'gfx::hidden))) ; => (GFX:SPRITE GFX::HIDDEN) ``` --- # FILE: references/reference/functions/uiop-absolute-pathname-p.md # uiop:absolute-pathname-p `(uiop:absolute-pathname-p pathspec)` The parsed PATHNAME when `pathspec` is absolute (a generalized boolean, as upstream), `nil` otherwise. A rontolisp namestring is the host spelling, so "absolute" is the leading `/` -- there is no device or host component to weigh. ```lisp (uiop:absolute-pathname-p "/tmp/x") ; => #P"/tmp/x" ``` ```lisp (uiop:absolute-pathname-p "tmp/x") ; => NIL ``` ## Backend support Works on all four backends (Lisp source, `uiop-pathname.lisp`). --- # FILE: references/reference/functions/uiop-add-package-local-nickname.md # uiop:add-package-local-nickname `(uiop:add-package-local-nickname nickname package &optional scope-package)` Registers `nickname` as a shorthand for `package`, so `nickname:symbol` resolves like `package:symbol` afterwards -- the idiom libraries recommend for shortening long package names (e.g. jzon's `(uiop:add-package-local-nickname '#:jzon '#:com.inuoe.jzon)`). Returns the target package's name symbol. The [`defpackage`](../special-forms/defpackage.md) `:local-nicknames` clause performs the same registration at package-definition time. Lite: the nickname is **global** -- rontolisp has no per-package nickname scoping, so the optional third argument (the package to scope the nickname to) is accepted and ignored, and the nickname follows the same collision rules as `defpackage` `:nicknames`. On the JVM/WASM compile path the call must be a literal top-level form (literal designator arguments); it is consumed at compile time like a `defpackage`. A runtime-computed call works on the interpreter only. ```lisp (defpackage #:com.example.deeply.nested (:use #:cl) (:export #:answer)) (in-package #:com.example.deeply.nested) (defun answer () 42) (in-package #:cl-user) (uiop:add-package-local-nickname '#:nick '#:com.example.deeply.nested) (nick:answer) ; => 42 ``` --- # FILE: references/reference/functions/uiop-collect-sub-directories.md # uiop:collect-sub*directories `(uiop:collect-sub*directories directory collectp recursep collector)` Walks a directory tree. Each directory reached is passed to `collectp`; when that answers true it is passed to `collector`. Each of its subdirectories is passed to `recursep`; when that answers true the walk descends into it. Every directory handed to the three functions is a pathname in directory form (trailing `/`), including the root, so the shape is the same at every level. Returns `nil`. The `(constantly t)` pair is the "walk everything" spelling: ```console $ cat walk.lisp (uiop:collect-sub*directories "src/" (constantly t) (constantly t) (lambda (dir) (print dir))) $ rontolisp walk.lisp #P"src/" #P"src/main/" #P"src/test/" ``` ## Backend support All four backends, over the same one primitive [`directory`](directory.md) uses. --- # FILE: references/reference/functions/uiop-directory-exists-p.md # uiop:directory-exists-p `(uiop:directory-exists-p pathname)` Answers whether a *directory* exists: the pathname (with a trailing `/`) when it does, `nil` when it does not. The directory twin of [`uiop:file-exists-p`](uiop-file-exists-p.md), and libraries use it to validate a directory root before walking it. It is also what tells an EMPTY directory from a missing one: [`directory`](directory.md) answers `nil` for both. ```lisp (uiop:directory-exists-p "definitely-missing-dir") ; => NIL ``` ## Backend support All four backends, over the same one primitive [`directory`](directory.md) uses -- so a host without a filesystem (the browser playground) answers `nil` rather than failing, and a WASM module answers `nil` for everything unless it was run with `--dir`. --- # FILE: references/reference/functions/uiop-directory-files.md # uiop:directory-files `(uiop:directory-files pathspec &optional pattern)` The non-directory entries of a directory: `(directory "/*.*")` with the subdirectories dropped. `pathspec` names the directory itself (with or without a trailing `/`) -- the wildcard is supplied here, so this is the "just list it" spelling. `pattern` is UIOP's own optional second argument, the namestring of a name-and-type wildcard. It is appended to the directory and matched exactly as [`directory`](directory.md) would match it (`*` any sequence, `?` one character), so `(uiop:directory-files "db/" "*.up.sql")` lists only the up-migrations. Omitting it lists everything. A pattern carrying a DIRECTORY component is an error, as it is in real UIOP -- the directory to scan is the first argument's job. ```lisp (uiop:directory-files "no-such-directory/" "*.up.sql") ; => NIL ``` ## Backend support All four backends, over the same one primitive `directory` uses. --- # FILE: references/reference/functions/uiop-directory-pathname-p.md # uiop:directory-pathname-p `(uiop:directory-pathname-p pathname)` `t` when the pathname is in directory form -- non-wild, with no name and no type, i.e. empty or ending in `/`. Does **not** check that the directory exists (that is `uiop:directory-exists-p`). ```lisp (uiop:directory-pathname-p "/a/b/") ; => T ``` ```lisp (uiop:directory-pathname-p "/a/b") ; => NIL ``` ## Backend support Works on all four backends (Lisp source, `uiop-pathname.lisp`). --- # FILE: references/reference/functions/uiop-emptyp.md # uiop:emptyp `(uiop:emptyp x)` Returns `t` when `x` is `nil` or a zero-length vector (a string included), `nil` otherwise. UIOP's one-liner, kept verbatim from upstream: ```lisp (defun uiop:emptyp (x) (or (null x) (and (vectorp x) (zerop (length x))))) ``` So it answers the "nothing here" question for the two shapes an empty value takes, without deciding what a non-empty non-sequence means -- a number is simply not empty. ```lisp (list (uiop:emptyp nil) (uiop:emptyp "") (uiop:emptyp "ab")) ; => (T T NIL) ``` ## Backend support Works on all four backends: it is a uiop library definition written in rontolisp itself and compiled into the program when used. --- # FILE: references/reference/functions/uiop-enough-pathname.md # uiop:enough-pathname `(uiop:enough-pathname maybe-subpath base-pathname)` The [`uiop:subpathp`](uiop-subpathp.md) remainder when there is one, the pathname itself otherwise -- the shortest spelling that still names the file given the base. rove keys its source-location printing on it. ```lisp (uiop:enough-pathname #P"/tmp/a/b.txt" #P"/tmp/") ; => #P"a/b.txt" ``` ```lisp (uiop:enough-pathname #P"/x/a.txt" #P"/tmp/") ; => #P"/x/a.txt" ``` `uiop:with-enough-pathname` / `uiop:call-with-enough-pathname` run a body with this value, under `*default-pathname-defaults*` bound to the base ([uiop/pathname](../uiop/pathname.md#relative-to-a-base)). ## Backend support Works on all four backends (Lisp source, `uiop-pathname.lisp`). --- # FILE: references/reference/functions/uiop-ensure-pathname.md # uiop:ensure-pathname `(uiop:ensure-pathname pathname &key on-error defaults type dot-dot empty-is-nil want-pathname want-relative want-absolute ensure-absolute ensure-subpath want-file want-directory ensure-directory want-non-wild want-wild wilden want-existing ensure-directories-exist truename &allow-other-keys)` The constraint machine the rest of uiop routes through: coerces a designator (a string goes through [`uiop:parse-unix-namestring`](uiop-parse-unix-namestring.md)), then applies the `:want-*` checks and `:ensure-*` transforms in upstream's order. A failed check signals an error naming the pathname and the constraint, or calls a custom `:on-error` function. ```lisp (uiop:ensure-pathname "a/b" :ensure-directory t) ; => #P"a/b/" ``` ```lisp (handler-case (uiop:ensure-pathname "/a/b" :want-relative t) (error () :err)) ; => :ERR ``` Lite next to upstream, deliberately: the report is `Invalid pathname ~S: ~A`, `:want-logical` always fails (no logical pathname exists), `:resolve-symlinks` / `:truenamize` are accepted and ignored, and `:truename` answers what [`probe-file`](probe-file.md) answers. ## Backend support Works on all four backends (Lisp source, `uiop-pathname.lisp`). --- # FILE: references/reference/functions/uiop-file-exists-p.md # uiop:file-exists-p `(uiop:file-exists-p pathname)` Answers whether a file exists: the pathname when it does, `nil` when it does not. It is exactly [`probe-file`](probe-file.md) under its ASDF/UIOP name -- same contract, same behavior -- and it lowers onto that primitive on every backend, so libraries spelling the question the UIOP way (postmodern's `execute-file`, for instance) need no shim. The "truename" answered on success is a pathname carrying the argument namestring: no backend resolves symbolic links or makes the path absolute. A directory counts as existing. ```lisp (uiop:file-exists-p "definitely-missing.txt") ; => NIL ``` ## Backend support Works on all four backends. The interpreter registers a global function delegating to `probe-file`; the compile paths rewrite the one-argument call into a direct `probe-file` call. --- # FILE: references/reference/functions/uiop-file-pathname-p.md # uiop:file-pathname-p `(uiop:file-pathname-p pathname)` The parsed PATHNAME when a name or type component is present -- the namestring names a FILE rather than a directory -- and `nil` otherwise. Does **not** check that the file exists (that is `uiop:file-exists-p`). ```lisp (uiop:file-pathname-p "/a/b") ; => #P"/a/b" ``` ```lisp (uiop:file-pathname-p "/a/b/") ; => NIL ``` ## Backend support Works on all four backends (Lisp source, `uiop-pathname.lisp`). --- # FILE: references/reference/functions/uiop-first-char.md # uiop:first-char `(uiop:first-char s)` Returns the first character of the string `s`, or `nil` when `s` is empty or is not a string. UIOP's one-liner, kept verbatim from upstream, so the "is there a first character" test and the access are one call rather than a `length` guard plus a `char`. ```lisp (list (uiop:first-char "hello") (uiop:first-char "")) ; => (#\h NIL) ``` The mirror image is [`uiop:last-char`](uiop-last-char.md); quri's `render-uri` calls both (plus [`uiop:emptyp`](uiop-emptyp.md)) to decide whether a path needs a leading slash. ## Backend support Works on all four backends: it is a uiop library definition written in rontolisp itself and compiled into the program when used. --- # FILE: references/reference/functions/uiop-getenv.md # uiop:getenv `(uiop:getenv name)` / `(setf (uiop:getenv name) value)` Returns the value of the named environment variable as a string, or `nil` if the variable is unset. Common Lisp has no `getenv`, so this is homed in the `uiop` package -- the portable spelling implementation-independent libraries already use; there is no unqualified `getenv`. Works on all four backends; the WASM backend reads the real host environment in Preview 1 and `wasi:cli/environment@0.3.0` in `--component` mode -- including a `rontolisp:http-handler` component under `wasmtime serve`, which imports the interface for it -- so pass `--env`/`-S inherit-env` to wasmtime to make variables visible. `(setf (uiop:getenv name) value)` records an **override** that later reads consult before the host, and a `nil` value makes the variable read as unset. It does not change the process environment: no backend can (the JVM cannot at all, WASI's is read-only), so the override lives for this program run only -- see [uiop/os](../uiop/os.md#environment-variables). ```lisp (uiop:getenv "PATH") ``` The result is whatever the host has assigned to the variable, so it is non-deterministic; `(uiop:getenv "DEFINITELY_UNSET")` returns `nil`. ```lisp (setf (uiop:getenv "RONTOLISP_EXAMPLE_VAR") "set-here") (uiop:getenv "RONTOLISP_EXAMPLE_VAR") ; => "set-here" ``` --- # FILE: references/reference/functions/uiop-last-char.md # uiop:last-char `(uiop:last-char s)` Returns the last character of the string `s`, or `nil` when `s` is empty or is not a string -- [`uiop:first-char`](uiop-first-char.md)'s mirror image, kept verbatim from upstream UIOP. ```lisp (list (uiop:last-char "hello") (uiop:last-char "")) ; => (#\o NIL) ``` ## Backend support Works on all four backends: it is a uiop library definition written in rontolisp itself and compiled into the program when used. --- # FILE: references/reference/functions/uiop-merge-pathnames-star.md # uiop:merge-pathnames* `(uiop:merge-pathnames* specified &optional defaults)` Merges the `specified` path onto `defaults` and returns the result -- UIOP's defaults-aware variant of `merge-pathnames`, and the one portable libraries call to build a path relative to a data directory. Both arguments take either spelling (a pathname or a namestring); the result is a pathname. ```lisp (uiop:merge-pathnames* "b.txt" "/tmp/") ; => #P"/tmp/b.txt" ``` Omitting `defaults` merges against `""` (the namestring designating the working directory), which leaves `specified` unchanged -- the same answer `uiop:get-pathname-defaults` gives with the initial `*default-pathname-defaults*`. ## Backend support Works on all four backends: it is a Lisp-source definition over [`merge-pathnames`](merge-pathnames.md), compiled into the program when used. The compile paths additionally **fold to a literal** every call whose arguments they can resolve at compile time -- a string literal, or a reference to a top-level `defparameter` bound to one -- which is what makes `(uiop:merge-pathnames* *data-directory* "UnicodeData.txt")` in a bundled library cost nothing at run time. --- # FILE: references/reference/functions/uiop-native-namestring.md # uiop:native-namestring `(uiop:native-namestring pathname)` The pathname's namestring in the host operating system's own spelling. A rontolisp namestring already IS the host spelling -- no backend translates between a Lisp and a native syntax -- so this is `namestring`. Libraries call it where a path leaves Lisp (jzon stringifies a pathname value through it, trivial-mimes hands one to an external probe). ```lisp (uiop:native-namestring #P"/tmp/data.json") ; => "/tmp/data.json" ``` ## Backend support All four backends: the interpreter as a built-in, the compile paths lowered onto `namestring`. --- # FILE: references/reference/functions/uiop-parse-unix-namestring.md # uiop:parse-unix-namestring `(uiop:parse-unix-namestring name &key type defaults dot-dot ensure-directory &allow-other-keys)` Coerces `name` into a pathname using Unix syntax -- UIOP's portable pathname reader. A pathname passes through, `nil` stays `nil`, a symbol is downcased and read as a string. Empty and `"."` directory components are dropped; `".."` is kept as one level up. `:type` a string makes the whole last component the NAME with that type; `:ensure-directory` (or `:type :directory`) forces directory form. Remaining keys go to [`uiop:ensure-pathname`](uiop-ensure-pathname.md) (so `:want-relative t` rejects an absolute string). ```lisp (uiop:parse-unix-namestring "a//b/./c.txt") ; => #P"a/b/c.txt" ``` ```lisp (uiop:parse-unix-namestring "foo/bar" :type "lisp") ; => #P"foo/bar.lisp" ``` ## Backend support Works on all four backends (Lisp source, `uiop-pathname.lisp`). --- # FILE: references/reference/functions/uiop-pathname-directory-pathname.md # uiop:pathname-directory-pathname `(uiop:pathname-directory-pathname pathname)` The pathname's directory as a pathname -- name and type dropped, everything up to and including the last `/` kept. What [`uiop:subpathname`](uiop-subpathname.md) merges under. ```lisp (uiop:pathname-directory-pathname #P"/a/b/c.txt") ; => #P"/a/b/" ``` ## Backend support Works on all four backends (Lisp source, `uiop-pathname.lisp`). --- # FILE: references/reference/functions/uiop-pathname-parent-directory-pathname.md # uiop:pathname-parent-directory-pathname `(uiop:pathname-parent-directory-pathname pathname)` One directory level up from the pathname's directory: `/foo/bar/baz/file.type` answers `#P"/foo/bar/"`. The root's parent is the root, and the parent of a single-level relative directory is the empty pathname. ```lisp (uiop:pathname-parent-directory-pathname #P"/a/b/c.txt") ; => #P"/a/" ``` ## Backend support Works on all four backends (Lisp source, `uiop-pathname.lisp`). --- # FILE: references/reference/functions/uiop-print-condition-backtrace.md # uiop/image:print-condition-backtrace `(uiop/image:print-condition-backtrace condition &key stream count)` Prints a report for `condition` to `stream` (default `*error-output*`). **Lite**: no backend carries a Lisp-level call stack, so there is no backtrace to print and the report is the condition alone; `count` is accepted and ignored. Real UIOP falls back to the same shape on an implementation with no backtrace API. ```lisp (handler-case (error "boom") (error (c) (uiop/image:print-condition-backtrace c :stream *standard-output*))) ``` ``` boom ``` The name lives in the `uiop/image` package, as upstream, and the `uiop` package re-exports it -- `uiop:print-condition-backtrace` names the same function. `lack-middleware-backtrace` is what asks for it. ## Backend support Works on all four backends: it is a uiop library definition written in rontolisp itself and compiled into the program when used. --- # FILE: references/reference/functions/uiop-read-file-string.md # uiop:read-file-string `(uiop:read-file-string file &rest keys)` The entire contents of a file as one string -- the one-call spelling of opening it, reading it to the end and closing it. A missing file signals, exactly as [`open`](open.md) does. Lite: real UIOP passes its `&rest` keys through to the open, and here they are accepted and ignored. The only one that could matter, `:external-format`, has no rontolisp surface at all -- every backend reads UTF-8. ```console (let ((sql (uiop:read-file-string "db/20260101.up.sql"))) (print (length sql))) ``` ## Backend support All four backends -- one definition in rontolisp source, over `with-open-file` and a chunked [`read-sequence`](read-sequence.md) loop, so it runs anywhere a file can be opened for input. It deliberately does not size a single buffer from [`file-length`](file-length.md), which answers `nil` on both WASM backends. --- # FILE: references/reference/functions/uiop-relative-pathname-p.md # uiop:relative-pathname-p `(uiop:relative-pathname-p pathspec)` The parsed PATHNAME when `pathspec` is relative -- it does not start with `/` (the empty pathname included) -- and `nil` otherwise, the mirror of [`uiop:absolute-pathname-p`](uiop-absolute-pathname-p.md). ```lisp (uiop:relative-pathname-p "a/b") ; => #P"a/b" ``` ```lisp (uiop:relative-pathname-p "/a/b") ; => NIL ``` ## Backend support Works on all four backends (Lisp source, `uiop-pathname.lisp`). --- # FILE: references/reference/functions/uiop-split-name-type.md # uiop:split-name-type `(uiop:split-name-type filename)` Two values, the NAME and TYPE of a filename with no directory component: the last dot separates them, except a lone leading dot, which belongs to the name (the type is then `uiop:*unspecific-pathname-type*`, i.e. `nil`). ```lisp (multiple-value-list (uiop:split-name-type "foo.lisp")) ; => ("foo" "lisp") ``` ```lisp (multiple-value-list (uiop:split-name-type ".hidden")) ; => (".hidden" NIL) ``` ## Backend support Works on all four backends (Lisp source, `uiop-pathname.lisp`). --- # FILE: references/reference/functions/uiop-split-string.md # uiop:split-string `(uiop:split-string string &key max separator)` Splits `string` into a list of substrings on ANY character of the `separator` sequence (a string or a character list; the default is space and tab), following upstream UIOP's semantics: the scan runs right to left, so `:max` bounds the number of pieces while keeping the UNsplit remainder in the head, and the empty string yields `("")`. sxql tokenizes dotted column names with `(uiop:split-string name :separator ".")`. ```lisp (uiop:split-string "a.b.c" :separator ".") ; => ("a" "b" "c") ``` ```lisp (uiop:split-string "a.b.c.d.e" :max 3 :separator ".") ; => ("a.b.c" "d" "e") ``` ```lisp (uiop:split-string "a-b_c" :separator "-_") ; => ("a" "b" "c") ``` ## Backend support Works on all four backends: it is a uiop library definition written in rontolisp itself and compiled into the program when used. --- # FILE: references/reference/functions/uiop-subdirectories.md # uiop:subdirectories `(uiop:subdirectories pathspec)` The subdirectories of a directory, each with its trailing `/`: the [`uiop:directory-files`](uiop-directory-files.md) twin, over the same `(directory "/*.*")`, and what [`uiop:collect-sub*directories`](uiop-collect-sub-directories.md) recurses through. ```lisp (uiop:subdirectories "no-such-directory/") ; => NIL ``` ## Backend support All four backends, over the same one primitive [`directory`](directory.md) uses. --- # FILE: references/reference/functions/uiop-subpathname.md # uiop:subpathname `(uiop:subpathname pathname subpath &key type)` Merges `subpath` under the DIRECTORY of `pathname` -- the portable way a library names a file relative to a base. An absolute pathname OBJECT passes through unchanged; anything else is parsed as a relative Unix namestring (given `type`, the whole last component becomes the NAME and `type` the type) and merged. An absolute STRING subpath is an error (`:want-relative`). ```lisp (uiop:subpathname #P"/tmp/foo/" "bar/baz.txt") ; => #P"/tmp/foo/bar/baz.txt" ``` `uiop:subpathname*` is the same with a nil-tolerant base: `nil` answers `nil`, and a non-nil base is first put in directory form. ## Backend support Works on all four backends (Lisp source, `uiop-pathname.lisp`). Like [`uiop:merge-pathnames*`](uiop-merge-pathnames-star.md), the compile paths fold a call over literal arguments to a pathname literal. --- # FILE: references/reference/functions/uiop-subpathp.md # uiop:subpathp `(uiop:subpathp maybe-subpath base-pathname)` When `maybe-subpath` sits under `base-pathname`, returns the relative pathname that merges back onto the base to give `maybe-subpath`; `nil` otherwise. Both arguments must be pathname OBJECTS, both absolute, and the base in directory form. ```lisp (uiop:subpathp #P"/tmp/foo/bar.txt" #P"/tmp/") ; => #P"foo/bar.txt" ``` ```lisp (uiop:subpathp #P"/other/x" #P"/tmp/") ; => NIL ``` ## Backend support Works on all four backends (Lisp source, `uiop-pathname.lisp`). --- # FILE: references/reference/functions/uiop-symbol-call.md # uiop:symbol-call `(uiop:symbol-call package name &rest arguments)` UIOP's late-binding call: look `name` up in `package` at run time and apply it to `arguments`. Both are designators -- a keyword, a symbol or a string. This is how a library calls into a system it does not depend on and may not have loaded, which is why it is spelled with a run-time lookup rather than a direct call. ```lisp (uiop:symbol-call :cl :+ 1 2 3) ; => 6 ``` A package that does not exist, or a name that package does not have, signals -- the caller is about to apply the result, so an absent name is an error rather than a `nil` that fails one frame later. It is also a function value, which is how the idiom is usually written: a library dispatching to one of several backends applies `#'uiop:symbol-call` to the arguments it was handed, and names the backend with uninterned symbols so that reading the form cannot require the package to exist. ```lisp (defpackage :backend-a (:use :cl) (:export :request)) (in-package :backend-a) (defun request (uri &rest args) (list uri args)) (in-package :cl-user) (apply #'uiop:symbol-call '#:backend-a '#:request "http://x" '(:method :get)) ; => ("http://x" (:METHOD :GET)) ``` Each arm of such a dispatch is resolved on its own, so an arm naming a backend this program does not have costs a call-time error if it ever runs, and nothing at all otherwise. ## Backend support - **Interpreter**: full support (the lookup runs against the live package and function tables). - **JVM** and **WASM**: full support -- the call is late-bound through the compiled name registry, like `funcall` of a runtime-interned symbol, and `#'uiop:symbol-call` is a value like any other function. An absent package still signals; an absent *name* signals at the call (the undefined-function error) rather than at the lookup, slightly later than the interpreter's own probe but just as loud. --- # FILE: references/reference/functions/uiop-unix-namestring.md # uiop:unix-namestring `(uiop:unix-namestring pathname)` The Unix-style namestring of a pathname. A rontolisp namestring already is the Unix spelling, so this is [`namestring`](namestring.md) with UIOP's tolerance: `nil` and strings pass through unchanged. ```lisp (uiop:unix-namestring #P"/a/b.c") ; => "/a/b.c" ``` ## Backend support Works on all four backends (Lisp source, `uiop-pathname.lisp`). --- # FILE: references/reference/functions/unbound-slot-instance.md # unbound-slot-instance `(unbound-slot-instance condition)` The object whose slot was unbound when an `unbound-slot` condition was signalled -- the companion of [`cell-error-name`](cell-error-name.md), which names the slot. See [`slot-boundp`](../macros/slot-boundp.md). ```lisp (defclass usi-box () ((v))) (handler-case (slot-value (make-instance 'usi-box) 'v) (unbound-slot (e) (type-of (unbound-slot-instance e)))) ; => USI-BOX ``` --- # FILE: references/reference/functions/unexport.md # unexport `(unexport symbols &optional package)` The inverse of [`export`](export.md): makes `symbols` internal again in `package` (the current package by default). The symbols stay present — they are still reachable with the two-colon spelling — but are no longer visible unqualified through a [`use-package`](use-package.md). Returns `t`. Consumed at compile time and subject to the same "before you define" rule as `export`; see that page for both. ```lisp (defpackage #:partial (:use #:cl) (:export #:pub #:priv)) (in-package #:partial) (unexport 'priv) (defun pub () 1) (defun priv () 2) (in-package #:cl-user) (+ (partial:pub) (partial::priv)) ; => 3 ``` --- # FILE: references/reference/functions/union.md # union `(union list1 list2 &key test key)` Returns a list containing every element that appears in either `list1` or `list2`, treating both as sets. The comparison is `eql` by default; the optional `:test` keyword takes a function designator to use a different comparison, and the optional `:key` keyword takes a selector function applied to both compared elements. The order of elements in the result is unspecified. ```lisp (union '(1 2 3) '(2 3 4)) ; => (4 1 2 3) ``` ```lisp (union '("a" "b") '("b" "c") :test #'string=) ; => ("c" "a" "b") ``` --- # FILE: references/reference/functions/unread-char.md # unread-char `(unread-char character &optional stream)` Puts `character` -- which must be the one just read -- back so the next read returns it again, and answers `nil`. On a [Gray stream](../../guides/gray-streams.md) instance it dispatches to `rontolisp:stream-unread-char`, whose default method parks the character in the protocol's one-slot pushback; a class that can rewind its own source defines that generic instead. On a stream HANDLE -- a file, a string input stream, a socket -- the character goes into a handle-side pushback of its own, which `read-char`, `peek-char`, `read-char-no-hang` and `read-line` drain. One character for one stream is all either cell holds, which is what CL promises: a second `unread-char` with the cell still full signals. `read-byte`, `read-sequence` and `read` do not consult the handle-side cell. ```lisp (let* ((s (make-string-input-stream "abc")) (c (read-char s))) (unread-char c s) (list c (peek-char nil s) (read-char s) (read-line s))) ; => (#\a #\a #\a "bc") ``` --- # FILE: references/reference/functions/use-package.md # use-package `(use-package packages &optional package)` Adds `packages` (a package designator or a list of them) to the use list of `package` (the current package by default), so the **external** symbols of the used packages are visible unqualified afterwards. Returns `t`. Using a package twice is a no-op; using a package in itself is an error, as is an unknown package (`No such package: NOSUCH`). It is the runtime form of the [`defpackage`](../special-forms/defpackage.md) `:use` clause. Packages are resolved at read/compile time here (see [Packages](../packages.md)), so a literal top-level call is consumed at compile time like `in-package` and takes effect for the forms that follow it — which is what makes it work on every backend. A runtime-computed call (a designator built at run time) works on the interpreter only. Internal symbols are never inherited: only what the used package `:export`s becomes visible. ```lisp (defpackage #:greeter (:use #:cl) (:export #:hello)) (in-package #:greeter) (defun hello () "hi") (in-package #:cl-user) (use-package '#:greeter) (hello) ; => "hi" ``` --- # FILE: references/reference/functions/use-value.md # use-value `(use-value value [condition])` Invokes the innermost active `use-value` restart with `value`, and returns `nil` when none is active (like [`continue`](continue.md), not an error). Calling it from a [`handler-bind`](../macros/handler-bind.md) handler transfers control to the matching [`restart-case`](../macros/restart-case.md) clause with `value` as its argument — the idiom libraries use to substitute a value at the signal point (trivia's pattern expander lifts guard tests this way). ```lisp (define-condition needs-value () ()) (handler-bind ((needs-value (lambda (c) (use-value 42)))) (restart-case (progn (signal 'needs-value) :not-restarted) (use-value (v) (list :used v)))) ; => (:USED 42) ``` --- # FILE: references/reference/functions/user-homedir-pathname.md # user-homedir-pathname `(user-homedir-pathname &optional host)` Returns the user's home directory as a **directory** pathname -- the namestring always ends in a separator, and the name and type components are nil. The value comes from the `HOME` environment variable; the `host` argument is accepted and ignored, since there is one host. `nil` when `HOME` is unset, which Common Lisp allows and is the honest answer on a WASI guest that was given no environment at all. ```console $ rontolisp -e '(print (user-homedir-pathname))' #P"/home/you/" ``` ## Backend support Works on all four backends: one definition in rontolisp source over the per-backend environment primitive. --- # FILE: references/reference/functions/usocket-accessors.md # usocket:get-local-port usocket:get-local-address usocket:get-local-name usocket:get-peer-address usocket:get-peer-port usocket:get-peer-name `(usocket:get-local-port socket)` -- `(usocket:get-local-address socket)` -- `(usocket:get-local-name socket)` -- `(usocket:get-peer-address socket)` -- `(usocket:get-peer-port socket)` -- `(usocket:get-peer-name socket)` The usocket address accessors, over the [`rontolisp:tcp-local-port`](rontolisp-tcp-local-port.md) / [`tcp-local-address` / `tcp-peer-address` / `tcp-peer-port`](rontolisp-tcp-addresses.md) built-ins. The `get-local-*` accessors work on listeners and connected sockets (reading an ephemeral port back after listening on `usocket:*auto-port*` is the main use); the `get-peer-*` accessors work on connected sockets only. ```lisp (let* ((listener (usocket:socket-listen "127.0.0.1" usocket:*auto-port*)) (port (usocket:get-local-port listener))) (usocket:socket-close listener) (> port 0)) ; => T ``` `get-local-name` / `get-peer-name` return `(values address port)`: a `multiple-value-bind` receives both parts, and an ordinary single-value context receives the address. ```lisp (let ((listener (usocket:socket-listen "127.0.0.1" usocket:*auto-port*))) (multiple-value-bind (address port) (usocket:get-local-name listener) (usocket:socket-close listener) (list address (> port 0)))) ; => ("127.0.0.1" T) ``` ## Backend support - **Interpreter** and **JVM**: full support. - **WASM**: component mode only; all six accessors return real addresses and ports, like the interpreter/JVM (a failure returns `nil` instead of signaling). Preview 1 is a compile error. - **Browser playground**: not supported. --- # FILE: references/reference/functions/usocket-host-names.md # usocket:host-to-hostname usocket:get-host-by-name `(usocket:host-to-hostname host)` -- `(usocket:get-host-by-name name)` The usocket host-designator pair. `host-to-hostname` renders any designator upstream accepts as a hostname/dotted-quad string: `nil` is the wildcard host `"0.0.0.0"`, a string passes through, a vector quad (or list of four octets) and a host-byte-order 32-bit integer become the dotted quad. ```lisp (list (usocket:host-to-hostname nil) (usocket:host-to-hostname "example.com") (usocket:host-to-hostname #(192 168 0 1)) (usocket:host-to-hostname 2130706433)) ; => ("0.0.0.0" "example.com" "192.168.0.1" "127.0.0.1") ``` `get-host-by-name` is **lite**: rontolisp has no name-resolution primitive on any backend, so it renders its argument through `host-to-hostname` instead of resolving it to upstream's vector quad. That keeps the normalize-then-hand-on chain libraries use -- `(usocket:host-to-hostname (usocket:get-host-by-name address))` -- an identity on the address it is given, and the [`usocket:socket-connect`](usocket-socket-connect.md) / [`usocket:socket-listen`](usocket-socket-listen.md) call that address reaches still resolves it for real (natively on the interpreter and the JVM; IPv4 literals only on WASM). ```lisp (usocket:host-to-hostname (usocket:get-host-by-name "127.0.0.1")) ; => "127.0.0.1" ``` ## Backend support Works on all four backends, and answers identically on each: both are pure Lisp in the shim and neither opens a socket, so unlike the rest of the usocket API they are available on WASM Preview 1 too. --- # FILE: references/reference/functions/usocket-socket-accept.md # usocket:socket-accept `(usocket:socket-accept socket &key element-type)` Blocks until a client connects to the given listener and returns the accepted connection socket -- the usocket-compatible wrapper over [`rontolisp:tcp-accept`](rontolisp-tcp-accept.md). `:element-type` is accepted for compatibility and ignored (a rontolisp socket handle is always bidirectional). ```lisp (let* ((listener (usocket:socket-listen "127.0.0.1" usocket:*auto-port*)) (port (usocket:get-local-port listener)) (client (usocket:socket-connect "127.0.0.1" port)) (server (usocket:socket-accept listener)) (peer (usocket:get-peer-address server))) (usocket:socket-close server) (usocket:socket-close client) (usocket:socket-close listener) peer) ; => "127.0.0.1" ``` ## Backend support - **Interpreter** and **JVM**: full support. - **WASM**: component mode only; Preview 1 is a compile error. - **Browser playground**: not supported. --- # FILE: references/reference/functions/usocket-socket-connect.md # usocket:socket-connect `(usocket:socket-connect host port &key protocol element-type timeout deadline nodelay local-host local-port)` Opens a blocking TCP connection to `host` and `port` and returns a socket -- the [usocket](https://github.com/usocket/usocket)-compatible entry point over [`rontolisp:tcp-connect`](rontolisp-tcp-connect.md). In this shim a socket IS its stream handle, so `usocket:socket-stream` is the identity function and the stream built-ins (`read-line`, `write-line`, `write-string`, `read-byte`, `write-byte`, `close`) work on it directly. Only TCP is supported: `:protocol :datagram` (UDP) signals an error. The other keyword arguments are accepted for compatibility and ignored -- `:element-type` in particular, because a rontolisp socket handle is always bidirectional and supports both the line and the byte built-ins (there is no separate binary/character stream construction to select). The cl-postgres (Postmodern) connection shape works verbatim: ```console (usocket:socket-stream (usocket:socket-connect "localhost" 5432 :element-type '(unsigned-byte 8))) ``` A loopback echo round trip (single-threaded choreography: connect before accept, write before the peer reads): ```lisp (let* ((listener (usocket:socket-listen "127.0.0.1" usocket:*auto-port*)) (port (usocket:get-local-port listener)) (client (usocket:socket-connect "127.0.0.1" port :element-type '(unsigned-byte 8)))) (write-line "hello" (usocket:socket-stream client)) (let* ((server (usocket:socket-accept listener)) (line (read-line (usocket:socket-stream server)))) (usocket:socket-close server) (usocket:socket-close client) (usocket:socket-close listener) line)) ; => "hello" ``` The `usocket` package is loaded automatically on first use (interpreter) or spliced into the compiled program (JVM / WASM component), and is also registered as the built-in ASDF system `"usocket"`, so `(asdf:load-system "usocket")`, `(ql:quickload :usocket)` and a third-party `.asd`'s `:depends-on ("usocket")` all resolve to it without touching the network. On the interpreter and the JVM a connection failure signals a typed `usocket:socket-error` condition (message preserved), so `(handler-case (usocket:socket-connect ...) (usocket:socket-error (e) ...))` works there; the subtypes (`usocket:connection-refused-error` etc.) are defined, but the re-signal always uses `socket-error`, so catch that. On the WASM component backend a failed connection returns `nil` instead of signaling -- test the returned handle for `nil`. ## Backend support - **Interpreter** and **JVM**: full support (delegates to `rontolisp:tcp-connect`). - **WASM**: component mode only (`--component`), IPv4 literals only; a failed connection returns `nil`. Preview 1 is a compile error. - **Browser playground**: not supported. --- # FILE: references/reference/functions/usocket-socket-listen.md # usocket:socket-listen `(usocket:socket-listen host port &key reuse-address backlog element-type)` Binds a listening TCP socket on `host` and `port` and returns a listener -- the usocket-compatible wrapper over [`rontolisp:tcp-listen`](rontolisp-tcp-listen.md) (note the flipped argument order: usocket passes the host first). A `host` of `usocket:*wildcard-host*` (`"0.0.0.0"`) or `nil` listens on all interfaces; a `port` of `usocket:*auto-port*` (`0`) picks a free ephemeral port, which `usocket:get-local-port` reads back. The keyword arguments are accepted for compatibility and ignored (the backlog is the runtime default). ```lisp (let* ((listener (usocket:socket-listen usocket:*wildcard-host* usocket:*auto-port*)) (port (usocket:get-local-port listener))) (usocket:socket-close listener) (> port 0)) ; => T ``` Accept connections with [`usocket:socket-accept`](usocket-socket-accept.md). ## Backend support - **Interpreter** and **JVM**: full support. - **WASM**: component mode only; a failed bind returns `nil`. Preview 1 is a compile error. - **Browser playground**: not supported. --- # FILE: references/reference/functions/usocket-socket-stream.md # usocket:socket-stream usocket:socket-close `(usocket:socket-stream socket)` -- `(usocket:socket-close socket)` `socket-stream` returns the stream associated with a socket; `socket-close` flushes and closes it. In this shim a socket IS its stream handle (the `rontolisp:tcp-*` handles share the file-stream handle space), so `socket-stream` is the identity function -- it exists so portable usocket code like `(read-line (usocket:socket-stream sock))` runs unchanged -- and `socket-close` is `close`. ```lisp (usocket:socket-stream 42) ; => 42 ``` ```console (let ((stream (usocket:socket-stream sock))) (write-line "ping" stream) (print (read-line stream)) (usocket:socket-close sock)) ``` ## Backend support - **Interpreter**, **JVM** and **WASM component**: wherever the socket itself works (`socket-stream` is pure and works everywhere). - **Browser playground**: `socket-stream` works; sockets do not. --- # FILE: references/reference/functions/values-list.md # values-list `(values-list list)` Spreads `list` as multiple values: the first element is the primary value and the rest reach the multiple-value consumers, so `(values-list '(1 2))` is equivalent to `(values 1 2)`. An empty list yields no values (nil). ```lisp (multiple-value-list (values-list '(1 2 3))) ; => (1 2 3) ``` ```lisp (multiple-value-bind (a b) (values-list '(10)) (list a b)) ; => (10 NIL) ``` --- # FILE: references/reference/functions/values.md # values `(values form...)` Returns multiple values. In an ordinary (single-value) context every argument is evaluated and the first is the result, like `prog1`; `(values)` reads as nil. The extra values are received by the consumers [`multiple-value-bind`](../macros/multiple-value-bind.md), [`multiple-value-list`](../macros/multiple-value-list.md), [`multiple-value-call`](../macros/multiple-value-call.md) and [`nth-value`](../macros/nth-value.md) -- either syntactically (a literal `(values ...)` producer) or, for a `values` in result position of a user function, through an internal channel that carries them across the call boundary to the consumer. A function that returns normally without calling `values` supplies a single value (extra variables read nil). Calling a `values`-returning function through `funcall #'values` or from compiled first-class contexts yields the primary value only. ```lisp (multiple-value-list (values 1 2 3)) ; => (1 2 3) ``` ```lisp (values 1 2 3) ; => 1 ``` --- # FILE: references/reference/functions/vector-pop.md # vector-pop `(vector-pop vector)` Decrements the fill pointer of a vector created with [`make-array`](make-array.md) `:fill-pointer` and returns the element it moved past (the last pushed element). Signals an error when the vector has no fill pointer or the fill pointer is already 0. ```lisp (defparameter *v* (make-array 3 :fill-pointer 0)) (vector-push 10 *v*) ; => 0 (vector-push 20 *v*) ; => 1 (vector-pop *v*) ; => 20 (length *v*) ; => 1 ``` --- # FILE: references/reference/functions/vector-push-extend.md # vector-push-extend `(vector-push-extend value vector &optional extension)` Like [`vector-push`](vector-push.md), but when the vector is full it grows the backing storage (by at least `extension` elements, default 1) instead of returning nil, so the push always succeeds and the new index is returned. Any vector with a fill pointer can be grown, whether or not it was created `:adjustable` (matching common practice; [`adjustable-array-p`](adjustable-array-p.md) reports the flag verbatim). Signals an error when the vector has no fill pointer. ```lisp (defparameter *v* (make-array 1 :fill-pointer 0 :adjustable t)) (vector-push-extend 10 *v*) ; => 0 (vector-push-extend 20 *v* 4) ; => 1 *v* ; => #(10 20) ``` --- # FILE: references/reference/functions/vector-push.md # vector-push `(vector-push value vector)` Stores `value` at the fill pointer of a vector created with [`make-array`](make-array.md) `:fill-pointer`, increments the fill pointer and returns the index the value was stored at, or nil (leaving the vector untouched) when the vector is already full. Use [`vector-push-extend`](vector-push-extend.md) to grow the vector instead. Signals an error when the vector has no fill pointer. ```lisp (defparameter *v* (make-array 2 :fill-pointer 0)) (vector-push 10 *v*) ; => 0 (vector-push 20 *v*) ; => 1 (vector-push 30 *v*) ; => NIL *v* ; => #(10 20) ``` --- # FILE: references/reference/functions/vector.md # vector `(vector &rest elements)` Creates and returns a fresh rank-1 array containing the given elements, evaluated left to right; `(vector)` returns an empty vector `#()`. It is equivalent to [`make-array`](make-array.md) with the element count as the dimension followed by [`aref`](aref.md)-style stores, but in one step. Like `make-array` and `aref`, `vector` is not a first-class function value -- `#'vector` is unavailable, so call it directly. ```lisp (vector 1 2 3) ; => #(1 2 3) (vector) ; => #() (aref (vector 10 20 30) 2) ; => 30 ``` --- # FILE: references/reference/functions/vectorp.md # vectorp `(vectorp value)` Returns `t` when `value` is a vector. Strings are vectors in Common Lisp, so they pass too. Like the `vector` type specifier in `typecase`, the rank is not checked — a multi-dimensional array also passes. ```lisp (vectorp (vector 1 2 3)) ; => T ``` ```lisp (list (vectorp "abc") (vectorp '(1 2))) ; => (T NIL) ``` --- # FILE: references/reference/functions/wild-pathname-p.md # wild-pathname-p `(wild-pathname-p pathname &optional field-key)` Whether the pathname carries a wildcard. With no `field-key` (or `nil`) it answers true when ANY component is wild; with one of `:directory`, `:name` or `:type` it tests just that component, and `:host` / `:device` / `:version` are always `nil` because those components do not exist ([`pathname-host`](pathname-host.md)). A component is wild when it holds a `*` (any run of characters) or a `?` (one character) -- the same wildcards [`directory`](directory.md) matches with, so this predicate and that matcher cannot disagree. A `**` directory component (`:wild-inferiors`, any number of levels) is wild by that same rule. ```lisp (list (wild-pathname-p "d/*.txt") (wild-pathname-p "d/a.txt") (wild-pathname-p "d/*.txt" :name) (wild-pathname-p "d/*.txt" :type) (wild-pathname-p "*/a.txt" :directory)) ; => (T NIL T NIL T) ``` An unknown field key signals. ## Backend support All four backends -- one definition in rontolisp source over primitives every backend has. --- # FILE: references/reference/functions/write-byte.md # write-byte `(write-byte byte stream)` Writes one byte -- an integer between 0 and 255 -- to a binary output stream (a stream opened with `:direction :output :element-type '(unsigned-byte 8)`) and returns the byte. Works in all four backends. The byte is written raw, with no newline or other framing added. `stream` takes the same designators every other stream operation takes: `t` is the process standard output, `nil` means the current `*standard-output*` (which holds `t` unless you bind it), and `*error-output*` is standard error. `(write-byte b *standard-output*)` therefore puts raw octets on standard output, in order with anything `princ` and `format` write there. Because it touches the filesystem, `write-byte` is shown here statically rather than as a runnable example: ```console (with-open-file (out "data.bin" :direction :output :element-type '(unsigned-byte 8)) (write-byte 137 out) ; => 137 (write-byte 80 out) (write-byte 78 out) (write-byte 71 out)) (write-byte 137 *standard-output*) ; one raw octet on stdout ``` This writes the four bytes `89 50 4E 47` (the start of a PNG signature) to `data.bin`. The interpreter and JVM signal an error for a value outside 0-255. --- # FILE: references/reference/functions/write-line.md # write-line `(write-line string &optional stream)` Writes the given string followed by a newline, and returns the string. With no stream argument it writes to standard output; given an output stream it writes there instead -- a file stream opened by `open` or `with-open-file`, a socket, or a `with-output-to-string` string stream. Works in all three backends. Unlike `print`/`prin1`, it writes the raw string contents without surrounding quotes. ```console (with-open-file (out "greeting.txt" :direction :output) (write-line "hello" out) (write-line "world" out)) ``` This writes two lines, `hello` and `world`, into `greeting.txt`. Each call appends its own trailing newline and returns the string it wrote. --- # FILE: references/reference/functions/write-sequence.md # write-sequence `(write-sequence sequence stream &key start end)` Writes the elements of `sequence` to `stream` and returns the sequence. Writing starts at index `:start` (default 0) and stops before index `:end` (default the sequence length). The `:start`/`:end` keywords must be literal; their values may be arbitrary expressions. Works in all three backends. When `sequence` is a string, the bounded slice is written as characters (like `write-string`), so it works with a text output stream such as the one from `with-output-to-string`: ```lisp (with-output-to-string (s) (write-sequence "abcd" s :start 1 :end 3)) ; => "bc" ``` When `sequence` is a one-dimensional array of integers between 0 and 255, it expands into a `write-byte` loop, so it requires a stream opened with `:direction :output :element-type '(unsigned-byte 8)`. Because it touches the filesystem, `write-sequence` is shown here statically rather than as a runnable example: ```console (let ((buf (make-array 4))) (setf (aref buf 0) 222) (setf (aref buf 1) 173) (setf (aref buf 2) 190) (setf (aref buf 3) 239) (with-open-file (out "data.bin" :direction :output :element-type '(unsigned-byte 8)) (write-sequence buf out))) ; => the array ``` This writes the four bytes `DE AD BE EF` to `data.bin`. Use `:start`/`:end` to write only a slice of the array. ## Packed buffers: raw binary elements in bulk When `sequence` is a **packed** array -- a packed float array of any rank (`single-float` / `double-float`, `#f(...)` / `#d(...)`) or a packed integer vector (`(unsigned-byte 8|16|32)`) -- its elements go out as **raw little-endian binary** in one bulk transfer, exactly the bytes [`read-sequence`](read-sequence.md) reads back into the same kind of buffer: 4 bytes per single-float, 8 per double-float, 2 per `(unsigned-byte 16)`, and so on, a rank-n float array in row-major order (`:end` defaults to the total size). A `#f` matrix written this way is a `float32` file numpy or C reads directly. ```console (with-open-file (out "weights.bin" :direction :output :element-type '(unsigned-byte 8)) (write-sequence #f((1.0 2.0) (3.0 4.0)) out)) ; => the array; 16 bytes written ``` --- # FILE: references/reference/functions/write-string.md # write-string `(write-string string &optional stream &key start end)` Writes the raw string contents -- without surrounding quotes and without a trailing newline -- and returns the string: `write-line` minus the newline. With no stream argument it writes to standard output; given an output stream (a file stream or a `with-output-to-string` string stream) it writes there instead. The `:start`/`:end` keywords bound the written substring (a `nil` `:end` means the string's length); the full string is still the return value. A CLOS instance extending rontolisp's Gray output-stream base class also works as the stream -- the write dispatches to `rontolisp:stream-write-string`. A [TCP or TLS socket handle](../../guides/tcp-sockets.md) works too: the string's UTF-8 bytes go on the wire immediately, with no trailing newline. ```lisp (write-string "one, ") (write-string "two") ``` ``` one, two ``` --- # FILE: references/reference/functions/write-to-string.md # write-to-string `(write-to-string object)` Returns `object`'s readable (`prin1`) printed representation as a string -- an alias for [prin1-to-string](prin1-to-string.md). The full Common Lisp `write` keyword arguments (`:escape`, `:base`, ...) are not supported. ```lisp (write-to-string '(a b 3)) ; => "(A B 3)" ``` --- # FILE: references/reference/functions/write.md # write `(write object &key stream escape readably pretty circle right-margin miser-width lines pprint-dispatch)` Writes `object` to `stream` (default standard output) and returns `object`. Each keyword binds the matching printer control variable around that one print, exactly as Common Lisp defines them. `:escape` (default: the value of `*print-escape*`, which is `t`) selects the readable `prin1` form; `:escape nil` selects the `princ` form. The rest are accepted and bind their variable, but the printer's layout never changes -- see `pprint` for why. Note that `write-to-string` takes no keywords here: use `(with-output-to-string (s) (write x :stream s ...))` when you need them. ```lisp (with-output-to-string (s) (write "hi" :stream s :escape nil)) ; => "hi" ``` --- # FILE: references/reference/functions/y-or-n-p.md # y-or-n-p `(y-or-n-p &optional format-control &rest format-arguments)` Asks a yes/no question and returns `t` or `nil`. The optional [`format`](../macros/format.md) control and its arguments are printed first, followed by `" (y or n) "`; then one line is read from standard input. A line starting with `y` or `Y` answers `t`, one starting with `n` or `N` answers `nil`, and anything else -- an empty line included -- re-asks, reprinting the whole prompt. Lite: Common Lisp reads single CHARACTERS without echo, where this reads a whole line, so the answer is only taken once the user presses return. End of input answers `nil` instead of signalling, because a backend with no interactive user has no way to ask again. ```console (if (y-or-n-p "Delete ~A?" "old.sql") (delete-file "old.sql") (print "kept")) ``` Answering `maybe` prints `Delete old.sql? (y or n) ` a second time; answering `yes` (or a bare `y`) then returns `t` and the file is deleted. ## Backend support All four backends -- one definition in rontolisp source over [`format`](../macros/format.md) and [`read-line`](read-line.md). --- # FILE: references/reference/functions/zerop.md # zerop `(zerop number)` Returns `t` if `number` is zero, else `nil`. It works for any numeric type; an integer, float or ratio of value zero all satisfy it. ```lisp (zerop 0) ; => T ``` ```lisp (zerop 0.0) ; => T ``` --- # FILE: references/reference/macros.md # Macros **Each macro name in the table links to its own page**, with a fuller description and a runnable example you can evaluate in your browser. | Macro | Syntax | Description | |-------|--------|-------------| | `cond` | `(cond (test1 body1...) ...)` | Conditional with multiple clauses. Returns body of first truthy test | | `case` | `(case key (k1 body1...) ((k2 k3) body2...) (otherwise body...))` | Dispatch on a key compared with `eql`. Keys are unevaluated; a list key matches any element; `t`/`otherwise` is the default. Returns nil if nothing matches | | `ecase` | `(ecase key (k1 body1...) ((k2 k3) body2...))` | Exhaustive `case`: no default clause (`t`/`otherwise` are ordinary keys), and an unmatched key signals an `error` | | `ccase` | `(ccase key (k1 body1...) ...)` | Like `ecase`; an unmatched key signals an `error`. rontolisp establishes no `store-value` restart around it, so it is identical to `ecase` (not correctable) | | `and` | `(and expr1 expr2...)` | Short-circuit AND. Returns first nil or last value. `(and)` returns `t` | | `or` | `(or expr1 expr2...)` | Short-circuit OR. Returns first non-nil value or nil. `(or)` returns `nil` | | `when` | `(when condition body...)` | Evaluates body when condition is true, returns nil otherwise | | `unless` | `(unless condition body...)` | Evaluates body when condition is nil, returns nil otherwise | | `dotimes` | `(dotimes (var count result?) body...)` | Evaluate body with `var` bound to `0`..`count-1`. Returns `result` (or nil) | | `do` | `(do ((var init step?)...) (end-test result...) body...)` | Iterate with parallel-stepped variables. Returns the `result` forms when `end-test` is true | | `do*` | `(do* ((var init step?)...) (end-test result...) body...)` | Like `do` but bindings and steps are sequential (`let*`-style): each init/step form sees the variables already updated this iteration | | `loop` | `(loop for i from 1 to n collect (f i))` | A bounded subset of the ANSI `loop`: numeric/list stepping (`for`), accumulation (`collect`/`sum`/`count`/...), and simple control clauses (`while`/`repeat`/`when`/`finally`/`return`). See the page for the full grammar and limitations | | `prog1` | `(prog1 first body...)` | Evaluate all forms in order, return the value of `first` | | `multiple-value-prog1` | `(multiple-value-prog1 (floor 17 5) (cleanup))` | Like `prog1` but returns ALL values of the first form | | `prog2` | `(prog2 first second body...)` | Evaluate all forms in order, return the value of `second` | | `time` | `(time form)` | Evaluate `form`, print the elapsed real time to standard output (`; Elapsed real time: N ms`), and return the form's value. `N` is an integer of milliseconds on the interpreter/JVM and a float of milliseconds on WASM | | `psetq` | `(psetq v1 e1 v2 e2 ...)` | Parallel assignment: every right-hand side is evaluated before any variable is assigned. Returns nil | | `psetf` | `(psetf place1 e1 place2 e2 ...)` | `psetq` generalized to `setf` places: place subforms and values are all evaluated before any assignment. Returns nil | | `block` | `(block name body...)` | Named block: returns the last form's value or the value of a matching `(return-from name v)`. The match is lexical on every backend, so a `return-from` inside a closure exits the block that encloses it in the source | | `typecase` | `(typecase x (integer body...) (string body...) (t default...))` | Dispatch on the type of `x`. Supported type names: `integer`, `float`, `number`, `rational`, `string`, `symbol`, `keyword`, `cons`, `list`, `null`, `atom`, `character`, `hash-table`, `boolean` (plus `t`/`otherwise`), and the compound specifiers `(or ...)`/`(and ...)`/`(not ...)`/`(member ...)`/`(eql ...)`/`(satisfies ...)` and ranged numeric types like `(integer 0 9)`. Returns nil if nothing matches | | `etypecase` | `(etypecase x (integer body...) (string body...))` | Exhaustive `typecase`: no default clause, and an object whose type matches no clause signals an `error` | | `ctypecase` | `(ctypecase x (integer body...) (string body...))` | Like `etypecase`; an object whose type matches no clause signals an `error`. rontolisp establishes no `store-value` restart around it, so it is identical to `etypecase` (not correctable) | | `error` | `(error "bad value: ~a" x)`, `(error 'my-error :v x)`, `(error obj)` | Signal an error, aborting execution unless a [`handler-case`](macros/handler-case.md) catches it. Designators: a literal control string (same directives as `format`), a quoted condition-type symbol with initargs (constructs a typed condition; the `define-condition` `:report` becomes the message), or a condition object. The interpreter and JVM throw an exception carrying the message and the condition; wasm-GC throws a WebAssembly exception when the program contains a catching form, and traps otherwise. Like `format`, it is a macro with no function value (`#'error` is unsupported) | | `cerror` | `(cerror continue-format datum args...)` | Signal a **continuable** error: a `continue` restart is established around the signal, so a `handler-bind` handler can call [`continue`](functions/continue.md) to resume past it with nil; otherwise behaves like `error` | | `signal` | `(signal 'my-condition :v x)` | Signal a **non-fatal** condition (same designators as `error`): raised to an established `handler-case`, otherwise returns nil and continues (always nil under `--no-gc`) | | `handler-case` | `(handler-case expr (type (var) body...)... (:no-error (v) body...))` | Evaluate `expr`, dispatching a signaled error to the first clause whose condition type matches (rethrown when none does); `:no-error` runs on normal completion. On wasm-GC needs `wasmtime -W exceptions=y`; compile error under `--no-gc` | | `ignore-errors` | `(ignore-errors form...)` | The forms' value, or nil when an error is signaled; sugar over `handler-case`. On wasm-GC needs `wasmtime -W exceptions=y`; compile error under `--no-gc` | | `handler-bind` | `(handler-bind ((type handler)...) body...)` | Establish handlers that run **at the signal point, before unwinding**, so they can invoke a [`restart-case`](macros/restart-case.md) restart; a handler that returns declines. Compile error under `--no-gc` | | `restart-case` | `(restart-case form (name (args...) body...)...)` | Evaluate `form` with named restarts established; an invoked restart unwinds back here and runs its clause inline (a clause may `go` into an enclosing `tagbody`). Compile error-free everywhere; `--no-gc` keeps the primary-form-only lowering | | `restart-bind` | `(restart-bind ((name fn)...) body...)` | Establish restarts whose functions run **at the invocation point** (no unwinding) | | `with-simple-restart` | `(with-simple-restart (name fmt args...) body...)` | Sugar over `restart-case`: invoking the restart returns `(values nil t)` from the form | | `setf` | `(setf place value)` | Generalized assignment. Supports `car`, `cdr`, `nth`, `first`..`fourth`, `rest`, `caXXXr` as places | | `push` | `(push item place)` | Prepend item to list at place. Returns the new list | | `pop` | `(pop place)` | Remove and return the first element from list at place | | `remf` | `(remf place indicator)` | Remove key-value pair from property list at place. Returns `t` if found, `nil` otherwise | | `let*` | `(let* ((x 1) (y x)) body...)` | Sequential bindings: each init form sees the previous bindings. Expands to nested `let` | | `dolist` | `(dolist (var list result?) body...)` | Evaluate body with `var` bound to each element. Returns `result` (or nil) with `var` bound to nil | | `incf` | `(incf place delta?)` | Expands to `(setf place (+ place delta))`. `delta` defaults to 1. Returns the new value | | `decf` | `(decf place delta?)` | Expands to `(setf place (- place delta))`. `delta` defaults to 1. Returns the new value | | `format` | `(format t "Hello ~a, ~d!~%" 'world 42)`, `(format nil "~a" x)` | Formatted output to standard output (`t`, returns nil) or to a string (`nil`) | | `with-open-file` | `(with-open-file (s "f.txt" :direction :output) (write-line "hi" s))` | Open a file, bind the stream to `s`, evaluate the body, close the file. Returns the body value. Supports the `:direction` option (`:input` default, `:output`) and the `:element-type` option (`'character` default, `'(unsigned-byte 8)` for a binary stream); both must be literal | | `with-open-stream` | `(with-open-stream (s (make-string-input-stream "hi")) (read-line s))` | Bind an ALREADY-OPEN stream, evaluate the body, close it. `with-open-file` without the `open` | | `check-type` | `(check-type place typespec [string])` | Signal an error when the value of `place` is not of the given type; return nil when it is. Lite version: no restarts, so the place is never re-stored | | `assert` | `(assert test-form [(place...) [datum args...]])` | Signal an error when `test-form` is false; return nil when it is true. The places list is accepted but ignored (no restarts) | | `declare` | `(declare declaration...)` | Parsed no-op: evaluates to nil, arguments never evaluated or validated | | `declaim` | `(declaim declaration...)` | Parsed no-op like `declare`, for file-level declarations | | `proclaim` | `(proclaim declaration)` | Parsed no-op like `declaim` (deviates from CL: classified as a macro, the argument is not evaluated) | | `the` | `(the type form)` | Returns the value of `form` unchanged; the type is not checked | | `eval-when` | `(eval-when (situation...) body...)` | Evaluates the body as a `progn`; every situation is treated as "evaluate now". Top-level bodies are spliced so nested `defun`/`defmacro` definitions are collected | | `locally` | `(locally declaration... form...)` | Evaluates the body as a `progn`; the leading `declare` forms are dropped (declarations are parsed no-ops) | | `with-standard-io-syntax` | `(with-standard-io-syntax form...)` | Binds `*package*` to `cl-user` and evaluates the body as a `progn`. Every other reader/printer control variable Common Lisp asks it to rebind is, in rontolisp, either informational (`*read-default-float-format*`) or unread by any reader/printer | | `write-char` | `(write-char char [stream])` | Write one character (returning it); expands to `write-string` of the one-character string, so file and string streams work | | `flet` | `(flet ((name lambda-list body...)...) body...)` | Local, non-recursive function bindings (Lisp-2: call position and `#'name`). A definition body sees the outer function of the same name, not its siblings. Lambda lists support the `defun` extensions | | `labels` | `(labels ((name lambda-list body...)...) body...)` | Like `flet` but the definitions see each other (recursion and mutual recursion) | | `symbol-macrolet` | `(symbol-macrolet ((name expansion)...) body...)` | Local symbol macros: a free reference to `name` evaluates `expansion` in its place, and `setq`/`setf` of `name` assigns through the expansion place. Inner bindings of the same name shadow | | `multiple-value-bind` | `(multiple-value-bind (var...) values-form body...)` | Binds the variables to the values of the producer form. A literal `(values ...)` call, the multi-value built-ins (`floor` family, `gethash`, `parse-integer`) and a user function returning `(values ...)` supply all of their values. Extra variables bind to nil | | `multiple-value-list` | `(multiple-value-list values-form)` | Collects the producer's values into a list (recognized like `multiple-value-bind`) | | `multiple-value-call` | `(multiple-value-call function values-form...)` | Calls the function with all values of every producer as the arguments, including a user function's values spread at runtime (deviates from CL: classified as a macro, not a special operator) | | `nth-value` | `(nth-value n values-form)` | The n-th (0-based) value of the producer, or nil; expands to `nth` over `multiple-value-list` | | `make-instance` | `(make-instance 'class-name :initarg value ...)` | Create an instance of a [`defclass`](special-forms/defclass.md) class (static CLOS subset). The class name may be literal or computed; the class SET is fixed at compile time | | `slot-value` | `(slot-value object 'slot-name)` | Read a slot of a [`defclass`](special-forms/defclass.md) instance; a `setf`-able place. The slot name must be a literal quoted symbol | | `with-slots` | `(with-slots (x (v y)) instance body...)` | Bind slot names as symbol-macro-style places for the body: reads see the slots, and `setf`/`push`/`incf` of a bound name writes back to the slot. Resolves `defstruct` slots too | | `with-accessors` | `(with-accessors ((x pt-x)) instance body...)` | Bind variables as symbol-macro-style places standing for accessor calls on the instance | | `change-class` | `(change-class obj 'class :initarg v)` | Change an instance's class in place (identity kept, shared slots kept, new slots from their `:initform`s) and return it | | `rontolisp:with-arena` | `(rontolisp:with-arena () body...)` | Run the body and return its value, naming a memory-reclamation boundary for the non-GC WASM backend (`--no-gc`): everything allocated inside is popped at the end, keeping only the body's value. A plain `progn` on the other backends (a real GC already reclaims) | | `rontolisp:with-mutex` | `(rontolisp:with-mutex (mutex-form) body...)` | Acquire the mutex, run the body, and release it on every exit (a signalled error included). Real mutual exclusion on the interpreter and the JVM backend, where a served handler runs one virtual thread per request; a no-op on the single-threaded WASM backends | | `torch:no-grad` | `(torch:no-grad body...)` | Run the body with gradient recording disabled: the `torch` operations inside compute values but record nothing on the autograd tape (the training-loop update / inference idiom) | | `uiop:if-let` | `(uiop:if-let ((a x) (b y)) then else)` | Bind the variables in parallel like `let`, then take the `then` branch only when EVERY variable came out non-nil. A single un-nested binding (`(uiop:if-let (x form) ...)`) is accepted too | | `uiop:when-let` | `(uiop:when-let ((a x)) body...)` | `uiop:if-let` with an implicit `progn` body and no else branch: the body runs only when every variable came out non-nil, otherwise nil | | `uiop:when-let*` | `(uiop:when-let* ((a x) (b (f a))) body...)` | The sequential `uiop:when-let`: each form sees the bindings before it, and the first nil one short-circuits to nil without evaluating the rest | | `uiop:with-deprecation` | `(uiop:with-deprecation (:style-warning) (defun old-f (x) x))` | Establish the wrapped definitions exactly as written. Lite: rontolisp has no deprecation-warning channel, so the level form is ignored and no warning is ever produced | | `prog` | `(prog ((v init)...) tag-or-form...)` | `let` + `tagbody` inside a block: `go` jumps between the body's tags and `(return x)` exits with `x` | | `prog*` | `(prog* ((v init)...) tag-or-form...)` | Like `prog` with sequential (`let*`-style) bindings | | `shiftf` | `(shiftf a b 9)` | Shift place values left, store the last value into the last place, return the first place's old value | | `load-time-value` | `(load-time-value form)` | Evaluates `form` once per occurrence in the source (lazily, on first use), not once per use | | `define-compiler-macro` | `(define-compiler-macro name (params...) body...)` | Rewrite calls to `name` at compile time; returning the `&whole` form declines. A hint: it is ignored when the body signals, when `name` is a standard operator, or under `apply`/`funcall` | | `typep` | `(typep x '(unsigned-byte 8))` | Type test over the `typecase` specifier set; the specifier must be a literal (quoted) type | | `slot-boundp` | `(slot-boundp obj 'slot)` | Whether the slot holds a value: `nil` for an unknown slot, one written with no `:initform` and never supplied, or one `slot-makunbound` emptied | | `slot-makunbound` | `(slot-makunbound obj 'slot)` | Makes the slot unbound and returns the instance; a later read signals `unbound-slot` | | `slot-exists-p` | `(slot-exists-p obj 'slot)` | Whether the instance's class declares the slot, regardless of boundness; `nil` for a non-instance | | `print-unreadable-object` | `(print-unreadable-object (obj stream :type t) body...)` | Writes `#<[type ]...>` around the body's output; returns nil (`:identity` accepted, prints no address) | | `with-package-iterator` | `(with-package-iterator (next pkgs :external) body...)` | Lite: binds the iterator name to a local FUNCTION always reporting no more symbols (no intern table) | | `do-external-symbols` | `(do-external-symbols (s :rontolisp) (print s))` | Iterate a package's exported symbols (interpreter only: the compiled backends carry no package registry) | | `do-symbols` | `(do-symbols (s :cl-user) (print s))` | Iterate every symbol ACCESSIBLE in a package -- its own plus what it inherits (interpreter only, same reason) | | `with-compilation-unit` | `(with-compilation-unit (:override t) body...)` | A `progn` around the body: the options only merge a deferred-warning report, and there is no `compile-file` to defer one from | Macros have no function value: `#'cond` or `(funcall 'setf ...)` is an error. Convenience accessors and predicates that expand inline in call position (`first`, `rest`, `nth`, `second`..`fourth`, `1+`, `1-`, `zerop`, `plusp`, `minusp`, `evenp`, `oddp`) are listed under [Functions](functions.md) because they are also usable as function values (`#'first`). --- # FILE: references/reference/macros/and.md # and `(and expr1 expr2...)` Evaluates its expressions left to right, short-circuiting as soon as one returns nil and yielding that nil; if every expression is non-nil it returns the value of the last one. `(and)` with no arguments returns `t`. It expands into nested `if` forms, so later expressions are not evaluated once a nil is found. ```lisp (and 1 2 3) ; => 3 ``` --- # FILE: references/reference/macros/assert.md # assert `(assert test-form [(place...) [datum args...]])` Evaluates `test-form` and signals an error when it is false; returns nil when it is true. The optional datum and arguments work like `error`'s control string and arguments and replace the default "The assertion ... failed." message. This is the lite version of Common Lisp's `assert`: it establishes no `continue` restart, so the places list is accepted but ignored (no interactive re-store loop). ```lisp (let ((x 1)) (assert (> x 0)) x) ; => 1 ``` A failing assertion aborts execution, so it is shown statically: ```console (let ((x 0)) (assert (> x 0) (x) "x must be positive, got ~a" x)) ; error: x must be positive, got 0 ``` --- # FILE: references/reference/macros/block.md # block `(block name body...)` Establishes a named block around `body` and returns the value of the last form, or the value thrown by a matching `(return-from name value)` fired during the body's execution. Names are matched on every backend: an inner `return-from` targeting an outer block passes through intervening blocks and loops, and `(block nil ...)` additionally catches plain `(return ...)` like the loop macros' implicit nil block. The match is LEXICAL on every backend — a `return-from` (or a plain `return`) inside a closure exits the block that encloses it in the SOURCE, so a `handler-bind` handler written inside `(block nil ...)` exits that block, not whichever same-named block is running where the condition was signalled. Exiting a block whose activation has already returned is an error. ```lisp (block scan (dotimes (i 10) (when (= i 4) (return-from scan (* i 100)))) :fell-through) ; => 400 ``` ```lisp (block nil (return 7) 9) ; => 7 ``` --- # FILE: references/reference/macros/case.md # case `(case key (k1 body...) ((k2 k3) body...) (otherwise body...))` Evaluates `key` once and compares it with `eql` against each clause's unevaluated key(s), running the body of the first match and returning its last value. A clause whose key is a list matches if `key` is `eql` to any element of that list. A final `t` or `otherwise` clause is the default; if nothing matches and there is no default, `case` returns nil. ```lisp (let ((x 2)) (case x (1 'one) ((2 3) 'two-or-three) (otherwise 'other))) ; => TWO-OR-THREE ``` --- # FILE: references/reference/macros/ccase.md # ccase `(ccase key (k1 body...) ...)` Like `ecase`, `ccase` dispatches on `key` with `eql` and signals an `error` when no clause matches. In full Common Lisp `ccase` is *correctable* -- it offers a restart to supply a new value -- but rontolisp's `ccase` establishes no `store-value` restart, so it behaves identically to `ecase` and is provided mainly for source compatibility. ```lisp (let ((x 1)) (ccase x (1 'one) (2 'two))) ; => ONE ``` --- # FILE: references/reference/macros/cerror.md # cerror `(cerror continue-format-control datum arg...)` Signals a **continuable** error like [`error`](error.md), with the same condition-designator surface: `datum` is a format control string (with `arg...` as format arguments) or a condition class name (with `arg...` as initargs). A `continue` restart described by `continue-format-control` is established around the signal, so a [`handler-bind`](handler-bind.md) handler — or anything else running at the signal point — can call [`continue`](../functions/continue.md) (or `invoke-restart` the `continue` restart) to make the `cerror` return `nil` and execution resume past it. When nothing invokes the restart, `cerror` behaves exactly like `error`: an uncaught one aborts, an enclosing [`handler-case`](handler-case.md) catches it. ```lisp (handler-bind ((error (lambda (c) (continue)))) (list :after (cerror "Ignore the error." "bad value: ~a" 42))) ; => (:AFTER NIL) ``` Uncaught, it aborts like `error` (shown statically): ```console (cerror "Ignore the error." "bad value: ~a" 42) (cerror "Skip this character." 'bad-input :position 7) ``` --- # FILE: references/reference/macros/change-class.md # change-class `(change-class instance 'class-name initarg value ...)` Changes the class of an existing instance **in place** and returns it: the object keeps its identity (every other reference sees the change), the slots the old and the new class share keep their values, the slots the new class adds are filled from their `:initform`s, and any supplied initargs are stored on top. The class argument is a class designator: a literal quoted name, a runtime symbol, or a class metaobject — `(change-class obj (find-class 'c))` works, as does a class name held in a variable. Out of scope: the MOP protocol around it (`update-instance-for-different-class` is never called), and changing between classes of unrelated inheritance chains keeps the slot values positionally instead of matching them by name. ```lisp (defclass cc-connection () ((host :initarg :host :accessor cc-host))) (defclass cc-pooled (cc-connection) ((kind :initarg :kind :accessor cc-kind :initform :none))) (let* ((c (make-instance 'cc-connection :host "db")) (alias c)) (change-class c 'cc-pooled :kind :shared) (list (type-of alias) (cc-host alias) (cc-kind alias))) ; => (CC-POOLED "db" :SHARED) ``` --- # FILE: references/reference/macros/check-type.md # check-type `(check-type place typespec [string])` Evaluates `place` and signals an error when the value is not of the given type; returns nil when it is. This is the lite version of Common Lisp's `check-type`: it establishes no `store-value` restart, so the error cannot be corrected interactively and the place is never re-stored. Supported type specifiers are the `typecase` type names (`integer`, `float`, `number`, `rational`, `string`, `symbol`, `keyword`, `cons`, `list`, `null`, `atom`, `character`, `hash-table`, `boolean`) plus the compound forms `(or ...)`, `(and ...)`, `(not ...)`, `(member item...)`, `(eql object)`, `(satisfies function)`, and ranged numeric types like `(integer 0 9)` (`*` means unbounded and a `(bound)` list is exclusive). The optional string replaces the "of type ..." part of the error message. ```lisp (let ((n 5)) (check-type n (integer 0 9)) n) ; => 5 ``` A failing check aborts execution, so it is shown statically: ```console (let ((n 12)) (check-type n (integer 0 9))) ; error: The value of n is 12, which is not of type (integer 0 9). ``` --- # FILE: references/reference/macros/complement.md # complement `(complement function)` Returns a one-argument predicate answering the opposite of `function`: the result is `t` where `function` returns `nil` and vice versa. Lite: unlike Common Lisp the returned function takes exactly one argument, and `complement` expands inline so `#'complement` is not available. ```lisp (funcall (complement #'evenp) 3) ; => T ``` ```lisp (remove-if (complement #'oddp) '(1 2 3 4 5)) ; => (1 3 5) ``` --- # FILE: references/reference/macros/complex.md # complex `(complex real &optional imaginary)` Lite: there is no complex number representation, so a zero (or omitted) imaginary part yields the real part and anything else signals an error. This keeps sources whose complex branch is never taken — like parse-number's `#C(...)` parser — loadable on every backend. ```lisp (complex 9 0) ; => 9 ``` ```console > (complex 1 2) Error: complex numbers are not supported (imaginary part 2) ``` --- # FILE: references/reference/macros/cond.md # cond `(cond (test body...)...)` Evaluates each clause's `test` in order and, for the first one that is truthy (non-nil), evaluates that clause's body forms and returns the value of the last. A clause with no body returns the value of its own test. If no test is truthy `cond` returns nil; a final `(t body...)` clause acts as a catch-all default. It expands into nested `if` forms. ```lisp (let ((x 5)) (cond ((< x 0) 'neg) ((= x 0) 'zero) (t 'pos))) ; => POS ``` --- # FILE: references/reference/macros/ctypecase.md # ctypecase `(ctypecase x (integer body...) (string body...))` The exhaustive variant of `typecase`: it dispatches on the type of `x` over the same set of type specifiers, but has no default clause. If `x` matches none of the clauses, `ctypecase` signals an `error` instead of returning nil, making it the right choice when every expected type should be handled explicitly. In full Common Lisp `ctypecase` is *correctable* -- it offers a `store-value` restart to supply a new value -- but rontolisp's `ctypecase` establishes no `store-value` restart, so it behaves identically to `etypecase` and is provided mainly for source compatibility. ```lisp (let ((x 42)) (ctypecase x (integer 'int) (string 'str))) ; => INT ``` --- # FILE: references/reference/macros/decf.md # decf `(decf place [delta])` Decrements the number stored in `place` by `delta` (default `1`), stores the result back into `place`, and returns the new value. `place` may be any location `setf` accepts. It expands into `(setf place (- place delta))`. ```lisp (let ((x 5)) (decf x)) ; => 4 ``` --- # FILE: references/reference/macros/declaim.md # declaim `(declaim declaration...)` File-level declarations, parsed as a no-op like `declare`: the form evaluates to nil and the declarations (`optimize`, `inline`, `type`, `special`, ...) are neither evaluated nor validated. It exists so library sources using `declaim` load unchanged. ```lisp (declaim (optimize (speed 3) (safety 1))) ; => NIL ``` --- # FILE: references/reference/macros/declare.md # declare `(declare declaration...)` Declarations never change what a program computes: the whole `declare` form evaluates to nil and its arguments are never evaluated or validated, so any standard declaration (`ignore`, `ignorable`, `type`, `optimize`, `inline`, `special`, ...) is accepted anywhere in a body, and source code written for other Common Lisp implementations loads unchanged. Two declaration families do affect compilation. A `(declare (special ...))` marks a variable dynamically scoped, as in standard CL. And on the WASM backend, a `type` declaration that names an array type -- `(simple-array (unsigned-byte 8) (*))`, `simple-vector`, `simple-string`, ... -- lets the compiler emit that one representation's element accessors directly, which makes the compiled module smaller and faster. Results are unaffected by a *correct* declaration; a *false* one, which is undefined behavior in Common Lisp, traps at the access on WASM while the other backends continue to ignore it. ```lisp (let ((x 10)) (declare (type integer x) (optimize (speed 3))) (* x 2)) ; => 20 ``` --- # FILE: references/reference/macros/define-compiler-macro.md # define-compiler-macro `(define-compiler-macro name lambda-list body...)` Defines a compiler macro for `name`: every later call to that function is rewritten by running `body` over the unevaluated argument forms, on all four backends. The definition itself is consumed (it produces no code) and the ordinary function definition stays in place for calls the macro declines to rewrite -- and for `apply`/`funcall`, which never consult a compiler macro. Returning the `&whole` parameter unchanged is the standard way to decline; a `defmacro` of the same name wins over the compiler macro, as in Common Lisp. A compiler macro is a hint, and Common Lisp lets an implementation ignore one. rontolisp uses that permission in three cases, all silent: the body signals (the call is left alone), `name` is a standard operator (never registered -- the shared expander lowers those before a compiler macro could see them), or the lambda list is one the macro machinery cannot bind. Limitations: `notinline` is not implemented, so a call declared `notinline` is still rewritten; the rewrite happens at most once per call site; and any output the body produces at expansion time is suppressed, so it stays identical across backends. ```lisp (defun myinc (x) (+ x 1)) (define-compiler-macro myinc (x) `(+ ,x 100)) (myinc 10) ; => 110 ``` ```lisp (defun mydec (x) (- x 1)) (define-compiler-macro mydec (&whole form x) (declare (ignore x)) form) ; declines (mydec 10) ; => 9 ``` --- # FILE: references/reference/macros/define-condition.md # define-condition `(define-condition name (parent...) (slot...) option...)` Defines a condition type as a CLOS-subset class (see [`defclass`](../special-forms/defclass.md)) over the built-in condition hierarchy `condition` > `serious-condition` > `error` (> `simple-error` and the standard error subtypes) and `warning`. The parent defaults to `condition`; with several parents, the **first** provides the slot layout (single inheritance) and the rest join the type hierarchy for `typep`/`typecase`/`handler-case` matching only (their slots are not inherited). Slots use the `defclass` subset (`:initarg`/`:initform`/`:reader`/`:accessor`, plus `:documentation`, which is dropped). Of the class options, `(:report x)` — a literal string or a `(lambda (condition stream) ...)` — is the condition's REPORT: it is the message when the condition is signaled by [`error`](error.md)/[`signal`](signal.md)/[`warn`](warn.md) **and** the text [`princ`](../functions/princ.md), [`princ-to-string`](../functions/princ-to-string.md) and [`format`](format.md)'s `~A` write for the condition object. [`prin1`](../functions/prin1.md) / `~S` are unaffected and keep the `#` instance syntax. A type that defines no `:report` inherits its parent's; a type that inherits none but carries the `simple-condition` slots (`format-control`/`format-arguments`, i.e. any subtype of `simple-error`/`simple-warning`/`simple-condition`) reports through `format` applied to them; a type with neither keeps the `#<...>` rendering under `princ` too. A [`print-object`](../functions/print-object.md) method on the type wins over the report, for both escape modes. `(:default-initargs :initarg value ...)` forwards to the generated class (defaults applied by `make-condition`/typed `error` for initargs not supplied), and `(:documentation ...)` is dropped. Returns the type name. On the compile path it is a top-level-only form, like `defclass`. ```lisp (define-condition my-parse-error (error) ((input :initarg :input :reader my-parse-error-input)) (:report "input did not parse")) ; => MY-PARSE-ERROR ``` The report is what `princ`/`~A` prints; `prin1`/`~S` still shows the instance: ```lisp (define-condition dc-report-demo (error) ((input :initarg :input :reader dc-report-demo-input)) (:report (lambda (c s) (format s "did not parse: ~a" (dc-report-demo-input c))))) (list (princ-to-string (make-condition 'dc-report-demo :input "x")) (prin1-to-string (make-condition 'dc-report-demo :input "x"))) ; => ("did not parse: x" "#") ``` ```console > (error 'my-parse-error :input "x") Error: input did not parse ``` --- # FILE: references/reference/macros/define-modify-macro.md # define-modify-macro `(define-modify-macro name (parameter...) function [documentation])` Defines a macro `name` so that `(name place argument...)` expands to `(setf place (function place argument...))` -- a read-modify-write of `place` through `function`. A trailing `&rest` parameter is spliced into the call. Lite: the `place` subforms may be evaluated more than once (no `get-setf-expansion` single-evaluation protocol) and the optional documentation string is ignored. ```lisp (define-modify-macro maxf (lo) max) ; => MAXF ``` --- # FILE: references/reference/macros/define-setf-expander.md # define-setf-expander `(define-setf-expander name lambda-list body...)` Defines how `(setf (name args...) value)` expands. The body (a macro-style form builder, usually with backquote) is run at expansion time with the lambda list bound to the place's argument forms, and must return the five setf-expansion values with [`values`](../functions/values.md): the temporary variables, their value forms, the store variables, the store form, and the access form. An `&environment` parameter is accepted and bound to nil, and [`get-setf-expansion`](../functions/get-setf-expansion.md) is available to expand a sub-place. Works on every backend (the compilers run the expander at compile time). `setf` template symbols resolve in the defining package, like a [`defmacro`](../special-forms/defmacro.md). ```lisp (defun my-first (x) (first x)) (define-setf-expander my-first (place) (let ((store (gensym))) (values '() '() (list store) `(progn (rplaca ,place ,store) ,store) `(my-first ,place)))) (let ((lst (list 1 2 3))) (setf (my-first lst) 99) lst) ; => (99 2 3) ``` --- # FILE: references/reference/macros/defsetf.md # defsetf `(defsetf access-fn update-fn)` Registers a `setf` expansion for `access-fn`. The **short form** above makes `(setf (access-fn args...) value)` expand to `(update-fn args... value)`. The **long form** `(defsetf access-fn (lambda-list) (store-var...) body...)` evaluates its body at expansion time -- the lambda list bound to temporaries for the argument forms and the store variables bound to the new-value temporaries -- and must return the store form. For the full five-value protocol use [`define-setf-expander`](define-setf-expander.md). Works on every backend. ```lisp (defun ref-value (box) (car box)) (defsetf ref-value rplaca) (let ((box (list 1))) (setf (ref-value box) 42) box) ; => (42) ``` --- # FILE: references/reference/macros/deftype.md # deftype `(deftype name lambda-list body...)` A **zero-parameter** `deftype` whose body is a literal (quoted) type specifier is registered, so the defined name resolves as a type in later [`typep`](typep.md)/[`typecase`](typecase.md) tests (a `(satisfies predicate)` body calls the named predicate; the name may itself expand to another type name). A **parameterized** or otherwise computed `deftype` stays a parsed no-op returning `nil` — there is no per-call expansion, so its name is not resolvable, which supports the common library shape where such a name only appears inside (equally no-op) `declaim`/`declare` declarations. ```lisp (deftype my-even () '(satisfies evenp)) (list (typep 4 'my-even) (typep 3 'my-even)) ; => (T NIL) ``` ```lisp (deftype array-index (&optional (length 1000)) `(integer 0 (,length))) ; => NIL ``` --- # FILE: references/reference/macros/destructuring-bind.md # destructuring-bind `(destructuring-bind pattern form body...)` Binds the variables of `pattern` to the corresponding parts of the value of `form` and evaluates the body. The pattern is a macro-style lambda list: patterns nest in required positions, and `&optional` (with defaults and supplied-p), `&rest`/`&body`, `&key` (with defaults and supplied-p), and `&aux` are supported -- also inside nested patterns. A dotted tail is shorthand for `&rest` (`((a &rest b) . rest)` binds `rest` to everything past the first element). `&whole` (as the pattern's first element) binds its variable to the whole source list; `&environment` is not supported. Matching is lenient: a missing position binds to nil and surplus elements are ignored (no mismatch error); only an undeclared keyword under `&key` signals, unless `&allow-other-keys` is given. ```lisp (destructuring-bind (a (b c) &optional (d 10)) '(1 (2 3)) (list a b c d)) ; => (1 2 3 10) ``` ```lisp (destructuring-bind (name &key (size 1) color) '(box :color red) (list name size color)) ; => (BOX 1 RED) ``` --- # FILE: references/reference/macros/do-external-symbols.md # do-external-symbols `(do-external-symbols (var [package [result]]) body...)` Evaluates the body once per external (exported) symbol of `package` -- the current package when omitted -- with `var` bound to the symbol, then evaluates `result` with `var` bound to nil and returns its value (nil when no result form is given). The symbols come in sorted order. This is an **interpreter-only** operator: the compiled backends carry no package registry at run time, so a call reaching them is a compile error. Inside a `#.` read-time form it works everywhere, because the macro-time evaluator resolves it before compilation. ```lisp (let ((names nil)) (do-external-symbols (s :rontolisp names) (push (symbol-name s) names)) (length names)) ; => 98 ``` --- # FILE: references/reference/macros/do-star.md # do* `(do* ((var init step?)...) (end-test result...) body...)` Like `do`, but the bindings and steps are sequential (`let*`-style) rather than parallel: each `init` form sees the variables bound earlier in the same list, and each `step` form sees the variables already updated this iteration. Otherwise it behaves identically -- `end-test` is checked before each pass, the `result` forms supply the return value, and `return` exits early. In the example below `j` is initialised from the freshly bound `i`, which a plain `do` could not do. ```lisp (do* ((i 5) (j i)) (t (list i j))) ; => (5 5) ``` --- # FILE: references/reference/macros/do-symbols.md # do-symbols `(do-symbols (var [package [result]]) body...)` Evaluates the body once per symbol ACCESSIBLE in `package` -- the current package when omitted -- with `var` bound to the symbol, then evaluates `result` with `var` bound to nil and returns its value (nil when no result form is given). Accessible means the symbols the package owns, internal and external alike, plus the exports it inherits through its use list; each is spelled against the package that owns it, so a package using `cl` yields the bare `cl` names. The symbols come in sorted order. This is an **interpreter-only** operator, like [`do-external-symbols`](do-external-symbols.md): the compiled backends carry no package registry at run time, so a call reaching them is a compile error. Inside a `#.` read-time form it works everywhere, because the macro-time evaluator resolves it before compilation. ```lisp (let ((n 0)) (do-symbols (s :rontolisp n) (setq n (1+ n)))) ; => 102 ``` --- # FILE: references/reference/macros/do.md # do `(do ((var init step?)...) (end-test result...) body...)` Iterates with one or more loop variables. Each `var` is bound to its `init` on entry and, after every pass through the body, updated *in parallel* to its `step` form (every step is evaluated against the old bindings before any variable is reassigned). Before each iteration `end-test` is checked; when it becomes truthy the loop stops and the `result` forms are evaluated, the last value being returned (nil if there are none). The body and steps are wrapped in the internal block boundary, so `return` exits the loop early. ```lisp (do ((i 0 (+ i 1)) (sum 0 (+ sum i))) ((= i 5) sum)) ; => 10 ``` --- # FILE: references/reference/macros/documentation.md # documentation `(documentation name doc-type)` / `(setf (documentation name doc-type) docstring)` Lite: docstrings are not stored anywhere, so reading returns `nil` and the `setf` form evaluates to the docstring while discarding it. Accepted so libraries that attach documentation at load time (`(setf (documentation 'f 'function) "...")`) load unchanged. ```lisp (defun greet () "hi") (setf (documentation 'greet 'function) "Says hi.") ; => "Says hi." (documentation 'greet 'function) ; => NIL ``` --- # FILE: references/reference/macros/dolist.md # dolist `(dolist (var list [result]) body...)` Evaluates `list` once and runs the body repeatedly with `var` bound to each successive element. After the list is exhausted, `var` is bound to nil and the optional `result` form is evaluated and returned (nil if omitted). The body is wrapped in the internal block boundary, so `return` inside it exits the loop early. ```lisp (dolist (x '(a b c)) (format t "~a~%" x)) ``` ``` A B C ``` --- # FILE: references/reference/macros/dotimes.md # dotimes `(dotimes (var count [result]) body...)` Evaluates `count` once, then runs the body repeatedly with `var` bound to the successive integers `0` through `count-1`. After the loop finishes, `var` is bound to `count` and the optional `result` form is evaluated and returned (nil if omitted). It expands into a counting loop wrapped in the internal block boundary, so `return` inside the body exits the loop. ```lisp (dotimes (i 3) (format t "~d~%" i)) ``` ``` 0 1 2 ``` --- # FILE: references/reference/macros/ecase.md # ecase `(ecase key (k1 body...) ((k2 k3) body...))` The exhaustive variant of `case`: it dispatches on `key` with `eql` exactly like `case`, but has no default clause -- `t` and `otherwise` are treated as ordinary keys. If `key` matches no clause, `ecase` signals an `error` rather than returning nil, so it is used when every valid value should be covered explicitly. ```lisp (let ((x 3)) (ecase x (1 'one) ((2 3) 'two-or-three))) ; => TWO-OR-THREE ``` --- # FILE: references/reference/macros/error.md # error `(error datum args...)` Signals an error, aborting execution unless an enclosing [`handler-case`](handler-case.md) catches it. The Common Lisp condition designators are supported: - `(error "control" args...)` — the control string uses the same directives as `format` (`~a`, `~s`, `~%`, ...); the remaining arguments fill them to build the message. The control may equally be computed (a variable, a slot read, a function call): a datum that is a string at runtime is a format control and the arguments after it are its format arguments, exactly as for a literal one. - `(error 'type :initarg value ...)` — constructs a condition instance of the named class (built-in or defined by [`define-condition`](define-condition.md)) and signals it. The message is the class's `:report` rendering -- its own, or the nearest ancestor's when it defines none -- and, for a class with no report anywhere but with the `simple-condition` family's slots, `format` applied to its `:format-control` over its `:format-arguments`. A class that reports nothing at all keeps the `Condition (type initargs...) was signalled.` shape. The message is exactly the text [`princ`](../functions/princ.md) writes for the same condition object (see [`define-condition`](define-condition.md)). - `(error obj)` — signals a pre-built condition object (e.g. from [`make-condition`](make-condition.md)); a value that turns out to be a string at runtime is signaled as the message it renders (see the first bullet). Every backend raises the condition so `handler-case` can dispatch on its type: the interpreter and JVM throw an exception carrying the message and the condition object, and the wasm-GC backends throw a WebAssembly exception when the program contains a catching form (`handler-case`/`ignore-errors`/`unwind-protect`; the emitted module then needs `wasmtime -W exceptions=y`) and trap otherwise. `#'error` IS a function value: the interpreter keeps the full designator protocol through it (`(apply #'error (list 'my-error :v x))` builds the typed condition), while the compiled backends forward the datum only -- a symbol datum signals a plain condition naming the class (still caught by `handler-case`'s `error` clause), and trailing initargs/format arguments are dropped. When nothing catches it, the condition reports itself as **one line on standard error** — `Unhandled condition: ` followed by the same text [`princ`](../functions/princ.md) writes for it — and the process exits non-zero. That line is identical on all four backends; what follows it is only the host's own note that the process died (the JVM's `Exception in thread "main" ...`, wasmtime's trap report). Setting the `RONTOLISP_DEBUG` environment variable to any value additionally prints the JVM stack trace, on the interpreter and on a compiled `.class`. The wasm-GC backends report only when the module carries the exception machinery — that is, when the program contains a catching form somewhere, as any program that loads a library does; without one the module traps with no message, which is what keeps a program that never signals from paying for the machinery. A `--no-wasi` reactor has no standard error to write to at all, so there the report goes into its discarding output sink (see the [wasm-GC module guide](../../guides/wasm-gc-module.md#what-the-build-tells-you-before-you-run-it)). Because an uncaught `error` aborts execution it is shown here statically rather than as a runnable example: ```console (error "bad value: ~a" x) (error 'type-error :datum x :expected-type 'integer) ``` --- # FILE: references/reference/macros/etypecase.md # etypecase `(etypecase x (integer body...) (string body...))` The exhaustive variant of `typecase`: it dispatches on the type of `x` over the same set of type specifiers, but has no default clause. If `x` matches none of the clauses, `etypecase` signals an `error` instead of returning nil, making it the right choice when every expected type should be handled explicitly. ```lisp (let ((x 42)) (etypecase x (integer 'int) (string 'str))) ; => INT ``` --- # FILE: references/reference/macros/eval-when.md # eval-when `(eval-when (situation...) body...)` Evaluates the body as a `progn`, treating every situation (`:compile-toplevel`, `:load-toplevel`, `:execute`, and the deprecated `compile`/`load`/`eval` spellings) as "evaluate now". The situation list is required but otherwise ignored, so code that runs under real Common Lisp situation rules also runs here. At top level the body forms are spliced into the surrounding program on the compile path, so the common macro-exporting idiom -- a `defmacro` wrapped in `(eval-when (:compile-toplevel :load-toplevel :execute) ...)` -- defines the macro for the rest of the compilation unit, and nested `defun`s are collected like ordinary top-level definitions. ```lisp (progn (eval-when (:compile-toplevel :load-toplevel :execute) (defun ew-double (x) (* x 2))) (ew-double 21)) ; => 42 ``` --- # FILE: references/reference/macros/flet.md # flet `(flet ((name lambda-list body...)...) body...)` Binds local functions for the body. Lisp-2: each name lives in the function namespace only, so it is reached from call position (`(name args...)`) and via `#'name`, while a bare `name` remains a variable reference. The definitions are NOT visible to each other or to themselves -- a definition body sees the outer function of the same name (use [`labels`](labels.md) for recursion). Lambda lists support the same extensions as `defun` (`&optional`/`&rest`/`&key`/`&aux`). Expands into `let`-bound lambdas plus a body rewrite, so a local function is an ordinary closure: it can capture surrounding variables and be passed to `mapcar`/`funcall`/`apply` via `#'name`. ```lisp (flet ((sq (x) (* x x)) (dbl (x) (* 2 x))) (list (sq (dbl 3)) (mapcar #'sq '(1 2 3)))) ; => (36 (1 4 9)) ``` --- # FILE: references/reference/macros/format.md # format `(format destination control-string args...)` A subset of Common Lisp's `format`, implemented as a macro shared by the interpreter and both compilers. A literal `control-string` is expanded at compile time; a computed one is rendered at run time (see [Runtime control strings](#runtime-control-strings)), with the same directives either way. With destination `t` the form expands into `princ`/`prin1`/`terpri` calls, writes to standard output, and returns nil; with destination `nil` it builds and returns the formatted string (expanding into `princ-to-string`/`prin1-to-string` calls folded with the internal string concatenation); with any other destination expression it builds the string the same way and then DISPATCHES on the value at run time -- a stream is written with one `write-string` call and nil is returned (a `with-output-to-string` string stream or a file stream), a `t` value writes to `*standard-output*`, and a nil value returns the string. That test has to happen at run time because nil does not name a stream: it is the "return the string" destination, so a function that forwards its own `&optional stream` argument (`(defun render (x &optional stream) (format stream ...))`, the Common Lisp convention) answers a string when called without one. All arguments are evaluated left to right before any output. ```lisp (format t "Hello ~a, you are ~d!~%" 'world 42) ``` ``` Hello WORLD, you are 42! ``` With destination `nil` the result is returned as a string instead of printed: ```lisp (format nil "~a+~a=~a" 1 2 3) ; => "1+2=3" ``` ## Directives | Directive | Meaning | |-----------|---------| | `~a`, `~A` | Aesthetic: prints the argument like `princ` (strings without quotes). With `:`, nil prints as `()` | | `~s`, `~S` | Standard: prints the argument like `prin1` (readable; a string is quoted and its embedded `"` / `\` escaped). With `:`, nil prints as `()` | | `~w`, `~W` | Write: prints the argument like `write` -- `prin1` under the printer control variables. It takes no prefix parameters, and its modifiers bind variables the printer does not honor (`~:W` binds `*print-pretty*`, `~@W` unbinds `*print-level*`/`*print-length*`), so all three spellings print the same text | | `~d`, `~D` | Decimal integer. With `:`, digits are grouped with commas; with `@`, a `+` sign precedes non-negative values | | `~x`, `~o`, `~b` | Hexadecimal / octal / binary integer (uppercase digits), with the same parameters and modifiers as `~d` | | `~R` | Radix: `~NR` prints the integer in radix `N` (2-36). Without the radix parameter the decimal digits are printed (English cardinal/ordinal output is not implemented) | | `~c`, `~C` | Character: prints the glyph like `write-char`. With `@`, the `#\` reader syntax (like `prin1`); with `:`, non-graphic characters print their name (`Newline`, `Space`, ...) | | `~f`, `~F` | Fixed-format floating point. `~,Df` prints `D` digits after the decimal point (rounded); with `@`, a leading `+`. Full parameters: `~w,d,k,overflowchar,padchar F` | | `~e`, `~E` | Exponential (scientific) floating point: `[-]d.ddde[+/-]xx`. `~,De` prints `D` digits after the decimal point (default 6, rounded); with `@`, a leading `+`. Full parameters: `~w,d,e,k,overflowchar,padchar,exponentchar E` (`k` must be 1) | | `~g`, `~G` | General floating point: the plain float representation for magnitudes in `[0.1, 1e16)` (and zero), the `~e` form otherwise | | `~$` | Monetary: `~D$` prints `D` digits after the decimal point (default 2); with `@`, a leading `+` | | `~%` | Newline (one, or the count given by a prefix parameter) | | `~&` | Fresh line: a newline only if not already at the start of a line | | `~~` | A literal `~` | | `~(str~)` | Case conversion of the processed `str`: downcase; `~:(` capitalizes every word, `~@(` capitalizes only the first word, `~:@(` upcases | | `~[str0~;str1~:;default~]` | Conditional: the argument (an integer) selects a clause; `~:;` introduces the default. `~N[` / `~#[` select by a literal / by the number of remaining arguments; `~:[false~;true~]` tests nil; `~@[str~]` processes `str` (re-using the tested argument) only when it is non-nil | | `~{str~}` | Iteration: applies `str` repeatedly to the elements of the list argument. `~N{` caps the passes at `N`; `~:{` iterates over a list of sublists; `~@{` iterates over the remaining arguments; `~:@{` treats each remaining argument as a sublist | | `~?` | Recursive format: consumes a control string and a list of its arguments, rendered through the runtime renderer. `~@?` takes the inner control's arguments from the remaining arguments instead of a list | | `~*` | Argument jump: `~N*` skips `N` arguments (default 1), `~N:*` moves back `N`, `~N@*` jumps to argument `N` (default 0) | Directives accept prefix parameters (written after the `~`, comma-separated) and the `:`/`@` modifiers. A parameter is a decimal number, a character (`'c`), `v` (consume an argument and use its value), or `#` (the number of remaining arguments). Field directives (`~a`/`~s`/`~d`/`~x`/`~o`/`~b`/`~f`/`~e`/`~$`) take a leading minimum-width parameter; text shorter than the width is padded (with the pad-character parameter -- a `'c` literal or a runtime `v` -- space by default). `~a`/`~s` pad on the right (left with `@`); numbers pad on the left. ```lisp (format t "Hello ~a, you are ~d years old.~%" 'world 42) (format t "~:d and ~@d~%" 1000000 42) (format t "~,2f and ~$~%" 3.14159 3.14159) (format t "~e and ~,4e~%" 1234.5 pi) (format t "~10a|~5,'0d|~%" "foo" 42) (format t "~w and ~a~%" "str" "str") (princ (format nil "Hello ~a!" 'world)) (terpri) (format t "~x ~o ~b ~8r~%" 255 64 5 4096) (format t "~c ~@c ~:c~%" #\a #\b #\Newline) (format t "~(~a~) ~:(~a~)~%" "FOO BAR" "foo bar") (format t "~[zero~;one~:;many~] ~:[no~;yes~] ~@[x=~a~]~%" 1 t 42) (format t "~{<~a>~} ~:{(~a,~a)~}~@{ ~a~}~%" '(1 2) '((x 1) (y 2)) 'a 'b) (format t "~{~a~^, ~}~%" '(1 2 3)) (format t "~a ~:* ~a~%" 1) ``` ``` Hello WORLD, you are 42 years old. 1,000,000 and +42 3.14 and 3.14 1.2345e+3 and 3.1416e+0 foo |00042| "str" and str Hello WORLD! FF 100 101 10000 a #\b Newline foo bar Foo Bar one yes x=42 <1><2> (X,1)(Y,2) A B 1, 2, 3 1 1 ``` ## Limitations Other destinations (strings with fill pointers) are not supported. The loop escape `~^` is supported at the top level and inside `~{ ... ~}` / `~@{ ... ~}` bodies (the join idiom `"~{~a~^, ~}"`; inside `~:{ ... ~}` it ends the current sublist's body), but its `~:^`/`~@^` variants and prefix parameters are not. Further notes: - A `~f` (and the fixed branch of `~g`) without a digit count falls back to the free-format float printing (the shortest round-trip decimal, identical on every backend), which may use exponent notation where `~f` with a digit count never would; supply a digit count for a fixed decimal layout. `~g` accepts no prefix parameters. - `~e` builds its mantissa from integer arithmetic (so the output is identical on every backend) and the digit count must be a literal, not a runtime `v`. The scaled mantissa is computed in 64-bit arithmetic, which limits `~,De` to roughly `D` <= 17 digits of precision (identically on every backend); the default (`~e`, 6 digits) is well within that bound. The scale factor of `~e` must be 1 (the default), and the overflow character of `~f`/`~e` requires a literal width. - The repeat count of `~%`/`~&`/`~~` must be a literal or `#` (a runtime `v` count there is not supported). `~&` decides whether to emit a newline from the actual output column for destination `t`, but from the surrounding literal text (a static approximation) for destination `nil` and inside composite (`~(`/`~[`/`~{`) bodies. - Composite directives nest freely: a `~[` conditional may hold another `~[` (or a `~{ ... ~}` iteration) in any of its clauses. When a runtime-selected `~[` has clauses that consume DIFFERENT numbers of arguments, the rest of the control string is expanded once per clause so each branch continues from its own argument position, exactly as Common Lisp's argument pointer would. A branch that would need more arguments than were supplied signals only if it is actually selected. - Because `format` expands statically, a `~@[` clause must consume exactly the tested argument, `#` and `~@{` are not available inside a `~{ ... ~}` body, and an argument-divergent `~[` nested inside another composite directive (`~(`/`~{`) is not supported. - `~:d` grouping and the radix directives `~x`/`~o`/`~b`/`~r` are exact for integers of any magnitude on every backend. - `~w` prints as `prin1` and does not read `*print-escape*` / `*print-readably*`, so binding one of them around it does not switch it to `princ` -- the same gap `write-to-string` has (`write` itself honors them). ## Runtime control strings A control string that is a runtime value -- a computed control expression, a call through the function value `#'format` (`funcall`/`apply`), the inner control of `~?`, or a condition's `format-control` slot -- is rendered by a runtime renderer instead of being expanded statically. It understands the same directives as the table above, so the same control string and arguments produce the same text whichever way `format` reaches them (on every backend). Two differences follow from the control being data rather than source: - The renderer never signals: a malformed control (`"abc~"`), an unknown directive (`~Q`), an unterminated `~{`, and a missing argument render as text (the directive verbatim, `NIL` for the missing argument) rather than raising an error. A literal control reports the same problems at expansion time, where a diagnostic belongs; a runtime control usually arrives with the data being reported, and a report must not fail while reporting. - The column-control directive `~t` (`~n,mT`, `~n@T`), the plural directive `~p`, the logical-block / justification directive `~<...~>` and the call-a-function directive `~/name/` are available here but not in the literal expansion -- a literal control using one falls back to this renderer, so all four work either way. `~t` measures the column from the text rendered so far. `~<...~>` is justification and `~<...~:>` a logical block; the closing directive decides which. Their SECTION rules are the standard ones: a justification's `~;` segments consume arguments in turn, while a logical block's first section is the prefix and, when there are three, the last is the suffix (neither consumes an argument), and a block without `@` takes one argument -- a list -- as its whole argument list. What does not happen is the LAYOUT: no padding to a minimum column, no wrapping at the right margin, and of the conditional newlines only the mandatory `~:@_` breaks a line (`~_` / `~:_` / `~@_` and `~i` do nothing). Deciding the others needs the stream's current column, which no rontolisp stream carries -- the same reason `pprint-newline` only honors `:mandatory`. `~/name/` calls the named function as `(name stream object colon-p at-p)` and splices what it writes. The name is looked up as if by `find-symbol`, where a single and a double colon are equivalent, so `~/mypkg:helper/` reaches an internal symbol too. **Compiled output carries `~/name/` only when the compiler can see the directive.** Resolving a function out of a control string at run time means any function in the program can be reached by name, which is exactly what stops the compiler from removing unused code -- so it includes that part of the renderer only when some string literal in the program spells a `~/name/` directive (anywhere: the control at the call site, a control bound to a variable, a control inside a spliced library). That covers every ordinary use. A control string *assembled at run time* out of pieces that never spell the directive signals instead of rendering it, naming the reason; compile with `--dynamic` to keep the directive available unconditionally. The interpreter always supports it. ```lisp (defun brackets (stream x &optional colonp atp) (princ (if colonp "[" "<") stream) (princ x stream) (princ (if atp "]" ">") stream)) (list (format nil "~@" 1) (format nil "~<~@;~a-~a~:>" (list "x" "y")) (format nil "~/brackets/ ~:@/brackets/" 1 2)) ; => ("a and 1" "x-y" "<1> [2]") ``` `~r` without a radix parameter prints the decimal digits; English cardinals and ordinals are not implemented. Like the other macros, `format` is not recognized by the embedded `eval` runtime in compiled output (see [Compiled `eval` limitations](../../guides/eval-limitations.md)). --- # FILE: references/reference/macros/handler-bind.md # handler-bind `(handler-bind ((type handler)...) body...)` Evaluates `body...` with the handlers established. When a condition is signaled during the body — by [`error`](error.md), [`signal`](signal.md), [`warn`](warn.md) or [`cerror`](cerror.md) — each matching handler is called **at the signal point, before any unwinding**, with the condition object as its one argument. That is the difference from [`handler-case`](handler-case.md): the stack between the signaler and the handler is still intact, so the handler can [`invoke-restart`](../functions/invoke-restart.md) a restart established by a [`restart-case`](restart-case.md) *inside* the body and transfer control there. A handler that returns normally **declines**: the search continues with outer handlers, and an unhandled `error` then aborts (or is caught by an enclosing `handler-case`) exactly as if no `handler-bind` were present. The `type` is any `handler-case` clause type, including classes from [`define-condition`](define-condition.md) and the class a built-in error carries (`type-error`, `division-by-zero`, ... -- see [`handler-case`](handler-case.md)); the handler expressions are evaluated when the `handler-bind` is entered. Supported on every backend except `--no-gc`. A program using the restart system compiles in EH mode on the wasm-GC backends, so add `-W exceptions=y` to `wasmtime run`/`wasmtime serve`. Handlers also run for the errors **built-ins** raise (a `(car 5)`-style type error, an out-of-range `aref`, an undefined function): the interpreter runs them at the signal point like a signaled condition; the compiled backends run them when the error unwinds past the `handler-bind` itself, so restarts established *inside* the body are gone and intervening [`unwind-protect`](../special-forms/unwind-protect.md) cleanups have already run by then (a **signaled** condition keeps the exact signal-point semantics everywhere). On the wasm-GC backends a failure that traps instead of signaling (`(car 5)` compiles to a failed cast; so does integer division by zero) still ends the program without running handlers — only what rides the condition channel (signaled conditions, an undefined-function call) reaches them. ```lisp (handler-bind ((error (lambda (c) (invoke-restart :use-value 42)))) (restart-case (error "boom") (:use-value (v) (list :recovered v)))) ; => (:RECOVERED 42) ``` A handler that returns declines, and the error keeps propagating: ```lisp (let ((log nil)) (handler-case (handler-bind ((error (lambda (c) (setq log :seen)))) (error "boom")) (error (e) (list :caught log)))) ; => (:CAUGHT :SEEN) ``` Handlers run **most recent first**, and a [`handler-case`](handler-case.md) established inside the body is one of them: a condition it matches is handled there and the enclosing handler is never called. Only when no clause of the inner `handler-case` matches does the search reach the outer handler. ```lisp (block b (handler-bind ((error (lambda (e) (return-from b :outer-ran)))) (handler-case (error "boom") (error () :caught)))) ; => :CAUGHT ``` An error a built-in raises runs the handlers too — how a test framework turns a broken test body into a recorded failure instead of an aborted run: ```lisp (block b (handler-bind ((error (lambda (e) (return-from b :caught)))) (car 1))) ; => :CAUGHT ``` --- # FILE: references/reference/macros/handler-case.md # handler-case `(handler-case expression (type ([var]) body...)... [(:no-error ([var]) body...)])` Evaluates `expression`; when an error is signaled during it, control transfers to the first clause whose condition `type` matches the signaled condition, with `var` (optional) bound to the condition object, and the clause body's value becomes the value of the whole form. When no clause matches, the error propagates outward (an enclosing `handler-case` may still catch it). The clause type is any `typecase` specifier, including condition classes defined by [`define-condition`](define-condition.md) and the built-in hierarchy (`condition` > `serious-condition` > `error`, `warning`); an error signaled without a condition object is caught as the class its cause names, with the message in the condition's `format-control` slot: a plain `(error "...")` is a `simple-error`, while a failure inside a built-in carries its own type -- a bad `car`, a wrong argument type or an out-of-range index is a `type-error`, a zero divisor a `division-by-zero`, a call to an undefined function an `undefined-function`, a read of an unbound variable an `unbound-variable` (on the wasm-GC backends only the undefined-function case is reachable at all, and it is caught as a `simple-error` there -- see the divergence below). The `:no-error` clause runs on normal completion with `var` bound to the (primary) value, outside the handler. Non-local exits (`return`/`return-from`) pass through uncaught, and an `unwind-protect` inside the expression runs its cleanup before the handler. `handler-case` is supported on **every backend** except `--no-gc` (a compile error there). On the wasm-GC backends (Preview 1 and `--component`, including `wasmtime serve`) it compiles through the WebAssembly exception-handling proposal, so running a program that uses a catching form needs wasmtime 37+ with the proposal enabled: add `-W exceptions=y` to the usual `wasmtime run`/`wasmtime serve` flags. A program without catching forms is byte-identical to before and keeps its usual command line. Divergence: the WASM backends catch **signaled conditions only** — a runtime trap (a `(car 5)`-style type failure, integer division by zero) stays uncatchable there, while the interpreter and the JVM catch it as an error. Handlers are per thread of control, so concurrent `rontolisp:http-handler` requests do not interfere. To run a handler at the signal point *without* unwinding — e.g. to invoke a [`restart-case`](restart-case.md) restart — use [`handler-bind`](handler-bind.md). ```lisp (handler-case (error "boom") (error (e) (list :caught (simple-condition-format-control e)))) ; => (:CAUGHT "boom") ``` Typed conditions dispatch through the class hierarchy, first matching clause wins: ```lisp (define-condition low-fuel (warning) ((level :initarg :level :reader low-fuel-level))) (handler-case (error 'low-fuel :level 5) (error (e) :error) (warning (w) (list :warned (low-fuel-level w)))) ; => (:WARNED 5) ``` ```lisp (handler-case (+ 1 2) (error (e) :err) (:no-error (v) (list :ok v))) ; => (:OK 3) ``` An error a built-in raises dispatches on its class -- what a test framework's `(signals form 'type-error)` asserts. On the wasm-GC backends this particular failure traps instead (see the divergence above), so it is caught on the interpreter and the JVM: ```lisp (handler-case (car 1) (type-error (e) :type-error) (error (e) :plain)) ; => :TYPE-ERROR ``` --- # FILE: references/reference/macros/ignore-errors.md # ignore-errors `(ignore-errors form...)` Evaluates the forms and returns the value of the last one; when an error is signaled, returns nil instead (with the condition object as the syntactic-tier secondary value). Sugar over `(handler-case (progn form...) (error (c) (values nil c)))` — see [`handler-case`](handler-case.md). Like `handler-case` it is supported on **every backend** except `--no-gc`; on the wasm-GC backends the emitted module needs `wasmtime run -W exceptions=y` (37+), and runtime traps remain uncatchable there — see [`handler-case`](handler-case.md). ```lisp (ignore-errors (error "boom")) ; => NIL ``` ```lisp (ignore-errors (+ 1 2)) ; => 3 ``` --- # FILE: references/reference/macros/incf.md # incf `(incf place [delta])` Increments the number stored in `place` by `delta` (default `1`), stores the result back into `place`, and returns the new value. `place` may be any location `setf` accepts. It expands into `(setf place (+ place delta))`. ```lisp (let ((x 5)) (incf x 3)) ; => 8 ``` --- # FILE: references/reference/macros/labels.md # labels `(labels ((name lambda-list body...)...) body...)` Like [`flet`](flet.md) but the definitions see each other, so the local functions can call themselves and each other (recursion and mutual recursion). ```lisp (labels ((ev (n) (if (= n 0) t (od (- n 1)))) (od (n) (if (= n 0) nil (ev (- n 1))))) (list (ev 10) (od 9))) ; => (T T) ``` --- # FILE: references/reference/macros/let-star.md # let* `(let* ((var1 init1) (var2 init2)...) body...)` Like `let`, but the bindings are established sequentially: each `init` form can refer to the variables bound earlier in the same binding list. After all variables are bound the body forms are evaluated in order and the value of the last is returned. It expands into nested `let` forms, one per binding. ```lisp (let* ((x 1) (y (+ x 1))) (list x y)) ; => (1 2) ``` --- # FILE: references/reference/macros/load-time-value.md # load-time-value `(load-time-value form [read-only-p])` Evaluates `form` once per occurrence in the source, not once per use: the result is computed the first time that occurrence is reached and reused from then on. `read-only-p` is accepted and ignored. The compiled backends hoist the result into a generated global filled on first use; the interpreter memoizes the occurrence. Filling is lazy rather than at program start, so an occurrence never reached is never evaluated -- which matters because a value form spliced out of a library routinely needs globals that later top-level forms initialize. A value form cheap enough not to be worth a slot -- an atom, a variable read, or a `quote`/`function`/`find-package` wrapper -- keeps the plain lowering and is simply re-evaluated. The `--no-gc` backend never hoists. ```lisp (load-time-value (+ 1 2)) ; => 3 ``` ```lisp (defvar *n* 0) (defun bump () (setq *n* (+ *n* 1)) *n*) (defun probe () (load-time-value (bump))) (list (probe) (probe) (probe)) ; => (1 1 1) ``` --- # FILE: references/reference/macros/locally.md # locally `(locally declaration... form...)` Evaluates the body as a `progn`. Declarations are parsed no-ops everywhere in rontolisp ([`declare`](declare.md)/[`the`](the.md)), so `locally` simply drops its leading `declare` forms and evaluates the rest — code that uses `locally` to scope real Common Lisp declarations runs unchanged. ```lisp (locally (declare (optimize (speed 3))) (+ 40 2)) ; => 42 ``` --- # FILE: references/reference/macros/loop.md # loop `(loop clause...)` (extended) or `(loop form...)` (simple) A bounded subset of the ANSI `loop` macro. It expands to the existing iteration core (`do*`-style stepping wrapped in the internal block boundary), so it works identically on the interpreter and both compilers. If every top-level subform is a compound form, `loop` is a **simple loop**: it repeats those forms forever until a `return` exits it. ```lisp (let ((i 0)) (loop (setq i (+ i 1)) (when (= i 5) (return i)))) ; => 5 ``` Otherwise it is an **extended loop** built from clauses. The supported clauses are: - Numeric stepping: `for VAR from LO [to|upto|below|downto|above HI] [by STEP]` (also `upfrom`/`downfrom`; a limit keyword with no `from` starts at 0). - List stepping: `for VAR in LIST [by FN]` and `for VAR on LIST [by FN]` (`VAR` may be a destructuring pattern). - Sequence stepping: `for VAR across SEQ` binds `VAR` to each character of a string or each element of a vector in turn. - General stepping: `for VAR = INIT [then STEP]` (`VAR` may be a destructuring pattern). - Local variables: `with VAR [= INIT]` (`VAR` may be a destructuring pattern; `and`-joined `with` bindings are parallel). - Type declaration: `for`/`as`/`with` accept an optional type spec right after `VAR`, in either ANSI spelling — `of-type TYPE` or the simple bare `fixnum`/`float`/`t`/`nil` — before the rest of the clause. It is parsed and discarded: rontolisp's loop expansion is untyped, so the declaration carries no semantics. - Accumulation: `collect`, `append`, `nconc`, `sum`, `count`, `maximize`, `minimize`, each with an optional `into VAR`. - Termination tests: `thereis EXPR`, `always EXPR`, `never EXPR`. - Control: `while`/`until` (honoring their textual position), `repeat N`, `do FORM...`, `return EXPR`, `(loop-finish)` inside body forms, `initially FORM...`, `finally FORM...`, and the conditionals `when`/`if`/`unless` with optional `else` and `end` (the tested value is available as `it` in the selected clauses). Multiple `for` clauses step together, and the loop ends as soon as the shortest driver is exhausted — the idiomatic indexed map. Sequential clauses step in order (a later clause's init and step forms see the values the earlier clauses just produced, and stepping stops at the first exhausted driver, so `for x in xs for a = (f x) then (g a x)` works as in CL): ```lisp (loop for x in '(a b c) for i from 0 collect (list i x)) ; => ((0 A) (1 B) (2 C)) ``` `and` joins `for` clauses into one group whose inits and steps are computed against the previous iteration's values (like `do`'s parallel stepping versus `do*`): ```lisp (loop for a = 0 then b and b = 1 then (+ a b) repeat 8 collect b) ; => (1 1 2 3 5 8 13 21) ``` A `for` variable is **one binding stepped in place**, not a fresh binding per iteration, so once the loop has ended it still holds the value from the final iteration — which is what `finally` sees, and what a closure built in the body answers when it is called afterwards. (`dolist` binds freshly, so its closures each keep their own element; the two are supposed to differ.) ```lisp (mapcar #'funcall (loop for x in '(1 2 3) collect (lambda () x))) ; => (3 3 3) ``` The variable is assigned only once its own termination test has passed, so the clause whose driver ran out keeps its last value while an earlier clause that still had an element does advance: ```lisp (loop for x in '(1 2 3) for y in '(10 20) finally (return (list x y))) ; => (3 20) ``` `for VAR on` is the apparent exception and is not one: there the variable *is* the cursor, so it legitimately ends at `nil`, as does a `being the hash-key`/`hash-value` variable. A numeric variable ends one step past its limit: ```lisp (loop for i from 1 to 3 finally (return i)) ; => 4 ``` Accumulation and numeric ranges cover the common cases directly: ```lisp (loop for i from 1 to 10 when (evenp i) sum i) ; => 30 ``` A `while`/`until` after body clauses (or after a `for` that assigns its variable at the top of the body, such as `in`/`on`/`across`) tests at its textual position, so it can reference the current element: ```lisp (loop for x in '(1 2 3 9 4) while (< x 4) collect x) ; => (1 2 3) ``` Inside a `when`/`if`/`unless`, the anaphoric `it` names the value the test produced: ```lisp (loop for x in '(1 nil 3 nil 5) when x collect it) ; => (1 3 5) ``` `thereis` returns the first non-nil value of its expression; `always`/`never` short-circuit to `nil` on the first failure and return `t` on normal completion. Like `return`, an early exit from these skips `finally`: ```lisp (loop for x in '(nil nil 7 9) thereis x) ; => 7 ``` `(loop-finish)` inside a body form terminates the iteration normally: `finally` still runs and the loop result is produced (unlike `return`, which skips both): ```lisp (loop for i from 1 collect i into xs do (when (>= i 3) (loop-finish)) finally (return (length xs))) ; => 3 ``` A `for`/`with` variable may be a destructuring pattern — a list of variables (nested patterns allowed, `nil` ignores a position): ```lisp (loop for (a b) in '((1 2) (3 4) (5 6)) collect (+ a b)) ; => (3 7 11) ``` A dotted pattern binds the rest of the list, so it walks an alist directly: ```lisp (loop for (k . v) in '((a . 1) (b . 2)) collect (list k v)) ; => ((A 1) (B 2)) ``` `for ... across` walks a string character by character, or a vector element by element: ```lisp (loop for c across "hello" count (eql c #\l)) ; => 2 ``` ```lisp (loop for x across #(1 2 3 4 5) collect (* x x)) ; => (1 4 9 16 25) ``` `for VAR being {the|each} {hash-keys|hash-key|hash-values|hash-value} {of|in} TABLE` drives the loop over a hash table, with an optional `using (hash-value V)` (or `using (hash-key K)`) to bind the other half: ```lisp (let ((h (make-hash-table))) (setf (gethash 'a h) 1) (loop for k being the hash-keys of h using (hash-value v) collect (list k v))) ; => ((A 1)) ``` The clause snapshots the table and walks the snapshot, so the iteration order is the table's, and mutating the table inside the body does not affect the walk in progress. The package form of `being` — `for VAR being {the|each} {symbols|present-symbols|external-symbols} {of|in} PACKAGE` — is accepted but **lite**: rontolisp has no runtime intern table, so the clause parses and iterates the *empty* sequence. The body never runs and accumulation yields `nil`. It exists so libraries whose load-time code walks a package (such as cl-who's hyperdoc table) load without error: ```lisp (loop for s being the external-symbols of :cl collect s) ; => NIL ``` A type spec after the variable — either spelling — is accepted and ignored: ```lisp (loop for v fixnum = 0 then (1+ v) for i from 1 to 3 collect v) ; => (0 1 2) ``` ```lisp (loop for v of-type fixnum = 0 then (1+ v) for i from 1 to 3 collect v) ; => (0 1 2) ``` Limitations: `named`/`return-from` is not supported. Destructuring patterns do not recognize lambda-list keywords (`&optional` and friends bind as ordinary variables rather than signalling). `(loop-finish)` must appear in statement position (not mid-expression) and not inside a nested iteration form. `thereis`/`always`/`never` cannot be combined with accumulation into the default result (use `into`). Accumulation clauses without `into` must all be of the same kind; collecting clauses build the result list in source order. --- # FILE: references/reference/macros/macrolet.md # macrolet `(macrolet ((name lambda-list body...)...) body...)` Defines local, lexically scoped macros for the body. Each local macro is expanded within the body exactly like a `defmacro`-defined macro -- its body runs at expansion time with the unevaluated argument forms bound to the parameters -- and the definitions do not leak past the body. Definition lambda lists accept the same shapes as `defmacro` (required parameters plus `&rest`/`&body`, and extended lists via `destructuring-bind`). Local macro bodies see global helper functions but not the surrounding runtime bindings. On the compile path (`UserMacroExpander`) the whole `macrolet` is expanded away before the JVM/WASM compilers run; the interpreter expands it natively. `macrolet` cannot locally shadow a standard operator (a name in the function namespace); for the variable namespace, see [`symbol-macrolet`](symbol-macrolet.md). ```lisp (macrolet ((sq (x) `(* ,x ,x)) (twice (x) `(+ ,x ,x))) (+ (sq 5) (twice 5))) ; => 35 ``` --- # FILE: references/reference/macros/make-condition.md # make-condition `(make-condition type &key initargs...)` Constructs a condition object of the given type — a CLOS-subset instance whose slots are filled from the initargs (missing slots take their `:initform`). The type must be a literal quoted symbol naming a type defined by [`define-condition`](define-condition.md) or a seeded built-in like `simple-error`. The instance can be passed to [`error`](error.md)/[`signal`](signal.md) (the condition-object designator) and tested with `typecase`. ```lisp (make-condition 'simple-error :format-control "something failed") ; => # ``` ```console > (error (make-condition 'simple-error :format-control "something failed")) Error: something failed ``` --- # FILE: references/reference/macros/make-instance.md # make-instance `(make-instance 'class-name :initarg value ...)` Creates an instance of a [`defclass`](../special-forms/defclass.md) class. Slots are supplied by their `:initarg` keywords (defaulting to the slot-name keyword); an unsupplied slot takes its `:initform` (or `nil`). A **literal quoted class name** naming a class defined by `defclass` compiles to a direct constructor call. A COMPUTED class also works — a name symbol built at run time (`(make-instance (intern (format nil "~A-~A" style '#:reporter) package) ...)`, matched under either the `pkg:name` or the `pkg::name` spelling, so the class need not be exported) or the metaobject [`find-class`](../functions/find-class.md) answers — and so does `#'make-instance` as a value; both dispatch at run time over the classes the program defines. On the compiled backends that SET is fixed at compile time: a class built from runtime data does not exist. ```lisp (defclass point () ((x :initarg :x :initform 0 :reader point-x) (y :initarg :y :initform 0 :reader point-y))) (setq p (make-instance 'point :x 3)) (list (point-x p) (point-y p)) ; => (3 0) ``` --- # FILE: references/reference/macros/multiple-value-bind.md # multiple-value-bind `(multiple-value-bind (var...) values-form body...)` Binds the variables to the values of `values-form` and evaluates the body. A literal [`values`](../functions/values.md) call supplies all of its values, and the two-value built-ins [`floor`](../functions/floor.md)/[`ceiling`](../functions/ceiling.md)/[`round`](../functions/round.md)/[`truncate`](../functions/truncate.md) (quotient and remainder) and [`gethash`](../functions/gethash.md) (value and present-p) supply both of theirs. A call to a USER function whose result is a `(values ...)` call also supplies all of its values: the extra values cross the function boundary through an internal channel the callee's `values` writes and the consumer reads. A user function or method whose result position holds one of the two-value built-ins supplies both values the same way, so a lookup function that just returns a `gethash` still answers present-p. Extra variables bind to nil; surplus values are evaluated and discarded. Deviation from Common Lisp: a producer that calls `values` in a non-tail position and then returns normally may leave stale extra values behind, so prefer `values` in result position. ```lisp (multiple-value-bind (q r) (floor 7 2) (list q r)) ; => (3 1) ``` ```lisp (let ((h (make-hash-table))) (setf (gethash 'a h) nil) (multiple-value-bind (v present-p) (gethash 'a h) (list v present-p))) ; => (NIL T) ``` ```lisp (defun div-mod (a b) (values (floor (/ a b)) (mod a b))) (multiple-value-bind (q r) (div-mod 17 5) (list q r)) ; => (3 2) ``` --- # FILE: references/reference/macros/multiple-value-call.md # multiple-value-call `(multiple-value-call function values-form...)` Calls `function` with all values of every `values-form` as the arguments. The function is evaluated first, then the producers left to right; each producer is recognized like in [`multiple-value-bind`](multiple-value-bind.md), including a user function whose result is a `(values ...)` call, whose values are spread at runtime — a compiled program using `multiple-value-call` therefore embeds the runtime `eval` support, like `apply` (deviates from CL: classified as a macro, not a special operator). Built-in function values passed as `function` are synthesized wrappers: the naturally variadic operators (`#'+`, `#'-`, `#'*`, `#'/`, `#'list`, `#'min`, `#'max`) accept any argument count, but every other multi-argument built-in keeps a fixed wrapper arity (e.g. `#'cons` and the comparison chains are binary); use a user-defined function or a `lambda` for other arities. ```lisp (multiple-value-call #'+ (values 1 2)) ; => 3 ``` ```lisp (defun collect (&rest args) args) (multiple-value-call #'collect 1 (values 2 3) (floor 9 4)) ; => (1 2 3 2 1) ``` --- # FILE: references/reference/macros/multiple-value-list.md # multiple-value-list `(multiple-value-list values-form)` Collects the values of `values-form` into a list. The producer is recognized like in [`multiple-value-bind`](multiple-value-bind.md): a literal `(values ...)` call, the multi-value built-ins (`floor`/`ceiling`/`round`/`truncate`, `gethash`, `parse-integer`, `values-list`) and a call to a user function whose result is a `(values ...)` call supply all of their values; any other producer (a variable, a literal, a function that returns normally) supplies a single value, so the result is a one-element list. ```lisp (multiple-value-list (floor 17 5)) ; => (3 2) ``` ```lisp (multiple-value-list (+ 1 2)) ; => (3) ``` ```lisp (defun two () (values 1 2)) (multiple-value-list (two)) ; => (1 2) ``` --- # FILE: references/reference/macros/multiple-value-prog1.md # multiple-value-prog1 `(multiple-value-prog1 first-form form...)` Evaluates `first-form`, then the remaining forms for effect, and returns **all** the values of `first-form` -- [`prog1`](prog1.md) widened to multiple values. The values are captured with [`multiple-value-list`](multiple-value-list.md) and republished with [`values-list`](../functions/values-list.md), so they survive whatever the intervening forms do. ```lisp (multiple-value-list (multiple-value-prog1 (floor 17 5) (list :cleanup))) ; => (3 2) ``` --- # FILE: references/reference/macros/multiple-value-setq.md # multiple-value-setq `(multiple-value-setq (var...) values-form)` Assigns the values of `values-form` to the existing variables with `setq` and returns the primary value. The producer is recognized like in [`multiple-value-bind`](multiple-value-bind.md) (a literal [`values`](../functions/values.md) call, the two-value built-ins [`floor`](../functions/floor.md)/[`ceiling`](../functions/ceiling.md)/[`round`](../functions/round.md)/[`truncate`](../functions/truncate.md) and [`gethash`](../functions/gethash.md)); extra variables receive nil. ```lisp (let (a b) (multiple-value-setq (a b) (floor 17 5)) (list a b)) ; => (3 2) ``` ```lisp (let (a b) (multiple-value-setq (a b) (values 1 2))) ; => 1 ``` --- # FILE: references/reference/macros/nth-value.md # nth-value `(nth-value n values-form)` Returns the `n`-th (0-based) value of `values-form`, or nil when there is no such value. `n` is evaluated before the form. Expands to `nth` over [`multiple-value-list`](multiple-value-list.md), so the producer is recognized like in [`multiple-value-bind`](multiple-value-bind.md), including a user function whose result is a `(values ...)` call. ```lisp (nth-value 1 (floor 7 2)) ; => 1 ``` ```lisp (nth-value 0 (values 'a 'b)) ; => A ``` --- # FILE: references/reference/macros/or.md # or `(or expr1 expr2...)` Evaluates its expressions left to right, short-circuiting as soon as one returns a non-nil value and yielding that value; if every expression is nil it returns nil. `(or)` with no arguments returns nil. It expands into nested `if` forms, so expressions after the first truthy one are not evaluated. ```lisp (or nil 2 3) ; => 2 ``` --- # FILE: references/reference/macros/pop.md # pop `(pop place)` Removes the first element from the list stored in `place`: it returns that element and stores the rest of the list (its `cdr`) back into `place`. `place` may be any location `setf` accepts. Calling `pop` on nil returns nil and leaves the place as nil. ```lisp (let ((s (list 1 2 3))) (pop s)) ; => 1 ``` --- # FILE: references/reference/macros/pprint-logical-block.md # pprint-logical-block `(pprint-logical-block (stream object &key prefix per-line-prefix suffix) body...)` Writes `prefix`, evaluates the body -- which prints to `stream` -- then writes `suffix`, and returns `nil`. When `object` is not a list it is printed with `write` and the body is skipped, which is Common Lisp's own rule and what makes the macro safe to wrap around a value that may or may not be a list. A rontolisp stream carries no column, so the block never WRAPS and `:per-line-prefix` is accepted as a synonym of `:prefix` (no line inside the block ever begins on its own). See `pprint` for the rest of that story. ```lisp (list (with-output-to-string (s) (pprint-logical-block (s '(1 2 3) :prefix "<" :suffix ">") (princ "body" s))) (with-output-to-string (s) (pprint-logical-block (s 5 :prefix "<" :suffix ">") (princ "body" s)))) ; => ("" "5") ``` --- # FILE: references/reference/macros/print-unreadable-object.md # print-unreadable-object `(print-unreadable-object (object stream &key type identity) body...)` Writes `#<`...`>` around the body's output to `stream` and returns `nil`. A true `:type` prints the [`type-of`](../functions/type-of.md) designator first, followed by a space when a body follows it; `:identity` is accepted but prints no address -- there is no object-identity token in the value model, and a per-backend one would make the same program print differently on each backend. The usual body of a [`print-object`](../functions/print-object.md) method. The type designator is written like any symbol, so `*print-escape*` decides whether its package qualifier appears: under `prin1`/`~S` a type defined in another package prints as `PKG:NAME`, under `princ`/`~A` as just `NAME`. The two agree for a type whose name needs no qualifier. ```lisp (with-output-to-string (s) (print-unreadable-object ('x s :type nil) (princ "thing" s))) ; => "#" ``` --- # FILE: references/reference/macros/proclaim.md # proclaim `(proclaim declaration)` A parsed no-op like `declaim`: the form evaluates to nil. This deviates from Common Lisp, where `proclaim` is a function whose argument is evaluated -- here it is classified as a macro, so the argument is not evaluated either (and `#'proclaim` is unsupported). ```lisp (proclaim '(special *state*)) ; => NIL ``` --- # FILE: references/reference/macros/prog-star.md # prog* `(prog* (bindings...) {tag | form}...)` Like [`prog`](prog.md) with sequential ([`let*`](let-star.md)-style) bindings: each init form sees the variables bound before it. ```lisp (prog* ((x 5) (y (* x 2))) (return (+ x y))) ; => 15 ``` --- # FILE: references/reference/macros/prog.md # prog `(prog (bindings...) {tag | form}...)` Binds the variables like [`let`](../special-forms/let.md), then runs the body as a [`tagbody`](../special-forms/tagbody.md) inside a block: [`go`](../special-forms/go.md) jumps between the body's tags and `(return value)` exits the `prog` with `value`. Falling off the end returns nil. ```lisp (prog ((n 5) (acc 1)) top (when (<= n 1) (return acc)) (setq acc (* acc n)) (setq n (- n 1)) (go top)) ; => 120 ``` --- # FILE: references/reference/macros/prog1.md # prog1 `(prog1 first body...)` Evaluates `first` and all the body forms in order, then returns the value of `first`. It is useful for capturing a value before the side effects in the rest of the body run -- for example, reading an old value out of a place before updating it. ```lisp (let ((x 10)) (prog1 x (setq x 20))) ; => 10 ``` --- # FILE: references/reference/macros/prog2.md # prog2 `(prog2 first second body...)` Evaluates `first`, then `second`, then any remaining body forms in order, and returns the value of `second`. It mirrors `prog1` but yields the *second* form's value, which is handy when the first form is purely a setup side effect. ```lisp (prog2 1 2 3) ; => 2 ``` --- # FILE: references/reference/macros/psetf.md # psetf `(psetf place1 e1 place2 e2 ...)` `psetq` generalized to `setf` places: every place subform and every right-hand side expression is evaluated into a temporary first, and only then are the places assigned, so a later place that reads a variable assigned by an earlier pair still sees the old value. `psetf` always returns nil. ```lisp (let ((a 1) (b 2)) (psetf a b b a) (list a b)) ; => (2 1) ``` ```lisp (let* ((tail (list 2)) (last-cdr tail) (fresh (list 3))) (psetf last-cdr fresh (cdr last-cdr) fresh) tail) ; => (2 3) ``` --- # FILE: references/reference/macros/psetq.md # psetq `(psetq v1 e1 v2 e2 ...)` Parallel assignment: every right-hand side expression is evaluated first, and only then are the variables assigned, so each value is computed against the variables' old bindings. This makes it possible to swap or rotate variables in one step without a temporary. `psetq` always returns nil. ```lisp (let ((a 1) (b 2)) (psetq a b b a) (list a b)) ; => (2 1) ``` --- # FILE: references/reference/macros/push.md # push `(push item place)` Prepends `item` to the list stored in `place`, stores the resulting longer list back into `place`, and returns that new list. `place` may be any location `setf` accepts, not just a variable. It expands into `(setf place (cons item place))`. ```lisp (let ((s (list 1 2))) (push 0 s) s) ; => (0 1 2) ``` --- # FILE: references/reference/macros/pushnew.md # pushnew `(pushnew item place &key test key)` Prepends `item` to the list stored in `place` only when it is not already a member (compared with `eql`, or the given `:test`), and stores the result back. Returns the (possibly unchanged) list. Like `push`, the place may be evaluated more than once. ```lisp (setq ns (list 2 3)) (pushnew 1 ns) ; => (1 2 3) (pushnew 2 ns) ; => (1 2 3) ns ; => (1 2 3) ``` --- # FILE: references/reference/macros/remf.md # remf `(remf place indicator)` Removes the first key/value pair matching `indicator` from the property list stored in `place`, updating `place` in place. It returns `t` if a matching pair was found and removed, or nil otherwise. Because it both mutates the plist and reports whether anything changed, inspect the place afterwards to see the result. ```lisp (let ((p (list :a 1 :b 2))) (remf p :a) p) ; => (:B 2) ``` --- # FILE: references/reference/macros/restart-bind.md # restart-bind `(restart-bind ((restart-name function [:report-function r]...)...) body...)` The primitive sibling of [`restart-case`](restart-case.md): evaluates `body...` with one restart record per binding on the dynamic restart stack, but an invoked restart **calls `function` at the invocation point** — there is no non-local transfer back to the `restart-bind` frame, and `invoke-restart` returns whatever the function returns (the CL semantics; the function must transfer control itself if it wants to). The function receives the invocation arguments (at most 10 on the WASM backends). `:report-function` is accepted and stored; other per-binding keyword options are accepted and ignored. Supported on every backend except `--no-gc`. ```lisp (let ((hit nil)) (restart-bind ((poke (lambda (v) (setq hit v)))) (invoke-restart 'poke 9) hit)) ; => 9 ``` --- # FILE: references/reference/macros/restart-case.md # restart-case `(restart-case form (restart-name (arg...) [:report r] [:interactive i] [:test t] body...)...)` Evaluates `form` with one **restart** established per clause for its dynamic extent. Nothing happens on normal completion — the form's values are returned and the restarts are disestablished. When code running inside the form (typically a [`handler-bind`](handler-bind.md) handler, running at the signal point) invokes one of the restarts with [`invoke-restart`](../functions/invoke-restart.md), control unwinds back to the `restart-case` (running intervening `unwind-protect` cleanups) and the clause body runs **in the restart-case's own lexical environment** with the invocation arguments bound to `arg...` — so a clause body can `return-from` an enclosing function or `go` to a tag of an enclosing `tagbody` (the retry-loop idiom). The restart name may be a symbol or a keyword; [`find-restart`](../functions/find-restart.md) returns the innermost active restart as a first-class object and [`compute-restarts`](../functions/compute-restarts.md) lists them all. The `:report`, `:interactive` and `:test` options are accepted and stored in the restart record (nothing in rontolisp renders reports or invokes restarts interactively — there is no debugger). Supported on every backend except `--no-gc`, which keeps the historical lowering to the primary form. A restart-system program compiles in EH mode on the wasm-GC backends (`wasmtime -W exceptions=y`). Lite deviations: `&optional` clause parameters take `nil` instead of their default when not supplied, restarts are not associated with conditions (the optional condition argument of `find-restart` is ignored), and a restart object prints as a plain list rather than `#`. ```lisp (restart-case (+ 1 2) (continue () 99)) ; => 3 ``` A handler invokes the restart by keyword name, with arguments; the clause body's value becomes the value of the whole form: ```lisp (handler-bind ((error (lambda (c) (invoke-restart :reconnect "db-1")))) (restart-case (error "connection lost") (:reconnect (host) (list :reconnected host)))) ; => (:RECONNECTED "db-1") ``` The retry idiom — a clause body `go`ing back into an enclosing `tagbody`: ```lisp (let ((n 0)) (handler-bind ((error (lambda (c) (invoke-restart 'retry)))) (tagbody start (restart-case (progn (setq n (+ n 1)) (when (< n 3) (error "again"))) (retry () (go start))))) n) ; => 3 ``` --- # FILE: references/reference/macros/return-from.md # return-from `(return-from name [value])` Returns `value` (default `nil`) from the enclosing block named `name`. A `defun` body is a block named after the function and a `defmethod` body one named after its generic, so a `return-from` exits the function even from inside a `do`/`loop` (whose implicit block is named nil); `(return-from nil v)` is `(return v)`. A same-function `return-from` compiles to a direct jump. A `return-from` inside a lambda that names an enclosing block — for example inside a lambda passed to `mapcar`/`mapl` — exits that block as a non-local exit on every backend, matching Common Lisp. (Such a cross-lambda exit compiles in exception-handling mode, so on the WASM backends the program needs `wasmtime -W exceptions=y`; a `return-from` that does not cross a lambda stays flag-free. A `return-from` that would have to cross an `flet`/`labels` local function is not yet supported on the compilers.) ```lisp (defun classify (n) (when (= n 0) (return-from classify :zero)) (* n 10)) (classify 0) ; => :ZERO ``` ```lisp (defun first-even (items) (dolist (x items) (when (evenp x) (return-from first-even x))) :none) (first-even '(1 3 4 5)) ; => 4 ``` --- # FILE: references/reference/macros/rontolisp-with-arena.md # rontolisp:with-arena `(rontolisp:with-arena () body...)` Runs the body and returns its value, naming a **memory-reclamation boundary** for the non-GC WASM backend ([`--no-gc`](../../guides/wasm-nogc.md)). On that backend nothing is freed within one export call — the bump allocator only pops at the export boundary — so a loop that allocates each iteration (string building, fresh `vec:` vectors) grows the linear memory. `with-arena` closes that gap: it snapshots the bump heap pointer, runs the body, and pops everything the body allocated, keeping only the body's own value (a string or packed float array result is copied down to the snapshot point). On the interpreter, the JVM backend and the default (wasm-GC) output a real garbage collector already reclaims, so it is observationally a plain `progn` and the same source runs on every backend. ```lisp (rontolisp:with-arena () (+ 1 2)) ; => 3 ``` ## Arguments - An option list, which must currently be empty (`()`); it is reserved for future options. - The body forms, evaluated in order like a `progn`. The value of the last form is the value of the whole expression (`nil` for an empty body). ## The escape contract (`--no-gc`) Nothing allocated inside the body may be reachable after it, **except the body's own value**. Storing an inner allocation somewhere that outlives the arena (for example writing it into an array created outside) leaves a dangling pointer once the arena pops — the same rule as the host-side `__ronto_alloc_reset` ([Reclaiming memory](../../guides/wasm-nogc.md#reclaiming-memory-the-arena-api)). A typical use — keeping a hot loop flat on `--no-gc`: ```lisp (defun train (epochs n) (let ((acc 0.0)) (dotimes (i epochs) (rontolisp:with-arena () ; everything allocated inside ... (setq acc (+ acc (vec:sum (vec:ones n)))) ; ... is popped here )) acc)) (train 3 4) ; => 12.0 ``` ## Limitations - The option list must be empty; a non-empty option list is an error. - On `--no-gc`, a `return` that exits across the arena boundary skips the pop (the allocations inside are simply not reclaimed; nothing is corrupted). - Macros have no function value: `#'rontolisp:with-arena` is an error. --- # FILE: references/reference/macros/rontolisp-with-mutex.md # rontolisp:with-mutex `(rontolisp:with-mutex (mutex-form) body...)` Evaluates `mutex-form` once, acquires the resulting lock (see [`rontolisp:make-mutex`](../functions/rontolisp-make-mutex.md)), runs the body, and releases the lock on **every** exit — including one caused by a signalled error. The value of the last body form is the value of the whole expression (`nil` for an empty body). This is the form to reach for when a served handler mutates state shared between requests: [`rontolisp:http-handler`](../functions/rontolisp-http-handler.md) puts one virtual thread per request on the interpreter and the JVM backend, so a read-modify-write of a global is a real race there. Both WASM backends run a single thread, so acquire and release are no-ops and the same source runs on all four. ```lisp (defvar *counter-lock* (rontolisp:make-mutex)) (defvar *counter* 0) (rontolisp:with-mutex (*counter-lock*) (setq *counter* (+ *counter* 1))) ; => 1 ``` ## Arguments - A one-element list holding the form that produces the mutex. It is evaluated once, before the body. - The body forms, evaluated in order like a `progn` while the lock is held. ## Reentrancy The lock is reentrant, so nesting is safe — a function that takes the lock may call another that takes the same lock: ```lisp (let ((m (rontolisp:make-mutex))) (rontolisp:with-mutex (m) (rontolisp:with-mutex (m) :nested))) ; => :NESTED ``` ## Limitations - There is no way to spawn a thread from Lisp; the concurrency comes from the runtime. - No timed or non-blocking variant: acquisition always blocks. - Macros have no function value: `#'rontolisp:with-mutex` is an error. --- # FILE: references/reference/macros/rotatef.md # rotatef `(rotatef place...)` Rotates the values of its [`setf`](setf.md)-able places to the left: the first place receives the old value of the second, and so on, and the last place receives the old value of the first. Returns nil. Every place is read into a temporary before any is written, so a two-place `rotatef` swaps. ```lisp (let ((x 1) (y 2)) (rotatef x y) (list x y)) ; => (2 1) ``` ```lisp (let ((a 1) (b 2) (c 3)) (rotatef a b c) (list a b c)) ; => (2 3 1) ``` ```lisp (let ((x (cons 1 2))) (rotatef (car x) (cdr x)) x) ; => (2 . 1) ``` --- # FILE: references/reference/macros/setf.md # setf `(setf place value [place2 value2 ...])` Generalized assignment: stores `value` into the location named by `place` and returns the value. Beyond plain variables, the supported places are the list accessors `car`, `cdr`, `nth`, `first` through `fourth`, `rest`, and the `caXXXr` compositions, plus `elt` (a runtime list/string/vector dispatch: a list cell and a vector -- including a mutable buffer from [`make-string`](../functions/make-string.md) -- are written in place, while a string literal is rebuilt and rebound like [`(setf (char ...))`](../functions/char.md), so its place must be a **variable** and an alias made before the write still sees the old content), so you can mutate a specific slot of an existing structure in place. It expands into the appropriate primitive mutator (such as `rplaca`/`rplacd`). Place subforms are evaluated before the value, so the tail-collection idiom `(setf (cdr tail) (setf tail (list x)))` links the old tail. ```lisp (let ((x (list 1 2 3))) (setf (second x) 99) x) ; => (1 99 3) ``` Multiple place/value pairs assign sequentially (each pair sees the effects of the previous ones), and the last value is returned: ```lisp (let ((x (list 1 2 3))) (setf (car x) 9 (second x) 8) x) ; => (9 8 3) ``` `(setf (getf place indicator) value)` writes a property list: when the indicator already has a cell its VALUE cell is written in place (so an alias of the same list sees the update), otherwise the pair is consed onto the front and the result stored back through `place`. A third subform in the place is `getf`'s default and is ignored by the write. ```lisp (let ((p (list :a 1))) (setf (getf p :b) 2) p) ; => (:B 2 :A 1) ``` Beyond the built-in places, a `defstruct` accessor, a CLOS `:accessor`, and a user-defined *setf-function* (`(defun (setf name) ...)`, or the generic `(defmethod (setf name) ...)` — see [defmethod](../special-forms/defmethod.md)) are also places: `(setf (name arg...) value)` calls the writer with the new value first. See [defun](../special-forms/defun.md) for setf-function definitions. `(setf (symbol-function 'name) fn)` / `(setf (fdefinition 'name) fn)` install a global function definition (see [symbol-function](../functions/symbol-function.md)). ```lisp (defvar *mode* :xml) (defun (setf my-mode) (m) (setq *mode* m)) (setf (my-mode) :html5) *mode* ; => :HTML5 ``` --- # FILE: references/reference/macros/shiftf.md # shiftf `(shiftf place... new-value)` Shifts values left through its [`setf`](setf.md)-able places: each place receives the old value of the place to its right, the last place receives `new-value`, and the OLD value of the first place is returned. All places and the new value are evaluated once, left to right. ```lisp (let ((a 1) (b 2)) (list (shiftf a b 9) a b)) ; => (1 2 9) ``` --- # FILE: references/reference/macros/signal.md # signal `(signal datum args...)` Signals a **non-fatal** condition with the same designator surface as [`error`](error.md): a control string, literal or computed, with the arguments after it as its format arguments (builds a `simple-condition`), a quoted condition-type symbol with initargs, or a condition object. When an established [`handler-case`](handler-case.md) has a clause matching the condition, the signal transfers control to it; otherwise -- no handler at all, or none whose clauses match -- `signal` returns nil and execution continues (the Common Lisp fall-through, CLHS 9.1.4.1). A `handler-case` whose clauses do not match is declined and stays armed for a later condition that does. This works on every backend except `--no-gc`, whose compiler rejects catching (`signal` there always evaluates its arguments and returns nil). ```lisp (signal "nothing is listening") ; => NIL ``` ```lisp (handler-case (progn (signal "caught mid-flight") :not-raised) (condition (c) :raised)) ; => :RAISED ``` ```lisp (handler-case (progn (signal "nobody handles this") :fell-through) (type-error (c) :caught)) ; => :FELL-THROUGH ``` --- # FILE: references/reference/macros/slot-boundp.md # slot-boundp `(slot-boundp instance 'slot-name)` Whether the named slot of the instance holds a value: `nil` when the instance's class has no such slot, when the slot was written with no `:initform` and no initarg supplied it, or after [`slot-makunbound`](slot-makunbound.md); `t` otherwise. Reading an unbound slot with [`slot-value`](slot-value.md) or an accessor signals `unbound-slot`, whose [`cell-error-name`](../functions/cell-error-name.md) is the slot and whose [`unbound-slot-instance`](../functions/unbound-slot-instance.md) is the object. On the JVM and WASM compilers the slot name must be a literal quoted symbol, like [`slot-value`](slot-value.md); a runtime-computed slot name works on the interpreter only. ```lisp (defclass sb-point () ((x :initarg :x) (y :initform 0))) (let ((p (make-instance 'sb-point))) (list (slot-boundp p 'x) (slot-boundp p 'y))) ; => (NIL T) ``` --- # FILE: references/reference/macros/slot-exists-p.md # slot-exists-p `(slot-exists-p instance 'slot-name)` Whether the instance's class declares a slot of that name, regardless of boundness: an unbound slot exists ([`slot-boundp`](slot-boundp.md) is the boundness test), an undeclared one does not, and a non-instance answers `nil`. The slot name may be a runtime-computed symbol on every backend. ```lisp (defclass se-point () ((x :initarg :x) (y :initform 0))) (let ((p (make-instance 'se-point))) (list (slot-exists-p p 'x) (slot-exists-p p 'z) (slot-exists-p 42 'x))) ; => (T NIL NIL) ``` --- # FILE: references/reference/macros/slot-makunbound.md # slot-makunbound `(slot-makunbound instance 'slot-name)` Makes the named slot unbound and returns the instance: [`slot-boundp`](slot-boundp.md) then answers `nil` and a read through [`slot-value`](slot-value.md) or an accessor signals `unbound-slot`. Storing into the slot binds it again. On the JVM and WASM compilers the slot name must be a literal quoted symbol, like [`slot-value`](slot-value.md); a runtime-computed slot name works on the interpreter only. ```lisp (defclass sm-point () ((x :initarg :x))) (let ((p (make-instance 'sm-point :x 1))) (slot-makunbound p 'x) (handler-case (slot-value p 'x) (unbound-slot (e) (cell-error-name e)))) ; => X ``` --- # FILE: references/reference/macros/slot-value.md # slot-value `(slot-value object 'slot-name)` Reads a slot of a [`defclass`](../special-forms/defclass.md) instance, and is also a `setf`-able place. A **literal quoted symbol** slot name resolves to the slot's fixed position at compile/expansion time. On the interpreter a computed slot name (a variable or expression) is also accepted and resolves the slot at runtime by name; on the compiled backends it is an error. With a literal name, a slot name used at *different* positions by two unrelated classes is rejected as ambiguous (within one inheritance chain positions always agree — prefer `:accessor`/`:reader` functions, which are per-class). Reading does not type-check the object, like `defstruct` accessors. A slot name no class in the program declares is a **run-time** error on every backend (`The slot X is missing`), not a compile-time one -- the same as reading it on the interpreter, and catchable with [`handler-case`](handler-case.md). ```lisp (defclass user () ((name :initarg :name))) (setq u (make-instance 'user :name "Alice")) (setf (slot-value u 'name) (concatenate 'string (slot-value u 'name) "!")) (slot-value u 'name) ; => "Alice!" ``` --- # FILE: references/reference/macros/symbol-macrolet.md # symbol-macrolet `(symbol-macrolet ((name expansion)...) body...)` Defines local, lexically scoped symbol macros for the body: each free reference to a `name` evaluates its `expansion` in its place, and a `setq`/`setf` of a `name` assigns through the expansion as a `setf` place. An inner binding of the same name (`let`/`let*`, a `lambda`/`defun` parameter, `do`, `dolist`, ...) shadows the symbol macro in its scope. Quoted data, function-namespace positions (`#'name`, call heads), `case`-family keys, `go` tags, and `block` names are never substituted, and declarations directly in the body are dropped. Sibling macros may reference each other; a self-referential expansion is substituted once, not expanded recursively. The global `define-symbol-macro` is not supported. ```lisp (let ((cell (list 1 2))) (symbol-macrolet ((head (car cell))) (setf head 99) cell)) ; => (99 2) ``` ```lisp (symbol-macrolet ((x 42)) (list (let ((x 1)) x) x)) ; => (1 42) ``` --- # FILE: references/reference/macros/the.md # the `(the type form)` Evaluates `form` and returns its value unchanged. The type is not checked (there is no compiler type system to inform), so `the` is pure documentation -- it exists so annotated library sources load unchanged. Use `check-type` for an actual runtime type assertion. ```lisp (the integer (+ 1 2)) ; => 3 ``` --- # FILE: references/reference/macros/time.md # time `(time form)` Evaluates `form`, prints a line of the form `; Elapsed real time: N ms` to standard output, and returns the value `form` produced. `N` is an integer of milliseconds on the interpreter and JVM backends and a float of milliseconds on WASM; because it reports wall-clock time it is non-deterministic and varies from run to run. The example below prints a timing line and returns `3`. ```lisp (time (+ 1 2)) ``` --- # FILE: references/reference/macros/torch-no-grad.md # torch:no-grad `(torch:no-grad body...)` Runs the body with gradient recording disabled: the `torch` operations inside compute their values as usual but record nothing on the autograd tape, so the results are constant leaves ([`torch:requires-grad-p`](../functions/torch-requires-grad-p.md) answers `nil`) and no history is retained. This is how a training loop's parameter update -- and inference in general -- stays off the tape; the per-tensor spelling is [`torch:detach`](../functions/torch-detach.md). Mechanically it dynamically rebinds the internal `torch::*grad-enabled*` variable to `nil` around the body, so recording resumes when the form exits. ```lisp (defparameter *w* (torch:tensor '(1.0 2.0) :requires-grad t)) (torch:no-grad (torch:requires-grad-p (torch:mul *w* 2.0))) ; => NIL (torch:requires-grad-p (torch:mul *w* 2.0)) ; => T ``` --- # FILE: references/reference/macros/typecase.md # typecase `(typecase x (integer body...) (string body...) (t default...))` Evaluates `x` once and selects the first clause whose type specifier `x` satisfies, evaluating that clause's body and returning its last value. The supported type names are `integer`, `float`, `number`, `rational`, `string`, `symbol`, `keyword`, `cons`, `list`, `null`, `atom`, `character`, `hash-table`, and `boolean`, plus a final `t`/`otherwise` default clause. Compound specifiers also work: `(or ...)`, `(and ...)`, `(not ...)`, `(member item...)`, `(eql object)`, `(satisfies function)`, and ranged numeric types like `(integer 0 9)` (see `check-type` for the details). A zero-parameter user [`deftype`](deftype.md) name resolves too. If no clause matches and there is no default, `typecase` returns nil. ```lisp (let ((x "hi")) (typecase x (integer 'int) (string 'str) (t 'other))) ; => STR ``` ```lisp (let ((n 5)) (typecase n ((integer 0 9) 'digit) (integer 'int))) ; => DIGIT ``` --- # FILE: references/reference/macros/typep.md # typep `(typep object 'type-specifier)` Tests whether `object` is of the given type. Lite: the type specifier is normally a literal (quoted) type — the same set [`typecase`](typecase.md) supports (atomic names, registered classes, zero-parameter user [`deftype`](deftype.md) names, and the compound specifiers `(or ...)`/`(and ...)`/`(not ...)`/`(member ...)`/`(eql ...)`/`(satisfies ...)`/ranged numerics/`(unsigned-byte n)`/`(signed-byte n)`); an unknown specifier matches nothing. A specifier computed at run time is supported when it is an ATOMIC type name (a registered class / struct / condition, or a built-in name) or a class metaobject — what [`find-class`](../functions/find-class.md) and [`class-of`](../functions/class-of.md) answer designates its own class. The compound specifiers above still require a literal. `class` is the class every class metaobject belongs to, so `(typep x 'class)` is the "is this a class?" test. ```lisp (typep 5 '(unsigned-byte 8)) ; => T ``` ```lisp (typep 500 '(unsigned-byte 8)) ; => NIL ``` ```lisp (defclass animal () ()) (defclass dog (animal) ()) (list (typep (make-instance 'dog) (find-class 'animal)) (typep (find-class 'dog) 'class) (typep 42 'class)) ; => (T T NIL) ``` --- # FILE: references/reference/macros/uiop-if-let.md # uiop:if-let `(uiop:if-let ((var form)...) then [else])` Binds the variables in parallel like [`let`](../special-forms/let.md), then evaluates `then` when **every** variable came out non-nil and `else` otherwise. The bindings are established for both branches, so `else` can still see them. A single un-nested binding is accepted as well — `(uiop:if-let (x form) ...)` — which is how UIOP itself spells the one-variable case; a binding list whose first element is a symbol *is* the one binding. ```lisp (list (uiop:if-let ((a 1) (b 2)) (list a b) :none) (uiop:if-let ((a 1) (b nil)) (list a b) :none) (uiop:if-let (x (+ 1 2)) (* x 10) :none)) ; => ((1 2) :NONE 30) ``` `uiop` is ASDF's portability layer, not part of Common Lisp: the name is only reachable with the `uiop:` qualifier. This is UIOP's own copy of alexandria's macro of the same name, and the two behave identically. ## Backend support Works on all four backends: it is a built-in macro expansion shared by the interpreter and both compilers. Like the other built-in macros it has no function value (`#'uiop:if-let` is an error). --- # FILE: references/reference/macros/uiop-when-let-star.md # uiop:when-let* `(uiop:when-let* ((var form)...) body...)` The sequential [`uiop:when-let`](uiop-when-let.md): each binding's form sees the variables bound before it, and the first variable that comes out nil short-circuits the whole form to nil **without evaluating the remaining forms**. That is the difference that matters — a later form may safely assume the earlier bindings are non-nil. ```lisp (list (uiop:when-let* ((a 5) (b (* a 2))) (+ a b)) (uiop:when-let* ((a nil) (b (/ 1 a))) b)) ; => (15 NIL) ``` The second line never evaluates `(/ 1 a)`, which would signal. `uiop` is ASDF's portability layer, not part of Common Lisp: the name is only reachable with the `uiop:` qualifier. ## Backend support Works on all four backends: it is a built-in macro expansion shared by the interpreter and both compilers. Like the other built-in macros it has no function value (`#'uiop:when-let*` is an error). --- # FILE: references/reference/macros/uiop-when-let.md # uiop:when-let `(uiop:when-let ((var form)...) body...)` [`uiop:if-let`](uiop-if-let.md) with an implicit `progn` body and no else branch: binds the variables in parallel, and evaluates the body only when **every** variable came out non-nil. Returns nil otherwise. The single un-nested binding spelling — `(uiop:when-let (x form) ...)` — is accepted here too. ```lisp (list (uiop:when-let ((a 3) (b 4)) (+ a b) (* a b)) (uiop:when-let ((a 3) (b nil)) (+ a 1))) ; => (12 NIL) ``` `uiop` is ASDF's portability layer, not part of Common Lisp: the name is only reachable with the `uiop:` qualifier. ## Backend support Works on all four backends: it is a built-in macro expansion shared by the interpreter and both compilers. Like the other built-in macros it has no function value (`#'uiop:when-let` is an error). --- # FILE: references/reference/macros/uiop-with-deprecation.md # uiop:with-deprecation `(uiop:with-deprecation (level) definitions...)` Establishes the definitions it wraps, exactly as written, and returns the last one's value. Real UIOP additionally marks them as deprecated so that a later caller gets a warning at `level`. **rontolisp drops that diagnostic.** There is no deprecation-warning machinery and no compile-time warning channel to route one through, so the honest lowering is `(progn definitions...)` — the level form is evaluated by nothing and ignored. A library that wraps part of its API in this macro therefore loads and runs normally; you simply never hear that a name is on its way out. The expansion splices at top level, so wrapped top-level `defun`s stay top-level definitions on the compile backends (that is the shape libraries use, usually inside an `eval-when`). ```lisp (uiop:with-deprecation (:style-warning) (defun old-double (x) (* x 2)) (defun old-triple (x) (* x 3))) (list (old-double 4) (old-triple 4)) ; => (8 12) ``` `uiop` is ASDF's portability layer, not part of Common Lisp: the name is only reachable with the `uiop:` qualifier. ## Backend support Works on all four backends: it is a built-in macro expansion shared by the interpreter and both compilers. Like the other built-in macros it has no function value (`#'uiop:with-deprecation` is an error). --- # FILE: references/reference/macros/unless.md # unless `(unless condition body...)` The complement of `when`: it evaluates `condition` and, only when it is nil, evaluates the body forms in order and returns the value of the last one. If `condition` is truthy the body is skipped and `unless` returns nil. It expands into an `if` whose then/else roles are reversed. ```lisp (unless (> 3 5) 'yes) ; => YES ``` --- # FILE: references/reference/macros/usocket-with-macros.md # usocket:with-client-socket usocket:with-connected-socket usocket:with-server-socket usocket:with-socket-listener `(usocket:with-client-socket (socket-var stream-var host port &rest connect-args) body...)` -- `(usocket:with-connected-socket (var socket-form) body...)` -- `(usocket:with-server-socket (var socket-form) body...)` -- `(usocket:with-socket-listener (socket-var host port &rest listen-args) body...)` The usocket convenience macros: each binds a socket for the extent of the body and closes it afterwards. `with-client-socket` connects (passing `connect-args` through to `usocket:socket-connect`) and additionally binds `stream-var` to the socket's stream (pass `nil` to skip that binding); `with-socket-listener` listens (passing `listen-args` through to `usocket:socket-listen`); `with-connected-socket` and `with-server-socket` (aliases in this shim) wrap an existing socket form such as a `usocket:socket-accept` call. ```lisp (usocket:with-socket-listener (listener "127.0.0.1" 0) (usocket:with-client-socket (client stream "127.0.0.1" (usocket:get-local-port listener)) (write-line "ping" stream) (usocket:with-connected-socket (server (usocket:socket-accept listener)) (read-line server)))) ; => "ping" ``` On the interpreter and the JVM the expansion wraps the body in [`unwind-protect`](../special-forms/unwind-protect.md), so the socket is closed on **every** exit -- normal return, an error signaled inside the body, or a `return`/`return-from` (usocket proper's semantics). This holds on the WASM component backend too since the exception-handling support landed (such a program compiles in EH mode and needs `wasmtime -W exceptions=y`, 37+). Like `rontolisp:with-arena`, these are built-in macro expansions, so they cannot be passed to `funcall`/`apply`. ## Backend support - **Interpreter**, **JVM** and **WASM component**: wherever the underlying socket functions work (the expansion is shared by all backends; the WASM variant closes on normal exit only, see above). - **Browser playground**: not supported. --- # FILE: references/reference/macros/warn.md # warn `(warn datum args...)` Prints a `WARNING:` message to the standard error stream and returns `nil`; execution continues. The same condition designators as [`error`](error.md) are accepted: a control string, literal or computed (same directives as `format`, with the arguments after it as its format arguments), a quoted condition-type symbol with initargs (the class's `:report` -- inherited from an ancestor when the class defines none -- becomes the message, `format` applied to `:format-control`/`:format-arguments` for a `simple-warning` subtype that reports nothing else, and the `Condition (type initargs...) was signalled.` shape when the class reports nothing at all), or a condition object. A [`handler-bind`](handler-bind.md) handler on `warning` runs at the signal point before the message is printed, and can call [`muffle-warning`](../functions/muffle-warning.md) to abort the output (`warn` then returns `nil` silently); `handler-case` catches errors and `signal`, not `warn`. Like `error`, `warn` has a function value: the interpreter keeps the full designator protocol through `#'warn`, the compiled backends forward the datum only. The message goes to standard error on every backend, including the WASM `--component` output (the WASI 0.3 adapter wires fd 2 to `wasi:cli/stderr`). Because the message goes to standard error (not standard output) it is shown here statically rather than as a runnable example: ```console (warn "unexpected value: ~a" x) ``` The call prints `WARNING: unexpected value: 42` (for `x` = 42) to standard error and evaluates to `nil`. ## Redirecting the report: `*error-output*` The destination is the current value of `*error-output*`, read at call time. Its default value is the designator for the process standard error -- unlike `*standard-output*`'s `t`, which names standard *output* -- so an unredirected `warn` reaches stderr, and `(format *error-output* ...)` does too. Binding the variable captures the report, the usual Common Lisp way to test warnings: ```lisp (string-right-trim '(#\Newline) (with-output-to-string (*error-output*) (warn "unexpected value: ~a" 42))) ; => "WARNING: unexpected value: 42" ``` The binding is dynamic, so it applies inside called functions as well, and the variable is restored on the way out. It works the same on all four backends. --- # FILE: references/reference/macros/when.md # when `(when condition body...)` Evaluates `condition`; if it is truthy, evaluates the body forms in order and returns the value of the last one. If `condition` is nil, the body is skipped and `when` returns nil. It is shorthand for an `if` with no else branch and an implicit `progn` body. ```lisp (when (> 5 3) 'yes) ; => YES ``` --- # FILE: references/reference/macros/with-accessors.md # with-accessors `(with-accessors ((var accessor)...) instance body...)` Binds each `var` as a symbol-macro-style place standing for `(accessor instance)` in the body -- the accessor-call twin of [`with-slots`](with-slots.md). The instance form is evaluated once. Reads call the accessor, and [`setf`](setf.md)/`push`/`incf` of a bound name writes through the accessor's `setf` place, so a class that exposes only accessors needs no `slot-value`. Lite: the substitution is textual over the body (quoted data is skipped); an inner binding shadowing one of the names is still substituted. ```lisp (defclass wa-point () ((x :initarg :x :accessor wa-x) (y :initarg :y :accessor wa-y))) (let ((p (make-instance 'wa-point :x 3 :y 4))) (with-accessors ((x wa-x) (y wa-y)) p (setf x (+ x y)) (list x y))) ; => (7 4) ``` --- # FILE: references/reference/macros/with-compilation-unit.md # with-compilation-unit `(with-compilation-unit (options...) body...)` Evaluates the body forms in order and returns the last one's value -- a `progn` around the body. The option list (`:override`, and any implementation extension) is accepted and ignored. A `progn` is the whole implementation, and a legitimate one. The options only control how an enclosing unit's deferred-warning report is merged into this one, and there is no [`compile-file`](../functions/compile-file.md) here to defer a warning from: a rontolisp program is compiled whole in one pass, and a loaded file is spliced into it. Libraries that wrap an operation sequence in one -- ASDF wraps every build -- get the dynamic extent they ask for. ```lisp (with-compilation-unit (:override t) 1 2 3) ; => 3 ``` --- # FILE: references/reference/macros/with-input-from-string.md # with-input-from-string `(with-input-from-string (stream string) body...)` Binds `stream` to an input stream reading from `string`, evaluates the body forms, and returns the value of the last one. `read-line` consumes the string line by line and returns nil at the end; `read` parses one datum per line (like `read` on a file stream, it is line-oriented, so the rest of a line after the first datum is skipped). Works in all three backends. ```lisp (with-input-from-string (s "(1 2 3)") (read s)) ; => (1 2 3) ``` Naming the bound variable `*standard-input*` redirects the whole stream-argument-less read family for the extent of the body -- including inside called functions -- because `read-line`, `read-char`, `read` and `peek-char` read the current (dynamically bound) value of `*standard-input*` at call time. A `nil` stream argument is the same designator, so a reader forwarding its own optional argument follows the redirect too; only `t` always names the process standard input. ```lisp (progn (defun next-line (&optional stream) (read-line stream)) (with-input-from-string (*standard-input* "from the string") (next-line))) ; => "from the string" ``` --- # FILE: references/reference/macros/with-open-file.md # with-open-file `(with-open-file (stream filename options...) body...)` Opens the file named by `filename`, binds the open stream to `stream`, evaluates the body forms with that binding, and closes the file afterwards, returning the value of the last body form. On the interpreter and the JVM the expansion wraps the body in [`unwind-protect`](../special-forms/unwind-protect.md), so the file is closed on every exit (normal return, an error signaled in the body, or a `return`/`return-from`); this holds on every backend, including wasm-GC since the exception-handling support landed (a `with-open-file` program compiles in EH mode there and needs `wasmtime -W exceptions=y`, 37+). The supported options are `:direction` -- `:input` (the default) or `:output` -- `:element-type` -- `'character` (the default, a text stream) or `'(unsigned-byte 8)` (a binary stream for `read-byte`/`write-byte`, with the unsized `'unsigned-byte` accepted as the same thing) -- and `:if-exists :append`, which opens an output stream WITHOUT truncating so every write lands at the end of an existing file. `:if-does-not-exist` and `:external-format` are accepted where they name the behavior already in place (`:create`/`:error` and `:utf-8`/`:default`). An option value may be COMPUTED: a function taking the options as arguments and passing them down is how a portable file wrapper opens a file, and such a value is read when the form runs and dispatched onto the matching literal shape. A literal value is still resolved at compile time, so the usual spelling compiles exactly as before. A value outside the supported set signals an error when the form runs. It expands into a plain `open`/`close` pair, so no special stream type is involved. Because it touches the filesystem, `with-open-file` is shown here statically rather than as a runnable example: ```console (with-open-file (s "out.txt" :direction :output) (write-line "hello" s)) (with-open-file (s "out.txt" :direction :input) (read-line s)) ; => "hello" (with-open-file (s "out.bin" :direction :output :element-type '(unsigned-byte 8)) (write-byte 255 s)) ; => 255 (with-open-file (s "out.txt" :direction :output :if-exists :append) (write-line "again" s)) (defun read-first-line (path element-type) (with-open-file (s path :direction :input :element-type element-type) (read-line s))) ``` --- # FILE: references/reference/macros/with-open-stream.md # with-open-stream `(with-open-stream (var stream-form) body...)` Binds `var` to the stream produced by `stream-form`, evaluates the body forms with that binding and closes the stream afterwards, returning the value of the last body form. It is [`with-open-file`](with-open-file.md) without the `open`: the stream is one you already have (a socket, a string stream, the result of a portable `open` call). On the interpreter and the JVM the body is wrapped in [`unwind-protect`](../special-forms/unwind-protect.md), so the stream closes on every exit; the WASM backends keep the close-after-body shape. ```lisp (with-input-from-string (in "hello") (read-line in)) ; => "hello" ``` That is the shorthand; `with-open-stream` is the general form, shown statically because it needs a stream you opened yourself: ```console (with-open-stream (s (open "f.txt" :direction :input)) (read-line s)) ; => "hello" ``` --- # FILE: references/reference/macros/with-output-to-string.md # with-output-to-string `(with-output-to-string (stream) body...)` Binds `stream` to a string output stream, evaluates the body forms, and returns everything written to the stream as a string. `princ`, `prin1`, `print`, `terpri`, `fresh-line`, `write-line`, `write-char` and `write-string` accept the stream as their optional stream argument, and `format` accepts it as the destination; each call appends to the stream. Works in all three backends. ```lisp (with-output-to-string (s) (princ "1 + 2 = " s) (princ (+ 1 2) s)) ; => "1 + 2 = 3" ``` Naming the bound variable `*standard-output*` redirects the whole stream-argument-less print family for the extent of the body -- including inside called functions, and including `format` with the `t` destination -- because those calls read the current (dynamically bound) value of `*standard-output*` at call time. The same redirect works for any `let` that binds `*standard-output*` to an output stream. ```lisp (progn (defun greet () (princ "hello")) (with-output-to-string (*standard-output*) (greet) (format t " ~a" 42))) ; => "hello 42" ``` A `nil` stream argument means the same thing as an omitted one -- it is the `*standard-output*` designator, not "raw standard output". That is what makes the common Common Lisp shape of a renderer forwarding its own optional argument work under the redirect: ```lisp (progn (defun emit (x &optional stream) (princ x stream)) (with-output-to-string (*standard-output*) (emit "forwarded"))) ; => "forwarded" ``` Binding `*error-output*` the same way captures the reports of [`warn`](warn.md) instead; that variable's default is the process standard error, so an unredirected warning never lands on standard output. --- # FILE: references/reference/macros/with-package-iterator.md # with-package-iterator `(with-package-iterator (name package-list symbol-type...) body...)` Lite expansion: binds `name` to a LOCAL FUNCTION (an `flet`, not CL's `macrolet`) that always reports no more symbols -- there is no intern table to iterate, so an iteration loop over it runs zero times (cl-ppcre's `regex-apropos`). The package-list form is evaluated once, for effect. ```lisp (with-package-iterator (next nil :external) (multiple-value-bind (morep sym) (next) (list morep sym))) ; => (NIL NIL) ``` --- # FILE: references/reference/macros/with-simple-restart.md # with-simple-restart `(with-simple-restart (restart-name format-control format-arg...) body...)` Sugar over [`restart-case`](restart-case.md): evaluates `body...` with a restart named `restart-name` established; invoking it returns `(values nil t)` from the `with-simple-restart` form, so the caller can tell "the body was abandoned" from "the body returned nil". The format control becomes the restart's report (lite: the format arguments are accepted and dropped — nothing renders reports). On normal completion the body's values are returned. Supported on every backend except `--no-gc`. ```lisp (handler-bind ((error (lambda (c) (invoke-restart 'skip)))) (multiple-value-list (with-simple-restart (skip "Skip the failing step.") (error "step failed")))) ; => (NIL T) ``` --- # FILE: references/reference/macros/with-slots.md # with-slots `(with-slots (slot-or-pair...) instance body...)` Binds the slot names of a CLOS-subset instance ([`defclass`](../special-forms/defclass.md) / [`define-condition`](define-condition.md)) as symbol-macro-style places for the body: each entry is a slot name, or a `(var slot)` pair binding `var` to slot `slot`. The instance form is evaluated once. Reads see the slots, and [`setf`](setf.md)/`push`/`incf` of a bound name writes back to the slot (the substitution is textual over the body; an inner binding shadowing a slot variable is still substituted). Lite: code GENERATED at run time inside the body (e.g. a `macrolet` template mentioning a slot name) resolves the name through a fallback binding holding the slot's value at entry -- reads work, but a write from such generated code updates that local copy only. `with-slots` only BINDS -- it never reads a slot on entry -- so a body that merely assigns a slot declared without an `:initform` works, and the fallback binding above holds `nil` for such a slot. A read the body really performs still signals `unbound-slot`. ```lisp (defclass buffered () ((buffer))) (let ((b (make-instance 'buffered))) (with-slots (buffer) b (setf buffer (list 1 2))) (slot-value b 'buffer)) ; => (1 2) ``` ```lisp (defclass ws-point () ((x :initarg :x) (y :initarg :y))) (with-slots (x (why y)) (make-instance 'ws-point :x 3 :y 4) (list x why)) ; => (3 4) ``` ```lisp (defclass counter () ((n :initform 0))) (let ((c (make-instance 'counter))) (with-slots (n) c (incf n) (incf n)) (slot-value c 'n)) ; => 2 ``` --- # FILE: references/reference/macros/with-standard-io-syntax.md # with-standard-io-syntax `(with-standard-io-syntax form...)` Binds `*package*` to `cl-user` and evaluates the body as a `progn`. In Common Lisp this macro dynamically rebinds the whole reader/printer control set to standard values so that a body reads and prints independently of the caller's settings; in rontolisp `*package*` is the one variable of that set with a run-time value to rebind (a body's `intern`/`read` homes in `cl-user`, as in Common Lisp). `*read-default-float-format*` is informational (every float shares the one double representation), `*print-circle*` and `*readtable*` exist so library code that reads them loads, and `*print-escape*`/`*print-readably*` exist and hold their standard values (`t`/`nil`) — `*print-escape*` is the one printer variable rontolisp really binds, around a [`print-object`](../functions/print-object.md) method call, so the method can tell [`prin1`](../functions/prin1.md) from [`princ`](../functions/princ.md). The remaining printer-mode variables (`*print-base*`, ...) hold their standard values, and the reader variables (`*read-base*` and friends) do not exist at all, which amounts to being permanently standard. Deviation: `*print-escape*`/`*print-readably*`/`*print-pretty*`, which the printer does honor, are not rebound here, so a non-standard binding of one of them around this form leaks into the body. ```lisp (with-standard-io-syntax (prin1-to-string (list 1 2 3))) ; => "(1 2 3)" (with-standard-io-syntax *package*) ; => :CL-USER ``` --- # FILE: references/reference/macros/write-char.md # write-char `(write-char character &optional stream)` Writes a single character to standard output (or to the given stream), returning the character. Expands to [`write-string`](../functions/write-string.md) of the one-character string on every backend, so it works wherever string output does — including file streams, string streams and [socket handles](../../guides/tcp-sockets.md). Classified as a macro (no function value), like `format`. ```lisp (write-char #\o) (write-char #\k) (terpri) ``` ``` ok ``` --- # FILE: references/reference/packages.md # Packages rontolisp has a small namespace (package) system with a set of built-in packages, plus [user-defined packages via `defpackage`](#user-defined-packages-defpackage): - **`cl`** — the standard package. All built-in functions, macros, special forms and the `*package*` variable belong here. - **`cl-user`** — the default working package. It *uses* `cl`, so standard symbols are available unqualified. The current package when a program starts. User definitions go here. - **`rontolisp`** — a package for implementation-specific symbols. `rl` is a built-in nickname. It does **not** use `cl`. It owns the `version`, `list-functions`, `list-macros` and `list-special-forms` functions. - **`linalg`** — numpy-style vector/matrix operations (`linalg:zeros`, `linalg:matmul`, `linalg:solve`, ...), implemented once in Lisp source and available in every backend. `la` is a built-in nickname. It does **not** use `cl`. See the [Vectors & Matrices guide](../guides/linear-algebra.md). - **`torch`** — a PyTorch-style tensor with reverse-mode automatic differentiation and an `nn`-style module layer, the optimizers and the training-loop plumbing (`torch:tensor`, `torch:matmul`, `torch:backward`, `torch:linear`, `torch:cross-entropy-loss`, `torch:adam`, `torch:step`, ...) over the `linalg` kernels, implemented once in Lisp source and available in every backend. It does **not** use `cl`. See the [Neural Networks guide](../guides/neural-networks.md). - **`java`** — Java interop by reflection, usable only under the JVM interpreter (`java -jar rontolisp.jar`), not the compilers or the native binary. It does **not** use `cl`. It owns `new`, `call`, `static`, `field` and `proxy`; see the [Java interop guide](../guides/java-interop.md). - **`asdf`** — a limited, API-compatible subset of ASDF (system definitions): `defsystem` and `load-system`. It does **not** use `cl`. See the [Systems guide](../guides/asdf-systems.md). - **`ql`** — a limited, API-compatible subset of Quicklisp: `quickload` downloads a system from the real Quicklisp distribution and loads it through the `asdf` subset, and `update-dist` refreshes a dist's index. `quicklisp` is a built-in nickname. It does **not** use `cl`. See the [Systems guide](../guides/asdf-systems.md#downloading-with-quickload). - **`ql-dist`** — Quicklisp's distribution machinery, of which the one member a program writes is `install-dist`: it adds another Quicklisp-format distribution ([Ultralisp](https://ultralisp.org/), or any distinfo URL) to the dists `ql:quickload` searches. It does **not** use `cl`. See the [Systems guide](../guides/asdf-systems.md#adding-a-dist-ultralisp). - **`uiop`** — ASDF's portability layer, registered as 15 sub-packages (`uiop/os`, `uiop/pathname`, ...) that `uiop` re-exports, so both spellings of a member name the same symbol. It does **not** use `cl`. See [The uiop Package](uiop.md). - **`usocket`** — a [usocket](https://github.com/usocket/usocket)-compatible shim over the `rontolisp:tcp-*` socket built-ins (`usocket:socket-connect`, `usocket:socket-listen`, ...), implemented once in Lisp source; also registered as the built-in ASDF system `"usocket"`. It does **not** use `cl`. See the [TCP Sockets guide](../guides/tcp-sockets.md#the-usocket-compatible-shim). A symbol can be referenced with a package qualifier: `package:symbol` (e.g. `cl:car`, `rontolisp:version`) reaches the package's external (exported) symbols, and `package::symbol` reaches any of its symbols, internal ones included — the same single/double colon distinction as Common Lisp (see [External and internal symbols](#external-and-internal-symbols)). `*package*` holds the current package — as the package keyword `find-package` answers, so `(eq *package* (find-package ...))` holds — and `(in-package name)` switches it (the name is a keyword, a symbol, or a string: `:rontolisp`, `rontolisp`, `"rontolisp"`). Like Common Lisp's, `*package*` is a dynamic variable read when a form runs: a function reads the package current at call time, `(let ((*package* ...)) ...)` binds it for the extent, `with-standard-io-syntax` binds it to `cl-user`, and `setq` assigns it. The standard Common Lisp names `common-lisp` and `common-lisp-user` are built-in **nicknames** for `cl` and `cl-user`, so portable `(:use #:common-lisp)` clauses and `common-lisp:car` references resolve; the shorthands `rl` and `la` are built-in nicknames for `rontolisp` and `linalg`, and `quicklisp` for `ql`. User packages can register their own nicknames with the `defpackage` `:nicknames` clause. ```lisp (print *package*) ; => :CL-USER (print (rontolisp:version)) ; the build's own version plist (print (gethash "n" (rl:json-parse "{\"n\": 41}"))) ; => 41 (print (la:to-list (la:from-list '(1 2 3)))) ; => (1.0 2.0 3.0) ``` [`rontolisp:version`](functions/rontolisp-version.md) returns the same information as `rontolisp --version`, as a property list `(:version "0.1.0-SNAPSHOT" :build-timestamp "..." :git-commit "..." :git-branch "...")`. Its timestamp and revision are whatever the running build was made from, so no fixed result is shown for it here. Because the `rontolisp` package does not use `cl`, standard symbols must be qualified with `cl:` inside it, while `version` (which it owns) is available unqualified: ```console (in-package rontolisp) (cl:print (version)) ; the rontolisp package owns version (cl:print (cl:car '(1 2))) ; standard symbols need the cl: prefix here ;; (car '(1 2)) would be an error: Undefined symbol: car (use cl:car) ``` The default package `cl-user` is empty and uses `cl`, so ordinary programs do not need any qualifiers. ## External and internal symbols As in Common Lisp, each package distinguishes its external (exported) symbols from its internal ones, and the two qualifier spellings differ in reach: - `package:symbol` (single colon) references an **external** symbol only. - `package::symbol` (double colon) references **any** symbol of the package, internal ones included. The built-in packages export their entire documented API: every standard `cl` symbol is external, and so are all the `rontolisp` and `java` functions in this manual (so the double colon is never *required* for them, though `rontolisp::version` is also accepted and means the same as `rontolisp:version`). Internal symbols follow the `%` prefix convention — for example `rontolisp::%json-parse`, the fixed-arity helper behind [`rontolisp:json-parse`](functions/rontolisp-json-parse.md) — and are implementation details that may change without notice. `cl-user` exports nothing, like the Common Lisp `COMMON-LISP-USER` package, so on the rare occasion a `cl-user` symbol needs a qualifier it is written `cl-user::name`. A single-colon reference to a non-external symbol is an error at read/compile time: ```console > (rontolisp:%json-parse "1") Error: The symbol %json-parse is not external in the rontolisp package (use rontolisp::%json-parse) ``` A package's export set comes from its definition — the built-in packages export their documented API, a user-defined package its `(:export ...)` clause — and [`export`](functions/export.md)/[`unexport`](functions/unexport.md) adjust it afterwards. A symbol defined while `(in-package rontolisp)` is in effect is interned into the `rontolisp` package as an internal symbol, so from other packages it must be referenced with the double colon. Exporting changes which qualifier *reaches* a symbol, never which symbol it is, so an `export` may come before or after the definitions it publishes. One deviation from Common Lisp: a symbol exported after it was first named keeps the double colon when *printed* — the qualifier is stored with the symbol here rather than recomputed at print time — though both spellings name the same symbol. ## User-defined packages (`defpackage`) New packages are defined with [`defpackage`](special-forms/defpackage.md): ```lisp (defpackage :mypkg (:use :cl) (:export :greet)) (in-package :mypkg) (defun greet (name) (concatenate 'string "hello, " name)) (defun helper () 42) ; not exported: internal (in-package :cl-user) (print (mypkg:greet "world")) ; => "hello, world" (print (mypkg::helper)) ; => 42 ``` Like `in-package`, `defpackage` is a **literal, top-level directive consumed at read/compile time**, so packages are defined in source order, before any use. The supported clauses are `(:use package...)`, `(:export symbol...)`, `(:nicknames name...)`, `(:import-from package symbol...)` and `(:shadow symbol...)`, plus `(:documentation "...")`/`(:size n)` which are accepted and ignored; the name and the clause arguments are keywords, bare symbols, strings, or uninterned symbols (`#:name`, the portable defpackage idiom). A `:shadow`ed name always resolves to the package's own symbol inside the package -- never to the `cl` (or any used package's) symbol of the same name -- so a library can define its own `digit-char-p` or `defconstant`. `:shadowing-import-from` is an error, and so is any other clause, redefining an existing package, or using a package that does not exist yet. - `:use` makes the **external** symbols of the used packages visible unqualified, as in Common Lisp — internal symbols of a used package still need the double colon. Without a `:use` clause nothing is inherited (like SBCL), so `cl` symbols would need the `cl:` prefix; ordinary packages should say `(:use :cl)` (or, portably, `(:use #:common-lisp)`). When several used packages export the same name, the first package in `:use` order wins (Common Lisp signals a conflict instead). - `:export` declares the package's external symbols. Symbols interned later (a `defun` under `(in-package name)` that is not in the `:export` clause, a free variable) are internal, exactly like the built-in packages. - `:nicknames` registers alternate names that resolve everywhere the canonical name does (in qualifiers, `in-package`, `:use`, ...). A nickname colliding with an existing package or nickname is an error — the built-in nicknames (`common-lisp`, `common-lisp-user`, `rl`, `la`, `quicklisp`) are reserved the same way as the built-in package names. - `:import-from` makes the named symbols of one package visible unqualified without using the whole package. Resolution is textual: an imported name resolves to the source package's canonical spelling, so importing and then re-exporting a symbol makes `mypkg:name` refer to the original definition. [`use-package`](functions/use-package.md) is the runtime form of the `:use` clause, and follows the same read/compile-time rule as `in-package`: a literal top-level `(use-package :mypkg)` widens the current package's use list for the forms that follow it, on every backend. [`export`](functions/export.md), [`unexport`](functions/unexport.md) and [`import`](functions/import.md) follow the same rule. `make-package` and `rename-package` are not available, and a `defpackage` inside another form (not top-level) is an error. ## Package introspection `rontolisp:list-functions`, `rontolisp:list-macros` and `rontolisp:list-special-forms` return the symbols of a package by category, sorted alphabetically. The optional argument is a package designator — a keyword, a bare symbol, a quoted symbol or a string (`:cl`, `cl`, `'cl`, `"cl"`) — and defaults to `:cl`. An unknown package is an error (`No such package: foo`). ```lisp (print (rontolisp:list-macros)) ; => (AND ASSERT BLOCK CASE CCASE CERROR CHANGE-CLASS CHECK-TYPE COMPLEMENT COMPLEX COND CTYPECASE DECF DECLAIM DECLARE DEFINE-COMPILER-MACRO DEFINE-CONDITION DEFINE-MODIFY-MACRO DEFINE-SETF-EXPANDER DEFSETF DEFTYPE DESTRUCTURING-BIND DO DO* DO-EXTERNAL-SYMBOLS DO-SYMBOLS DOCUMENTATION DOLIST DOTIMES ECASE ERROR ETYPECASE EVAL-WHEN FLET FORMAT HANDLER-BIND HANDLER-CASE IGNORE-ERRORS INCF LABELS LET* LOAD-TIME-VALUE LOCALLY LOOP MACROLET MAKE-CONDITION MAKE-INSTANCE MAKE-SEQUENCE MULTIPLE-VALUE-BIND MULTIPLE-VALUE-CALL MULTIPLE-VALUE-LIST MULTIPLE-VALUE-PROG1 MULTIPLE-VALUE-SETQ NTH-VALUE OR POP PPRINT-LOGICAL-BLOCK PRINT-UNREADABLE-OBJECT PROCLAIM PROG PROG* PROG1 PROG2 PSETF PSETQ PUSH PUSHNEW REMF RESTART-BIND RESTART-CASE RETURN-FROM ROTATEF SETF SHIFTF SIGNAL SLOT-BOUNDP SLOT-EXISTS-P SLOT-MAKUNBOUND SLOT-VALUE SYMBOL-MACROLET THE TIME TYPECASE TYPEP UNLESS WARN WHEN WITH-ACCESSORS WITH-COMPILATION-UNIT WITH-INPUT-FROM-STRING WITH-OPEN-FILE WITH-OPEN-STREAM WITH-OUTPUT-TO-STRING WITH-PACKAGE-ITERATOR WITH-SIMPLE-RESTART WITH-SLOTS WITH-STANDARD-IO-SYNTAX WRITE-CHAR) (print (rontolisp:list-special-forms)) ; => (CATCH DEFCLASS DEFCONSTANT DEFGENERIC DEFMACRO DEFMETHOD DEFPACKAGE DEFPARAMETER DEFSTRUCT DEFUN DEFVAR FUNCTION GO IF IN-PACKAGE LAMBDA LET PROGN PROGV QUOTE RETURN SETQ TAGBODY THROW UNWIND-PROTECT WHILE) (print (length (rontolisp:list-functions))) ; => 436 (defun square (x) (* x x)) (print (rontolisp:list-functions :cl-user)) ; => (SQUARE) (print (rontolisp:list-functions :rontolisp)) ; => (AWAIT CATCH FETCH FINALLY HTTP-HANDLER JSON-PARSE JSON-STRINGIFY LIST-FUNCTIONS LIST-MACROS LIST-SPECIAL-FORMS MAKE-MUTEX MUTEX-ACQUIRE MUTEX-RELEASE QUERY-PARAM QUERY-PARAMS RANDOM-BYTES TCP-ACCEPT TCP-CONNECT TCP-LISTEN TCP-LOCAL-ADDRESS TCP-LOCAL-PORT TCP-PEER-ADDRESS TCP-PEER-PORT TCP-SET-TIMEOUT THEN THEN* TLS-CONNECT TLS-LISTEN TLS-LISTEN-PEM TLS-UPGRADE URL-DECODE URL-ENCODE URL-PATH URL-QUERY VERSION WIT-ERROR-PAYLOAD WIT-PROVIDE) (print (rontolisp:list-functions :java)) ; => (CALL FIELD NEW PROXY STATIC) ``` The classification follows the function namespace: a name is listed as a function exactly when it is usable as a function value via `#'name` (so `first`, `length`, `1+`, ... are functions even though they compile via inline expansion), and `list-macros`/`list-special-forms` list the operators that have no function value. Notes: - `list-functions` of `cl-user` lists the user-defined functions (`defun`s); names that are package-qualified, `%`-prefixed internals or shadow a `cl` symbol are excluded. In compiled output it is a **compile-time snapshot** of the program's `defun`s — functions defined at runtime through `load`/`eval` (even with `--dynamic`) are not included, and functions defined while `(in-package :rontolisp)` is in effect are not listed for any package. - `list-functions` of a [user-defined package](#user-defined-packages-defpackage) lists the package's `defun`s under their canonical qualified names — `mypkg:fn` for an exported function, `mypkg::fn` for an internal one. `list-macros` and `list-special-forms` of a user package are `nil`. - Car/cdr compositions (`cadr`, `caddr`, ...) are recognized by pattern, not enumerated, so they do not appear in `list-functions`. - The package designator must be a literal; a computed designator is rejected at read/compile time (the interpreter additionally accepts a computed designator through `funcall`, where an unknown package yields `nil` instead of an error — user packages are known only at read/compile time). - Like `version`, these functions are not supported inside the compiled runtime `eval`/`load`. Packages are resolved at read/compile time (in source order), so `in-package` is a top-level directive: which package a symbol in the source belongs to is decided by the `in-package` above it, not by a runtime `setq` of `*package*` (in compiled output the whole file is resolved before it runs; the interpreter resolves each top-level form as it reaches it, so a runtime assignment does affect the forms after it there). In compiled output a runtime-loaded file's package directives are not processed; the `rontolisp` package's functions (`version`, `list-functions`, ...) are not available as first-class values (they cannot be passed to `mapcar`/`funcall`); and a `cl` symbol name must not be shadowed as a local variable inside a package that does not use `cl`. ## rontolisp Package Extensions The symbols the `rontolisp` package owns are **implementation-specific and not part of Common Lisp**. They must be referenced with the `rontolisp:` qualifier (or used unqualified after `(in-package rontolisp)`). Besides the introspection helpers above (`version`, `list-functions`, `list-macros`, `list-special-forms`), the package provides asynchronous outgoing HTTP via `rontolisp:fetch` (which returns a future) together with `rontolisp:await` (resolve) and `rontolisp:futurep` (type predicate), and JSON conversion via [`rontolisp:json-parse`](functions/rontolisp-json-parse.md) / [`rontolisp:json-stringify`](functions/rontolisp-json-stringify.md) (JavaScript `JSON.parse`/`JSON.stringify` style). All of these have their own pages in the [Functions](functions.md#rontolisp-package-functions) reference, including the full [`rontolisp:fetch`](functions/rontolisp-fetch.md) / [`rontolisp:await`](special-forms/rontolisp-await.md) / [`rontolisp:futurep`](functions/rontolisp-futurep.md) documentation. Two members of the package are neither functions nor macros but read-time literals: `rontolisp:current-file` and `rontolisp:current-line`, which the reader replaces with the position they stand on. Because they are resolved before any `in-package` directive is interpreted, they are the exception to the rule above — they must always be written qualified. See [Source position literals](data-types.md#source-position-literals-rontolispcurrent-file-rontolispcurrent-line). --- # FILE: references/reference/special-forms.md # Special Forms **Each form name in the table links to its own page**, with a fuller description and a runnable example you can evaluate in your browser. | Form | Syntax | Description | |------|--------|-------------| | `quote` | `(quote expr)` or `'expr` | Returns the expression unevaluated | | `if` | `(if cond then else?)` | Conditional. `nil` is false, everything else is true | | `let` | `(let ((x 1) (y 2)) body...)` | Local variable bindings (parallel). A name proclaimed special (`defvar`/`declaim`) is bound dynamically instead of lexically | | `progv` | `(progv symbols values body...)` | Dynamically bind a runtime-computed list of `symbols` to `values` for the body, restored on exit (interpreter only) | | `lambda` | `(lambda (params...) body...)` | Anonymous function | | `progn` | `(progn expr1 expr2...)` | Evaluate expressions in sequence, return the last | | `setq` | `(setq name value ...)` | Assign values to variables; accepts multiple `name value` pairs, assigned left to right, and returns the last value | | `while` | `(while test body...)` | Evaluate body repeatedly while test is non-nil. Returns nil | | `return` | `(return value?)` | Non-local exit from the nearest enclosing loop (`do`/`dolist`/`dotimes`/`loop`), which evaluates to `value` (or nil) | | `unwind-protect` | `(unwind-protect protected cleanup...)` | Evaluate `protected` and run the `cleanup` forms on every exit from it -- normal return, `error` unwind, or `return`/`return-from` (on wasm-GC needs `wasmtime -W exceptions=y`; compile error under `--no-gc`) | | `defun` | `(defun name (params...) body...)` | Define a function in the function namespace. Returns the function name | | `defmacro` | `(defmacro name (params...) body...)` | Define a user macro; a call is expanded (the body runs with unevaluated argument forms bound) and the expansion is evaluated. Supports `&rest`/`&body`. Returns the name | | `defclass` | `(defclass name (super?) ((slot options...)...))` | Define a class (static CLOS subset: single inheritance; `:initarg`/`:initform`/`:reader`/`:accessor` slot options). Returns the name | | `defgeneric` | `(defgeneric name (param...))` | Define a generic function dispatching on its first argument. Returns the name | | `defmethod` | `(defmethod name (param...) body...)` | Add a method to a generic function; the first parameter may carry an `(var (eql literal))`, class, or built-in-type specializer. Returns the name | | `defvar` | `(defvar name value?)` | Define a global variable and proclaim it special, binding `value` only if `name` is not already bound (idempotent). With no `value`, leaves it unbound. Returns the name | | `defparameter` | `(defparameter name value)` | Define a global variable and proclaim it special, **always** (re)binding `value` even if `name` is already bound. Returns the name | | `defconstant` | `(defconstant name value)` | Like `defparameter` (rontolisp does not enforce constancy). Returns the name | | `function` | `(function name)` or `#'name` | Look up a function in the function namespace and return it as a value | | `defpackage` | `(defpackage name (:use ...) (:export ...))` | Define a new package (a top-level, read/compile-time directive; `:use` and `:export` clauses only). Returns the name | | `rontolisp:async` | `(rontolisp:async (defun ...))` or `(rontolisp:async (lambda ...))` | Turn the wrapped `defun`/`lambda` into its asynchronous counterpart (`async-defun`/`async-lambda`) — the JavaScript-style spelling | | `rontolisp:async-defun` | `(rontolisp:async-defun name (params...) body...)` | Define an asynchronous function: calling it starts the body eagerly and returns a future that settles with the body's value (or error) | | `rontolisp:async-lambda` | `(rontolisp:async-lambda (params...) body...)` | Anonymous asynchronous function; each invocation returns a future | | `rontolisp:await` | `(rontolisp:await value)` | Suspend the current asynchronous function until a future settles and return its value; a non-future passes through unchanged. Legal only in `async-defun`/`async-lambda` bodies and at top level | | `tagbody` | `(tagbody tag-or-form...)` | Body forms with go tags: `go` jumps to a tag (forward or backward), falling off the end returns nil | | `go` | `(go tag)` | Transfer control to a tag of the enclosing `tagbody` (compiled `go` is lexical, but one crossing a `lambda` re-enters the `tagbody` as a non-local exit) | | `catch` | `(catch tag body...)` | Establish a dynamic exit point named by `tag` (an `eq`-compared runtime value): the form's value is the body's, or the value of a matching `throw` fired in its dynamic extent (on wasm-GC needs `wasmtime -W exceptions=y`; compile error under `--no-gc`) | | `throw` | `(throw tag result)` | Transfer control (and `result`) to the innermost active `catch` with an `eq` tag, running every intervening `unwind-protect` cleanup; a `handler-case` in between does not catch it | rontolisp is a **Lisp-2** like Common Lisp: functions and variables live in separate namespaces. A bare symbol evaluates as a variable (`car` alone is an unbound-variable error), a symbol in call position resolves in the function namespace only (a variable named `car` does not shadow the function `car`), and a function is obtained as a value with `#'name`, `(function name)` or `(symbol-function 'name)`. See [Function Namespace and First-Class Functions](function-namespace.md). --- # FILE: references/reference/special-forms/catch.md # catch `(catch tag body...)` Establishes a **dynamic** exit point named by `tag` and returns the value of the last `body` form -- or, if a [`throw`](throw.md) to an `eq` tag fires anywhere in the body's dynamic extent, the thrown value. The tag is an ordinary runtime value evaluated once on entry (usually a quoted symbol), so unlike [`block`](../macros/block.md)/`return-from` the thrower does not need the catcher in lexical scope: it only has to run while the `catch` is active. The innermost active `catch` with a matching tag wins; a non-matching one lets the exit pass through. `catch`/`throw` work on **every backend** except `--no-gc` (a compile error there). On the wasm-GC backends (Preview 1 and `--component`) they compile through the WebAssembly exception-handling proposal, so running a program that uses them needs wasmtime 37+ with `-W exceptions=y`, exactly like [`unwind-protect`](unwind-protect.md) and `handler-case`. A `throw` is a non-local exit, not a condition: it unwinds **through** a `handler-case` without being caught, while every intervening `unwind-protect` cleanup does run. ```lisp (catch 'done (throw 'done :thrown) :not-reached) ; => :THROWN ``` The exit crosses function boundaries, which is what makes it useful for bailing out of a callback: ```lisp (defun first-even (xs) (catch 'found (map nil (lambda (x) (if (evenp x) (throw 'found x))) xs) :none)) (list (first-even '(1 3 4 5)) (first-even '(1 3 5))) ; => (4 :NONE) ``` Tags are compared with `eq`, so a freshly consed tag matches only itself: ```lisp (catch 'outer (catch (list 1) (throw 'outer :to-outer))) ; => :TO-OUTER ``` --- # FILE: references/reference/special-forms/defclass.md # defclass `(defclass name (superclass*) ((slot slot-option...) ...) class-option...)` Defines a class and returns the name symbol. This is a **static CLOS subset**: every superclass must be defined by an earlier `defclass`, and instances are first-class objects created with [`make-instance`](../macros/make-instance.md) (like [`defstruct`](defstruct.md) instances they are not lists — `consp` is `nil` — and `print` shows them as `#`). Slot options: - `:initarg keyword` — the constructor keyword for the slot (defaults to the slot-name keyword) - `:initform expr` — the default value, evaluated at construction time when the slot is not supplied. **Omitting it leaves the slot UNBOUND**, as in CL: [`slot-boundp`](../macros/slot-boundp.md) is `nil` and a read signals `unbound-slot` - `:reader fn` — defines `fn` as a reader function - `:accessor fn` — like `:reader`, and additionally a `setf`-able place A subclass inherits every slot of its superclasses and its instances match the superclasses' [`defmethod`](defmethod.md) class specializers. **Multiple inheritance** is supported: the first superclass's slots come first and each later superclass appends the slots whose name is not present yet, a diamond keeps ONE copy of the shared slot, and both method dispatch and re-declared slot options follow the **class precedence list** (CLHS 4.3.5 -- the class, then its superclasses left to right, each before its own superclasses; conflicting local orders are an error). A subclass may re-declare an inherited slot: the storage stays the one inherited slot, while the subclass's `:initform`/`:initarg` override the inherited ones and its readers/accessors are added to them. Reader/accessor functions are ordinary defuns, so they are first-class. The class options `(:documentation "...")` (accepted and ignored) and `(:default-initargs :initarg value ...)` (defaults applied by [`make-instance`](../macros/make-instance.md) for initargs not supplied) are supported. Beyond `:reader`/`:accessor`, the slot options `:type` (recorded, no checking), `:writer` (a symbol names a two-argument new-value-first generic, `(setf name)` the setf function of `name`) and `:allocation` are supported: `:allocation :class` stores the slot's value in ONE cell shared by the declaring class, its instances and its subclasses — every access path (accessors, [`slot-value`](../macros/slot-value.md), a `make-instance` initarg) reads and writes the shared cell, the `:initform` is evaluated once at class-definition time, and a subclass re-declaring the slot with `:allocation :class` gets a cell of its own (CLHS 7.5.3). Without a `:metaclass` (below), other class options and other slot options are errors. On the compilation path `defclass` is only supported as a top-level form; [`find-class`](../functions/find-class.md) and [`class-of`](../functions/class-of.md) answer the class metaobject, and of the runtime class operations only [`change-class`](../macros/change-class.md) exists (the target may be computed). Evaluating a `defclass` for an **existing name** updates the definition — a `:metaclass` class re-runs the class-definition protocol through `reinitialize-instance` on the SAME metaobject (see below) — but existing instances are not updated (`update-instance-for-redefined-class` does not exist), and on the compilation path the statically compiled parts (slot layout, constructors, accessors) always follow the LAST definition. **Metaclasses** (the static MOP subset): the class option `(:metaclass M)` names a class inheriting `standard-class`, defined earlier by `defclass`. The class definition then runs the class-definition protocol at **definition time**: the metaclass is instantiated — its `shared-initialize` methods see every other unknown class option as an initarg whose value is the option's tail list, and a `:before` method's write to a slot with a declared `:initarg` is overwritten by the supplied initarg afterwards, CL's fill order (a slot without a declared `:initarg`, like `table-name` below, keeps the `:before`'s write) — each slot's non-standard options (`:col-type`, ...) are handed to `closer-mop:direct-slot-definition-class` as initargs and its answer is instantiated as that slot's direct-slot-definition metaobject, effective slots are computed through `closer-mop:compute-effective-slot-definition` (the default method picks and instantiates `closer-mop:effective-slot-definition-class` inside the dynamic extent of a user override's `call-next-method`), and `closer-mop:finalize-inheritance` runs **eagerly** (CL finalizes lazily; inputs are static, so only the timing of definition errors differs). `find-class` and `class-of` answer the metaclass instance from then on, while instances of the class itself stay ordinary objects. The definition routes through `closer-mop:ensure-class-using-class` — its user `:around` methods dispatch on the EXISTING class metaobject, so they fire when the same name is defined again (the first definition passes `nil`), and the redefinition takes the `reinitialize-instance` path on the same metaobject. Initialization runs through the ordinary generic chain, so user `initialize-instance`/`reinitialize-instance` `:around` methods on the metaclass (or on a slot-definition class) may rewrite the initargs — `:direct-superclasses` and `:direct-slots` included — before `(apply #'call-next-method ...)`, as mito's table classes do. Slot-definition metaobjects also carry `closer-mop:slot-definition-initfunction` (a zero-argument function evaluating the `:initform`; `nil` when the slot has none). The protocol is static: a `defclass` in a non-top-level position, or protocol calls on classes unknown at definition time, signal an error. ```lisp (defclass animal () ((name :initarg :name :accessor animal-name))) (defclass dog (animal) ((breed :initarg :breed :initform "mixed" :reader dog-breed))) (setq d (make-instance 'dog :name "Rex")) (list (animal-name d) (dog-breed d)) ; => ("Rex" "mixed") ``` ```lisp (defclass shape () ((sides :initform 0 :reader sides) (label :initarg :label))) (defclass square (shape) ((sides :initform 4 :accessor square-sides))) (let ((s (make-instance 'square))) (list (sides s) (square-sides s) (slot-boundp s 'label))) ; => (4 4 NIL) ``` ```lisp (defclass fast-stream () ((buffer :initform nil :accessor stream-buffer))) (defclass output-mixin () ((pos :initform 0 :accessor stream-pos))) (defclass fast-output-stream (fast-stream output-mixin) ()) (defgeneric stream-kind (s)) (defmethod stream-kind ((s fast-stream)) :fast) (defmethod stream-kind ((s output-mixin)) :output) (let ((s (make-instance 'fast-output-stream))) (setf (stream-pos s) 10) (list (stream-kind s) (stream-pos s) (typep s 'output-mixin))) ; => (:FAST 10 T) ``` ```lisp (defclass counter () ((n :initform 0 :accessor counter-n))) (setq c (make-instance 'counter)) (incf (counter-n c)) (setf (counter-n c) (+ (counter-n c) 10)) (counter-n c) ; => 11 ``` ```lisp (defclass table-class (standard-class) ((table-name))) (defmethod closer-mop:validate-superclass ((c table-class) (s standard-class)) t) (defmethod shared-initialize :before ((c table-class) slots &key table-name &allow-other-keys) (if table-name (setf (slot-value c 'table-name) (car table-name)) nil)) (defclass account () ((id :initarg :id)) (:metaclass table-class) (:table-name "accounts")) (list (class-name (find-class 'account)) (slot-value (find-class 'account) 'table-name) (typep (find-class 'account) 'table-class)) ; => (ACCOUNT "accounts" T) ``` --- # FILE: references/reference/special-forms/defconstant.md # defconstant `(defconstant name value)` Defines a global named `name` bound to `value`, evaluating `value` and returning the name symbol. It behaves like `defparameter` -- rontolisp does not enforce constancy, so the binding can still be changed afterward. It is intended to document values meant to stay fixed. ```lisp (defconstant +pi+ 3.14159) ; => +PI+ ``` ```lisp (defconstant +pi+ 3.14159) +pi+ ; => 3.14159 ``` --- # FILE: references/reference/special-forms/defgeneric.md # defgeneric `(defgeneric name (param...) option...)` Defines a generic function and returns the name symbol. Methods are added with [`defmethod`](defmethod.md) — specializers may appear on **any** required parameter, and a call runs the most specific matching method (parameters ranked leftmost-first); calling the generic with no matching method signals an error. A `defgeneric` is optional — the first `defmethod` implicitly creates the generic — but declares the lambda list every method must match. The generic function is an ordinary function, so `#'name` and `funcall` work. `name` may also be a setf function name `(setf reader)` (see [defmethod](defmethod.md)). The lambda list may continue past the required parameters with `&optional`/`&rest` (the dispatcher forwards the tail to the selected method), and inline `(:method [qualifier] (param...) body...)` clauses define methods in the `defgeneric` itself. `(:documentation "...")` is recorded and ignored. `(:method-combination NAME [:most-specific-first | :most-specific-last])` selects one of the CLHS **short-form** combinations — `progn`, `and`, `or`, `+`, `list`, `nconc`, `append`, `max`, `min`. The effective method is then that operator applied to EVERY applicable method whose qualifier is the combination name, most specific first (`:most-specific-last` reverses the order); `:around` methods wrap the combined form as usual, while `:before`/`:after` are rejected, as CLHS requires. A primary method must carry the combination name as its qualifier: `(defmethod encode-slots progn ((o point)) ...)`. ```lisp (defclass shape () ()) (defclass square (shape) ()) (defgeneric describe-parts (x) (:method-combination list)) (defmethod describe-parts list ((x shape)) 'shape) (defmethod describe-parts list ((x square)) 'square) (describe-parts (make-instance 'square)) ; => (SQUARE SHAPE) ``` Lite subset: `&key` in the generic's lambda list, `define-method-combination` (the long form) and the remaining options are errors. ```lisp (defgeneric area (shape) (:documentation "The area of a shape.")) (defmethod area (shape) 0) (defmethod area ((shape (eql :unit-square))) 1) (list (area :unit-square) (area :dot) (funcall #'area :unit-square)) ; => (1 0 1) ``` Calling a generic that has no applicable method signals an error (`No applicable method: G on INTEGER`), so it is shown here statically rather than as a runnable example: ```console (defgeneric g (x)) (g 1) ``` --- # FILE: references/reference/special-forms/defmacro.md # defmacro `(defmacro name lambda-list body...)` Defines a user macro named `name` and returns the name symbol. A macro call receives its argument forms **unevaluated**: the `body` runs at expansion time with the parameters bound to the raw forms, and the form it returns (the expansion) is evaluated in place of the call. The lambda list is a macro lambda list, destructured over the argument forms like [`destructuring-bind`](../macros/destructuring-bind.md): patterns nest in required positions, and `&optional` (with defaults), `&rest`/`&body`, `&key`, and `&aux` are supported; `&whole` is not; `&environment` is accepted and its parameter is bound to nil (there is no environment object), which suffices to thread it into `constantp`/`get-setf-expansion`. With such an extended lambda list, matching is lenient (a missing position binds to nil, surplus forms are ignored); a plain lambda list (required parameters plus one trailing `&rest`/`&body`) keeps the strict argument-count check. A standard operator (`when`, `setf`, ...) cannot be redefined, and a macro has no function value (`#'name` is an error). Macro bodies usually build the expansion with the backquote template syntax, which is also available anywhere else in a program: - `` `form `` quotes `form` except where a comma unquotes it - `,expr` inserts the value of `expr` - `,@expr` splices the value of `expr` (a list) into the surrounding list - `',@expr` (a splice directly after `'` or `#'`) builds a `(quote ...)`/`(function ...)` list from the spliced elements; with the customary one-element splice the result reads back as `'x`/`#'x` Nested backquote (a backquote template inside another) is supported and fully expanded at read time, so classic macro-writing macros such as `once-only` work. Use [`gensym`](../functions/gensym.md) to generate capture-safe temporaries in macro bodies, and [`macroexpand-1`](../functions/macroexpand-1.md)/[`macroexpand`](../functions/macroexpand.md) to inspect an expansion. The interpreter expands macro calls at evaluation time (so `defmacro` also works in the REPL and via `load`/`eval`). On the compilation path the CLI fully expands every macro call **before** the JVM/WASM compilers run and removes the definitions, so compiled output contains only ordinary forms; consequently the runtime `eval`/`read` of a compiled program does not know `defmacro` or the backquote character, and a macro must be defined before its first use. ```lisp (defmacro my-unless (test &body body) `(if ,test nil (progn ,@body))) (my-unless (> 1 3) 'a 'b) ; => B ``` ```lisp (defmacro swap! (a b) `(let ((__tmp ,a)) (setq ,a ,b) (setq ,b __tmp))) (setq x 1) (setq y 2) (swap! x y) (list x y) ; => (2 1) ``` ```lisp (defmacro with-point ((x y) form &key (scale 1)) `(destructuring-bind (,x ,y) ,form (list (* ,x ,scale) (* ,y ,scale)))) (with-point (px py) '(3 4) :scale 10) ; => (30 40) ``` --- # FILE: references/reference/special-forms/defmethod.md # defmethod `(defmethod name [qualifier] (param... ) body...)` Adds a method to the generic function `name` (creating it when no [`defgeneric`](defgeneric.md) preceded it) and returns the name symbol. **Any** required parameter may carry a specializer, written `(var specializer)`: - `(var (eql literal))` — matches when the first argument is the literal (a keyword, quoted symbol, number, or character) - `(var class-name)` — matches instances of a [`defclass`](defclass.md) class and its subclasses - `(var struct-name)` — matches instances of a [`defstruct`](defstruct.md) type (the dispatcher tests the instance tag, like the struct predicate) - `(var type-name)` — matches a built-in type (`integer`, `float`, `number`, `string`, `symbol`, `keyword`, `character`, `cons`, `list`, `null`, `hash-table`, `function`, `pathname`, `package`, ...). A `package` parameter matches exactly what [`typep`](../macros/typep.md) calls a package, and is tried BEFORE `keyword`/`symbol`, so the designator idiom "a `package` method plus an unspecialized method that calls [`find-package`](../functions/find-package.md) and recurses" terminates - `(var t)` or a plain `var` — the default method A call runs the most specific matching method: parameters are ranked leftmost-first, and per parameter `eql` methods win over class methods (subclass before superclass), then built-in types (subtypes such as `integer` before their supertypes such as `number`), then the default; with no match the call signals an error. Defining the same specializer combination again replaces the previous method. The lambda list may continue past the required parameters with `&optional`/`&rest` (the dispatcher forwards the tail via `apply`). The body may start with a docstring and `(declare ...)` (both are ignored). ```lisp (defclass animal () ()) (defclass dog (animal) ()) (defgeneric speak (x)) (defmethod speak ((x dog)) "woof") (defmethod speak ((x animal)) "some sound") (defmethod speak ((x integer)) "a number") (defmethod speak ((x (eql :cat))) "meow") (defmethod speak (x) "?") (list (speak (make-instance 'dog)) (speak (make-instance 'animal)) (speak 42) (speak :cat) (speak "s")) ; => ("woof" "some sound" "a number" "meow" "?") ``` ## Setf methods `name` may also be the function name `(setf reader)`: the method becomes part of the *setf function* of `reader`, and `(setf (reader arg...) value)` dispatches through it with the new value as the FIRST parameter (CL's setf-function argument order). A setf method on the same name as a [`defclass`](defclass.md) `:accessor` merges with the accessor's writer methods instead of shadowing them, and `#'(setf reader)` is the writer as a first-class function. `(defgeneric (setf reader) ...)` works the same way, inline `(:method ...)` clauses included. ```lisp (defclass sbox () ((v :initarg :v :reader content))) (defmethod (setf content) (new (b sbox)) (setf (slot-value b 'v) new)) (let ((b (make-instance 'sbox :v 1))) (setf (content b) 42) (content b)) ; => 42 ``` ## Method qualifiers and `call-next-method` An optional `:before`, `:after`, or `:around` **qualifier** before the lambda list adds an auxiliary method (standard method combination). For one call: - every applicable `:around` method runs, most specific first, each wrapping the rest; - then every `:before` method runs for effect, most specific first; - then the most specific applicable primary (unqualified) method runs — its value is the result; - then every `:after` method runs for effect, **least** specific first. Under a short-form [`:method-combination`](defgeneric.md) the qualifier set is different: a primary method carries the COMBINATION NAME instead (`(defmethod total + ((x account)) ...)`), `:around` still wraps, and `:before`/`:after` are rejected. Inside a primary or `:around` method, `(call-next-method)` invokes the next less specific method (passing the current arguments, or new ones if given as `(call-next-method arg...)`), and `(next-method-p)` returns whether such a method exists. Calling `call-next-method` with no next method signals an error. ```lisp (defclass point () ((x :initarg :x :accessor px))) (defclass point3d (point) ((z :initarg :z :accessor pz))) (defgeneric describe-point (p)) (defmethod describe-point ((p point)) (list :x (px p))) (defmethod describe-point ((p point3d)) (append (call-next-method) (list :z (pz p)))) (defmethod describe-point :around ((p point)) (list :point (call-next-method))) (describe-point (make-instance 'point3d :x 1 :z 3)) ; => (:POINT (:X 1 :Z 3)) ``` Lite subset: `&key` is an error, and standard method combination is supported for class and default methods (an `:around`/`:before`/`:after` with an `eql` or built-in-type specializer combines only with primaries of the same specializer plus the default method). On the compilation path `defmethod` is only supported as a top-level form; the dispatched method set of a compiled program is fixed at compile time. ## A method on a built-in name Defining a method on the name of a built-in function (`close`, `open-stream-p`, `stream-element-type`, ...) makes the **built-in the generic function's default method**: instances of the specialized class run the method, and every other argument keeps the built-in behavior — including through a `(call-next-method)` out of the least specific primary method. A method on `close` for your own stream class therefore leaves `(close stream)` on a real file stream working. A user default (unspecialized) method still replaces the built-in outright. ```lisp (defclass counter () ((n :initform 3))) (defmethod length ((c counter)) (slot-value c 'n)) (list (length (make-instance 'counter)) (length "abcd")) ; => (3 4) ``` Lite subset: this works on every backend — the compilation paths route calls to such a name through the generated dispatcher, whose fall-through is the original built-in. Only names backed by a native built-in function participate; a name implemented as an expansion (`mapcar`, `sort`, `format`, ...) cannot take methods, and a plain `defun` on a built-in name is still ignored on the compilation paths. --- # FILE: references/reference/special-forms/defpackage.md # defpackage `(defpackage name (:use package...) (:export symbol...) (:nicknames name...) (:import-from package symbol...) (:local-nicknames (nick actual)...))` Defines a new package named `name` and returns the name symbol. Like `in-package`, it is a literal, top-level directive consumed at read/compile time, so packages are defined in source order. The name and the clause arguments are keywords, bare symbols, strings, or uninterned symbols (`:mypkg`, `mypkg`, `"mypkg"`, `#:mypkg` — the last is the portable defpackage idiom). - `(:use package...)` makes the external (exported) symbols of the listed packages visible unqualified. The used packages must already exist. Without a `:use` clause **nothing** is visible unqualified — write `(:use :cl)` to use the standard symbols without a `cl:` prefix. `common-lisp` and `common-lisp-user` are built-in nicknames for `cl` and `cl-user`, so `(:use #:common-lisp)` works too. - `(:export symbol...)` declares the package's external symbols: they are reachable as `name:symbol` from other packages, and inherited by packages that use this one. Symbols interned later (for example `defun`s made under `(in-package name)` that are not in the `:export` clause) are internal and require the double colon, `name::symbol`. - `(:nicknames name...)` registers alternate names for the package; a nickname resolves everywhere the canonical name does. A nickname that collides with an existing package (or nickname) is an error; the built-in nicknames — `common-lisp` and `common-lisp-user` (`cl`/`cl-user`), `rl` (`rontolisp`), `la` (`linalg`) and `quicklisp` (`ql`) — are reserved the same way. - `(:import-from package symbol...)` makes the named symbols of `package` visible unqualified without using the whole package. Resolution is textual: an imported name resolves to the source package's canonical spelling, so `(:import-from #:common-lisp #:car)` gives a use-nothing package just `car`. - `(:local-nicknames (nick actual)...)` registers a shorthand for **another** package, so `nick:symbol` resolves like `actual:symbol` — the idiom libraries use to shorten long package names. Lite: the nickname is **global** (rontolisp has no per-package nickname scoping), so it follows the same collision rules as `:nicknames`. The same registration is available outside a `defpackage` as [`uiop:add-package-local-nickname`](../functions/uiop-add-package-local-nickname.md). - `(:documentation "...")` and `(:size n)` are accepted and ignored. Redefining an existing package is an error, and so are `:shadow`/`:shadowing-import-from` (rontolisp has no symbol shadowing) and any other clause (`:intern`, ...). See [Packages](../packages.md#user-defined-packages-defpackage) for the full rules. ```lisp (defpackage :util (:use :cl) (:export :trim)) ; => UTIL ``` ```lisp (defpackage #:mypkg (:use #:common-lisp) (:nicknames #:mp) (:import-from #:rontolisp #:version) (:export #:greet)) (in-package :mypkg) (defun greet (name) (concatenate 'string "hello, " name)) (in-package :cl-user) (mp:greet "world") ; => "hello, world" ``` --- # FILE: references/reference/special-forms/defparameter.md # defparameter `(defparameter name value)` Defines a global variable `name` and binds it to `value`, evaluating `value` and **always** (re)assigning even if `name` is already bound -- unlike `defvar`, which only binds an unbound name. Like `defvar`, it proclaims `name` **special**, so a later [`let`](let.md) of it establishes a dynamic binding. Returns the name symbol. ```lisp (defparameter *limit* 100) ; => *LIMIT* ``` ```lisp (defparameter *limit* 100) (list (let ((*limit* 5)) *limit*) *limit*) ; => (5 100) ``` --- # FILE: references/reference/special-forms/defstruct.md # defstruct `(defstruct name slot...)` Defines a structure type named `name` and returns the name symbol. Each `slot` is either a symbol or `(slot-name default)`, where `default` is evaluated at construction time when the slot is not supplied (and may refer to variables in scope). The form generates ordinary functions: - `make-name (&key slot...)` — the constructor; slots are supplied as keyword arguments, an unknown keyword is an error - `name-p (object)` — the type predicate, `t` for instances of this structure only - `copy-name (object)` — a shallow copier - `name-slot (object)` — one accessor per slot; accessors are also `setf`-able places, so `setf`/`incf`/`push` on `(name-slot obj)` work Because the generated names are plain functions they are first-class (`#'point-x`, `mapcar`, `funcall`). On the compilation path `defstruct` is only supported as a top-level form; the interpreter also accepts it in the REPL and via `load`. Under a [user-defined package](../packages.md#user-defined-packages-defpackage) the generated names are interned as internal symbols of that package (`geo::make-pt`); listing them in a `defpackage` `:export` clause is not supported. An instance is a first-class structure object, not a list: `print` shows it in the standard `#S(NAME :SLOT value ...)` syntax, `consp`/`listp` are `nil` on instances, and `equal` compares instances slot-wise (Common Lisp compares distinct structures as unequal). The options syntax `(defstruct (name option...) slot...)` supports `(:constructor name)`, `(:conc-name prefix)`, `(:predicate name)`, `(:copier name)`, `(:include parent (slot new-default) ...)`, `(:type (vector ...))`, `(:print-object fn)` and `(:print-function fn)` on every backend, and a documentation string before the slots is accepted and dropped. A BOA constructor -- `(:constructor name (lambda-list))` -- is supported in a lite form: a slot named by the lambda list reads that parameter, and every other slot evaluates its initform in the constructor body. Slot options `:type` and `:read-only` are parsed and ignored. The struct name is usable as a [`defmethod`](defmethod.md) parameter specializer, and the runtime `eval` of a compiled program knows neither `defstruct` nor accessor `setf` places (calling the generated functions from `eval` works). `(:include parent)` is single struct inheritance: the parent's slots come first (so its accessors, its predicate and `(typep x 'parent)` all work on a child instance), and the child adds its own after them. A trailing slot-override -- `(:include parent (slot new-default) ...)` -- re-defaults one inherited slot in THIS child's layout (the parent's own default is untouched); the slot keeps its inherited index, so the parent's accessors still read it. Overriding a slot the parent does not define is an error. `(:type (vector ...))` makes the "instance" a plain vector instead of a structure object: the element type is dropped (rontolisp vectors are generic), accessors are `aref` reads and `setf`-able places, the copier is `copy-seq`, and since there is no structure tag such a type has no predicate, no `#S(...)` syntax and cannot be a `defmethod` specializer (Common Lisp agrees -- a typed struct is not a `structure-object`). `:include` on a `:type` struct is an error. `(:print-object fn)` and `(:print-function fn)` give the structure its own printer instead of the `#S(...)` syntax. `fn` is a function designator -- a symbol or a `lambda` expression; `:print-object` calls it with `(object stream)` and the older `:print-function` with `(object stream depth)`, where `depth` is always `0` (no print level is tracked). Either one is exactly a [`print-object`](defmethod.md) method on the structure type, so every printing operator honors it -- `print`, `princ`, `prin1`, `format`'s `~A`/`~S` -- and a later `defmethod print-object` on the same type replaces it. [`print-unreadable-object`](../macros/print-unreadable-object.md) is the usual way to write the body. Giving both options is an error, and so is combining either with `:type` (a typed structure is a plain vector, with no type to dispatch on). ```lisp (defstruct (celsius (:print-object (lambda (obj stream) (format stream "~D deg" (celsius-c obj))))) (c 0)) (list (princ-to-string (make-celsius :c 21)) (format nil "~A" (make-celsius))) ; => ("21 deg" "0 deg") ``` ```lisp (defstruct shape (kind :none)) (defstruct (circle (:include shape)) (r 1)) (setq c (make-circle :kind :round :r 2)) (list (shape-kind c) (circle-r c) (shape-p c) (circle-p (make-shape))) ; => (:ROUND 2 T NIL) ``` ```lisp (defstruct point x (y 10)) (setq p (make-point :x 1)) (list (point-x p) (point-y p) (point-p p) (point-p '(1 2))) ; => (1 10 T NIL) ``` The same `#S(NAME :SLOT value ...)` syntax is also read back: a `#S(...)` literal in source is a self-evaluating instance, so it works quoted, inside a backquote template and inside a `#(...)` vector literal. The `defstruct` must appear in an EARLIER top-level form (the literal is built as the source is processed, just as it is in Common Lisp). Slot values are read as data and are never evaluated, so `#S(BOX :V (+ 1 2))` stores the list `(+ 1 2)`; a slot named twice keeps its leftmost value; an omitted slot takes its `default`, which here must be a constant rather than an expression to evaluate. A type name that is not a structure, a slot the type does not have, and an odd number of slot items are errors. The runtime [`read`](../functions/read.md) / `read-from-string` builds the instance too, on every backend, so `(read-from-string (prin1-to-string p))` round-trips everywhere. One compiled-reader nuance: an omitted slot whose `default` is `nil` or a simple constant is honored at run time, while a `default` outside the re-readable constant set signals instead of substituting a wrong value (see [Compiled read/load Limitations](../../guides/read-load-limitations.md)). ```lisp (defstruct point x (y 10)) (list #S(POINT :X 1 :Y 2) #S(POINT :X 7) (equal #S(POINT :X 1 :Y 2) (make-point :x 1 :y 2))) ; => (#S(POINT :X 1 :Y 2) #S(POINT :X 7 :Y 10) T) ``` ```lisp (defstruct book title (sold 0)) (setq b (make-book :title "RontoLisp")) (incf (book-sold b)) (setf (book-title b) "RontoLisp 2e") (setq c (copy-book b)) (incf (book-sold c)) (list (book-title b) (book-sold b) (book-sold c)) ; => ("RontoLisp 2e" 1 2) ``` --- # FILE: references/reference/special-forms/defun.md # defun `(defun name (params...) body...)` Defines a function named `name` in the function namespace, with the given parameter list and body, and returns the name symbol. The `body` is not evaluated at definition time; it runs on each call, returning the value of the last body form. Per Lisp-2 the definition lives in the function namespace, so the name is reachable in call position (and via `#'name`) without colliding with any like-named variable. ```lisp (defun sq (x) (* x x)) ; => SQ ``` ```lisp (defun sq (x) (* x x)) (sq 6) ; => 36 ``` ## Lambda list keywords The parameter list supports the Common Lisp lambda-list keywords `&optional`, `&rest`, `&key`, `&allow-other-keys`, and `&aux` (in that order). A default form is evaluated only when the argument is absent, and it can reference parameters bound to its left. An optional or keyword parameter may declare a supplied-p variable that is `t` when the caller passed the argument. ```lisp (defun greet (name &optional (greeting "Hello")) (concatenate 'string greeting ", " name)) (greet "world" "Hi") ; => "Hi, world" ``` ```lisp (defun sum (&rest xs) (reduce #'+ xs :initial-value 0)) (sum 1 2 3 4) ; => 10 ``` ```lisp (defun make-point (&key (x 0) (y 0 y-supplied-p)) (list x y y-supplied-p)) (make-point :y 5) ; => (0 5 T) ``` An unknown keyword argument signals an error unless the lambda list declares `&allow-other-keys` or the caller passes `:allow-other-keys t`. `&aux` introduces auxiliary variables bound like a trailing `let*`. `&whole` is not supported. ```lisp (defun area (w &optional (h w) &aux (a (* w h))) a) (area 3) ; => 9 ``` Calling a function with too few required arguments (or too many, for a fixed-arity function) signals an error in the interpreter and is a compile error on the JVM/WASM backends. ```console > (defun f (a b) (+ a b)) > (f 1) Function expects 2 arguments, got 1 ``` ## setf-function names The `name` may be a `(setf name)` list instead of a plain symbol. This defines a *setf-function*: the writer invoked when `name` is used as a `setf` place. The new value is passed as the first argument (it is the last required parameter of the setf lambda list, per the Common Lisp convention), so `(setf (name arg...) value)` calls the writer with `value` followed by `arg...`. The function is also first-class through `#'(setf name)`. ```lisp (defvar *mode* :xml) (defun (setf my-mode) (m) (setq *mode* m)) (setf (my-mode) :html5) *mode* ; => :HTML5 ``` Only the `(setf name)` form is supported (a two-element list); `symbol-function`/`fboundp` of a `(setf ...)` name is not. --- # FILE: references/reference/special-forms/defvar.md # defvar `(defvar name [value])` Defines a global variable `name`, binding it to `value` only if `name` is not already bound; if it already has a value, `defvar` leaves it unchanged (it is idempotent). With no `value`, the variable is declared but left unbound. The `value` is evaluated only when a binding is actually established, and the name symbol is returned. `defvar` also proclaims `name` **special**: a later [`let`](let.md)/`let*` of it establishes a dynamic binding (visible to functions called within the extent, restored on exit) rather than a lexical one. See [`let`](let.md) and [`progv`](progv.md). ```lisp (defvar *counter* 0) ; => *COUNTER* ``` ```lisp (defvar *scale* 1) (defun scaled (n) (* n *scale*)) (let ((*scale* 10)) (scaled 5)) ; => 50 ``` --- # FILE: references/reference/special-forms/function.md # function `(function name)` or `#'name` Looks up `name` in the function namespace and returns the corresponding function as a first-class value; `#'name` is reader shorthand for `(function name)`. The argument is a name, not an evaluated expression. This is how a named function (or a `lambda`) is obtained so it can be passed to `funcall`/`apply` or a higher-order function like `mapcar` -- necessary in a Lisp-2 because a bare symbol refers to the variable namespace. ```lisp (funcall (function +) 2 3) ; => 5 ``` --- # FILE: references/reference/special-forms/go.md # go `(go tag)` Transfers control to the given go tag of the (dynamically) enclosing [`tagbody`](tagbody.md); the forms after the jump point continue executing. It is an error when no enclosing `tagbody` has the tag. On the JVM and WASM compilers `go` is lexical: it must target a tag of a `tagbody` that lexically encloses it (the interpreter additionally supports dynamic `go` across function-call boundaries, i.e. a tag established by the *caller*). A tag reached from inside a nested `lambda` -- what a [`handler-bind`](../macros/handler-bind.md) handler that resumes its loop with a `go` produces -- is lowered to a non-local exit that re-enters the `tagbody` at the tag, so it works on every backend; such a program compiles in exception-handling mode, so the wasm runs need `wasmtime -W exceptions=y`. ```lisp (let ((acc nil)) (tagbody (push :a acc) (go skip) (push :never acc) skip (push :b acc)) (nreverse acc)) ; => (:A :B) ``` --- # FILE: references/reference/special-forms/if.md # if `(if test then [else])` Evaluates `test`; if it is non-nil the `then` branch is evaluated and returned, otherwise the optional `else` branch is evaluated and returned. Only one of the two branches is ever evaluated. `nil` is the only false value -- every other value, including `0` and the empty string, counts as true. With no `else` branch and a false `test`, the result is `nil`. ```lisp (if (> 3 2) 'yes 'no) ; => YES ``` --- # FILE: references/reference/special-forms/lambda.md # lambda `(lambda (params...) body...)` Creates an anonymous function with the given parameter list and body, closing over the lexical variables in scope. The `body` is not evaluated when the `lambda` is created; it runs each time the resulting function is called, returning the value of the last body form. The function value can be called with `funcall`/`apply` or passed to higher-order functions like `mapcar`. In call position a `lambda` form may also be used directly, e.g. `((lambda (x) x) 1)`. ```lisp (funcall (lambda (x) (* x x)) 5) ; => 25 ``` The parameter list supports the same lambda-list keywords as [`defun`](defun.md) (`&optional`, `&rest`, `&key`, `&allow-other-keys`, `&aux`): ```lisp (funcall (lambda (&rest xs) xs) 1 2 3) ; => (1 2 3) ``` ```lisp (mapcar (lambda (x &optional (y 100)) (+ x y)) (list 1 2 3)) ; => (101 102 103) ``` --- # FILE: references/reference/special-forms/let.md # let `(let ((var init)...) body...)` Establishes local variable bindings: each `init` is evaluated (in the surrounding scope, so the bindings are parallel, not sequential) and bound to its `var` for the duration of `body`. The `body` forms are evaluated in order and the value of the last one is returned; with an empty body the result is `nil`. The bindings are variable bindings only -- per Lisp-2 they do not shadow the function namespace, so a `let`-bound `car` does not affect calls to the function `car`. If a `var` names a variable that was proclaimed **special** (by [`defvar`](defvar.md)/[`defparameter`](defparameter.md) or `(declaim (special ...))`), that binding is **dynamic**: it is visible to any function called during `body`, not just lexically nested code, and is restored when `body` exits. Ordinary (non-special) names are bound lexically as usual. See also [`progv`](progv.md) for a runtime-computed list of special bindings. ```lisp (let ((x 2) (y 3)) (+ x y)) ; => 5 ``` ```lisp (defvar *depth* 0) (let ((*depth* (+ *depth* 1))) *depth*) ; => 1 ``` --- # FILE: references/reference/special-forms/progn.md # progn `(progn expr1 expr2...)` Evaluates each expression in order and returns the value of the last one; the earlier expressions are evaluated only for their side effects. It is the way to group several forms where a single form is expected (such as a branch of `if`). An empty `(progn)` returns `nil`. ```lisp (progn 1 2 3) ; => 3 ``` --- # FILE: references/reference/special-forms/progv.md # progv `(progv symbols values body...)` Evaluates `symbols` and `values` (each a list), then dynamically binds each symbol to the corresponding value for the duration of `body`, restoring the previous values on exit. When `values` is shorter than `symbols`, the extra symbols are bound to `nil`. Unlike [`let`](let.md), the symbols are computed at runtime and need not have been proclaimed special. Returns the value of the last body form. `progv` runs on all backends. The compilers lower it to a dispatch over the program's statically known special variables, so a symbol that is a special of the program gets a true dynamic binding; any other symbol is bound so that `symbol-value` and `boundp` see it for the extent. On the compiled backends a WASM program using `progv` compiles in exception-handling mode (run it with `-W exceptions=y`). ```lisp (progv '(a b) '(1 2) (list (symbol-value 'a) (symbol-value 'b))) ; => (1 2) ``` --- # FILE: references/reference/special-forms/quote.md # quote `(quote expr)` or `'expr` Returns `expr` without evaluating it, yielding the literal datum -- a symbol, list, or atom -- as written. This is how you produce list and symbol literals; the reader macro `'expr` is shorthand for `(quote expr)`. The argument is never evaluated, so `(quote (+ 1 2))` returns the three-element list, not `3`. ```lisp (quote (a b c)) ; => (A B C) ``` --- # FILE: references/reference/special-forms/return.md # return `(return [value])` Performs a non-local exit from the nearest enclosing loop (`do`, `dolist`, `dotimes`, or `loop`), causing that loop form to evaluate to `value` (or `nil` if omitted). The `value` is evaluated before the exit. Because it targets the innermost loop boundary, `return` is only valid inside such a loop. It may appear anywhere in the loop body, including in the middle of an expression, which abandons the enclosing expression. ```lisp (dotimes (i 10) (when (= i 3) (return i))) ; => 3 ``` --- # FILE: references/reference/special-forms/rontolisp-async-defun.md # rontolisp:async-defun `(rontolisp:async-defun name (params...) body...)` Defines an asynchronous function. The surface is the same as [`defun`](defun.md) — the full lambda-list keywords (`&optional`, `&rest`, `&key`, ...) are supported — but calling the function starts the body immediately and returns a *future* instead of a value: the body runs until its first [`rontolisp:await`](rontolisp-await.md) of an unsettled future (or until completion), then the caller resumes ("eager start"). The future settles with the value of the last body form, or with the error the body signaled (re-signaled when the future is awaited). ```lisp (rontolisp:async-defun add-later (a b) (+ a b)) (rontolisp:await (add-later 20 22)) ; => 42 ``` The call itself yields an opaque future ([`rontolisp:futurep`](../functions/rontolisp-futurep.md) recognizes it, and it prints as `#`): ```lisp (add-later 1 2) ; => # ``` An error signaled by the body does not escape at call time; it settles the future and re-signals at the `await` — see [`rontolisp:await`](rontolisp-await.md) for catching it with `handler-case`. The anonymous counterpart is [`rontolisp:async-lambda`](rontolisp-async-lambda.md). `(rontolisp:async (defun name ...))` — the [`rontolisp:async`](rontolisp-async.md) wrapper — is an equivalent JavaScript-style spelling. ## Backend support - **Interpreter / JVM**: the body runs on a virtual thread — real parallelism with the caller after the first suspension. - **WASM `--component`**: the body compiles into a state machine; an `await` of a pending future genuinely suspends it, and the component's event loop resumes it when the awaited host operation (e.g. a `fetch` response) completes. The tasks of one component instance are cooperative (single-threaded). An asynchronous component needs `wasmtime -W exceptions=y` on top of `-W gc=y`. - **Preview 1 WASM**: the body runs to completion immediately (no asynchronous host I/O exists there). - **`--no-gc`**: rejected at compile time. --- # FILE: references/reference/special-forms/rontolisp-async-lambda.md # rontolisp:async-lambda `(rontolisp:async-lambda (params...) body...)` The anonymous counterpart of [`rontolisp:async-defun`](rontolisp-async-defun.md): evaluates to a function value whose invocation returns a future. The parameter list supports the same lambda-list keywords as [`lambda`](lambda.md), and the body follows the same semantics as an `async-defun` body — it starts eagerly on invocation, may use [`rontolisp:await`](rontolisp-await.md), and its value (or error) settles the returned future. `(rontolisp:async (lambda ...))` — the [`rontolisp:async`](rontolisp-async.md) wrapper — is an equivalent JavaScript-style spelling. ```lisp (rontolisp:await (funcall (rontolisp:async-lambda (x) (* x 2)) 21)) ; => 42 ``` Being a function value, it can be passed around like any other function; each invocation returns a fresh future: ```lisp (let ((double-later (rontolisp:async-lambda (x) (* x 2)))) (rontolisp:futurep (funcall double-later 3))) ; => T ``` ## Backend support Identical to [`rontolisp:async-defun`](rontolisp-async-defun.md): virtual threads on the interpreter and JVM, the component's asynchronous task under WASM `--component`, immediate completion on Preview 1 WASM, and a compile error under `--no-gc`. --- # FILE: references/reference/special-forms/rontolisp-async.md # rontolisp:async `(rontolisp:async (defun name (params...) body...))` / `(rontolisp:async (lambda (params...) body...))` Wraps an ordinary defining form and turns it into its asynchronous counterpart, for a notation closer to JavaScript's `async function` / `async (...) =>`. Wrapping a [`defun`](defun.md) is exactly [`rontolisp:async-defun`](rontolisp-async-defun.md), and wrapping a [`lambda`](lambda.md) is exactly [`rontolisp:async-lambda`](rontolisp-async-lambda.md) — the wrapper is a pure rewrite, so the semantics (eager start, futures, [`rontolisp:await`](rontolisp-await.md) placement) and the backend support are those of the canonical forms. ```lisp (rontolisp:async (defun add-later (a b) (+ a b))) (rontolisp:await (add-later 20 22)) ; => 42 ``` ```lisp (rontolisp:await (funcall (rontolisp:async (lambda (x) (* x 2))) 21)) ; => 42 ``` Anything other than a single `defun` or `lambda` form inside the wrapper is an error: ```console > (rontolisp:async (+ 1 2)) Error: rontolisp:async expects a single (defun ...) or (lambda ...) form to make asynchronous, got: (rontolisp:async (+ 1 2)) ``` --- # FILE: references/reference/special-forms/rontolisp-await.md # rontolisp:await `(rontolisp:await value)` Given a future, suspends the current asynchronous function until the future settles and returns its settled value. Settled futures never suspend, nested futures flatten, and a value that is not a future passes through unchanged — like a JavaScript `await` on a non-promise — so `await` can be applied uniformly to a value that may or may not be a future. ```lisp (rontolisp:await 42) ; => 42 ``` ```lisp (rontolisp:async-defun inner () 10) (rontolisp:async-defun outer () (+ (rontolisp:await (inner)) 1)) (rontolisp:await (outer)) ; => 11 ``` `await` is a special form, legal only inside [`rontolisp:async-defun`](rontolisp-async-defun.md) / [`rontolisp:async-lambda`](rontolisp-async-lambda.md) bodies and at top level (the top level is implicitly asynchronous). Anywhere else — a plain `defun` or `lambda` body, even one nested inside an asynchronous body — it is an error at compile/definition time: ```console > (defun bad () (rontolisp:await 1)) rontolisp:await is only allowed inside rontolisp:async-defun/async-lambda or at top level ``` ## Errors A future that settled with an error re-signals that condition at the `await` — catch it with `handler-case` around the await: ```lisp (rontolisp:async-defun failing () (error "boom")) (handler-case (rontolisp:await (failing)) (error (e) "caught")) ; => "caught" ``` --- # FILE: references/reference/special-forms/setq.md # setq `(setq name value ...)` Assigns `value` to the variable `name`, evaluating `value` but not `name`. Multiple `name value` pairs may be given; they are assigned left to right, so a later `value` can read an earlier assignment. The value of the last assignment is returned. `setq` operates in the variable namespace only (Lisp-2). ```lisp (let ((x 0)) (setq x 1 x (+ x 9)) x) ; => 10 ``` --- # FILE: references/reference/special-forms/tagbody.md # tagbody `(tagbody {tag | form}...)` Evaluates its body forms in order for effect. A bare symbol (or integer) in the body is a *go tag*: [`go`](go.md) transfers control to the form after that tag, forward or backward, so loops and state machines can be written with explicit jumps. Falling off the end returns nil. On the JVM and WASM compilers `go` is lexical: it must target a tag of a `tagbody` that lexically encloses it (the interpreter additionally supports dynamic `go` across function-call boundaries, i.e. a tag established by the *caller*). A tag reached from inside a nested `lambda` -- what a [`handler-bind`](../macros/handler-bind.md) handler that resumes its loop with a `go` produces -- is lowered to a non-local exit that re-enters the `tagbody` at the tag, so it works on every backend; such a program compiles in exception-handling mode, so the wasm runs need `wasmtime -W exceptions=y`. ```lisp (let ((n 0)) (tagbody top (incf n) (when (< n 5) (go top))) n) ; => 5 ``` --- # FILE: references/reference/special-forms/throw.md # throw `(throw tag result)` Transfers control to the innermost active [`catch`](catch.md) whose tag is `eq` to `tag`, making `result` that `catch` form's value. The stack is really unwound, so every intervening [`unwind-protect`](unwind-protect.md) cleanup runs on the way out (innermost first) -- and a `handler-case` in between does **not** intercept it, because a `throw` is a non-local exit, not a signaled condition. `throw` is an error when no matching `catch` is active: the interpreter reports `THROW: no enclosing catch for tag ...`, the JVM backend raises the equivalent runtime error, and the wasm-GC backends trap (the same way an uncaught `error` does). The `result` form is evaluated before the unwind starts. ```lisp (let ((log nil)) (list (catch 'up (unwind-protect (throw 'up :out) (setq log (cons :cleaned log)))) log)) ; => (:OUT (:CLEANED)) ``` A `throw` unwinding through a `handler-case` is not caught by it: ```lisp (catch 'up (handler-case (throw 'up :through) (error (e) :caught))) ; => :THROUGH ``` Because an unmatched `throw` aborts the program, that path is shown statically: ```console > (throw 'nope 1) Error: THROW: no enclosing catch for tag NOPE ``` --- # FILE: references/reference/special-forms/unwind-protect.md # unwind-protect `(unwind-protect protected cleanup...)` Evaluates the `protected` form and returns its value, running the `cleanup` forms on **every** exit from it: a normal return, an `error` unwind, and a `return`/`return-from` non-local exit. The cleanup values are discarded -- their count included, so a protected form returning [multiple values](../functions/values.md) answers all of them however many the cleanup returns. A cleanup form that itself signals replaces the pending exit (the newer error wins, as in Common Lisp). `unwind-protect` is supported on **every backend** except `--no-gc` (a compile error there). On the wasm-GC backends (Preview 1 and `--component`) it compiles through the WebAssembly exception-handling proposal, so running a program that uses it needs wasmtime 37+ with `-W exceptions=y`; a program without catching/cleanup forms is byte-identical to before and keeps its usual command line. Divergence: the cleanups run on a **signaled** error unwind (`error`/`signal`), but a runtime trap (a `(car 5)`-style type failure, integer division by zero) still aborts the instance without running them — the interpreter and the JVM run cleanups for those too. The `with-*` macros (`with-open-file`, `with-output-to-string`, `with-input-from-string`, and the `usocket:with-*` family) expand over `unwind-protect` on every backend, so they release their handle on every exit; on wasm-GC a `with-*` program therefore also compiles in EH mode and needs `-W exceptions=y`. ```lisp (let ((n 1)) (list (unwind-protect (+ n 1) (setq n 10)) n)) ; => (2 10) ``` A cleanup that returns its own values -- `(values)`, the idiomatic "I return nothing" of a release helper, included -- does not truncate the protected form's: ```lisp (multiple-value-list (unwind-protect (values 1 2 3) (values 7 8))) ; => (1 2 3) ``` The cleanup also runs when the protected form exits early via `return`: ```lisp (let ((log nil)) (dolist (x '(1 2 3)) (unwind-protect (when (= x 2) (return)) (setq log (cons x log)))) log) ; => (2 1) ``` Because an uncaught `error` aborts the program, the error path is shown statically: ```console > (unwind-protect (error "boom") (print :cleaned)) :cleaned Error: boom ``` --- # FILE: references/reference/special-forms/while.md # while `(while test body...)` Evaluates `test`; while it is non-nil, evaluates the `body` forms in order and repeats. The loop ends as soon as `test` evaluates to `nil`, and `while` itself always returns `nil`. It is used for side-effecting iteration, typically updating variables with `setq` in the body. ```lisp (let ((i 0) (sum 0)) (while (< i 5) (setq sum (+ sum i)) (setq i (+ i 1))) sum) ; => 10 ``` --- # FILE: references/reference/uiop.md # The uiop Package `uiop` is ASDF's portability layer — the spelling implementation-independent libraries already use for the operations Common Lisp never standardized: reading an environment variable, probing a file, walking a directory, splitting a string. It is **not part of Common Lisp**; reference its symbols with a qualifier (`uiop:getenv`), never unqualified. The coverage target is **uiop 3.3.7**, the release the built-in [`ql:quickload`](../guides/asdf-systems.md#downloading-with-quickload) client fetches. That release exports **429 symbols**, and rontolisp implements a subset of them; the rest resolve and signal, so a library that merely *names* one in an `(:import-from #:uiop)` clause still reads, compiles and runs. ## Sub-packages Upstream's `uiop` is `uiop/driver`, a re-export of 15 sub-packages, and a library may name either spelling — `lack-middleware-backtrace` writes `(:import-from :uiop/image :print-condition-backtrace)`. rontolisp registers all 15, with each sub-package owning the members it defines and `uiop` importing them, so **both spellings name the same symbol** rather than two functions with one member name: ```lisp (list (uiop:emptyp "") (uiop/utility:emptyp "")) ; => (T T) ``` | Sub-package | What lives there | Implemented | |-------------|------------------|-------------| | `uiop/package` | symbol and package surgery (`find-symbol*`, `intern*`, `define-package`) | 4 / 31 | | `uiop/package-local-nicknames` | the package-local nickname API | 1 / 3 | | `uiop/package*` | the three condition/type names `uiop/package` defines but does not export | 0 / 3 | | [`uiop/utility`](uiop/utility.md) | the portable helpers (`strcat`, `split-string`, `if-let`, `not-implemented-error`) | 68 / 68 | | `uiop/version` | version comparison and the deprecation conditions | 1 / 15 | | [`uiop/os`](uiop/os.md) | host identity, the environment, the working directory | 22 / 22 | | [`uiop/pathname`](uiop/pathname.md) | the pathname algebra (`subpathname`, `parse-unix-namestring`, `enough-pathname`) | 50 / 50 | | `uiop/filesystem` | probe, walk and mutate the file system | 8 / 32 | | `uiop/stream` | file contents, temporary files, encodings, the standard streams | 3 / 66 | | [`uiop/image`](uiop/image.md) | exit, fatal conditions, the dump hooks (and the command line, which is not implemented) | 25 / 30 | | `uiop/launch-program` | asynchronous subprocesses | 0 / 19 | | `uiop/run-program` | synchronous subprocesses | 0 / 7 | | `uiop/lisp-build` | `compile-file*` and the deferred warnings | 1 / 44 | | `uiop/configuration` | XDG paths and the configuration search | 0 / 38 | | `uiop/backward-driver` | the deprecated aliases | 0 / 7 | The full export list is checked in as `src/main/resources/am/ik/rontolisp/uiop-exports.txt` (one row per export: sub-package, symbol, and the definition form upstream gives it). It is the target the counts above are measured against, so both move together. ## What is implemented Four sub-packages have their own page. Three are complete: `uiop/utility` — the 68 portable helpers everything else in uiop is written in ([uiop/utility](uiop/utility.md)) — `uiop/pathname`, the 50-member pathname algebra ([uiop/pathname](uiop/pathname.md)), and `uiop/os`, the 22 host-identity, environment and working-directory members ([uiop/os](uiop/os.md), which is where [`uiop:getenv`](functions/uiop-getenv.md) lives). The fourth is [uiop/image](uiop/image.md), where [`uiop:quit`](uiop/image.md#exiting) ends the process with a status code on all four backends, and where the fatal-condition, backtrace and image-hook families live. The rest: | Function | Example | Result | |----------|---------|--------| | `uiop:file-exists-p` | `(uiop:file-exists-p "f.txt")` | the pathname when the file exists, `nil` otherwise — the same contract as `probe-file`, which it lowers onto on every backend | | `uiop:directory-exists-p` | `(uiop:directory-exists-p "src/")` | the pathname (with a trailing `/`) when the DIRECTORY exists, `nil` otherwise — the directory twin of `file-exists-p`, and what tells an empty directory from a missing one | | `uiop:directory-files` | `(uiop:directory-files "db/" "*.up.sql")` | the non-directory entries of a directory — `(directory "db/*.*")` with the subdirectories dropped. UIOP's optional second argument, the namestring of a name-and-type wildcard, filters them exactly as `directory` matches; omitting it lists everything, and a pattern carrying a directory component is an error | | `uiop:subdirectories` | `(uiop:subdirectories "src/")` | the subdirectories of a directory, each with its trailing `/` | | `uiop:collect-sub*directories` | `(uiop:collect-sub*directories "src/" (constantly t) (constantly t) #'print)` | walk a directory tree: `collectp` decides what reaches `collector`, `recursep` what is descended into. Every directory handed over is in directory form, root included | | `uiop:read-file-string` | `(uiop:read-file-string "db/up.sql")` | the whole file as one string. Runs on every backend that can open a file for input. Lite: real UIOP's `&rest` keys are accepted and ignored (`:external-format` has no rontolisp surface — every backend reads UTF-8) | | `uiop:compile-file-type` | `(uiop:compile-file-type)` | `nil` — the pathname type a compiled file carries. There is no `compile-file` here, so there is no such type, and a caller asking "is this path a fasl?" gets `no` for a source path | | `uiop:default-temporary-directory` | `(uiop:default-temporary-directory)` | `$TMPDIR` in directory form, or `#P"/tmp/"` when the environment is empty (both WASM backends without `--env`) | | `uiop:delete-file-if-exists` | `(uiop:delete-file-if-exists "scratch.txt")` | delete a file, answering `nil` instead of signalling when it is not there — the whole reason UIOP exports it | | `uiop:get-pathname-defaults` | `(uiop:get-pathname-defaults)` | the defaults relative names resolve against — `*default-pathname-defaults*` (initially `#P""`, the pathname designating the host working directory) unless an absolute defaults argument is given | | `uiop:native-namestring` | `(uiop:native-namestring #P"/tmp/x")` | `"/tmp/x"` — the host-OS spelling of a pathname, which here IS the namestring, so this is `namestring` | | `uiop:add-package-local-nickname` | `(uiop:add-package-local-nickname '#:j '#:com.example.pkg)` | register a package shorthand (lite: global, no per-package scoping). A literal top-level call is a compile-time directive, so it works on every backend | | `uiop:symbol-call` | `(uiop:symbol-call :cl :+ 1 2)` | look the name up in the package at run time and apply it — UIOP's late-binding call into a system the caller does not depend on | Three members outside the complete sub-packages are **macros**, expanded by the compiler rather than called: `uiop:with-temporary-file`, [`uiop:with-deprecation`](macros/uiop-with-deprecation.md) and `uiop:define-package` (a literal top-level call is consumed like `defpackage`). `uiop/pathname`'s two macros — `uiop:with-pathname-defaults` and `uiop:with-enough-pathname` — are [on its page](uiop/pathname.md#relative-to-a-base). `uiop/utility`'s own macros — [`uiop:if-let`](macros/uiop-if-let.md), `uiop:nest`, `uiop:while-collecting`, `uiop:with-upgradability` and the rest — are [on its page](uiop/utility.md#macros). ## What is not Every other export **resolves and signals `uiop:not-implemented-error`**, naming the operation. That is the whole point of registering the inventory: a program that reaches an unfilled corner of uiop gets one clear answer instead of an `undefined function` from the middle of a library, and a handler can catch it: ```console $ rontolisp -e '(uiop:run-program "ls")' Unhandled condition: Not (currently) implemented on rontolisp: UIOP/RUN-PROGRAM:RUN-PROGRAM ``` ```lisp (handler-case (uiop:run-program "ls") (uiop:not-implemented-error () :cannot)) ; => :CANNOT ``` The behaviour is identical on all four backends — the interpreter, the JVM and both WASM outputs signal the same condition with the same report. ## rontolisp extras Two names live in `uiop` that upstream does not export there: - `uiop:namestring` — upstream only *inherits* Common Lisp's; here it is exported and is the very [`namestring`](functions/namestring.md) function, so both spellings name one function. - [`uiop:when-let`](macros/uiop-when-let.md) and [`uiop:when-let*`](macros/uiop-when-let-star.md) — alexandria's names, kept because programs already spell them. Real UIOP exports `if-let` only. --- # FILE: references/reference/uiop/image.md # uiop/image `uiop/image` is what a program does at its edges: end the process with a status code, report a condition nobody handled, and register work to run when an image is restored or dumped. **25 of the 30 exports are implemented**; the five that are not are the command-line family, named at the [bottom of this page](#what-is-missing-the-command-line). Every name is reachable through either spelling: `uiop:quit` and `uiop/image:quit` are the same function ([The uiop Package](../uiop.md#sub-packages)). Three decisions here are rontolisp's own, and each is a decision rather than a gap: - **`uiop:quit` is the host's exit on all four backends**, and nothing runs after it — see [Exiting](#exiting). - **Backtraces carry no frames.** No backend keeps a Lisp-level call stack, so the honest rendering of "the backtrace for this condition" is the condition and nothing else. Real UIOP's own fallback for an implementation without a backtrace API has the same shape. - **There is no image to dump, restore or create**, so those three signal — but the hooks around them are real, because registering into one is just a list push. See [Image hooks](#image-hooks). ## Exiting | Function | What it does | |----------|--------------| | `uiop:quit` | end the process with a status code (`0` by default), after finishing the standard output streams | | `uiop:die` | report a `format` message on `*error-output*`, then quit with the given code | | `uiop:shell-boolean-exit` | quit with `0` when the argument is true and `1` when it is `nil` — a shell's idea of a boolean | ```console $ cat quit.lisp (print :before) (uiop:quit 3) (print :after) $ rontolisp quit.lisp :BEFORE $ echo $? 3 ``` The same program compiled to a class, to a Preview 1 module or to a component prints the same line and exits `3`: the primitive underneath is `System.exit` on the JVM, `proc_exit` on WASM Preview 1 and `wasi:cli/exit`'s `exit-with-code` under `--component`, and the interpreter raises an exit signal the CLI turns into the process code. Two consequences follow from that being a real host exit, and they hold on every backend: - **Nothing runs afterwards.** An `unwind-protect` cleanup around a `quit` does not run — the process ends where the call stands. - **It is not a condition.** `handler-case`, `ignore-errors` and a `catch` tag cannot see it, so a `quit` inside a library's error handling still quits. The status code is masked to eight bits, which is what a POSIX host does with it anyway and what `wasi:cli/exit`'s `u8` accepts: `(uiop:quit 300)` exits `44` everywhere rather than 300 on one backend and 44 on another. A test runner's exit code is the usual reason to reach for this: ```console $ cat run-tests.lisp (uiop:quit (if (rove:run :my-app/tests) 0 1)) ``` `uiop:quit` needs a host process to end, so it is **refused at compile time** under `--no-wasi` and `--no-gc`: those emit a reactor whose entry points are exports a host calls, and a reactor returns from an export rather than exiting. ## Fatal conditions A *fatal condition* is a `serious-condition` — the type is a `deftype` alias, so `typep` and a `handler-bind` clause both match it. | Name | What it does | |------|--------------| | `uiop:fatal-condition` | the type: `serious-condition` | | `uiop:fatal-condition-p` | `(typep c 'uiop:fatal-condition)` | | `uiop:handle-fatal-condition` | report the condition on `*error-output*` and `uiop:die` with status `99` | | `uiop:call-with-fatal-condition-handler` | call a thunk with that handler bound | | `uiop:with-fatal-condition-handler` | the macro over it: `(uiop:with-fatal-condition-handler () body...)` | | `uiop:*lisp-interaction*` | `nil` | ```lisp (list (uiop:fatal-condition-p (make-condition 'error)) (uiop:fatal-condition-p (make-condition 'warning)) (uiop:fatal-condition-p 42)) ; => (T NIL NIL) ``` `uiop:*lisp-interaction*` is `nil` here where upstream defaults to `t`. It asks "is this an interactive Lisp environment, or is it batch processing?", and every rontolisp backend runs a program and ends: there is no debugger to enter and no REPL underneath a compiled artifact. That value is what makes `uiop:handle-fatal-condition` report and exit rather than call an `invoke-debugger` that does not exist. ```console $ cat fatal.lisp (print :start) (uiop:with-fatal-condition-handler () (error "the sky is falling")) (print :unreachable) $ rontolisp fatal.lisp :START Fatal condition: the sky is falling the sky is falling the sky is falling $ echo $? 99 ``` The condition appears three times because upstream prints it three times — once as the report, once with the backtrace, once as `die`'s message — and the middle one has no frames above it here. ## Backtraces | Function | What it prints | |----------|----------------| | `uiop:raw-print-backtrace` | the `:condition` argument, when there is one | | `uiop:print-backtrace` | the same, through `uiop:raw-print-backtrace` | | `uiop:print-condition-backtrace` | its condition argument, on `:stream` (`*error-output*` by default) | ```lisp (let ((report (with-output-to-string (s) (uiop:print-condition-backtrace (make-condition 'simple-error :format-control "boom") :stream s)))) (string-right-trim (list #\Newline) report)) ; => "boom" ``` `:count` is accepted and ignored: there are no frames to limit. `lack-middleware-backtrace` is the library that reaches for this — its error report opens with `uiop/image:print-condition-backtrace`, and here that report is one line long. ## Image hooks | Name | What it does | |------|--------------| | `uiop:register-image-restore-hook` | push a function onto `uiop:*image-restore-hook*`, calling it now unless the second argument is `nil` | | `uiop:register-image-dump-hook` | push a function onto `uiop:*image-dump-hook*`, calling it now only if the second argument is true | | `uiop:call-image-restore-hook` | call the restore hooks, in registration order | | `uiop:call-image-dump-hook` | call the dump hooks | | `uiop:*image-restore-hook*` / `uiop:*image-dump-hook*` | the two lists | | `uiop:*image-prelude*` / `uiop:*image-entry-point*` / `uiop:*image-postlude*` / `uiop:*image-dumped-p*` | `nil` | The hooks are **real** even though nothing here can dump an image: a library may register into one while it loads, and that must not be an error. ```lisp (defvar *log* nil) (uiop:register-image-restore-hook (lambda () (push :restored *log*)) nil) (uiop:call-image-restore-hook) *log* ; => (:RESTORED) ``` `uiop:*image-dumped-p*` is `nil` and stays `nil` — nothing sets it, because nothing dumps. ## Dumping an image | Function | What it signals | |----------|-----------------| | `uiop:dump-image` | `uiop:not-implemented-error` — no backend can save its heap; compile the program instead | | `uiop:restore-image` | `uiop:not-implemented-error` — a program is started from source, never resumed | | `uiop:create-image` | `uiop:not-implemented-error` — there are no Lisp object files to link | rontolisp has no image in the SBCL sense. A program is read and run, or read and compiled into one artifact: ```bash rontolisp app.lisp -o App.class # a JVM class rontolisp app.lisp -o app.wasm # a WASM module ``` which is what `uiop:dump-image` would have been for. ## What is missing: the command line `uiop:argv0`, `uiop:command-line-arguments`, `uiop:raw-command-line-arguments` and `uiop:setup-command-line-arguments` are **not implemented yet**: they signal `uiop:not-implemented-error` like every other unfilled uiop name, and `uiop:*command-line-arguments*` is `nil`. A program that needs input today reads it from the environment ([`uiop:getenv`](../functions/uiop-getenv.md)) or from standard input. --- # FILE: references/reference/uiop/os.md # uiop/os `uiop/os` is what a portable library asks about the host: which operating system this is, which implementation, what the environment says, where the working directory is. **All 22 exports are implemented**, and every host answer is derived from one source — upstream's own `uiop:featurep` over [`*features*`](../data-types.md#comments-feature-conditionals-and-features) — so the same rule decides them on all four backends. Every name is reachable through either spelling: `uiop:getenv` and `uiop/os:getenv` are the same function ([The uiop Package](../uiop.md#sub-packages)). Three answers here are rontolisp's own, and each is a decision rather than a gap: - **`uiop:os-unix-p` is `t` outright.** Every backend presents the POSIX-shaped file and namestring model, so the answer is right — but `*features*` deliberately carries no `:unix`, because that would flip the `#+unix` reader branch of every library the frontend reads, which is a far wider claim than one OS predicate. - **The environment is read from the host and written to an override map.** No backend can rewrite its own process environment (the JVM cannot at all, WASI's is read-only), so `(setf (uiop:getenv name) value)` records the value in a per-program map that `uiop:getenv` consults first. - **There is no `chdir` anywhere**, and no working directory on the WASM backends. See [The working directory](#the-working-directory). ## Host identity | Function | What it answers | |----------|-----------------| | `uiop:implementation-type` / `uiop:*implementation-type*` | `:rontolisp` | | `uiop:lisp-version-string` | this build's version, the same string `(rontolisp:version)` carries | | `uiop:operating-system` | `:unix` | | `uiop:detect-os` | `:os-unix` — upstream's own body minus the loop: only one predicate can win here, so it pushes that feature onto `*features*` and returns it | | `uiop:architecture` | the ABI the artifact targets: `:jvm` on the interpreter and the JVM (a class file is CPU-independent), `:wasm32` on both WASM outputs | | `uiop:implementation-identifier` | all four joined and downcased, the way upstream builds a fasl-cache directory name: `"rontolisp--unix-jvm"` | | `uiop:hostname` | `nil` — no backend has a host-identity primitive, and this is exactly what upstream answers on an implementation its own `#+` clauses do not name | | `uiop:os-unix-p` | `t` (see above) | | `uiop:os-macosx-p` / `uiop:os-windows-p` / `uiop:os-genera-p` | `nil` — upstream's own derivations over `*features*`, which carries no host-OS feature | | `uiop:os-cond` | a `cond` over the predicates above, choosing the first clause whose test is true | ```lisp (print (list (uiop:implementation-type) (uiop:operating-system) (uiop:detect-os))) (print (list (uiop:os-unix-p) (uiop:os-windows-p) (uiop:hostname))) (print (uiop:os-cond ((uiop:os-windows-p) :windows) ((uiop:os-unix-p) :unix) (t :other))) ``` ``` (:RONTOLISP :UNIX :OS-UNIX) (T NIL NIL) :UNIX ``` ## Feature expressions `uiop:featurep` evaluates a feature expression against `*features*` at **run time**, exactly as `#+` does at read time: an atom is a membership test, and `(:not e)`, `(:or e...)` and `(:and e...)` combine them. ```lisp (print (list (uiop:featurep :rontolisp) (uiop:featurep '(:and :rontolisp :unicode)) (uiop:featurep '(:or :no-such-feature :rontolisp)) (uiop:featurep '(:not :no-such-feature)) (uiop:featurep :no-such-feature))) ``` ``` (T T T T NIL) ``` The feature set differs per backend by design — `:rontolisp-interpreter`, `:rontolisp-jvm` and `:rontolisp-wasm` are what tell them apart — so `uiop:featurep` (and `uiop:architecture` with it) answers for the backend that is running. A second, optional argument tests a feature set of your own: `(uiop:featurep :x '(:x))` is `T`. Rebinding `*features*` around the call, which upstream's parameter list invites, works on every backend — it is an ordinary special variable. ## Environment variables | Function | What it does | |----------|--------------| | [`uiop:getenv`](../functions/uiop-getenv.md) | the value of a variable as a string, or `nil` when unset | | `(setf uiop:getenv)` | record an override the reads above consult first; a `nil` value is an unset | | `uiop:getenvp` | the value, but `nil` for the empty string as well — "is this variable really set?" | ```lisp (setf (uiop:getenv "RONTOLISP_DOC_VAR") "hello") (print (list (uiop:getenv "RONTOLISP_DOC_VAR") (uiop:getenvp "RONTOLISP_DOC_VAR"))) (setf (uiop:getenv "RONTOLISP_DOC_VAR") "") (print (list (uiop:getenv "RONTOLISP_DOC_VAR") (uiop:getenvp "RONTOLISP_DOC_VAR"))) (setf (uiop:getenv "RONTOLISP_DOC_VAR") nil) (print (uiop:getenv "RONTOLISP_DOC_VAR")) ``` ``` ("hello" "hello") ("" NIL) NIL ``` The override is **per program run**, not a change to the process environment: a subprocess would not see it, and neither would anything outside this image. That is the honest shape of `(setf (uiop:getenv ...))` on hosts that do not allow the write, and it is what makes the option-setting idiom libraries use — bind some variables, run a body, put them back — behave the same on all four backends. ## The working directory | Function | What it does | |----------|--------------| | `uiop:getcwd` | the host working directory in directory form, or a signalled `uiop:not-implemented-error` where the host has none | | `uiop:chdir` | signals `uiop:not-implemented-error` on every backend | `uiop:getcwd` answers on the interpreter and the JVM, where the host process has a working directory. **Both WASM backends signal**: a WASI program is given preopened directories and no current one, so there is nothing to answer. `uiop:chdir` signals everywhere, including the JVM. Java reads `user.dir` once at startup and cannot move the process working directory, and WASI has no `chdir` at all — an answer that only changed what merges look like, while `open` kept resolving elsewhere, would be worse than the error. ```lisp (handler-case (uiop:chdir "/tmp") (uiop:not-implemented-error () :cannot-change-directory)) ; => :CANNOT-CHANGE-DIRECTORY ``` ## Windows shortcuts Upstream carries a small `.lnk` reader, and its two octet primitives are generally useful: | Function | What it does | |----------|--------------| | `uiop:read-little-endian` | read an unsigned little-endian integer of *n* octets (4 by default) from a binary stream | | `uiop:read-null-terminated-string` | read octets up to a `0` and answer them as a string | | `uiop:parse-windows-shortcut` / `uiop:parse-file-location-info` | signal `uiop:not-implemented-error` | The two readers are real stream work and run everywhere `read-byte` does. The two `.lnk` parsers navigate the file with `file-position`, which no rontolisp file stream supports, so they name that primitive instead of misparsing silently. --- # FILE: references/reference/uiop/pathname.md # uiop/pathname `uiop/pathname` is the pathname algebra — the portable layer libraries use to build, take apart and compare pathnames without touching the file system. **All 50 exports are implemented**, as pure computation over the pathname value, so every one runs identically on all four backends — the interpreter, the JVM, and both WASM outputs. Every name is reachable through either spelling: `uiop:subpathname` and `uiop/pathname:subpathname` are the same function ([The uiop Package](../uiop.md#sub-packages)). A rontolisp [pathname](../data-types.md) carries one flat namestring, so the component-wise algebra collapses onto namestring computation — with two consequences worth knowing: - **Logical pathnames do not exist** (no logical host can be defined), so `uiop:logical-pathname-p` answers `nil` for everything, `uiop:physical-pathname-p` is `pathnamep`, `uiop:physicalize-pathname` is the coercing identity, and `uiop:make-pathname-logical` signals `uiop:not-implemented-error`. - **Nothing is absolutized**: `uiop:ensure-absolute-pathname` answers a relative path as itself where upstream signals — rontolisp resolves relative paths against the host working directory for the whole run, so the path as given is already the file's identity. ## Building and merging | Function | What it does | |----------|--------------| | [`uiop:merge-pathnames*`](../functions/uiop-merge-pathnames-star.md) | the defaults-aware merge — an absolute `specified` wins, a relative one is appended to the defaults' directory | | [`uiop:subpathname`](../functions/uiop-subpathname.md) | a relative subpath merged under a base pathname's directory | | `uiop:subpathname*` | `nil` when the base is `nil`, otherwise `subpathname` with the base first put in directory form | | `uiop:ensure-directory-pathname` | the pathname in directory form (a trailing `/`) | | `uiop:ensure-absolute-pathname` | an absolute path passes through; a relative one is merged against the defaults (a pathname, or a function answering one) | | `uiop:nil-pathname` / `uiop:*nil-pathname*` | the neutral defaults — the empty pathname `#P""` | | `uiop:pathname-root` | the root of the pathname's host and device — `#P"/"`, the only root here | | `uiop:pathname-host-pathname` | a pathname carrying only the host — `#P""`, since no host is modeled | | `uiop:make-pathname*` | `make-pathname`, kept for callers of the deprecated spelling | | `uiop:make-pathname-component-logical` | `:unspecific` becomes `nil`; everything else passes through | | `uiop:normalize-pathname-directory-component` | a directory component in CLHS list form (`"foo"` → `(:absolute "foo")`) | | `uiop:denormalize-pathname-directory-component` | the identity — the normalized form is the native one | | `uiop:merge-pathname-directory-components` | the directory-list half of the merge, `:back` handling included | | `uiop:*unspecific-pathname-type*` | `nil` — a component that is not present is `nil` here | ```lisp (print (uiop:subpathname #P"/tmp/foo/" "bar/baz.txt")) (print (uiop:subpathname* "/tmp/foo" "x.txt")) (print (uiop:ensure-absolute-pathname "b.txt" "/tmp/")) (print (uiop:merge-pathname-directory-components '(:relative :back "x") '(:absolute "a" "b"))) ``` ``` #P"/tmp/foo/bar/baz.txt" #P"/tmp/foo/x.txt" #P"/tmp/b.txt" (:ABSOLUTE "a" "x") ``` ## Predicates `absolute-pathname-p`, `relative-pathname-p` and `file-pathname-p` answer the parsed **pathname** when true (a generalized boolean, as upstream); the rest answer `t`/`nil`. None of them touches the file system. | Function | True when | |----------|-----------| | [`uiop:absolute-pathname-p`](../functions/uiop-absolute-pathname-p.md) | the namestring starts with `/` | | [`uiop:relative-pathname-p`](../functions/uiop-relative-pathname-p.md) | it does not (the empty pathname included) | | [`uiop:directory-pathname-p`](../functions/uiop-directory-pathname-p.md) | non-wild, with no name and no type — empty or ending in `/` | | [`uiop:file-pathname-p`](../functions/uiop-file-pathname-p.md) | a name or type component is present | | `uiop:hidden-pathname-p` | the name starts with a dot | | `uiop:pathname-equal` | the two designators carry the same namestring | | `uiop:logical-pathname-p` | never (no logical pathnames exist) | | `uiop:physical-pathname-p` | the argument is a pathname | ```lisp (print (list (uiop:absolute-pathname-p "/a/b") (uiop:relative-pathname-p "a/b"))) (print (list (uiop:directory-pathname-p "/a/b/") (uiop:file-pathname-p "/a/b"))) (print (list (uiop:hidden-pathname-p ".gitignore") (uiop:pathname-equal "/a/b" #P"/a/b"))) ``` ``` (#P"/a/b" #P"a/b") (T #P"/a/b") (T T) ``` ## Directories | Function | What it does | |----------|--------------| | [`uiop:pathname-directory-pathname`](../functions/uiop-pathname-directory-pathname.md) | the pathname's directory, name and type dropped | | [`uiop:pathname-parent-directory-pathname`](../functions/uiop-pathname-parent-directory-pathname.md) | one directory level up (the root's parent is the root) | ```lisp (print (uiop:pathname-directory-pathname #P"/a/b/c.txt")) (print (uiop:pathname-parent-directory-pathname #P"/a/b/c.txt")) ``` ``` #P"/a/b/" #P"/a/" ``` ## Parsing | Function | What it does | |----------|--------------| | [`uiop:parse-unix-namestring`](../functions/uiop-parse-unix-namestring.md) | a Unix-syntax string as a pathname: `""` and `"."` components dropped, `:type` appended, `:ensure-directory` forcing directory form | | [`uiop:unix-namestring`](../functions/uiop-unix-namestring.md) | the Unix-style namestring — which here *is* the namestring | | [`uiop:split-name-type`](../functions/uiop-split-name-type.md) | two values, NAME and TYPE of a filename (the last dot separates them; a lone leading dot belongs to the name) | | `uiop:split-unix-namestring-directory-components` | four values: `:absolute`/`:relative`, the directory components, the last component, and whether the string was a bare filename | ```lisp (print (uiop:parse-unix-namestring "a//b/./c.txt")) (print (uiop:parse-unix-namestring "foo/bar" :type "lisp")) (print (multiple-value-list (uiop:split-name-type "foo.lisp"))) (print (multiple-value-list (uiop:split-unix-namestring-directory-components "/a/b/c.txt"))) ``` ``` #P"a/b/c.txt" #P"foo/bar.lisp" ("foo" "lisp") (:ABSOLUTE ("a" "b") "c.txt" NIL) ``` ## Relative to a base | Function | What it does | |----------|--------------| | [`uiop:subpathp`](../functions/uiop-subpathp.md) | when the first pathname sits under the second, the relative remainder that merges back onto it; `nil` otherwise | | [`uiop:enough-pathname`](../functions/uiop-enough-pathname.md) | that remainder when there is one, the pathname itself otherwise | | `uiop:call-with-enough-pathname` | calls a function on `enough-pathname`, with `*default-pathname-defaults*` bound to the base | | `uiop:with-enough-pathname` | macro shorthand for the above — `(uiop:with-enough-pathname (p :defaults d) ...)` rebinds `p` | | `uiop:with-pathname-defaults` | macro: run the body with `*default-pathname-defaults*` bound to the given form, or to `*nil-pathname*` when none is given | ```lisp (print (uiop:subpathp #P"/tmp/foo/bar.txt" #P"/tmp/")) (print (uiop:enough-pathname #P"/x/a.txt" #P"/tmp/")) (let ((p #P"/tmp/a/b.txt")) (uiop:with-enough-pathname (p :defaults #P"/tmp/") (print p))) (uiop:with-pathname-defaults (#P"/wpd/") (print *default-pathname-defaults*)) ``` ``` #P"foo/bar.txt" #P"/x/a.txt" #P"a/b.txt" #P"/wpd/" ``` ## Checking constraints [`uiop:ensure-pathname`](../functions/uiop-ensure-pathname.md) is the constraint machine the rest of uiop routes through: it coerces a designator (a string goes through `parse-unix-namestring`), then applies the `:want-*` checks and `:ensure-*` transforms in upstream's order. A failed check signals, or calls a custom `:on-error` function. ```lisp (print (uiop:ensure-pathname "a/b" :ensure-directory t)) (print (handler-case (uiop:ensure-pathname "/a/b" :want-relative t) (error () :err))) ``` ``` #P"a/b/" :ERR ``` Lite next to upstream, deliberately: a failed check reports `Invalid pathname ~S: ~A` (not upstream's `~?` chain), `:want-logical` always fails, and `:resolve-symlinks` / `:truenamize` are accepted and ignored (no backend resolves a symlink); `:truename` answers what `probe-file` answers. ## Wildcards and translation The `*wild*` family are namestring literals over the two wildcards the [`directory`](../functions/directory.md) matcher reads (`*` and `?`), so a wild constant and the matcher can never disagree. | Name | Value / what it does | |------|----------------------| | `uiop:*wild*` | `"*"` | | `uiop:*wild-file*` / `uiop:*wild-file-for-directory*` | `#P"*.*"` | | `uiop:*wild-directory*` | `#P"*/"` | | `uiop:*wild-inferiors*` | `#P"**/"` | | `uiop:*wild-path*` | `#P"**/*.*"` | | `uiop:wilden` | any file in any subdirectory of the pathname's directory | | `uiop:translate-pathname*` | the output-translations wrapper over [`translate-pathname`](../functions/translate-pathname.md): a function destination is called, `t` answers the path, a relative destination is first merged with the root | | `uiop:relativize-directory-component` | `(:absolute ...)` becomes `(:relative ...)` | | `uiop:relativize-pathname-directory` | the pathname with its leading `/` dropped | | `uiop:directory-separator-for-host` | `#\/` | | `uiop:directorize-pathname-host-device` | the identity — a Unix-shaped physical pathname is already in that form | | `uiop:*output-translation-function*` | `'identity` — no output translations run here | ```lisp (print (uiop:wilden #P"/tmp/foo")) (print (uiop:translate-pathname* #P"/src/a/b.lisp" #P"/src/**/*.*" #P"/out/**/*.*")) (print (uiop:relativize-pathname-directory #P"/a/b/c.txt")) ``` ``` #P"/tmp/**/*.*" #P"/out/a/b.lisp" #P"a/b/c.txt" ``` ## Compile-time folding Like `uiop:merge-pathnames*`, a `uiop:subpathname` whose arguments are literals (or references to a top-level `defparameter` bound to one) is folded to a pathname literal by the compile paths, so a bundled library's data-file path costs nothing at run time. --- # FILE: references/reference/uiop/utility.md # uiop/utility `uiop/utility` is the layer the rest of uiop is written in: string, list, plist, hash-table, timestamp and condition helpers that need nothing from the operating system. **All 68 exports are implemented**, and because none of them touches the file system, a subprocess or the network, every one runs on all four backends — the interpreter, the JVM, and both WASM outputs. Every name is reachable through either spelling: `uiop:strcat` and `uiop/utility:strcat` are the same function ([The uiop Package](../uiop.md#sub-packages)). ## Strings | Function | What it does | |----------|--------------| | `uiop:strcat` | concatenate string designators, where `nil` is the empty string and a character is a string of length one | | `uiop:reduce/strcat` | `strcat` over a LIST, with `:key`, `:start` and `:end` as `reduce` takes them | | `uiop:string-prefix-p` | does the string begin with the prefix? | | `uiop:string-suffix-p` | does the string end with the suffix? | | `uiop:string-enclosed-p` | both at once | | `uiop:stripln` | strip a trailing CR, LF or CRLF; two values, the stripped string and the ending removed | | `uiop:frob-substrings` | replace (or remove) each of several substrings, left to right, never inside an earlier match | | `uiop:first-char` / `uiop:last-char` | the first / last character of a non-empty string, else `nil` | | `uiop:split-string` | split on any character of a separator sequence | | `uiop:emptyp` | true for `nil` and for a zero-length vector or string | | `uiop:+cr+` / `uiop:+lf+` / `uiop:+crlf+` | the three line endings as strings | | `uiop:standard-case-symbol-name` | a name designator as a string, upcasing a string one | | `uiop:find-standard-case-symbol` | that name looked up in a package | `strcat`'s tolerance is the point of it: an optional piece concatenates without a test around it. ```lisp (print (uiop:strcat "a" nil #\b "c")) (print (uiop:reduce/strcat (list "aa" "bb" "cc") :start 1)) (print (list (uiop:string-prefix-p "ab" "abc") (uiop:string-suffix-p "abc" "bc") (uiop:string-enclosed-p "a" "abc" "c"))) (print (uiop:frob-substrings "hello world" (list "o") "0")) ``` ``` "abc" "bbcc" (T T T) "hell0 w0rld" ``` `stripln` returns what it removed as a second value, so `strcat` of the two reconstitutes the original line. ```lisp (multiple-value-bind (line ending) (uiop:stripln (uiop:strcat "hi" uiop:+crlf+)) (print (list line (length ending))) (print (string= (uiop:strcat line ending) (uiop:strcat "hi" uiop:+crlf+)))) ``` ``` ("hi" 2) T ``` ## Lists, plists and hash tables | Function | What it does | |----------|--------------| | `uiop:ensure-list` | wrap a non-list in a one-element list | | `uiop:length=n-p` | is the list exactly `n` long? — without walking past `n` | | `uiop:appendf` | `(appendf place list...)`, i.e. `(setf place (append place list...))` | | `uiop:remove-plist-key` / `uiop:remove-plist-keys` | a plist without the given key(s) — keyword-argument cleanup | | `uiop:ensure-gethash` | the entry, computing and storing a default on a miss; a second value says whether it was already there | | `uiop:list-to-hash-set` | a list as an `equal` hash set | | `uiop:lexicographic<` / `uiop:lexicographic<=` | compare two lists element by element with a supplied `element<` | ```lisp (print (uiop:remove-plist-keys (list :b :c) (list :a 1 :b 2 :c 3))) (print (let ((l (list 1))) (uiop:appendf l (list 2 3)) l)) (print (let ((h (make-hash-table :test 'equal))) (list (multiple-value-list (uiop:ensure-gethash "k" h (constantly 5))) (multiple-value-list (uiop:ensure-gethash "k" h (constantly 6)))))) ``` ``` (:A 1) (1 2 3) ((5 NIL) (5 T)) ``` ## Timestamps A timestamp is a real number or a boolean, where `t` is minus infinity and `nil` is plus infinity — so a missing file is "infinitely old" and an unknown one "infinitely new", which is how ASDF orders a build. | Function | What it does | |----------|--------------| | `uiop:timestamp<` / `uiop:timestamp<=` | compare two timestamps | | `uiop:timestamps<` / `uiop:timestamp*<` | is a list (or an argument list) strictly increasing? | | `uiop:earlier-timestamp` / `uiop:later-timestamp` | the smaller / larger of two | | `uiop:timestamps-earliest` / `uiop:timestamps-latest` | over a list | | `uiop:earliest-timestamp` / `uiop:latest-timestamp` | over an argument list | | `uiop:latest-timestamp-f` | `(latest-timestamp-f place timestamp...)`, accumulating into the place | ```lisp (print (list (uiop:timestamp< 1 2) (uiop:timestamp< t 3) (uiop:timestamp< 3 nil))) (print (list (uiop:earliest-timestamp 3 1 2) (uiop:latest-timestamp 3 1 2))) (print (let ((newest 1)) (uiop:latest-timestamp-f newest 5 3) newest)) ``` ``` (T T T) (1 3) 5 ``` `timestamps<` chains from `nil` = plus infinity, so a non-empty list is never "increasing" — upstream's own answer, kept rather than corrected. ## Function designators `uiop:ensure-function` coerces a *designator* into a function: a function is itself, a constant (boolean, keyword, character, number, pathname) becomes `(constantly it)`, a hash table becomes its lookup, a symbol its `fdefinition`, a cons a partially applied call (or an evaluated `lambda` form), and a string is read and evaluated as a function name. | Function | What it does | |----------|--------------| | `uiop:ensure-function` | the coercion above | | `uiop:call-function` | `(apply (ensure-function spec) args)` | | `uiop:call-functions` | `call-function` over a list, in order | | `uiop:access-at` | apply a chain of accessors: an integer is `elt`, a keyword is `getf`, `nil` is identity, a symbol or function is called, a cons is `ensure-function` | | `uiop:access-at-count` | how many sub-objects an `access-at` specifier reads | | `uiop:register-hook-function` | push a hook onto a variable — see [What is missing](#what-is-missing) | ```lisp (print (funcall (uiop:ensure-function 'car) (list 9 8))) (print (uiop:call-function (list '+ 1) 2)) (print (uiop:access-at (list :a (list 10 20)) (list :a 1))) ``` ``` 9 3 20 ``` ## Conditions | Name | What it does | |------|--------------| | `uiop:not-implemented-error` | the condition, and the function that signals it, naming an operation this implementation does not have | | `uiop:parameter-error` | the operation exists but does not accept that parameter combination | | `uiop:simple-style-warning` | uiop's own style warning — a real `style-warning`, so a handler for the standard type catches it | | `uiop:style-warn` | signal one, from a format string, a condition type or a condition | | `uiop:match-condition-p` | does a condition match a pattern? (a type name, a `#(name package)` vector, a predicate, or a `simple-condition` format string) | | `uiop:match-any-condition-p` | any of several patterns | | `uiop:call-with-muffled-conditions` | run a thunk with matching conditions muffled | | `uiop:with-muffled-conditions` | the macro over it | | `uiop:boolean-to-feature-expression` | `(:and)` or `(:or)` — an always-true / always-false `#+` test | | `uiop:symbol-test-to-feature-expression` | the same, from "does this package export this name?" | ```lisp (print (uiop:with-muffled-conditions ('(warning)) (warn "not shown") :muffled)) (print (handler-bind ((style-warning (lambda (c) (muffle-warning c)))) (uiop:style-warn "deprecated: ~A" 'old-name) :warned)) (print (list (uiop:boolean-to-feature-expression t) (uiop:symbol-test-to-feature-expression "CAR" :cl))) ``` ``` :MUFFLED :WARNED ((:AND) (:AND)) ``` The one deviation worth knowing: a STRING pattern for `match-condition-p` is compared against `simple-condition-format-control`, which in rontolisp holds the already-formatted message. A pattern with format directives in it therefore cannot match; a pattern without them still does. ## Macros | Macro | What it does | |-------|--------------| | [`uiop:if-let`](../macros/uiop-if-let.md) | bind, then take the `then` branch only if every variable is non-nil | | `uiop:nest` | nest each form inside the previous one's tail — indentation control | | `uiop:while-collecting` | bind one collector FUNCTION per name; the form answers one list each, in order | | `uiop:with-upgradability` | upstream wraps every definition in it; here it is `progn` — see below | | `uiop:with-muffled-conditions` | shorthand for `call-with-muffled-conditions` | | `uiop:appendf` / `uiop:latest-timestamp-f` | the two `define-modify-macro`s above | | `uiop:compatfmt` | strip pretty-printer directives a weaker `format` cannot read; rontolisp reads them all, so the string is returned unchanged | | `uiop:uiop-debug` | load a developer's personal debug file — see [What is missing](#what-is-missing) | | `uiop:parse-body` | (a function, not a macro) split a body into forms, declarations and a docstring — what a macro-writing library calls | ```lisp (print (uiop:nest (list 1) (list 2) (list 3))) (print (multiple-value-list (uiop:while-collecting (names numbers) (dolist (row (list (list 'a 1) (list 'b 2))) (names (first row)) (numbers (second row)))))) (print (multiple-value-list (uiop:parse-body '("doc" (declare (ignore x)) (+ 1 2)) :documentation t))) ``` ``` (1 (2 (3))) ((A B) (1 2)) (((+ 1 2)) ((DECLARE (IGNORE X))) "doc") ``` **`uiop:with-upgradability` expands to `progn`.** Upstream wraps every one of its definitions in it so that ASDF can redefine itself inside a running image: the body is evaluated at compile, load and run time and each function is declared `notinline`. rontolisp has no image to upgrade — a program is compiled once and run — so `progn` is the whole meaning of it here. This is a deliberate choice, not a gap: the definitions are established exactly as written, and they stay top-level definitions on the compile backends. ```lisp (uiop:with-upgradability () (defun double-it (x) (* x 2)) (defvar *scale* 5)) (print (list (double-it 3) *scale*)) ``` ``` (6 5) ``` ## Characters: one character type Upstream's character quartet exists because `base-char` and `character` are different types on some implementations, so a string's element type has to be discovered. rontolisp has **one** character type — `(subtypep 'character 'base-char)` is true — and running upstream's own derivation on that gives one element, index 0, and a false `+non-base-chars-exist-p+`. Everything else follows: every string is a base string, and the common element type of any group of strings is `character`. ```lisp (print (list uiop:+max-character-type-index+ (uiop:character-type-index #\a) uiop:+non-base-chars-exist-p+)) (print (list (uiop:base-string-p "abc") (uiop:strings-common-element-type (list "a" #\b)))) ``` ``` (0 0 NIL) (T CHARACTER) ``` ## What is missing Two members name what rontolisp does not have, rather than pretending, and both signal `uiop:not-implemented-error` with the reason: - **`uiop:register-hook-function`** would push onto a variable named at run time, which needs `(setf (symbol-value var) ...)` — not a place on any backend. - **`uiop:load-uiop-debug-utility`** (and `uiop:uiop-debug`, which calls it) would `load` a computed pathname at run time; `load` is a compile-time splice on every backend. `uiop:*uiop-debug-utility*` still holds upstream's default form. ```console $ rontolisp -e '(uiop:register-hook-function (quote *h*) (lambda () 1))' Unhandled condition: Not (currently) implemented on rontolisp: UIOP/UTILITY:REGISTER-HOOK-FUNCTION pushing onto a hook needs (setf (symbol-value ...)), which is not a place on any backend ``` --- # FILE: references/examples/README.md # rontolisp examples Practical, self-contained rontolisp programs. Unless noted otherwise each one runs identically on the interpreter, the JVM and WASM. | Directory | Programs | | --- | --- | | [`console/`](https://github.com/making/rontolisp/blob/develop/examples/console) | Algorithms and console I/O — pure, cross-backend | | [`ml/`](https://github.com/making/rontolisp/blob/develop/examples/ml) | Numerical computing and machine learning (arrays, `linalg`, `--simd`) | | [`deep-learning-from-scratch/`](https://github.com/making/rontolisp/blob/develop/examples/deep-learning-from-scratch) | The book *Deep Learning from Scratch* (ゼロから作るDeep Learning) ch02-ch08, ported | | [`llama2/`](https://github.com/making/rontolisp/blob/develop/examples/llama2) | llama2.c's `run.c` ported whole: a Llama 2 inference engine over the real TinyStories checkpoints, and the example `--simd` is for | | [`llm-from-scratch/`](https://github.com/making/rontolisp/blob/develop/examples/llm-from-scratch) | 『作ってわかる大規模言語モデルの仕組み』 chapters 2 and 3, ported: attention, an encoder/decoder Transformer, then a GPT trained on 漱石 and sampled from — all on the `torch` package | | [`net/`](https://github.com/making/rontolisp/blob/develop/examples/net) | Sockets, HTTP servers and JSON web services | | [`db/`](https://github.com/making/rontolisp/blob/develop/examples/db) | PostgreSQL through the real cl-postgres driver and postmodern, up to a REST API on top | | [`jvm/`](https://github.com/making/rontolisp/blob/develop/examples/jvm) | `java:` interop and Swing GUIs (JVM only) | | [`browser/`](https://github.com/making/rontolisp/blob/develop/examples/browser) | Browser demos: compile to WASM, run in a page | | [`count-vowels/`](https://github.com/making/rontolisp/blob/develop/examples/count-vowels), [`wit/`](https://github.com/making/rontolisp/blob/develop/examples/wit) | Crossing the WASM boundary: exporting to a host, implementing a WIT world, calling one, composing with Rust | | [`asdf/`](https://github.com/making/rontolisp/blob/develop/examples/asdf) | Loading real third-party libraries with `asdf:load-system` / `ql:quickload` | | [`wasmcloud/`](https://github.com/making/rontolisp/blob/develop/examples/wasmcloud), [`cloudflare-workers/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers) | Platform templates | How big the compiled artifacts are is measured, not documented here: [`size-report/`](https://github.com/making/rontolisp/blob/develop/size-report). Assuming the executable JAR is built (`./mvnw clean package`): ```bash JAR=target/rontolisp-0.1.0-SNAPSHOT-exec.jar ``` ## Console & algorithms — `console/` | File | What it demonstrates | | --- | --- | | [`nqueens.lisp`](console/nqueens.lisp) | Backtracking search: recursion, list manipulation, ASCII board output | | [`life.lisp`](console/life.lisp) | Conway's Game of Life on a toroidal 2-D `make-array`; `(load ...)`s the rendering-free `life-core.lisp` | | [`sorting.lisp`](console/sorting.lisp) | Quicksort and merge sort parameterized by a first-class comparator | | [`calc.lisp`](console/calc.lisp) | A prefix-arithmetic interpreter: recursive evaluation over an alist environment | | [`mandelbrot.lisp`](console/mandelbrot.lisp) | ASCII Mandelbrot: floating-point arithmetic and nested loops | | [`mandelbrot-nogc.lisp`](console/mandelbrot-nogc.lisp) | The same, as a `--no-gc` export typed by a checked-in WIT world. One directive, two builds: a plain MVP module whose host reads the string out of linear memory, and a `--component` one where `wasmtime --invoke` returns it | | [`line-numbers.lisp`](console/line-numbers.lisp) | A `cat -n` clone: `with-open-file`, `read-line`, `write-line`, `format nil` | | [`parse-numbers.lisp`](console/parse-numbers.lisp) | `parse-integer` and character classification over file lines | | [`sieve.lisp`](console/sieve.lisp) | Sieve of Eratosthenes over a boolean array, plus prime factorization | | [`hanoi.lisp`](console/hanoi.lisp) | Tower of Hanoi, in a printing and a list-returning variant | | [`roman.lisp`](console/roman.lisp) | Roman numerals both ways, and an example that **checks itself**: the full 3999-value round-trip is a [rove](#an-example-that-checks-itself) assertion | | [`word-frequency.lisp`](console/word-frequency.lisp) | Hash-table accumulation, custom-comparator `sort`, `maphash` | | [`contact-book.lisp`](console/contact-book.lisp) | `defstruct` with `setf`-able accessors and `&key` lambda lists | | [`error-handling.lisp`](console/error-handling.lisp) | Typed conditions on a bank account: `define-condition`, `handler-case` dispatch by class, `ignore-errors`, `unwind-protect`, non-fatal `signal`. **Interpreter/JVM only** | | [`l-system.lisp`](console/l-system.lisp) | L-system fractals: string rewriting by hash-table rule dispatch, `&rest` args | ## Numerical & machine learning — `ml/` | File | What it demonstrates | | --- | --- | | [`nn.lisp`](ml/nn.lisp) | XOR by backpropagation with hand-written loops over rank-1/rank-2 arrays | | [`nn-vec.lisp`](ml/nn-vec.lisp) | The same net over the `vec`/`linalg` packages and single-float (`#f`) packed arrays | | [`simd-dot.lisp`](ml/simd-dot.lisp) | The smallest thing `--simd` speeds up: one `vec:dot` over 1024 doubles, 4000 times. The answer is an exact integer, so only the elapsed time moves | | [`simd-gemv.lisp`](ml/simd-gemv.lisp) | `vec:matvec` (GEMV) + `vec:dot` — the two kernels LLM inference lives in. Prints `argmax` indices, so acceleration cannot change the output. See the [SIMD guide](https://github.com/making/rontolisp/blob/develop/doc/en/guides/simd-acceleration.md) | | [`simd-gemv-nogc.lisp`](ml/simd-gemv-nogc.lisp) | The same inner loop as a `--no-gc` reactor: the host calls the exported `fingerprint`. The `-into` kernels keep the never-freed bump heap at three blocks | | [`tiny-llm.lisp`](ml/tiny-llm.lisp) | A 2-layer transformer decoder — llama2's `forward()` without the tokenizer or weight loader: RMSNorm, causal attention over a KV cache, SwiGLU, greedy decode. Thirteen GEMVs per pass. The KV cache stores **V transposed**, which keeps the attention sum one `vec:matvec`. The whole engine, over real checkpoints, is [`llama2/`](https://github.com/making/rontolisp/blob/develop/examples/llama2) | | [`mlp.lisp`](ml/mlp.lisp) | A generalized multi-layer perceptron for 2-D circle classification | | [`maze-rl.lisp`](ml/maze-rl.lisp) | Tabular Q-learning on a grid maze. **Non-deterministic:** `random` is unseeded and per-backend | | [`linear-regression.lisp`](ml/linear-regression.lisp) | Least-squares polynomial fitting through the normal equations, in exact rationals | | [`deep-digits.lisp`](ml/deep-digits.lisp) | A 15-16-16-10 leaky-ReLU MLP over pixel bitmaps, trained by full-batch matrix backprop. Fully deterministic on every backend | | [`numerical-calculus.lisp`](ml/numerical-calculus.lisp) | `linalg:diff` / `linalg:gradient` (numpy's `np.diff`/`np.gradient`), including non-uniform spacing | | [`heat3d.lisp`](ml/heat3d.lisp) | Rank-3 arrays: 3-subscript `aref`, `#nA` syntax, `row-major-aref`, rank-generic `linalg`, and exact rational heat conservation | ## Deep Learning from Scratch — `deep-learning-from-scratch/` A chapter-by-chapter port of the sample code of *Deep Learning from Scratch* (ゼロから作るDeep Learning, O'Reilly Japan) by Koki Saitoh, with the book's `common/` library rebuilt on `linalg:` and CLOS layer classes. MNIST scripts need a one-time `./download-mnist.sh`. Per-program table in [its README](deep-learning-from-scratch/README.md). ## llama2.c — `llama2/` [`llama2.lisp`](llama2/llama2.lisp) is Andrej Karpathy's `run.c` in one Lisp file -- checkpoint loader, tokenizer + BPE encoder, forward pass, sampler, generate loop -- and tells the same stories as the C program, token for token, from the checked-in 1 MB `stories260K.bin` or the downloadable `stories15M.bin`. Its 15 million weights load through `read-sequence` over packed single-float arrays; its decode is all `vec:matvec`, which is why `--simd` takes wasm-GC from 0.4 to 46 tokens/s. Setup, knobs and numbers in [its README](llama2/README.md). ## LLM from Scratch — `llm-from-scratch/` The Transformer chapter of 『作ってわかる大規模言語モデルの仕組み』 (Elith Inc., Nikkei BP), rewritten on the [`torch` package](https://github.com/making/rontolisp/blob/develop/doc/en/guides/neural-networks.md): scaled dot-product and multi-head attention, sinusoidal positional encoding, LayerNorm, the encoder/decoder Transformer with its padding and causal masks, and a Japanese-English training loop with greedy decoding over a twelve-pair corpus that lives in the file. `nn.Module` becomes `torch:module` plus a `forward` defun, `nn.ModuleList` a plain list, `DataLoader` `torch:shuffled-batches`. Chapter 3 continues into GPT: a character-level tokenizer, a decoder-only stack with learned positions, pre-LayerNorm blocks and a causal mask, AdamW over two parameter groups with gradient clipping and a warmup-then-cosine schedule, and temperature / top-k sampling. It trains on the public-domain opening of 『吾輩は猫である』, inlined — nothing is downloaded — and because the sampler draws from the same seeded generator, the generated passages are byte-identical on every backend. Section 3.2's byte-pair encoder needs no `torch` at all, and its hundred merges come out in the book's exact order. Mapping table and the book-vs-tested shapes in [its README](llm-from-scratch/README.md). ## Networking, HTTP & services — `net/` Servers on WASM need `--component` plus `wasmtime run ... -W exceptions=y -S tcp=y -S inherit-network=y`; the `http-handler` ones run under `wasmtime serve -W gc=y -W exceptions=y`. | File | What it demonstrates | | --- | --- | | [`echo-server.lisp`](net/echo-server.lisp) | TCP echo server: `rontolisp:tcp-listen`/`tcp-accept`, then the ordinary stream functions over the socket handle | | [`echo-client.lisp`](net/echo-client.lisp) | Its client, via `rontolisp:tcp-connect`. Either end can run on a different backend | | [`http-hello.lisp`](net/http-hello.lisp) | Minimal HTTP/1.1 by hand: `read-line` over the request, `Content-Length`, `Connection: close` | | [`https-hello.lisp`](net/https-hello.lisp) | The same over TLS via `rontolisp:tls-listen` (PKCS12 keystore). Everything after the listen call is unchanged. Interpreter/JVM only | | [`http-handler.lisp`](net/http-handler.lisp) | The `rontolisp:http-handler` hello world: Clack environment plist in, `(status headers body)` out. See the [Serving HTTP guide](https://github.com/making/rontolisp/blob/develop/doc/en/guides/http-handler.md) | | [`http-handler-cl-who.lisp`](net/http-handler-cl-who.lisp) | The same, rendering through the real **cl-who**: the markup DSL expands at macro-expansion time, `esc` escapes at run time | | [`httpbin.lisp`](net/httpbin.lisp) | A mini **httpbin**: `/get`, `/post`, `/put`, `/patch`, `/delete` echo the request as JSON, plus 405 and 404 | | [`httpbin-clos.lisp`](net/httpbin-clos.lisp) | The **CLOS** flavour: the envelope is a `defclass`, so `json-stringify` serializes slots in definition order — byte-identical output, and the same shape jzon produces | | [`httpbin-jzon.lisp`](net/httpbin-jzon.lisp) | The **jzon** flavour: only the two JSON call sites change, since `rontolisp:json-*` is a subset of jzon | | [`httpbin-clack.lisp`](net/httpbin-clack.lisp) | The **Clack** flavour: an application *function*, a `cond` over `:path-info` (clack has no router) and one middleware — a function from application to application. rontolisp's server protocol *is* Clack's, so this one file is also, unchanged, the Worker of [`cloudflare-workers/httpbin-clack-one-source/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin-clack-one-source). See the [Clack guide](https://github.com/making/rontolisp/blob/develop/doc/en/guides/clack.md) | | [`httpbin-tiny-routes.lisp`](net/httpbin-tiny-routes.lisp) | The **tiny-routes** flavour: routes composed with `define-routes`, threaded through the library's own middleware by `pipe`, and a wrong method *declining* into the catch-all that tells 405 from 404 | | [`httpbin-ningle.lisp`](net/httpbin-ningle.lisp) | The **ningle** flavour, the other routing model: routes are assigned to an application *object*, a controller returns the body and mutates `*response*`, the request arrives already parsed, and the 404 is an overridden `ningle:not-found` method | | [`magic-8-ball.lisp`](net/magic-8-ball.lisp) | The Spin tutorial's Magic 8 Ball JSON API. Inside a serve component `random` works via `wasi:random` | | [`dog-fetcher.lisp`](net/dog-fetcher.lisp) | `rontolisp:fetch` *inside* a handler — the proxy/aggregator shape | | [`linalg-api.lisp`](net/linalg-api.lisp) | A linear-algebra JSON service: `POST /solve` and `POST /fit`, with 400s for bad input. Integer inputs are solved exactly | | [`kv-server.lisp`](net/kv-server.lisp) | A mini **Redis**: enough RESP2 that the real `redis-cli` works, plus inline commands for `nc` | | [`kv-server-tls.lisp`](net/kv-server-tls.lisp) | The same over TLS on 6380 (`redis-cli --tls --insecure`). Interpreter/JVM only | [`net/http-handler/`](https://github.com/making/rontolisp/blob/develop/examples/net/http-handler) holds a `spin.toml` for the `http-handler.lisp` component: `spin build && spin up` serves it on `:3000`. It needs the [Spin canary](https://github.com/spinframework/spin/releases/tag/canary) build (4.1.0-pre0+); 4.0.2 speaks an older `wasi:http` snapshot. ## Java interop / GUI (JVM only) — `jvm/` These drive real Java APIs through the `java:` package, so they need the JVM (as interpreter or via `-o Prog.class`) and a display — not the WASM backend, and not the GraalVM native binary, which carries no reflection metadata for them. See the [Java interop guide](https://github.com/making/rontolisp/blob/develop/doc/en/guides/java-interop.md). | File | What it demonstrates | | --- | --- | | [`java-interop.lisp`](jvm/java-interop.lisp) | A Swing window through `java:new`/`call`/`field`/`proxy`, with a Lisp lambda as the `ActionListener` | | [`swing.lisp`](jvm/swing.lisp) | A reusable Swing grid-window helper written entirely on `java:`, in its own package; the demos splice it in with `(require :swing "swing.lisp")` | | [`life-gui.lisp`](jvm/life-gui.lisp) | Game of Life animated on a `javax.swing.Timer`, loading the same `life-core.lisp` as `life.lisp` | | [`minesweeper-swing.lisp`](browser/minesweeper/minesweeper-swing.lisp) | Minesweeper on the desktop, loading the same core as the browser build | ## Browser demos A Lisp program compiled to `.wasm` and driven from plain HTML/JavaScript — except [`wit-component/`](https://github.com/making/rontolisp/blob/develop/examples/browser/wit-component), which loads a *component* and needs no glue at all. Each directory has its own README. | Directory | What it demonstrates | | --- | --- | | [`wit-component/`](https://github.com/making/rontolisp/blob/develop/examples/browser/wit-component) | The first rontolisp component in a browser: a Mandelbrot/Julia explorer whose page supplies *nothing* — no `instantiate`, no import object, no WASI shim, no `(ptr, len)` decoding. A WIT world types the exports and `jco transpile` produces one self-contained ES module | | [`rainbow/`](https://github.com/making/rontolisp/blob/develop/examples/browser/rainbow) | HSV↔RGB and shortest-arc hue interpolation in Lisp, behind one `rainbow-html(string) -> string` export | | [`wasm-browser/`](https://github.com/making/rontolisp/blob/develop/examples/browser/wasm-browser) | The plumbing: running a rontolisp `.wasm` from plain HTML + JavaScript, stdin included | | [`minesweeper/`](https://github.com/making/rontolisp/blob/develop/examples/browser/minesweeper) | A playable Minesweeper whose rules live in a `minesweeper-core.lisp` shared with the Swing build — and checked head-less by [`minesweeper-core-test.lisp`](browser/minesweeper/minesweeper-core-test.lisp) | | [`hiragana/`](https://github.com/making/rontolisp/blob/develop/examples/browser/hiragana) | A 46-class handwriting recognizer: the ch07 SimpleConvNet trained offline on Kuzushiji-49, its weights read back at startup, driven from a `` | | [`webgl-triangle/`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-triangle) | The WebGL hello world and the smallest `rontolisp:wasm-import` program: ten imported host functions, no exports, no frame loop | | [`webgl-cube/`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-cube) | Hello 3D: perspective and rotation matrices computed in Lisp every frame; bulk floats cross through a staging array | | [`webgl-galaxy/`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-galaxy) | A spiral galaxy driven entirely from Lisp, GLSL sources included, over 32 host functions declared by a WIT — the JavaScript is generated one-line bindings | | [`webgl-heat3d/`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-heat3d) | The rank-3 array showcase: the page's whole state is one `(n n n)` array, diffused and projected every frame | | [`webgl-robot-arm/`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-robot-arm) | A 3-D arm that reaches where you click: damped-least-squares Jacobian IK every frame (FABRIK and the analytic closed form on a HUD toggle), on a minimum-jerk trajectory | | [`webgl-platformer/`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-platformer) | A one-stage 3D platformer: gravity, coyote time, per-axis AABB collision, enemy patrols and the follow camera, all in Lisp | | [`webgl-battlefront/`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-battlefront) | A Pointer-Lock snow battle: third-person aim camera, blaster bolts, a lightsaber that hits *and* deflects, and stormtrooper/AT-AT/boss AI | | [`webgl-common/`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-common) | Not a demo but the shared `gl` package the others splice in with `(require :gl ...)`; `--optimize` tree-shakes the entries a demo never calls | ## Crossing the WASM boundary — `count-vowels/`, `wit/` | Directory | What it demonstrates | | --- | --- | | [`count-vowels/`](https://github.com/making/rontolisp/blob/develop/examples/count-vowels) | *Share a string through Wasm memory.* A `--no-gc` MVP module any engine runs: the host allocates through `__ronto_alloc`, writes UTF-8 bytes, then calls `count-vowels(ptr, len)`. The export's type lives in a checked-in WIT world, so a drifted signature is a compile error naming the WIT line. Driven from a pure-Java host and a three-line Node script; the `--component` build lets the canonical ABI do the memory work instead | | [`wit/world/`](https://github.com/making/rontolisp/blob/develop/examples/wit/world) | *Someone handed me a `.wit`, now what.* `--scaffold-wit` turns a world nobody wrote for rontolisp into a compiling skeleton — one `defun` stub per export, the WIT's own parameter names and docs — which you fill in one export at a time. Renaming a `defun` fails the build with the WIT line number | | [`wit/keyvalue/`](https://github.com/making/rontolisp/blob/develop/examples/wit/keyvalue) | The other direction: **calling** a WIT interface. `wit-import` binds the real upstream `wasi:keyvalue/store` as ordinary `defun`s, and what those calls reach is bound separately — two Lisp providers here, or wasmtime's own store under `--component` | | [`wit/lisp-calls-rust/`](https://github.com/making/rontolisp/blob/develop/examples/wit/lisp-calls-rust) | **Lisp calls Rust**: a Lisp command imports an interface a Rust component exports, `wac plug`ged into one component. The app also runs standalone, where a bundled Lisp provider answers the same interface | | [`wit/rust-calls-lisp/`](https://github.com/making/rontolisp/blob/develop/examples/wit/rust-calls-lisp) | **Rust calls Lisp**: a Lisp component exports a plain function a Rust component imports and calls | | [`wit/pipeline/`](https://github.com/making/rontolisp/blob/develop/examples/wit/pipeline) | **Both directions chained**: Lisp → Rust → Lisp across three components. `wac plug` cannot wire a plug into a plug, so a `composition.wac` spells out each edge for one `wac compose` | ## Third-party libraries & platform templates | Directory | What it demonstrates | | --- | --- | | [`asdf/`](https://github.com/making/rontolisp/blob/develop/examples/asdf) | Loading unmodified upstream libraries — split-sequence, parse-number, cl-utilities, cl-who, cl-mustache, assoc-utils, cl-base64, md5, chipz, cl-ppcre, jzon, ironclad, jose, uax-15, tiny-routes, clack — on all four backends | | [`wasmcloud/`](https://github.com/making/rontolisp/blob/develop/examples/wasmcloud) | The wasmCloud Rust templates ported to `rontolisp:http-handler`, each with a `.wash/config.yaml` so `wash dev` builds and serves it | | [`cloudflare-workers/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers) | Twelve independent Workers: two subjects written once with no library and then in the idiom of each web library, plus two that call out over HTTP on the two `--host-boundary` shapes — from a `--no-gc` module with zero imports to a routed application deployed by `npx wrangler deploy` | ## Running Any example can be interpreted, compiled to a JVM `.class`, or compiled to `.wasm`: ```bash # 1. Interpreter java -jar $JAR examples/console/nqueens.lisp # 2. JVM (the class is named after the output file, so keep it path-free) java -jar $JAR examples/console/nqueens.lisp -o Prog.class && java Prog # 3. WASM (requires wasmtime 14+) java -jar $JAR examples/console/nqueens.lisp -o nqueens.wasm && wasmtime run -W gc nqueens.wasm ``` Programs that touch files need a preopened directory on WASM: ```bash java -jar $JAR examples/console/line-numbers.lisp -o ln.wasm wasmtime run -W gc --dir . ln.wasm ``` ## An example that checks itself Most examples print a result and leave the checking to [`examples.yaml`](examples.yaml). Three of them do it in the Lisp instead, with [rove](https://github.com/making/rontolisp/blob/develop/doc/en/guides/testing.md) — the shape to copy when what you are writing has a right answer rather than only an output: | File | What it asserts | | --- | --- | | [`console/roman.lisp`](console/roman.lisp) | The demo prints its tables, then asserts the encodings, the out-of-range errors and the whole 1..3999 round-trip | | [`cloudflare-workers/httpbin/check.lisp`](cloudflare-workers/httpbin/check.lisp) | Drives the Worker's `handle-request` over six requests and asserts the **parsed** reply, field by field | | [`browser/minesweeper/minesweeper-core-test.lisp`](browser/minesweeper/minesweeper-core-test.lisp) | A test file beside a GUI example: its rules live in a rendering-free core, so they can be checked head-less | The recipe is four lines. Load rove, silence its ANSI colors, write `deftest`s, and make the verdict the exit code: ```lisp (asdf:load-system :rove) (use-package :rove) (setf *enable-colors* nil) (deftest arithmetic (testing "adding two integers" (ok (= (add 1 2) 3)))) (uiop:quit (if (run-suite *package*) 0 1)) ``` rove is vendored in this repository, so its three directories go on `--system-path` and nothing is downloaded; outside it, `(ql:quickload "rove")` fetches the same sources. The systems are spliced in at compile time, so the compiled class / module is self-contained: ```bash SP=src/test/resources/rove:src/test/resources/dissect:src/test/resources/cl-ppcre java -jar $JAR examples/console/roman.lisp --system-path $SP ``` Both WASM runs need `-W exceptions=y`: rove records a failing test through `handler-bind`. The full story — the entry points, the exit code, and what does not work — is the [Testing guide](https://github.com/making/rontolisp/blob/develop/doc/en/guides/testing.md). ## Verifying every non-GUI example at once [`examples.yaml`](examples.yaml) lists each non-GUI example and the backends it can be verified on; [`ExamplesE2eTest`](https://github.com/making/rontolisp/blob/develop/src/test/java/am/ik/rontolisp/e2e/ExamplesE2eTest.java) turns every *(example × backend)* pair into one dynamic test. `interpreter` / `jvm` / `wasm` run the program and check its output; `jvm-compile` / `wasm-component` / `no-gc` only build it, because blocking servers and host-invoked modules never return on their own. Each entry declares its `args` / `stdin` and one `expect`: `equals` (stdout matches this text), `file` (matches a file under `examples/`), `contains` (every listed substring appears) or `skip: true`. Omitting `expect` means "exit 0 and non-empty output". `equals`/`file` are checked against **all** run backends, so a per-backend divergence is a real failure. An example that loads an ASDF system names its directory — or, like the rove ones above, the LIST of directories — under `systemPath`. The suite is opt-in, so a plain `./mvnw test` skips it: ```bash ./mvnw clean package -DskipTests ./mvnw -Dtest=ExamplesE2eTest -DfailIfNoTests=false -Drontolisp.examples=true test ``` To run against the native binary instead, pass `-Drontolisp.binary="$PWD/target/rontolisp"` and drop `-Drontolisp.examples`. The full suite takes minutes; narrow it with a comma-separated `-Drontolisp.examples.only=` matched against the manifest path (`console/,ml/`). A pattern matching nothing produces `Tests run: 0` rather than the whole suite. Adding an example means appending an entry to `examples.yaml` — no Java changes. To regenerate an externalised expected file, run the example and save its stdout. GUI examples (`jvm/` and the `browser/` demos) are excluded: they open a window or run in a page and cannot be checked headless — though the part of one that is not GUI can be, which is what [`minesweeper-core-test.lisp`](browser/minesweeper/minesweeper-core-test.lisp) is. --- # FILE: references/examples/asdf/README.md # Loading real ASDF libraries These demos load REAL third-party Common Lisp libraries — unmodified upstream sources — through `asdf:load-system` and exercise their public API. All run identically on all four backends (interpreter, JVM, WASM Preview 1, `--component`), and each is pinned by its own cross-backend E2E test. | Demo | Library | Upstream | | --- | --- | --- | | [`alexandria-demo.lisp`](alexandria-demo.lisp) | alexandria 1.0.1 (public domain / 0-clause MIT) | | | [`split-sequence-demo.lisp`](split-sequence-demo.lisp) | split-sequence v2.0.1 (MIT) | | | [`parse-number-demo.lisp`](parse-number-demo.lisp) | parse-number v1.8 (BSD 3-Clause) | | | [`cl-utilities-demo.lisp`](cl-utilities-demo.lisp) | cl-utilities v1.2.4 (public domain) | | | [`cl-who-demo.lisp`](cl-who-demo.lisp) | cl-who v1.1.5 (BSD 2-Clause) | | | [`mustache-demo.lisp`](mustache-demo.lisp) | cl-mustache 0.12.3 (MIT) — Mustache templates from strings AND from `greeting.mustache`, its only runtime file I/O (so its WASM runs need `--dir .`). The missing-partial demo invokes a `use-value` restart, so both WASM runs need `-W exceptions=y` | | | [`assoc-utils-demo.lisp`](assoc-utils-demo.lisp) | assoc-utils (public domain) | | | [`cl-base64-demo.lisp`](cl-base64-demo.lisp) | cl-base64 v3.4 (BSD-style) | | | [`jzon-demo.lisp`](jzon-demo.lisp) | com.inuoe.jzon v1.1.4 (MIT) | | | [`md5-demo.lisp`](md5-demo.lisp) | md5 v2.0.4 (public domain) | | | [`chipz-demo.lisp`](chipz-demo.lisp) | chipz 0.8 (BSD) — gzip/zlib/deflate decompression. Uses `catch`/`throw`, so both WASM runs need `-W exceptions=y` | | | [`cl-ppcre-demo.lisp`](cl-ppcre-demo.lisp) | cl-ppcre v2.1.2 (BSD 2-Clause) | | | [`ironclad-demo.lisp`](ironclad-demo.lisp) | ironclad v0.61, SHA-256/HMAC/PBKDF2/HKDF/SCRAM slice (BSD 3-Clause) | | | [`ironclad-rsa-demo.lisp`](ironclad-rsa-demo.lisp) | ironclad v0.61, the SHA-384/512 digests and the RSA public-key stack — `sign-message`/`verify-signature` with and without PSS, and `generate-key-pair` | | | [`jose-demo.lisp`](jose-demo.lisp) | jose (BSD 2-Clause) — JSON Object Signing and Encryption / JWT over HS256/384/512, RS256/384/512, PS256 and the unsecured `none`. Needs eight `--system-path` directories (jose plus cl-json, ironclad, cl-base64, split-sequence, assoc-utils, alexandria, trivial-utf-8) and, because it handles the correctable claim conditions, `-W exceptions=y` on both WASM runs | | | [`uax-15-demo.lisp`](uax-15-demo.lisp) | uax-15 v0.1.3 (MIT) | | | [`tiny-routes-demo.lisp`](tiny-routes-demo.lisp) | tiny-routes v0.1.1 (BSD 3-Clause). For a size-constrained module load the opt-in `"tiny-routes/lite"`, which drops the cl-ppcre dependency — see the [asdf-systems guide](https://github.com/making/rontolisp/blob/develop/examples/doc/en/guides/asdf-systems.md) | | | [`clack-hello.lisp`](clack-hello.lisp) | clack v2.1.0 + lack (MIT), served by the built-in `clack-handler-rontolisp` backend; loads via `ql:quickload` (network on the first run) | | jzon's three numeric leaf components (the eisel-lemire float reader and Schubfach float printer) are replaced at load time by built-in shims over rontolisp's native float arithmetic, so float text takes rontolisp's cross-backend-identical shape rather than Schubfach's shortest-round-trip string. ## Where the libraries come from The sources are vendored under `src/test/resources//` for the test suite, so the demos run out of the box from the repository root. Two of them have a wrinkle: jzon's `.asd` lives in its `src/` subdirectory, and only the SHA-2/HMAC/PBKDF2/HKDF/SCRAM/RSA slice of ironclad is vendored (its executable `ironclad.asd` is kept for provenance, but a bundled replacement is what loads). Alternatively, download the same versions from upstream and point `--system-path` (or the `RONTOLISP_SOURCE_REGISTRY` environment variable) at the directory containing the `.asd` file: ```bash curl -sL https://github.com/sharplispers/split-sequence/archive/refs/tags/v2.0.1.tar.gz | tar xz ``` ## Running (all four backends) From the repository root. `rontolisp` is the native binary; `java -jar target/rontolisp-0.1.0-SNAPSHOT-exec.jar` works identically: ```bash SYS=src/test/resources/split-sequence # 1. Interpreter rontolisp examples/asdf/split-sequence-demo.lisp --system-path $SYS # 2. JVM (the class is named after the output file, so keep it path-free) rontolisp examples/asdf/split-sequence-demo.lisp -o Prog.class --system-path $SYS && java Prog # 3. WASM Preview 1 (requires wasmtime 14+) rontolisp examples/asdf/split-sequence-demo.lisp -o demo.wasm --system-path $SYS && \ wasmtime run -W gc demo.wasm # 4. WASM component / WASI 0.3 (requires wasmtime 46+) rontolisp examples/asdf/split-sequence-demo.lisp -o demo-comp.wasm --component --system-path $SYS && \ wasmtime run -W gc=y demo-comp.wasm ``` `--system-path` takes ONE value, a `:`-joined list of directories, so a library with dependencies of its own names them all in the same argument: ```bash SYS=src/test/resources/uax-15:src/test/resources/split-sequence:src/test/resources/cl-ppcre rontolisp examples/asdf/uax-15-demo.lisp --system-path $SYS SYS=src/test/resources/tiny-routes:src/test/resources/cl-ppcre rontolisp examples/asdf/tiny-routes-demo.lisp --system-path $SYS ``` The compile path splices the system's component files in at compile time (the `.asd` must be on disk when compiling), so the produced `.class`/`.wasm` is self-contained. A demo using `handler-case`/`unwind-protect` compiles in EH mode, so both wasm run commands need `-W exceptions=y` (wasmtime 37+). `alexandria-demo.lisp` is one, and so is `tiny-routes-demo.lisp` (`with-input-from-string` expands to an `unwind-protect`). `mustache-demo.lisp` is the one demo that reads a file at run time — its `greeting.mustache` template, the path relative to the repository root — so its two WASM runs also preopen that root: add `--dir .` to both `wasmtime run` commands (which already carry `-W exceptions=y` for the missing-partial section). ## Expected output Each demo prints one line per API call it exercises (`mustache-demo.lisp`'s file rendering is the exception — it prints the rendered template, which spans several lines); `split-sequence-demo.lisp` starts: ```console ("a" "b" "" "c") ("a" "b" "c") ((1 2) (4 5) (6)) ``` The output is identical on every backend, which is what the E2E tests assert — so the demo itself is the specification, and any divergence is a real failure. ## What can be loaded today A library qualifies when it stays inside plain `defun`/`defmacro`/`defpackage` code, `loop`, multiple values, `check-type`/`etypecase` with the supported type specifiers, declarations (parsed no-ops) and the lite `define-condition`/`make-condition`/`warn`/`restart-case`/`return-from` idioms. Libraries built on the CLOS static subset, the lite condition system, dynamic (special) variables, Gray output streams and adjustable fill-pointered string buffers load too — jzon exercises all of these on every backend, and `restart-case`/`invoke-restart` are in (cl-mustache's missing-partial `use-value` restart runs on every backend). The [ASDF systems guide](https://github.com/making/rontolisp/blob/develop/examples/doc/en/guides/asdf-systems.md) has the supported subset. --- # FILE: references/examples/asdf/alexandria-demo.lisp ;; Loads the REAL alexandria 1.0.1 (public domain / 0-clause MIT) via ;; asdf:load-system and exercises its public API. Run with: ;; rontolisp examples/asdf/alexandria-demo.lisp --system-path src/test/resources/alexandria ;; (see examples/asdf/README.md for the compile-path variants; this demo uses ;; handler-case, so both wasm run commands need -W exceptions=y). (asdf:load-system :alexandria) ;; binding constructs (print (alexandria:if-let ((x (find 3 '(1 2 3)))) (* x 10) :none)) (print (alexandria:if-let ((x (find 9 '(1 2 3)))) (* x 10) :none)) (print (alexandria:when-let ((x 5)) (* x 2))) (print (alexandria:when-let* ((x 1) (y (+ x 1))) (list x y))) ;; control flow (print (alexandria:switch (3) (1 :one) (3 :three) (t :other))) (print (alexandria:eswitch (:a) (:a :got-a) (:b :got-b))) (print (alexandria:cswitch (2) (1 :one) (2 :two))) (print (alexandria:xor nil 3 nil)) (print (alexandria:whichever 42)) (print (alexandria:nth-value-or 0 (values nil :second) :fallback)) ;; definitions (alexandria:define-constant +greeting+ "hello" :test #'string=) (print +greeting+) ;; macro-writing macros (defmacro my-square (x) (alexandria:once-only (x) `(* ,x ,x))) (let ((calls 0)) (flet ((bump () (incf calls))) (print (my-square (bump))) (print calls))) (defmacro my-double (x) (alexandria:with-gensyms (g) `(let ((,g ,x)) (+ ,g ,g)))) (print (my-double 21)) (print (funcall (alexandria:named-lambda greeter (who) (list :hi who)) :you)) (print (alexandria:destructuring-case '(:add 1 2) ((:add a b) (list :sum (+ a b))) ((t &rest rest) rest))) (print (multiple-value-list (alexandria:parse-body '("doc" (+ 1 2)) :documentation t))) ;; functions (print (funcall (alexandria:compose #'1+ #'1+) 40)) (print (mapcar (alexandria:compose #'car #'cdr) '((1 2 3) (4 5 6)))) (print (funcall (alexandria:multiple-value-compose #'list (lambda (x) (values x (* x 10)))) 4)) (print (funcall (alexandria:curry #'+ 1 2) 3)) (print (funcall (alexandria:rcurry #'- 1) 10)) (print (funcall (alexandria:conjoin #'evenp #'plusp) 4)) (print (funcall (alexandria:disjoin #'evenp #'minusp) 3)) (print (funcall (alexandria:ensure-function #'car) '(1 2))) ;; lists (print (alexandria:iota 5)) (print (alexandria:iota 4 :start 1 :step 2)) (print (alexandria:flatten '(1 (2 (3 (4))) 5))) (print (alexandria:mappend #'list '(1 2) '(3 4))) (print (list (alexandria:proper-list-p '(1 2 3)) (alexandria:circular-list-p '(1 2 3)))) (print (list (alexandria:lastcar '(1 2 3)) (alexandria:ensure-list 1) (alexandria:ensure-cons '(1 2)))) (print (alexandria:alist-plist '((:a . 1) (:b . 2)))) (print (alexandria:plist-alist '(:a 1 :b 2))) (print (alexandria:remove-from-plist '(:a 1 :b 2 :c 3) :b)) (print (alexandria:delete-from-plist (list :a 1 :b 2) :a)) (print (list (alexandria:assoc-value '((:a . 1) (:b . 2)) :b) (alexandria:rassoc-value '((:a . 1) (:b . 2)) 1))) (print (list (alexandria:set-equal '(1 2 3) '(3 2 1)) (alexandria:setp '(1 2 2)))) (print (alexandria:doplist (k v '(:a 1 :b 2) :done) (print (list k v)))) ;; modify macros over setf places (let ((xs (list 1 2)) (n 3)) (alexandria:appendf xs '(3 4)) (alexandria:removef xs 2) (alexandria:maxf n 10) (alexandria:minf n 4) (print (list xs n))) ;; sequences (print (list (alexandria:emptyp '()) (alexandria:emptyp "x"))) (print (alexandria:length= '(1 2) #(3 4))) (print (list (alexandria:first-elt "abc") (alexandria:last-elt "abc") (alexandria:last-elt '(1 2 3)))) (print (list (alexandria:starts-with-subseq "he" "hello") (alexandria:ends-with-subseq "lo" "hello"))) (print (list (alexandria:starts-with 1 '(1 2 3)) (alexandria:ends-with 3 '(1 2 3)))) (print (alexandria:extremum '(3 1 4 1 5) #'<)) (print (alexandria:extremum '(3 1 4 1 5) #'>)) (print (alexandria:sequence-of-length-p '(1 2 3) 3)) (alexandria:map-combinations (lambda (c) (print c)) '(1 2 3) :length 2) (alexandria:map-permutations (lambda (p) (print p)) '(1 2) :length 2) (let* ((a (vector 1 2 3)) (b (alexandria:copy-array a))) (setf (aref b 0) 99) (print (list (aref a 0) (aref b 0)))) (print (alexandria:copy-sequence 'list #(1 2 3))) (print (alexandria:copy-sequence 'vector '(1 2 3))) (print (alexandria:rotate (list 1 2 3 4 5) 2)) (print (alexandria:rotate (list 1 2 3 4 5) -2)) (print (let ((x '(1 2))) (alexandria:coercef x 'vector) x)) ;; io (print (with-input-from-string (s "stream content") (alexandria:read-stream-content-into-string s))) ;; hash tables (results sorted -- iteration order is unspecified) (let ((h (alexandria:alist-hash-table '((:a . 1) (:b . 2)) :test #'eq))) (print (sort (alexandria:hash-table-keys h) #'string< :key #'symbol-name)) (print (sort (alexandria:hash-table-values h) #'<)) (print (alexandria:ensure-gethash :c h 3)) (print (gethash :c h))) (print (alexandria:hash-table-plist (alexandria:plist-hash-table '(:x 1)))) (print (alexandria:hash-table-alist (alexandria:copy-hash-table (alexandria:plist-hash-table '(:x 1))))) (alexandria:maphash-keys #'print (alexandria:plist-hash-table '(:x 1))) (alexandria:maphash-values #'print (alexandria:plist-hash-table '(:x 1))) ;; numbers (print (list (alexandria:clamp 15 0 10) (alexandria:clamp -1 0 10) (alexandria:clamp 5 0 10))) (print (alexandria:lerp 1/2 0 10)) (print (alexandria:mean '(1 2 3 4))) (print (alexandria:variance '(1 2 3 4))) (print (alexandria:factorial 10)) (print (alexandria:binomial-coefficient 10 3)) (print (list (alexandria:subfactorial 5) (alexandria:count-permutations 5 2))) (print (alexandria:median '(3 1 4 1 5))) ;; symbols (print (alexandria:symbolicate 'foo '- 'bar)) (print (alexandria:make-keyword "HELLO")) ;; conditions (print (handler-case (alexandria:required-argument :x) (error () :signalled))) (print (handler-case (alexandria:simple-parse-error "bad ~A" 1) (error () :parse-error))) (print (alexandria:ignore-some-conditions (error) (error "boom"))) (alexandria:unwind-protect-case () (print :body) (:normal (print :normal)) (:abort (print :abort))) ;; features (print (alexandria:featurep :rontolisp)) ;; alexandria-2 (print (alexandria-2:subseq* "hello world" 6)) (print (alexandria-2:line-up-first 5 (+ 1) (* 2))) (print (alexandria-2:line-up-last 5 (+ 1) (- 20))) (print (alexandria-2:delete-from-plist* (list :a 1 :b 2) :a)) (print (list (alexandria-2:dim-in-bounds-p '(2 3) 1 2) (alexandria-2:dim-in-bounds-p '(2 3) 2 2))) (print (alexandria-2:row-major-index '(2 3) 1 2)) (print (alexandria-2:rmajor-to-indices '(2 3) 5)) --- # FILE: references/examples/asdf/assoc-utils-demo.lisp ;; Loads the REAL assoc-utils (public domain, Eitaro Fukamachi) via ;; asdf:load-system and exercises its public alist read/convert API. Run with: ;; rontolisp examples/asdf/assoc-utils-demo.lisp --system-path src/test/resources/assoc-utils ;; (see examples/asdf/README.md for the compile-path variants). (asdf:load-system :assoc-utils) (defvar *a* (list (cons "name" "eitaro") (cons "loc" "vienna"))) ;; aget (assoc with *assoc-test*) + default (print (assoc-utils:aget *a* "name")) (print (assoc-utils:aget *a* "missing" "none")) ;; alist-keys / alist-values (mapcar car/cdr) (print (assoc-utils:alist-keys *a*)) (print (assoc-utils:alist-values *a*)) ;; alist-plist ((intern name :keyword) over the keys) and the inverse plist-alist ;; ((string-downcase key) over the keyword keys -- string-downcase accepts a string ;; designator, so a keyword coerces to its name). (print (assoc-utils:alist-plist *a*)) (print (assoc-utils:plist-alist (list :name "eitaro" :loc "vienna"))) ;; remove-from-alist (remove-if) and the define-modify-macro place variant (print (assoc-utils:remove-from-alist *a* "loc")) (let ((b (list (cons "x" 1) (cons "y" 2)))) (assoc-utils:delete-from-alistf b "x") (print b)) ;; alist-hash / hash-alist (loop being the hash-keys ... using (hash-value ...)) (let ((h (assoc-utils:alist-hash (list (cons "k" "v"))))) (print (assoc-utils:hash-alist h))) ;; with-keys (the alist equivalent of with-slots) (print (assoc-utils:with-keys ((nm "name") (lc "loc")) *a* (format nil "~a in ~a" nm lc))) ;; alist-get (reduce #'aget* over a key path; integer keys index into lists) (print (assoc-utils:alist-get (list (cons "user" (list (cons "age" 42)))) (list "user" "age"))) ;; alist= (equalp over string< :key #'car sorted copies) (print (if (assoc-utils:alist= (list (cons "a" "1") (cons "b" "2")) (list (cons "b" "2") (cons "a" "1"))) "equal" "different")) --- # FILE: references/examples/asdf/chipz-demo.lisp ;;;; chipz 0.8 (BSD): gzip / zlib / deflate decompression from unmodified ;;;; upstream sources. ;;;; ;;;; Run (all four backends), from the repository root: ;;;; SYS=src/test/resources/chipz ;;;; rontolisp examples/asdf/chipz-demo.lisp --system-path $SYS ;;;; rontolisp examples/asdf/chipz-demo.lisp -o Prog.class --system-path $SYS && java Prog ;;;; rontolisp examples/asdf/chipz-demo.lisp -o demo.wasm --system-path $SYS \ ;;;; && wasmtime run -W gc -W exceptions=y demo.wasm ;;;; rontolisp examples/asdf/chipz-demo.lisp -o demo-c.wasm --component --system-path $SYS \ ;;;; && wasmtime run -W gc=y -W exceptions=y demo-c.wasm ;;;; ;;;; chipz uses catch/throw, so every compiled artifact is in EH mode -- hence ;;;; -W exceptions=y on both WASM backends. (asdf:load-system "chipz") ;;; "Hello, chipz!" gzipped, as the 33 octets a gzip writer produces. (defparameter *gzipped* (make-array 33 :element-type '(unsigned-byte 8) :initial-contents '(31 139 8 0 0 0 0 0 2 255 243 72 205 201 201 215 81 72 206 200 44 168 82 4 0 46 239 228 135 13 0 0 0))) (defun octets-to-string (octets) (let ((out (make-array (length octets) :element-type 'character))) (dotimes (i (length octets) out) (setf (aref out i) (code-char (aref octets i)))))) ;;; Decompress into a fresh vector. (let ((raw (chipz:decompress nil 'chipz:gzip *gzipped*))) (format t "~a octets -> ~s~%" (length *gzipped*) (octets-to-string raw))) ;;; Decompress into a buffer you supply, which answers how much of each side ;;; was used -- the shape a streaming caller wants. (let ((buffer (make-array 64 :element-type '(unsigned-byte 8))) (state (chipz:make-dstate 'chipz:gzip))) (multiple-value-bind (consumed produced) (chipz:decompress buffer state *gzipped*) (format t "consumed ~a, produced ~a~%" consumed produced))) --- # FILE: references/examples/asdf/cl-base64-demo.lisp ;; Loads the REAL cl-base64 (BSD, Kevin M. Rosenberg) via asdf:load-system and ;; exercises its public encode/decode API. Run with: ;; rontolisp examples/asdf/cl-base64-demo.lisp --system-path src/test/resources/cl-base64 ;; (see examples/asdf/README.md for the compile-path variants). (asdf:load-system :cl-base64) ;; string <-> base64 string (the names are synthesized at macro-expansion time ;; by (intern (concatenate 'string (symbol-name input-type) ...))) (print (cl-base64:string-to-base64-string "Hello, World!")) (print (cl-base64:base64-string-to-string "SGVsbG8sIFdvcmxkIQ==")) ;; :columns wraps the output with newlines (print (cl-base64:string-to-base64-string "Hello, World!" :columns 5)) ;; :uri t uses the URI-safe alphabet (- _ and . padding) (print (cl-base64:string-to-base64-string "Hello?>>" :uri t)) (print (cl-base64:base64-string-to-string "SGVsbG8_Pj4." :uri t)) ;; (unsigned-byte 8) arrays (print (cl-base64:usb8-array-to-base64-string (make-array 3 :element-type '(unsigned-byte 8) :initial-contents '(1 2 3)))) (print (cl-base64:base64-string-to-usb8-array "AQID")) ;; integers (exact on every backend within the signed 64-bit range the ;; WASM backends carry) (print (cl-base64:integer-to-base64-string 1234567)) (print (cl-base64:base64-string-to-integer "EtaH")) ;; a bad input character signals bad-base64-character, caught by handler-case (print (handler-case (cl-base64:base64-string-to-string "SGVsbG8@") (error (e) :caught-bad-char))) --- # FILE: references/examples/asdf/cl-ppcre-demo.lisp ;; Loads the REAL cl-ppcre (BSD-2-Clause, Dr. Edmund Weitz) via asdf:load-system ;; and runs the Perl-compatible regex API. Run with: ;; rontolisp examples/asdf/cl-ppcre-demo.lisp --system-path src/test/resources/cl-ppcre ;; Runs on all four backends: the scanner closures rely on named block/return-from ;; crossing loops, which the compile backends implement as lexical named exits. (asdf:load-system :cl-ppcre) ;; scan: match bounds plus register bounds as four values (print (multiple-value-list (cl-ppcre:scan "(a)*b" "xaaabd"))) ;; scan-to-strings: the whole match plus a register vector (print (multiple-value-list (cl-ppcre:scan-to-strings "(\\d+)-(\\d+)" "phone 03-1234"))) ;; split, with a regex separator (print (cl-ppcre:split "\\s+" "foo bar baz")) ;; replacement, single and global (print (cl-ppcre:regex-replace "fo+" "foo bar" "frob")) (print (cl-ppcre:regex-replace-all "a" "banana" "o")) ;; all matches as strings (print (cl-ppcre:all-matches-as-strings "[a-z]+" "one 2 three 4 five")) ;; the iteration macros (do-scans family builds on &environment + get-setf-expansion) (let ((acc nil)) (cl-ppcre:do-matches-as-strings (m "[0-9]+" "a1 b22 c333") (push m acc)) (print (nreverse acc))) ;; register-groups-bind destructures the registers by name (print (cl-ppcre:register-groups-bind (area num) ("(\\d+)-(\\d+)" "tel 03-1234 end") (list area num))) ;; a parse tree instead of a regex string (print (cl-ppcre:scan-to-strings '(:sequence "b" (:greedy-repetition 1 nil #\a)) "xbaaay")) ;; (?i) inline modifier (print (cl-ppcre:scan-to-strings "(?i)hello|bye" "say HELLO now")) --- # FILE: references/examples/asdf/cl-utilities-demo.lisp ;; Loads the REAL cl-utilities v1.2.4 (public domain) via asdf:load-system and ;; exercises its whole public API. Run with: ;; rontolisp examples/asdf/cl-utilities-demo.lisp --system-path src/test/resources/cl-utilities ;; (see examples/asdf/README.md for the compile-path variants). (asdf:load-system :cl-utilities) ;; split-sequence (cl-utilities' own copy, via apply #'position) (print (cl-utilities:split-sequence #\, "a,b,,c")) (print (cl-utilities:split-sequence #\, "a,b,,c" :remove-empty-subseqs t)) (print (cl-utilities:split-sequence-if #'evenp '(1 2 3 4 5))) (print (cl-utilities:split-sequence-if-not #'oddp '(1 2 3 4 5))) ;; extremum family (once-only / with-check-length macro templates) (print (cl-utilities:extremum '(3 1 4 1 5 9 2 6) #'<)) (print (cl-utilities:extremum '(3 1 4 1 5 9 2 6) #'>)) (print (cl-utilities:extremum '((1 . "one") (3 . "three") (2 . "two")) #'> :key #'car)) (print (cl-utilities:extremum '(9 8 3 1 2) #'< :start 2)) (print (cl-utilities:extremum-fastkey '(3 1 4 1 5) #'< :key #'identity)) (print (cl-utilities:extrema '(3 1 4 1 5 9 2 6 1) #'<)) (print (cl-utilities:n-most-extreme 3 '(3 1 4 1 5 9 2 6) #'<)) ;; read-delimited (read-char + multiple-value-setq + (setf (elt ...))) (print (with-input-from-string (s "hello,world") (let ((buf (make-array 20 :initial-element nil))) (multiple-value-bind (pos found) (cl-utilities:read-delimited buf s :delimiter #\,) (list pos found (subseq (coerce buf 'list) 0 pos)))))) ;; expt-mod (print (cl-utilities:expt-mod 2 10 1000)) (print (cl-utilities:expt-mod 12 34 235)) ;; collecting / with-collectors (tail collection) (print (cl-utilities:collecting (dotimes (x 5) (cl-utilities:collect (* x x))))) (print (multiple-value-list (cl-utilities:with-collectors (evens odds) (dolist (n '(1 2 3 4 5 6)) (if (evenp n) (evens n) (odds n)))))) ;; once-only / with-unique-names / with-gensyms used from user macros (defmacro my-square (x) (cl-utilities:once-only (x) `(* ,x ,x))) (let ((counter 0)) (flet ((bump () (incf counter))) (print (my-square (bump))) (print counter))) (defmacro my-swap (a b) (cl-utilities:with-unique-names (tmp) `(let ((,tmp ,a)) (setq ,a ,b) (setq ,b ,tmp) (list ,a ,b)))) (let ((p 1) (q 2)) (print (my-swap p q))) (defmacro my-double (x) (cl-utilities:with-gensyms (g) `(let ((,g ,x)) (+ ,g ,g)))) (print (my-double 21)) ;; rotate-byte (return-from) (print (cl-utilities:rotate-byte 3 (byte 8 0) 1)) (print (cl-utilities:rotate-byte 2 (byte 8 0) 255)) (print (cl-utilities:rotate-byte -1 (byte 4 0) 1)) ;; copy-array (apply #'make-array) (let* ((a (vector 1 2 3)) (b (cl-utilities:copy-array a))) (setf (aref b 0) 99) (print (list (aref a 0) (aref b 0)))) ;; compose (reduce #'funcall :from-end) (print (funcall (cl-utilities:compose #'1+ #'1+) 40)) (print (mapcar (cl-utilities:compose #'car #'cdr) '((1 2 3) (4 5 6)))) --- # FILE: references/examples/asdf/cl-who-demo.lisp ;;;; cl-who via asdf:load-system ;;;; Loads the REAL cl-who v1.1.5 (Edi Weitz's unmodified upstream sources, ;;;; BSD) and renders (X)HTML through with-html-output-to-string. cl-who's ;;;; markup macros run a chain of ordinary defuns (and a generic function) AT ;;;; MACRO-EXPANSION TIME, so the whole template is expanded before codegen and ;;;; the produced HTML string is a compile-time constant on the compile paths. ;;;; str / esc / fmt splice evaluated / escaped / formatted content in, and ;;;; (setf (html-mode) ...) switches between the default :xml self-closing tags ;;;; and :html5 void tags. Runs identically on all four backends; see README.md ;;;; in this directory for the run commands (the library directory is passed ;;;; with --system-path). ;;;; ;;;; Run (library vendored in this repository): ;;;; rontolisp examples/asdf/cl-who-demo.lisp --system-path src/test/resources/cl-who (asdf:load-system :cl-who) ;; A full document: nested tags, an attribute, and text nodes. Attributes ;; render with single quotes; :xml (the default) self-closes empty tags. (princ (cl-who:with-html-output-to-string (s) (:html (:head (:title "Hi")) (:body (:p "Hello" (:a :href "/x" "link")))))) (terpri) ;; str evaluates a Lisp form and inserts its princ output; esc HTML-escapes ;; the special characters; fmt is an inline (format nil ...). (princ (cl-who:with-html-output-to-string (s) (:div (:span (cl-who:str (+ 1 2))) (:span (cl-who:esc "")) (:span (cl-who:fmt "~a-~a" 3 4))))) (terpri) ;; Default :xml mode -- empty elements self-close as "
". (princ (cl-who:with-html-output-to-string (s) (:br))) (terpri) ;; Switch to HTML5 -- void elements render as bare "
". (setf (cl-who:html-mode) :html5) (princ (cl-who:with-html-output-to-string (s) (:br))) (terpri) ;; Back to :xml; esc emits a numeric character reference for non-ASCII. (setf (cl-who:html-mode) :xml) (princ (cl-who:with-html-output-to-string (s) (:p (cl-who:esc (string (code-char 233)))))) (terpri) --- # FILE: references/examples/asdf/clack-hello.lisp ;;;; Clack via ql:quickload -- clackup on the built-in rontolisp handler backend ;;;; Loads the REAL clack (unmodified upstream sources; Eitaro Fukamachi, MIT) ;;;; together with lack, and serves a Clack application through clack:clackup. ;;;; The :server :rontolisp backend is the built-in clack-handler-rontolisp ;;;; shim system, resolved by name at run time exactly the way clack finds any ;;;; handler backend. :use-thread nil keeps the process serving in the ;;;; foreground (Ctrl-C to stop), the script shape; in a REPL-style program the ;;;; default :use-thread t returns a handler object instead and ;;;; (clack:stop handler) shuts the server down. ;;;; ;;;; Run (downloads clack/lack into ~/.rontolisp/quicklisp on the first run): ;;;; rontolisp examples/asdf/clack-hello.lisp ;;;; curl http://127.0.0.1:5000/hello ;;;; ;;;; JVM (the served program needs the rontolisp jar on the runtime classpath): ;;;; rontolisp examples/asdf/clack-hello.lisp -o ClackHello.class ;;;; java -cp rontolisp-exec.jar:. ClackHello ;;;; ;;;; WASM component (the host owns the socket; the port argument is ignored): ;;;; rontolisp examples/asdf/clack-hello.lisp -o clack-hello.wasm --component ;;;; wasmtime serve -W gc=y -W exceptions=y -S cli=y -S tcp=y -S inherit-network=y clack-hello.wasm ;;;; ;;;; WASM Preview 1 has no incoming TCP by design: the program compiles, and ;;;; clackup signals "HTTP-HANDLER requires --component ..." at run time. (ql:quickload "clack") (clack:clackup (lambda (env) (list 200 '(:content-type "text/plain") (list (format nil "Hello, Clack on rontolisp! ~A ~A~%" (getf env :request-method) (getf env :path-info))))) :server :rontolisp :port 5000 :use-thread nil) --- # FILE: references/examples/asdf/ironclad-demo.lisp ;; Loads the SHA-256 / HMAC / PBKDF2 / HKDF slice of the REAL ironclad (BSD 3-Clause, ;; Nathan Froyd / Guillaume LE VAILLANT) via asdf:load-system and reproduces ;; published test vectors. Run with: ;; rontolisp examples/asdf/ironclad-demo.lisp --system-path src/test/resources/ironclad ;; Runs on all four backends. ironclad's own ironclad.asd is an executable ;; program (component classes, a defsystem-generating macro), so rontolisp ;; substitutes a bundled replacement .asd declaring the loadable slice -- the ;; component files loaded are the library's real ones. The slice also carries ;; SHA-384/512 and the RSA public-key stack -- see ironclad-rsa-demo.lisp; ;; ciphers, AEAD modes, the Fortuna PRNG and the non-RSA public-key algorithms ;; are outside it. (asdf:load-system :ironclad) ;; FIPS 180-2: SHA-256 of "abc" and of a two-block message (print (ironclad:byte-array-to-hex-string (ironclad:digest-sequence :sha256 (ironclad:ascii-string-to-byte-array "abc")))) (print (ironclad:byte-array-to-hex-string (ironclad:digest-sequence :sha256 (ironclad:ascii-string-to-byte-array "abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq")))) ;; SHA-224 shares the SHA-256 compression function (print (ironclad:byte-array-to-hex-string (ironclad:digest-sequence :sha224 (ironclad:ascii-string-to-byte-array "abc")))) ;; RFC 4231 test case 2: HMAC-SHA-256 through the make-mac API (let ((mac (ironclad:make-mac :hmac (ironclad:ascii-string-to-byte-array "Jefe") :sha256))) (ironclad:update-mac mac (ironclad:ascii-string-to-byte-array "what do ya want for nothing?")) (print (ironclad:byte-array-to-hex-string (ironclad:produce-mac mac)))) ;; RFC 5869 test case 1: HKDF-SHA-256 (make-kdf :hmac-kdf), whose output builder ;; concatenates its blocks into a '(vector (unsigned-byte 8)) (let ((kdf (ironclad:make-kdf :hmac-kdf :digest :sha256 :additional-data (ironclad:hex-string-to-byte-array "f0f1f2f3f4f5f6f7f8f9")))) (print (ironclad:byte-array-to-hex-string (ironclad:derive-key kdf (ironclad:hex-string-to-byte-array "0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b") (ironclad:hex-string-to-byte-array "000102030405060708090a0b0c") 1 42)))) ;; PBKDF2-HMAC-SHA-256, 4096 iterations (let ((kdf (ironclad:make-kdf :pbkdf2 :digest :sha256))) (print (ironclad:byte-array-to-hex-string (ironclad:derive-key kdf (ironclad:ascii-string-to-byte-array "password") (ironclad:ascii-string-to-byte-array "salt") 4096 32)))) ;; SCRAM-SHA-256 (RFC 7677 section 3), the sequence a PostgreSQL client runs to ;; authenticate: SaltedPassword -> ClientKey -> StoredKey -> ClientSignature -> ;; ClientProof. The proof step XORs two 32-byte digests as 256-bit INTEGERS, so ;; it needs arbitrary-precision exact integers -- which every backend has. (defun scram-hmac (key message) (ironclad:hmac-digest (ironclad:update-hmac (ironclad:make-hmac key :sha256) (ironclad:ascii-string-to-byte-array message)))) ;; integer-to-octets returns the MINIMAL vector, so a proof whose high bytes ;; cancel comes back shorter than 32 and has to be padded back on the left. (defun pad-octet-vector (vector desired-length) (let ((length (length vector))) (if (= desired-length length) vector (replace (make-array desired-length :element-type '(unsigned-byte 8) :initial-element 0) vector :start1 (- desired-length length))))) (let* ((salted (ironclad:pbkdf2-hash-password (ironclad:ascii-string-to-byte-array "pencil") :salt (ironclad:hex-string-to-byte-array "5b6d99689d12358eeca04b141236fa81") :digest :sha256 :iterations 4096)) (nonce "rOprNGfwEbeRWgbNEkqO%hvYDpWUa2RaTCAfuxFIlj)hNlF$k0") (auth-message (concatenate 'string "n=user,r=rOprNGfwEbeRWgbNEkqO,r=" nonce ",s=W22ZaJ0SNY7soEsUEjb6gQ==,i=4096,c=biws,r=" nonce)) (client-key (scram-hmac salted "Client Key")) (stored-key (ironclad:digest-sequence :sha256 client-key)) (client-signature (scram-hmac stored-key auth-message))) (print (ironclad:byte-array-to-hex-string salted)) (print (ironclad:byte-array-to-hex-string (pad-octet-vector (ironclad:integer-to-octets (logxor (ironclad:octets-to-integer client-key) (ironclad:octets-to-integer client-signature))) 32)))) --- # FILE: references/examples/asdf/ironclad-rsa-demo.lisp ;; The SHA-384/512 and RSA half of the REAL ironclad slice (BSD 3-Clause, ;; Nathan Froyd / Guillaume LE VAILLANT), loaded via asdf:load-system. Run with: ;; rontolisp examples/asdf/ironclad-rsa-demo.lisp --system-path src/test/resources/ironclad ;; Runs on all four backends. Its sibling ironclad-demo.lisp covers the ;; SHA-256 / HMAC / PBKDF2 / HKDF / SCRAM half; this one loads the same real ;; sources for src/math.lisp and src/public-key/{public-key,pkcs1,rsa}.lisp. (asdf:load-system :ironclad) ;; FIPS 180-2: SHA-384 and SHA-512 of "abc". One file defines both -- one ;; 64-bit compression function, two initial states. (print (ironclad:byte-array-to-hex-string (ironclad:digest-sequence :sha384 (ironclad:ascii-string-to-byte-array "abc")))) (print (ironclad:byte-array-to-hex-string (ironclad:digest-sequence :sha512 (ironclad:ascii-string-to-byte-array "abc")))) ;; RFC 4231 test case 2: HMAC-SHA-512 (let ((mac (ironclad:make-mac :hmac (ironclad:ascii-string-to-byte-array "Jefe") :sha512))) (ironclad:update-mac mac (ironclad:ascii-string-to-byte-array "what do ya want for nothing?")) (print (ironclad:byte-array-to-hex-string (ironclad:produce-mac mac)))) ;; A fixed 2048-bit RSA key pair, so the raw signature below is a CONSTANT -- ;; it is m^d mod n of the SHA-256 digest of "abc", reproducible outside ;; ironclad with any bignum library. (defparameter *n* (ironclad:octets-to-integer (ironclad:hex-string-to-byte-array "d01c708f91a9038f62a5fd55ce3d1454857a220f92b33c4fb8c1b86840b6064099088053a3be5a1aeda9c54fb0c44b7d373bd097f282ab99e4b8fed626aafc41739597981387370ca20abe05839567c21422b42392eba320e4cafe0bece676420cbbe501a36cd19b9947bf18f5d6708651a3d5286085ff42cbd16d76573cc166486fabcdad197ce8756a905309d87c30a07e4c313a3c721b1e25b4cbc2f9c1e94275ee7eabc37a987bd646aa4c4d04ed4bea3912e4fd2980f9486352cc5282a5ffac929a6430e3cef133e1af96562de2abecd6a39c03dc30f5237fff1f67a45243d32ee53d8f4c2aa8febdda3e3953fcca0f762f0c24ac6ea88c047a80a2e245"))) (defparameter *d* (ironclad:octets-to-integer (ironclad:hex-string-to-byte-array "04cb4eed83c3c2bcfc1f0a4bbe7d4a395b3cd1c98d8ddafb142cc448848b1cf042863f5c8de65df1865d9599cd1eec85452f37d234483dd744fd5d03866f0472268d40e98433a67940476292c271ffeaa8e796c246096f1fdc1d70064ace1155dab0be6910007b00ac628a7cb2f71e6efdb4fa3d5ca1e19c42913fc60ce2edaa98a8266dc28bc5bbe1ff7a2aa61c604dce677c50e6b5b00cc560724afa64a23915bb78476f42b348a55cb33312e0712b084f24a85d2fe8ea425028757145430b38522048f1463c4a538d640c8caf109a15e8df2c0cbf4df93fc3e957734b92bbf18f577aedc5ad34de6a82dbf14b3ab601f719aed8422df046c3bfd79653fa51"))) (defparameter *private-key* (ironclad:make-private-key :rsa :d *d* :n *n*)) (defparameter *public-key* (ironclad:make-public-key :rsa :e 65537 :n *n*)) (defparameter *message* (ironclad:digest-sequence :sha256 (ironclad:ascii-string-to-byte-array "abc"))) (defparameter *signature* (ironclad:sign-message *private-key* *message*)) (print (ironclad:byte-array-to-hex-string *signature*)) (print (ironclad:verify-signature *public-key* *message* *signature*)) ;; ... and it rejects a signature over a different message. (print (ironclad:verify-signature *public-key* (ironclad:digest-sequence :sha256 (ironclad:ascii-string-to-byte-array "abd")) *signature*)) ;; PSS salts every signature, so what repeats is the round trip, not the bytes. ;; The salt comes from rontolisp:random-bytes, the entropy source every backend ;; has (ironclad's own Fortuna generator is outside the slice). (let ((signature (ironclad:sign-message *private-key* *message* :pss :sha256))) (print (ironclad:verify-signature *public-key* *message* signature :pss :sha256))) ;; Key generation draws its primes from the same source. (multiple-value-bind (private public) (ironclad:generate-key-pair :rsa :num-bits 1024) (let ((signature (ironclad:sign-message private *message*))) (print (integer-length (getf (ironclad:destructure-private-key private) :n))) (print (ironclad:verify-signature public *message* signature)))) --- # FILE: references/examples/asdf/jose-demo.lisp ;; jose (BSD 2-Clause, Eitaro Fukamachi) -- JSON Object Signing and Encryption ;; / JWT -- loaded via asdf:load-system from the REAL upstream sources. Run with: ;; SYS=src/test/resources/jose:src/test/resources/cl-json:src/test/resources/ironclad:src/test/resources/cl-base64:src/test/resources/split-sequence:src/test/resources/assoc-utils:src/test/resources/alexandria:src/test/resources/trivial-utf-8 ;; rontolisp examples/asdf/jose-demo.lisp --system-path $SYS ;; Runs on all four backends. It uses handler-case, so both wasm runs need ;; -W exceptions=y. (asdf:load-system :jose) (defparameter *secret* (ironclad:ascii-string-to-byte-array "my$ecret")) ;; HS256. The token is a pure function of the claims, and this one is the very ;; token jose's own README publishes -- Python's hmac/hashlib agrees byte for ;; byte, which is the only thing that makes a JWT worth issuing. (defparameter *token* (jose:encode :hs256 *secret* '(("hello" . "world")))) (print *token*) ;; decode VERIFIES the signature, then hands back the claims and the headers. (multiple-value-bind (claims headers) (jose:decode :hs256 *secret* *token*) (print claims) (print headers)) ;; inspect-token does not verify. Its third value is the raw signature, which ;; for HS256 is the 32 bytes of the HMAC. (multiple-value-bind (claims headers signature) (jose:inspect-token *token*) (print claims) (print headers) (print (length signature))) ;; HS384 and HS512 need SHA-384/512; :none is the unsecured token, whose ;; signature is the empty string. (print (jose:encode :hs384 *secret* '(("hello" . "world")))) (print (jose:encode :hs512 *secret* '(("hello" . "world")))) (print (jose:encode :none nil '(("hello" . "world")))) ;; The registered claims. iat / nbf / exp must decode as INTEGERS -- every one ;; of jose's checks is an integerp guard -- and :issuer / :audience / :subject ;; are checked against the claims of the same name. (defparameter *now* (- (get-universal-time) 2208988800)) (defparameter *claims-token* (jose:encode :hs256 *secret* `(("iss" . "rontolisp") ("aud" . "example") ("sub" . "42") ("iat" . ,*now*) ("nbf" . ,*now*) ("exp" . ,(+ *now* 3600))))) (print (mapcar #'car (jose:decode :hs256 *secret* *claims-token* :issuer "rontolisp" :audience "example" :subject "42"))) ;; A claim that fails its check signals. An expired exp goes through cerror, so ;; a handler-bind that continues decodes anyway -- that is how a caller says ;; "I know, give me the claims". (defparameter *expired* (jose:encode :hs256 *secret* `(("exp" . ,(- *now* 1000))))) (print (handler-case (jose:decode :hs256 *secret* *expired*) (jose/errors:jwt-claims-expired () :expired))) (print (mapcar #'car (handler-bind ((jose/errors:jwt-claims-expired #'continue)) (jose:decode :hs256 *secret* *expired*)))) ;; The wrong key is a jws-verification-error (also correctable); a token that is ;; not three dot-separated parts is a jws-invalid-format. (print (handler-case (jose:decode :hs256 (ironclad:ascii-string-to-byte-array "wrong") *token*) (jose/errors:jws-verification-error () :bad-signature))) (print (handler-case (jose:decode :hs256 *secret* "not.a.jwt") (jose/errors:jws-invalid-format () :malformed))) ;; RS256 over a fixed 2048-bit key pair. RSA keys are ironclad objects, so any ;; source of one works -- generate-key-pair, or a parser for a PEM file. (defparameter *n* (ironclad:octets-to-integer (ironclad:hex-string-to-byte-array "d01c708f91a9038f62a5fd55ce3d1454857a220f92b33c4fb8c1b86840b6064099088053a3be5a1aeda9c54fb0c44b7d373bd097f282ab99e4b8fed626aafc41739597981387370ca20abe05839567c21422b42392eba320e4cafe0bece676420cbbe501a36cd19b9947bf18f5d6708651a3d5286085ff42cbd16d76573cc166486fabcdad197ce8756a905309d87c30a07e4c313a3c721b1e25b4cbc2f9c1e94275ee7eabc37a987bd646aa4c4d04ed4bea3912e4fd2980f9486352cc5282a5ffac929a6430e3cef133e1af96562de2abecd6a39c03dc30f5237fff1f67a45243d32ee53d8f4c2aa8febdda3e3953fcca0f762f0c24ac6ea88c047a80a2e245"))) (defparameter *d* (ironclad:octets-to-integer (ironclad:hex-string-to-byte-array "04cb4eed83c3c2bcfc1f0a4bbe7d4a395b3cd1c98d8ddafb142cc448848b1cf042863f5c8de65df1865d9599cd1eec85452f37d234483dd744fd5d03866f0472268d40e98433a67940476292c271ffeaa8e796c246096f1fdc1d70064ace1155dab0be6910007b00ac628a7cb2f71e6efdb4fa3d5ca1e19c42913fc60ce2edaa98a8266dc28bc5bbe1ff7a2aa61c604dce677c50e6b5b00cc560724afa64a23915bb78476f42b348a55cb33312e0712b084f24a85d2fe8ea425028757145430b38522048f1463c4a538d640c8caf109a15e8df2c0cbf4df93fc3e957734b92bbf18f577aedc5ad34de6a82dbf14b3ab601f719aed8422df046c3bfd79653fa51"))) (defparameter *private-key* (ironclad:make-private-key :rsa :d *d* :n *n*)) (defparameter *public-key* (ironclad:make-public-key :rsa :e 65537 :n *n*)) ;; RS* is deterministic PKCS#1 v1.5, so this token is a constant. (defparameter *rs-token* (jose:encode :rs256 *private-key* '(("hello" . "world")))) (print *rs-token*) (print (jose:decode :rs256 *public-key* *rs-token*)) ;; PS* is RSA-PSS, which salts every signature -- what repeats is the round ;; trip, not the token. (print (jose:decode :ps256 *public-key* (jose:encode :ps256 *private-key* '(("hello" . "world"))))) --- # FILE: references/examples/asdf/jzon-demo.lisp ;;;; com.inuoe.jzon via ql:quickload / asdf:load-system ;;;; Loads the REAL jzon v1.1.4 (unmodified upstream sources) and exercises ;;;; JSON parsing and stringification: scalars, nested structures and a ;;;; round-trip. Its dependencies (closer-mop, flexi-streams, ;;;; float-features, trivial-gray-streams, uiop) resolve to rontolisp's ;;;; built-in shim systems, and its numeric leaf components (the ;;;; eisel-lemire float reader / Schubfach float printer) are replaced at ;;;; load time by shims over rontolisp's native float arithmetic -- so the ;;;; same program runs on all four backends; see the systems guide. ;;;; ;;;; Run (library vendored in this repository): ;;;; rontolisp examples/asdf/jzon-demo.lisp --system-path src/test/resources/jzon/src (asdf:load-system :com.inuoe.jzon) ;; A shorthand for the long package name (the README idiom). (uiop:add-package-local-nickname '#:jzon '#:com.inuoe.jzon) ;; Scalars parse to the natural rontolisp values. (print (jzon:parse "42")) (print (jzon:parse "-1.5")) (print (jzon:parse "\"hello\"")) (print (jzon:parse "true")) (print (jzon:parse "null")) ;; An array parses to a vector, an object to a hash table. (print (jzon:parse "[1, 2, 3]")) (let ((obj (jzon:parse "{\"name\": \"rontolisp\", \"tags\": [\"lisp\", \"wasm\"]}"))) (print (gethash "name" obj)) (print (gethash "tags" obj))) ;; Stringify renders JSON text; :pretty adds newlines and indentation. (write-string (jzon:stringify #(1 2 3))) (terpri) (let ((table (make-hash-table))) (setf (gethash "a" table) 1) (write-string (jzon:stringify table)) (terpri)) ;; Round-trip: parse then stringify. (write-string (jzon:stringify (jzon:parse "{\"k\": [true, null, 7]}"))) (terpri) --- # FILE: references/examples/asdf/md5-demo.lisp ;; Loads the REAL md5 (public domain, Pierre R. Mai) via asdf:load-system and ;; digests the RFC 1321 A.5 test vectors. Run with: ;; rontolisp examples/asdf/md5-demo.lisp --system-path src/test/resources/md5 ;; Runs on all four backends: the unsigned 32-bit MD5 working state rides the ;; WASM backends' boxed 64-bit integer path. (asdf:load-system :md5) (defun hex (digest) (string-downcase (with-output-to-string (s) (dotimes (i (length digest)) (format s "~2,'0X" (aref digest i)))))) ;; RFC 1321 A.5 test vectors (print (hex (md5:md5sum-sequence ""))) (print (hex (md5:md5sum-sequence "abc"))) (print (hex (md5:md5sum-sequence "message digest"))) ;; the (unsigned-byte 8) vector shape (what a database driver hands it) (let ((v (make-array 3 :element-type '(unsigned-byte 8) :initial-contents '(97 98 99)))) (print (hex (md5:md5sum-sequence v)))) ;; md5sum-string UTF-8-encodes through the flexi-streams shim first (print (hex (md5:md5sum-string "日本語"))) ;; the incremental API: same digest, fed in two chunks (let ((state (md5:make-md5-state))) (md5:update-md5-state state "ab") (md5:update-md5-state state "c") (print (hex (md5:finalize-md5-state state)))) --- # FILE: references/examples/asdf/mustache-demo.lisp ;;;; cl-mustache via asdf:load-system ;;;; Loads the REAL cl-mustache 0.12.3 (Kan-Ru Chen's unmodified upstream ;;;; sources, MIT/Expat) and renders Mustache templates. render* answers a ;;;; string, render writes to mustache:*output-stream* or to a stream argument, ;;;; and compile-template returns a closure you can call many times. The ;;;; template is a STRING (its body) or a PATHNAME (read from a file -- ;;;; greeting.mustache in this directory, this demo's only runtime file I/O, ;;;; so the WASM runs need `wasmtime run --dir .`). A context is an alist, a ;;;; hash table, or a mustache:make-context carrying :partials. The library is ;;;; spec 1.1.2 compliant (194-case suite; the 36 it does not pass fail ;;;; identically on SBCL), and it renders the same on all four backends; see ;;;; README.md in this directory for the run commands (the library directory ;;;; is passed with --system-path). ;;;; ;;;; Run (library vendored in this repository): ;;;; rontolisp examples/asdf/mustache-demo.lisp --system-path src/test/resources/cl-mustache (asdf:load-system :cl-mustache) ;; Interpolation from an alist context. {{name}} escapes HTML, ;; {{{name}}} (and {{&name}}) inserts it raw. (princ (mustache:render* "Hello, {{name}}! {{tag}} vs {{{tag}}}" '((:name . "World") (:tag . "")))) (terpri) ;; A section over a list of alists repeats its body once per element; an ;; inverted section renders only when the key is absent or false. (princ (mustache:render* "{{#items}}- {{name}} x{{qty}} {{/items}}{{^items}}(nothing){{/items}}" '((:items . (((:name . "pen") (:qty . 2)) ((:name . "ink") (:qty . 3))))))) ;; A hash table works as a context too -- keys are looked up upcased. (princ (mustache:render* "{{greeting}}, {{name}}!" (let ((ctx (make-hash-table :test #'equal))) (setf (gethash "GREETING" ctx) "Hi") (setf (gethash "NAME" ctx) "mustache") ctx))) (terpri) ;; The template itself can live in a file: render and compile-template ;; dispatch on it -- a PATHNAME reads the file, a STRING is the template body ;; itself, so the namestring "greeting.mustache" verbatim would render the ;; filename, not the file. The read is this demo's only runtime file I/O; the ;; WASM builds need `wasmtime run --dir .` so the path resolves against the ;; preopened directory. The same file, the two context kinds: (mustache:render (pathname "examples/asdf/greeting.mustache") '((:name . "rontolisp") (:items . (((:name . "interpreter") (:qty . 1)) ((:name . "jvm") (:qty . 2)) ((:name . "wasm") (:qty . 3)))))) (mustache:render (pathname "examples/asdf/greeting.mustache") (let ((ctx (make-hash-table :test #'equal))) (setf (gethash "NAME" ctx) "mustache") (setf (gethash "ITEMS" ctx) nil) ctx)) ;; make-context carries partials: {{>name}} splices one in, and {{>*name}} ;; picks the partial whose name the data supplies (a "dynamic name"). (princ (mustache:render* "{{>greet}} / [{{>*which}}]" (mustache:make-context :data '((:name . "Ronto") (:which . "b")) :partials '(("greet" . "Hello, {{name}}") ("a" . "A") ("b" . "B"))))) (terpri) ;; A function in the context is a lambda section: it receives the raw section ;; text and its result is rendered as a template in turn. (princ (mustache:render* "{{#shout}}hello {{name}}{{/shout}}" (list (cons :name "world") (cons :shout (lambda (text) (string-upcase text)))))) (terpri) ;; compile-template parses once and returns a renderer; mustache:define binds ;; that renderer to a name. (let ((row (mustache:compile-template "| {{a}} | {{b}} |"))) (princ (with-output-to-string (out) (funcall row '((:a . 1) (:b . 2)) out))) (terpri) (princ (with-output-to-string (out) (funcall row '((:a . 3) (:b . 4)) out))) (terpri)) (mustache:define banner "== {{title}} ==") (princ (with-output-to-string (mustache:*output-stream*) (banner '((:title . "done"))))) (terpri) ;; A partial that cannot be found signals mustache:partial-cant-be-found with a ;; use-value restart, so a handler can substitute a template instead. (princ (handler-bind ((mustache:partial-cant-be-found (lambda (c) (declare (ignore c)) (use-value "{{fallback}}")))) (mustache:render* "<{{>missing}}>" (mustache:make-context :data '((:fallback . "substituted")) :partials nil)))) (terpri) --- # FILE: references/examples/asdf/parse-number-demo.lisp ;;;; parse-number via asdf:load-system ;;;; Loads the REAL parse-number v1.8 (unmodified upstream sources) and ;;;; parses integers, ratios, floats, radix-prefixed literals and exponent ;;;; markers without going through the reader. Runs identically on all four ;;;; backends; see README.md in this directory for the run commands (the ;;;; library directory is passed with --system-path). ;;;; ;;;; Run (library vendored in this repository): ;;;; rontolisp examples/asdf/parse-number-demo.lisp --system-path src/test/resources/parse-number (asdf:load-system :parse-number) ;; Integers, signs and surrounding whitespace. (print (parse-number:parse-number "42")) (print (parse-number:parse-number "-13")) (print (parse-number:parse-number " 3.14 ")) ;; Ratios are exact and normalized. (print (parse-number:parse-number "1/3")) (print (parse-number:parse-number "-4/8")) ;; Exponent markers produce floats. (print (parse-number:parse-number "1e3")) (print (parse-number:parse-number "2.5e2")) (print (parse-number:parse-number "5d0")) ;; Radix-prefixed literals, including the general #NNr form. (print (parse-number:parse-number "#xFF")) (print (parse-number:parse-number "#b101")) (print (parse-number:parse-number "#o777")) (print (parse-number:parse-number "#3r12")) ;; The entry points for reals only / positive reals only. (print (parse-number:parse-real-number "-42.5")) (print (parse-number:parse-positive-real-number "17")) --- # FILE: references/examples/asdf/split-sequence-demo.lisp ;;;; split-sequence via asdf:load-system ;;;; Loads the REAL split-sequence v2.0.1 (unmodified upstream sources) and ;;;; exercises split-sequence / split-sequence-if / split-sequence-if-not on ;;;; strings and lists, including the second return value (the resume index), ;;;; the :count/:from-end/:start/:end bounds and the :test/:test-not/:key ;;;; designators. Runs identically on all four backends; see README.md in ;;;; this directory for the run commands (the library directory is passed ;;;; with --system-path). ;;;; ;;;; Run (library vendored in this repository): ;;;; rontolisp examples/asdf/split-sequence-demo.lisp --system-path src/test/resources/split-sequence (asdf:load-system :split-sequence) ;; Strings: empty subsequences are kept unless :remove-empty-subseqs. (print (split-sequence:split-sequence #\, "a,b,,c")) (print (split-sequence:split-sequence #\, "a,b,,c" :remove-empty-subseqs t)) ;; Lists, and predicate variants. (print (split-sequence:split-sequence 3 '(1 2 3 4 5 3 6))) (print (split-sequence:split-sequence-if #'evenp '(1 2 3 4 5))) (print (split-sequence:split-sequence-if-not #'oddp '(1 2 3 4 5))) ;; The second return value is the index where processing stopped -- it ;; crosses the function boundary through the multiple-value channel. (multiple-value-bind (parts index) (split-sequence:split-sequence #\space "hello world lisp") (print parts) (print index)) ;; Bounds and counts. (print (split-sequence:split-sequence #\, "a,b,c,d" :count 2)) (print (split-sequence:split-sequence #\, "a,b,c,d" :count 2 :from-end t)) (print (split-sequence:split-sequence #\, "a,b,c,d" :start 2)) (print (split-sequence:split-sequence #\, "a,b,c,d" :end 3)) ;; Custom :test and :key designators. (print (split-sequence:split-sequence 2 '(1 2 3 2 4) :test #'eql)) (print (split-sequence:split-sequence #\A "aAbAc" :key #'char-upcase :test #'char=)) --- # FILE: references/examples/asdf/tiny-routes-demo.lisp ;; Loads the REAL tiny-routes (BSD 3-Clause, Johnny Ruiz) via asdf:load-system ;; and routes requests through it. Run with: ;; rontolisp examples/asdf/tiny-routes-demo.lisp --system-path src/test/resources/tiny-routes:src/test/resources/cl-ppcre ;; tiny-routes targets Clack, and this demo is deliberately TRANSPORT-FREE: it ;; calls the composed handler with hand-built request plists, so it runs on all ;; four backends -- WASM Preview 1 has no incoming TCP. To serve the same routes ;; over HTTP, hand the composed handler to clack:clackup (see ;; doc/en/guides/clack.md); that works on the interpreter, the JVM and the WASM ;; component. ;; The demo uses with-input-from-string, whose expansion is an unwind-protect, so ;; BOTH wasm run commands need -W exceptions=y. (asdf:load-system :tiny-routes) ;; An application uses the library from its own package; :tiny is tiny-routes' ;; own nickname. (defpackage :tiny-routes-demo (:use :cl :tiny-routes)) (in-package :tiny-routes-demo) ;; define-routes binds a handler that tries each route in turn and answers with ;; the first non-nil response. Each define-VERB is itself a handler: a lambda ;; wrapped in the method matcher and the path-template matcher, so its arguments ;; are the path template, the request lambda list and a body. (define-routes *app* (define-get "/hello" () (ok "hello world")) ;; A :name segment in the template binds a path parameter. (define-get "/users/:id" (req) (ok (format nil "user ~A" (path-parameter req :id)))) ;; wrap-query-parameters below parses the query string; its keys are interned ;; VERBATIM, so "q" is the |q| keyword, not :Q. (define-get "/search" (req) (ok (format nil "q=~A" (getf (request-get req :query-parameters) :|q|)))) ;; wrap-request-body below reads the request body stream into a string. (define-post "/echo" (req) (ok (format nil "echo:~A" (request-body req)))) (define-put "/put" () (created "/put" "made")) ;; A route with no template at all: match on anything you like. (define-route (req) (when (ppcre:scan "^/v[0-9]+/ping$" (path-info req)) (ok "pong"))) ;; "*" matches every path and :any every method, so this is the fallback. (define-any "*" () (not-found "nope"))) ;; pipe threads the handler through middleware left to right. (defparameter *handler* (pipe *app* (wrap-request-body) (wrap-query-parameters))) ;; The Clack request environment, as a server would hand it over. (defun env (method path &optional (query "")) (list :request-method method :request-uri path :path-info path :url-scheme "http" :query-string query)) (defun show (res) (format t "~A ~A ~A~%" (response-status res) (response-body res) (response-headers res))) (show (funcall *handler* (env :get "/hello"))) (show (funcall *handler* (env :get "/users/42"))) (show (funcall *handler* (env :get "/search" "q=lisp&n=2"))) (show (funcall *handler* (env :get "/v2/ping"))) (show (funcall *handler* (env :put "/put"))) (show (funcall *handler* (env :get "/zzz"))) ;; The method matcher declines a POST to a GET-only route, so it falls through. (show (funcall *handler* (env :post "/hello"))) ;; A request WITH a body: :raw-body is the stream, :content-length its size. (with-input-from-string (in "abc") (let ((request (append (env :post "/echo") (list :content-length 3 :raw-body in)))) (show (funcall *handler* request)))) ;; Response combinators, as middleware and directly. A route is an ordinary ;; value, so it can be named and wrapped on its own. (defparameter *ct-route* (define-get "/ct" () (ok "y"))) (defparameter *typed* (wrap-response-content-type *ct-route* "text/plain")) (show (funcall *typed* (env :get "/ct"))) (show (clone-response (ok "x") :status 202)) --- # FILE: references/examples/asdf/uax-15-demo.lisp ;; Loads the REAL uax-15 (MIT, Chris Bagley and Sabra Crolleton) via ;; asdf:load-system and runs the four Unicode normalization forms. Run with: ;; rontolisp examples/asdf/uax-15-demo.lisp --system-path src/test/resources/uax-15:src/test/resources/split-sequence:src/test/resources/cl-ppcre ;; It is the one demo here whose library has dependencies of its own, so the ;; system path carries three directories. Runs on all four backends: the tables ;; the library builds by parsing 2.7 MB of bundled Unicode text are DERIVED from ;; the same files at compile/load time, emitted as data, and built only when one ;; is first read -- so the load itself is milliseconds where the real library ;; takes minutes, and the results are identical. (asdf:load-system :uax-15) ;; The API is stringly typed and the results carry combining marks, so print ;; code-point lists rather than the raw normalized strings. (defun codes (s) (map 'list #'char-code s)) ;; NFC composes: A + U+030A (COMBINING RING ABOVE) -> U+00C5. (print (codes (uax-15:normalize (format nil "A~C" (code-char #x030A)) :nfc))) ;; NFD decomposes it back. (print (codes (uax-15:normalize (string (code-char #x00C5)) :nfd))) ;; NFKC compatibility-composes: U+2460 (circled 1) -> 1, U+00BD -> 1 / 2. (print (codes (uax-15:normalize (format nil "~C~C" (code-char #x2460) (code-char #x00BD)) :nfkc))) ;; NFKD of U+FB00 (LATIN SMALL LIGATURE FF) -> two 'f' characters. (print (codes (uax-15:normalize (string (code-char #xFB00)) :nfkd))) ;; U+212B (ANGSTROM SIGN) canonically decomposes, so NFC yields U+00C5. (print (codes (uax-15:normalize (string (code-char #x212B)) :nfc))) ;; The canonical combining class of U+0301 (COMBINING ACUTE ACCENT). (print (gethash #x0301 (uax-15:get-canonical-combining-class-map) 0)) ;; The NFC illegal-character list: its length and both endpoints. (let ((illegal (uax-15:get-illegal-char-list :nfc))) (print (list (length illegal) (first illegal) (car (last illegal))))) ;; unicode-letter-p over a Latin letter, a hiragana, a digit and a CJK ideograph. ;; rontolisp answers T for the letters; the upstream load answers NIL for every ;; character outside nine hardcoded ranges (its key computation reads #+utf-32, ;; a feature a file's own pushnew never gets to the reader). (print (mapcar (lambda (code) (uax-15:unicode-letter-p (code-char code))) (list #x41 #x3042 #x30 #x4E00))) --- # FILE: references/examples/browser/hiragana/README.md # 手書きひらがな認識 (rontolisp で CNN を学習 → WASM → ブラウザ canvas) 全 46 文字（五十音 あ〜ん、濁点・拗音は除く）の**畳み込みニューラルネット (CNN)** を rontolisp で書き、**実際の手書きかな (Kuzushiji-49) と複数フォントの合成字形**で **オフラインで学習**し、推論だけを WebAssembly にコンパイルして、ブラウザの `` に 書いた文字を認識させるデモです。 学習（バックプロパゲーション）はコストが高いので一度だけオフラインで行い、ブラウザでは **学習済みの重み (`weights.bin`) を読み込んで推論するだけ**にしています。同じ推論プログラムは インタプリタ・JVM・WASM Preview1・WASM コンポーネントのいずれでも動きます。 ネットワークは新しく書き起こしたものではなく、[`examples/deep-learning-from-scratch`](https://github.com/making/rontolisp/blob/develop/examples/deep-learning-from-scratch) の **ch07 SimpleConvNet をそのまま**このデモの入力サイズで使っています（`linalg:` の im2col 行列積・CLOS レイヤ・Adam・trainer をすべて再利用）。 ``` input 1 x 24 x 24 二値ビットマップ（ブラウザが canvas を縮小したもの） conv 16 filters, 5x5, pad 2 -> 16 x 24 x 24 pool 2x2 max -> 16 x 12 x 12 (= 2304) affine 2304 -> 64, relu affine 64 -> 46 -> softmax（46 クラス） ``` ## 精度 同じ held-out テスト集合（K49 テスト分割 4,600 枚 = **実際の手書き**）での比較です。 | モデル | 実手書き (K49 test) | 参考字形 46 字 | 全フォント変種 184 字 | | --- | --- | --- | --- | | 旧: 576-20-46 MLP（合成フォントのみで学習・重み焼き込み） | 650 / 4600 (14.1%) | 46 / 46 | — | | **新: CNN（K49 + 合成フォント）** | **3476 / 4600 (75.6%)** | **46 / 46** | **184 / 184** | 旧デモは「参考字形どおりに書けば当たるが、自由な手書きは外す」テンプレート記憶器でした。 実データを混ぜた CNN にしたことで、**参考字形の 46/46 を保ったまま**、実手書きが 14% → 76% になりました。 ## 仕組み ``` [オフライン] tools/k49/prepare-k49.py K49 (.npz) をダウンロードし、ブラウザと同じ 前処理で 24x24 二値化 -> data/k49-*.bin train.lisp dataset.lisp (K49 + 合成字形の水増し) と net.lisp (ch07 SimpleConvNet) を load し、 Adam で学習 -> weights.bin (RLW1 バイナリ) recognize.lisp 同じネットを wasm-export し infer.wasm に コンパイル [ブラウザ] infer.wasm を1回だけインスタンス化 -> _start が weights.bin を読み込む canvas 描画 -> JS が 24x24 に縮小・中心化・二値化して平坦化 -> 文字列 "(0.0 1.0 ...)" を recognize() に渡す（:s-expr ABI） -> "pred " と各クラスのスコアを受け取り、かなで表示 ``` 各ファイルは連結ではなく `(load ...)` で合成します。コンパイラではトップレベルのリテラル `(load ...)` は**コンパイル時インクルード**として展開され、相対パスは**その `load` を書いた ファイルからの相対**で解決されます（だからデモは deep-learning-from-scratch のレイヤ群を そのまま読み込めます）。 ### 重みを「焼き込まない」 旧デモは重みを Lisp のリテラルとして推論プログラムに焼き込んでいました。JVM バックエンドは スタックマップを持たないクラス版 50 を出力するため、**焼き込む浮動小数定数が概ね 1.3 万個を 超えるとクラスがロードできなくなり**、隠れ層 20・重み 1.25 万個という 「4 バックエンドすべてで動く上限」がモデルの上限そのものになっていました。 いまは重みを **`weights.bin`（RLW1 バイナリ）から起動時に読み込みます**。定数として焼き込まないので この上限は効かず、パラメータは 15 万個（604KB）に増えました。コンパイラ側の課題 （大量の定数を焼き込むプログラム一般）は未解決のままですが、**このデモはもう当たりません**。 - 書き出し: `net.lisp` の `save-rlw1`（`write-byte` でビッグエンディアン f32） - 読み込み: `deep-learning-from-scratch/dataset/rlw1.lisp` の `load-rlw1`（本の学習済み重みと同じ形式） - ブラウザ: `wasi-shim.js` に**読み込み専用の仮想ファイルシステム**を足し、fetch した `weights.bin` を WASI の `path_open`/`fd_read` に見せています（WASM 側は「ファイルを開いて 読む」だけで、ブラウザを意識しません）。 ### なぜ「1 回だけインスタンス化」なのか 起動時の重み読み込みは 15 万パラメータ分の `read-byte`（ブラウザで約 330ms）です。ストロークごとに モジュールを作り直すとこれを毎回払うことになるため、`recognize.lisp` はネットを `rontolisp:wasm-export` で**ホストから呼べる関数**として公開しています。ページはモジュールを 1 回だけインスタンス化して `_start`（= 重みロード）を走らせ、以後は `recognize()` を呼ぶだけ ——**1 認識あたり約 27ms**（Chrome 実測）です。 ## ビルド リポジトリルートで JAR を用意し、実データを一度だけ準備してから `gen.sh` を実行します。 ```bash ./mvnw clean package # target/rontolisp-...-exec.jar python3 examples/browser/hiragana/tools/k49/prepare-k49.py # K49 を DL + 前処理（初回のみ・約80MB） examples/browser/hiragana/gen.sh # 学習 + infer.wasm 生成 ``` `gen.sh` は学習を **JVM コンパイル + `--simd`** で走らせます（実測 約 4 分半 / 12 エポック / 39,537 サンプル。畳み込みは im2col で `linalg:matmul` になるので `--simd` がそのまま効きます）。 同じプログラムはインタプリタでも動きますが数時間かかります。 参考字形（`prototypes.lisp` / `glyphs.js` / `samples/`）はコミット済みの生成物です。 フォント・解像度・対象文字を変えたときだけ再生成します（macOS のフォントが必要）。 ```bash examples/browser/hiragana/regen-glyphs.sh ``` ## ブラウザで動かす `fetch` で `.wasm` と `weights.bin` を読むため `http://` で配信してください。 ```bash python3 -m http.server 8000 --directory examples/browser/hiragana # ブラウザで http://localhost:8000/ を開く ``` マスに字を書くと、書くそばから**リアルタイムで**予測クラスとスコアが更新されます （ストロークごとに自動認識。「認識」ボタンは手動での再実行用）。 **ブラウザ要件**: WebAssembly GC 対応（Chrome/Edge 119+, Firefox 120+, Safari 18.2+）。 ## ブラウザなしで試す（4 バックエンド） canvas はブラウザ専用ですが、認識ロジックはバックエンド非依存です。`samples/` に各クラスの 参考字形を平坦化したビットマップ（`(0.0 1.0 ...)` 1 行）を置いてあります。`infer.lisp` は それを標準入力から読み、`weights.bin` を**カレントディレクトリから**開くので、このディレクトリで 実行してください（WASM は `--dir .` のプリオープンが必要）。 ```bash JAR=../../../target/rontolisp-0.1.0-SNAPSHOT-exec.jar cd examples/browser/hiragana # 1. インタプリタ java -jar $JAR infer.lisp < samples/u.txt # -> pred 2 u # 2. JVM（クラス名は出力ファイル名になるのでパスを含めない） java -jar $JAR infer.lisp -o Infer.class && java -cp .:$JAR Infer < samples/u.txt # 3. WASM Preview1 java -jar $JAR infer.lisp -o /tmp/infer-p1.wasm && \ wasmtime run -W gc --dir . /tmp/infer-p1.wasm < samples/u.txt # 4. WASM コンポーネント (WASI 0.3) java -jar $JAR infer.lisp -o /tmp/infer-c.wasm --component && \ wasmtime run -W gc=y --dir . /tmp/infer-c.wasm < samples/u.txt ``` 4 バックエンドとも 46 個の参考字形をすべて正しく分類します（WASM は浮動小数の表示桁数だけが 異なります）。 ## ファイル | ファイル | 役割 | | ------------------------------ | ----------------------------------------------------------- | | `net.lisp` | ネットワーク定義（ch07 SimpleConvNet の再利用）＋ RLW1 の書き出し | | `dataset.lisp` | 学習データ（K49 バイナリの読み込み・合成字形の水増し） | | `train.lisp` | オフライン学習（Adam）→ `weights.bin` を出力 | | `infer.lisp` | CLI 推論（stdin のビットマップ → 予測。4 バックエンド共通） | | `recognize.lisp` | ブラウザ用推論（`recognize` を `wasm-export`）→ `infer.wasm` | | `gen.sh` | 学習 → `infer.wasm` 生成のパイプライン | | `tools/k49/prepare-k49.py` | K49 のダウンロードと前処理（[README](tools/k49/README.md)） | | `glyphgen/GlyphGen.java` | 字形レンダラ（実フォント → 24x24 二値化。`prototypes.lisp` / `glyphs.js` / `samples/` を生成） | | `regen-glyphs.sh` | `GlyphGen.java` を実行する薄いラッパ | | `prototypes.lisp` | 各クラスの 24x24 字形テンプレート（生成物・合成データの種） | | `glyphs.js` | ブラウザ表示用の参考字形 `GLYPHS`/`KANA`/`ORDER`（生成物） | | `index.html` | canvas で描いて認識するブラウザページ | | `wasi-shim.js` | WASI Preview1 シム（`../wasm-browser/` のコピー ＋ 仮想 FS） | | `samples/*.txt` | ブラウザなし確認用の平坦化ビットマップ（全 46 クラス・生成物） | | `weights.bin` | 学習済みの重み（生成物・コミット対象。ブラウザが fetch する） | | `infer.wasm` | 推論モジュール（生成物・コミット対象） | ## 限界 - **K49 は崩し字（古典籍のくずし字）**で、現代の手書きとは字形分布がずれます。合成フォント字形を 混ぜて現代の字形を押さえていますが、実手書き 76% という数字は、このずれと 24x24 二値・小さな CNN という制約の合計です。ネットを大きくすれば伸びますが、学習時間と `weights.bin` が増えます。 - 字形が似たかな（は/ほ/ま、ね/れ/わ、る/ろ、さ/き など）では依然として混同が起きます。 - 濁点・半濁点・拗音（が・ぱ・きゃ 等）は対象外です。 - ブラウザのシムの仮想 FS は**読み込み専用**です（データファイルは読めますが書き込みはできません）。 --- # FILE: references/examples/browser/hiragana/dataset.lisp ;;;; dataset.lisp -- the training data: real handwriting (Kuzushiji-49) mixed ;;;; with augmented copies of the synthetic multi-font glyphs. ;;;; ;;;; Why both. The synthetic glyphs (prototypes.lisp, rendered from four real ;;;; Japanese fonts by glyphgen/GlyphGen.java) are MODERN letterforms -- exactly ;;;; what somebody drawing in the browser writes -- but they are type, not ;;;; handwriting: augmentation only ever wobbles them rigidly, so a net trained ;;;; on them alone never learns that a stroke may be hooked, disconnected or ;;;; differently proportioned -- the ceiling the previous version of this demo ;;;; hit, and the reason it read a freehand あ as さ. K49 ;;;; is real handwritten kana with all of that variation, but it is cursive ;;;; (kuzushiji) and its shapes drift from the modern ones. Trained together, ;;;; the real data supplies the handwriting variation and the synthetic set pins ;;;; the modern letterform. ;;;; ;;;; Both halves arrive in the SAME representation the browser produces: a 24x24 ;;;; binary bitmap, ink-bbox-cropped, centred with a 1px margin, binarized at ;;;; 0.35 (index.html's toBitmap; GlyphGen.render and tools/k49/prepare-k49.py ;;;; reproduce it pixel for pixel). ;;;; ;;;; Everything below hands the trainer packed linalg arrays: the images as a ;;;; rank-4 (N 1 24 24) batch (the shape the convolution wants) and the labels as ;;;; a rank-1 vector of class indices. (load "net.lisp") (load "prototypes.lisp") ;;; --------------------------------------------------------------------------- ;;; A seeded RNG for the augmentation. ;;; ;;; linalg:seed / linalg:rand exist, but they hand back arrays; the augmenters ;;; want one number at a time, so this is the same i31-safe LCG the ml examples ;;; use. Seeded, so a rerun of the trainer sees the same augmented set. ;;; --------------------------------------------------------------------------- (defparameter *rng* 12345) (defun rnd () ; uniform in [0, 1) (setq *rng* (mod (+ (* *rng* 1103515) 12345) 2097152)) (/ *rng* 2097152.0)) (defun rnd-int (n) (floor (* (rnd) n))) (defun rnd-range (lo hi) (+ lo (* (rnd) (- hi lo)))) ;;; --------------------------------------------------------------------------- ;;; Image ops. An image is a packed length-576 double vector of 0.0 / 1.0 ;;; (row-major); every op returns a fresh one. ;;; --------------------------------------------------------------------------- (defun blank-image () (linalg:zeros *pixels*)) (defun ink-at (v x y) ; 1.0 if (x, y) is in range and ink (if (and (>= x 0) (< x *grid*) (>= y 0) (< y *grid*)) (aref v (+ (* y *grid*) x)) 0.0)) (defun glyph->image (rows) ; one prototypes.lisp glyph -> an image (let ((v (blank-image)) (i 0)) (dolist (e (glyph->list rows)) (setf (aref v i) e) (setq i (+ i 1))) v)) (defun shift-image (v dx dy) (let ((dst (blank-image))) (dotimes (y *grid*) (dotimes (x *grid*) (setf (aref dst (+ (* y *grid*) x)) (ink-at v (- x dx) (- y dy))))) dst)) ;; Thicken every stroke by one pixel (4-neighbourhood dilation). A drawn stroke ;; is heavier than the thin font outline after the browser downsamples it. (defun dilate (v) (let ((dst (blank-image))) (dotimes (y *grid*) (dotimes (x *grid*) (when (> (+ (ink-at v x y) (ink-at v (- x 1) y) (ink-at v (+ x 1) y) (ink-at v x (- y 1)) (ink-at v x (+ y 1))) 0.5) (setf (aref dst (+ (* y *grid*) x)) 1.0)))) dst)) ;; Affine warp about the grid centre: rotate by ANG radians, then shear ;; horizontally by SHEAR, nearest-neighbour, inverse-mapped. M = Shear(k)*Rot(a) ;; has det 1, so M^-1 = [[c, s-k*c], [-s, c+k*s]] with c = cos a, s = sin a. ;; Handwriting is rarely upright. (defun warp-image (v ang shear) (let* ((dst (blank-image)) (c (cos ang)) (s (sin ang)) (m00 c) (m01 (- s (* shear c))) (m10 (- 0.0 s)) (m11 (+ c (* shear s))) (cc (floor *grid* 2))) (dotimes (y *grid*) (dotimes (x *grid*) (let* ((rx (- x cc)) (ry (- y cc)) (fx (+ (* m00 rx) (* m01 ry))) (fy (+ (* m10 rx) (* m11 ry))) (ix (+ (round fx) cc)) (iy (+ (round fy) cc))) (setf (aref dst (+ (* y *grid*) x)) (ink-at v ix iy))))) dst)) ;; Flip K random pixels: the browser's downsample leaves ragged edges. (defun add-noise (v k) (let ((dst (linalg:add v 0))) ; a copy, same width (dotimes (n k) (let ((i (rnd-int *pixels*))) (setf (aref dst i) (if (> (aref dst i) 0.5) 0.0 1.0)))) dst)) ;;; --------------------------------------------------------------------------- ;;; The synthetic half: every (class, font) glyph, augmented. ;;; --------------------------------------------------------------------------- (defparameter *variants-per-glyph* 8) ; random variants on top of the fixed ones (defun augment-glyph (base) ;; BASE plus a fixed spine (thickened / extra-thick / shifted) and ;; *variants-per-glyph* random rotate+shear+shift+thicken+noise copies. (let ((out (list base (dilate base) (dilate (dilate base)) (shift-image base -1 0) (shift-image base 1 0) (shift-image base 0 -1) (shift-image base 0 1)))) (dotimes (i *variants-per-glyph*) (let* ((img (warp-image base (rnd-range -0.25 0.25) (rnd-range -0.25 0.25))) (img (shift-image img (- (rnd-int 3) 1) (- (rnd-int 3) 1))) (img (if (> (rnd) 0.35) (dilate img) img)) (img (if (> (rnd) 0.5) (add-noise img (rnd-int 6)) img))) (setq out (cons img out)))) out)) (defun synthetic-samples () ;; ((image . class) ...) over every class x font x augmentation. (let ((acc nil) (class 0)) (dolist (variants *glyphs*) (dolist (glyph variants) (dolist (img (augment-glyph (glyph->image glyph))) (setq acc (cons (cons img class) acc)))) (setq class (+ class 1))) acc)) ;;; --------------------------------------------------------------------------- ;;; The real half: the HKB1 files written by tools/k49/prepare-k49.py. ;;; ;;; magic "HKB1" | count u32 be | grid u8 | COUNT x (label u8, grid*grid bytes) ;;; ;;; Read strictly front-to-back with read-byte (there is no seek), so LIMIT just ;;; stops early -- the file is pre-shuffled, so a prefix is a fair sample. ;;; --------------------------------------------------------------------------- (defun k49-load (path limit) ;; -> (images labels): ((n 1 24 24) packed batch, rank-1 label vector). (with-open-file (s path :element-type '(unsigned-byte 8)) (unless (and (= (read-byte s) 72) (= (read-byte s) 75) (= (read-byte s) 66) (= (read-byte s) 49)) (error "not an HKB1 file (run tools/k49/prepare-k49.py first)")) (let* ((count (%read-be32 s)) (grid (read-byte s)) (pixels (* grid grid)) (n (min limit count)) (x (make-array (list n 1 grid grid) :element-type 'double-float :initial-element 0.0)) (lab (make-array n :element-type 'double-float :initial-element 0.0))) (dotimes (i n) (setf (aref lab i) (read-byte s)) (let ((base (* i pixels))) (dotimes (k pixels) (setf (row-major-aref x (+ base k)) (* 1.0 (read-byte s)))))) (list x lab)))) ;;; --------------------------------------------------------------------------- ;;; Assembling one batch tensor out of both halves. ;;; --------------------------------------------------------------------------- (defun samples->batch (samples) ;; ((image . class) ...) -> ((n 1 24 24) batch, label vector). (let* ((n (length samples)) (x (make-array (list n 1 *grid* *grid*) :element-type 'double-float :initial-element 0.0)) (lab (make-array n :element-type 'double-float :initial-element 0.0)) (i 0)) (dolist (sample samples) (let ((img (car sample)) (base (* i *pixels*))) (dotimes (k *pixels*) (setf (row-major-aref x (+ base k)) (aref img k))) (setf (aref lab i) (cdr sample)) (setq i (+ i 1)))) (list x lab))) (defun concat-batches (a b) ;; Two (images labels) pairs -> one. Rank-4 batches concatenate along axis 0, ;; which linalg has no operator for (take-rows selects, it does not join), so ;; this is a straight row-major copy of both slabs. (let* ((xa (first a)) (la (second a)) (xb (first b)) (lb (second b)) (na (car (linalg:shape xa))) (nb (car (linalg:shape xb))) (n (+ na nb)) (x (make-array (list n 1 *grid* *grid*) :element-type 'double-float :initial-element 0.0)) (lab (make-array n :element-type 'double-float :initial-element 0.0))) (dotimes (k (linalg:size xa)) (setf (row-major-aref x k) (row-major-aref xa k))) (dotimes (k (linalg:size xb)) (setf (row-major-aref x (+ (linalg:size xa) k)) (row-major-aref xb k))) (dotimes (i na) (setf (aref lab i) (aref la i))) (dotimes (i nb) (setf (aref lab (+ na i)) (aref lb i))) (list x lab))) (defun shuffle-batch (b) ;; A seeded permutation of the rows (linalg:permutation), so the synthetic and ;; real halves interleave instead of arriving in two blocks. (let ((p (linalg:permutation (car (linalg:shape (first b)))))) (list (linalg:take-rows (first b) p) (linalg:take-rows (second b) p)))) --- # FILE: references/examples/browser/hiragana/gen.sh #!/usr/bin/env bash # Train the hiragana recognizer offline, then compile the inference half to WASM # for the browser demo. # # examples/browser/hiragana/gen.sh # # Pipeline (the .lisp files compose via (load ...), not concatenation -- a # top-level literal load is a compile-time include on the compilers, and the # interpreter loads at runtime; relative load paths resolve next to each file): # # 1. train.lisp -> weights.bin loads dataset.lisp (K49 + synthetic glyphs) # and net.lisp (the ch07 SimpleConvNet), trains # with Adam and writes the RLW1 weight file. # Run compiled to the JVM with --simd; the # interpreter would take hours. # 2. recognize.lisp -> infer.wasm the same net, exported as a host-callable # recognize() for the page. It READS # weights.bin at startup -- nothing is baked in, # so the model is not capped by the JVM's # baked-constant ceiling. # # The real-handwriting half of the training set is Kuzushiji-49, which # tools/k49/prepare-k49.py downloads and converts once (see tools/k49/README.md). set -euo pipefail here="$(cd "$(dirname "$0")" && pwd)" repo_root="$(cd "$here/../../.." && pwd)" jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar" if [[ ! -f "$jar" ]]; then echo "JAR not found: $jar" >&2 echo "Build it first from the repo root: ./mvnw clean package" >&2 exit 1 fi if [[ ! -f "$here/data/k49-train.bin" ]]; then echo "training data not found: $here/data/k49-train.bin" >&2 echo "Prepare it once (downloads Kuzushiji-49, ~80 MB; needs numpy + Pillow):" >&2 echo " python3 $here/tools/k49/prepare-k49.py" >&2 exit 1 fi echo "[1/2] training (Adam over K49 + the synthetic glyphs; a few minutes)" # The compiled class is named after the -o file, so it must be path-free: compile # and run from inside this directory (which also makes the relative data paths # resolve here). --simd routes the convolution's matrix products through the # Vector API, which the JVM run then needs the incubator module for. ( cd "$here" \ && java -jar "$jar" train.lisp -o Train.class --simd \ && java --add-modules jdk.incubator.vector -Xmx8g -cp ".:$jar" Train \ && rm -f Train.class ) echo "[2/2] compiling recognize.lisp -> infer.wasm" # --optimize tree-shakes the runtime and the WASI import surface down to what the # inference half reaches; the convnet itself dominates the module, so the win is # small here -- but the page then links against exactly the imports the shim has. java -jar "$jar" "$here/recognize.lisp" -o "$here/infer.wasm" --optimize echo "done." echo " weights.bin $(wc -c < "$here/weights.bin") bytes" echo " infer.wasm $(wc -c < "$here/infer.wasm") bytes" echo "Serve this dir over http and open index.html, e.g.:" echo " python3 -m http.server 8000 --directory \"$here\"" --- # FILE: references/examples/browser/hiragana/glyphgen/GlyphGen.java // GlyphGen.java -- offline glyph generator for the hiragana demo. // // Renders the 46 gojuon hiragana from a single real font and downsamples each // to a GRID x GRID binary bitmap using the SAME crop/center/binarize pipeline // the browser applies to a drawn stroke (index.html's toBitmap), so a template // equals what the browser produces for a perfectly drawn glyph. // // It writes three artifacts (single source -> no drift): // /prototypes.lisp the trainer's reference glyphs (regenerated) // /glyphs.js GLYPHS/KANA/ORDER for index.html // /samples/.txt each template flattened to 576 floats // // Run it only when changing the font / resolution / class set: // java examples/browser/hiragana/glyphgen/GlyphGen.java examples/browser/hiragana // (or via examples/browser/hiragana/regen-glyphs.sh). JDK 25 single-file launch; this // file lives outside the Maven source root, so it is not built or formatted by // the main project. import java.awt.Color; import java.awt.Font; import java.awt.Graphics2D; import java.awt.RenderingHints; import java.awt.image.BufferedImage; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; public class GlyphGen { static final int GRID = 24; // output bitmap is GRID x GRID // Each kana is rendered from SEVERAL fonts so the network sees stroke-shape // variation (hooks/sweeps, brush vs round), not one exemplar -- this is what // lets it tolerate real handwriting. FONTS[0] is the "display" font: the // reference shown on the page (glyphs.js) and the parity samples come from it, // so "draw to match the reference" still works; the rest only widen training. static final String[] FONTS = { "Hiragino Maru Gothic ProN", // round gothic (the displayed reference) "Klee", // textbook / pen style -- proper hane (hooks) like handwriting "YuGothic", // gothic with a hooked left radical "Hiragino Mincho ProN" // brush / serif shapes }; static final int HIRES = 320; // hi-res render canvas (px) static final int FONT_PX = 220; // glyph point size on that canvas static final double INK_BBOX = 0.3; // ink threshold for the bounding box static final double BINARIZE = 0.35; // cell on/off threshold (matches browser) // The 46 gojuon, in output-unit order. {kana, romaji}. romaji is the ASCII // label (kept multibyte-free across the WASM boundary) and the Lisp var name. static final String[][] KANA = { { "あ", "a" }, { "い", "i" }, { "う", "u" }, { "え", "e" }, { "お", "o" }, { "か", "ka" }, { "き", "ki" }, { "く", "ku" }, { "け", "ke" }, { "こ", "ko" }, { "さ", "sa" }, { "し", "shi" }, { "す", "su" }, { "せ", "se" }, { "そ", "so" }, { "た", "ta" }, { "ち", "chi" }, { "つ", "tsu" }, { "て", "te" }, { "と", "to" }, { "な", "na" }, { "に", "ni" }, { "ぬ", "nu" }, { "ね", "ne" }, { "の", "no" }, { "は", "ha" }, { "ひ", "hi" }, { "ふ", "fu" }, { "へ", "he" }, { "ほ", "ho" }, { "ま", "ma" }, { "み", "mi" }, { "む", "mu" }, { "め", "me" }, { "も", "mo" }, { "や", "ya" }, { "ゆ", "yu" }, { "よ", "yo" }, { "ら", "ra" }, { "り", "ri" }, { "る", "ru" }, { "れ", "re" }, { "ろ", "ro" }, { "わ", "wa" }, { "を", "wo" }, { "ん", "n" } }; public static void main(String[] args) throws IOException { Path out = Path.of(args.length > 0 ? args[0] : "."); Path samples = out.resolve("samples"); Files.createDirectories(samples); // bitmaps[class][font] = one GRID x GRID bitmap (GRID rows of GRID chars). String[][][] bitmaps = new String[KANA.length][FONTS.length][]; for (int c = 0; c < KANA.length; c++) { for (int f = 0; f < FONTS.length; f++) { bitmaps[c][f] = render(KANA[c][0], FONTS[f]); } } writePrototypes(out.resolve("prototypes.lisp"), bitmaps); // The page reference and the parity samples use the display font only. writeGlyphsJs(out.resolve("glyphs.js"), bitmaps); for (int c = 0; c < KANA.length; c++) { Files.writeString(samples.resolve(KANA[c][1] + ".txt"), flattenSample(bitmaps[c][0])); } System.out.println("generated " + KANA.length + " kana x " + FONTS.length + " fonts at " + GRID + "x" + GRID + " into " + out); } // Render one kana from one font and downsample to a GRID x GRID '#'/'.' // bitmap, mirroring index.html's toBitmap: crop the ink bbox, scale it to fit // a (GRID-2) box centred with a 1px margin, accumulate ink per cell, binarize. static String[] render(String kana, String font) { BufferedImage img = new BufferedImage(HIRES, HIRES, BufferedImage.TYPE_INT_RGB); Graphics2D g = img.createGraphics(); g.setColor(Color.WHITE); g.fillRect(0, 0, HIRES, HIRES); g.setColor(Color.BLACK); g.setRenderingHint(RenderingHints.KEY_ANTIALIASING, RenderingHints.VALUE_ANTIALIAS_ON); g.setRenderingHint(RenderingHints.KEY_TEXT_ANTIALIASING, RenderingHints.VALUE_TEXT_ANTIALIAS_ON); g.setFont(new Font(font, Font.PLAIN, FONT_PX)); var fm = g.getFontMetrics(); int tw = fm.stringWidth(kana); int x = (HIRES - tw) / 2; int y = (HIRES - fm.getHeight()) / 2 + fm.getAscent(); g.drawString(kana, x, y); g.dispose(); // ink(px) = 1 - luminance/255 (black stroke -> 1). double[] ink = new double[HIRES * HIRES]; int minx = HIRES, miny = HIRES, maxx = -1, maxy = -1; for (int yy = 0; yy < HIRES; yy++) { for (int xx = 0; xx < HIRES; xx++) { int rgb = img.getRGB(xx, yy); int lum = (rgb >> 16) & 0xff; // grayscale -> any channel double v = 1.0 - lum / 255.0; ink[yy * HIRES + xx] = v; if (v > INK_BBOX) { if (xx < minx) minx = xx; if (xx > maxx) maxx = xx; if (yy < miny) miny = yy; if (yy > maxy) maxy = yy; } } } double[] acc = new double[GRID * GRID]; int[] cnt = new int[GRID * GRID]; if (maxx >= 0) { int bw = maxx - minx + 1, bh = maxy - miny + 1; double fit = GRID - 2, scale = fit / Math.max(bw, bh); double ox = (GRID - bw * scale) / 2, oy = (GRID - bh * scale) / 2; for (int yy = miny; yy <= maxy; yy++) { for (int xx = minx; xx <= maxx; xx++) { int gx = (int) Math.floor((xx - minx) * scale + ox); int gy = (int) Math.floor((yy - miny) * scale + oy); if (gx < 0 || gx >= GRID || gy < 0 || gy >= GRID) continue; acc[gy * GRID + gx] += ink[yy * HIRES + xx]; cnt[gy * GRID + gx] += 1; } } } String[] rows = new String[GRID]; for (int r = 0; r < GRID; r++) { StringBuilder sb = new StringBuilder(GRID); for (int cc = 0; cc < GRID; cc++) { int i = r * GRID + cc; boolean on = cnt[i] > 0 && acc[i] / cnt[i] > BINARIZE; sb.append(on ? '#' : '.'); } rows[r] = sb.toString(); } return rows; } static void writePrototypes(Path file, String[][][] bitmaps) throws IOException { StringBuilder b = new StringBuilder(); b.append(";;;; prototypes.lisp -- GENERATED by glyphgen/GlyphGen.java -- DO NOT EDIT BY HAND.\n"); b.append(";;;;\n"); b.append(";;;; The reference glyphs (the \"training alphabet\"). Each class has one glyph\n"); b.append(";;;; per font (").append(FONTS.length).append(" fonts) so the trainer sees stroke-shape variation.\n"); b.append(";;;; Each glyph is ").append(GRID).append(" rows of ").append(GRID) .append(" characters, '#' = ink, '.' = blank, rendered and\n"); b.append(";;;; binarized with the same crop/center/binarize the browser applies to a drawn\n"); b.append(";;;; stroke. Fonts: "); for (int f = 0; f < FONTS.length; f++) b.append(f == 0 ? "" : ", ").append(FONTS[f]); b.append(".\n"); b.append(";;;; Used only by the OFFLINE trainer (interpreter/JVM); index.html's glyphs.js\n"); b.append(";;;; (display font = the first one) is generated from the same run. Class order\n"); b.append(";;;; defines the output-unit order and must match *romaji* / *labels*.\n\n"); b.append("(defparameter *romaji* (list"); for (String[] k : KANA) b.append(" \"").append(k[1]).append("\""); b.append("))\n\n"); // One defun per (class, font) returning its glyph rows. These are defuns, // not defparameters, on purpose: the JVM trainer compiles each defun body // into its own method, whereas top-level defparameter literals all land in // one `main` method -- 184 glyphs of literals there blow the 64 KB method // cap. (Same reason the weights are chunked into gN functions.) for (int c = 0; c < KANA.length; c++) { b.append(";; ").append(KANA[c][0]).append("\n"); for (int f = 0; f < FONTS.length; f++) { b.append("(defun glyph-").append(KANA[c][1]).append("-f").append(f).append(" () (list\n"); for (int r = 0; r < GRID; r++) { b.append(" \"").append(bitmaps[c][f][r]).append("\""); b.append(r == GRID - 1 ? "))\n" : "\n"); } } b.append("\n"); } // *glyphs* groups the font variants per class: a list (one entry per class) // of lists (the per-font glyphs). build-dataset trains on every variant. b.append("(defparameter *glyphs* (list\n"); for (int c = 0; c < KANA.length; c++) { b.append(" (list"); for (int f = 0; f < FONTS.length; f++) b.append(" (glyph-").append(KANA[c][1]).append("-f").append(f).append(")"); b.append(")").append(c == KANA.length - 1 ? "))\n\n" : "\n"); } b.append(";; Convert one glyph (list of equal-length rows) into a flat list of 0.0 / 1.0,\n"); b.append(";; row-major. Size-agnostic: it reads the grid width off the row strings.\n"); b.append("(defun glyph->list (rows)\n"); b.append(" (let ((acc nil))\n"); b.append(" (dolist (row rows)\n"); b.append(" (dotimes (j (length row))\n"); b.append(" (setq acc (cons (if (char= (char row j) #\\#) 1.0 0.0) acc))))\n"); b.append(" (reverse acc)))\n"); Files.writeString(file, b.toString()); } static void writeGlyphsJs(Path file, String[][][] bitmaps) throws IOException { StringBuilder b = new StringBuilder(); b.append("// glyphs.js -- GENERATED by glyphgen/GlyphGen.java -- DO NOT EDIT BY HAND.\n"); b.append("// The reference glyphs shown on the page (display font = FONTS[0]); the network\n"); b.append("// is additionally trained on other fonts (prototypes.lisp), same run.\n\n"); b.append("export const GRID = ").append(GRID).append(";\n\n"); b.append("export const GLYPHS = {\n"); for (int c = 0; c < KANA.length; c++) { b.append(" ").append(KANA[c][1]).append(": ["); for (int r = 0; r < GRID; r++) { b.append("\"").append(bitmaps[c][0][r]).append("\""); b.append(r == GRID - 1 ? "" : ", "); } b.append("],\n"); } b.append("};\n\n"); b.append("export const KANA = {"); for (int c = 0; c < KANA.length; c++) { b.append(" ").append(KANA[c][1]).append(": \"").append(KANA[c][0]).append("\","); } b.append(" };\n\n"); b.append("export const ORDER = ["); for (int c = 0; c < KANA.length; c++) { b.append("\"").append(KANA[c][1]).append("\""); b.append(c == KANA.length - 1 ? "" : ", "); } b.append("];\n"); Files.writeString(file, b.toString()); } // "(0.0 1.0 ...)\n" -- GRID*GRID floats, row-major, matching infer's (read). static String flattenSample(String[] rows) { StringBuilder b = new StringBuilder("("); for (int r = 0; r < GRID; r++) { for (int cc = 0; cc < GRID; cc++) { if (r != 0 || cc != 0) b.append(' '); b.append(rows[r].charAt(cc) == '#' ? "1.0" : "0.0"); } } b.append(")\n"); return b.toString(); } } --- # FILE: references/examples/browser/hiragana/glyphs.js // glyphs.js -- GENERATED by glyphgen/GlyphGen.java -- DO NOT EDIT BY HAND. // The reference glyphs shown on the page (display font = FONTS[0]); the network // is additionally trained on other fonts (prototypes.lisp), same run. export const GRID = 24; export const GLYPHS = { a: ["........................", "........##..............", "........##..............", "........##.......###....", "...#################....", "..#################.....", "........##..............", "........##....##........", "........##....##........", "........##########......", ".......#############....", ".....#####...##..####...", "....###.##...##....##...", "...###..##..###....###..", "...##...##..##......##..", "..###...##.###......##..", "..##....#####.......##..", ".###.....###.......###..", ".###.....###.......##...", "..###..#####......###...", "..##########...#####....", "....####....#######.....", "............####........", "........................"], i: ["........................", "........................", ".##.....................", ".##.............##......", ".##.............###.....", ".##.............###.....", ".##..............###....", ".##..............###....", ".##...............###...", ".##...............###...", ".##................###..", ".###...............###..", ".###...............###..", ".###......#.........##..", ".###.....###........###.", "..##.....###........###.", "..##.....##.........###.", "..###...###.........###.", "..###...###..........##.", "...#######..............", "....######..............", ".....###................", "........................", "........................"], u: ["........................", "......#####.............", "......#############.....", "........###########.....", "................##......", "........................", "........................", ".........########.......", ".....##############.....", "..##################....", "..#####..........####...", "..................###...", "...................##...", "...................##...", "..................###...", "..................###...", ".................###....", "................####....", "...............####.....", "............######......", "........########........", ".......#######..........", ".......####.............", "........................"], e: ["........................", ".....####...............", ".....############.......", ".......###########......", ".............####.......", "........................", "........................", "...##############.......", "..###############.......", "...######....###........", "............###.........", "...........###..........", "..........###...........", ".........###............", "........######..........", ".......########.........", ".....#####...##.........", "....####.....##.........", "...####......##.........", "..####.......###........", ".####........##########.", ".###..........#########.", ".##............######...", "........................"], o: ["........................", ".......##...............", ".......##...............", ".......##.......##......", ".......##.......###.....", ".......##.####...####...", ".##############...####..", ".############......####.", ".......##...........###.", ".......##............#..", ".......##...............", ".......##...#####.......", ".......############.....", ".....#######....####....", "....#####.........###...", "...###.##..........##...", "..###..##..........###..", ".###...##..........###..", ".##....##..........###..", ".###...##...##....###...", ".####.###...####.####...", "..#######...########....", "....####......####......", "........................"], ka: ["........................", "........##..............", "........##..............", ".......###..............", ".......###.......#......", ".......###......###.....", ".......###.......##.....", ".#############...###....", ".##############...###...", "......###....###..###...", "......##.....###...###..", "......##......##...###..", ".....###......##....##..", ".....##.......##....###.", "....###.......##.....##.", "....##.......###.....##.", "....##.......###........", "...###.......###........", "...##........##.........", "..###..#....###.........", ".###...#######..........", ".###...#######..........", "..#.......##............", "........................"], ki: ["........................", "..........#.............", "..........##............", "..........##....####....", "........############....", "..###############.......", "..########.###..........", "............##......#...", "............##########..", "...##################...", "..##############........", "...#####......###.......", "...............##.......", ".........####...##......", "......#############.....", "....#####....#######....", "....##...........###....", "...###..................", "...###..................", "...###..................", "....####.......####.....", ".....##############.....", ".......##########.......", "........................"], ku: ["........................", "................##......", "...............###......", "..............###.......", "............####........", "...........####.........", "..........####..........", "........####............", ".......####.............", "......####..............", ".....###................", "....###.................", ".....###................", "......####..............", ".......####.............", "........#####...........", "..........####..........", "...........####.........", "............####........", "..............####......", "...............####.....", "................###.....", ".................##.....", "........................"], ke: ["........................", "...##...........##......", "...##..........###......", "...##...........##......", "..###...........##......", "..###...........##......", "..###...........######..", "..##...###############..", "..##...#############....", ".###............##......", ".###............##......", ".###............##......", ".###............##......", ".###............##......", ".###............##......", ".###...........###......", "..##...........##.......", "..##...........##.......", "..###.........###.......", "..###........###........", "...##......####.........", "...##.....####..........", "..........###...........", "........................"], ko: ["........................", "...#################....", "...##################...", "....################....", ".............#####......", "...........#####........", "...........####.........", "...........##...........", "........................", "........................", "........................", "....##..................", "...###..................", "..####..................", "..###...................", ".###....................", ".###....................", ".###....................", ".###....................", "..####..............##..", "..#####################.", "....###################.", "......##############....", "........................"], sa: ["........................", "..........##............", "..........###...........", "..........###...........", "...........##......###..", "...........###########..", "..###################...", "..##############........", "...###.......###........", "..............##........", "..............###.......", "...............###......", ".........#####..###.....", "......##############....", "....######..#########...", "....###..........####...", "...###..................", "...###..................", "...###..................", "....###.................", "....#####......#####....", ".....###############....", ".......###########......", "........................"], shi: ["........................", "...###..................", "...###..................", "...###..................", "...###..................", "...###..................", "...###..................", "...###..................", "...##...................", "...##...................", "..###...................", "..###...................", "..###..............##...", "..###.............###...", "..###.............###...", "..###.............##....", "...##............###....", "...###...........##.....", "...###..........###.....", "...####.......####......", "....#############.......", ".....###########........", ".......#######..........", "........................"], su: ["........................", ".............##.........", ".............##.........", ".............##.........", "....###################.", ".######################.", ".###############........", ".............##.........", ".........##..##.........", ".......########.........", "......####.####.........", ".....###.....##.........", ".....###.....###........", ".....###.....###........", ".....###....####........", "......##########........", ".......######.##........", ".............###........", "............###.........", "...........####.........", "........######..........", "......######............", ".......###..............", "........................"], se: ["........................", "........................", "......#.........##......", "......##........##......", "......##........##......", "......##........##......", "......##........##......", "......##...############.", ".######################.", ".###########....##......", "......##........##......", "......##........##......", "......##........##......", "......##...#...###......", "......##..#######.......", "......##...######.......", "......##................", "......##................", "......###...............", "......######...######...", ".......##############...", ".........##########.....", "........................", "........................"], so: ["........................", "..............####......", ".....#############......", ".....#############......", "..............###.......", ".............###........", "...........####.........", ".........####...........", ".......#####............", "......####......#######.", "...####################.", ".###################....", ".#######....####........", "...........###..........", "..........##............", ".........###............", ".........##.............", ".........##.............", ".........###............", ".........#####..........", "..........#########.....", "............#######.....", "...............####.....", "........................"], ta: ["........................", "........##..............", ".......###..............", ".......###..............", ".......##...............", "..#############.........", ".##############.........", ".########...............", "......##......#######...", "......##...###########..", ".....###...##########...", ".....##.................", ".....##.................", "....###.................", "....###.................", "....##....###...........", "...###....##............", "...##....###............", "..###....##.............", "..###....###............", "..##.....####.......###.", ".###......#############.", "..#.........##########..", "........................"], chi: ["........................", "..........##............", ".........###............", ".........##.............", ".........##.......###...", "..###################...", "..##################....", ".......###..............", ".......###..............", ".......##...............", "......###...............", "......##.....####.......", ".....###.###########....", ".....################...", "....######.........###..", "....####...........###..", "....##..............##..", "....................##..", "...................###..", "..................###...", ".....#####.....######...", ".....##############.....", ".......##########.......", "........................"], tsu: ["........................", "........................", "........................", ".........#########......", "....################....", ".####################...", ".######...........####..", ".##................###..", "....................###.", "....................###.", "....................###.", "....................###.", "....................###.", "....................###.", "...................###..", "..................####..", ".................####...", "..............######....", ".......############.....", "......##########........", ".......######...........", "........................", "........................", "........................"], te: ["........................", "........................", "..........#############.", ".######################.", ".####################...", "..#..........####.......", "............###.........", "...........###..........", "..........###...........", ".........###............", ".........###............", ".........##.............", "........###.............", "........###.............", "........###.............", ".........##.............", ".........###............", ".........####...........", "..........####..........", "...........#########....", "............########....", "..............######....", "........................", "........................"], to: ["........................", ".......##...............", ".......##...............", ".......##...............", ".......###..............", ".......###..............", "........##........##....", "........##.....######...", "........###.########....", "........#########.......", "........######..........", "......#####.............", ".....####...............", "....###.................", "...###..................", "...###..................", "..###...................", "..###...................", "..###...................", "...###..................", "....########.########...", ".....#################..", ".......#############....", "........................"], na: ["........................", ".......##...............", ".......###..............", ".......##...............", ".......##.......#.......", "..###########..####.....", ".############...#####...", ".########.........####..", "......##............###.", ".....###.......##.......", ".....##........##.......", "....###........##.......", "....##.........###......", "...###.........###......", "...###.........###......", "..###....#########......", "..###...###########.....", ".###...###......#####...", ".###...##.......######..", ".......###.....###.####.", "........###...####...##.", "........#########.......", "..........######........", "........................"], ni: ["........................", "...##...................", "...##...................", "...##...................", "..###...#############...", "..###...#############...", "..###.....####..........", "..##....................", "..##....................", ".###....................", ".###....................", ".###....................", ".###.....##.............", ".###....###.............", ".###....##..............", ".###...###..............", ".###...###..............", ".###...###..............", ".###....###.........##..", "..##....###############.", "..##......############..", "..###...................", "..##....................", "........................"], nu: ["........................", ".............#..........", "............###.........", "....##......###.........", "....##......##..........", "....##......##..........", "....##..#########.......", "....###############.....", "....#####..##....###....", "....###....##.....###...", "...####....##......##...", "..#####...##.......##...", "..##..##..##.......###..", ".###..##.###.......###..", ".##...#####........###..", ".##....####..#########..", ".##....###..#########...", ".##....###.##.....###...", ".###..#######.....####..", "..######.#.###...######.", "...####.....#######..##.", ".............#####......", "........................", "........................"], ne: ["........................", "......#.................", "......##................", ".....###................", ".....###................", ".....###......###.......", ".#######...########.....", ".#######..####..####....", ".....##..###......##....", ".....#####........###...", ".....####.........###...", ".....###...........##...", "....###............##...", "...####............##...", "...####......###...##...", "..#####.....#########...", ".###.##....###..#####...", ".##..##...##......###...", ".....##...###....######.", ".....##....###..###..##.", ".....##....#######....#.", ".....##......###........", "......#.................", "........................"], no: ["........................", "........................", "........#########.......", "......#############.....", ".....####..##...####....", "....###....##.....###...", "...###.....##......###..", "..###......##......###..", "..##......###.......###.", ".###......###.......###.", ".###......###.......###.", ".##.......##.........##.", ".##.......##.........##.", ".##......###........###.", ".##......###........###.", ".###....###.........##..", ".###....###........###..", "..###..###........###...", "..########......#####...", "...######....######.....", "............######......", "............####........", "........................", "........................"], ha: ["........................", "...##..........##.......", "..###..........##.......", "..###..........##.......", "..###..........##.......", "..##...........###.###..", "..##...###############..", ".###...##############...", ".###...........##.......", ".###...........###......", ".###...........###......", ".###...........###......", ".###...........###......", ".###...........###......", ".###.......#######......", ".###.....#########......", ".###....###...######....", ".###...###......#####...", ".###...###......##.###..", ".###...###.....###..###.", "..##....##########...#..", "..##.....########.......", "..##.......####.........", "........................"], hi: ["........................", "........................", "....######....##........", ".##########...###.......", ".#####.###.....##.......", "......##.......###......", ".....##........####.....", "....###........####.....", "...###.........#####....", "...##..........##.###...", "..###..........###.###..", "..##...........###..###.", "..##...........###...##.", "..##...........###......", "..##...........###......", "..##...........##.......", "..##...........##.......", "..###.........###.......", "...###.......###........", "...#####...####.........", "....##########..........", "......#######...........", "........................", "........................"], fu: ["........................", "......#.................", ".....#####..............", ".....###########........", "........##########......", ".............####.......", "...........####.........", "..........###...........", ".........###............", ".........##.............", ".........###............", "....##....##......##....", "...###....###.....##....", "...###.....###....###...", "...##.......###....##...", "...##........###...###..", "..###.........##....##..", "..##..........##....###.", ".###..##......##....###.", ".###..###....###.....##.", ".##...#########......#..", "........######..........", "...........#............", "........................"], he: ["........................", "........................", "........................", "........................", "........###.............", ".......#####............", "......######............", ".....###..###...........", ".....###...###..........", "....###.....###.........", "...###.......###........", "...###.......####.......", "..###.........####......", ".###...........####.....", ".##.............####....", ".................####...", "..................####..", "...................####.", "....................###.", ".....................##.", "........................", "........................", "........................", "........................"], ho: ["........................", "...##...................", "..###...##############..", "..###...##############..", "..###..........##.......", "..##...........##.......", "..##...........##.......", ".###...........###......", ".###....##############..", ".###...###############..", ".###...........###......", ".###...........###......", ".###...........###......", ".###............##......", ".###........##..##......", ".###.....#########......", ".###....############....", ".###...###......#####...", ".###...###......######..", ".###...###.....###..###.", "..##....####..####...##.", "..##.....########.......", "..##......#####.........", "........................"], ma: ["........................", "...........##...........", "...........##...........", "...........##...........", "..####################..", ".####################...", "...........##...........", "...........##...........", "...........##...........", "...##################...", "..###################...", "...###########..........", "...........##...........", "...........##...........", "........##.###..........", "....##########..........", "...##############.......", "...##.......#######.....", "..###......###..####....", "..###......###...####...", "...###....###......###..", "...##########.......##..", ".....#######............", "........................"], mi: ["........................", "......#####.............", "...#########............", "...##########...........", "..........###...........", "..........##......#.....", "..........##.....##.....", ".........###.....##.....", ".........##......##.....", ".......#####.....##.....", "....###########..##.....", "...####.###########.....", "..###...##....#####.....", ".###....##......#####...", ".##....##.......######..", ".##....##.......##.####.", ".##...##.......###...##.", ".###.###.......##.......", ".######.......###.......", "..####......####........", "...........####.........", "...........##...........", "........................", "........................"], mu: ["........................", ".......##...............", "......###...............", "......###.......##......", "..############..###.....", ".#############...###....", "..#######.........###...", "......###..........###..", "......###...........###.", "...######............##.", "..#######...............", ".###...##...............", ".##....##...............", ".##....##........###....", ".##....##.........##....", ".###..###.........##....", "..######..........###...", "...####...........###...", ".....##...........###...", ".....##...........##....", ".....###.......#####....", ".....##############.....", ".......##########.......", "........................"], me: ["........................", "..............##........", "....###.......##........", "....###.......##........", ".....##......###........", ".....##..#########......", ".....##############.....", ".....#####...##.#####...", "....####.....##...####..", "...#####....###....###..", "..###.##....##......###.", "..##..###..###......###.", ".###...##..###......###.", ".##....##..##........##.", ".##....######.......###.", ".##.....####........###.", ".##.....###.........###.", ".##.....####.......###..", ".###..######......####..", "..#######..#.....####...", "...#####......######....", "............######......", "............####........", "........................"], mo: ["........................", ".........##.............", ".........##.............", "........###.............", "........###.............", "...##############.......", "...##############.......", ".......###..............", ".......###..............", ".......###..............", ".......##...............", ".#########...###..##....", ".################.###...", ".....###########...###..", ".......##..........###..", "......###...........##..", "......###...........###.", "......###...........##..", ".......##..........###..", ".......###........###...", ".......#####....#####...", "........############....", "..........########......", "........................"], ya: ["........................", ".............##.........", ".....#.......##.........", "....###......##.........", ".....##......##.........", ".....###...#########....", ".....################...", ".....#######.......###..", "...######...........###.", ".########...........###.", ".###...##...........###.", ".......##...........###.", ".......###..........###.", "........##..###...####..", "........##..#########...", "........###..#######....", "........###.............", ".........##.............", ".........##.............", ".........###............", ".........###............", ".........###............", "..........##............", "........................"], yu: ["........................", "............##..........", "............##..........", "...##.......###.........", "..###.......#####.......", "..###....##########.....", "..##....#######.####....", "..##..####...##...###...", "..##..##.....##....###..", "..##.##......##....###..", "..####.......##.....##..", ".#####.......##.....###.", ".####........##.....##..", ".####..#.....##.....##..", ".####..##....##....###..", "..##...###..###...###...", "..##....###.##...###....", "..##.....##########.....", "..##......########......", "..........###...........", ".........###............", "........###.............", "........##..............", "........................"], yo: ["........................", "...........##...........", "...........##...........", "...........##...........", "...........##...........", "...........##...........", "...........###########..", "...........###########..", "...........#####........", "...........##...........", "...........###..........", "...........###..........", "...........###..........", "............##..........", ".....#########..........", "...#############........", "...###.....######.......", "..###.......#######.....", "..##........##..####....", "..###.......##....####..", "..####....####.....###..", "...##########.......#...", ".....#######............", "........................"], ra: ["........................", "......####..............", "......###########.......", ".......###########......", "............######......", "........................", "....##..................", "....##..................", "...###..................", "...###..................", "...##.....#######.......", "...##...###########.....", "...##.######....####....", "..#######.........###...", "..#####............###..", "..####.............###..", "...#...............###..", "...................###..", "..................###...", ".................####...", "....######..########....", "....##############......", "......##########........", "........................"], ri: ["........................", "......##................", ".....###................", ".....##....#####........", ".....##..#########......", "....###.###....###......", "....###.##......###.....", "....#####........###....", "....#####........###....", "....####.........###....", "....####.........###....", "....###..........###....", "....###..........###....", "....###..........###....", "....###..........###....", "....###..........##.....", ".....#..........###.....", "................##......", "..............####......", ".............####.......", ".........#######........", ".......#######..........", "........####............", "........................"], ru: ["........................", ".....#############......", "....###############.....", ".....#######...###......", "..............###.......", ".............###........", "...........####.........", "..........####..........", ".........###............", ".......####.#####.......", "......##############....", "....#########...#####...", "...######..........###..", ".#####..............##..", ".###................###.", ".......#####........###.", "......########......###.", ".....###...####.....###.", ".....###....###....###..", ".....###.....##...###...", "......###....#######....", ".......############.....", "........#########.......", "........................"], re: ["........................", "......#.................", "......##................", "......##................", ".....###................", ".....###....######......", ".#######...#######......", ".#######..###...###.....", ".....##.###......##.....", ".....#####.......##.....", ".....####.......###.....", ".....###........###.....", "....###.........##......", "...####.........##......", "...####.........##......", "..#####.........##......", ".###.##........###...##.", ".##..##........###..###.", ".....##.........##.###..", ".....##.........######..", ".....##..........####...", ".....##.................", "......#.................", "........................"], ro: ["........................", "....###############.....", "....###############.....", ".....#######...####.....", "..............####......", ".............###........", "............###.........", "..........####..........", ".........####...........", "........###.............", ".......#############....", ".....################...", "....######........####..", "..######............###.", ".#####..............###.", ".###................###.", "....................###.", "....................###.", "...................###..", "..................####..", ".....######..########...", ".....###############....", ".......##########.......", "........................"], wa: ["........................", "......##................", "......##................", "......##................", "......##................", ".....###................", ".########...#######.....", "..######..###########...", "......##.####.....####..", "......#####........###..", "......####..........###.", ".....###............###.", "....####.............##.", "....####............###.", "...#####............###.", "..###.##............###.", ".###..##...........###..", ".##...##..........####..", "......##........#####...", "......##....########....", "......##....######......", "......##................", "......##................", "........................"], wo: ["........................", "..........##............", ".........###............", ".........##.......#.....", "..##################....", "..#################.....", "....######..............", ".......##...............", "......###...............", ".....#########..........", "....###########...#####.", "....####.....#########..", "...###......#######.....", "..###.....######........", ".###.....######.........", "..#....####..##.........", ".......###...##.........", "......###....##.........", "......##......#.........", "......###...............", "......####........###...", ".......###############..", ".........############...", "........................"], n: ["........................", "........##..............", "........##..............", ".......###..............", ".......###..............", ".......##...............", "......###...............", "......##................", "......##................", ".....###................", ".....##...##............", "....###.######..........", "....##########..........", "....####....###.........", "...####.....###......##.", "...###......###.....###.", "...###......###.....###.", "..###.......###.....###.", "..###.......###....###..", ".###.........##....###..", ".###.........########...", ".##..........#######....", ".##............####.....", "........................"], }; export const KANA = { a: "あ", i: "い", u: "う", e: "え", o: "お", ka: "か", ki: "き", ku: "く", ke: "け", ko: "こ", sa: "さ", shi: "し", su: "す", se: "せ", so: "そ", ta: "た", chi: "ち", tsu: "つ", te: "て", to: "と", na: "な", ni: "に", nu: "ぬ", ne: "ね", no: "の", ha: "は", hi: "ひ", fu: "ふ", he: "へ", ho: "ほ", ma: "ま", mi: "み", mu: "む", me: "め", mo: "も", ya: "や", yu: "ゆ", yo: "よ", ra: "ら", ri: "り", ru: "る", re: "れ", ro: "ろ", wa: "わ", wo: "を", n: "ん", }; export const ORDER = ["a", "i", "u", "e", "o", "ka", "ki", "ku", "ke", "ko", "sa", "shi", "su", "se", "so", "ta", "chi", "tsu", "te", "to", "na", "ni", "nu", "ne", "no", "ha", "hi", "fu", "he", "ho", "ma", "mi", "mu", "me", "mo", "ya", "yu", "yo", "ra", "ri", "ru", "re", "ro", "wa", "wo", "n"]; --- # FILE: references/examples/browser/hiragana/index.html rontolisp ひらがな認識 (WASM + canvas)

手書きひらがな認識 — Lisp で学習し、WASM で推論

全 46 文字（五十音 あ〜ん）の畳み込みニューラルネット (CNN) を rontolisp で書き、 実際の手書きかな (Kuzushiji-49) と複数フォントの合成字形で、オフラインで学習 しました。推論だけを WebAssembly (infer.wasm) にコンパイルし、学習済みの 重みは weights.bin から読み込みます。下のマスに字を書くと、ブラウザが 描画を 24×24 に縮小して WASM の recognize 関数に渡し、 書くそばからリアルタイムで予測します。

予測:
―


    
認識できる 46 文字

    


    
  



---

# FILE: references/examples/browser/hiragana/infer.lisp

;;;; infer.lisp -- the command-line inference program (interpreter, JVM, WASM
;;;; Preview 1 and the WASI 0.3 component all run this file unchanged).
;;;;
;;;; It reads ONE flattened 24x24 bitmap from stdin as a Lisp list of 576 values
;;;; in [0, 1] -- e.g. "(0.0 0.0 1.0 ... 0.0)", which is what samples/*.txt hold
;;;; -- runs the trained convnet forward and prints the predicted class plus
;;;; every class score.
;;;;
;;;; The weights come from weights.bin at startup, so the WASM backends need a
;;;; preopened directory (wasmtime --dir .) and the weights path is relative to
;;;; the directory you run in.  The (load ...) path, by contrast, resolves
;;;; relative to THIS file, so the program compiles from anywhere.
;;;;
;;;; The browser front-end does NOT use this program -- it cannot pipe stdin into
;;;; a module it wants to keep alive across strokes.  recognize.lisp is the same
;;;; net exported as a host-callable function instead.

(load "net.lisp")

(defparameter *net* (load-hiragana-net "weights.bin"))

(print-prediction (classify *net* (linalg:from-list (read))))


---

# FILE: references/examples/browser/hiragana/net.lisp

;;;; net.lisp -- the network shared by the trainer and the two inference
;;;; programs (infer.lisp for the CLI, recognize.lisp for the browser).
;;;;
;;;; The demo is a CONVOLUTIONAL net now, and it is not a new one: it is the
;;;; SimpleConvNet of examples/deep-learning-from-scratch/ch07 (the book's
;;;; Conv -> Relu -> Pool -> Affine -> Relu -> Affine over im2col), re-used
;;;; verbatim at this demo's geometry.  Everything it needs -- the CLOS layers,
;;;; Adam, the trainer loop, the RLW1 weight reader -- already lives there, so
;;;; this file only fixes the shape of the problem:
;;;;
;;;;   input   1 x 24 x 24 binary bitmap (the browser's downsampled canvas)
;;;;   conv    16 filters, 5x5, pad 2   -> 16 x 24 x 24
;;;;   pool    2x2 max                  -> 16 x 12 x 12  (= 2304)
;;;;   affine  2304 -> 64, relu
;;;;   affine  64 -> 46                 -> softmax over the 46 gojuon
;;;;
;;;; The weights are NOT baked into the program any more: they are read at
;;;; startup from weights.bin, an RLW1 binary file (the same format the book's
;;;; pretrained params use).  That is what lifts the old 12.5k-parameter ceiling
;;;; -- the JVM backend's class-version-50 verifier caps the number of float
;;;; constants a class may BAKE (~12.8k), not the number it may read.  The
;;;; browser gets the file through the WASI shim's virtual filesystem
;;;; (wasi-shim.js), wasmtime through --dir .

(load "../../deep-learning-from-scratch/ch07/simple-convnet.lisp")

(defparameter *grid* 24)    ; bitmap edge; the input is *grid* x *grid*
(defparameter *pixels* 576) ; *grid* * *grid*
(defparameter *nclasses* 46)

;; Class index -> romaji label, the output-unit order (must match *romaji* in
;; prototypes.lisp and the K49 remap in tools/k49/prepare-k49.py).  Romaji, not
;; kana, so nothing multibyte crosses the WASM string boundary; the browser maps
;; the label back to a kana for display.
(defparameter *labels*
  (list "a" "i" "u" "e" "o" "ka" "ki" "ku" "ke" "ko" "sa" "shi" "su" "se" "so"
        "ta" "chi" "tsu" "te" "to" "na" "ni" "nu" "ne" "no" "ha" "hi" "fu" "he"
        "ho" "ma" "mi" "mu" "me" "mo" "ya" "yu" "yo" "ra" "ri" "ru" "re" "ro"
        "wa" "wo" "n"))

(defun make-hiragana-net ()
  ;; The ch07 SimpleConvNet at this demo's geometry.  weight-init-std 0.01 is
  ;; the book's; Adam copes with it (see train.lisp).
  (make-simple-convnet :input-dim (list 1 *grid* *grid*)
                       :filter-num 16
                       :filter-size 5
                       :filter-pad 2
                       :filter-stride 1
                       :hidden-size 64
                       :output-size *nclasses*
                       :weight-init-std 0.01))

(defun load-hiragana-net (path)
  ;; A net with the trained weights.bin read back in (RLW1, W1 b1 W2 b2 W3 b3 --
  ;; the ch07 net-load-params contract).
  (net-load-params (make-hiragana-net) path nil))

;; --- RLW1 writer (the trainer's half of the format) ---------------------------
;;
;; dataset/rlw1.lisp READS the format; nothing wrote it in Lisp before (the
;; book's params were exported from Python).  Now the trainer produces one, so
;; the writer lives here: "RLW1", u8 array count, then per array u8 ndim, u32
;; dims (big-endian), and the elements as big-endian IEEE-754 f32, row-major.
;; Only the offline trainer runs this (interpreter / JVM), so it needs no WASM
;; care -- but it is plain arithmetic and would run there too.

(defun %write-be32 (v s)
  (write-byte (mod (floor v 16777216) 256) s)
  (write-byte (mod (floor v 65536) 256) s)
  (write-byte (mod (floor v 256) 256) s)
  (write-byte (mod v 256) s))

(defun %f32-bits (x)
  ;; The IEEE-754 single-precision encoding of X, as the four bytes (b0 b1 b2 b3)
  ;; most-significant first.  Built without bit operations on the raw word: the
  ;; sign, the exponent and the 23-bit mantissa are computed in arithmetic and
  ;; packed, so every intermediate stays small (the i31-safe style of
  ;; dataset/rlw1.lisp's reader, which this inverts).
  (let* ((sign (if (< x 0.0) 128 0)) (a (abs x)))
    (if (= a 0.0)
        (list sign 0 0 0)
        (let ((e 0))
          ;; Normalize a into [1, 2) tracking the unbiased exponent.
          (while (>= a 2.0)
            (setq a (/ a 2.0))
            (setq e (+ e 1)))
          (while (< a 1.0)
            (setq a (* a 2.0))
            (setq e (- e 1)))
          (let ((m (round (* (- a 1.0) 8388608.0))))
            ;; Rounding the mantissa up to 2^23 carries into the exponent.
            (when (>= m 8388608)
              (setq m 0)
              (setq e (+ e 1)))
            (cond ((> e 127) ; overflow -> +/- infinity
                   (list (+ sign 127) 128 0 0))
                  ((< e -126) ; underflow -> signed zero (no subnormals)
                   (list sign 0 0 0))
                  (t (let ((be (+ e 127)))
                       (list (+ sign (floor be 2))
                             (+ (* (mod be 2) 128) (floor m 65536))
                             (mod (floor m 256) 256) (mod m 256))))))))))

(defun %write-f32 (x s) (dolist (b (%f32-bits x)) (write-byte b s)))

(defun save-rlw1 (path arrays)
  ;; Write ARRAYS (a list of packed arrays, any rank) to PATH as one RLW1 file.
  (with-open-file (s path :direction :output :element-type '(unsigned-byte 8))
    (write-byte 82 s)
    (write-byte 76 s)
    (write-byte 87 s)
    (write-byte 49 s)
    (write-byte (length arrays) s)
    (dolist (a arrays)
      (let ((dims (array-dimensions a)))
        (write-byte (length dims) s)
        (dolist (d dims) (%write-be32 d s))
        (dotimes (i (array-total-size a))
          (%write-f32 (row-major-aref a i) s))))))

(defun save-hiragana-net (net path)
  ;; The six parameter arrays in the ch07 key order (*scn-keys*), which is the
  ;; order net-load-params reads them back in.
  (let ((params (scn-params net)))
    (save-rlw1 path
               (list (gethash "W1" params) (gethash "b1" params)
                     (gethash "W2" params) (gethash "b2" params)
                     (gethash "W3" params) (gethash "b3" params)))))

;; --- prediction over ONE bitmap ------------------------------------------------

(defun classify (net flat)
  ;; FLAT is a length-576 packed vector of 0.0/1.0.  Returns the softmax score
  ;; vector over the 46 classes (the net wants a rank-4 (N C H W) batch).
  (linalg:flatten
   (softmax (predict net (linalg:reshape flat (list 1 1 *grid* *grid*))))))

(defun print-prediction (scores)
  ;; The machine-readable lines both front-ends parse: the winner, then every
  ;; class score.
  (let ((pred (linalg:argmax scores)))
    (format t "pred ~a ~a~%" pred (nth pred *labels*))
    (dotimes (i *nclasses*)
      (format t "score ~a ~a~%" (nth i *labels*) (aref scores i)))))


---

# FILE: references/examples/browser/hiragana/prototypes.lisp

;;;; prototypes.lisp -- GENERATED by glyphgen/GlyphGen.java -- DO NOT EDIT BY HAND.
;;;;
;;;; The reference glyphs (the "training alphabet").  Each class has one glyph
;;;; per font (4 fonts) so the trainer sees stroke-shape variation.
;;;; Each glyph is 24 rows of 24 characters, '#' = ink, '.' = blank, rendered and
;;;; binarized with the same crop/center/binarize the browser applies to a drawn
;;;; stroke.  Fonts: Hiragino Maru Gothic ProN, Klee, YuGothic, Hiragino Mincho ProN.
;;;; Used only by the OFFLINE trainer (interpreter/JVM); index.html's glyphs.js
;;;; (display font = the first one) is generated from the same run.  Class order
;;;; defines the output-unit order and must match *romaji* / *labels*.

(defparameter *romaji*
  (list "a" "i" "u" "e" "o" "ka" "ki" "ku" "ke" "ko" "sa" "shi" "su" "se" "so"
        "ta" "chi" "tsu" "te" "to" "na" "ni" "nu" "ne" "no" "ha" "hi" "fu" "he"
        "ho" "ma" "mi" "mu" "me" "mo" "ya" "yu" "yo" "ra" "ri" "ru" "re" "ro"
        "wa" "wo" "n"))

;; あ
(defun glyph-a-f0 ()
  (list "........................" "........##.............."
        "........##.............." "........##.......###...."
        "...#################...." "..#################....."
        "........##.............." "........##....##........"
        "........##....##........" "........##########......"
        ".......#############...." ".....#####...##..####..."
        "....###.##...##....##..." "...###..##..###....###.."
        "...##...##..##......##.." "..###...##.###......##.."
        "..##....#####.......##.." ".###.....###.......###.."
        ".###.....###.......##..." "..###..#####......###..."
        "..##########...#####...." "....####....#######....."
        "............####........" "........................"))
(defun glyph-a-f1 ()
  (list "........................" "..........#............."
        "..........#............." "..........#............."
        ".........##..####......." "......#########........."
        "....#######............." ".........#....##........"
        ".........#....##........" ".........#########......"
        "........###..##..###...." ".......###...#.....#...."
        "......####..##.....##..." ".....#..##.##......##..."
        "....##..####.......##..." "...##....##........##..."
        "...##...##.........#...." "...##..###........##...."
        "...#####.##......##....." "....##..........##......"
        "..............###......." "............###........."
        "...........##..........." "........................"))
(defun glyph-a-f2 ()
  (list "........................" ".........##............."
        ".........##............." "........###............."
        "........###.######......" "...##############......."
        "....######.............." "........##.....#........"
        "........##.....##......." "........##..#####......."
        "........###########....." "......####....#...###..."
        ".....#####...##....##..." "....##..##..##......##.."
        "...##...##..##......##.." "...##...##.##.......##.."
        "..##....####........##.." "..##....###.........##.."
        ".###...####........##..." "..#########.......###..."
        "..#####.........####...." ".............#####......"
        "..............##........" "........................"))
(defun glyph-a-f3 ()
  (list "........................" "........###............."
        ".........###............" ".........##....##......."
        ".........##...###......." "....##...#######........"
        ".....#######............" ".........#.............."
        "........##...#.........." "........##...##........."
        "........##########......" ".......###...#...###...."
        "......####..##.....##..." "....##..##..#......##..."
        "...##...##.##......###.." "...##...####.......###.."
        "..##.....##........###.." "..##....###........##..."
        "...#..#####.......###..." "...#####.##......###...."
        "....##....#.....###....." "..............###......."
        "............###........." "........................"))

;; い
(defun glyph-i-f0 ()
  (list "........................" "........................"
        ".##....................." ".##.............##......"
        ".##.............###....." ".##.............###....."
        ".##..............###...." ".##..............###...."
        ".##...............###..." ".##...............###..."
        ".##................###.." ".###...............###.."
        ".###...............###.." ".###......#.........##.."
        ".###.....###........###." "..##.....###........###."
        "..##.....##.........###." "..###...###.........###."
        "..###...###..........##." "...#######.............."
        "....######.............." ".....###................"
        "........................" "........................"))
(defun glyph-i-f1 ()
  (list "........................" "........................"
        "........................" ".##....................."
        ".##....................." ".##..............#......"
        ".##..............##....." ".##...............##...."
        "..#...............##...." "..#................##..."
        "..##...............###.." "..##................##.."
        "..##................###." "..##.................##."
        "..##....#............##." "...##...#..............."
        "...##..##..............." "....##.#................"
        "....####................" ".....###................"
        "......##................" "........................"
        "........................" "........................"))
(defun glyph-i-f2 ()
  (list "........................" "........................"
        ".#......................" ".##....................."
        ".##....................." ".##....................."
        ".##.............##......" ".##..............##....."
        ".##...............##...." ".##...............###..."
        ".##................##..." ".##................###.."
        ".###................##.." "..##................###."
        "..##................###." "..##.....#...........##."
        "..##.....##............." "...##...###............."
        "...###.###.............." "....#####..............."
        ".....####..............." "......#................."
        "........................" "........................"))
(defun glyph-i-f3 ()
  (list "........................" "........................"
        "........................" ".#......................"
        "..#....................." "..##...................."
        "..###............#......" "..###.............##...."
        "..##...............##..." "..##...............##..."
        "..##................##.." "..##................##.."
        "..##.....#..........###." "..##.....#...........##."
        "...#....#...........###." "...##..##........######."
        "...##..##...........##.." "....####................"
        "....####................" ".....###................"
        "......##................" "........................"
        "........................" "........................"))

;; う
(defun glyph-u-f0 ()
  (list "........................" "......#####............."
        "......#############....." "........###########....."
        "................##......" "........................"
        "........................" ".........########......."
        ".....##############....." "..##################...."
        "..#####..........####..." "..................###..."
        "...................##..." "...................##..."
        "..................###..." "..................###..."
        ".................###...." "................####...."
        "...............####....." "............######......"
        "........########........" ".......#######.........."
        ".......####............." "........................"))
(defun glyph-u-f1 ()
  (list "........................" ".........###............"
        "..........#####........." ".............###........"
        "........................" "........................"
        "............###........." ".........########......."
        "......#####.....##......" "......###.......##......"
        ".................#......" ".................##....."
        ".................##....." ".................##....."
        "................##......" "................##......"
        "................#......." "...............##......."
        "..............##........" ".............##........."
        "...........###.........." "..........##............"
        "........##.............." "........................"))
(defun glyph-u-f2 ()
  (list "........................" ".........###............"
        ".........#######........" "...........######......."
        "...............##......." "........................"
        "........................" "...........#####........"
        "........#########......." ".....######....###......"
        "......##........###....." "................###....."
        ".................##....." ".................##....."
        "................###....." "................###....."
        "................##......" "...............###......"
        "..............###......." ".............###........"
        "............###........." "..........####.........."
        "...........##..........." "........................"))
(defun glyph-u-f3 ()
  (list "........................" "..........#............."
        "..........#####........." "............####........"
        "...........#####........" "..........#............."
        "........................" "............###........."
        "......#...###..##......." "......#####....##......."
        ".......###......##......" "........#.......##......"
        "................##......" "................##......"
        "................##......" "...............###......"
        "...............##......." "...............##......."
        "..............##........" "..............##........"
        ".............##........." "............##.........."
        "...........#............" "........................"))

;; え
(defun glyph-e-f0 ()
  (list "........................" ".....####..............."
        ".....############......." ".......###########......"
        ".............####......." "........................"
        "........................" "...##############......."
        "..###############......." "...######....###........"
        "............###........." "...........###.........."
        "..........###..........." ".........###............"
        "........######.........." ".......########........."
        ".....#####...##........." "....####.....##........."
        "...####......##........." "..####.......###........"
        ".####........##########." ".###..........#########."
        ".##............######..." "........................"))
(defun glyph-e-f1 ()
  (list "........................" "........###............."
        ".........#####.........." "............###........."
        "..............#........." "........................"
        "........................" "...........#####........"
        ".......########........." "....######..##.........."
        "...........##..........." "...........##..........."
        "..........##............" ".........##............."
        "........##.............." ".......##..............."
        "......######............" ".....####..#............"
        ".....##....##..........." "....##.....##..........."
        "...##.......#..........." "..###.......##########.."
        "..##.........#########.." "........................"))
(defun glyph-e-f2 ()
  (list "........................" "........##.............."
        "........######.........." "..........######........"
        ".............##........." "........................"
        "........................" "..............##........"
        "........########........" "...#############........"
        "....####....##.........." "...........###.........."
        "..........###..........." ".........###............"
        "........###............." ".......######..........."
        "......#######..........." ".....###....##.........."
        "....###.....##.........." "...###......##.........."
        "..###........##........." ".###.........##########."
        "..............#########." "........................"))
(defun glyph-e-f3 ()
  (list "........................" ".........#.............."
        ".........###............" "...........####........."
        "...........#####........" "........####............"
        "........................" "........................"
        "...........####........." "....#...####.###........"
        "....#####...###........." ".....##.....##.........."
        "...........##..........." "..........##............"
        ".........##............." "........##.............."
        ".......######..........." "......###...##.........."
        ".....##.....##.........." "....##.......#.........."
        "..###........##........." "..###........#########.."
        "..##...........#######.." "........................"))

;; お
(defun glyph-o-f0 ()
  (list "........................" ".......##..............."
        ".......##..............." ".......##.......##......"
        ".......##.......###....." ".......##.####...####..."
        ".##############...####.." ".############......####."
        ".......##...........###." ".......##............#.."
        ".......##..............." ".......##...#####......."
        ".......############....." ".....#######....####...."
        "....#####.........###..." "...###.##..........##..."
        "..###..##..........###.." ".###...##..........###.."
        ".##....##..........###.." ".###...##...##....###..."
        ".####.###...####.####..." "..#######...########...."
        "....####......####......" "........................"))
(defun glyph-o-f1 ()
  (list "........................" "......##................"
        ".......#................" ".......#................"
        ".......#........#......." ".......#........###....."
        ".......#...###....###..." "......######.......###.."
        ".########............#.." "...##..#................"
        ".......#................" ".......#................"
        ".......#................" ".......#....#######....."
        ".......#.###########...." ".......####........##..."
        "......###..........##..." "....####...........##..."
        "...##..#...........##..." "..##...#..........###..."
        "..##...#...###...###...." "...#####....#######....."
        ".....###................" "........................"))
(defun glyph-o-f2 ()
  (list "........................" "........#..............."
        "........##.............." "........##.............."
        "........##.............." "........##...#...###...."
        ".......#######....####.." "..##########........###."
        "...#######...........#.." "........#..............."
        "........#..............." "........#.....####......"
        "........#..#########...." "........#####.....###..."
        "......####.........##..." ".....#####.........##..."
        "...####.##.........##..." ".####...##.........##..."
        ".#####..##........###..." ".....#####...#######...."
        "......####....####......" ".......###.............."
        "........#..............." "........................"))
(defun glyph-o-f3 ()
  (list "........................" "......###..............."
        ".......###.............." ".......###.............."
        ".......##........###...." ".......##..........###.."
        ".......##..###......###." ".#.....######......####."
        "..#########.......##...." ".......##..............."
        ".......##..............." ".......##..............."
        ".......##.....####......" ".......##.##########...."
        ".......####........##..." ".....####...........##.."
        "...###.##...........##.." "..###..##...#.......##.."
        "..##...##...#.......##.." "..##...##....#.....###.."
        "...######....########..." ".....###.......#####...."
        "......##................" "........................"))

;; か
(defun glyph-ka-f0 ()
  (list "........................" "........##.............."
        "........##.............." ".......###.............."
        ".......###.......#......" ".......###......###....."
        ".......###.......##....." ".#############...###...."
        ".##############...###..." "......###....###..###..."
        "......##.....###...###.." "......##......##...###.."
        ".....###......##....##.." ".....##.......##....###."
        "....###.......##.....##." "....##.......###.....##."
        "....##.......###........" "...###.......###........"
        "...##........##........." "..###..#....###........."
        ".###...#######.........." ".###...#######.........."
        "..#.......##............" "........................"))
(defun glyph-ka-f1 ()
  (list "........................" "........##.............."
        "........##.............." "........##.............."
        "........##.............." "........#..............."
        ".......##..............." ".......#####....##......"
        ".....########....##....." ".#######....#.....##...."
        "..#...#.....#......##..." ".....##.....#......##..."
        ".....#.....##.......##.." "....##.....##........##."
        "....##.....##........##." "...##......##.........#."
        "...##.....##............" "..........##............"
        ".....#....##............" ".....##..##............."
        "......#####............." ".......###.............."
        "........#..............." "........................"))
(defun glyph-ka-f2 ()
  (list "........................" ".........#.............."
        "........###............." "........###............."
        "........##.............." "........##.............."
        "........##.......##....." ".......######....###...."
        ".##############...###..." ".########....##....##..."
        "......###....##.....##.." "......##.....##.....###."
        "......##.....##......##." ".....##......##......##."
        ".....##......##........." "....###......##........."
        "....##.......##........." "...###.......##........."
        "...##.......###........." "..###.......##.........."
        "..##...#######.........." "..##....#####..........."
        ".........###............" "........................"))
(defun glyph-ka-f3 ()
  (list "........................" "........................"
        ".......###.............." "........###............."
        "........##.............." "........##.............."
        "........#..............." ".......##.#............."
        ".#....#######....##....." ".#######....##.....#...."
        "..##..##.....#.....##..." "......#......#......##.."
        ".....##.....##......###." ".....#......##.......##."
        "....##......##.....####." "....##......##...##..##."
        "...##.......##.........." "...##......##..........."
        "..##..#....##..........." "..##...##.##............"
        "..#.....####............" "........###............."
        "........................" "........................"))

;; き
(defun glyph-ki-f0 ()
  (list "........................" "..........#............."
        "..........##............" "..........##....####...."
        "........############...." "..###############......."
        "..########.###.........." "............##......#..."
        "............##########.." "...##################..."
        "..##############........" "...#####......###......."
        "...............##......." ".........####...##......"
        "......#############....." "....#####....#######...."
        "....##...........###...." "...###.................."
        "...###.................." "...###.................."
        "....####.......####....." ".....##############....."
        ".......##########......." "........................"))
(defun glyph-ki-f1 ()
  (list ".........#.............." ".........##............."
        ".........##............." "..........##............"
        "..........##..####......" "..........#######......."
        "......#######..........." "....####....##.........."
        ".............#....#....." ".............#######...."
        "...........######......." ".......######.##........"
        "...............##......." "................#......."
        "..............####......" "...............###......"
        ".....#.................." ".....##................."
        ".....###................" "......###..............."
        "........####............" "..........#####........."
        "............###........." "........................"))
(defun glyph-ki-f2 ()
  (list "........................" ".........###............"
        "..........##............" "..........##....###....."
        "...........########....." "...#############........"
        "....#########..........." "............##.....#...."
        "............##..####...." "..........#########....."
        ".....##########........." ".....####.....##........"
        "..............##........" "...............##......."
        "........##########......" "......#############....."
        ".....##.........###....." ".....##................."
        "....###................." ".....##................."
        ".....####..............." "......##########........"
        "........########........" "........................"))
(defun glyph-ki-f3 ()
  (list "........................" ".......####............."
        ".........###............" "..........##....###....."
        "...........#..####......" "...#.......####........."
        "....##########...#......" ".............#....###..."
        ".............##.####...." ".............#####......"
        ".....###.#######........" ".......#####...#........"
        "...............##......." "................##......"
        "................##......" ".......############....."
        ".....##........####....." "....##...........#......"
        "....##.................." "....##.................."
        ".....###......#........." "......##########........"
        "........########........" "........................"))

;; く
(defun glyph-ku-f0 ()
  (list "........................" "................##......"
        "...............###......" "..............###......."
        "............####........" "...........####........."
        "..........####.........." "........####............"
        ".......####............." "......####.............."
        ".....###................" "....###................."
        ".....###................" "......####.............."
        ".......####............." "........#####..........."
        "..........####.........." "...........####........."
        "............####........" "..............####......"
        "...............####....." "................###....."
        ".................##....." "........................"))
(defun glyph-ku-f1 ()
  (list "........................" "...............#........"
        "...............##......." "..............##........"
        ".............##........." "............##.........."
        "...........##..........." "..........##............"
        ".........##............." "........##.............."
        ".......##..............." "......###..............."
        "........##.............." ".........##............."
        "..........##............" "...........##..........."
        "...........###.........." "............##.........."
        ".............##........." "..............##........"
        "...............##......." "...............###......"
        "................##......" "........................"))
(defun glyph-ku-f2 ()
  (list "........................" "...............#........"
        "..............###......." ".............###........"
        "............###........." "...........###.........."
        "..........###..........." ".........###............"
        "........###............." ".......###.............."
        "......###..............." ".....###................"
        ".....###................" "......####.............."
        ".......####............." ".........###............"
        "..........###..........." "...........###.........."
        "............####........" ".............####......."
        "..............####......" "...............####....."
        "................##......" "........................"))
(defun glyph-ku-f3 ()
  (list "........................" ".............##........."
        ".............###........" ".............###........"
        "............###........." "............##.........."
        "...........##..........." "..........##............"
        ".........##............." "........##.............."
        ".......##..............." ".......##..............."
        "........#..............." "........##.............."
        ".........##............." "..........##............"
        "...........##..........." "............##.........."
        "............###........." ".............###........"
        "..............###......." "..............###......."
        "...............#........" "........................"))

;; け
(defun glyph-ke-f0 ()
  (list "........................" "...##...........##......"
        "...##..........###......" "...##...........##......"
        "..###...........##......" "..###...........##......"
        "..###...........######.." "..##...###############.."
        "..##...#############...." ".###............##......"
        ".###............##......" ".###............##......"
        ".###............##......" ".###............##......"
        ".###............##......" ".###...........###......"
        "..##...........##......." "..##...........##......."
        "..###.........###......." "..###........###........"
        "...##......####........." "...##.....####.........."
        "..........###..........." "........................"))
(defun glyph-ke-f1 ()
  (list "........................" "...............#........"
        "...............##......." "....#..........##......."
        "...##..........##......." "...##..........##......."
        "...##..........######..." "...##.....##########...."
        "...##...####...##......." "...##..........##......."
        "...##..........##......." "...#...........##......."
        "...#...........##......." "...#...........##......."
        "...#...........##......." "...#.#.........##......."
        "...###.........#........" "...###........##........"
        "...###........##........" "...##........##........."
        "....#........#.........." "............##.........."
        "...........#............" "........................"))
(defun glyph-ke-f2 ()
  (list "........................" "...#...........###......"
        "...##..........###......" "...##..........###......"
        "...##..........###......" "...##...........##......"
        "...##..........#######.." "..##.....#############.."
        "..##.....#########......" "..##............##......"
        "..##............##......" "..##............##......"
        "..##............##......" "..##............##......"
        "..##..#.........##......" "..##.##........###......"
        "..##.##........##......." "..##.#.........##......."
        "...###........###......." "...###........##........"
        "...##........##........." "....#......####........."
        "...........##..........." "........................"))
(defun glyph-ke-f3 ()
  (list "........................" "..............###......."
        "...#...........###......" "...##..........###......"
        "...##..........##......." "...##..........##..###.."
        "...##..........#######.." "...#.....###########...."
        "..##.......######......." "..##............#......."
        "..##............#......." "..##..#.........#......."
        "..##..#.........#......." "..##.#.........##......."
        "..##.#.........##......." "..##.#.........##......."
        "..####.........##......." "..###.........###......."
        "...##.........##........" "...##........##........."
        "....#........#.........." "............#..........."
        "...........#............" "........................"))

;; こ
(defun glyph-ko-f0 ()
  (list "........................" "...#################...."
        "...##################..." "....################...."
        ".............#####......" "...........#####........"
        "...........####........." "...........##..........."
        "........................" "........................"
        "........................" "....##.................."
        "...###.................." "..####.................."
        "..###..................." ".###...................."
        ".###...................." ".###...................."
        ".###...................." "..####..............##.."
        "..#####################." "....###################."
        "......##############...." "........................"))
(defun glyph-ko-f1 ()
  (list "........................" "....############........"
        "....###############....." "...............####....."
        "..............####......" ".............###........"
        "............##.........." "........................"
        "........................" "........................"
        "........................" "........................"
        "........................" "........................"
        "........................" "..#....................."
        ".###...................." "..##...................."
        "...##..................." "...####................."
        ".....######............." ".......###############.."
        "..........############.." "........................"))
(defun glyph-ko-f2 ()
  (list "........................" "....#######..#######...."
        "....################...." ".......#############...."
        "............####........" "..........####.........."
        ".........####..........." "..........#............."
        "........................" "........................"
        "........................" "........................"
        "........................" "...#...................."
        "..###..................." ".###...................."
        ".###...................." ".###...................."
        "..###..................." "..####.................."
        "...########...########.." "....###################."
        ".......#############...." "........................"))
(defun glyph-ko-f3 ()
  (list "........................" "....##.........###......"
        ".....##############....." ".......############....."
        "...........####........." ".........###............"
        "........##.............." "........................"
        "........................" "........................"
        "........................" "........................"
        "........................" "........................"
        "...#...................." "..##...................."
        "..##...................." "..##...................."
        "...##..................." "...###............##...."
        "....#####....########..." ".....#################.."
        "........############...." "........................"))

;; さ
(defun glyph-sa-f0 ()
  (list "........................" "..........##............"
        "..........###..........." "..........###..........."
        "...........##......###.." "...........###########.."
        "..###################..." "..##############........"
        "...###.......###........" "..............##........"
        "..............###......." "...............###......"
        ".........#####..###....." "......##############...."
        "....######..#########..." "....###..........####..."
        "...###.................." "...###.................."
        "...###.................." "....###................."
        "....#####......#####...." ".....###############...."
        ".......###########......" "........................"))
(defun glyph-sa-f1 ()
  (list "........................" "..........##............"
        "..........##............" "...........##..........."
        "...........##..........." "............##...####..."
        "............########...." ".......########........."
        "...########..##........." "..............##........"
        "...............##......." "...............##......."
        "..............####......" ".............#####......"
        "........................" "......#................."
        ".....##................." "......##................"
        ".......##..............." "........###............."
        ".........####..........." "...........######......."
        "..............###......." "........................"))
(defun glyph-sa-f2 ()
  (list "........................" "..........##............"
        "..........##............" "..........###..........."
        "...........##......#...." "...........##..#####...."
        "..........#########....." "...############........."
        "....######...##........." ".............##........."
        "..............##........" "..............###......."
        "...............##......." ".......########.##......"
        "......#############....." ".....##.........###....."
        "....##............#....." "....##.................."
        "....##.................." ".....##................."
        ".....####..............." "......###########......."
        ".........########......." "........................"))
(defun glyph-sa-f3 ()
  (list "........................" "........#####..........."
        "........###............." "..........##....####...."
        "...........##....####..." "............##..####...."
        "...#.........#####......" "....############........"
        "......######..##........" "...............##......."
        "...............##......." "................##......"
        "................##......" "........########.##....."
        "......###....######....." ".....##.........####...."
        ".....#............#....." "....##.................."
        ".....#.................." ".....##................."
        "......####....#........." ".......#########........"
        ".........#######........" "........................"))

;; し
(defun glyph-shi-f0 ()
  (list "........................" "...###.................."
        "...###.................." "...###.................."
        "...###.................." "...###.................."
        "...###.................." "...###.................."
        "...##..................." "...##..................."
        "..###..................." "..###..................."
        "..###..............##..." "..###.............###..."
        "..###.............###..." "..###.............##...."
        "...##............###...." "...###...........##....."
        "...###..........###....." "...####.......####......"
        "....#############......." ".....###########........"
        ".......#######.........." "........................"))
(defun glyph-shi-f1 ()
  (list "........................" "...###.................."
        "....##.................." "....##.................."
        "....##.................." "....##.................."
        "....##.................." "....##.................."
        "....##.................." "....##.................."
        "...##..................." "...##..................."
        "...##..................." "...##..................."
        "...##..................." "...##..................."
        "...##..................." "...##..............##..."
        "....##............##...." "....##...........##....."
        ".....##.......####......" ".....###########........"
        ".......#######.........." "........................"))
(defun glyph-shi-f2 ()
  (list "........................" "....###................."
        "....###................." "....###................."
        "....###................." "....###................."
        "....###................." "....##.................."
        "....##.................." "....##.................."
        "....##.................." "....##.................."
        "....##.................." "....##.................."
        "....##.................." "....##.................."
        "....##............##...." "....##...........###...."
        "....##..........####...." "....##........####......"
        "....###.....#####......." ".....##########........."
        "......#######..........." "........................"))
(defun glyph-shi-f3 ()
  (list "........................" "....###................."
        ".....###................" "......##................"
        "......##................" "......##................"
        "......##................" "......##................"
        ".....###................" ".....##................."
        ".....##................." ".....##................."
        ".....##................." ".....##................."
        ".....##................." ".....##................."
        ".....##................." ".....##............#...."
        ".....##...........#....." "......#.........##......"
        "......##.....####......." ".......#########........"
        "........######.........." "........................"))

;; す
(defun glyph-su-f0 ()
  (list "........................" ".............##........."
        ".............##........." ".............##........."
        "....###################." ".######################."
        ".###############........" ".............##........."
        ".........##..##........." ".......########........."
        "......####.####........." ".....###.....##........."
        ".....###.....###........" ".....###.....###........"
        ".....###....####........" "......##########........"
        ".......######.##........" ".............###........"
        "............###........." "...........####........."
        "........######.........." "......######............"
        ".......###.............." "........................"))
(defun glyph-su-f1 ()
  (list "........................" ".............#.........."
        ".............#.........." ".............#.........."
        ".............#......##.." "..........############.."
        "..#############........." "..####.......#.........."
        ".............#.........." "..........####.........."
        ".........##..##........." ".........#...##........."
        ".........#...##........." ".........##..##........."
        ".........######........." "..........#####........."
        ".............#.........." "............##.........."
        "............#..........." "...........##..........."
        "..........##............" ".........##............."
        "........##.............." "........................"))
(defun glyph-su-f2 ()
  (list "........................" "............###........."
        "............###........." "............##.........."
        "............##.........." ".....##################."
        ".######################." ".####.......##.........."
        ".............#.........." "..........#..#.........."
        "........######.........." ".......##...##.........."
        ".......#....###........." "......##....###........."
        ".......##...###........." ".......########........."
        "........####.##........." "............##.........."
        "............##.........." "...........##..........."
        "..........###..........." "........###............."
        "........##.............." "........................"))
(defun glyph-su-f3 ()
  (list "........................" "...........###.........."
        "............###........." "............##.........."
        "............##......#..." "............##########.."
        ".##..#########.....####." "..#####.....##.........."
        "...##.......##.........." "..........####.........."
        ".........#..##.........." "........#...##.........."
        "........#....##........." "........#....##........."
        "........##..###........." "........#######........."
        "..........####.........." "............##.........."
        "............##.........." "...........##..........."
        "..........##............" ".........##............."
        "........#..............." "........................"))

;; せ
(defun glyph-se-f0 ()
  (list "........................" "........................"
        "......#.........##......" "......##........##......"
        "......##........##......" "......##........##......"
        "......##........##......" "......##...############."
        ".######################." ".###########....##......"
        "......##........##......" "......##........##......"
        "......##........##......" "......##...#...###......"
        "......##..#######......." "......##...######......."
        "......##................" "......##................"
        "......###..............." "......######...######..."
        ".......##############..." ".........##########....."
        "........................" "........................"))
(defun glyph-se-f1 ()
  (list "........................" "........................"
        "...............##......." "...............##......."
        "...............##......." ".......##......##......."
        ".......##......##......." ".......##......##......."
        ".......##......########." ".......##..##########..."
        "......########.#........" ".########......#........"
        ".###....#......#........" "........#.....##........"
        "........#...#.##........" "........#....###........"
        "........##....#........." "........##.............."
        ".........##............." "..........####.........."
        "...........########....." "..............#####....."
        "........................" "........................"))
(defun glyph-se-f2 ()
  (list "........................" "........................"
        "................##......" "................##......"
        "......##........##......" "......###.......##......"
        ".......##.......##......" ".......##.......##......"
        ".......##.....#########." "......#################."
        ".############..##......." ".#######.......##......."
        ".......#.......##......." ".......#.......##......."
        ".......#.......##......." ".......#....#####......."
        ".......#....####........" ".......##..............."
        ".......##..............." ".......##..............."
        "........############...." ".........###########...."
        "........................" "........................"))
(defun glyph-se-f3 ()
  (list "........................" "........................"
        "..............##........" "...............##......."
        "...............###......" "......#........##......."
        ".......##......##......." ".......##......##......."
        ".......##......########." ".......##...###########."
        ".......#####...##......." ".##.#####......##......."
        "..####.##......#........" "...#...##..#...#........"
        ".......##...####........" ".......##....###........"
        "........#....##........." "........#..............."
        "........#..............." ".........##....####....."
        "..........##########...." "............######......"
        "........................" "........................"))

;; そ
(defun glyph-so-f0 ()
  (list "........................" "..............####......"
        ".....#############......" ".....#############......"
        "..............###......." ".............###........"
        "...........####........." ".........####..........."
        ".......#####............" "......####......#######."
        "...####################." ".###################...."
        ".#######....####........" "...........###.........."
        "..........##............" ".........###............"
        ".........##............." ".........##............."
        ".........###............" ".........#####.........."
        "..........#########....." "............#######....."
        "...............####....." "........................"))
(defun glyph-so-f1 ()
  (list "........................" ".............###........"
        "......##########........" "......####...##........."
        "............##.........." "...........##..........."
        "..........##............" ".........##............."
        "........##.............." ".......##......#######.."
        "......##..#########....." ".....##########........."
        "...######..##..........." "..####....##............"
        "..........#............." ".........##............."
        ".........#.............." ".........#.............."
        ".........##............." ".........###............"
        "..........####.........." "...........######......."
        "..............####......" "........................"))
(defun glyph-so-f2 ()
  (list "........................" "..............###......."
        "......############......" "......######..###......."
        "......#......###........" "............###........."
        "...........##..........." "..........##............"
        ".........##............." ".......###......######.."
        "......###...##########.." ".....#############......"
        "...########.###........." "..#####....###.........."
        "..##.......##..........." "..........##............"
        "..........##............" "..........##............"
        "..........##............" "..........###..........."
        "...........####........." "............#######....."
        "..............#####....." "........................"))
(defun glyph-so-f3 ()
  (list "........................" "..............##........"
        ".....#....#######......." "......######..####......"
        "......###.....###......." ".............##........."
        "............##.........." "...........##..........."
        ".........###.......#...." "........##......######.."
        ".......##...##########.." "......##.#######........"
        ".....######.##.........." "..#######..##..........."
        "..####....##............" "...#......#............."
        ".........##............." ".........##............."
        "..........#............." "..........##............"
        "...........###.........." "............######......"
        ".............#####......" "........................"))

;; た
(defun glyph-ta-f0 ()
  (list "........................" "........##.............."
        ".......###.............." ".......###.............."
        ".......##..............." "..#############........."
        ".##############........." ".########..............."
        "......##......#######..." "......##...###########.."
        ".....###...##########..." ".....##................."
        ".....##................." "....###................."
        "....###................." "....##....###..........."
        "...###....##............" "...##....###............"
        "..###....##............." "..###....###............"
        "..##.....####.......###." ".###......#############."
        "..#.........##########.." "........................"))
(defun glyph-ta-f1 ()
  (list "........................" "........##.............."
        ".........#.............." "........##.............."
        "........##.............." "........##..###........."
        ".......#######.........." "..########.............."
        "...#...##..............." ".......##......#######.."
        ".......##....#########.." ".......#................"
        "......##................" "......##................"
        "......#................." "......#................."
        ".....##................." ".....##................."
        ".....#......#..........." "....##.......##........."
        "....##........####......" "...##...........######.."
        "...##..............##..." "........................"))
(defun glyph-ta-f2 ()
  (list "........................" ".........##............."
        ".........##............." ".........##............."
        "........##.............." "........##...#.........."
        "........#######........." ".############..........."
        ".#########.............." ".......##..............."
        ".......##....#########.." "......###....#########.."
        "......##................" "......##................"
        ".....###................" ".....##................."
        ".....##................." "....##.................."
        "....##.....##..........." "...###.....###.........."
        "...##.......###........." "..###........##########."
        "...#..........#########." "........................"))
(defun glyph-ta-f3 ()
  (list "........................" "........###............."
        ".........###............" ".........##............."
        ".........##............." "........##...##........."
        "........##.####........." ".##.....######.........."
        "..#########............." "....#####........####..."
        ".......##.....########.." ".......#..........##...."
        "......##.........#......" "......##................"
        "......#................." ".....##................."
        ".....##................." "....##......#..........."
        "...###......#..........." "...###......##.........."
        "..###........##.....#..." "...##........##########."
        "...#...........########." "........................"))

;; ち
(defun glyph-chi-f0 ()
  (list "........................" "..........##............"
        ".........###............" ".........##............."
        ".........##.......###..." "..###################..."
        "..##################...." ".......###.............."
        ".......###.............." ".......##..............."
        "......###..............." "......##.....####......."
        ".....###.###########...." ".....################..."
        "....######.........###.." "....####...........###.."
        "....##..............##.." "....................##.."
        "...................###.." "..................###..."
        ".....#####.....######..." ".....##############....."
        ".......##########......." "........................"))
(defun glyph-chi-f1 ()
  (list "........................" ".........#.............."
        ".........#.............." "........##.............."
        "........##.............." "........##....##........"
        "........########........" "...#########............"
        "....##..#..............." "........#..............."
        ".......##..............." ".......##..............."
        ".......##......##......." ".......##..#########...."
        ".......##.###.....###..." ".......####........##..."
        ".......##..........##..." "...................##..."
        "..................##...." ".................##....."
        "...............###......" "............####........"
        "..........###..........." "........................"))
(defun glyph-chi-f2 ()
  (list "........................" "........###............."
        "........###............." "........##.............."
        "........##.............." "........##.#####........"
        "..##############........" "..########.............."
        "........##.............." ".......##..............."
        ".......##..............." ".......##....######....."
        ".......##..#########...." ".......######......##..."
        "......#####........###.." "......###...........##.."
        "......###...........##.." ".......#...........###.."
        "...................##..." ".................####..."
        "..............#####....." "..........########......"
        "..........#####........." "........................"))
(defun glyph-chi-f3 ()
  (list "........................" "........###............."
        ".........###............" ".........##............."
        ".........##............." ".........#....###......."
        "...#....########........" "....#########..........."
        "........##.............." "........#..............."
        "........#..............." ".......##..............."
        ".......##..............." ".......##...#######....."
        ".......##.###.....###..." ".......####........##..."
        ".......###.........###.." ".......##..........###.."
        "..................###..." ".................###...."
        "................###....." "..............####......"
        "..........#####........." "........................"))

;; つ
(defun glyph-tsu-f0 ()
  (list "........................" "........................"
        "........................" ".........#########......"
        "....################...." ".####################..."
        ".######...........####.." ".##................###.."
        "....................###." "....................###."
        "....................###." "....................###."
        "....................###." "....................###."
        "...................###.." "..................####.."
        ".................####..." "..............######...."
        ".......############....." "......##########........"
        ".......######..........." "........................"
        "........................" "........................"))
(defun glyph-tsu-f1 ()
  (list "........................" "........................"
        "........................" "........................"
        "...........########....." "........############...."
        ".....######........###.." "...####.............##.."
        ".####................##." ".....................##."
        ".....................##." ".....................##."
        ".....................##." "....................##.."
        "...................###.." "..................###..."
        "................###....." ".............####......."
        "..........#####........." ".........###............"
        "........................" "........................"
        "........................" "........................"))
(defun glyph-tsu-f2 ()
  (list "........................" "........................"
        "........................" "..............###......."
        "..........##########...." ".......##############..."
        "....#######........###.." ".#######............###."
        ".####...............###." "..#..................##."
        ".....................##." ".....................##."
        "....................###." "....................###."
        "...................###.." "..................###..."
        "................####...." ".............######....."
        "..........#######......." "..........####.........."
        "........................" "........................"
        "........................" "........................"))
(defun glyph-tsu-f3 ()
  (list "........................" "........................"
        "........................" "........................"
        "............#######....." ".........######..####..."
        ".......####........###.." ".#..#####...........##.."
        ".######..............##." "..####...............##."
        ".....................##." "....................###."
        "....................###." "...................###.."
        "...................###.." "..................###..."
        "................####...." "..............####......"
        "............####........" "..........###..........."
        "........................" "........................"
        "........................" "........................"))

;; て
(defun glyph-te-f0 ()
  (list "........................" "........................"
        "..........#############." ".######################."
        ".####################..." "..#..........####......."
        "............###........." "...........###.........."
        "..........###..........." ".........###............"
        ".........###............" ".........##............."
        "........###............." "........###............."
        "........###............." ".........##............."
        ".........###............" ".........####..........."
        "..........####.........." "...........#########...."
        "............########...." "..............######...."
        "........................" "........................"))
(defun glyph-te-f1 ()
  (list "........................" "..................#####."
        ".............##########." "........###########....."
        ".....#######...##......." ".#######......##........"
        "..##.........##........." ".............##........."
        "............##.........." "...........##..........."
        "...........##..........." "...........#............"
        "..........##............" "..........##............"
        "..........##............" "..........##............"
        "...........##..........." "...........##..........."
        "............##.........." "............###........."
        ".............#####......" "...............######..."
        ".................####..." "........................"))
(defun glyph-te-f2 ()
  (list "........................" ".....................##."
        "...............########." "........###############."
        ".##################....." ".########.....###......."
        "..#.........###........." "...........###.........."
        "...........##..........." "..........##............"
        "..........##............" ".........##............."
        ".........##............." ".........##............."
        ".........##............." ".........##............."
        ".........###............" "..........##............"
        "..........####.........." "...........#####........"
        ".............#######...." "...............#####...."
        "..................##...." "........................"))
(defun glyph-te-f3 ()
  (list "........................" "....................#..."
        ".................#####.." "............###########."
        ".#......######..##......" ".#########....##........"
        "..#####......##........." "...##.......##.........."
        "............#..........." "...........#............"
        "..........##............" "..........##............"
        "..........#............." "..........#............."
        "..........##............" "..........##............"
        "..........##............" "...........##..........."
        "...........###.........." "............######......"
        "..............######...." "................####...."
        "..................#....." "........................"))

;; と
(defun glyph-to-f0 ()
  (list "........................" ".......##..............."
        ".......##..............." ".......##..............."
        ".......###.............." ".......###.............."
        "........##........##...." "........##.....######..."
        "........###.########...." "........#########......."
        "........######.........." "......#####............."
        ".....####..............." "....###................."
        "...###.................." "...###.................."
        "..###..................." "..###..................."
        "..###..................." "...###.................."
        "....########.########..." ".....#################.."
        ".......#############...." "........................"))
(defun glyph-to-f1 ()
  (list "........................" "......##................"
        ".......##..............." ".......##..............."
        ".......##..............." "........##.............."
        "........##.............." "........##.......##....."
        ".........##....####....." ".........##..####......."
        "..........####.........." ".........###............"
        "........##.............." "......###..............."
        ".....###................" "....##.................."
        "....##.................." "...##..................."
        "...##..................." "...###.................."
        "....###...........##...." ".....################..."
        "........###########....." "........................"))
(defun glyph-to-f2 ()
  (list "........................" ".......###.............."
        ".......###.............." "........##.............."
        "........##.............." "........##.............."
        "........##.......#......" "........##......###....."
        ".........##...#####....." ".........##.####........"
        ".........#####.........." "........####............"
        ".......###.............." "......###..............."
        ".....##................." "....###................."
        "....##.................." "...###.................."
        "...###.................." "....##.................."
        "....####.........###...." ".....###############...."
        ".......#############...." "........................"))
(defun glyph-to-f3 ()
  (list "........................" ".......###.............."
        "........###............." "........###............."
        "........###............." "........###............."
        "........##.............." ".........#.............."
        ".........#.............." ".........#....###......."
        ".........##.#####......." "..........######........"
        ".........####..........." "........##.............."
        "......###..............." "......#................."
        ".....##................." "....##.................."
        "....##.................." ".....#.................."
        ".....##.........##......" "......#############....."
        "........##########......" "........................"))

;; な
(defun glyph-na-f0 ()
  (list "........................" ".......##..............."
        ".......###.............." ".......##..............."
        ".......##.......#......." "..###########..####....."
        ".############...#####..." ".########.........####.."
        "......##............###." ".....###.......##......."
        ".....##........##......." "....###........##......."
        "....##.........###......" "...###.........###......"
        "...###.........###......" "..###....#########......"
        "..###...###########....." ".###...###......#####..."
        ".###...##.......######.." ".......###.....###.####."
        "........###...####...##." "........#########......."
        "..........######........" "........................"))
(defun glyph-na-f1 ()
  (list "........................" ".........##............."
        ".........##............." ".........#.............."
        "........##..##.........." ".......######...##......"
        "..########.......###...." "...##..##..........##..."
        ".......#............##.." "......##..........####.."
        "......##................" ".....##.......#........."
        ".....##.......#........." "....##........#........."
        "...##.........#........." "...##.........#........."
        "...#..........##........" "..............##........"
        "........########........" ".......##.....####......"
        ".......#.....##.###....." ".......#######....##...."
        "........#####..........." "........................"))
(defun glyph-na-f2 ()
  (list "........................" "........##.............."
        "........##.............." "........##.............."
        "........##.............." ".......###.##..........."
        ".############...###....." ".#########......#####..."
        "......##...........####." "......##............##.."
        "......##......##........" ".....##.......##........"
        ".....##.......##........" "....##........##........"
        "...###........##........" "...##.........##........"
        "..###.....######........" "..##.....#########......"
        "..##....##....######...." ".......##......######..."
        "........##....##...##..." "........########........"
        ".........######........." "........................"))
(defun glyph-na-f3 ()
  (list "........................" ".......###.............."
        "........###............." ".........##............."
        "........##.............." "........##.###.........."
        "..#.....######.........." "...########.....###....."
        ".......#..........###..." ".......#..........####.."
        "......##........######.." "......#........#........"
        ".....##.......##........" "....##........##........"
        "....##........##........" "...##.........##........"
        "..###.........##........" "..##......#######......."
        "...#.....##...#####....." "........#.....######...."
        "........#.....##..###..." "........###.####...##..."
        "..........#####........." "........................"))

;; に
(defun glyph-ni-f0 ()
  (list "........................" "...##..................."
        "...##..................." "...##..................."
        "..###...#############..." "..###...#############..."
        "..###.....####.........." "..##...................."
        "..##...................." ".###...................."
        ".###...................." ".###...................."
        ".###.....##............." ".###....###............."
        ".###....##.............." ".###...###.............."
        ".###...###.............." ".###...###.............."
        ".###....###.........##.." "..##....###############."
        "..##......############.." "..###..................."
        "..##...................." "........................"))
(defun glyph-ni-f1 ()
  (list "........................" "...##..................."
        "...##..................." "...##...........#####..."
        "...#.......##########..." "...#......####.........."
        "..##...................." "..##...................."
        "..##...................." "..##...................."
        "..##...................." "..##...................."
        "..##...................." ".###...................."
        ".##....................." ".##..#....#............."
        ".##.##....##............" ".##.##....###..........."
        ".####.......####........" ".####........#########.."
        "..###...........#######." "..##...................."
        "..##...................." "........................"))
(defun glyph-ni-f2 ()
  (list "........................" "...##..................."
        "...##..................." "..###..................."
        "..###.....############.." "..##......############.."
        "..##...................." "..##...................."
        "..##...................." ".###...................."
        ".###...................." ".##....................."
        ".##....................." ".##....................."
        ".##....................." ".##..#.................."
        ".##..##..##............." ".##.##...###............"
        ".#####....###..........." "..###......############."
        "..###.......###########." "..###..................."
        "..##...................." "........................"))
(defun glyph-ni-f3 ()
  (list "........................" "..#....................."
        "..##...................." "...##..............#...."
        "...##...........#####..." "..###.....###########..."
        "..###........#####......" "..##.........##........."
        "..##........#..........." "..#....................."
        ".##....................." ".##....................."
        ".##....................." ".##..#.................."
        ".##..#...#.............." ".##.##...#.............."
        ".##.#....#.............." ".##.#....#.............."
        ".####....##............." ".####.....####....####.."
        "..##.......############." "..##..........########.."
        "..##...................." "........................"))

;; ぬ
(defun glyph-nu-f0 ()
  (list "........................" ".............#.........."
        "............###........." "....##......###........."
        "....##......##.........." "....##......##.........."
        "....##..#########......." "....###############....."
        "....#####..##....###...." "....###....##.....###..."
        "...####....##......##..." "..#####...##.......##..."
        "..##..##..##.......###.." ".###..##.###.......###.."
        ".##...#####........###.." ".##....####..#########.."
        ".##....###..#########..." ".##....###.##.....###..."
        ".###..#######.....####.." "..######.#.###...######."
        "...####.....#######..##." ".............#####......"
        "........................" "........................"))
(defun glyph-nu-f1 ()
  (list "........................" ".............#.........."
        ".............##........." ".............##........."
        "....#........#.........." "....##......##.........."
        "....##......######......" ".....#....##########...."
        ".....#..#####......##..." ".....####..##.......##.."
        ".....###...#........##.." "....###...##.........##."
        "...##.#...#..........##." "..##..#..##..........##."
        "..#...####...........#.." ".##...###...........##.."
        ".##....##.....##....##.." ".##...###...#########..."
        ".##..####...#....####..." "..####..##..#....####..."
        "...#........#######.##.." "..............##.....##."
        "......................#." "........................"))
(defun glyph-nu-f2 ()
  (list "........................" "........................"
        "...........##..........." "...........##..........."
        "...##......##..........." "...##......##..........."
        "...##......####........." "...##....#########......"
        "....##.#####....###....." "....#####.##......##...."
        "....###...##.......##..." "...###....#........##..."
        "...###...##........##..." "..##.##..##........##..."
        "..##.##.##.........##..." ".##...####.........##..."
        ".##...###....########..." ".##...###...###.#####..."
        ".##...###...##....###..." ".######.##..##...#####.."
        "..####......#######..##." ".............#####...#.."
        "........................" "........................"))
(defun glyph-nu-f3 ()
  (list "........................" "........................"
        ".........###............" "..........##............"
        "...........##..........." "...#.......##..........."
        "...##......##..........." "....#.....########......"
        "....#...####.....##....." "....####..##......##...."
        "....###...#........#...." "....##...##........##..."
        "...###...#.........##..." "..##.##.##.........##..."
        ".##..##.#..........##..." ".##...###..........##..."
        ".#....##.....########..." ".#....###...##...####..."
        "..#.#####...#.....####.." "..####..#...#....##..##."
        "..##.........#####...##." "..............###......."
        "........................" "........................"))

;; ね
(defun glyph-ne-f0 ()
  (list "........................" "......#................."
        "......##................" ".....###................"
        ".....###................" ".....###......###......."
        ".#######...########....." ".#######..####..####...."
        ".....##..###......##...." ".....#####........###..."
        ".....####.........###..." ".....###...........##..."
        "....###............##..." "...####............##..."
        "...####......###...##..." "..#####.....#########..."
        ".###.##....###..#####..." ".##..##...##......###..."
        ".....##...###....######." ".....##....###..###..##."
        ".....##....#######....#." ".....##......###........"
        "......#................." "........................"))
(defun glyph-ne-f1 ()
  (list "........................" ".......#................"
        ".......#................" ".......#................"
        ".......#................" "......##......####......"
        "......##....#######....." ".....####..##.....##...."
        "...#####..##......##...." "..##..#####........#...."
        "......###..........##..." ".....###...........##..."
        ".....###...........##..." "....###............##..."
        "....###............##..." "...##.#............##..."
        "...##.#............##..." "..##..#.....########...."
        ".##...#....##....####..." ".##...#....##....#####.."
        "......##....######...##." "......##.....####.....#."
        "......#................." "........................"))
(defun glyph-ne-f2 ()
  (list "........................" "......##................"
        "......##................" "......##................"
        "......##................" "......##................"
        "......##.....######....." ".....####..####.###....."
        ".#######..###.....##...." ".###..##.###......##...."
        "......####........##...." ".....####.........##...."
        ".....###..........###..." "....####..........###..."
        "....####..........##...." "...##.##..........##...."
        "..##..##.....#######...." ".###..##....########...."
        ".##...##...##.....###..." "......##...##....#####.."
        "......##....#######..##." "......##.....#####......"
        "......##................" "........................"))
(defun glyph-ne-f3 ()
  (list "........................" ".....#.................."
        "......#................." "......##................"
        "......##................" "......##................"
        "......##.......##......." ".....###....##..###....."
        ".#######..##......#....." "..##..##.##.......##...."
        "......#.##........##...." ".....###..........###..."
        "....####...........##..." "....###............##..."
        "...##.#...........###..." "..##..#...........##...."
        "..##..#......#######...." ".##...#.....#....####..."
        ".####.#.....#....#####.." "....###.....#....##.###."
        ".....##.....######...##." ".....##................."
        "......#................." "........................"))

;; の
(defun glyph-no-f0 ()
  (list "........................" "........................"
        "........#########......." "......#############....."
        ".....####..##...####...." "....###....##.....###..."
        "...###.....##......###.." "..###......##......###.."
        "..##......###.......###." ".###......###.......###."
        ".###......###.......###." ".##.......##.........##."
        ".##.......##.........##." ".##......###........###."
        ".##......###........###." ".###....###.........##.."
        ".###....###........###.." "..###..###........###..."
        "..########......#####..." "...######....######....."
        "............######......" "............####........"
        "........................" "........................"))
(defun glyph-no-f1 ()
  (list "........................" "...........#####........"
        "........###########....." ".......######...####...."
        ".....###...##.....###..." "....##.....#........##.."
        "...##.....##........##.." "..###.....##.........##."
        "..##......##.........##." ".##......##..........##."
        ".##......##..........##." ".##......##..........##."
        ".##.....##...........##." ".##.....##...........##."
        ".##.....#...........##.." "..##...##...........##.."
        "..##..##...........##..." "...#####..........##...."
        "....###..........##....." "...............###......"
        "..............###......." "............###........."
        "...........##..........." "........................"))
(defun glyph-no-f2 ()
  (list "........................" "........................"
        "..........######........" ".......###########......"
        ".....#####.##...####...." "....###....##.....###..."
        "...###.....##......###.." "..###.....###.......##.."
        "..##......##........###." ".###......##.........##."
        ".##.......##.........##." ".##......###.........##."
        "###......##..........##." "###.....###.........###."
        ".##.....##..........###." ".##....###..........##.."
        ".###...##..........###.." "..#######.........###..."
        "...#####........####...." "..............#####....."
        "............#####......." ".............##........."
        "........................" "........................"))
(defun glyph-no-f3 ()
  (list "........................" "........................"
        "..........######........" ".......#####...###......"
        ".....###...##....###...." "....##.....##......##..."
        "...##......##......###.." "..##.......##.......##.."
        "..##......##........###." ".##.......##........###."
        ".##......##..........##." "###......##.........###."
        "###.....###.........###." ".##.....##..........###."
        "..#....##...........##.." "..#...###..........###.."
        "...#.###...........##..." "...####...........###..."
        "...###...........###...." "...............###......"
        ".............###........" "............##.........."
        "........................" "........................"))

;; は
(defun glyph-ha-f0 ()
  (list "........................" "...##..........##......."
        "..###..........##......." "..###..........##......."
        "..###..........##......." "..##...........###.###.."
        "..##...###############.." ".###...##############..."
        ".###...........##......." ".###...........###......"
        ".###...........###......" ".###...........###......"
        ".###...........###......" ".###...........###......"
        ".###.......#######......" ".###.....#########......"
        ".###....###...######...." ".###...###......#####..."
        ".###...###......##.###.." ".###...###.....###..###."
        "..##....##########...#.." "..##.....########......."
        "..##.......####........." "........................"))
(defun glyph-ha-f1 ()
  (list "...............#........" "...............##......."
        "...##..........##......." "...##...........#......."
        "...##...........#......." "...##...........#......."
        "...##........#########.." "...#....##########......"
        "...#.....#......#......." "..##............#......."
        "..##............#......." "..##............#......."
        "..##............#......." "..##............#......."
        "..##............#......." "..##............#......."
        "..##.#..........#......." "..####....#######......."
        "..####..###...#####....." "..###...##.....#####...."
        "..###...###..###...###.." "..###....######.....##.."
        "...#...................." "........................"))
(defun glyph-ha-f2 ()
  (list "........................" "...#...........##......."
        "...##..........###......" "..###..........##......."
        "..##...........##......." "..##...........##......."
        "..##...........#######.." "..##.....#############.."
        "..##.....#########......" ".###...........##......."
        ".##............##......." ".##............##......."
        ".##............##......." ".##............##......."
        ".##............##......." ".##..#.........###......"
        ".##.##....########......" ".##.#....##########....."
        ".####...##.....######..." ".####...##.....##..####."
        "..##....###...###...##.." "..##.....#######........"
        "..##.......###.........." "........................"))
(defun glyph-ha-f3 ()
  (list "........................" "..##..........###......."
        "...#...........###......" "...##...........##......"
        "...##...........##......" "..###...........##......"
        "..###...........##..###." "..##.....#......#######."
        "..##......##########...." "..##........######......"
        ".##.............##......" ".##.............##......"
        ".##.............##......" ".##..#..........##......"
        ".##..#..........##......" ".##.##..........##......"
        ".##.#........#####......" ".####.....##########...."
        ".####....##.....######.." ".####....#......##.####."
        "..##.....#.....###..###." "..##......#######....##."
        "...#.......#####........" "........................"))

;; ひ
(defun glyph-hi-f0 ()
  (list "........................" "........................"
        "....######....##........" ".##########...###......."
        ".#####.###.....##......." "......##.......###......"
        ".....##........####....." "....###........####....."
        "...###.........#####...." "...##..........##.###..."
        "..###..........###.###.." "..##...........###..###."
        "..##...........###...##." "..##...........###......"
        "..##...........###......" "..##...........##......."
        "..##...........##......." "..###.........###......."
        "...###.......###........" "...#####...####........."
        "....##########.........." "......#######..........."
        "........................" "........................"))
(defun glyph-hi-f1 ()
  (list "........................" ".........##............."
        ".....######............." ".#########.......##....."
        "..##...##........##....." "......##.........##....."
        ".....##..........##....." ".....##.........###....."
        "....##..........###....." "....##..........#.##...."
        "...##..........##.##...." "...##..........##..#...."
        "...##..........##..##..." "...#..........##...##..."
        "..##..........##....##.." "..##.........##.....###."
        "..##.........##......#.." "...#........##.........."
        "...##.......##.........." "...##......##..........."
        "...###....##............" "....#######............."
        ".....#####.............." "........................"))
(defun glyph-hi-f2 ()
  (list "........................" "........................"
        ".......###.....#........" ".#########.....##......."
        ".########......##......." "......##.......###......"
        ".....##........###......" "....##.........####....."
        "....##.........####....." "...##..........##.##...."
        "...##..........##.###..." "..##...........##..###.."
        "..##...........##...###." "..##...........##....##."
        "..##...........##......." "..##..........###......."
        "..##..........##........" "..##.........###........"
        "...##........##........." "...###.....###.........."
        "....#########..........." ".....#######............"
        "........................" "........................"))
(defun glyph-hi-f3 ()
  (list "........................" ".........#.....##......."
        ".#.....####....###......" ".##########.....##......"
        "...######.......##......" ".....###........##......"
        ".....##.........###....." "....##..........###....."
        "....#...........#.##...." "...##...........#.##...."
        "...#............#.##...." "...#............#..##..."
        "..##............#..###.." "..##............#...###."
        "..##............#...###." "..##...........##....##."
        "..##...........##......." "...#..........###......."
        "...##.........##........" "...##........###........"
        "....###...#####........." ".....########..........."
        ".......#####............" "........................"))

;; ふ
(defun glyph-fu-f0 ()
  (list "........................" "......#................."
        ".....#####.............." ".....###########........"
        "........##########......" ".............####......."
        "...........####........." "..........###..........."
        ".........###............" ".........##............."
        ".........###............" "....##....##......##...."
        "...###....###.....##...." "...###.....###....###..."
        "...##.......###....##..." "...##........###...###.."
        "..###.........##....##.." "..##..........##....###."
        ".###..##......##....###." ".###..###....###.....##."
        ".##...#########......#.." "........######.........."
        "...........#............" "........................"))
(defun glyph-fu-f1 ()
  (list "........................" "........................"
        "..........##............" "...........##..........."
        "............##.........." ".............##........."
        "..............##........" "............###........."
        "............#..........." "........................"
        ".........#.............." ".........##............."
        "..........##............" "...........##..........."
        "............##....##...." ".............##....##..."
        ".#...........##.....##.." ".#............#......##."
        ".##..##.......#......##." "..####.#.....##.......#."
        "..####..#######........." "...##.....####.........."
        "........................" "........................"))
(defun glyph-fu-f2 ()
  (list "........................" "........................"
        "..........##............" "..........####.........."
        "............####........" ".............###........"
        "........................" "........................"
        "..........##............" "..........##............"
        ".........##............." ".........##............."
        "..........##......#....." "..........###.....###..."
        "...........###.....##..." "............###.....##.."
        ".##..........###....###." ".##...#.......##.....##."
        "..#####.......##........" "..####..#....###........"
        "........#######........." ".........#####.........."
        "........................" "........................"))
(defun glyph-fu-f3 ()
  (list "........................" "........................"
        ".........#.............." "..........#............."
        "..........##............" "...........###.........."
        "............###........." "...........#####........"
        "..........#............." ".........#.............."
        ".........#.............." ".........#.............."
        "..........#............." "..........##............"
        "...........########....." ".........#####.....##..."
        ".......###...##.....##.." "......##.....##.....###."
        ".#..###......##.....###." ".#######.....##..######."
        "..###...######.........." "..##.....####..........."
        "........................" "........................"))

;; へ
(defun glyph-he-f0 ()
  (list "........................" "........................"
        "........................" "........................"
        "........###............." ".......#####............"
        "......######............" ".....###..###..........."
        ".....###...###.........." "....###.....###........."
        "...###.......###........" "...###.......####......."
        "..###.........####......" ".###...........####....."
        ".##.............####...." ".................####..."
        "..................####.." "...................####."
        "....................###." ".....................##."
        "........................" "........................"
        "........................" "........................"))
(defun glyph-he-f1 ()
  (list "........................" "........................"
        "........................" "........................"
        "........................" "........................"
        "........##.............." ".......#####............"
        "......##..###..........." ".....##.....##.........."
        "....##.......##........." "..###.........##........"
        ".###...........##......." "................###....."
        ".................###...." "...................###.."
        "....................###." ".....................##."
        "........................" "........................"
        "........................" "........................"
        "........................" "........................"))
(defun glyph-he-f2 ()
  (list "........................" "........................"
        "........................" "........................"
        "........................" "........................"
        "........###............." ".......#####............"
        "......#######..........." ".....###...###.........."
        "....###.....###........." "...###.......####......."
        ".####..........###......" ".###............####...."
        ".................####..." "...................####."
        "....................###." ".....................#.."
        "........................" "........................"
        "........................" "........................"
        "........................" "........................"))
(defun glyph-he-f3 ()
  (list "........................" "........................"
        "........................" "........................"
        "........................" "........................"
        "........###............." ".......##.##............"
        "......##....##.........." ".....##......##........."
        "....##........##........" ".#####........###......."
        "..###..........###......" "...##............###...."
        "..................####.." "...................####."
        "....................###." ".....................##."
        "........................" "........................"
        "........................" "........................"
        "........................" "........................"))

;; ほ
(defun glyph-ho-f0 ()
  (list "........................" "...##..................."
        "..###...##############.." "..###...##############.."
        "..###..........##......." "..##...........##......."
        "..##...........##......." ".###...........###......"
        ".###....##############.." ".###...###############.."
        ".###...........###......" ".###...........###......"
        ".###...........###......" ".###............##......"
        ".###........##..##......" ".###.....#########......"
        ".###....############...." ".###...###......#####..."
        ".###...###......######.." ".###...###.....###..###."
        "..##....####..####...##." "..##.....########......."
        "..##......#####........." "........................"))
(defun glyph-ho-f1 ()
  (list "........................" "...#...........######..."
        "...#.....############..." "..##.....####...#......."
        "..##............#......." "..##............#......."
        "..##............##......" "..##............##.###.."
        "..##.....#############.." "..#.......########......"
        ".##.............##......" ".##.............##......"
        ".##.............##......" ".##.............##......"
        ".##.............##......" ".##.............##......"
        ".##.#...........##......" ".##.#.....########......"
        ".####...###########....." ".####...##.....######..."
        ".###....##....##...###.." ".###.....######.....###."
        "..##.......##........#.." "........................"))
(defun glyph-ho-f2 ()
  (list "........................" "...#...................."
        "..###.......#########..." "..##.....############..."
        "..##.....##....##......." "..##...........##......."
        "..##...........##......." "..##...........##......."
        "..##...........##...#..." ".##......#############.."
        ".##......###########...." ".##............##......."
        ".##............##......." ".##............##......."
        ".##............###......" ".##............###......"
        ".##.##....########......" ".##.#....##########....."
        ".####....##.....#####..." ".####....##....###.####."
        ".###.....##...###...###." "..##......######........"
        "..##.......####........." "........................"))
(defun glyph-ho-f3 ()
  (list "..#....................." "..##...................."
        "...#..............###..." "...##....##...#######..."
        "...##......#######......" "...##...........##......"
        "..###...........##......" "..##............##......"
        "..##............##..#..." "..#.......#.....######.."
        ".##.......##########...." ".##.........######......"
        ".##..#..........##......" ".##..#..........##......"
        ".##.##..........##......" ".##.#...........##......"
        ".##.#.........#.##......" ".####.....##########...."
        ".####....##.....######.." ".###.....#......##.####."
        "..##.....#.....###..###." "..##......#######....##."
        "..##.......#####........" "........................"))

;; ま
(defun glyph-ma-f0 ()
  (list "........................" "...........##..........."
        "...........##..........." "...........##..........."
        "..####################.." ".####################..."
        "...........##..........." "...........##..........."
        "...........##..........." "...##################..."
        "..###################..." "...###########.........."
        "...........##..........." "...........##..........."
        "........##.###.........." "....##########.........."
        "...##############......." "...##.......#######....."
        "..###......###..####...." "..###......###...####..."
        "...###....###......###.." "...##########.......##.."
        ".....#######............" "........................"))
(defun glyph-ma-f1 ()
  (list "........................" "............##.........."
        "............##.........." "............##.........."
        "............##.........." "..........#########....."
        "....##########.........." ".....###....##.........."
        "............##.........." "............##.........."
        "............#######....." "......############......"
        ".......#######.........." ".............#.........."
        ".............#.........." ".............#.........."
        ".............#.........." ".............#.........."
        ".....##########........." "....##......#####......."
        "....##.....###.####....." ".....########....###...."
        "......#####.......##...." "........................"))
(defun glyph-ma-f2 ()
  (list "........................" "...........##..........."
        "...........##..........." "...........##..........."
        "...........##..........." "...#################...."
        "...################....." "...........##..........."
        "...........##..........." "...........##..........."
        "...........##..####....." ".....##############....."
        ".....##########........." "...........##..........."
        "...........##..........." "...........##..........."
        ".......###.##..........." ".....##########........."
        "....##.....######......." "...##......##.#####....."
        "...###.....##...#####..." "....#########.....##...."
        ".....######............." "........................"))
(defun glyph-ma-f3 ()
  (list "........................" "..........####.........."
        "...........###.........." "............##.........."
        "............##....##...." "...##.......##.######..."
        ".....#############......" "......########.........."
        "............#..........." "............#..........."
        ".....#......#...####...." "......#############....."
        ".......#########........" "............#..........."
        "............#..........." "............#..........."
        "............##.........." ".......#########........"
        ".....##.....######......" ".....#......##..####...."
        ".....#.....###...###...." ".....########.....##...."
        "......######............" "........................"))

;; み
(defun glyph-mi-f0 ()
  (list "........................" "......#####............."
        "...#########............" "...##########..........."
        "..........###..........." "..........##......#....."
        "..........##.....##....." ".........###.....##....."
        ".........##......##....." ".......#####.....##....."
        "....###########..##....." "...####.###########....."
        "..###...##....#####....." ".###....##......#####..."
        ".##....##.......######.." ".##....##.......##.####."
        ".##...##.......###...##." ".###.###.......##......."
        ".######.......###......." "..####......####........"
        "...........####........." "...........##..........."
        "........................" "........................"))
(defun glyph-mi-f1 ()
  (list "........................" "............#..........."
        ".........####..........." "......#######..........."
        "......##...##..........." "..........##............"
        "..........##............" ".........##............."
        ".........##............." ".........#.......#......"
        "........##.......##....." "........#........##....."
        ".....########....##....." "...###############......"
        "..##..##.......#####...." ".##...#.........#######."
        ".##..##.........#....##." ".##.##.........##......."
        ".####..........#........" "..............##........"
        ".............##........." ".............#.........."
        "............#..........." "........................"))
(defun glyph-mi-f2 ()
  (list "........................" "...........##..........."
        "....##########.........." ".....########..........."
        "...........##..........." "..........###..........."
        "..........##............" "..........##.....##....."
        ".........##......###...." ".........##......##....."
        "........###......##....." "....##########...##....."
        "...################....." "..##...##.....#####....."
        ".##....##.......#####..." ".##...##........##.####."
        ".##..###.......##...###." ".######........##......."
        "..####........##........" ".............##........."
        "............###........." "...........###.........."
        "...........##..........." "........................"))
(defun glyph-mi-f3 ()
  (list "........................" "...........#............"
        ".....########..........." ".....####..###.........."
        "...........##..........." "..........##............"
        "..........##.....##....." ".........##.......##...."
        ".........##.......##...." "........##........##...."
        "........##.......###...." "....###########..##....."
        "...###.##...#######....." "..#...##.......#####...."
        ".#....##........#####..." ".#...##.........##.####."
        ".#...#.........##...###." ".#####.........##....##."
        ".####.........##........" ".............##........."
        ".............#.........." "...........##..........."
        "...........#............" "........................"))

;; む
(defun glyph-mu-f0 ()
  (list "........................" ".......##..............."
        "......###..............." "......###.......##......"
        "..############..###....." ".#############...###...."
        "..#######.........###..." "......###..........###.."
        "......###...........###." "...######............##."
        "..#######..............." ".###...##..............."
        ".##....##..............." ".##....##........###...."
        ".##....##.........##...." ".###..###.........##...."
        "..######..........###..." "...####...........###..."
        ".....##...........###..." ".....##...........##...."
        ".....###.......#####...." ".....##############....."
        ".......##########......." "........................"))
(defun glyph-mu-f1 ()
  (list "......#................." "......##................"
        ".......#................" ".......#................"
        ".......#..........#....." ".......#..........##...."
        "......##..###......##..." ".....########.......##.."
        ".#######............###." "..##..##.............##."
        "......##................" "......##................"
        "......##................" "......##..........#....."
        "...#####..........#....." "...#####..........#....."
        "..##..##..........#....." "..##..##..........#....."
        "..##..##.........##....." "...##.##.........##....."
        "...#####........##......" ".......##########......."
        "........#######........." "........................"))
(defun glyph-mu-f2 ()
  (list "........................" "......###..............."
        "......###..............." "......###..............."
        "......##................" "......#######..........."
        ".###########.....##....." "..###.##.........###...."
        "......##..........####.." "......##............###."
        "....####.............#.." "..######................"
        "..##..##................" ".##...##................"
        ".##...##.........#......" "..##..##.........##....."
        "..#####..........##....." "....###..........##....."
        ".....##..........##....." "....##...........##....."
        ".....##........###......" ".....#############......"
        ".......#########........" "........................"))
(defun glyph-mu-f3 ()
  (list "........................" ".....###................"
        "......###..............." "......###..............."
        ".......#...##..........." ".#.....#.####..........."
        "..#########.......#....." "...#####.........#.##..."
        "......##.........#..##.." "......##.........#...##."
        "...#####.........#..###." "..##..##.........#.####."
        "..#...##.........#......" "..#...##.........#......"
        "..#...##.........#......" "..##..##.........#......"
        "...####..........#......" "....##...........##....."
        ".....#...........##....." ".....#...........##....."
        ".....#.........####....." ".....#############......"
        ".......#########........" "........................"))

;; め
(defun glyph-me-f0 ()
  (list "........................" "..............##........"
        "....###.......##........" "....###.......##........"
        ".....##......###........" ".....##..#########......"
        ".....##############....." ".....#####...##.#####..."
        "....####.....##...####.." "...#####....###....###.."
        "..###.##....##......###." "..##..###..###......###."
        ".###...##..###......###." ".##....##..##........##."
        ".##....######.......###." ".##.....####........###."
        ".##.....###.........###." ".##.....####.......###.."
        ".###..######......####.." "..#######..#.....####..."
        "...#####......######...." "............######......"
        "............####........" "........................"))
(defun glyph-me-f1 ()
  (list "........................" "...............#........"
        "...............#........" ".....#........##........"
        ".....##.......##........" ".....##.......##........"
        "......#....########....." "......#..######..###...."
        "......####..##.....##..." ".....###....##......##.."
        "....####...##.......###." "...##.##...##........##."
        "...#...#..##.........##." "..##...####..........##."
        ".##....###..........###." ".##....###..........##.."
        ".##....##..........##..." "..##.#####.........##..."
        "..#####.##........##...." "...##...........###....."
        "...............###......" ".............###........"
        "............##.........." "........................"))
(defun glyph-me-f2 ()
  (list "........................" "..............##........"
        "..............###......." "....##........##........"
        "....##........##........" "....##.......##........."
        "....##.....######......." ".....##.###########....."
        ".....#####..##...####..." ".....###....##.....###.."
        "....###....##.......##.." "...#####...##.......###."
        "..##..##..##.........##." "..##..###.##.........##."
        ".##....####..........##." ".##....####..........##."
        ".##.....##..........###." ".##....####.........##.."
        ".##...##.###.......###.." "..#####...........###..."
        "..####.........#####...." ".............#####......"
        ".............###........" "........................"))
(defun glyph-me-f3 ()
  (list "........................" "............###........."
        ".............###........" "....#........###........"
        "....##.......##........." ".....#.......##........."
        ".....##...########......" ".....#####..##...###...."
        ".....###....##....###..." "....###.....#......###.."
        "...##.##...##.......##.." "..##..##...##.......###."
        "..##..##..##........###." ".##....####.........###."
        ".##....####.........###." ".##....###..........###."
        ".##....###..........##.." "..#...#####........###.."
        "..#####..###......###..." "...###...........###...."
        "................###....." "..............###......."
        "............###........." "........................"))

;; も
(defun glyph-mo-f0 ()
  (list "........................" ".........##............."
        ".........##............." "........###............."
        "........###............." "...##############......."
        "...##############......." ".......###.............."
        ".......###.............." ".......###.............."
        ".......##..............." ".#########...###..##...."
        ".################.###..." ".....###########...###.."
        ".......##..........###.." "......###...........##.."
        "......###...........###." "......###...........##.."
        ".......##..........###.." ".......###........###..."
        ".......#####....#####..." "........############...."
        "..........########......" "........................"))
(defun glyph-mo-f1 ()
  (list "........................" "..........##............"
        "..........##............" "..........##............"
        "..........##............" "..........#............."
        "..........#............." ".....########..........."
        "......########.........." ".........##............."
        ".........#.....#........" ".........#......#......."
        "....###.##......##......" ".....#########...#......"
        "........##.......##....." "........##........#....."
        "........##........##...." "........##........##...."
        "........##.......##....." "........##.......##....."
        ".........##.....##......" "..........#######......."
        "...........####........." "........................"))
(defun glyph-mo-f2 ()
  (list "........................" ".........##............."
        ".........###............" ".........##............."
        ".........##............." "....#....##............."
        "....#######..#.........." "....###########........."
        "........###............." "........##.............."
        "........##.............." "........##.............."
        "...####.##.......##....." "...###########...##....."
        ".......#######....##...." "........##........###..."
        ".......##.........###..." ".......##.........###..."
        ".......##.........###..." "........##........##...."
        "........###.....####...." ".........##########....."
        "..........#######......." "........................"))
(defun glyph-mo-f3 ()
  (list "........................" ".........##............."
        "..........##............" "..........##............"
        "..........##............" "....#.....##............"
        ".....#...##............." "......#####............."
        ".......#####............" "......######............"
        ".....#...#....#........." ".....#..##.....#........"
        ".....#..##......#......." ".....##.#.......##......"
        "......####.......##....." ".......#####.....###...."
        "........#.##......##...." "........#.........##...."
        "........#........###...." "........#........##....."
        "........##......###....." ".........#########......"
        "...........#####........" "........................"))

;; や
(defun glyph-ya-f0 ()
  (list "........................" ".............##........."
        ".....#.......##........." "....###......##........."
        ".....##......##........." ".....###...#########...."
        ".....################..." ".....#######.......###.."
        "...######...........###." ".########...........###."
        ".###...##...........###." ".......##...........###."
        ".......###..........###." "........##..###...####.."
        "........##..#########..." "........###..#######...."
        "........###............." ".........##............."
        ".........##............." ".........###............"
        ".........###............" ".........###............"
        "..........##............" "........................"))
(defun glyph-ya-f1 ()
  (list "........................" "..........##............"
        "...........##..........." "....##......##.........."
        ".....##....####........." ".....##....####..##....."
        "......#......#########.." "......##..#####.....###."
        "......######.........##." ".....####............##."
        "...######............##." ".####..##...........##.."
        "..#.....##...########..." "........##....#####....."
        ".........#.............." ".........##............."
        ".........##............." "..........##............"
        "..........##............" "...........##..........."
        "...........##..........." "............#..........."
        "............##.........." "........................"))
(defun glyph-ya-f2 ()
  (list "........................" "............##.........."
        "............###........." ".....##......###........"
        ".....##.......#........." ".....###........####...."
        "......##....##########.." "......##..#####.....###."
        "......######.........##." ".....#####...........##."
        "..#######............##." ".####...##..........###."
        "..#.....##.........###.." "........##....#######..."
        ".........##....####....." ".........##............."
        ".........##............." "..........##............"
        "..........##............" "..........##............"
        "...........##..........." "...........##..........."
        "...........##..........." "........................"))
(defun glyph-ya-f3 ()
  (list "........................" "............#..........."
        ".............##........." "..............##........"
        ".....###########........" ".....##.......##........"
        ".....##.......######...." "......##....###.....##.."
        ".......#.###.........##." ".......####..........##."
        "......###............##." ".######.##..........###."
        "..####..##..#......###.." "...#.....##..########..."
        ".........##......##....." ".........##............."
        "..........##............" "..........##............"
        "..........###..........." "...........##..........."
        "...........##..........." "...........##..........."
        "...........##..........." "........................"))

;; ゆ
(defun glyph-yu-f0 ()
  (list "........................" "............##.........."
        "............##.........." "...##.......###........."
        "..###.......#####......." "..###....##########....."
        "..##....#######.####...." "..##..####...##...###..."
        "..##..##.....##....###.." "..##.##......##....###.."
        "..####.......##.....##.." ".#####.......##.....###."
        ".####........##.....##.." ".####..#.....##.....##.."
        ".####..##....##....###.." "..##...###..###...###..."
        "..##....###.##...###...." "..##.....##########....."
        "..##......########......" "..........###..........."
        ".........###............" "........###............."
        "........##.............." "........................"))
(defun glyph-yu-f1 ()
  (list "........................" ".............##........."
        ".............##........." ".............##........."
        ".............#####......" "..##........########...."
        "..##......#####....##..." "...#.....##..##.....##.."
        "...#...###...##.....##.." "...#..##.....##.....##.."
        "...#..#......##.....##.." "...#.##......##.....##.."
        "...###.......##....##..." "...##........##....##..."
        "...##........##...##...." "...##.....##.##..##....."
        "...##.....##.##.##......" "...##......######......."
        "...##.......###........." "............##.........."
        "............##.........." "...........##..........."
        "...........#............" "........................"))
(defun glyph-yu-f2 ()
  (list "........................" ".............##........."
        ".............##........." "..##.........##........."
        "..##.........##........." "..##........######......"
        "..##......##########...." "..##....####..##..###..."
        "..##...###....##...###.." "..##..##......##....##.."
        ".###.##.......##....##.." ".##.###.......##....###."
        ".##.##........##....##.." ".####.........##....##.."
        ".####...##...##....###.." ".###....###..##...###..."
        "..##.....######.####...." "..##.......########....."
        "............###........." "...........###.........."
        "..........###..........." ".........###............"
        ".........##............." "........................"))
(defun glyph-yu-f3 ()
  (list "........................" "..........####.........."
        ".........#...##........." ".........#...##........."
        "..#.....#....#####......" "..##....#.###.##..##...."
        "..###...###...##...##..." "..##....#.....##....##.."
        "..##...##.....##....##.." "..##..###.....##....###."
        "..#..#..#.....##....###." "..#.##..#.....##....###."
        "..#.#...#.....##....###." "..###...#.....##....##.."
        "..###...##...###...###.." "..##.....##..##...###..."
        "..##......#####.####...." "...#.......#######......"
        "............##.........." "............#..........."
        "...........#............" "..........#............."
        ".........#.............." "........................"))

;; よ
(defun glyph-yo-f0 ()
  (list "........................" "...........##..........."
        "...........##..........." "...........##..........."
        "...........##..........." "...........##..........."
        "...........###########.." "...........###########.."
        "...........#####........" "...........##..........."
        "...........###.........." "...........###.........."
        "...........###.........." "............##.........."
        ".....#########.........." "...#############........"
        "...###.....######......." "..###.......#######....."
        "..##........##..####...." "..###.......##....####.."
        "..####....####.....###.." "...##########.......#..."
        ".....#######............" "........................"))
(defun glyph-yo-f1 ()
  (list "........................" "............#..........."
        "............##.........." "............##.........."
        "............##.........." "............##.........."
        "............##.....#...." "............#########..."
        "............#####......." "............##.........."
        "............##.........." "............##.........."
        "............##.........." "............##.........."
        "............##.........." "............##.........."
        "............##.........." ".....######.##.........."
        "....###########........." "...##.......#####......."
        "...##......##..####....." "....########.....####..."
        ".....#####.........##..." "........................"))
(defun glyph-yo-f2 ()
  (list "........................" "..........###..........."
        "..........###..........." "...........##..........."
        "...........##..........." "...........##..........."
        "...........##....###...." "...........#########...."
        "...........#######......" "...........##..........."
        "...........##..........." "...........##..........."
        "...........##..........." "...........##..........."
        "...........##..........." "...........##..........."
        ".....#########.........." "....############........"
        "...##......#######......" "...##......##...####...."
        "...##.....###.....###..." "....########.......#...."
        ".....######............." "........................"))
(defun glyph-yo-f3 ()
  (list "........................" ".........####..........."
        "..........###..........." "...........##..........."
        "...........##..........." "...........##..........."
        "...........##....##....." "...........##..#####...."
        "...........######......." "...........##..........."
        "...........##..........." "...........##..........."
        "...........##..........." "...........##..........."
        "...........##..........." "...........##..........."
        ".......#######.........." ".....############......."
        "....#......##.#####....." "...##......##...####...."
        "...##......##.....###..." "....#########......##..."
        ".....######............." "........................"))

;; ら
(defun glyph-ra-f0 ()
  (list "........................" "......####.............."
        "......###########......." ".......###########......"
        "............######......" "........................"
        "....##.................." "....##.................."
        "...###.................." "...###.................."
        "...##.....#######......." "...##...###########....."
        "...##.######....####...." "..#######.........###..."
        "..#####............###.." "..####.............###.."
        "...#...............###.." "...................###.."
        "..................###..." ".................####..."
        "....######..########...." "....##############......"
        "......##########........" "........................"))
(defun glyph-ra-f1 ()
  (list ".......#................" ".......###.............."
        "........####............" "..........###..........."
        "...........###.........." ".........####..........."
        "......#................." ".....##................."
        ".....##................." ".....##................."
        ".....#.................." "....##.................."
        "....##......####........" "....##...#########......"
        "....##.###.......##....." "....####..........##...."
        ".....##...........##...." ".................##....."
        "................###....." "...............###......"
        ".............####......." "..........#####........."
        "........####............" "........................"))
(defun glyph-ra-f2 ()
  (list "........................" ".......###.............."
        ".......####............." ".........####..........."
        "..........###..........." "........................"
        "........................" "....###................."
        "....##.................." "....##.................."
        "....##.......###........" "....##....########......"
        "....##..#####...###....." "...###.###........##...."
        "...######.........##...." "...####...........###..."
        "...###............###..." "....##............##...."
        ".................##....." "...............####....."
        "............######......" "........########........"
        ".........####..........." "........................"))
(defun glyph-ra-f3 ()
  (list "........................" "........#..............."
        ".........#.............." ".........####..........."
        "...........####........." ".........######........."
        "......###..............." ".....##................."
        ".....##................." ".....##................."
        ".....##................." ".....##................."
        ".....#......#####......." ".....#...####...##......"
        "....##..###......##....." "....#####........###...."
        "....####.........###...." ".....##..........###...."
        ".................##....." "................##......"
        "..............###......." "............####........"
        ".........####..........." "........................"))

;; り
(defun glyph-ri-f0 ()
  (list "........................" "......##................"
        ".....###................" ".....##....#####........"
        ".....##..#########......" "....###.###....###......"
        "....###.##......###....." "....#####........###...."
        "....#####........###...." "....####.........###...."
        "....####.........###...." "....###..........###...."
        "....###..........###...." "....###..........###...."
        "....###..........###...." "....###..........##....."
        ".....#..........###....." "................##......"
        "..............####......" ".............####......."
        ".........#######........" ".......#######.........."
        "........####............" "........................"))
(defun glyph-ri-f1 ()
  (list "........................" ".......#.......##......."
        ".......##......##......." ".......##......##......."
        ".......##......##......." ".......#.......##......."
        ".......#........#......." "......##........##......"
        "......##........##......" "......##........##......"
        "......##.#......##......" "......####.....##......."
        ".......##......##......." ".......##......##......."
        ".......##......##......." "...............##......."
        "..............##........" "..............##........"
        ".............##........." "............##.........."
        "............#..........." "..........##............"
        ".........##............." "........................"))
(defun glyph-ri-f2 ()
  (list "........................" ".......##......###......"
        ".......##.......##......" ".......##.......##......"
        "......##........##......" "......##........##......"
        "......##........##......" "......##........##......"
        "......##........##......" "......##.#......##......"
        "......##.#......##......" "......####......##......"
        "......###.......##......" "......###.......##......"
        "......###.......##......" "...............##......."
        "...............##......." "..............###......."
        "..............##........" ".............##........."
        "............###........." "..........###..........."
        "..........##............" "........................"))
(defun glyph-ri-f3 ()
  (list "........................" ".......##..............."
        ".......###.............." ".......###.............."
        ".......##...###........." ".......##..#..##........"
        ".......##.#....##......." ".......#.#.....##......."
        "......##.#.....##......." "......###......###......"
        "......###.......##......" "......###.......##......"
        "......###......###......" ".......##......###......"
        ".......##......##......." ".......##......##......."
        "...............##......." "..............##........"
        ".............##........." ".............##........."
        "............##.........." "...........##..........."
        ".........##............." "........................"))

;; る
(defun glyph-ru-f0 ()
  (list "........................" ".....#############......"
        "....###############....." ".....#######...###......"
        "..............###......." ".............###........"
        "...........####........." "..........####.........."
        ".........###............" ".......####.#####......."
        "......##############...." "....#########...#####..."
        "...######..........###.." ".#####..............##.."
        ".###................###." ".......#####........###."
        "......########......###." ".....###...####.....###."
        ".....###....###....###.." ".....###.....##...###..."
        "......###....#######...." ".......############....."
        "........#########......." "........................"))
(defun glyph-ru-f1 ()
  (list "........................" "............####........"
        "........########........" ".....#####...##........."
        "............##.........." "...........###.........."
        "...........##..........." "..........##............"
        ".........##............." "........##.............."
        ".......##..######......." "......#############....."
        "......####........##...." ".....###...........##..."
        "....###............###.." "...##..............###.."
        "..###..............###.." "........##.........##..."
        "......######......###..." "......##..###....###...."
        "......##....##..###....." "......############......"
        "........#######........." "........................"))
(defun glyph-ru-f2 ()
  (list "........................" ".............####......."
        ".....############......." ".....#######..###......."
        "......#......###........" "............###........."
        "...........###.........." "..........###..........."
        ".........###............" "........###............."
        "........##.#######......" ".......######...####...."
        "......####........##...." ".....###..........###..."
        "...####............##..." "..####.............###.."
        "....#..............##..." "........#####......##..."
        ".......###.###....###..." ".......##....##..###...."
        ".......##....######....." "........##########......"
        ".........#######........" "........................"))
(defun glyph-ru-f3 ()
  (list "........................" "..............##........"
        ".....#....#######......." ".....#######..###......."
        "......###....###........" "............###........."
        "...........###.........." "..........###..........."
        "..........##............" ".........##............."
        "........##.............." ".......##.########......"
        "......#####......##....." ".....####.........##...."
        "....###...........###..." "...###............###..."
        "...##.............###..." "..................###..."
        "........#####.....##...." ".......#....##...###...."
        ".......#.....######....." "........###.#####......."
        ".........######........." "........................"))

;; れ
(defun glyph-re-f0 ()
  (list "........................" "......#................."
        "......##................" "......##................"
        ".....###................" ".....###....######......"
        ".#######...#######......" ".#######..###...###....."
        ".....##.###......##....." ".....#####.......##....."
        ".....####.......###....." ".....###........###....."
        "....###.........##......" "...####.........##......"
        "...####.........##......" "..#####.........##......"
        ".###.##........###...##." ".##..##........###..###."
        ".....##.........##.###.." ".....##.........######.."
        ".....##..........####..." ".....##................."
        "......#................." "........................"))
(defun glyph-re-f1 ()
  (list "........................" "......##................"
        ".......#................" ".......#................"
        "......##................" "......##......###......."
        "......##....##..#......." "......###..##...##......"
        "...#####..##....#......." "..###.##.##.....#......."
        "......###.......#......." ".....####.......#......."
        ".....###.......##......." "....###........##......."
        "....###........##......." "...##.#........##......."
        "...#..#........##......." "..##..#........##....##."
        ".##...#........#######.." ".##...#.........####...."
        "......#................." "......#................."
        "......#................." "........................"))
(defun glyph-re-f2 ()
  (list "........................" "......##................"
        "......##................" "......##................"
        "......##................" "......##......##........"
        "......##....#####......." ".....####..##...##......"
        ".########.##....##......" ".####.##.##.....##......"
        "......####......##......" "......###.......##......"
        ".....###........##......" "....####........##......"
        "....####........##......" "...##.##........##......"
        "..##..##.......##......." ".###..##.......##....##."
        ".##...##.......###..###." "......##........######.."
        "......##........#####..." "......##................"
        "......##................" "........................"))
(defun glyph-re-f3 ()
  (list "........................" "........................"
        ".....##................." "......##................"
        "......##................" "......##................"
        "......##....#####......." "......##...#....#......."
        ".#.#####..#.....##......" "..##..#.##......##......"
        ".....####.......#......." ".....###.......##......."
        "....###........##......." "...##.#........##......."
        "...##.#........##......." "..##..#........##......."
        ".###..#........##......." ".###..#........##....##."
        "..#.###........######..." "....###.........####...."
        ".....##................." ".....##................."
        "........................" "........................"))

;; ろ
(defun glyph-ro-f0 ()
  (list "........................" "....###############....."
        "....###############....." ".....#######...####....."
        "..............####......" ".............###........"
        "............###........." "..........####.........."
        ".........####..........." "........###............."
        ".......#############...." ".....################..."
        "....######........####.." "..######............###."
        ".#####..............###." ".###................###."
        "....................###." "....................###."
        "...................###.." "..................####.."
        ".....######..########..." ".....###############...."
        ".......##########......." "........................"))
(defun glyph-ro-f1 ()
  (list "........................" "............###........."
        "........#######........." ".....#####..##.........."
        "............##.........." "...........##..........."
        "..........##............" "..........#............."
        ".........##............." "........##.............."
        ".......##..#######......" ".......#####....####...."
        "......####........###..." ".....###...........##..."
        "....###............##..." "...###.............##..."
        "...##..............##..." "..................##...."
        ".................###...." "................###....."
        "..............###......." "...........####........."
        "........####............" "........................"))
(defun glyph-ro-f2 ()
  (list "........................" ".............###........"
        "......###########......." ".....########.###......."
        "......#......###........" "............###........."
        "...........###.........." "..........###..........."
        ".........###............" ".........##....#........"
        "........##.########....." ".......######....###...."
        "......####........###..." ".....###...........###.."
        "....###............###.." "...###.............###.."
        "..###..............###.." "...................##..."
        "..................###..." "................####...."
        "..............####......" "..........#######......."
        "...........###.........." "........................"))
(defun glyph-ro-f3 ()
  (list "........................" ".....#.......###........"
        "......#...######........" "......######..###......."
        ".......##....###........" "............###........."
        "...........###.........." "..........###..........."
        "..........##............" ".........##............."
        "........##...####......." ".......##.#########....."
        "......#####.......##...." ".....####.........###..."
        "....###............##..." "...###.............##..."
        "...##.............###..." "...#..............###..."
        ".................###...." ".................##....."
        "...............###......" ".............####......."
        "..........####.........." "........................"))

;; わ
(defun glyph-wa-f0 ()
  (list "........................" "......##................"
        "......##................" "......##................"
        "......##................" ".....###................"
        ".########...#######....." "..######..###########..."
        "......##.####.....####.." "......#####........###.."
        "......####..........###." ".....###............###."
        "....####.............##." "....####............###."
        "...#####............###." "..###.##............###."
        ".###..##...........###.." ".##...##..........####.."
        "......##........#####..." "......##....########...."
        "......##....######......" "......##................"
        "......##................" "........................"))
(defun glyph-wa-f1 ()
  (list ".......#................" ".......##..............."
        ".......##..............." ".......#................"
        ".......#................" ".......#................"
        ".......#................" "......###....######....."
        "...######..####..####..." "..###.##.###.......###.."
        "......####..........##.." "......###............##."
        "......##.............##." ".....###.............##."
        ".....###.............##." "....####............##.."
        "...##.##............##.." "...##.##...........##..."
        "..##..##..........##...." ".##...##.........##....."
        "..#...##.......###......" "......##.....###........"
        "......##................" "........................"))
(defun glyph-wa-f2 ()
  (list "........................" ".......##..............."
        ".......##..............." ".......##..............."
        ".......##..............." ".......##..............."
        ".......###.............." "....######.....###......"
        "..#######...########...." "..##...##.####....###..."
        ".......#####........##.." "......####..........###."
        ".....####............##." ".....####............##."
        "....##.##............##." "...###.##...........###."
        "..###..##...........##.." "..##...##..........###.."
        ".##....##.........###..." "..#....##.......####...."
        ".......##.....#####....." ".......##......#........"
        ".......##..............." "........................"))
(defun glyph-wa-f3 ()
  (list "........................" "......##................"
        ".......##..............." ".......###.............."
        ".......##..............." ".......##..............."
        ".......##..............." ".#....####.............."
        ".########....######....." "..###..##..###....###..."
        ".......####........###.." "......####..........##.."
        ".....####...........###." ".....#.#............###."
        "....##.#............###." "...##..#............###."
        "..###..#...........###.." ".###...#...........##..."
        ".####..#..........###..." "..#..####........##....."
        ".....####......###......" "......##.....##........."
        "......##................" "........................"))

;; を
(defun glyph-wo-f0 ()
  (list "........................" "..........##............"
        ".........###............" ".........##.......#....."
        "..##################...." "..#################....."
        "....######.............." ".......##..............."
        "......###..............." ".....#########.........."
        "....###########...#####." "....####.....#########.."
        "...###......#######....." "..###.....######........"
        ".###.....######........." "..#....####..##........."
        ".......###...##........." "......###....##........."
        "......##......#........." "......###..............."
        "......####........###..." ".......###############.."
        ".........############..." "........................"))
(defun glyph-wo-f1 ()
  (list "........................" ".........##............."
        ".........##............." ".........##............."
        ".........######........." "..###########..........."
        "........##.............." ".......##..............."
        ".......##..............." "......##............#..."
        "......#.####......####.." ".....####.###...####...."
        "....###....##.###......." "...###.....####........."
        "..........###..........." ".........####..........."
        "........##.##..........." ".......##..##..........."
        ".......#...##..........." ".......#................"
        ".......##..............." "........###########....."
        ".........#########......" "........................"))
(defun glyph-wo-f2 ()
  (list "........................" "..........##............"
        "..........##............" "..........##............"
        ".........########......." "...##############......."
        "...########............." "........##.............."
        ".......##..............." "......#######.......#..."
        ".....#########....####.." ".....###....##..#####..."
        "....##......######......" "..###......#####........"
        "..##......####.........." "........###..#.........."
        "........##...#.........." ".......##....#.........."
        ".......##....#.........." ".......##..............."
        ".......##..............." ".......#############...."
        ".........###########...." "........................"))
(defun glyph-wo-f3 ()
  (list "........................" ".........###............"
        "..........##............" "..........##..##........"
        ".........##.####........" "...##...#######........."
        "....#######............." "........#.........#....."
        ".......#...........##..." "......##..........####.."
        ".....#######.....#####.." "....###....##..####....."
        "...###......####........" "..###......###.........."
        "..##......###..........." "........##..#..........."
        ".......##...#..........." ".......#...##..........."
        "......##....#..........." "......##................"
        ".......#.........#......" ".......#############...."
        ".........##########....." "........................"))

;; ん
(defun glyph-n-f0 ()
  (list "........................" "........##.............."
        "........##.............." ".......###.............."
        ".......###.............." ".......##..............."
        "......###..............." "......##................"
        "......##................" ".....###................"
        ".....##...##............" "....###.######.........."
        "....##########.........." "....####....###........."
        "...####.....###......##." "...###......###.....###."
        "...###......###.....###." "..###.......###.....###."
        "..###.......###....###.." ".###.........##....###.."
        ".###.........########..." ".##..........#######...."
        ".##............####....." "........................"))
(defun glyph-n-f1 ()
  (list "........................" ".........#.............."
        ".........##............." ".........##............."
        "........##.............." "........##.............."
        ".......##..............." ".......##..............."
        "......##................" "......##................"
        "......#................." ".....##................."
        ".....#..###...........#." "....##.###............#."
        "....####.#...........##." "...####..#...........#.."
        "...###...##.........##.." "..###....##........##..."
        "..##.....##.......##...." "..##......##.....##....."
        ".##........##..###......" ".##.........#####......."
        ".#......................" "........................"))
(defun glyph-n-f2 ()
  (list "........................" "..........#............."
        "..........##............" "..........##............"
        ".........###............" ".........##............."
        "........##.............." "........##.............."
        ".......##..............." ".......##..............."
        "......##................" "......##................"
        ".....##.####............" ".....########..........."
        "....####...###.........." "....###.....##.......##."
        "...###......##.......##." "...##.......##......###."
        "..###.......##......##.." "..##........###...###..."
        ".###.........#######...." ".##..........######....."
        "...............###......" "........................"))
(defun glyph-n-f3 ()
  (list "........................" "........#..............."
        ".........##............." ".........###............"
        ".........###............" ".........##............."
        "........##.............." "........##.............."
        ".......##..............." ".......#................"
        "......##................" "......#................."
        ".....##................." ".....#.####...........#."
        "....####..##..........#." "...###.....##........##."
        "...##......##........#.." "..###......##.......##.."
        "..##.......##......##..." ".###........##....##...."
        ".##.........#######....." ".##.........######......"
        "..............###......." "........................"))

(defparameter *glyphs*
  (list (list (glyph-a-f0) (glyph-a-f1) (glyph-a-f2) (glyph-a-f3))
        (list (glyph-i-f0) (glyph-i-f1) (glyph-i-f2) (glyph-i-f3))
        (list (glyph-u-f0) (glyph-u-f1) (glyph-u-f2) (glyph-u-f3))
        (list (glyph-e-f0) (glyph-e-f1) (glyph-e-f2) (glyph-e-f3))
        (list (glyph-o-f0) (glyph-o-f1) (glyph-o-f2) (glyph-o-f3))
        (list (glyph-ka-f0) (glyph-ka-f1) (glyph-ka-f2) (glyph-ka-f3))
        (list (glyph-ki-f0) (glyph-ki-f1) (glyph-ki-f2) (glyph-ki-f3))
        (list (glyph-ku-f0) (glyph-ku-f1) (glyph-ku-f2) (glyph-ku-f3))
        (list (glyph-ke-f0) (glyph-ke-f1) (glyph-ke-f2) (glyph-ke-f3))
        (list (glyph-ko-f0) (glyph-ko-f1) (glyph-ko-f2) (glyph-ko-f3))
        (list (glyph-sa-f0) (glyph-sa-f1) (glyph-sa-f2) (glyph-sa-f3))
        (list (glyph-shi-f0) (glyph-shi-f1) (glyph-shi-f2) (glyph-shi-f3))
        (list (glyph-su-f0) (glyph-su-f1) (glyph-su-f2) (glyph-su-f3))
        (list (glyph-se-f0) (glyph-se-f1) (glyph-se-f2) (glyph-se-f3))
        (list (glyph-so-f0) (glyph-so-f1) (glyph-so-f2) (glyph-so-f3))
        (list (glyph-ta-f0) (glyph-ta-f1) (glyph-ta-f2) (glyph-ta-f3))
        (list (glyph-chi-f0) (glyph-chi-f1) (glyph-chi-f2) (glyph-chi-f3))
        (list (glyph-tsu-f0) (glyph-tsu-f1) (glyph-tsu-f2) (glyph-tsu-f3))
        (list (glyph-te-f0) (glyph-te-f1) (glyph-te-f2) (glyph-te-f3))
        (list (glyph-to-f0) (glyph-to-f1) (glyph-to-f2) (glyph-to-f3))
        (list (glyph-na-f0) (glyph-na-f1) (glyph-na-f2) (glyph-na-f3))
        (list (glyph-ni-f0) (glyph-ni-f1) (glyph-ni-f2) (glyph-ni-f3))
        (list (glyph-nu-f0) (glyph-nu-f1) (glyph-nu-f2) (glyph-nu-f3))
        (list (glyph-ne-f0) (glyph-ne-f1) (glyph-ne-f2) (glyph-ne-f3))
        (list (glyph-no-f0) (glyph-no-f1) (glyph-no-f2) (glyph-no-f3))
        (list (glyph-ha-f0) (glyph-ha-f1) (glyph-ha-f2) (glyph-ha-f3))
        (list (glyph-hi-f0) (glyph-hi-f1) (glyph-hi-f2) (glyph-hi-f3))
        (list (glyph-fu-f0) (glyph-fu-f1) (glyph-fu-f2) (glyph-fu-f3))
        (list (glyph-he-f0) (glyph-he-f1) (glyph-he-f2) (glyph-he-f3))
        (list (glyph-ho-f0) (glyph-ho-f1) (glyph-ho-f2) (glyph-ho-f3))
        (list (glyph-ma-f0) (glyph-ma-f1) (glyph-ma-f2) (glyph-ma-f3))
        (list (glyph-mi-f0) (glyph-mi-f1) (glyph-mi-f2) (glyph-mi-f3))
        (list (glyph-mu-f0) (glyph-mu-f1) (glyph-mu-f2) (glyph-mu-f3))
        (list (glyph-me-f0) (glyph-me-f1) (glyph-me-f2) (glyph-me-f3))
        (list (glyph-mo-f0) (glyph-mo-f1) (glyph-mo-f2) (glyph-mo-f3))
        (list (glyph-ya-f0) (glyph-ya-f1) (glyph-ya-f2) (glyph-ya-f3))
        (list (glyph-yu-f0) (glyph-yu-f1) (glyph-yu-f2) (glyph-yu-f3))
        (list (glyph-yo-f0) (glyph-yo-f1) (glyph-yo-f2) (glyph-yo-f3))
        (list (glyph-ra-f0) (glyph-ra-f1) (glyph-ra-f2) (glyph-ra-f3))
        (list (glyph-ri-f0) (glyph-ri-f1) (glyph-ri-f2) (glyph-ri-f3))
        (list (glyph-ru-f0) (glyph-ru-f1) (glyph-ru-f2) (glyph-ru-f3))
        (list (glyph-re-f0) (glyph-re-f1) (glyph-re-f2) (glyph-re-f3))
        (list (glyph-ro-f0) (glyph-ro-f1) (glyph-ro-f2) (glyph-ro-f3))
        (list (glyph-wa-f0) (glyph-wa-f1) (glyph-wa-f2) (glyph-wa-f3))
        (list (glyph-wo-f0) (glyph-wo-f1) (glyph-wo-f2) (glyph-wo-f3))
        (list (glyph-n-f0) (glyph-n-f1) (glyph-n-f2) (glyph-n-f3))))

;; Convert one glyph (list of equal-length rows) into a flat list of 0.0 / 1.0,
;; row-major.  Size-agnostic: it reads the grid width off the row strings.
(defun glyph->list (rows)
  (let ((acc nil))
    (dolist (row rows)
      (dotimes (j (length row))
        (setq acc (cons (if (char= (char row j) #\#) 1.0 0.0) acc))))
    (reverse acc)))


---

# FILE: references/examples/browser/hiragana/recognize.lisp

;;;; recognize.lisp -- the browser build: the same net as infer.lisp, but
;;;; exported as a host-callable WASM function instead of reading stdin.
;;;;
;;;;   rontolisp recognize.lisp -o infer.wasm      (WASI Preview 1, WASM GC)
;;;;
;;;; Why an export and not a command module.  The weights are no longer baked
;;;; into the program: startup reads ~150k parameters out of weights.bin, one
;;;; byte at a time through WASI.  A command module would redo that on every
;;;; keystroke.  As an export, the page instantiates the module ONCE, runs
;;;; _start (which loads the weights into the module's globals), and then calls
;;;; recognize(...) per stroke -- the load happens once per page, and each
;;;; recognition is just the forward pass.
;;;;
;;;; The :s-expr parameter crosses as (ptr, len) into linear memory and is parsed
;;;; by the embedded reader; the :string result comes back the same way.  See
;;;; index.html for the ten lines of host glue (__ronto_alloc + memory), and
;;;; wasi-shim.js for the virtual filesystem that serves weights.bin.

(load "net.lisp")

(defparameter *net* (load-hiragana-net "weights.bin"))

(defun recognize (image)
  ;; IMAGE is the flattened 24x24 bitmap as a list of 576 numbers.  Returns the
  ;; same text infer.lisp prints: "pred  " then one "score" line per
  ;; class, which the page parses.
  (let* ((scores (classify *net* (linalg:from-list image)))
         (pred (linalg:argmax scores))
         (out (format nil "pred ~a ~a~%" pred (nth pred *labels*))))
    (dotimes (i *nclasses*)
      (setq out
       (format nil "~ascore ~a ~a~%" out (nth i *labels*) (aref scores i))))
    out))

(rontolisp:wasm-export 'recognize :params '(:s-expr) :returns :string)


---

# FILE: references/examples/browser/hiragana/regen-glyphs.sh

#!/usr/bin/env bash
# Regenerate the reference glyphs for the hiragana demo.
#
#   examples/browser/hiragana/regen-glyphs.sh
#
# Runs the offline Java glyph renderer (glyphgen/GlyphGen.java) and rewrites the
# generated artifacts in this directory:
#   prototypes.lisp        the trainer's reference glyphs
#   glyphs.js              GLYPHS/KANA/ORDER for index.html
#   samples/.txt   each template flattened to a stdin bitmap
#
# Run this ONLY when changing the font, resolution (GRID), or class set in
# GlyphGen.java -- the artifacts above are committed, and the normal build
# (gen.sh) consumes them as-is.  Requires a JDK with the configured font
# installed (macOS ships "Hiragino Maru Gothic ProN").
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
java "$here/glyphgen/GlyphGen.java" "$here"


---

# FILE: references/examples/browser/hiragana/samples/a.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/chi.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/e.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/fu.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/ha.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/he.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/hi.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/ho.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/i.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/ka.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/ke.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/ki.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/ko.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/ku.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/ma.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/me.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/mi.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/mo.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/mu.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/n.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/na.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/ne.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/ni.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/no.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/nu.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 1.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/o.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/ra.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/re.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/ri.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/ro.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/ru.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/sa.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/se.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/shi.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/so.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/su.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/ta.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/te.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/to.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/tsu.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/u.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/wa.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/wo.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/ya.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/yo.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/samples/yu.txt

(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 1.0 1.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)


---

# FILE: references/examples/browser/hiragana/tools/k49/README.md

# 実データ (Kuzushiji-49) の取得と前処理

デモの CNN は、**実際の手書きかな**と合成フォント字形の混合で学習します。その実データ側を
用意するのがこのツールです。学習そのものは rontolisp（`../../train.lisp`）で行うので、
ここでやるのは 2 つだけです。

1. Kuzushiji-49 の `.npz`（約 80MB）をダウンロードしてキャッシュする
2. 各画像を**ブラウザと同じ前処理**（インクの外接矩形で切り出し → 縦横比を保って 24x24 に
   中心化 → 0.35 で二値化）にかけ、rontolisp が `read-byte` で読める素朴なバイナリに書き出す

rontolisp は `.npz`（NumPy の zip）を読めないので、この 2 つだけを外部で済ませます。

## 必要なもの

- Python 3 + NumPy + Pillow（`pip install numpy pillow`）

## 使い方

```bash
python3 examples/browser/hiragana/tools/k49/prepare-k49.py
```

出力（`.gitignore` 済み。リポジトリにはコミットしません）:

| ファイル | 中身 |
| --- | --- |
| `tools/k49/data/*.npz` | ダウンロードした元データのキャッシュ |
| `../../data/k49-train.bin` | 学習用（既定: 1 クラス最大 800 枚 → 36,777 枚） |
| `../../data/k49-test.bin` | 評価用（既定: 1 クラス最大 100 枚 → 4,600 枚） |

主なオプション:

```bash
python3 prepare-k49.py --per-class-cap 0        # クラス上限なし（全 23 万枚。学習は長くなる）
                       --test-per-class-cap 200
```

## HKB1 フォーマット

`dataset.lisp` の `k49-load` が先頭から順に読むだけの形式です（シークは使いません）。

```
"HKB1"                    4 bytes
count                     u32 big-endian
grid                      u8  (24)
count 個のレコード:
  label                   u8  (0..45、デモのかな順)
  pixels                  grid*grid bytes (0 か 1、行優先)
```

## クラス

K49 の 49 クラスのうち、合成字形に対応がない **ゐ (wi) / ゑ (we) / 繰り返し記号 ゝ** を落とし、
残り 46 クラスをデモの五十音順（`*romaji*`）に振り直しています。

## ライセンス

Kuzushiji-49 (c) ROIS-DS 人文学オープンデータ共同利用センター (CODH),
[CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/) —



---

# FILE: references/examples/browser/hiragana/tools/k49/prepare-k49.py

#!/usr/bin/env python3
# prepare-k49.py -- turn the real-handwriting dataset Kuzushiji-49 into the tiny
# binary rontolisp can read, so the TRAINING itself happens in Lisp.
#
# The demo trains a CNN on a mix of synthetic multi-font glyphs (prototypes.lisp)
# and real handwritten kana.  rontolisp cannot read numpy's .npz (a zip of binary
# arrays), and the preprocessing (crop / centre / binarize) must match what the
# browser does to a drawn stroke to the pixel -- so this script does BOTH offline
# and writes a flat, sequential binary file that dataset.lisp reads with plain
# read-byte on every backend.  Nothing about the model lives here.
#
# Output format ("HKB1", one file per split, read strictly front-to-back):
#
#     magic  "HKB1"                     4 bytes
#     count  u32 big-endian             number of samples
#     grid   u8                         bitmap edge (24)
#     then COUNT records of 1 + grid*grid bytes:
#         label u8                      class index 0..45 (the demo's kana order)
#         pixels grid*grid bytes        0 or 1, row-major
#
# Class set: K49's 49 classes minus ゐ (wi) / ゑ (we) / the iteration mark ゝ,
# remapped onto the demo's 46 gojuon order (see LABELS below).
#
# Dataset: Kuzushiji-49, (c) ROIS-DS Center for Open Data in the Humanities
# (CODH), licensed CC BY-SA 4.0.  http://codh.rois.ac.jp/kmnist/  The .npz files
# are downloaded on first run and cached under --data-dir (NOT committed).
#
# Usage (from anywhere):
#   python3 examples/browser/hiragana/tools/k49/prepare-k49.py
#   python3 .../prepare-k49.py --per-class-cap 800 --test-per-class-cap 100

import argparse
import struct
import sys
import urllib.request
from pathlib import Path

import numpy as np
from PIL import Image

BASE_URL = "http://codh.rois.ac.jp/kmnist/dataset/k49/"
TRAIN_IMGS = "k49-train-imgs.npz"
TRAIN_LBLS = "k49-train-labels.npz"
TEST_IMGS = "k49-test-imgs.npz"
TEST_LBLS = "k49-test-labels.npz"

GRID = 24  # output bitmap edge (matches the browser / GlyphGen)
INK_BBOX = 0.3  # ink threshold for the bounding box (matches GlyphGen)
BINARIZE = 0.35  # cell on/off threshold (matches GlyphGen / index.html)

# The demo's 46 classes, in output-unit order (GlyphGen.KANA / *romaji*).
LABELS = [
    "a", "i", "u", "e", "o", "ka", "ki", "ku", "ke", "ko", "sa", "shi", "su",
    "se", "so", "ta", "chi", "tsu", "te", "to", "na", "ni", "nu", "ne", "no",
    "ha", "hi", "fu", "he", "ho", "ma", "mi", "mu", "me", "mo", "ya", "yu",
    "yo", "ra", "ri", "ru", "re", "ro", "wa", "wo", "n",
]

# K49's own class order (k49_classmap.csv) is the demo's a..wa for 0..43, then
# 44 ゐ, 45 ゑ, 46 を, 47 ん, 48 ゝ.  Map the shared classes onto the demo's
# indices and drop the three the synthetic set has no glyph for.
K49_TO_DEMO = {k: k for k in range(44)}
K49_TO_DEMO[46] = 44  # wo
K49_TO_DEMO[47] = 45  # n


def download(data_dir: Path, name: str) -> Path:
    path = data_dir / name
    if path.exists():
        return path
    data_dir.mkdir(parents=True, exist_ok=True)
    url = BASE_URL + name
    print(f"downloading {url}", file=sys.stderr)
    urllib.request.urlretrieve(url, path)
    return path


def load_npz(path: Path) -> np.ndarray:
    with np.load(path) as z:
        return z["arr_0"]


def to_bitmap(img: np.ndarray) -> np.ndarray:
    """One 28x28 uint8 image -> a GRID*GRID uint8 vector of 0/1.

    Mirrors index.html's toBitmap / GlyphGen.render: ink = pixel/255 (K49 is a
    white stroke on black, so high = ink), crop to the ink bounding box, scale
    it to fit a (GRID-2) box preserving aspect, centre it with a 1px margin,
    binarize.  The network therefore sees real handwriting in exactly the
    representation the browser will hand it."""
    ink = img.astype(np.float32) / 255.0
    mask = ink > INK_BBOX
    if not mask.any():
        return np.zeros(GRID * GRID, dtype=np.uint8)
    ys, xs = np.where(mask)
    crop = ink[ys.min() : ys.max() + 1, xs.min() : xs.max() + 1]
    bh, bw = crop.shape
    scale = (GRID - 2) / max(bw, bh)
    new_w = max(1, int(round(bw * scale)))
    new_h = max(1, int(round(bh * scale)))
    resized = np.asarray(
        Image.fromarray((crop * 255).astype(np.uint8)).resize(
            (new_w, new_h), Image.BILINEAR
        ),
        dtype=np.float32,
    ) / 255.0
    canvas = np.zeros((GRID, GRID), dtype=np.float32)
    oy = (GRID - new_h) // 2
    ox = (GRID - new_w) // 2
    canvas[oy : oy + new_h, ox : ox + new_w] = resized
    return (canvas > BINARIZE).astype(np.uint8).reshape(-1)


def build_split(imgs, lbls, per_class_cap, rng):
    """Keep the 46 shared classes, cap each class (K49 is imbalanced, and the
    cap is also what bounds the Lisp trainer's runtime), shuffle, preprocess."""
    demo = np.full(len(lbls), -1, dtype=np.int64)
    for k, d in K49_TO_DEMO.items():
        demo[lbls == k] = d
    idx = np.flatnonzero(demo >= 0)
    if per_class_cap > 0:
        keep = []
        for c in range(len(LABELS)):
            ci = idx[demo[idx] == c]
            if len(ci) > per_class_cap:
                ci = rng.choice(ci, per_class_cap, replace=False)
            keep.append(ci)
        idx = np.concatenate(keep)
    rng.shuffle(idx)
    print(f"preprocessing {len(idx)} images to {GRID}x{GRID} bitmaps...", file=sys.stderr)
    x = np.empty((len(idx), GRID * GRID), dtype=np.uint8)
    for i, j in enumerate(idx):
        x[i] = to_bitmap(imgs[j])
    return x, demo[idx].astype(np.uint8)


def write_hkb1(path: Path, x: np.ndarray, y: np.ndarray) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    with open(path, "wb") as f:
        f.write(b"HKB1")
        f.write(struct.pack(">I", len(y)))
        f.write(bytes([GRID]))
        for i in range(len(y)):
            f.write(bytes([int(y[i])]))
            f.write(x[i].tobytes())
    print(f"wrote {path} ({len(y)} samples, {path.stat().st_size} bytes)", file=sys.stderr)


def main():
    here = Path(__file__).resolve().parent
    demo_root = here.parent.parent  # examples/browser/hiragana
    ap = argparse.ArgumentParser(description="Preprocess Kuzushiji-49 into HKB1 bitmaps for the Lisp trainer")
    ap.add_argument("--data-dir", type=Path, default=here / "data",
                    help="cache directory for the downloaded .npz files")
    ap.add_argument("--out-dir", type=Path, default=demo_root / "data",
                    help="where the .bin files land (read by dataset.lisp)")
    ap.add_argument("--per-class-cap", type=int, default=800,
                    help="max TRAIN images per class (0 = all); the Lisp trainer's runtime scales with this")
    ap.add_argument("--test-per-class-cap", type=int, default=100,
                    help="max TEST images per class (0 = all)")
    ap.add_argument("--seed", type=int, default=0)
    args = ap.parse_args()

    rng = np.random.default_rng(args.seed)
    xtr, ytr = build_split(
        load_npz(download(args.data_dir, TRAIN_IMGS)),
        load_npz(download(args.data_dir, TRAIN_LBLS)),
        args.per_class_cap, rng)
    write_hkb1(args.out_dir / "k49-train.bin", xtr, ytr)

    xte, yte = build_split(
        load_npz(download(args.data_dir, TEST_IMGS)),
        load_npz(download(args.data_dir, TEST_LBLS)),
        args.test_per_class_cap, rng)
    write_hkb1(args.out_dir / "k49-test.bin", xte, yte)


if __name__ == "__main__":
    main()


---

# FILE: references/examples/browser/hiragana/train.lisp

;;;; train.lisp -- the offline trainer.  Runs on the interpreter or (much
;;;; faster) compiled to the JVM; see gen.sh, which is what you should run.
;;;;
;;;; It trains the ch07 SimpleConvNet (net.lisp) with Adam on real handwritten
;;;; kana (Kuzushiji-49) mixed with augmented multi-font synthetic glyphs
;;;; (dataset.lisp), reports accuracy on a held-out K49 test split AND on the
;;;; clean reference glyphs, and writes the learned parameters to weights.bin
;;;; (RLW1).  The inference programs read that file at startup -- nothing is
;;;; baked into them, which is what let the model grow past the old 12.5k-weight
;;;; limit (see net.lisp).
;;;;
;;;; The convolution is the expensive part; im2col turns it into linalg:matmul,
;;;; so --simd pays for itself here (gen.sh passes it).

(load "dataset.lisp")
(load "../../deep-learning-from-scratch/common/trainer.lisp")

(defparameter *k49-train-limit* 40000) ; real samples used (the file holds 36777)
(defparameter *k49-test-limit* 4600)   ; held-out real samples (the whole split)
(defparameter *epochs* 12)
(defparameter *batch-size* 64)
(defparameter *lr* 0.001) ; Adam

(linalg:seed 42)

(format t ";; loading K49 (~a train / ~a test)~%" *k49-train-limit*
        *k49-test-limit*)
(defparameter *real* (k49-load "data/k49-train.bin" *k49-train-limit*))
(defparameter *test* (k49-load "data/k49-test.bin" *k49-test-limit*))

(format t ";; augmenting the synthetic glyphs~%")
(defparameter *synth* (samples->batch (synthetic-samples)))

(defparameter *train* (shuffle-batch (concat-batches *synth* *real*)))

(format t ";; dataset: ~a synthetic + ~a real = ~a samples~%"
        (car (linalg:shape (first *synth*))) (car (linalg:shape (first *real*)))
        (car (linalg:shape (first *train*))))

(defparameter *net* (make-hiragana-net))

(time
  (train *net* (scn-params *net*) (first *train*) (second *train*)
         (first *test*) (second *test*)
         :epochs *epochs*
         :mini-batch-size *batch-size*
         :optimizer (make-instance 'adam :lr *lr*)
         :eval-limit 2000))

;;; The clean reference glyphs (the display font, one per class) are what the
;;; page shows and what a user tries to copy, so they get their own score:
;;; a model can be good on K49 and still fumble the letterforms the page
;;; advertises.
(defun reference-accuracy (net)
  (let ((correct 0) (class 0))
    (dolist (variants *glyphs*)
      (let ((scores (classify net (glyph->image (first variants)))))
        (when (= (linalg:argmax scores) class) (setq correct (+ correct 1))))
      (setq class (+ class 1)))
    correct))

(format t ";; reference-glyph accuracy ~a/~a~%" (reference-accuracy *net*)
        *nclasses*)

(save-hiragana-net *net* "weights.bin")
(format t ";; wrote weights.bin~%")


---

# FILE: references/examples/browser/hiragana/wasi-shim.js

// wasi-shim.js
//
// A tiny, dependency-free WASI Preview 1 shim, just large enough to run a
// program compiled by rontolisp (`rontolisp prog.lisp -o prog.wasm`) in a
// browser. rontolisp's WASM output imports exactly nine functions from the
// "wasi_snapshot_preview1" module:
//
//   fd_write, fd_read, path_open, fd_close, fd_readdir,
//   random_get, clock_time_get, environ_sizes_get, environ_get
//
// A module built with --optimize imports only the ones it actually reaches, so
// the shim provides all nine and lets the link pick.
//
// This shim implements them over plain JavaScript:
//   - stdout/stderr (fd 1/2) are captured into strings instead of a real tty
//   - stdin (fd 0) is fed from a string you provide
//   - files (path_open) are served from an in-memory `files` map: a read-only
//     virtual filesystem, so a program that reads a data file (the hiragana
//     demo opens its weights.bin) works in the browser with the bytes fetched
//     over HTTP. rontolisp resolves every path against the first preopened
//     directory (fd 3), so the map is keyed by the plain relative path.
//   - randomness uses crypto.getRandomValues, the clock uses Date.now()
//   - environment variables come from the `env` option
//
// It is intentionally minimal and easy to read; it is NOT a complete WASI
// implementation (no directories, no writing, no seek). For a fuller one, use a
// package such as `@bjorn3/browser_wasi_shim`.
//
// This copy has diverged from examples/browser/wasm-browser/wasi-shim.js, which
// keeps the "files are not supported" stub, by exactly that virtual filesystem.

const WASI_ESUCCESS = 0;
const WASI_EBADF = 8; // bad file descriptor
const WASI_ENOENT = 44; // no such file or directory
const WASI_ENOSYS = 52; // function not supported

/**
 * Create a WASI Preview 1 import object plus helpers for one run of a module.
 *
 * @param {Object}  [opts]
 * @param {string}  [opts.stdin]  text delivered to the program's stdin (fd 0)
 * @param {Object}  [opts.env]    environment variables, e.g. { NAME: "Ada" }
 * @param {Object}  [opts.files]  virtual read-only files, path -> Uint8Array
 *                                (e.g. { "weights.bin": bytes })
 * @returns {{ imports: object, setMemory: (m: WebAssembly.Memory) => void,
 *             getStdout: () => string, getStderr: () => string }}
 */
export function createWasi({ stdin = "", env = {}, files = {} } = {}) {
  const encoder = new TextEncoder();
  const decoder = new TextDecoder();

  let memory = null; // set after instantiation via setMemory()
  let stdoutText = "";
  let stderrText = "";

  const stdinBytes = encoder.encode(stdin);
  let stdinPos = 0;

  // Open virtual files, keyed by the fd we hand back from path_open. fds 0-2 are
  // the standard streams and 3 is the (notional) preopened directory, so real
  // files start above them.
  const openFiles = new Map();
  let nextFd = 4;

  // environ entries are "KEY=VALUE\0" byte arrays (WASI layout).
  const envEntries = Object.entries(env).map(([k, v]) =>
    encoder.encode(`${k}=${v}\0`),
  );

  const view = () => new DataView(memory.buffer);
  const bytes = () => new Uint8Array(memory.buffer);

  const imports = {
    // fd_write(fd, iovs, iovs_len, nwritten) -> errno
    // Concatenate the iovec slices and route fd 1 -> stdout, fd 2 -> stderr.
    fd_write(fd, iovs, iovsLen, nwritten) {
      let written = 0;
      let chunk = "";
      for (let i = 0; i < iovsLen; i++) {
        const base = iovs + i * 8;
        const ptr = view().getUint32(base, true);
        const len = view().getUint32(base + 4, true);
        chunk += decoder.decode(new Uint8Array(memory.buffer, ptr, len));
        written += len;
      }
      if (fd === 1) stdoutText += chunk;
      else if (fd === 2) stderrText += chunk;
      view().setUint32(nwritten, written, true);
      return WASI_ESUCCESS;
    },

    // fd_read(fd, iovs, iovs_len, nread) -> errno
    // fd 0 reads the provided stdin string; any other fd is an open virtual
    // file, served sequentially from its cursor. EOF reads zero bytes.
    fd_read(fd, iovs, iovsLen, nread) {
      let src, pos;
      if (fd === 0) {
        src = stdinBytes;
        pos = stdinPos;
      } else {
        const f = openFiles.get(fd);
        if (!f) return WASI_EBADF;
        src = f.bytes;
        pos = f.pos;
      }
      let total = 0;
      const mem = bytes();
      for (let i = 0; i < iovsLen; i++) {
        const base = iovs + i * 8;
        const ptr = view().getUint32(base, true);
        const len = view().getUint32(base + 4, true);
        let j = 0;
        for (; j < len && pos < src.length; j++) {
          mem[ptr + j] = src[pos++];
        }
        total += j;
        if (j < len) break; // ran out of input
      }
      if (fd === 0) stdinPos = pos;
      else openFiles.get(fd).pos = pos;
      view().setUint32(nread, total, true);
      return WASI_ESUCCESS;
    },

    // path_open(dirfd, dirflags, path_ptr, path_len, oflags, rights_base,
    //           rights_inheriting, fdflags, fd_out) -> errno
    // Only reading is supported: look the path up in `files` and hand back a
    // cursor. (The rights arguments are i64 and arrive as BigInt; unused.)
    path_open(_dirfd, _dirflags, pathPtr, pathLen, _oflags, _rights, _inherit, _fdflags, fdOut) {
      const path = decoder.decode(new Uint8Array(memory.buffer, pathPtr, pathLen));
      const data = files[path] ?? files[path.replace(/^\.\//, "")];
      if (!data) return WASI_ENOENT;
      const fd = nextFd++;
      openFiles.set(fd, { bytes: new Uint8Array(data), pos: 0 });
      view().setUint32(fdOut, fd, true);
      return WASI_ESUCCESS;
    },

    fd_close(fd) {
      openFiles.delete(fd);
      return WASI_ESUCCESS;
    },

    // fd_readdir(fd, buf, buf_len, cookie, bufused_out) -> errno
    // The virtual filesystem is a flat path -> bytes map with no directories, so
    // a program that lists one gets "not supported" rather than an empty listing
    // it would read as "no files".
    fd_readdir() {
      return WASI_ENOSYS;
    },

    // random_get(ptr, len) -> errno
    random_get(ptr, len) {
      crypto.getRandomValues(new Uint8Array(memory.buffer, ptr, len));
      return WASI_ESUCCESS;
    },

    // clock_time_get(id, precision, out) -> errno ; out is an i64 of nanoseconds
    clock_time_get(_id, _precision, out) {
      const nanos = BigInt(Date.now()) * 1_000_000n;
      view().setBigUint64(out, nanos, true);
      return WASI_ESUCCESS;
    },

    // environ_sizes_get(count_out, bufsize_out) -> errno
    environ_sizes_get(countOut, bufsizeOut) {
      const bufSize = envEntries.reduce((sum, e) => sum + e.length, 0);
      view().setUint32(countOut, envEntries.length, true);
      view().setUint32(bufsizeOut, bufSize, true);
      return WASI_ESUCCESS;
    },

    // environ_get(ptrs_out, buf_out) -> errno
    environ_get(ptrsOut, bufOut) {
      const mem = bytes();
      let bufPtr = bufOut;
      for (let i = 0; i < envEntries.length; i++) {
        view().setUint32(ptrsOut + i * 4, bufPtr, true);
        mem.set(envEntries[i], bufPtr);
        bufPtr += envEntries[i].length;
      }
      return WASI_ESUCCESS;
    },

    // Programs that exit explicitly would call this. rontolisp's output never
    // imports proc_exit, but we provide it so the shim also works with modules
    // that do (a thrown sentinel ends _start cleanly).
    proc_exit(code) {
      throw new WasiExit(code);
    },
  };

  return {
    imports: { wasi_snapshot_preview1: imports },
    setMemory: (m) => {
      memory = m;
    },
    getStdout: () => stdoutText,
    getStderr: () => stderrText,
  };
}

/** Thrown by proc_exit to unwind out of _start. */
export class WasiExit extends Error {
  constructor(code) {
    super(`WASI exit ${code}`);
    this.code = code;
  }
}

/**
 * Instantiate and run an already-loaded rontolisp-compiled `.wasm` command
 * module (a `BufferSource`: `ArrayBuffer` or typed array), returning whatever
 * it wrote to stdout/stderr. Use this when the bytes are already in memory —
 * e.g. compiled in the browser by the playground's `rontoCompileWasm`, with no
 * `.wasm` file to fetch.
 *
 * @param {BufferSource} wasmBytes  the module bytes
 * @param {Object} [opts]           same options as createWasi()
 * @returns {Promise<{ stdout: string, stderr: string, exitCode: number }>}
 */
export async function runWasmModule(wasmBytes, opts = {}) {
  const wasi = createWasi(opts);
  const { instance } = await WebAssembly.instantiate(wasmBytes, wasi.imports);

  wasi.setMemory(instance.exports.memory);

  let exitCode = 0;
  try {
    instance.exports._start();
  } catch (e) {
    if (e instanceof WasiExit) exitCode = e.code;
    else throw e;
  }
  return {
    stdout: wasi.getStdout(),
    stderr: wasi.getStderr(),
    exitCode,
  };
}

/**
 * Instantiate a module and run its top level ONCE, keeping the instance alive so
 * the host can go on calling its `rontolisp:wasm-export`ed functions against the
 * state that top level built. The hiragana demo needs this: `_start` reads
 * ~150k weights out of the virtual `weights.bin`, and every later stroke is just
 * a `recognize(...)` call on the same instance.
 *
 * @param {BufferSource} wasmBytes  the module bytes
 * @param {Object} [opts]           same options as createWasi()
 * @returns {Promise<{ exports: object, getStdout: () => string,
 *                     getStderr: () => string }>}
 */
export async function instantiateWasm(wasmBytes, opts = {}) {
  const wasi = createWasi(opts);
  const { instance } = await WebAssembly.instantiate(wasmBytes, wasi.imports);
  wasi.setMemory(instance.exports.memory);
  // A WASI command runs its top level in _start (a --no-wasi reactor exports
  // _initialize instead); either way it runs once, here.
  if (instance.exports._start) instance.exports._start();
  else if (instance.exports._initialize) instance.exports._initialize();
  return {
    exports: instance.exports,
    getStdout: wasi.getStdout,
    getStderr: wasi.getStderr,
  };
}

/**
 * Fetch, instantiate and run a rontolisp-compiled `.wasm` command module,
 * returning whatever it wrote to stdout/stderr.
 *
 * @param {string} url            URL of the .wasm file
 * @param {Object} [opts]         same options as createWasi()
 * @returns {Promise<{ stdout: string, stderr: string, exitCode: number }>}
 */
export async function runWasm(url, opts = {}) {
  const response = await fetch(url);
  if (!response.ok) {
    throw new Error(`failed to fetch ${url}: ${response.status}`);
  }
  // instantiateStreaming needs the correct application/wasm MIME type; fall
  // back to ArrayBuffer instantiation when the server doesn't send it.
  const wasmBytes = await response.arrayBuffer();
  return runWasmModule(wasmBytes, opts);
}


---

# FILE: references/examples/browser/minesweeper/README.md

# Minesweeper (one Lisp rulebook, two front-ends)

A complete Minesweeper whose rules are written once in rontolisp and played two
ways: in the **browser** (the rules compiled ahead of time to a WebAssembly
reactor) and on the **desktop** (the same rules run on the interpreter behind a
Java/Swing window). All the interesting work -- the flood-fill reveal and
win/lose detection -- happens in Lisp; only the drawing differs between the two.

| File | Role |
| --- | --- |
| [`minesweeper-core.lisp`](minesweeper-core.lisp) | The shared rules: a pure state machine, no rendering (see below) |
| [`minesweeper-wasm.lisp`](minesweeper-wasm.lisp) | Browser front-end: loads the core, adds HTML rendering + WASM exports |
| [`minesweeper.html`](minesweeper.html) | The page: grid, mouse/touch input, mine placement, timer |
| [`minesweeper.wasm`](https://github.com/making/rontolisp/blob/develop/examples/browser/minesweeper/minesweeper.wasm) | Prebuilt module (regenerate with `build.sh`) |
| [`build.sh`](build.sh) | Recompile `minesweeper-wasm.lisp` to `minesweeper.wasm` |
| [`minesweeper-swing.lisp`](minesweeper-swing.lisp) | Desktop front-end: loads the core, paints a Swing grid |
| [`minesweeper-core-test.lisp`](minesweeper-core-test.lisp) | The rules, checked with [rove](https://github.com/making/rontolisp/blob/develop/examples/browser/doc/en/guides/testing.md) -- neither front-end can be run head-less, the core can |

The two front-ends share `minesweeper-core.lisp` verbatim -- the browser build
inlines it at compile time (a top-level literal `load`), and the Swing build
loads it at run time. Swapping the rendering layer is all it takes to move the
same game between WebAssembly and the JVM.

Because the core touches neither the screen nor entropy, it is also the part
that can be TESTED, and `minesweeper-core-test.lisp` does — the flood fill, the
win and loss transitions, flagging, and the placement rule that keeps the first
click safe. rove is vendored in this repository, so its three directories go on
`--system-path`, and the run's exit code is the verdict:

```bash
# from the repo root, after ./mvnw package
SP=src/test/resources/rove:src/test/resources/dissect:src/test/resources/cl-ppcre
java -jar target/rontolisp-0.1.0-SNAPSHOT-exec.jar \
  examples/browser/minesweeper/minesweeper-core-test.lisp --system-path $SP
```

## Play it on the desktop (Java / Swing)

```bash
# from the repo root, after ./mvnw package
java -jar target/rontolisp-0.1.0-SNAPSHOT-exec.jar examples/browser/minesweeper/minesweeper-swing.lisp
```

Left-click opens a cell, right-click flags it, and any click after the game ends
starts a fresh board. The Swing front-end runs on the JVM -- interpret it, or
compile it to a `.class` (`-o Minesweeper.class`; WASM cannot lower a Java
object) -- and needs a display. Unlike the
entropy-free browser reactor, this build has `random`, so it lays
its own mines -- keeping them off the first click. The rendering layer reuses the
[`../../jvm/swing.lisp`](../../jvm/swing.lisp) `swing` package (spliced in with
`(require :swing "../../jvm/swing.lisp")`), whose clickable, text-capable label grid
(`swing:label-grid-window`) was built for this game.

## Play it in the browser (WebAssembly)

```bash
./build.sh                      # (optional) rebuild minesweeper.wasm
jwebserver -p 8000              # or: python3 -m http.server 8000
# open http://localhost:8000/minesweeper.html
```

It must be served over `http://` (a `file://` page cannot `fetch` the `.wasm`)
and needs a browser with WebAssembly GC: Chrome 119+, Firefox 120+, or
Safari 18.2+.

Left-click opens a cell, right-click (or long-press on touch) flags it, and the
smiley starts a new game.

## How it works

The module is built with `--no-wasi`, which produces a **reactor**: it has no
WASI imports (so `WebAssembly.instantiate(bytes, {})` needs no import object) and
exposes an `_initialize` export the page runs once after instantiation. Because
a reactor has no entropy source, the page cannot ask Lisp to roll the mines; but
it does not reimplement the placement rule either. Instead the page only shuffles
a random ordering of the cell indices (its sole job) and hands it to the shared
Lisp `place-mines`, which applies the first-click-safe placement rule -- the same
`place-mines` the Swing front-end calls with an interpreter-shuffled ordering. So
entropy stays host-side while the rule stays in Lisp, and the two front-ends
place mines identically.

The game is a **pure state machine**. The state is a nested list of integers
`(status w h mines revealed flags)`; the page treats it as an opaque string that
round-trips through the WebAssembly `:s-expr` ABI, so the JavaScript never parses
Lisp. Each interaction is one export call:

| Export | Signature | Purpose |
| --- | --- | --- |
| `place-mines` | `(:int :int :int :int :s-expr) -> :s-expr` | The shared first-click-safe placement rule: pick mines from a host-supplied random ordering |
| `new-game` | `(:int :int :s-expr) -> :s-expr` | Build a fresh state from width, height, and a mine bit-list |
| `reveal` | `(:s-expr :int) -> :s-expr` | Open a cell; flood-fill blanks; detect win/loss |
| `toggle-flag` | `(:s-expr :int) -> :s-expr` | Flag / unflag a covered cell |
| `render` | `(:s-expr) -> :string` | The board as a run of `
` elements |
| `game-status` | `(:s-expr) -> :int` | 0 playing, 1 won, 2 lost |
| `mines-remaining` | `(:s-expr) -> :int` | Mines minus flags placed |

The `:string` / `:s-expr` boundary is a `(ptr, len)` into the module's linear
memory. The page writes UTF-8 via the exported `__ronto_alloc` bump allocator,
passes the pointer and length, and reads the returned `(ptr, len)` back out --
the same pattern as [`../rainbow/rainbow.html`](../rainbow/rainbow.html). See the top of
`minesweeper-core.lisp` for the state layout and the `CLAUDE.md` notes on
`rontolisp:wasm-export` and `--no-wasi` for the ABI details.


---

# FILE: references/examples/browser/minesweeper/build.sh

#!/usr/bin/env bash
# Recompile minesweeper-wasm.lisp to a browser-loadable WebAssembly reactor.
# The --no-wasi flag drops all WASI imports and exports _initialize, so the
# module instantiates with an empty import object and runs with just
# WebAssembly GC support -- no shim, no server-side runtime. --optimize runs the
# tree-shaker so only the reachable functions ship, shrinking the .wasm.
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling minesweeper-wasm.lisp -> minesweeper.wasm"
java -jar "$jar" "$here/minesweeper-wasm.lisp" -o "$here/minesweeper.wasm" --no-wasi --optimize

echo "done. Serve this directory over http, e.g.:"
echo "  jwebserver -p 8000 --directory \"$here\""
echo "then open http://localhost:8000/minesweeper.html"


---

# FILE: references/examples/browser/minesweeper/minesweeper-core-test.lisp

;;;; minesweeper-core-test.lisp -- the rules, checked with rove.
;;;;
;;;; minesweeper-core.lisp is a pure state machine: every action takes the
;;;; current state and returns the next one, and nothing in it touches the
;;;; screen, entropy or the host. That is exactly what makes it testable
;;;; head-less -- the two front-ends beside it (the browser/WASM one and the
;;;; Swing one) cannot be, so this file is where the rules are pinned.
;;;;
;;;; rove is loaded with asdf, so pass the directories holding its .asd files
;;;; (rove, dissect and cl-ppcre, all vendored in this repository) with
;;;; --system-path; outside this repository (ql:quickload "rove") fetches the
;;;; same sources. See the Testing guide: doc/en/guides/testing.md
;;;;
;;;; Run:
;;;;   SP=src/test/resources/rove:src/test/resources/dissect:src/test/resources/cl-ppcre
;;;;   rontolisp minesweeper-core-test.lisp --system-path $SP
;;;;   rontolisp minesweeper-core-test.lisp --system-path $SP -o Tests.class && java Tests
;;;;   rontolisp minesweeper-core-test.lisp --system-path $SP -o tests.wasm --optimize && \
;;;;     wasmtime run -W gc -W exceptions=y tests.wasm

(asdf:load-system :rove)
(use-package :rove)
;; rove colors its report for a terminal; a checked pipeline wants plain text.
(setf *enable-colors* nil)

(load "minesweeper-core.lisp")

;;; A 3x3 board whose only mine is the top-left corner:
;;;
;;;   * 1 .        index 0 1 2
;;;   1 1 .              3 4 5
;;;   . . .              6 7 8
;;;
;;; Cells 2, 5, 6, 7 and 8 have no mine next to them, so revealing any of them
;;; floods over the whole board.
(defun corner-mine-board () (list 1 0 0 0 0 0 0 0 0))

;;; A 4x4 board mined at both diagonal corners, where no single click wins.
(defun two-mine-board () (list 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1))

(deftest geometry
  (testing "a corner has three neighbours, an edge five, the middle eight"
    (ok (= (length (neighbors 0 3 3)) 3))
    (ok (= (length (neighbors 1 3 3)) 5))
    (ok (= (length (neighbors 4 3 3)) 8)))
  (testing "the count is of MINED neighbours"
    (let ((mines (corner-mine-board)))
      (ok (= (adjacent-count mines 1 3 3) 1))
      (ok (= (adjacent-count mines 4 3 3) 1))
      (ok (= (adjacent-count mines 8 3 3) 0)))))

(deftest a-fresh-board
  (let ((state (new-game 3 3 (corner-mine-board))))
    (ok (= (game-status state) 0))
    (testing "nothing is uncovered and nothing is flagged yet"
      (ok (= (count-ones (st-revealed state)) 0))
      (ok (= (mines-remaining state) 1)))))

(deftest revealing-a-blank-cell-floods-and-can-win
  ;; Cell 8 has no mine beside it, so the flood runs until it meets the
  ;; numbered cells around the mine -- which here is every remaining cell, so
  ;; the one click also wins the game.
  (let ((state (reveal (new-game 3 3 (corner-mine-board)) 8)))
    (ok (= (count-ones (st-revealed state)) 8))
    (ok (= (nth 0 (st-revealed state)) 0) "the mine itself stays covered")
    (ok (= (game-status state) 1) "every safe cell uncovered wins")))

(deftest revealing-a-numbered-cell-stops-there
  ;; Cell 5 on the 4x4 board touches the corner mine, so it uncovers alone and
  ;; the game is still on.
  (let ((state (reveal (new-game 4 4 (two-mine-board)) 5)))
    (ok (= (count-ones (st-revealed state)) 1))
    (ok (= (game-status state) 0))))

(deftest stepping-on-a-mine-loses
  (let ((state (reveal (new-game 3 3 (corner-mine-board)) 0)))
    (ok (= (game-status state) 2))
    (ok (= (nth 0 (st-revealed state)) 1) "the mine you stepped on is shown")
    (testing "a finished game ignores further moves"
      (reveal state 8)
      (ok (= (count-ones (st-revealed state)) 1)))))

(deftest flagging
  (let ((state (new-game 3 3 (corner-mine-board))))
    (toggle-flag state 0)
    (ok (= (mines-remaining state) 0) "a flag counts against the mine counter")
    (testing "flagging is a toggle"
      (toggle-flag state 0)
      (ok (= (mines-remaining state) 1)))
    (testing "a flagged cell is protected from a click"
      (toggle-flag state 8)
      (reveal state 8)
      (ok (= (count-ones (st-revealed state)) 0)))))

(deftest the-first-click-is-never-a-mine
  ;; The host supplies the random ORDER; the rule that keeps the opening move
  ;; safe lives here. With the identity order every mine would land on the low
  ;; indices, which is precisely where the safe cell and its neighbours are.
  (let* ((order (list 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15))
         (mines (place-mines 4 4 3 5 order)))
    (ok (= (count-ones mines) 3) "exactly as many mines as asked for")
    (ok (= (nth 5 mines) 0) "not the cell that was clicked")
    (testing "and none of its neighbours either"
      (let ((safe t))
        (dolist (n (neighbors 5 4 4))
          (when (= (nth n mines) 1) (setq safe nil)))
        (ok safe)))))

;;; Loading this file runs its suite (rove's file-driven entry point), and the
;;; exit code is the verdict.
(uiop:quit (if (run-suite *package*) 0 1))


---

# FILE: references/examples/browser/minesweeper/minesweeper-core.lisp

;;;; minesweeper-core.lisp -- Minesweeper rules, with no rendering.
;;;;
;;;; This is the shared game logic behind BOTH front-ends: the browser/WASM one
;;;; (minesweeper-wasm.lisp, which adds HTML rendering and host-callable exports)
;;;; and the desktop/Swing one (minesweeper-swing.lisp, which paints a Swing grid).
;;;; Only the drawing differs between them; the rules below are identical.
;;;;
;;;; The game is a pure state machine: every action takes the current state and
;;;; returns the next one. Nothing here touches the screen, entropy, or the host.
;;;;
;;;; State layout (a list):
;;;;   (status w h mines revealed flags)
;;;;     status   -- 0 playing, 1 won, 2 lost
;;;;     w h      -- board width / height (columns / rows)
;;;;     mines    -- w*h list of 0/1, the hidden truth (1 = mine)
;;;;     revealed -- w*h list of 0/1 (1 = uncovered)
;;;;     flags    -- w*h list of 0/1 (1 = flagged)
;;;;   A cell index is  i = r*w + c  (row-major).
;;;;
;;;; The mine layout is supplied by the caller (the browser uses JavaScript's RNG;
;;;; the Swing front-end uses the interpreter's `random`), which lets each host
;;;; keep the mines away from the very first click.
;;;;
;;;; This uses cons/list operations, so it needs a GC backend
;;;; (interpreter / JVM / WASM-GC); it is NOT in the --no-gc subset.

;;; --- small list helpers ------------------------------------------------------

;;; Destructively set element I of LST to VAL, returning LST. The lists we mutate
;;; are freshly built for each game (or parsed from the :s-expr argument on every
;;; host call), so this never leaks state between actions.
(defun set-nth (lst i val)
  (rplaca (nthcdr i lst) val)
  lst)

;;; A freshly allocated list of N zeros (make-list only fills with nil).
(defun zeros (n)
  (let ((lst nil))
    (dotimes (i n) (push 0 lst))
    lst))

;;; Count the 1s in a 0/1 list.
(defun count-ones (lst)
  (let ((c 0))
    (dolist (x lst) (when (= x 1) (setq c (1+ c))))
    c))

;;; --- board geometry ----------------------------------------------------------

;;; The indices of the (up to eight) neighbours of cell I on a W x H board.
(defun neighbors (i w h)
  (let ((r (floor (/ i w))) (c (mod i w)) (result nil))
    (dolist (dr (list -1 0 1))
      (dolist (dc (list -1 0 1))
        (unless (and (= dr 0) (= dc 0))
          (let ((nr (+ r dr)) (nc (+ c dc)))
            (when (and (>= nr 0) (< nr h) (>= nc 0) (< nc w))
              (push (+ (* nr w) nc) result))))))
    result))

;;; How many of cell I's neighbours are mines.
(defun adjacent-count (mines i w h)
  (let ((count 0))
    (dolist (n (neighbors i w h))
      (when (= (nth n mines) 1) (setq count (1+ count))))
    count))

;;; --- state accessors ---------------------------------------------------------

(defun st-status (state) (nth 0 state))
(defun st-w (state) (nth 1 state))
(defun st-h (state) (nth 2 state))
(defun st-mines (state) (nth 3 state))
(defun st-revealed (state) (nth 4 state))
(defun st-flags (state) (nth 5 state))

;;; The game is won once every non-mine cell has been revealed.
(defun won-p (state)
  (= (count-ones (st-revealed state))
     (- (* (st-w state) (st-h state)) (count-ones (st-mines state)))))

;;; --- game actions ------------------------------------------------------------

;;; Build the initial state. MINES is a w*h list of 0/1 supplied by the host.
(defun new-game (w h mines)
  (let ((n (* w h))) (list 0 w h mines (zeros n) (zeros n))))

;;; Reveal cell IDX. A safe cell floods outward through all connected zero-count
;;; cells (classic Minesweeper auto-open); a mine ends the game.
(defun reveal (state idx)
  (let ((w (st-w state))
        (h (st-h state))
        (mines (st-mines state))
        (revealed (st-revealed state))
        (flags (st-flags state)))
    (when (and (= (st-status state) 0) (= (nth idx flags) 0)
               (= (nth idx revealed) 0))
      (if (= (nth idx mines) 1)
          ;; Stepped on a mine: uncover it and lose.
          (progn
            (set-nth revealed idx 1)
            (set-nth state 0 2))
          ;; Safe: iterative flood fill using an explicit stack of indices.
          (let ((stack (list idx)))
            (loop while stack
                  do
                    (let ((j (pop stack)))
                      (when (and (= (nth j revealed) 0) (= (nth j flags) 0))
                        (set-nth revealed j 1)
                        ;; Only keep spreading out of blank (zero-count) cells.
                        (when (= (adjacent-count mines j w h) 0)
                          (dolist (nb (neighbors j w h))
                            (when (= (nth nb revealed) 0) (push nb stack)))))))
            (when (won-p state) (set-nth state 0 1)))))
    state))

;;; A w*h mine bit-list with COUNT mines, placed at the first COUNT cells of
;;; ORDER (a host-supplied ordering of cell indices, normally a random
;;; permutation of 0..w*h-1) that are neither the safe first-click cell SAFE nor
;;; one of its neighbours -- so the opening move is always safe. This is the
;;; shared placement RULE; the host only supplies the random ORDER, because the
;;; entropy-free --no-wasi WASM reactor cannot call `random` (the browser shuffles
;;; in JavaScript, the Swing front-end uses the interpreter's `random`).
(defun place-mines (w h count safe order)
  (let ((mines (zeros (* w h)))
        (forbidden (cons safe (neighbors safe w h)))
        (placed 0))
    (dolist (idx order)
      (when (and (< placed count) (not (member idx forbidden)))
        (set-nth mines idx 1)
        (setq placed (1+ placed))))
    mines))

;;; Toggle a flag on cell IDX (only while playing and still covered).
(defun toggle-flag (state idx)
  (let ((revealed (st-revealed state)) (flags (st-flags state)))
    (when (and (= (st-status state) 0) (= (nth idx revealed) 0))
      (set-nth flags idx (- 1 (nth idx flags))))
    state))

;;; --- queries for the host ----------------------------------------------------

;;; 0 playing, 1 won, 2 lost.
(defun game-status (state) (st-status state))

;;; Mines minus flags placed -- the "mines remaining" counter.
(defun mines-remaining (state)
  (- (count-ones (st-mines state)) (count-ones (st-flags state))))


---

# FILE: references/examples/browser/minesweeper/minesweeper-swing.lisp

;;;; minesweeper-swing.lisp -- Minesweeper desktop front-end (Java / Swing).
;;;;
;;;; The rules are shared verbatim with the browser build: this file loads the
;;;; same minesweeper-core.lisp, so ONLY the drawing differs. Where the WASM
;;;; front-end (minesweeper-wasm.lisp) renders the board to HTML, this one paints a
;;;; Swing grid of clickable labels through the reusable `swing` package
;;;; (../../jvm/swing.lisp).
;;;;
;;;; Swing runs on the JVM -- interpret this file, or compile it to a .class (the
;;;; WASM backend cannot lower a java object) -- and needs a display. Run it from
;;;; anywhere; the load and the require resolve relative to this file (the
;;;; compile path inlines them):
;;;;   java -jar target/rontolisp-0.1.0-SNAPSHOT-exec.jar examples/browser/minesweeper/minesweeper-swing.lisp
;;;;   java -jar ...-exec.jar examples/browser/minesweeper/minesweeper-swing.lisp -o Minesweeper.class && java Minesweeper
;;;;
;;;; Left-click opens a cell, right-click flags it, and any click after the game
;;;; ends starts a fresh board. Unlike the entropy-free WASM reactor, the
;;;; interpreter has `random`, so this front-end lays its own mines -- keeping
;;;; them off the very first click so the opening move is always safe.

(load "minesweeper-core.lisp")
(require :swing "../../jvm/swing.lisp")

;;; --- board configuration (Beginner) ------------------------------------------

(defparameter *w* 9)
(defparameter *h* 9)
(defparameter *mines-count* 10)
(defparameter *cell-size* 34)

;;; The live game state and whether the mines have been laid yet (deferred until
;;; the first click so it can be made safe).
(defparameter *state* nil)
(defparameter *started* nil)

;;; --- palette -----------------------------------------------------------------

(defparameter *c-hidden* (swing:rgb 188 192 200)) ; a covered cell
(defparameter *c-open* (swing:rgb 225 227 232))   ; an uncovered cell
(defparameter *c-boom* (swing:rgb 214 69 65))     ; the mine you stepped on

(defparameter *fg-mine* (swing:rgb 24 24 28))
(defparameter *fg-flag* (swing:rgb 200 44 44))
(defparameter *fg-wrong* (swing:rgb 150 44 44))

(defparameter *glyph-mine* "✸")
(defparameter *glyph-flag* "⚑")
(defparameter *glyph-wrong* "✗")

;;; Classic per-number text colours (1 blue, 2 green, 3 red, ...).
(defun number-color (n)
  (cond ((= n 1) (swing:rgb 25 60 210))
        ((= n 2) (swing:rgb 20 130 40))
        ((= n 3) (swing:rgb 210 40 40))
        ((= n 4) (swing:rgb 20 20 140))
        ((= n 5) (swing:rgb 140 30 30))
        ((= n 6) (swing:rgb 20 130 130))
        ((= n 7) (swing:rgb 24 24 28))
        (t (swing:rgb 90 90 90))))

;;; --- the drawing layer (the only part that differs from the WASM build) ------

;; The window: rows = height, cols = width. Created up front so the drawing
;; functions can refer to it.
(defparameter *win*
  (swing:label-grid-window "rontolisp minesweeper" *h* *w* *cell-size*))

;; Paint one cell from the current state. This mirrors the WASM front-end's
;; cell-html, but sets a Swing label's background / colour / text instead of
;; emitting a 
.
(defun paint-cell (i r c)
  (let* ((state *state*)
         (over (> (game-status state) 0))
         (is-mine (= (nth i (st-mines state)) 1))
         (is-rev (= (nth i (st-revealed state)) 1))
         (is-flag (= (nth i (st-flags state)) 1)))
    (swing:cell-text *win* r c "")
    (cond
          ;; The mine you actually stepped on.
          ((and is-rev is-mine)
           (swing:paint *win* r c *c-boom*)
           (swing:cell-fg *win* r c *fg-mine*)
           (swing:cell-text *win* r c *glyph-mine*))
          ;; A normally revealed cell: blank, or a neighbour count 1..8.
          (is-rev
           (swing:paint *win* r c *c-open*)
           (let ((cnt (adjacent-count (st-mines state) i *w* *h*)))
             (when (> cnt 0)
               (swing:cell-fg *win* r c (number-color cnt))
               (swing:cell-text *win* r c (princ-to-string cnt)))))
          ;; Game over: reveal every remaining mine.
          ((and over is-mine)
           (swing:paint *win* r c *c-open*)
           (swing:cell-fg *win* r c *fg-mine*)
           (swing:cell-text *win* r c *glyph-mine*))
          ;; Game over: a flag that turned out to be wrong.
          ((and over is-flag)
           (swing:paint *win* r c *c-open*)
           (swing:cell-fg *win* r c *fg-wrong*)
           (swing:cell-text *win* r c *glyph-wrong*))
          ;; A flag still standing.
          (is-flag
           (swing:paint *win* r c *c-hidden*)
           (swing:cell-fg *win* r c *fg-flag*)
           (swing:cell-text *win* r c *glyph-flag*))
          ;; An ordinary covered cell.
          (t (swing:paint *win* r c *c-hidden*)))))

(defun update-status ()
  (let ((s (game-status *state*)))
    (swing:status *win*
                  (cond ((= s 1) "  You win!  Click any cell for a new game.")
                        ((= s 2) "  Boom!  Click any cell for a new game.")
                        (t (concatenate 'string "  Mines remaining: "
                                        (princ-to-string
                                         (if *started*
                                             (mines-remaining *state*)
                                             *mines-count*))
                                        "     (left: open   right: flag)"))))))

(defun draw ()
  (dotimes (i (* *w* *h*)) (paint-cell i (floor (/ i *w*)) (mod i *w*)))
  (update-status))

;;; --- host-side entropy (the shared core owns the placement rule) -------------

;; A random permutation of 0..n-1 (Fisher-Yates). This is the only host-specific
;; piece of mine layout: the first-click-safe placement RULE lives in the shared
;; core (place-mines); each front-end merely supplies a random ordering. The
;; browser does this shuffle in JavaScript because the --no-wasi reactor has no
;; `random`; here we use the interpreter's.
(defun shuffle-indices (n)
  (let ((v (make-array n)))
    (dotimes (i n) (setf (aref v i) i))
    (let ((i (- n 1)))
      (while (> i 0)
        (let ((j (random (+ i 1))) (tmp (aref v i)))
          (setf (aref v i) (aref v j))
          (setf (aref v j) tmp))
        (setq i (- i 1))))
    (let ((order nil))
      (dotimes (i n) (push (aref v (- n 1 i)) order))
      order)))

;; The first-click-safe mine layout, built by the shared core rule over a random
;; ordering -- exactly what the browser does, only the shuffle differs.
(defun make-mines (w h count safe)
  (place-mines w h count safe (shuffle-indices (* w h))))

;; Start (or restart) with an empty, mine-free board; mines are laid on the
;; first click.
(defun reset ()
  (setq *started* nil)
  (setq *state* (new-game *w* *h* (zeros (* *w* *h*))))
  (draw))

;; The single click handler bound to every cell: (row col button).
(defun on-click (r c button)
  (let ((idx (+ (* r *w*) c)))
    (cond
          ;; Any click once the game is over starts a new one.
          ((> (game-status *state*) 0) (reset))
          ;; Right-click toggles a flag.
          ((= button 3)
           (setq *state* (toggle-flag *state* idx))
           (draw))
          ;; Left-click opens; the first one lays the (safe) mines.
          (t
           (unless *started*
             (setq *state*
                   (new-game *w* *h* (make-mines *w* *h* *mines-count* idx)))
             (setq *started* t))
           (setq *state* (reveal *state* idx))
           (draw)))))

;;; --- wire it up --------------------------------------------------------------

(swing:on-cell-click *win* (function on-click))
(reset)

(print "minesweeper window is open; close it to quit")


---

# FILE: references/examples/browser/minesweeper/minesweeper-wasm.lisp

;;;; minesweeper-wasm.lisp -- Minesweeper browser front-end.
;;;;
;;;; This is the WebAssembly rendering layer: it shares all the rules with the
;;;; Swing front-end by loading minesweeper-core.lisp, then adds HTML rendering
;;;; and host-callable exports. Compiled ahead of time to a --no-wasi reactor and
;;;; driven from the browser (see minesweeper.html). A top-level literal `load`
;;;; is inlined at compile time, so the compiler sees the core `defun`s natively.
;;;;
;;;; The browser holds the game state as an opaque string that round-trips through
;;;; the WASM :s-expr ABI -- it never parses Lisp. Randomness: a --no-wasi reactor
;;;; has no entropy source, so the page supplies only a random ordering of cells
;;;; and the shared core `place-mines` applies the (first-click-safe) placement
;;;; rule -- the same rule the Swing front-end uses, entropy the only difference.
;;;;
;;;; See minesweeper-core.lisp for the state layout and the rules.

(load "minesweeper-core.lisp")

;;; --- rendering to HTML -------------------------------------------------------

;;; HTML-escape one character to safe markup; ordinary characters pass through.
;;; Matched by code point so the reader never sees the tricky #\" / #\' literals.
(defun escape-char (ch)
  (let ((c (char-code ch)))
    (cond ((= c 38) "&")  ; &
          ((= c 60) "<")   ; <
          ((= c 62) ">")   ; >
          ((= c 34) """) ; "
          ((= c 39) "'")  ; '
          (t (princ-to-string ch)))))

;;; Escape every HTML-special character in a string (& < > " '), so any cell
;;; label is safe to drop into markup. Same helper as rainbow.lisp.
(defun html-escape (s)
  (reduce (lambda (acc piece) (concatenate 'string acc piece))
          (map 'list #'escape-char s)
          :initial-value ""))

;;; One 
 for a cell: a class the CSS styles and a data-i the host reads back
;;; on click. HTML attributes use single quotes so the Lisp string needs no
;;; escapes (double quotes would); the cell label is HTML-escaped defensively.
(defun cell-div (i cls txt)
  (concatenate 'string "
"
               (html-escape txt) "
"))

;;; The class/label for cell I given the whole state (which decides how covered
;;; cells and mines are shown once the game is over).
(defun cell-html (state i)
  (let* ((status (st-status state))
         (w (st-w state))
         (h (st-h state))
         (mines (st-mines state))
         (over (> status 0))
         (is-mine (= (nth i mines) 1))
         (is-rev (= (nth i (st-revealed state)) 1))
         (is-flag (= (nth i (st-flags state)) 1)))
    (cond
          ;; The mine you actually stepped on (revealed) -- highlight it.
          ((and is-rev is-mine) (cell-div i "cell mine boom" ""))
          ;; A normally revealed cell: blank, or a neighbour count 1..8.
          (is-rev
           (let ((cnt (adjacent-count mines i w h)))
             (if (= cnt 0)
                 (cell-div i "cell open" "")
                 (cell-div i
                  (concatenate 'string "cell open n" (princ-to-string cnt))
                  (princ-to-string cnt)))))
          ;; Game over: reveal every remaining mine.
          ((and over is-mine) (cell-div i "cell mine" ""))
          ;; Game over: a flag that turned out to be wrong.
          ((and over is-flag) (cell-div i "cell wrongflag" ""))
          ;; A flag still standing.
          (is-flag (cell-div i "cell flag" ""))
          ;; An ordinary covered cell.
          (t (cell-div i "cell hidden" "")))))

;;; The whole board as a run of cell 
s (the host wraps them in a CSS grid
;;; sized to the board width).
(defun render (state)
  (let ((n (* (st-w state) (st-h state))) (out ""))
    (dotimes (i n) (setq out (concatenate 'string out (cell-html state i))))
    out))

;;; --- host-callable exports ---------------------------------------------------

(rontolisp:wasm-export 'place-mines
                       :params '(:int :int :int :int :s-expr)
                       :returns :s-expr)
(rontolisp:wasm-export 'new-game :params '(:int :int :s-expr) :returns :s-expr)
(rontolisp:wasm-export 'reveal :params '(:s-expr :int) :returns :s-expr)
(rontolisp:wasm-export 'toggle-flag :params '(:s-expr :int) :returns :s-expr)
(rontolisp:wasm-export 'render :params '(:s-expr) :returns :string)
(rontolisp:wasm-export 'game-status :params '(:s-expr) :returns :int)
(rontolisp:wasm-export 'mines-remaining :params '(:s-expr) :returns :int)


---

# FILE: references/examples/browser/minesweeper/minesweeper.html



  
    
    
    minesweeper.lisp — a Lisp listener that plays Minesweeper
    
    
    
    
  
  
    

      

        MINE■SWEEPER.LISP
        rontolisp · wasm reactor
        
      


      

        


        

          
010

          
          
000

        


        

          

        


        

          

            >
            (loading minesweeper.wasm ...)
            
          

        

      

    


    

      Every click is a real evaluation. The rules — flood fill, win/lose, and the
      board's own markup — live in
      minesweeper-wasm.lisp, compiled ahead of time to a WebAssembly reactor with --no-wasi.
      Left-click opens, right-click (or long-press) flags.
    

    


    
  



---

# FILE: references/examples/browser/rainbow/rainbow.html



  
    
    
    rontolisp rainbow text (WASM)
    
  
  
    
Rainbow text, compiled from Lisp to WebAssembly

    

      As you type, each character is colored along a rainbow gradient. The work
      is done by rainbow-html from
      rainbow.lisp, compiled ahead of time to a WebAssembly reactor with
      --no-wasi (no imports, no GC host flags needed beyond
      WebAssembly GC support). This page just feeds your text into the module's
      rainbow-html(string) -> string export and drops the
      returned HTML below. No server framework, no JavaScript color math.
    


    
    

    
(loading WebAssembly...)

    


    
  



---

# FILE: references/examples/browser/rainbow/rainbow.lisp

;;;; Color a string one character at a time, building a rainbow across it.
;;;;
;;;; The work is split into small, composable functions:
;;;;   rgb-hsv / hsv-rgb     -- color-space conversions (lists of numbers)
;;;;   interpolate-hue       -- shortest-arc interpolation of a hue (degrees)
;;;;   interpolate-hsv       -- interpolate a whole (h s v) color
;;;;   color-at              -- the palette color at a position in [0,1]
;;;;   rainbow-text          -- string -> alist of (character . "#rrggbb")
;;;;   decorate-html         -- that alist -> a string of colored s
;;;;   rainbow-html          -- string -> decorated HTML (the two combined)
;;;;
;;;; Per character we walk the input, map its index to a position p in [0,1],
;;;; sample the anchor palette at p by interpolating in HSV (hue on the shorter
;;;; arc), and emit one c. When the string is
;;;; longer or shorter than the palette the colors are interpolated, so any
;;;; length comes out as a smooth gradient.
;;;;
;;;; This uses cons/list/char/string operations, so it needs the GC backends
;;;; (interpreter / JVM / WASM-GC); it is NOT in the --no-gc subset (which has no
;;;; per-character string access). It still runs in Node as a WASM-GC reactor
;;;; built with --no-wasi (see the build/run notes at the bottom).

;;; --- small numeric helpers ---------------------------------------------------

;;; Linear interpolation a -> b by fraction f in [0,1].
(defun lerp (a b f) (+ a (* f (- b a))))

;;; A 0..1 float color component to an integer 0..255, clamped.
(defun clamp255 (x) (let ((n (round (* x 255.0)))) (max 0 (min 255 n))))

;;; An integer 0..255 to a two-digit lowercase hex string.
(defun hex2 (n)
  (let ((digits "0123456789abcdef") (hi (floor (/ n 16))) (lo (mod n 16)))
    (concatenate 'string (subseq digits hi (1+ hi))
                 (subseq digits lo (1+ lo)))))

;;; An (r g b) list (0..255 each) to a "#rrggbb" CSS color string.
(defun rgb-hex (rgb)
  (concatenate 'string "#" (hex2 (first rgb)) (hex2 (second rgb))
               (hex2 (third rgb))))

;;; --- color-space conversion --------------------------------------------------

;;; (r g b) with each component 0..255  ->  (h s v) with h in [0,360), s,v in [0,1].
(defun rgb-hsv (c)
  (let* ((r (/ (float (first c)) 255.0))
         (g (/ (float (second c)) 255.0))
         (b (/ (float (third c)) 255.0))
         (mx (max r g b))
         (mn (min r g b))
         (d (- mx mn))
         (v mx)
         (s (if (= mx 0.0) 0.0 (/ d mx)))
         (h
          (cond ((= d 0.0) 0.0)
                ((= mx r) (* 60.0 (mod (/ (- g b) d) 6.0)))
                ((= mx g) (* 60.0 (+ (/ (- b r) d) 2.0)))
                (t (* 60.0 (+ (/ (- r g) d) 4.0))))))
    (list h s v)))

;;; (h s v)  ->  (r g b) with each component an integer 0..255.
(defun hsv-rgb (h s v)
  (let* ((c (* v s))
         (x (* c (- 1.0 (abs (- (mod (/ h 60.0) 2.0) 1.0)))))
         (m (- v c))
         ;; (r g b) for this 60-degree sector, before adding the m offset.
         (rgb
          (cond ((< h 60.0) (list c x 0.0))
                ((< h 120.0) (list x c 0.0))
                ((< h 180.0) (list 0.0 c x))
                ((< h 240.0) (list 0.0 x c))
                ((< h 300.0) (list x 0.0 c))
                (t (list c 0.0 x)))))
    (list (clamp255 (+ (first rgb) m)) (clamp255 (+ (second rgb) m))
          (clamp255 (+ (third rgb) m)))))

;;; --- interpolation -----------------------------------------------------------

;;; Interpolate a hue from h0 to h1 by fraction f, going the short way around the
;;; 360-degree color wheel (so red 350 -> 10 passes through 0, not through cyan).
(defun interpolate-hue (h0 h1 f)
  (let* ((raw (- h1 h0))
         ;; the signed delta taken the short way around the 360-degree wheel
         (d
          (cond ((> raw 180.0) (- raw 360.0))
                ((< raw -180.0) (+ raw 360.0))
                (t raw))))
    (mod (+ h0 (* f d)) 360.0)))

;;; Interpolate a whole color: hue on the short arc, saturation/value linearly.
;;; a and b are (h s v) lists.
(defun interpolate-hsv (a b f)
  (list (interpolate-hue (first a) (first b) f) (lerp (second a) (second b) f)
        (lerp (third a) (third b) f)))

;;; --- palette -----------------------------------------------------------------

;;; The anchor colors (RGB 0..255), in gradient order.
(defun rainbow-anchors ()
  (list (list 153 50 204) (list 71 111 240) (list 69 139 116) (list 50 205 50)
        (list 255 215 0) (list 255 127 0) (list 238 99 99) (list 238 64 0)
        (list 205 38 38) (list 139 26 26)))

;;; The interpolated "#rrggbb" color at position p in [0,1] across the palette.
(defun color-at (p)
  (let* ((anchors (rainbow-anchors))
         (n (length anchors))
         (pos (* p (float (1- n))))
         ;; segment index, clamped so k and k+1 are valid anchor indices
         (k (max 0 (min (- n 2) (floor pos))))
         (frac (- pos (float k)))
         (a (rgb-hsv (nth k anchors)))
         (b (rgb-hsv (nth (1+ k) anchors)))
         (hsv (interpolate-hsv a b frac)))
    (rgb-hex (hsv-rgb (first hsv) (second hsv) (third hsv)))))

;;; --- the public functions ----------------------------------------------------

;;; A string to an alist mapping each character to its "#rrggbb" rainbow color.
;;; `loop for i below len` walks the character indices (a string is not a list,
;;; so an index also drives the color position): index i -> (char . color-at p).
(defun rainbow-text (s)
  (let ((len (length s)))
    (loop for i below len
          collect
            (cons (char s i)
             (color-at (if (<= len 1) 0.0 (/ (float i) (float (1- len)))))))))

;;; HTML-escape one character to safe markup; ordinary characters pass through.
;;; Matched by code point so the reader never sees the tricky #\" / #\' literals.
(defun escape-char (ch)
  (let ((c (char-code ch)))
    (cond ((= c 38) "&")  ; &
          ((= c 60) "<")   ; <
          ((= c 62) ">")   ; >
          ((= c 34) """) ; "
          ((= c 39) "'")  ; '
          (t (princ-to-string ch)))))

;;; Escape every HTML-special character in a string (& < > " '), so arbitrary
;;; input is safe to drop into markup. `map 'list` escapes each character to a
;;; string, then reduce joins the pieces.
(defun html-escape (s)
  (reduce (lambda (acc piece) (concatenate 'string acc piece))
          (map 'list #'escape-char s)
          :initial-value ""))

;;; One (character . color) pair to its colored  string, with the
;;; character HTML-escaped.
(defun span-html (pair)
  (concatenate 'string ""
               (html-escape (princ-to-string (car pair))) ""))

;;; An alist of (character . color) to a string of colored  elements:
;;; mapcar each pair to its span, then reduce the spans into one string. (The
;;; join uses reduce rather than `apply #'concatenate` because concatenate takes
;;; a result-type argument and has no first-class function value in the
;;; compilers; here concatenate stays in call position and is inlined.)
(defun decorate-html (pairs)
  (reduce (lambda (acc span) (concatenate 'string acc span))
          (mapcar #'span-html pairs)
          :initial-value ""))

;;; A string straight to its rainbow HTML (rainbow-text then decorate-html).
(defun rainbow-html (s) (decorate-html (rainbow-text s)))

;;; Export rainbow-html as a host-callable WASM function (string in, string out).
;;; A no-op on the interpreter/JVM; under --no-wasi it makes a Node-loadable
;;; WASM-GC reactor.
(rontolisp:wasm-export 'rainbow-html :params '(:string) :returns :string)


---

# FILE: references/examples/browser/wasm-browser/README.md

# Running rontolisp WASM in the browser (plain HTML + JavaScript)

Take a Lisp program, compile it to WebAssembly with rontolisp, and run it in a
browser from ordinary HTML and JavaScript — no framework, no bundler, no
server-side component.

Different from [`web/`](https://github.com/making/rontolisp/blob/develop/examples/browser/web) at the repository root: that playground
compiles **rontolisp itself** to WASM (via GraalVM Web Image). Here we compile a
**user Lisp program** with rontolisp's own WASM backend and call the result from
JavaScript.

**Live demo:** 

## What's in here

| File            | Purpose                                                            |
| --------------- | ----------------------------------------------------------------- |
| `index.html`    | The demo page: buttons that run the WASM modules and show stdout. |
| `wasi-shim.js`  | A tiny, dependency-free WASI Preview 1 shim (the glue code).       |
| `hello.lisp` / `hello.wasm` | A self-contained program (Fibonacci + rational math). |
| `greet.lisp` / `greet.wasm` | Reads a line from stdin and greets — shows input.     |
| `dice.lisp` / `dice.wasm`   | Rolls dice with `random` — different result every run. |
| `build.sh`      | Recompiles the `.lisp` files to `.wasm`.                          |

The `.wasm` files are checked in, so you can run the demo without building
anything.

## How it works

A rontolisp-compiled module is a WASI "command": it exports `memory` and
`_start`, and imports eight functions from `wasi_snapshot_preview1`:

```
fd_write  fd_read  path_open  fd_close
random_get  clock_time_get  environ_sizes_get  environ_get
```

There is no host runtime in a browser, so `wasi-shim.js` implements those eight
functions in JavaScript:

- **stdout / stderr** (`fd_write`) are captured into strings instead of a tty.
- **stdin** (`fd_read`) is served from a string you pass in.
- **files** (`path_open`) are unsupported and report "no entry".
- **randomness** uses `crypto.getRandomValues`; the **clock** uses `Date.now()`.
- **environment variables** come from an `env` option.

Running a module is then three steps (see `runWasm` in `wasi-shim.js`):

```js
import { runWasm } from "./wasi-shim.js";

// 1. fetch + instantiate with the shim's imports, 2. call _start,
// 3. collect what it printed.
const { stdout } = await runWasm("./hello.wasm");
console.log(stdout);

// Passing input via stdin:
const res = await runWasm("./greet.wasm", { stdin: "Ada\n" });

// Passing input via environment variables:
const res2 = await runWasm("./prog.wasm", { env: { NAME: "Ada" } });

// If you already have the module bytes in memory (e.g. compiled in the
// browser, with no file to fetch), skip the fetch with runWasmModule:
const res3 = await runWasmModule(wasmBytes, { stdin: "Ada\n" });
```

`runWasm(url)` is just `fetch` + `runWasmModule(bytes)`. The root
[`web/` playground](https://github.com/making/rontolisp/blob/develop/examples/browser/web) uses `runWasmModule` on bytes it compiled in the
browser a moment earlier, so compile-to-WASM and run-the-WASM both happen
client-side: it compiles the definitions once with a `(print (eval (read)))`
driver, then passes each call expression as **stdin** without recompiling.

Because the module's only outward interface is stdout (and optionally stdin /
env / exit code), "calling it from JavaScript" means *running it and reading
what it printed* — it is a whole program, not a library of exported functions.

## Run it

The page fetches `.wasm` files, so it must be served over `http://` (opening
`index.html` as a `file://` URL will fail the `fetch`). Any static server works:

```bash
cd examples/browser/wasm-browser
python3 -m http.server 8000
# then open http://localhost:8000/
```

Click **Run hello.wasm** to run the self-contained program, **Run greet.wasm**
to feed the textarea contents to the program as stdin, and **Roll dice.wasm**
(then **Roll again**) to see `random` produce a fresh result each run from the
host's `random_get`.

## Browser requirements

rontolisp emits **WebAssembly GC** (the cons cell is a GC struct), so you need a
browser that supports the WASM GC proposal — enabled by default in:

- Chrome / Edge 119+
- Firefox 120+
- Safari 18.2+

No special flags are required in those versions. (In Node.js the same modules
run under `node` 22+ without flags, which is how the shim is regression-tested.)

## Rebuilding the `.wasm` files

If you edit the `.lisp` sources, rebuild from the repository root:

```bash
./mvnw clean package          # produces target/rontolisp-...-exec.jar
examples/browser/wasm-browser/build.sh
```

## Notes and limitations

- **Input channels.** A WASI command cannot be handed arguments the way a
  function call can. To pass data in, use **stdin** (`read-line`), **environment
  variables** (`uiop:getenv`), or compile the input into the program. This shim
  supports stdin (demoed by `greet.wasm`) and env (pass `{ env: { NAME: "Ada" } }`
  to `runWasm`); command-line args are not wired (rontolisp's output does not
  import `args_get`).
- **Files.** `open` / `load` / `with-open-file` will fail in the browser —
  there is no filesystem behind `path_open`. Use stdin/env instead.
- This shim is intentionally minimal and readable. For a more complete browser
  WASI implementation, use a package such as `@bjorn3/browser_wasi_shim`.


---

# FILE: references/examples/browser/wasm-browser/build.sh

#!/usr/bin/env bash
# Recompile the .lisp sources in this directory to .wasm using rontolisp.
# Run from the repository root or from this directory; it locates the JAR
# relative to the repo root.
#
# --optimize tree-shakes the runtime so only the reachable functions -- and only
# the WASI imports they actually reach -- ship. These are the smallest programs
# in the examples, so it is also the largest difference: without it each module
# carries the whole runtime and is ~40x bigger.
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

for src in "$here"/*.lisp; do
  name="$(basename "$src" .lisp)"
  echo "compiling $name.lisp -> $name.wasm"
  java -jar "$jar" "$src" -o "$here/$name.wasm" --optimize
done

echo "done. Serve this directory over http, e.g.:"
echo "  python3 -m http.server 8000 --directory \"$here\""


---

# FILE: references/examples/browser/wasm-browser/dice.lisp

;; dice.lisp -- rolls a handful of dice using `random`.
;; On WASI Preview 1 `random` draws real entropy from the host's `random_get`
;; (the JavaScript shim backs it with `crypto.getRandomValues`), so every click
;; produces a different roll -- it is NOT a fixed pseudo-random sequence.

(defun roll () (+ 1 (random 6)))

(let ((total 0))
  (format t "Rolling five six-sided dice:~%")
  (dotimes (i 5)
    (let ((r (roll)))
      (setq total (+ total r))
      (format t "  die ~a -> ~a~%" (+ i 1) r)))
  (format t "~%")
  (format t "Total: ~a~%" total))


---

# FILE: references/examples/browser/wasm-browser/greet.lisp

;; greet.lisp -- reads one line from stdin and prints a greeting.
;; The JavaScript shim feeds the text from a 
      

        
      

    

    
(output appears here)


    
3. Real randomness from the host

    

      dice.wasm rolls five dice with random. On WASI
      Preview 1 random draws real entropy from the host's
      random_get (the shim backs it with
      crypto.getRandomValues), so click Roll again and the
      result changes every time -- it is not a fixed pseudo-random sequence.
      Source:
      dice.lisp.
    

    

      
    

    
(output appears here)


    
  



---

# FILE: references/examples/browser/wasm-browser/wasi-shim.js

// wasi-shim.js
//
// A tiny, dependency-free WASI Preview 1 shim, just large enough to run a
// program compiled by rontolisp (`rontolisp prog.lisp -o prog.wasm`) in a
// browser. rontolisp's WASM output imports exactly nine functions from the
// "wasi_snapshot_preview1" module:
//
//   fd_write, fd_read, path_open, fd_close, fd_readdir,
//   random_get, clock_time_get, environ_sizes_get, environ_get
//
// A module built with --optimize imports only the ones it actually reaches, so
// the shim provides all nine and lets the link pick.
//
// This shim implements them over plain JavaScript:
//   - stdout/stderr (fd 1/2) are captured into strings instead of a real tty
//   - stdin (fd 0) is fed from a string you provide
//   - files (path_open) are not supported and report "no entry"
//   - directories (fd_readdir) are not supported and report "not supported"
//   - randomness uses crypto.getRandomValues, the clock uses Date.now()
//   - environment variables come from the `env` option
//
// It is intentionally minimal and easy to read; it is NOT a complete WASI
// implementation. For a fuller one, use a package such as
// `@bjorn3/browser_wasi_shim`.

const WASI_ESUCCESS = 0;
const WASI_EBADF = 8; // bad file descriptor
const WASI_ENOENT = 44; // no such file or directory
const WASI_ENOSYS = 52; // function not supported

/**
 * Create a WASI Preview 1 import object plus helpers for one run of a module.
 *
 * @param {Object}  [opts]
 * @param {string}  [opts.stdin]  text delivered to the program's stdin (fd 0)
 * @param {Object}  [opts.env]    environment variables, e.g. { NAME: "Ada" }
 * @returns {{ imports: object, setMemory: (m: WebAssembly.Memory) => void,
 *             getStdout: () => string, getStderr: () => string }}
 */
export function createWasi({ stdin = "", env = {} } = {}) {
  const encoder = new TextEncoder();
  const decoder = new TextDecoder();

  let memory = null; // set after instantiation via setMemory()
  let stdoutText = "";
  let stderrText = "";

  const stdinBytes = encoder.encode(stdin);
  let stdinPos = 0;

  // environ entries are "KEY=VALUE\0" byte arrays (WASI layout).
  const envEntries = Object.entries(env).map(([k, v]) =>
    encoder.encode(`${k}=${v}\0`),
  );

  const view = () => new DataView(memory.buffer);
  const bytes = () => new Uint8Array(memory.buffer);

  const imports = {
    // fd_write(fd, iovs, iovs_len, nwritten) -> errno
    // Concatenate the iovec slices and route fd 1 -> stdout, fd 2 -> stderr.
    fd_write(fd, iovs, iovsLen, nwritten) {
      let written = 0;
      let chunk = "";
      for (let i = 0; i < iovsLen; i++) {
        const base = iovs + i * 8;
        const ptr = view().getUint32(base, true);
        const len = view().getUint32(base + 4, true);
        chunk += decoder.decode(new Uint8Array(memory.buffer, ptr, len));
        written += len;
      }
      if (fd === 1) stdoutText += chunk;
      else if (fd === 2) stderrText += chunk;
      view().setUint32(nwritten, written, true);
      return WASI_ESUCCESS;
    },

    // fd_read(fd, iovs, iovs_len, nread) -> errno
    // Serve bytes from the provided stdin string; EOF reads zero bytes.
    fd_read(fd, iovs, iovsLen, nread) {
      if (fd !== 0) return WASI_EBADF;
      let total = 0;
      const mem = bytes();
      for (let i = 0; i < iovsLen; i++) {
        const base = iovs + i * 8;
        const ptr = view().getUint32(base, true);
        const len = view().getUint32(base + 4, true);
        let j = 0;
        for (; j < len && stdinPos < stdinBytes.length; j++) {
          mem[ptr + j] = stdinBytes[stdinPos++];
        }
        total += j;
        if (j < len) break; // ran out of input
      }
      view().setUint32(nread, total, true);
      return WASI_ESUCCESS;
    },

    // Files are not backed by anything in the browser.
    path_open() {
      return WASI_ENOENT;
    },
    fd_close() {
      return WASI_ESUCCESS;
    },

    // fd_readdir(fd, buf, buf_len, cookie, bufused_out) -> errno
    // There are no directories here, so a program that lists one gets "not
    // supported" rather than an empty listing it would read as "no files".
    fd_readdir() {
      return WASI_ENOSYS;
    },

    // random_get(ptr, len) -> errno
    random_get(ptr, len) {
      crypto.getRandomValues(new Uint8Array(memory.buffer, ptr, len));
      return WASI_ESUCCESS;
    },

    // clock_time_get(id, precision, out) -> errno ; out is an i64 of nanoseconds
    clock_time_get(_id, _precision, out) {
      const nanos = BigInt(Date.now()) * 1_000_000n;
      view().setBigUint64(out, nanos, true);
      return WASI_ESUCCESS;
    },

    // environ_sizes_get(count_out, bufsize_out) -> errno
    environ_sizes_get(countOut, bufsizeOut) {
      const bufSize = envEntries.reduce((sum, e) => sum + e.length, 0);
      view().setUint32(countOut, envEntries.length, true);
      view().setUint32(bufsizeOut, bufSize, true);
      return WASI_ESUCCESS;
    },

    // environ_get(ptrs_out, buf_out) -> errno
    environ_get(ptrsOut, bufOut) {
      const mem = bytes();
      let bufPtr = bufOut;
      for (let i = 0; i < envEntries.length; i++) {
        view().setUint32(ptrsOut + i * 4, bufPtr, true);
        mem.set(envEntries[i], bufPtr);
        bufPtr += envEntries[i].length;
      }
      return WASI_ESUCCESS;
    },

    // Programs that exit explicitly would call this. rontolisp's output never
    // imports proc_exit, but we provide it so the shim also works with modules
    // that do (a thrown sentinel ends _start cleanly).
    proc_exit(code) {
      throw new WasiExit(code);
    },
  };

  return {
    imports: { wasi_snapshot_preview1: imports },
    setMemory: (m) => {
      memory = m;
    },
    getStdout: () => stdoutText,
    getStderr: () => stderrText,
  };
}

/** Thrown by proc_exit to unwind out of _start. */
export class WasiExit extends Error {
  constructor(code) {
    super(`WASI exit ${code}`);
    this.code = code;
  }
}

/**
 * Instantiate and run an already-loaded rontolisp-compiled `.wasm` command
 * module (a `BufferSource`: `ArrayBuffer` or typed array), returning whatever
 * it wrote to stdout/stderr. Use this when the bytes are already in memory —
 * e.g. compiled in the browser by the playground's `rontoCompileWasm`, with no
 * `.wasm` file to fetch.
 *
 * @param {BufferSource} wasmBytes  the module bytes
 * @param {Object} [opts]           same options as createWasi()
 * @returns {Promise<{ stdout: string, stderr: string, exitCode: number }>}
 */
export async function runWasmModule(wasmBytes, opts = {}) {
  const wasi = createWasi(opts);
  const { instance } = await WebAssembly.instantiate(wasmBytes, wasi.imports);

  wasi.setMemory(instance.exports.memory);

  let exitCode = 0;
  try {
    instance.exports._start();
  } catch (e) {
    if (e instanceof WasiExit) exitCode = e.code;
    else throw e;
  }
  return {
    stdout: wasi.getStdout(),
    stderr: wasi.getStderr(),
    exitCode,
  };
}

/**
 * Fetch, instantiate and run a rontolisp-compiled `.wasm` command module,
 * returning whatever it wrote to stdout/stderr.
 *
 * @param {string} url            URL of the .wasm file
 * @param {Object} [opts]         same options as createWasi()
 * @returns {Promise<{ stdout: string, stderr: string, exitCode: number }>}
 */
export async function runWasm(url, opts = {}) {
  const response = await fetch(url);
  if (!response.ok) {
    throw new Error(`failed to fetch ${url}: ${response.status}`);
  }
  // instantiateStreaming needs the correct application/wasm MIME type; fall
  // back to ArrayBuffer instantiation when the server doesn't send it.
  const wasmBytes = await response.arrayBuffer();
  return runWasmModule(wasmBytes, opts);
}


---

# FILE: references/examples/browser/webgl-battlefront/README.md

# webgl-battlefront — a one-arena snow-battle skirmish, in Lisp

Move with WASD while the mouse aims
(the page takes Pointer Lock), Space jumps, click attacks,
F swaps lightsaber and blaster, R restarts, scroll zooms.
Cut down the stormtroopers, bring the two walkers down, and the boss strides in
— his blade deflects blaster fire, so finish him with your own.

On a touch-primary device the page swaps in a touch layer instead: a floating
joystick moves, dragging the right half aims, and a FIRE / JUMP / F cluster
covers the rest. It skips Pointer Lock entirely (iOS Safari has none) and
carries no instruction prose — the on-screen controls say what they do.

**Live demo:** 

## What is in the Lisp

Everything that makes it a game lives in `battlefront.lisp`, compiled ahead of
time to WebAssembly:

- **Movement and the aim camera** — camera-relative acceleration, the
  third-person follow camera whose yaw is the mouse aim, and its look-at /
  perspective matrices.
- **Blaster bolts** — one pool shared by you, the troopers and the walkers:
  travel, lifetimes, per-owner collisions, muzzle and impact sparks.
- **The lightsaber** — a swing that deals a frontal-arc hit *and* **deflects**
  any incoming bolt that reaches it, so blocking is a real defense.
- **The AI** — troopers that close in and fire, two ranged walkers, and a boss
  who stays dormant until both walkers fall, then chases and swings in melee.
- **Every triangle** — you, the enemies, the walkers, the blades and bolts are
  tessellated from scratch each frame. The blades, bolts, damage flashes and
  fireworks are a second **additive-blended pass** over the same vertex buffer,
  so they bloom against the snow.
- **Materials** — each part carries a *shine* value driving a Blinn-Phong
  highlight plus a soft rim light, so rounded surfaces read as lit rather than
  flat.

JavaScript is the same one-line WebGL2 host boundary as the other `webgl-*`
demos — bindings generated from the shared `gl.wit` (see
[`../webgl-common/`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-common)) plus two staging entries (`setEmissive`,
`setShine`) — the Pointer-Lock mouse, and the HUD. The page maps input to small
integers; every rule is Lisp's.

## The shape vocabulary

Four primitives, all with smooth per-vertex normals (face culling is off, so
only the explicit normal matters):

- **`emit-limb`** — a tapered tube between two *arbitrary* 3D points, with the
  side normal tilted by the taper so a cone is lit as a cone. This is the one
  that changes what the cast can be: with upright cylinders and horizontal
  beams alone a figure has straight posts for limbs, which is why the walkers
  used to read as tables. With a free-standing segment, a leg becomes thigh +
  knee ball + shin at real angles.
- **`emit-rbox`** — a box with genuinely rounded edges: the Minkowski sum of a
  core box and a sphere, sampled by taking a point on the outer box, clamping
  it into the core, and placing the surface at `q + br * normalize(P - q)` —
  whose normal is that same direction. Splitting each face at the core boundary
  puts the sample lines where the curvature starts, so a 3×3 grid per face is
  enough.
- **`emit-ellipsoid` / `emit-cylinder` / `emit-cyl-beam`** — heads, helmet
  domes, joint balls, barrels, blade rods, bolts, shadows, drifts, boulders.
- **soft box normals** — a plain `emit-box` stamped with per-corner normals
  bent toward each corner's outward direction. The silhouette stays a box but
  the shading gradient is a bevelled edge's, for not one extra vertex.

Two things keep that affordable by buying detail where it is *visible*:
`set-lod` picks a per-figure detail tier from apparent size, and `part-rbox`
then asks the same question about the individual part — a fillet costs nine
times a plain box, so it is spent only when that part's largest half-extent
over its distance clears a threshold.

Two consequences worth knowing if you edit the models. The walker's legs are
**solved, not drawn**: the foot is animated and the knee placed by the two-link
inverse kinematics the fixed thigh and shin lengths force — that is what makes
it stride instead of slide. And the boss's cape is a **parametric surface**,
sampled over (down the back × across the back), normals taken from finite
differences of the same function, because a cape is the one part a box can
never stand in for.

The horizon is the same `emit-limb`: a ring of squat cones with off-centre
apexes in two rows, on an apron of very wide low cones. It replaced four tall
boxes, which from inside the arena is a flat grey wall with two corners running
up the sky.

## Building

```bash
./build.sh          # battlefront.lisp -> battlefront.wasm (--no-wasi --optimize)
# the page imports the generated ../webgl-common/gl-imports.js, so serve
# examples/browser rather than this directory:
jwebserver -p 8000 --directory ..
# open http://localhost:8000/webgl-battlefront/
```

`build.sh` uses a `rontolisp` binary on `PATH` if it finds one, otherwise the
built exec JAR. `--no-wasi` makes the module a reactor whose only imports are
host functions; `--optimize` tree-shakes the runtime and the unused WebGL
entries, leaving only the ~40 the program reaches. The page instantiates the
module, calls `_initialize()` (which compiles the shaders and bakes the snow
field, from Lisp), then calls the exported `frame` per animation tick.

The module's exports are on `window.lisp`, so the DevTools console can poke the
game directly: `lisp.getHp()`, `lisp.getWeapon()`, `lisp.restart()`.

## A note on the geometry math

The **game** math is the `linalg` package throughout. Every coordinate is a
packed single-float vector (`#f(x y z)`), and movement, distance and heading are
`linalg:add`/`sub`/`mul`/`dot`/`norm` straight on those vectors — single-float
in, single-float out, no boxing path. The view-projection is `linalg:matmul` of
two `(4 4)` matrices, flattened from its **transpose** into the column-major run
`gl:uniform-matrix4fv` wants. There are no bespoke vector helpers: the `linalg:`
calls are the vector algebra.

The **tessellation** inner loop deliberately is not. A box's eight corners used
to be one `linalg:matmul` plus a broadcast add — the right tool for a camera
matrix and the wrong one for eight corners: four array allocations and a general
nested-loop matmul, paid per box per frame. With hundreds of boxes a frame that
allocation dominated; `emit-box` writes the same rotation out as scalars into
one corner buffer allocated once. The same holds for the cylinder / ellipsoid /
limb / rounded-box samplers, where each vertex is a handful of scalar
`sin`/`cos` calls.

One more constraint shapes those signatures. The WASM backend's callable types
stop at **seven parameters**; a wider fixed-arity `defun` still compiles, but
only because the compiler bundles the surplus arguments into a freshly consed
list at every call site — invisible in a cold helper, ruinous in one called per
triangle. So every per-vertex function stays at seven parameters or fewer, and
whatever is constant across a primitive is latched in a global instead. That is
why there is no `emit-tri`: three `emit-vertex` calls cost nothing, an
18-parameter helper costs twelve cons cells a triangle.


---

# FILE: references/examples/browser/webgl-battlefront/battlefront.lisp

;;;; battlefront.lisp -- a one-arena snow-battle skirmish, entirely in Lisp.
;;;;
;;;; Play as you on a Hoth snow field. Move with W/A/S/D (Minecraft style, the
;;;; mouse aims), switch between the lightsaber and the blaster with F, attack
;;;; with the left mouse button. Cut down the stormtroopers, bring down the two
;;;; AT-AT walkers, and once the walkers are gone Vader wakes -- his red
;;;; blade deflects blaster fire, so finish the boss with your own lightsaber.
;;;;
;;;; Everything that makes it a game lives here: the movement and the
;;;; camera-relative steering, the third-person follow/aim camera and its
;;;; look-at/perspective matrices, the blaster bolts (fired by you, the
;;;; troopers and the walkers) with their travel, lifetimes and collisions, the
;;;; lightsaber swing that both damages enemies and DEFLECTS incoming fire, the
;;;; trooper / AT-AT / Vader AI, the win/lose state machine, and every triangle
;;;; of the world -- flat parts (armor plates, packs, belts) are tessellated
;;;; from rotated boxes each frame, round ones (heads, limbs, gun/saber
;;;; barrels, bolts, sparks) from smooth-normaled cylinders and ellipsoids.
;;;; The glowing lightsabers and bolts are a second additive-blended pass over
;;;; the same buffer, so the blades bloom against the snow.
;;;;
;;;; The geometry math is the linalg package throughout. Every position is a
;;;; packed single-float vector (#f(x y z)); movement, distances and headings
;;;; are linalg:add / sub / mul / dot / norm on those vectors; a box's eight
;;;; corners are one linalg:matmul of the yaw rotation against the local-corner
;;;; matrix (then a broadcast add of the center); and the view-projection is a
;;;; linalg:matmul of two rank-2 (4 4) matrices flattened (transposed) into the
;;;; column-major run the GPU wants.
;;;;
;;;; JavaScript is the same one-line WebGL2 host boundary as the other webgl-*
;;;; demos, plus Pointer-Lock mouse/keyboard forwarding and the HUD.
;;;;
;;;; Compiled ahead of time to a --no-wasi reactor (build.sh), so the module
;;;; imports nothing but the host functions declared below and instantiates in
;;;; any wasm-GC-capable browser.

;; --- the host boundary ------------------------------------------------------
;;
;; The WebGL2 API itself -- the wasm-import directives, the enum constants and
;; the shader helpers -- lives in the shared gl package (../webgl-common/gl.lisp),
;; spliced in here at compile time; --optimize drops the entries this demo never
;; calls. Only the imports specific to this page stay below.

(require :gl "../webgl-common/gl.lisp")

;; The bulk-float staging path (see webgl-robot-arm / webgl-platformer): each
;; vertex is 11 floats -- position (3), normal (3), color (3), an emissive
;; term (1) that makes the blades and bolts self-lit, and a shine term (1)
;; that drives the specular highlight on polished materials (helmet domes,
;; gun/saber metal). Color, emissive and shine are constant per part, so
;; set-color / set-emissive / set-shine latch them and set-vertex stages
;; position + normal + those latched values (a call crosses the WASM boundary
;; with at most 7 parameters).
(rontolisp:wasm-import 'set-color
                       :from "gl"
                       :as "setColor"
                       :params '(:float :float :float)
                       :returns :void)
(rontolisp:wasm-import 'set-emissive
                       :from "gl"
                       :as "setEmissive"
                       :params '(:float)
                       :returns :void)
;; shine (0 = matte cloth/snow, 1 = polished metal/glass) drives the specular
;; highlight in the fragment shader -- latched like color/emissive, so armor
;; plates, helmet domes and blaster/saber metal can read as harder materials
;; than the fabric and snow around them.
(rontolisp:wasm-import 'set-shine
                       :from "gl"
                       :as "setShine"
                       :params '(:float)
                       :returns :void)
(rontolisp:wasm-import 'set-vertex
                       :from "gl"
                       :as "setVertex"
                       :params '(:int :float :float :float :float :float :float)
                       :returns :void)
(rontolisp:wasm-import 'gl-upload-vertices
                       :from "gl"
                       :as "uploadVertices"
                       :params '(:int :int)
                       :returns :void)
(rontolisp:wasm-import 'set-float
                       :from "gl"
                       :as "setFloat"
                       :params '(:int :float)
                       :returns :void)
(rontolisp:wasm-import 'gl-uniform-matrix4fv
                       :from "gl"
                       :as "uniformMatrix4fv"
                       :params '(:int)
                       :returns :void)

;; Canvas metrics, owned by the page.
(rontolisp:wasm-import 'canvas-width
                       :from "canvas"
                       :as "width"
                       :params '()
                       :returns :float)
(rontolisp:wasm-import 'canvas-height
                       :from "canvas"
                       :as "height"
                       :params '()
                       :returns :float)

;; The WASM backend has no transcendental built-ins, so borrow the host's; the
;; host also supplies entropy (trooper fire timing, muzzle jitter).
(rontolisp:wasm-import 'sin :from "math" :params '(:float) :returns :float)
(rontolisp:wasm-import 'cos :from "math" :params '(:float) :returns :float)
(rontolisp:wasm-import 'atan2
                       :from "math"
                       :params '(:float :float)
                       :returns :float)
(rontolisp:wasm-import 'host-random
                       :from "math"
                       :as "random"
                       :params '()
                       :returns :float)

(defconstant +pi+ 3.141592653589793)
(defconstant +two-pi+ 6.283185307179586)

(defun rand01 () (host-random))
(defun rand-range (a b) (+ a (* (- b a) (host-random))))

;; --- shaders ----------------------------------------------------------------
;;
;; One program: lit triangles under a cold Hoth key light, a sky-tinted
;; hemisphere ambient and a distance fog that fades the far scenery into the
;; pale sky. An emissive term (aEmit) lets a vertex ignore the lighting and fog
;; and glow at its own color -- that is what the lightsaber blades and the
;; blaster bolts use.

(defconstant +solid-vs+
  "#version 300 es
layout(location=0) in vec3 aPos;     // world-space position from Lisp
layout(location=1) in vec3 aNormal;  // world-space normal from Lisp
layout(location=2) in vec3 aColor;
layout(location=3) in float aEmit;   // 0 = lit surface, 1 = self-lit
layout(location=4) in float aShine;  // 0 = matte, 1 = polished metal/glass
uniform mat4 uVP;                    // view-projection, computed in Lisp
out vec3 vN;
out vec3 vC;
out vec3 vW;
out float vE;
out float vS;
void main() {
  gl_Position = uVP * vec4(aPos, 1.0);
  vN = aNormal;
  vC = aColor;
  vW = aPos;
  vE = aEmit;
  vS = aShine;
}")

(defconstant +solid-fs+
  "#version 300 es
precision mediump float;
in vec3 vN;
in vec3 vC;
in vec3 vW;
in float vE;
in float vS;
uniform vec3 uEye;
out vec4 color;
void main() {
  vec3 n = normalize(vN);
  vec3 l = normalize(vec3(0.35, 0.82, 0.45));
  vec3 v = normalize(uEye - vW);
  float diff = max(dot(n, l), 0.0);
  float amb  = 0.52 + 0.18 * n.y;           // hemisphere: tops brighter
  vec3 lit = vC * (amb + 0.55 * diff);

  // a Blinn-Phong highlight, harder and brighter the shinier the material --
  // this is what separates polished helmet domes / gun metal / saber hilts
  // from the matte cloth and snow around them (both packed in aShine).
  vec3 hFace = normalize(l + v);
  float shininess = mix(10.0, 90.0, vS);
  float spec = pow(max(dot(n, hFace), 0.0), shininess) * vS * 1.3;
  lit += spec;

  // a cheap outdoor rim light: the low Hoth sun grazes silhouette edges with
  // a touch of sky color, which reads as soft ambient occlusion's opposite --
  // it keeps rounded (ellipsoid/cylinder) surfaces from looking flat-lit.
  float rim = pow(1.0 - max(dot(n, v), 0.0), 2.5) * 0.22;
  vec3 sky = vec3(0.74, 0.83, 0.93);
  lit += rim * sky;

  vec3 shown = mix(lit, vC, vE);            // emissive: ignore the light
  // the fog reaches further than it used to: the horizon is a range of peaks
  // now rather than a wall, and saturating it at 95m erased the whole skyline
  float fog = smoothstep(45.0, 150.0, distance(uEye, vW)) * (1.0 - vE);
  color = vec4(mix(shown, sky, fog), 1.0);
}")

;; --- 4x4 matrix math: rank-2 arrays + the linalg package ----------------------
;;
;; The matrices are ordinary rank-2 (4 4) linalg arrays in textbook (row, col)
;; convention, VP = P x V is one linalg:matmul, and upload-vp flattens the
;; transpose into the column-major order WebGL expects.

(defconstant +half-fov+ 0.44) ; a touch under 50-degree fov

(defun mat4-perspective (aspect near far)
  (let* ((f (/ (cos +half-fov+) (sin +half-fov+)))
         (nf (/ 1.0 (- near far)))
         (m (linalg:full '(4 4) 0.0)))
    (setf (aref m 0 0) (/ f aspect))
    (setf (aref m 1 1) f)
    (setf (aref m 2 2) (* (+ far near) nf))
    (setf (aref m 2 3) (* 2.0 far near nf))
    (setf (aref m 3 2) -1.0)
    m))

(defun build-view (eye target)
  ;; The look-at view matrix for eye -> target: rows are the camera basis
  ;; (right / up / -forward), the translation column is -(basis . eye). The
  ;; basis comes out of a couple of cross products -- kept in local scalars,
  ;; the matrix itself is the linalg (4 4) that combines with the projection.
  (let* ((ex (aref eye 0))
         (ey (aref eye 1))
         (ez (aref eye 2))
         (fx0 (- (aref target 0) ex))
         (fy0 (- (aref target 1) ey))
         (fz0 (- (aref target 2) ez))
         (fl (sqrt (+ (* fx0 fx0) (* fy0 fy0) (* fz0 fz0))))
         (fx (/ fx0 fl))
         (fy (/ fy0 fl))
         (fz (/ fz0 fl))
         ;; right = normalize(forward x up), up = (0 1 0)
         (rx0 (- 0.0 fz))
         (rz0 fx)
         (rl (sqrt (+ (* rx0 rx0) (* rz0 rz0))))
         (rx (/ rx0 rl))
         (rz (/ rz0 rl))
         ;; up = right x forward (right has ry = 0)
         (ux (- 0.0 (* rz fy)))
         (uy (- (* rz fx) (* rx fz)))
         (uz (* rx fy))
         (v (linalg:full '(4 4) 0.0)))
    (setf (aref v 0 0) rx)
    (setf (aref v 0 1) 0.0)
    (setf (aref v 0 2) rz)
    (setf (aref v 0 3) (- 0.0 (+ (* rx ex) (* rz ez))))
    (setf (aref v 1 0) ux)
    (setf (aref v 1 1) uy)
    (setf (aref v 1 2) uz)
    (setf (aref v 1 3) (- 0.0 (+ (* ux ex) (* uy ey) (* uz ez))))
    (setf (aref v 2 0) (- 0.0 fx))
    (setf (aref v 2 1) (- 0.0 fy))
    (setf (aref v 2 2) (- 0.0 fz))
    (setf (aref v 2 3) (+ (* fx ex) (* fy ey) (* fz ez)))
    (setf (aref v 3 3) 1.0)
    v))

;; --- the follow / aim camera --------------------------------------------------
;;
;; A third-person camera orbiting a smoothed copy of the player. The mouse
;; drives the orbit (Pointer Lock on the page), so it doubles as the aim: you
;; face the way the camera looks and fire along that heading.

(defvar *cam* #f(0.0 0.0 0.0)) ; the smoothed follow point
(defconstant +cam-yaw-0+ 0.0)  ; 0 looks along +x, into the field
(defconstant +cam-pitch-0+ 0.32)
(defconstant +cam-dist-0+ 7.5)

(defvar *cam-yaw* +cam-yaw-0+)
(defvar *cam-pitch* +cam-pitch-0+)
(defvar *cam-dist* +cam-dist-0+)
(defvar *eye* #f(0.0 0.0 0.0))
(defvar *aspect* 1.0)
(defvar *vp* nil) ; the current view-projection matrix

;; The aim frame, refreshed from the camera yaw each frame: forward *aimf* is the
;; horizontal heading you face and fire along (y = 0), right *aimr* is
;; forward x up. Both are unit vectors in the ground plane.
(defvar *aimf* #f(1.0 0.0 0.0))
(defvar *aimr* #f(0.0 0.0 1.0))

(defun update-aim ()
  (setq *aimf*
        (linalg:from-list (list (cos *cam-yaw*) 0.0 (sin *cam-yaw*))
                          :element-type 'single-float))
  (setq *aimr*
        (linalg:from-list (list (- 0.0 (sin *cam-yaw*)) 0.0 (cos *cam-yaw*))
                          :element-type 'single-float)))

(defun orbit (dx dy)
  ;; Exported: mouse-look deltas, normalized by the canvas height.
  (setq *cam-yaw* (- *cam-yaw* (* 2.6 dx)))
  (let ((p (+ *cam-pitch* (* 2.4 dy))))
    (setq *cam-pitch* (max 0.08 (min 1.15 p)))))

(defun zoom (dz)
  ;; Exported: scroll-wheel deltas.
  (setq *cam-dist* (max 4.5 (min 13.0 (+ *cam-dist* dz)))))

(defun update-camera (dt)
  ;; smooth the follow point toward the player, then place the eye behind it
  (let ((k (min 1.0 (* 8.0 dt))))
    (setq *cam* (linalg:add *cam* (linalg:mul (linalg:sub *ppos* *cam*) k))))
  (let ((cp (cos *cam-pitch*))
        (sp (sin *cam-pitch*))
        (cy (cos *cam-yaw*))
        (sy (sin *cam-yaw*)))
    (setq *eye*
          (linalg:add *cam*
                      (linalg:from-list (list (- 0.0 (* *cam-dist* cp cy))
                                              (+ 1.1 (* *cam-dist* sp))
                                              (- 0.0 (* *cam-dist* cp sy)))
                                        :element-type 'single-float)))
    (let ((target
           (linalg:add *cam*
                       (linalg:from-list (list (* 2.0 cy) 1.1 (* 2.0 sy))
                                         :element-type 'single-float))))
      (setq *vp*
            (linalg:matmul (mat4-perspective *aspect* 0.1 160.0)
                           (build-view *eye* target))))))

(defun upload-vp (loc)
  ;; WebGL wants column-major, which is the row-major layout of the transpose;
  ;; flatten it into the 16-float scratch and hand it over.
  (let ((flat (linalg:flatten (linalg:transpose *vp*))))
    (dotimes (i 16) (set-float i (aref flat i))))
  (gl-uniform-matrix4fv loc))

;; --- GL pipeline setup --------------------------------------------------------

(defvar *prog* 0)
(defvar *u-vp* 0)
(defvar *u-eye* 0)
(defvar *vao* 0)
(defvar *buf* 0)

;; Rounded parts (ellipsoid heads, tapered limbs, rounded-box armour and the
;; round bolts/sparks/shadows) cost far more triangles per feature than a box,
;; so the capacity is well above the old boxes-only budget; still trivial GPU
;; memory (+max-verts+ * +stride+ bytes ~ 8 MB). It MUST match the page's own
;; staging Float32Array (MAX_VERTS in index.html): the staging array is what
;; setVertex writes into, and a JS typed-array store past the end is silently
;; dropped, so a Lisp-side cap above the page's would lose triangles with no
;; error anywhere.
(defconstant +max-verts+ 190000) ; lit-triangle vertex capacity
(defconstant +stride+ 44)        ; 11 floats per vertex

(defun setup-gl ()
  (setq *prog* (gl:build-program +solid-vs+ +solid-fs+))
  (setq *u-vp* (gl:get-uniform-location *prog* "uVP"))
  (setq *u-eye* (gl:get-uniform-location *prog* "uEye"))
  (gl:enable gl:+depth-test+)
  ;; one VAO: position + normal + color + emissive + shine
  (setq *vao* (gl:create-vertex-array))
  (gl:bind-vertex-array *vao*)
  (setq *buf* (gl:create-buffer))
  (gl:bind-buffer gl:+array-buffer+ *buf*)
  (gl:buffer-data gl:+array-buffer+ (* +max-verts+ +stride+) gl:+dynamic-draw+)
  (gl:enable-vertex-attrib-array 0)
  (gl:vertex-attrib-pointer 0 3 gl:+float+ nil +stride+ 0)
  (gl:enable-vertex-attrib-array 1)
  (gl:vertex-attrib-pointer 1 3 gl:+float+ nil +stride+ 12)
  (gl:enable-vertex-attrib-array 2)
  (gl:vertex-attrib-pointer 2 3 gl:+float+ nil +stride+ 24)
  (gl:enable-vertex-attrib-array 3)
  (gl:vertex-attrib-pointer 3 1 gl:+float+ nil +stride+ 36)
  (gl:enable-vertex-attrib-array 4)
  (gl:vertex-attrib-pointer 4 1 gl:+float+ nil +stride+ 40))

;; --- box tessellation ---------------------------------------------------------
;;
;; Everything in the world is a yaw-rotated box. emit-box builds the eight local
;; corners as the columns of a rank-2 (3 8) matrix, rotates them ALL with one
;; linalg:matmul against the 3x3 yaw matrix, broadcasts the center on with a
;; linalg:add, and stamps the 6 faces (12 triangles) with the face normals read
;; straight off the rotation's columns. Corner index bits: bit 0 = +x, bit 1 =
;; +y, bit 2 = +z in local space; *corners* holds the resulting world (3 8).

(defvar *v* 0)            ; vertex write cursor
(defvar *static-verts* 0) ; the baked snow field, uploaded once

;; Edge softening: a box face can be stamped with per-corner normals bent
;; towards the corner's own outward direction instead of the flat face normal.
;; The silhouette stays a box, but the shading gradient across each face is the
;; one a bevelled edge would produce -- which is most of what separates a
;; "moulded panel" from a "cardboard carton" at gameplay distance, and it costs
;; not one extra vertex. The factor is latched by `soften` and CONSUMED by
;; emit-box (reset to 0 afterwards), so a box that does not ask for softening
;; can never inherit the previous one's.
(defvar *ccx* 0.0)
(defvar *ccy* 0.0)
(defvar *ccz* 0.0)
(defvar *csoft* 0.0)

(defun soften (k) (setq *csoft* k))

(defun emit-v (col nx ny nz)
  ;; stages corner `col` of *corners* with the latched color + the given normal
  (when (< *v* +max-verts+)
    (let ((x (aref *corners* 0 col))
          (y (aref *corners* 1 col))
          (z (aref *corners* 2 col)))
      (if (> *csoft* 0.0)
          (let* ((dx (- x *ccx*))
                 (dy (- y *ccy*))
                 (dz (- z *ccz*))
                 (dl (max 0.000001 (sqrt (+ (* dx dx) (* dy dy) (* dz dz)))))
                 (k *csoft*)
                 (j (- 1.0 k))
                 (mx (+ (* j nx) (* k (/ dx dl))))
                 (my (+ (* j ny) (* k (/ dy dl))))
                 (mz (+ (* j nz) (* k (/ dz dl))))
                 (ml (max 0.000001 (sqrt (+ (* mx mx) (* my my) (* mz mz))))))
            (set-vertex *v* x y z (/ mx ml) (/ my ml) (/ mz ml)))
          (set-vertex *v* x y z nx ny nz))
      (setq *v* (+ *v* 1)))))

(defun emit-face (a b c d nx ny nz)
  (emit-v a nx ny nz)
  (emit-v b nx ny nz)
  (emit-v c nx ny nz)
  (emit-v a nx ny nz)
  (emit-v c nx ny nz)
  (emit-v d nx ny nz))

;; The corner buffer is allocated ONCE and refilled in place. It used to be
;; rebuilt per box as (linalg:add (linalg:matmul rot local) centre) -- elegant,
;; and the right tool for the camera matrices, but the wrong one here: that is
;; four array allocations and a general nested-loop matmul for eight corners,
;; paid for every box in every frame. The cast is now boxes-plus-rounded-boxes
;; in the hundreds per frame, and the allocation dominated the frame. The
;; algebra below is the SAME rotation, written out: yaw only mixes x and z, so
;; the rotation's two interesting columns are the local +x and +z edge vectors,
;; and each corner is the centre plus or minus each of them.
(defvar *corners* (linalg:full '(3 8) 0.0 :element-type 'single-float))

(defun emit-box (cx cy cz hx hy hz yaw)
  (setq *ccx* cx *ccy* cy *ccz* cz)
  (let* ((c (cos yaw))
         (s (sin yaw))
         (axx (* c hx))
         (axz (- 0.0 (* s hx))) ; the local +x edge, in world
         (azx (* s hz))
         (azz (* c hz))) ; the local +z edge, in world
    (dotimes (i 8)
      (let ((sx (if (= (logand i 1) 1) 1.0 -1.0))
            (sy (if (= (logand i 2) 2) hy (- 0.0 hy)))
            (sz (if (= (logand i 4) 4) 1.0 -1.0)))
        (setf (aref *corners* 0 i) (+ cx (* sx axx) (* sz azx)))
        (setf (aref *corners* 1 i) (+ cy sy))
        (setf (aref *corners* 2 i) (+ cz (* sx axz) (* sz azz)))))
    ;; the face normals are the same two edge directions, normalized (yaw only
    ;; tilts x / z, so +y and -y stay axis-aligned)
    (emit-face 4 5 7 6 s 0.0 c)                 ; +z
    (emit-face 1 0 2 3 (- 0.0 s) 0.0 (- 0.0 c)) ; -z
    (emit-face 5 1 3 7 c 0.0 (- 0.0 s))         ; +x
    (emit-face 0 4 6 2 (- 0.0 c) 0.0 s)         ; -x
    (emit-face 6 7 3 2 0.0 1.0 0.0)             ; +y
    (emit-face 0 1 5 4 0.0 -1.0 0.0)            ; -y
    (setq *csoft* 0.0)))

;; --- rounded primitives: cylinders and ellipsoids ------------------------------
;;
;; Boxes read as toy-blocky wherever a real silhouette is round: heads, helmet
;; domes, limbs, gun barrels, blade cores. These three build low-poly meshes
;; with SMOOTH per-vertex normals (the radial direction, not a flat per-face
;; one) -- under the lit shader a smooth normal gradient across a flat
;; triangle reads as curved, so even an 8-sided prism looks like a round rod.
;; No face culling is enabled (setup-gl), so triangle winding never matters
;; here -- only the explicit per-vertex normal does.

(defun emit-vertex (x y z nx ny nz)
  ;; the cursor is checked here (and in emit-v) rather than trusted: a frame
  ;; that overruns the staging array would otherwise upload a slice longer than
  ;; the page's buffer and take the whole canvas down, instead of just dropping
  ;; the triangles nobody budgeted for.
  (when (< *v* +max-verts+)
    (set-vertex *v* x y z nx ny nz)
    (setq *v* (+ *v* 1))))

;; NOTE ON SIGNATURES. The WASM backend's callable types stop at seven
;; parameters; a wider fixed-arity defun still compiles, but only because the
;; compiler rewrites it to bundle the surplus arguments into a freshly consed
;; list at EVERY call site. That is invisible in a cold helper and ruinous in
;; one called per triangle, so every function below that runs per vertex or per
;; triangle is kept at seven parameters or fewer, and the values that are
;; constant across a whole primitive (the ellipsoid's centre/radii/yaw, the
;; rounded box's extents) are latched in globals instead of threaded through
;; the signature. That is why there is no emit-tri: three emit-vertex calls
;; cost nothing, an 18-parameter helper costs twelve cons cells a triangle.

;; A vertical (local +y axis) cylinder: radius r, half-height hy, yaw-rotated
;; and centered at (cx cy cz) exactly like emit-box. Used for upright limbs
;; (thighs, shins, neck).
(defun emit-cylinder (cx cy cz r hy yaw nsides)
  (let ((c (cos yaw)) (s (sin yaw)) (ytop (+ cy hy)) (ybot (- cy hy)))
    (dotimes (i nsides)
      (let* ((a0 (* +two-pi+ (/ (float i) (float nsides))))
             (a1 (* +two-pi+ (/ (float (+ i 1)) (float nsides))))
             (lx0 (cos a0))
             (lz0 (sin a0))
             (lx1 (cos a1))
             (lz1 (sin a1))
             ;; the local radial direction, rotated by yaw -- already unit,
             ;; so it doubles as the smooth side normal
             (n0x (+ (* c lx0) (* s lz0)))
             (n0z (- (* c lz0) (* s lx0)))
             (n1x (+ (* c lx1) (* s lz1)))
             (n1z (- (* c lz1) (* s lx1)))
             (x0 (+ cx (* r n0x)))
             (z0 (+ cz (* r n0z)))
             (x1 (+ cx (* r n1x)))
             (z1 (+ cz (* r n1z))))
        (emit-vertex x0 ytop z0 n0x 0.0 n0z)
        (emit-vertex x0 ybot z0 n0x 0.0 n0z)
        (emit-vertex x1 ybot z1 n1x 0.0 n1z)
        (emit-vertex x0 ytop z0 n0x 0.0 n0z)
        (emit-vertex x1 ybot z1 n1x 0.0 n1z)
        (emit-vertex x1 ytop z1 n1x 0.0 n1z)
        (emit-vertex cx ytop cz 0.0 1.0 0.0)
        (emit-vertex x0 ytop z0 0.0 1.0 0.0)
        (emit-vertex x1 ytop z1 0.0 1.0 0.0)
        (emit-vertex cx ybot cz 0.0 -1.0 0.0)
        (emit-vertex x1 ybot z1 0.0 -1.0 0.0)
        (emit-vertex x0 ybot z0 0.0 -1.0 0.0)))))

;; A horizontal beam cylinder, the round counterpart of emit-arm-to's oriented
;; box: its axis is the yaw-rotated local +x direction (half-len long), radius
;; r in the perpendicular plane. Used for arms, gun barrels and blade/hilt
;; rods -- anything currently built as a "beam between two points".
(defun emit-cyl-beam (cx cy cz half-len r yaw nsides)
  (let* ((c (cos yaw))
         (s (sin yaw))
         (axx c)
         (axz (- 0.0 s)) ; axis direction (matches box hx)
         (perpx s)
         (perpz c)) ; horizontal perpendicular (box hz)
    (dotimes (i nsides)
      (let* ((a0 (* +two-pi+ (/ (float i) (float nsides))))
             (a1 (* +two-pi+ (/ (float (+ i 1)) (float nsides))))
             (ca0 (cos a0))
             (sa0 (sin a0))
             (ca1 (cos a1))
             (sa1 (sin a1))
             (n0x (* sa0 perpx))
             (n0y ca0)
             (n0z (* sa0 perpz))
             (n1x (* sa1 perpx))
             (n1y ca1)
             (n1z (* sa1 perpz))
             (bx0 (+ cx (* r n0x)))
             (by0 (+ cy (* r n0y)))
             (bz0 (+ cz (* r n0z)))
             (bx1 (+ cx (* r n1x)))
             (by1 (+ cy (* r n1y)))
             (bz1 (+ cz (* r n1z)))
             (fx0 (+ bx0 (* half-len axx)))
             (fz0 (+ bz0 (* half-len axz)))
             (kx0 (- bx0 (* half-len axx)))
             (kz0 (- bz0 (* half-len axz)))
             (fx1 (+ bx1 (* half-len axx)))
             (fz1 (+ bz1 (* half-len axz)))
             (kx1 (- bx1 (* half-len axx)))
             (kz1 (- bz1 (* half-len axz))))
        (emit-vertex fx0 by0 fz0 n0x n0y n0z)
        (emit-vertex kx0 by0 kz0 n0x n0y n0z)
        (emit-vertex kx1 by1 kz1 n1x n1y n1z)
        (emit-vertex fx0 by0 fz0 n0x n0y n0z)
        (emit-vertex kx1 by1 kz1 n1x n1y n1z)
        (emit-vertex fx1 by1 fz1 n1x n1y n1z)
        (emit-vertex (+ cx (* half-len axx)) cy (+ cz (* half-len axz)) axx 0.0
                     axz)
        (emit-vertex fx0 by0 fz0 axx 0.0 axz)
        (emit-vertex fx1 by1 fz1 axx 0.0 axz)
        (emit-vertex (- cx (* half-len axx)) cy (- cz (* half-len axz))
                     (- 0.0 axx) 0.0 (- 0.0 axz))
        (emit-vertex kx1 by1 kz1 (- 0.0 axx) 0.0 (- 0.0 axz))
        (emit-vertex kx0 by0 kz0 (- 0.0 axx) 0.0 (- 0.0 axz))))))

;; A yaw-rotated ellipsoid (rx ry rz half-extents; a sphere when they match) --
;; a UV mesh of lon-segs longitude wedges x lat-segs latitude bands, poles
;; included. Non-uniform radii need the inverse-square normal correction
;; (normalize(n/r) rather than n) to stay lit correctly.
;; The ellipsoid being sampled: latched once per primitive so the per-vertex
;; call carries only the unit-sphere direction (see the signature note above).
(defvar *el-cx* 0.0)
(defvar *el-cy* 0.0)
(defvar *el-cz* 0.0)
(defvar *el-rx* 1.0)
(defvar *el-ry* 1.0)
(defvar *el-rz* 1.0)
(defvar *el-c* 1.0)
(defvar *el-s* 0.0)

;; One ellipsoid vertex from a UNIT-sphere direction: scales it into the
;; ellipsoid, yaw-rotates the result into world space, and derives the
;; correctly-lit normal (scale by 1/r, then renormalize) the same way.
(defun ellipsoid-vertex (ux uy uz)
  (let* ((nx (/ ux *el-rx*))
         (ny (/ uy *el-ry*))
         (nz (/ uz *el-rz*))
         (nl (max 0.000001 (sqrt (+ (* nx nx) (* ny ny) (* nz nz)))))
         (nnx (/ nx nl))
         (nny (/ ny nl))
         (nnz (/ nz nl))
         (lx (* ux *el-rx*))
         (lz (* uz *el-rz*))
         (wx (+ *el-cx* (* *el-c* lx) (* *el-s* lz)))
         (wz (+ *el-cz* (- (* *el-c* lz) (* *el-s* lx))))
         (wy (+ *el-cy* (* uy *el-ry*)))
         (rnx (+ (* *el-c* nnx) (* *el-s* nnz)))
         (rnz (- (* *el-c* nnz) (* *el-s* nnx))))
    (emit-vertex wx wy wz rnx nny rnz)))

(defun emit-ellipsoid (cx cy cz rx ry rz yaw lon-segs lat-segs)
  (setq *el-cx* cx *el-cy* cy *el-cz* cz)
  (setq *el-rx* rx *el-ry* ry *el-rz* rz)
  (setq *el-c* (cos yaw) *el-s* (sin yaw))
  (dotimes (j lat-segs)
    (let* ((th0 (* +pi+ (/ (float j) (float lat-segs))))
           (th1 (* +pi+ (/ (float (+ j 1)) (float lat-segs))))
           (y0 (cos th0))
           (rad0 (sin th0))
           (y1 (cos th1))
           (rad1 (sin th1)))
      (dotimes (i lon-segs)
        (let* ((p0 (* +two-pi+ (/ (float i) (float lon-segs))))
               (p1 (* +two-pi+ (/ (float (+ i 1)) (float lon-segs))))
               (cp0 (cos p0))
               (sp0 (sin p0))
               (cp1 (cos p1))
               (sp1 (sin p1))
               ;; the four unit-sphere corners of this lat/lon quad
               (u00x (* rad0 cp0))
               (u00z (* rad0 sp0))
               (u01x (* rad0 cp1))
               (u01z (* rad0 sp1))
               (u10x (* rad1 cp0))
               (u10z (* rad1 sp0))
               (u11x (* rad1 cp1))
               (u11z (* rad1 sp1)))
          (ellipsoid-vertex u00x y0 u00z)
          (ellipsoid-vertex u10x y1 u10z)
          (ellipsoid-vertex u11x y1 u11z)
          (ellipsoid-vertex u00x y0 u00z)
          (ellipsoid-vertex u11x y1 u11z)
          (ellipsoid-vertex u01x y0 u01z))))))

;; --- the articulated limb ------------------------------------------------------
;;
;; A tapered tube (a truncated cone) between two ARBITRARY world points. This is
;; the primitive the older shape vocabulary was missing: emit-cylinder is
;; upright and emit-cyl-beam is horizontal, so every limb built from them had to
;; be a straight vertical post or a straight horizontal plank -- which is
;; exactly why the walkers read as tables and the arms as broomsticks. With a
;; free-standing segment, a leg becomes thigh + knee ball + shin at real angles,
;; and it can taper the way a limb does.
;;
;; The frame: `a` is the unit axis, and t1/t2 are any two unit vectors
;; perpendicular to it (built from a reference vector deliberately chosen NOT to
;; be near-parallel to the axis). The side normal is the radial direction tilted
;; along the axis by the taper's slope, so a strongly-tapered cone is lit as a
;; cone and not as a cylinder.

;; The far-end radius and the ring count are latched rather than passed: the
;; signature is already at the seven-parameter ceiling with the two endpoints
;; and the near radius. `taper` is CONSUMED -- emit-limb resets it after use --
;; so a forgotten taper yields a plain cylinder rather than a stale cone.
(defvar *limb-r1* -1.0) ; < 0 = "same as r0", i.e. no taper
(defvar *limb-n* 8)     ; ring segments, before the LOD scale
(defvar *limb-caps* t)  ; end discs; off for a buried base

(defun taper (r1) (setq *limb-r1* r1))
(defun limb-sides (n) (setq *limb-n* n))
(defun limb-caps (b) (setq *limb-caps* b))

(defvar *lx0* 0.0) ; the limb's cached perpendicular frame
(defvar *ly0* 0.0)
(defvar *lz0* 0.0)
(defvar *lx1* 0.0)
(defvar *ly1* 0.0)
(defvar *lz1* 0.0)

(defun limb-frame (ax ay az)
  ;; t1 = normalize(u x a), t2 = a x t1, for a reference u that is (0 1 0)
  ;; unless the axis is itself near-vertical, in which case (1 0 0).
  (let* ((vert (> (* ay ay) 0.86))
         (ux (if vert 1.0 0.0))
         (uy (if vert 0.0 1.0))
         (p1x (* uy az))
         (p1y (- 0.0 (* ux az)))
         (p1z (- (* ux ay) (* uy ax)))
         (p1l (max 0.000001 (sqrt (+ (* p1x p1x) (* p1y p1y) (* p1z p1z))))))
    (setq *lx0* (/ p1x p1l) *ly0* (/ p1y p1l) *lz0* (/ p1z p1l))
    (setq *lx1* (- (* ay *lz0*) (* az *ly0*)) *ly1*
          (- (* az *lx0*) (* ax *lz0*)) *lz1* (- (* ax *ly0*) (* ay *lx0*)))))

(defun emit-limb (x0 y0 z0 x1 y1 z1 r0)
  (let* ((dx (- x1 x0))
         (dy (- y1 y0))
         (dz (- z1 z0))
         (r1 (if (< *limb-r1* 0.0) r0 *limb-r1*))
         (nsides (segs *limb-n*))
         (len (sqrt (+ (* dx dx) (* dy dy) (* dz dz)))))
    (setq *limb-r1* -1.0) ; consumed
    (when (> len 0.000001)
      (let* ((ax (/ dx len))
             (ay (/ dy len))
             (az (/ dz len))
             (slope (/ (- r0 r1) len)))
        (limb-frame ax ay az)
        (dotimes (i nsides)
          (let* ((a0 (* +two-pi+ (/ (float i) (float nsides))))
                 (a1 (* +two-pi+ (/ (float (+ i 1)) (float nsides))))
                 (c0 (cos a0))
                 (s0 (sin a0))
                 (c1 (cos a1))
                 (s1 (sin a1))
                 ;; the two radial directions bounding this side quad
                 (d0x (+ (* c0 *lx0*) (* s0 *lx1*)))
                 (d0y (+ (* c0 *ly0*) (* s0 *ly1*)))
                 (d0z (+ (* c0 *lz0*) (* s0 *lz1*)))
                 (d1x (+ (* c1 *lx0*) (* s1 *lx1*)))
                 (d1y (+ (* c1 *ly0*) (* s1 *ly1*)))
                 (d1z (+ (* c1 *lz0*) (* s1 *lz1*)))
                 ;; the taper tilts the side normal along the axis
                 (m0x (+ d0x (* slope ax)))
                 (m0y (+ d0y (* slope ay)))
                 (m0z (+ d0z (* slope az)))
                 (m0l
                  (max 0.000001 (sqrt (+ (* m0x m0x) (* m0y m0y) (* m0z m0z)))))
                 (n0x (/ m0x m0l))
                 (n0y (/ m0y m0l))
                 (n0z (/ m0z m0l))
                 (m1x (+ d1x (* slope ax)))
                 (m1y (+ d1y (* slope ay)))
                 (m1z (+ d1z (* slope az)))
                 (m1l
                  (max 0.000001 (sqrt (+ (* m1x m1x) (* m1y m1y) (* m1z m1z)))))
                 (n1x (/ m1x m1l))
                 (n1y (/ m1y m1l))
                 (n1z (/ m1z m1l))
                 (a0x (+ x0 (* r0 d0x)))
                 (a0y (+ y0 (* r0 d0y)))
                 (a0z (+ z0 (* r0 d0z)))
                 (b0x (+ x1 (* r1 d0x)))
                 (b0y (+ y1 (* r1 d0y)))
                 (b0z (+ z1 (* r1 d0z)))
                 (a1x (+ x0 (* r0 d1x)))
                 (a1y (+ y0 (* r0 d1y)))
                 (a1z (+ z0 (* r0 d1z)))
                 (b1x (+ x1 (* r1 d1x)))
                 (b1y (+ y1 (* r1 d1y)))
                 (b1z (+ z1 (* r1 d1z))))
            (emit-vertex a0x a0y a0z n0x n0y n0z)
            (emit-vertex b0x b0y b0z n0x n0y n0z)
            (emit-vertex b1x b1y b1z n1x n1y n1z)
            (emit-vertex a0x a0y a0z n0x n0y n0z)
            (emit-vertex b1x b1y b1z n1x n1y n1z)
            (emit-vertex a1x a1y a1z n1x n1y n1z)
            ;; both ends are capped by default: a limb is nearly always met by
            ;; a joint ball or sunk into a hull, but the one time it is not, an
            ;; open tube shows the inside of the figure through it. A buried
            ;; end (a mountain's base) turns the caps off instead -- a lid
            ;; underground is not just wasted, it is the thing that surfaces as
            ;; a dark plate the moment the cone leans.
            (when *limb-caps*
              (emit-vertex x0 y0 z0 (- 0.0 ax) (- 0.0 ay) (- 0.0 az))
              (emit-vertex a1x a1y a1z (- 0.0 ax) (- 0.0 ay) (- 0.0 az))
              (emit-vertex a0x a0y a0z (- 0.0 ax) (- 0.0 ay) (- 0.0 az))
              (emit-vertex x1 y1 z1 ax ay az)
              (emit-vertex b0x b0y b0z ax ay az)
              (emit-vertex b1x b1y b1z ax ay az))))))))

;; --- the rounded box ------------------------------------------------------------
;;
;; A box with genuinely rounded edges and corners: the Minkowski sum of a
;; smaller "core" box (half-extents shrunk by the fillet radius br) and a sphere
;; of radius br. Sampling it is one uniform rule -- take a point P on the outer
;; box, clamp it into the core box to get q, and place the surface at
;; q + br * normalize(P - q), whose normal is exactly that same normalized
;; direction. On a face interior the clamp does nothing but move q inward by br,
;; so the flat face comes back exactly; near an edge or a corner the direction
;; swings and traces the fillet. Splitting each face's parameter range at the
;; core boundary (-h, -a, +a, +h) puts the sample lines exactly where the
;; curvature starts, so a 3x3 grid per face is enough: one flat centre quad,
;; four edge fillets, four corner fillets.
;;
;; This is what armour plates, hulls, packs and boots want -- shapes that ARE
;; boxes but were never machined with knife edges.

(defvar *rb-hx* 1.0) ; the box being sampled (local frame)
(defvar *rb-hy* 1.0)
(defvar *rb-hz* 1.0)
(defvar *rb-ax* 1.0) ; ... and its core half-extents
(defvar *rb-ay* 1.0)
(defvar *rb-az* 1.0)
(defvar *rb-br* 0.1)
(defvar *rb-cx* 0.0)
(defvar *rb-cy* 0.0)
(defvar *rb-cz* 0.0)
(defvar *rb-c* 1.0)
(defvar *rb-s* 0.0)

(defun clamp1 (v lim) (max (- 0.0 lim) (min lim v)))

(defun rbox-vertex (px py pz)
  ;; one sample of the rounded surface, from a point on the *outer* box
  (let* ((qx (clamp1 px *rb-ax*))
         (qy (clamp1 py *rb-ay*))
         (qz (clamp1 pz *rb-az*))
         (dx (- px qx))
         (dy (- py qy))
         (dz (- pz qz))
         (dl (max 0.000001 (sqrt (+ (* dx dx) (* dy dy) (* dz dz)))))
         (nx (/ dx dl))
         (ny (/ dy dl))
         (nz (/ dz dl))
         (lx (+ qx (* *rb-br* nx)))
         (ly (+ qy (* *rb-br* ny)))
         (lz (+ qz (* *rb-br* nz))))
    (emit-vertex (+ *rb-cx* (* *rb-c* lx) (* *rb-s* lz)) (+ *rb-cy* ly)
                 (+ *rb-cz* (- (* *rb-c* lz) (* *rb-s* lx)))
                 (+ (* *rb-c* nx) (* *rb-s* nz)) ny
                 (- (* *rb-c* nz) (* *rb-s* nx)))))

;; one quad of a face, given in that face's (u v) parameters plus the pinned
;; coordinate `f`; `axis` selects the permutation back to (x y z).
(defun rbox-quad (axis f u0 v0 u1 v1)
  (cond ((= axis 0)
         (rbox-vertex f u0 v0)
         (rbox-vertex f u1 v0)
         (rbox-vertex f u1 v1)
         (rbox-vertex f u0 v0)
         (rbox-vertex f u1 v1)
         (rbox-vertex f u0 v1))
        ((= axis 1)
         (rbox-vertex u0 f v0)
         (rbox-vertex u1 f v0)
         (rbox-vertex u1 f v1)
         (rbox-vertex u0 f v0)
         (rbox-vertex u1 f v1)
         (rbox-vertex u0 f v1))
        (t
         (rbox-vertex u0 v0 f)
         (rbox-vertex u1 v0 f)
         (rbox-vertex u1 v1 f)
         (rbox-vertex u0 v0 f)
         (rbox-vertex u1 v1 f)
         (rbox-vertex u0 v1 f))))

;; one face of the outer box, as a 3x3 grid split at the core boundary. The
;; face is addressed through a small permutation: `axis` says which coordinate
;; is pinned to the face (0 = x, 1 = y, 2 = z) and `sgn` which side.
(defun rbox-face (axis sgn)
  (let* ((f
          (if (= axis 0)
              (* sgn *rb-hx*)
              (if (= axis 1) (* sgn *rb-hy*) (* sgn *rb-hz*))))
         ;; the two in-plane axes and their split points
         (uh (if (= axis 0) *rb-hy* *rb-hx*))
         (ua (if (= axis 0) *rb-ay* *rb-ax*))
         (vh (if (= axis 2) *rb-hy* *rb-hz*))
         (va (if (= axis 2) *rb-ay* *rb-az*)))
    (dotimes (iu 3)
      (let ((u0 (if (= iu 0) (- 0.0 uh) (if (= iu 1) (- 0.0 ua) ua)))
            (u1 (if (= iu 0) (- 0.0 ua) (if (= iu 1) ua uh))))
        (dotimes (iv 3)
          (let ((v0 (if (= iv 0) (- 0.0 vh) (if (= iv 1) (- 0.0 va) va)))
                (v1 (if (= iv 0) (- 0.0 va) (if (= iv 1) va vh))))
            (rbox-quad axis f u0 v0 u1 v1)))))))

;; The yaw is latched (see the signature note): every call site is a figure
;; part, and the figure's frame already knows its heading.
(defun rbox-yaw (yaw) (setq *rb-c* (cos yaw) *rb-s* (sin yaw)))

(defun emit-rbox (cx cy cz hx hy hz br)
  (let ((r (max 0.004 (min br (* 0.98 (min hx (min hy hz)))))))
    (setq *rb-cx* cx *rb-cy* cy *rb-cz* cz)
    (setq *rb-hx* hx *rb-hy* hy *rb-hz* hz *rb-br* r)
    (setq *rb-ax* (max 0.0 (- hx r)) *rb-ay* (max 0.0 (- hy r)) *rb-az*
          (max 0.0 (- hz r)))
    (rbox-face 0 1.0)
    (rbox-face 0 -1.0)
    (rbox-face 1 1.0)
    (rbox-face 1 -1.0)
    (rbox-face 2 1.0)
    (rbox-face 2 -1.0)))

;; --- level of detail -----------------------------------------------------------
;;
;; A rounded box is nine quads a face and a limb is a ring per side, so the cast
;; costs far more per figure than the old all-boxes one did. What keeps the
;; frame honest is that detail nobody can resolve is not drawn: *lod* is picked
;; per figure from its APPARENT size (world height over distance to the eye), so
;; the trooper in your face is fully machined and the one across the arena is
;; the same model at a coarser ring count with its fillets dropped. `segs`
;; scales a ring/band count, and `roundp` is the switch a part uses to fall back
;; from a rounded box to a soft-normalled plain one.

(defvar *lod* 2)

(defun set-lod (x z size)
  (let* ((dx (- x (aref *eye* 0)))
         (dy (- 1.0 (aref *eye* 1)))
         (dz (- z (aref *eye* 2)))
         (d (max 0.5 (sqrt (+ (* dx dx) (* dy dy) (* dz dz)))))
         (k (/ size d)))
    ;; the top tier is deliberately narrow: a rounded box is nine quads a face,
    ;; so full detail is worth paying for only on a figure that genuinely fills
    ;; part of the screen. At the default follow distance that is you, whoever
    ;; is in melee range, and a walker you are standing under
    (setq *lod* (cond ((> k 0.32) 2) ((> k 0.075) 1) (t 0)))))

(defun segs (n)
  (cond ((>= *lod* 2) n)
        ((= *lod* 1) (max 4 (floor (* 0.7 (float n)))))
        (t (max 3 (floor (* 0.45 (float n)))))))

(defun roundp () (>= *lod* 2))

;; A local frame for composite figures: set-origin latches a world position +
;; yaw, and part emits one box given in that frame's local coordinates. The
;; character part-lists below stay in readable scalar coordinates -- they are
;; fixed geometry offsets, not the game's moving state.
(defvar *ox* 0.0)
(defvar *oy* 0.0)
(defvar *oz* 0.0)
(defvar *oyaw* 0.0)
(defvar *oc* 1.0)
(defvar *os* 0.0)

(defun set-origin (x y z yaw)
  (setq *ox* x)
  (setq *oy* y)
  (setq *oz* z)
  (setq *oyaw* yaw)
  (setq *oc* (cos yaw))
  (setq *os* (sin yaw))
  (rbox-yaw yaw))

;; local -> world, one coordinate at a time (a limb needs both endpoints
;; converted, and this Lisp has no cheap 3-value return worth the ceremony)
(defun lwx (lx lz) (+ *ox* (* lx *oc*) (* lz *os*)))
(defun lwy (ly) (+ *oy* ly))
(defun lwz (lx lz) (+ *oz* (- (* lz *oc*) (* lx *os*))))

;; The figure-local wrappers. There is no plain `part` any more: with the
;; rounded box, the tapered limb and the joint ball in the vocabulary, nothing
;; in the cast turned out to want a bare yaw-rotated box -- the handful of
;; genuinely machined slabs left (a pistol's receiver, its grip) call emit-box
;; directly, preceded by `soften`.
(defun part-cyl (lx ly lz r hy &optional (nsides 8))
  (emit-cylinder (lwx lx lz) (lwy ly) (lwz lx lz) r hy *oyaw* (segs nsides)))

(defun part-ellipsoid (lx ly lz rx ry rz &optional (lon-segs 8) (lat-segs 5))
  (emit-ellipsoid (lwx lx lz) (lwy ly) (lwz lx lz) rx ry rz *oyaw*
                  (segs lon-segs) (segs lat-segs)))

;; A rounded box in the local frame -- when the fillet is actually worth nine
;; times the triangles of a plain box, which is a question about THIS PART, not
;; about the figure it belongs to. A torso at arm's length earns its fillet; the
;; 2cm brow ridge on the same figure never will, and neither will anything at
;; all on a trooper across the arena. So the test is the part's own apparent
;; size -- its largest half-extent over its distance to the eye -- with the
;; figure's detail tier as a floor. Everything below falls back to the plain
;; box with soft edge normals, which at that size is indistinguishable.
(defun rbox-worth-it (x y z h)
  (let* ((dx (- x (aref *eye* 0)))
         (dy (- y (aref *eye* 1)))
         (dz (- z (aref *eye* 2)))
         (d (max 0.5 (sqrt (+ (* dx dx) (* dy dy) (* dz dz))))))
    (> (/ h d) 0.013)))

(defun part-rbox (lx ly lz hx hy hz br)
  (let ((wx (lwx lx lz)) (wy (lwy ly)) (wz (lwz lx lz)))
    (if (and (roundp) (rbox-worth-it wx wy wz (max hx (max hy hz))))
        (emit-rbox wx wy wz hx hy hz br)
        (progn
          (soften 0.5)
          (emit-box wx wy wz hx hy hz *oyaw*)))))

;; A free-standing tapered segment between two LOCAL points -- the articulated
;; limb in figure coordinates. Precede it with (taper r1) for a cone.
(defun part-limb (lx0 ly0 lz0 lx1 ly1 lz1 r0)
  (emit-limb (lwx lx0 lz0) (lwy ly0) (lwz lx0 lz0) (lwx lx1 lz1) (lwy ly1)
             (lwz lx1 lz1) r0))

;; A joint ball at a local point -- what turns two limb segments into a knee or
;; a shoulder instead of two sticks that happen to touch.
(defun part-joint (lx ly lz r)
  ;; coarser than a head or a helmet on purpose: a joint ball is small on
  ;; screen and there are a dozen of them per figure
  (emit-ellipsoid (lwx lx lz) (lwy ly) (lwz lx lz) r r r *oyaw* (segs 6)
                  (segs 3)))

;; colour + emissive + shine helpers
;; *hit-tint* (0..1) reddens and lights whatever `col` draws next -- set around
;; an enemy's body while its damage flash is active, 0 otherwise, so a struck
;; enemy glows red without touching any call site.
(defvar *hit-tint* 0.0)
(defun col (r g b &optional (shine 0.05))
  ;; shine defaults low (cloth/skin/snow); pass an explicit shine for armor
  ;; plates, helmet domes, gun metal and blade hilts -- see (metal ...) below.
  (set-shine shine)
  (if (> *hit-tint* 0.0)
      (let ((k *hit-tint*))
        (set-emissive (* 0.55 k))
        (set-color (+ r (* k (- 1.0 r))) (* g (- 1.0 k)) (* b (- 1.0 k))))
      (progn
        (set-emissive 0.0)
        (set-color r g b))))
(defun metal (r g b) (col r g b 0.85))
(defun glow-col (r g b)
  (set-shine 0.0)
  (set-emissive 1.0)
  (set-color r g b))

;; --- the snow field -----------------------------------------------------------
;;
;; Static geometry, baked once: the snow plate, low drifts, the ring of Hoth
;; mountains (all boxes -- broken terrain reads fine as facets), plus ice
;; boulders and clouds (ellipsoids -- the two static shapes that actually read
;; as round in life, so worth the baked-once extra triangles). Each block is
;; (x0 y0 z0 x1 y1 z1 r g b); nothing here collides -- the arena is open.

;; The snow plate. It runs far beyond the mountains, not merely up to them: the
;; plate is a solid slab, so wherever its rim falls inside the view you see the
;; slab's own edge and underside -- mid-grey and near-black under this light --
;; ruled across the snow. Pushing the rim past the fog's saturation distance is
;; what makes the field read as endless. It is also kept thin, so a stray
;; sightline under a peak sees as little of the edge as possible.
(defconstant +scenery+ '((-260.0 -0.6 -260.0 300.0 0.0 260.0 0.90 0.93 0.98)))

;; The horizon used to be four tall boxes, which from inside the arena is a
;; flat grey wall with two conspicuous vertical corners running up the sky. It
;; is a broken RIDGE now: squat cones whose apexes are nudged off centre and
;; whose skirts overlap, at jittered sizes, so the skyline is a range. All of
;; it is baked once with the rest of the field, so the triangles cost nothing
;; per frame.
(defconstant +peak-base+ -2.6) ; how far the cone is planted below the snow

(defun emit-peak (px pz r h)
  ;; The apex leans off the base's centre -- that lean is the difference
  ;; between a mountain and a party hat. Two bounds on it: a share of the
  ;; SMALLER of radius and height, or a squat wide cone leans over into a
  ;; wedge; and, decisively, little enough that the (tilted) base ring stays
  ;; buried. Tilting a cone tilts its base ring with it, and the ring's uphill
  ;; edge rises by about radius * lean / height -- for a 25m-wide apron that is
  ;; metres, so an unbounded lean lifts the buried end clean out of the snow.
  (let* ((rise (- h +peak-base+))
         (lean
          (min (* 0.30 (min r h)) (* 0.85 (/ (* (- 0.0 +peak-base+) rise) r)))))
    (col (rand-range 0.79 0.87) (rand-range 0.85 0.91) (rand-range 0.93 0.98))
    (limb-sides 7)
    (limb-caps nil) ; the base is underground, the tip a point
    (taper 0.0)     ; a true point, so no lid is missed
    (emit-limb px +peak-base+ pz (+ px (rand-range (- 0.0 lean) lean)) h
               (+ pz (rand-range (- 0.0 lean) lean)) r)
    (limb-caps t)))

;; The apron the ranges stand on: very wide, very low cones. A box shelf would
;; do the same job of stopping daylight showing between the peaks' skirts, but
;; a box has vertical faces, and the one facing away from the sun becomes a
;; dark grey band ruled across the horizon -- exactly the wall the peaks were
;; brought in to replace. A cone has no vertical face; every normal on it
;; points mostly up, so the whole apron stays snow-bright from any angle.
(defun emit-apron (x0 z0 dx dz n)
  (dotimes (i n)
    (let ((t0 (float i)))
      (emit-peak (+ x0 (* dx t0) (rand-range -8.0 8.0))
                 (+ z0 (* dz t0) (rand-range -8.0 8.0)) (rand-range 22.0 34.0)
                 (rand-range 1.5 3.4)))))

(defun emit-peaks (x0 z0 dx dz n hlo hhi)
  ;; the perpendicular jitter is deliberately as large as the step: a range
  ;; laid out on a straight line reads as a fence of cones
  (dotimes (i n)
    (let ((t0 (float i)))
      (emit-peak (+ x0 (* dx t0) (rand-range -9.0 9.0))
                 (+ z0 (* dz t0) (rand-range -9.0 9.0)) (rand-range 9.0 20.0)
                 (rand-range hlo hhi)))))

(defun emit-block (b)
  (col (nth 6 b) (nth 7 b) (nth 8 b))
  (emit-box (* 0.5 (+ (nth 0 b) (nth 3 b))) (* 0.5 (+ (nth 1 b) (nth 4 b)))
            (* 0.5 (+ (nth 2 b) (nth 5 b))) (* 0.5 (- (nth 3 b) (nth 0 b)))
            (* 0.5 (- (nth 4 b) (nth 1 b))) (* 0.5 (- (nth 5 b) (nth 2 b)))
            0.0))

;; low snow drifts: smooth wind-blown mounds, not plateaus -- same
;; (x0 y0 z0 x1 y1 z1 r g b) AABB shape as a +scenery+ block, but drawn as a
;; squashed ellipsoid whose base sits at y0 (flush with the snow plate) and
;; crests at y1, so it reads as a dome rather than a box. Baked once, so the
;; extra triangles over emit-block are free.
(defconstant +drifts+
  '((6.0 0.0 -14.0 12.0 0.7 -9.0 0.95 0.97 1.00)
    (18.0 0.0 10.0 25.0 0.9 15.0 0.95 0.97 1.00)
    (-8.0 0.0 6.0 -2.0 0.6 11.0 0.95 0.97 1.00)
    (34.0 0.0 -10.0 41.0 1.0 -4.0 0.95 0.97 1.00)
    (28.0 0.0 18.0 36.0 0.8 24.0 0.95 0.97 1.00)))

(defun emit-drift (b)
  (col (nth 6 b) (nth 7 b) (nth 8 b))
  (emit-ellipsoid (* 0.5 (+ (nth 0 b) (nth 3 b))) (nth 1 b)
                  (* 0.5 (+ (nth 2 b) (nth 5 b)))
                  (* 0.5 (- (nth 3 b) (nth 0 b))) (- (nth 4 b) (nth 1 b))
                  (* 0.5 (- (nth 5 b) (nth 2 b))) 0.0 12 6))

;; ice boulders (cold blue-grey): center + radius + a squash/yaw so a plain
;; sphere reads as a lumpy rock, not a ball-bearing.
(defconstant +boulders+
  '((3.2 0.9 13.2 1.2 0.62 0.9) (14.9 0.65 -5.1 0.9 0.55 -0.9)
    (31.3 1.1 5.3 1.3 0.68 1.7) (-5.0 0.75 -11.0 1.0 0.60 -1.3)
    (45.5 1.3 9.5 1.5 0.75 0.4)))

(defun emit-boulder (b)
  (col 0.64 0.72 0.81)
  (emit-ellipsoid (nth 0 b) (nth 1 b) (nth 2 b) (nth 3 b) (nth 4 b)
                  (* 0.9 (nth 3 b)) (nth 5 b) 10 6))

;; a few clouds: each is a small cluster of overlapping ellipsoid puffs
;; (center + half-width/height/depth), fluffier than one stretched blob.
(defconstant +clouds+
  '((15.0 16.0 -27.5 5.0 1.0 2.5) (39.5 18.0 25.0 5.5 1.0 3.0)
    (-9.0 17.0 7.0 5.0 1.0 3.0)))

(defun emit-cloud (c)
  (let ((cx (nth 0 c))
        (cy (nth 1 c))
        (cz (nth 2 c))
        (w (nth 3 c))
        (h (nth 4 c))
        (d (nth 5 c)))
    (col 0.99 0.99 1.00)
    (emit-ellipsoid cx cy cz (* 0.55 w) h (* 0.85 d) 0.0 10 5)
    (emit-ellipsoid (- cx (* 0.32 w)) (- cy (* 0.15 h)) cz (* 0.40 w) (* 0.75 h)
                    (* 0.70 d) 0.0 8 4)
    (emit-ellipsoid (+ cx (* 0.34 w)) (- cy (* 0.10 h)) cz (* 0.42 w) (* 0.80 h)
                    (* 0.72 d) 0.0 8 4)
    (emit-ellipsoid cx (+ cy (* 0.35 h)) (+ cz (* 0.10 d)) (* 0.36 w) (* 0.70 h)
                    (* 0.60 d) 0.0 8 4)))

(defun bake-static ()
  (setq *v* 0)
  (dolist (b +scenery+) (emit-block b))
  ;; the apron first, then the four ranges: each is a near row of low hills
  ;; with a far row of tall peaks behind it, so the horizon has depth instead
  ;; of one silhouette
  (emit-apron -90.0 -66.0 22.0 0.0 10)
  (emit-apron -90.0 66.0 22.0 0.0 10)
  (emit-apron -70.0 -66.0 0.0 20.0 8)
  (emit-apron 90.0 -66.0 0.0 20.0 8)
  (emit-peaks -84.0 -60.0 13.0 0.0 15 5.0 11.0)
  (emit-peaks -84.0 -76.0 13.0 0.0 15 12.0 24.0)
  (emit-peaks -84.0 60.0 13.0 0.0 15 5.0 10.0)
  (emit-peaks -84.0 76.0 13.0 0.0 15 11.0 22.0)
  (emit-peaks -62.0 -60.0 0.0 12.0 11 5.0 11.0)
  (emit-peaks -80.0 -60.0 0.0 12.0 11 12.0 25.0)
  (emit-peaks 82.0 -60.0 0.0 12.0 11 5.0 11.0)
  (emit-peaks 100.0 -60.0 0.0 12.0 11 13.0 26.0)
  (limb-sides 8) ; restore the default ring count
  (dolist (b +drifts+) (emit-drift b))
  (dolist (b +boulders+) (emit-boulder b))
  (dolist (c +clouds+) (emit-cloud c))
  (setq *static-verts* *v*)
  (gl:bind-buffer gl:+array-buffer+ *buf*)
  (gl-upload-vertices 0 (* *static-verts* 11)))

;; --- the player (you) ---------------------------------------------------------

(defconstant +run-speed+ 6.0)
(defconstant +field-min-x+ -34.0)
(defconstant +field-max-x+ 56.0)
(defconstant +field-min-z+ -34.0)
(defconstant +field-max-z+ 34.0)
(defconstant +player-max-hp+ 120.0)
(defconstant +gravity+ 26.0)
(defconstant +jump-v+ 9.2) ; a roomy hop, apex ~1.6
(defconstant +invuln+ 0.6) ; i-frames after taking a hit

(defvar *ppos* #f(0.0 0.0 0.0)) ; player position
(defvar *pvel* #f(0.0 0.0 0.0)) ; velocity (y = vertical, jump / gravity)
(defvar *grounded* t)
(defvar *pyaw* 0.0) ; render facing = -cam-yaw
(defvar *php* 120.0)
(defvar *inv-t* 0.0)      ; invulnerability countdown
(defvar *hurt-flash* 0.0) ; brief red vignette timer (HUD)
(defvar *run-phase* 0.0)

(defvar *weapon* 1)   ; 0 lightsaber, 1 blaster (start armed with the blaster)
(defvar *attack* 0.0) ; 1.0 while the mouse button is held
(defvar *attack-prev* nil)
(defvar *swing-t* 0.0) ; lightsaber swing countdown
(defvar *swing-cd* 0.0)
(defvar *swing-hit* nil) ; damage applied once per swing
(defvar *fire-cd* 0.0)   ; blaster cadence

;; game state: 0 playing, 1 victory, 2 defeat
(defvar *state* 0)
(defvar *state-t* 0.0)
(defvar *last-tm* 0.0)
(defvar *pending-reset* nil)

;; keyboard state, forwarded by the page: 1.0 while held
(defvar *in-l* 0.0)    ; A
(defvar *in-r* 0.0)    ; D
(defvar *in-f* 0.0)    ; W
(defvar *in-b* 0.0)    ; S
(defvar *in-jump* 0.0) ; Space
(defvar *jump-prev* nil)

(defun set-key (code down)
  ;; Exported: 0 = A, 1 = D, 2 = W, 3 = S, 4 = Space; down is 1 or 0.
  (let ((val (if (= down 1) 1.0 0.0)))
    (cond ((= code 0) (setq *in-l* val))
          ((= code 1) (setq *in-r* val))
          ((= code 2) (setq *in-f* val))
          ((= code 3) (setq *in-b* val))
          ((= code 4) (setq *in-jump* val)))))

(defun set-attack (down)
  ;; Exported: mouse button, 1 down / 0 up.
  (setq *attack* (if (= down 1) 1.0 0.0)))

(defun switch-weapon ()
  ;; Exported: F toggles the lightsaber and the blaster.
  (setq *weapon* (if (= *weapon* 0) 1 0)))

(defun restart () (setq *pending-reset* t))

;; --- blaster bolts ------------------------------------------------------------
;;
;; One pool shared by you, the troopers and the walkers. owner 0 = player,
;; 1 = enemy. Player bolts damage enemies; enemy bolts damage you unless your
;; lightsaber swing deflects them. Every bolt is a glowing capsule; position and
;; velocity are #f vectors, the rest scalars.

(defconstant +nbolt+ 48)
(defvar *bpos* (make-array +nbolt+ :initial-element nil))
(defvar *bvel* (make-array +nbolt+ :initial-element nil))
(defvar *bt* (make-array +nbolt+ :initial-element 0.0)) ; life left
(defvar *bown* (make-array +nbolt+ :initial-element 0)) ; 0 player / 1 enemy
(defvar *bdmg* (make-array +nbolt+ :initial-element 0.0))
(defvar *bsz* (make-array +nbolt+ :initial-element 1.0)) ; visual scale
(defvar *br* (make-array +nbolt+ :initial-element 1.0))
(defvar *bg* (make-array +nbolt+ :initial-element 1.0))
(defvar *bb* (make-array +nbolt+ :initial-element 1.0))
(defvar *balive* (make-array +nbolt+ :initial-element nil))

(defun spawn-bolt (pos dir speed owner dmg sz r g b)
  ;; dir is a (not necessarily unit) heading vector; normalise and launch.
  (let ((len (linalg:norm dir)) (slot -1))
    (dotimes (i +nbolt+)
      (when (and (< slot 0) (not (aref *balive* i))) (setq slot i)))
    (when (and (>= slot 0) (> len 0.0001))
      (setf (aref *bpos* slot) pos)
      (setf (aref *bvel* slot) (linalg:mul dir (/ speed len)))
      (setf (aref *bt* slot) 3.0)
      (setf (aref *bown* slot) owner)
      (setf (aref *bdmg* slot) dmg)
      (setf (aref *bsz* slot) sz)
      (setf (aref *br* slot) r)
      (setf (aref *bg* slot) g)
      (setf (aref *bb* slot) b)
      (setf (aref *balive* slot) t))))

;; --- flashes (muzzle, hit sparks, explosions) ---------------------------------
;;
;; Short-lived additive glow puffs, drawn in the bloom pass.

(defconstant +nflash+ 24)
(defvar *fpos* (make-array +nflash+ :initial-element nil))
(defvar *ft* (make-array +nflash+ :initial-element 0.0))
(defvar *fttl* (make-array +nflash+ :initial-element 0.3))
(defvar *fsz* (make-array +nflash+ :initial-element 0.4))
(defvar *fr* (make-array +nflash+ :initial-element 1.0))
(defvar *fg* (make-array +nflash+ :initial-element 1.0))
(defvar *fb* (make-array +nflash+ :initial-element 1.0))

(defun spawn-flash (pos ttl sz r g b)
  (let ((slot -1))
    (dotimes (i +nflash+)
      (when (and (< slot 0) (<= (aref *ft* i) 0.0)) (setq slot i)))
    (when (>= slot 0)
      (setf (aref *fpos* slot) pos)
      (setf (aref *ft* slot) ttl)
      (setf (aref *fttl* slot) ttl)
      (setf (aref *fsz* slot) sz)
      (setf (aref *fr* slot) r)
      (setf (aref *fg* slot) g)
      (setf (aref *fb* slot) b))))

(defun update-flashes (dt)
  (dotimes (i +nflash+)
    (when (> (aref *ft* i) 0.0) (setf (aref *ft* i) (- (aref *ft* i) dt)))))

;; --- victory fireworks --------------------------------------------------------
;;
;; A particle pool: on victory, bursts launch over the arena, each scattering a
;; shell of glowing sparks that arc under a little gravity and fade. All drawn in
;; the additive bloom pass.

(defconstant +nfw+ 128)
(defvar *fwpos* (make-array +nfw+ :initial-element nil))
(defvar *fwvel* (make-array +nfw+ :initial-element nil))
(defvar *fwt* (make-array +nfw+ :initial-element 0.0)) ; life left
(defvar *fwmax* (make-array +nfw+ :initial-element 1.0))
(defvar *fwr* (make-array +nfw+ :initial-element 1.0))
(defvar *fwg* (make-array +nfw+ :initial-element 1.0))
(defvar *fwb* (make-array +nfw+ :initial-element 1.0))
(defvar *fw-launch* 0.0) ; time to the next burst

(defun spawn-firework (center r g b)
  ;; scatter a shell of sparks from the burst point
  (dotimes (n 26)
    (let ((slot -1))
      (dotimes (i +nfw+)
        (when (and (< slot 0) (<= (aref *fwt* i) 0.0)) (setq slot i)))
      (when (>= slot 0)
        (let ((spd (rand-range 3.5 7.5)) (life (rand-range 1.3 2.2)))
          (setf (aref *fwpos* slot) center)
          (setf (aref *fwvel* slot)
                (linalg:from-list (list (* (- (rand01) 0.5) 2.0 spd)
                                        (* (- (rand01) 0.35) 2.0 spd)
                                        (* (- (rand01) 0.5) 2.0 spd))
                                  :element-type 'single-float))
          (setf (aref *fwt* slot) life)
          (setf (aref *fwmax* slot) life)
          (setf (aref *fwr* slot) r)
          (setf (aref *fwg* slot) g)
          (setf (aref *fwb* slot) b))))))

(defun launch-firework ()
  ;; a burst in view above the arena, in one of a few bright colors -- kept low
  ;; and near the player so it fills the frame rather than drifting off-screen
  (let ((center
         (linalg:from-list (list (+ (aref *ppos* 0) (rand-range -9.0 9.0))
                                 (rand-range 3.5 7.5)
                                 (+ (aref *ppos* 2) (rand-range -9.0 9.0)))
                           :element-type 'single-float))
        (c (floor (* 6.0 (rand01)))))
    (cond ((= c 0) (spawn-firework center 1.0 0.28 0.28)) ; red
          ((= c 1) (spawn-firework center 1.0 0.78 0.20)) ; gold
          ((= c 2) (spawn-firework center 0.30 0.68 1.0)) ; blue
          ((= c 3) (spawn-firework center 0.35 1.0 0.40)) ; green
          ((= c 4) (spawn-firework center 0.82 0.40 1.0)) ; violet
          (t (spawn-firework center 1.0 1.0 0.9)))))      ; white

(defun update-fireworks (dt)
  (let ((g
         (linalg:from-list (list 0.0 (* -3.2 dt) 0.0)
                           :element-type 'single-float))) ; one gravity impulse, reused
    (dotimes (i +nfw+)
      (when (> (aref *fwt* i) 0.0)
        (setf (aref *fwvel* i) (linalg:add (aref *fwvel* i) g))
        (setf (aref *fwpos* i)
              (linalg:add (aref *fwpos* i) (linalg:mul (aref *fwvel* i) dt)))
        (setf (aref *fwt* i) (- (aref *fwt* i) dt))))))

;; --- stormtroopers ------------------------------------------------------------

(defconstant +trooper-list+
  '((11.0 -5.0) (15.0 4.0) (13.0 -1.0) (20.0 -7.0) (19.0 8.0) (25.0 2.0)))

(defconstant +trooper-hp+ 20.0)
(defconstant +trooper-speed+ 2.6)
(defconstant +trooper-standoff+ 5.5) ; how close they press before holding

(defvar *ntrooper* 0)
(defvar *tpos* nil) ; array of ground positions (#f vectors)
(defvar *thp* nil)
(defvar *tyaw* nil)
(defvar *tfire* nil) ; cooldown to the next shot
(defvar *talive* nil)
(defvar *tstep* nil) ; walk-cycle phase
(defvar *thit* nil)  ; damage-flash timer

(defun parse-troopers ()
  (setq *ntrooper* (length +trooper-list+))
  (setq *tpos* (make-array *ntrooper* :initial-element nil))
  (setq *thp* (make-array *ntrooper* :initial-element 0.0))
  (setq *tyaw* (make-array *ntrooper* :initial-element 0.0))
  (setq *tfire* (make-array *ntrooper* :initial-element 0.0))
  (setq *talive* (make-array *ntrooper* :initial-element t))
  (setq *tstep* (make-array *ntrooper* :initial-element 0.0))
  (setq *thit* (make-array *ntrooper* :initial-element 0.0)))

(defun reset-troopers ()
  (let ((i 0))
    (dolist (e +trooper-list+)
      (setf (aref *tpos* i)
            (linalg:from-list (list (nth 0 e) 0.0 (nth 1 e))
                              :element-type 'single-float))
      (setf (aref *thp* i) +trooper-hp+)
      (setf (aref *tyaw* i) 0.0)
      (setf (aref *tfire* i) (rand-range 1.4 3.4))
      (setf (aref *talive* i) t)
      (setf (aref *tstep* i) 0.0)
      (setf (aref *thit* i) 0.0)
      (setq i (+ i 1)))))

(defun troopers-alive ()
  (let ((n 0))
    (dotimes (i *ntrooper*) (when (aref *talive* i) (setq n (+ n 1))))
    n))

;; Your ground position (y = 0), shared by the enemy AI: distances and
;; headings on the snow ignore his jump height.
(defun player-ground ()
  (linalg:from-list (list (aref *ppos* 0) 0.0 (aref *ppos* 2))
                    :element-type 'single-float))

(defun update-troopers (dt)
  (let ((pg (player-ground)))
    (dotimes (i *ntrooper*)
      (when (aref *talive* i)
        (let* ((p (aref *tpos* i))
               (to (linalg:sub pg p)) ; horizontal offset to the player
               (d (linalg:norm to)))
          (setf (aref *tyaw* i) (atan2 (- 0.0 (aref to 2)) (aref to 0)))
          (when (> d 0.001)
            (if (> d +trooper-standoff+)
                (progn ; press in toward the player
                  (setf (aref *tpos* i)
                   (linalg:add p (linalg:mul to (/ (* +trooper-speed+ dt) d))))
                  (setf (aref *tstep* i) (+ (aref *tstep* i) (* 7.0 dt))))
                (setf (aref *tstep* i) 0.0)))
          ;; fire at your chest when in range (a slower, lighter cadence so the
          ;; field is survivable and the saber can be waded in behind)
          (setf (aref *tfire* i) (- (aref *tfire* i) dt))
          (when (and (<= (aref *tfire* i) 0.0) (< d 22.0))
            (setf (aref *tfire* i) (rand-range 1.9 3.6))
            (let ((muzzle
                   (linalg:from-list (list (aref p 0) 1.02 (aref p 2))
                                     :element-type 'single-float)))
              (spawn-bolt muzzle
                          (linalg:sub (linalg:from-list
                                       (list (+ (aref *ppos* 0)
                                                (rand-range -0.5 0.5))
                                             (+ (aref *ppos* 1) 1.0)
                                             (+ (aref *ppos* 2)
                                                (rand-range -0.5 0.5)))
                                       :element-type 'single-float) muzzle) 24.0
                          1 4.0 1.0 1.0 0.28 0.20)
              (spawn-flash muzzle 0.08 0.35 1.0 0.4 0.25))))))))

;; --- AT-AT walkers ------------------------------------------------------------

(defconstant +atat-list+ '((30.0 -6.0) (36.0 9.0)))

(defconstant +atat-hp+ 150.0)
(defconstant +atat-speed+ 0.9)

(defvar *natat* 0)
(defvar *apos* nil) ; array of ground positions (#f vectors)
(defvar *ahp* nil)
(defvar *ayaw* nil)
(defvar *afire* nil)
(defvar *aalive* nil)
(defvar *awreck* nil) ; collapse animation timer
(defvar *ahit* nil)   ; damage-flash timer

(defun parse-atats ()
  (setq *natat* (length +atat-list+))
  (setq *apos* (make-array *natat* :initial-element nil))
  (setq *ahp* (make-array *natat* :initial-element 0.0))
  (setq *ayaw* (make-array *natat* :initial-element 0.0))
  (setq *afire* (make-array *natat* :initial-element 0.0))
  (setq *aalive* (make-array *natat* :initial-element t))
  (setq *awreck* (make-array *natat* :initial-element 0.0))
  (setq *ahit* (make-array *natat* :initial-element 0.0)))

(defun reset-atats ()
  (let ((i 0))
    (dolist (e +atat-list+)
      (setf (aref *apos* i)
            (linalg:from-list (list (nth 0 e) 0.0 (nth 1 e))
                              :element-type 'single-float))
      (setf (aref *ahp* i) +atat-hp+)
      (setf (aref *ayaw* i) 0.0)
      (setf (aref *afire* i) (rand-range 1.0 2.5))
      (setf (aref *aalive* i) t)
      (setf (aref *awreck* i) 0.0)
      (setf (aref *ahit* i) 0.0)
      (setq i (+ i 1)))))

(defun atats-alive ()
  (let ((n 0))
    (dotimes (i *natat*) (when (aref *aalive* i) (setq n (+ n 1))))
    n))

(defun update-atats (dt)
  (let ((pg (player-ground)))
    (dotimes (i *natat*)
      (if (aref *aalive* i)
          (let* ((p (aref *apos* i))
                 (to (linalg:sub pg p))
                 (d (linalg:norm to)))
            (setf (aref *ayaw* i) (atan2 (- 0.0 (aref to 2)) (aref to 0)))
            (when (> d 13.0) ; a ranged behemoth: keep its stand-off
              (setf (aref *apos* i)
                    (linalg:add p (linalg:mul to (/ (* +atat-speed+ dt) d)))))
            (setf (aref *afire* i) (- (aref *afire* i) dt))
            (when (<= (aref *afire* i) 0.0)
              (setf (aref *afire* i) (rand-range 3.0 4.6))
              ;; the muzzle is the tip of the chin blasters (local x 3.4, y 3.04)
              (let ((muzzle
                     (linalg:from-list (list (+ (aref p 0)
                                                (* (cos (aref *ayaw* i)) 3.4))
                                             3.04
                                             (- (aref p 2)
                                                (* (sin (aref *ayaw* i)) 3.4)))
                                       :element-type 'single-float)))
                (spawn-bolt muzzle
                            (linalg:sub (linalg:from-list
                                         (list (+ (aref *ppos* 0)
                                                  (rand-range -0.7 0.7))
                                               (+ (aref *ppos* 1) 1.0)
                                               (+ (aref *ppos* 2)
                                                  (rand-range -0.7 0.7)))
                                         :element-type 'single-float) muzzle)
                            26.0 1 9.0 1.8 1.0 0.32 0.16)
                (spawn-flash muzzle 0.14 0.7 1.0 0.5 0.2))))
          (when (< (aref *awreck* i) 3.0)
            (setf (aref *awreck* i) (+ (aref *awreck* i) dt)))))))

;; --- Vader (the boss) ---------------------------------------------------
;;
;; Dormant at the far edge until both walkers fall, then he engages: he chases
;; you and swings his red blade in melee. His own lightsaber deflects blaster
;; bolts, so only your lightsaber wounds him.

(defconstant +vader-hp+ 160.0)
(defconstant +vader-speed+ 3.0)

(defvar *vpos* #f(44.0 0.0 0.0)) ; Vader's ground position
(defvar *vhp* 160.0)
(defvar *vyaw* 0.0)
(defvar *vactive* nil)
(defvar *valive* t)
(defvar *vswing* 0.0) ; melee swing animation / hit window
(defvar *vcd* 0.0)    ; melee cooldown
(defvar *vhit* nil)
(defvar *vhitf* 0.0) ; damage-flash timer

(defun reset-vader ()
  (setq *vpos* #f(46.0 0.0 0.0))
  (setq *vhp* +vader-hp+)
  (setq *vyaw* +pi+)
  (setq *vactive* nil)
  (setq *valive* t)
  (setq *vswing* 0.0)
  (setq *vcd* 0.0)
  (setq *vhit* nil)
  (setq *vhitf* 0.0))

(defun update-enemy-flashes (dt)
  ;; decay every enemy's damage-flash timer, alive or not
  (dotimes (i *ntrooper*)
    (when (> (aref *thit* i) 0.0)
      (setf (aref *thit* i) (- (aref *thit* i) dt))))
  (dotimes (i *natat*)
    (when (> (aref *ahit* i) 0.0)
      (setf (aref *ahit* i) (- (aref *ahit* i) dt))))
  (when (> *vhitf* 0.0) (setq *vhitf* (- *vhitf* dt))))

(defun update-vader (dt)
  (when *valive*
    (unless *vactive*
      (when (= (atats-alive) 0)
        (setq *vactive* t)
        ;; a dramatic entrance: Vader strides in ahead of the player
        (setq *vpos*
              (linalg:add (player-ground)
                          (linalg:from-list (list (* (cos *cam-yaw*) 15.0) 0.0
                                                  (* (sin *cam-yaw*) 15.0))
                                            :element-type 'single-float)))
        (spawn-flash (linalg:from-list
                      (list (aref *vpos* 0) 1.4 (aref *vpos* 2))
                      :element-type 'single-float) 1.1 3.8 1.0 0.2 0.18)
        (spawn-flash (linalg:from-list
                      (list (aref *vpos* 0) 0.5 (aref *vpos* 2))
                      :element-type 'single-float) 0.9 3.0 0.9 0.15 0.15)))
    (when *vactive*
      (let* ((to (linalg:sub (player-ground) *vpos*)) (d (linalg:norm to)))
        (setq *vyaw* (atan2 (- 0.0 (aref to 2)) (aref to 0)))
        (when (> d 2.1)
          (setq *vpos*
                (linalg:add *vpos* (linalg:mul to (/ (* +vader-speed+ dt) d)))))
        (when (> *vcd* 0.0) (setq *vcd* (- *vcd* dt)))
        (when (> *vswing* 0.0) (setq *vswing* (- *vswing* dt)))
        ;; open a melee swing when in reach
        (when (and (< d 2.6) (<= *vcd* 0.0))
          (setq *vswing* 0.45)
          (setq *vcd* 0.9)
          (setq *vhit* nil))
        ;; the blade connects at mid-swing
        (when (and (> *vswing* 0.0) (< *vswing* 0.28) (not *vhit*) (< d 3.0))
          (setq *vhit* t)
          (hurt-player 15.0))))))

;; --- damage plumbing ----------------------------------------------------------

(defun hurt-player (amount)
  ;; ignore hits during the brief i-frame window so fire cannot stack-kill
  (when (and (= *state* 0) (<= *inv-t* 0.0))
    (setq *php* (- *php* amount))
    (setq *inv-t* +invuln+)
    (setq *hurt-flash* 0.35)
    (when (<= *php* 0.0)
      (setq *php* 0.0)
      (setq *state* 2)
      (setq *state-t* 0.0))))

(defun hit-trooper (i dmg)
  (setf (aref *thp* i) (- (aref *thp* i) dmg))
  (setf (aref *thit* i) 0.16) ; flash red
  (when (<= (aref *thp* i) 0.0)
    (setf (aref *talive* i) nil)
    (let ((p (aref *tpos* i)))
      (spawn-flash (linalg:from-list (list (aref p 0) 0.7 (aref p 2))
                                     :element-type 'single-float) 0.4 1.1 1.0
                   0.7 0.4))))

(defun hit-atat (i dmg)
  (setf (aref *ahp* i) (- (aref *ahp* i) dmg))
  (setf (aref *ahit* i) 0.16) ; flash red
  (when (<= (aref *ahp* i) 0.0)
    (setf (aref *aalive* i) nil)
    (setf (aref *awreck* i) 0.0)
    (let ((p (aref *apos* i)))
      (spawn-flash (linalg:from-list (list (aref p 0) 3.0 (aref p 2))
                                     :element-type 'single-float) 0.7 3.2 1.0
                   0.7 0.3)
      (spawn-flash (linalg:from-list (list (aref p 0) 5.0 (aref p 2))
                                     :element-type 'single-float) 0.6 2.2 1.0
                   0.5 0.2))))

(defun hit-vader (dmg)
  (setq *vhp* (- *vhp* dmg))
  (setq *vhitf* 0.16) ; flash red
  (spawn-flash (linalg:from-list (list (aref *vpos* 0) 1.4 (aref *vpos* 2))
                                 :element-type 'single-float) 0.25 0.8 1.0 0.4
               0.3)
  (when (<= *vhp* 0.0)
    (setq *vhp* 0.0)
    (setq *valive* nil)
    (spawn-flash (linalg:from-list (list (aref *vpos* 0) 1.4 (aref *vpos* 2))
                                   :element-type 'single-float) 0.9 3.0 1.0 0.6
                 0.3)
    (launch-firework)
    (launch-firework)
    (launch-firework) ; an instant volley
    (launch-firework)
    (launch-firework)
    (setq *state* 1)
    (setq *state-t* 0.0)))

;; --- bolt integration + collisions --------------------------------------------

(defun bolt-hits-enemies (i)
  ;; a player bolt: test the troopers, the walkers and Vader; return t on a hit.
  (let ((bp (aref *bpos* i)) (hit nil))
    ;; troopers -- a generous body capsule so aimed fire connects
    (dotimes (j *ntrooper*)
      (when (and (not hit) (aref *talive* j))
        (let* ((tp (aref *tpos* j))
               (diff
                (linalg:sub bp
                            (linalg:from-list (list (aref tp 0) 1.0 (aref tp 2))
                                              :element-type 'single-float))))
          (when (< (linalg:dot diff diff) 0.45)
            (hit-trooper j (aref *bdmg* i))
            (spawn-flash bp 0.16 0.45 1.0 0.7 0.4)
            (setq hit t)))))
    ;; walkers -- a tall body, so test a horizontal disc over a vertical span
    (dotimes (j *natat*)
      (when (and (not hit) (aref *aalive* j))
        (let* ((ap (aref *apos* j))
               (dx (- (aref bp 0) (aref ap 0)))
               (dz (- (aref bp 2) (aref ap 2))))
          (when (and (< (+ (* dx dx) (* dz dz)) 3.6) (> (aref bp 1) 0.5)
                     (< (aref bp 1) 6.2))
            (hit-atat j (aref *bdmg* i))
            (spawn-flash bp 0.18 0.5 1.0 0.7 0.4)
            (setq hit t)))))
    ;; Vader deflects blaster fire with his blade
    (when (and (not hit) *valive* *vactive*)
      (let ((diff
             (linalg:sub bp
                         (linalg:from-list
                          (list (aref *vpos* 0) 1.3 (aref *vpos* 2))
                          :element-type 'single-float))))
        (when (< (linalg:dot diff diff) 1.3)
          (spawn-flash bp 0.2 0.5 1.0 0.3 0.25)
          (setq hit t))))
    hit))

(defun player-chest ()
  (linalg:from-list
   (list (aref *ppos* 0) (+ (aref *ppos* 1) 1.0) (aref *ppos* 2))
   :element-type 'single-float))

(defun bolt-hits-player (i)
  (let ((diff (linalg:sub (aref *bpos* i) (player-chest))))
    (< (linalg:dot diff diff) 0.4)))

(defun deflecting-p ()
  ;; the lightsaber swing sweeps a shield in front of you
  (and (= *weapon* 0) (> *swing-t* 0.0)))

(defun update-bolts (dt)
  (dotimes (i +nbolt+)
    (when (aref *balive* i)
      (setf (aref *bpos* i)
            (linalg:add (aref *bpos* i) (linalg:mul (aref *bvel* i) dt)))
      (setf (aref *bt* i) (- (aref *bt* i) dt))
      (cond ((<= (aref *bt* i) 0.0) (setf (aref *balive* i) nil))
            ((< (aref (aref *bpos* i) 1) 0.05) (setf (aref *balive* i) nil))
            ((= (aref *bown* i) 0)
             (when (bolt-hits-enemies i) (setf (aref *balive* i) nil)))
            (t ; enemy bolt
             (cond ((and (deflecting-p)
                         (let ((diff
                                (linalg:sub (aref *bpos* i) (player-chest))))
                           (< (linalg:dot diff diff) 2.5)))
                    ;; deflected: bat it away as a harmless spark
                    (spawn-flash (aref *bpos* i) 0.18 0.5 0.6 0.85 1.0)
                    (setf (aref *balive* i) nil))
                   ((bolt-hits-player i)
                    (hurt-player (aref *bdmg* i))
                    (spawn-flash (aref *bpos* i) 0.14 0.4 1.0 0.5 0.3)
                    (setf (aref *balive* i) nil))))))))

;; --- your attacks -----------------------------------------------------------

;; A generous frontal reach: hit anything within `reach` whose bearing is
;; within ~120 degrees of the aim (dot > -0.5*d), so a swing near an enemy
;; connects even when the aim is a little off. `epos` is the enemy's ground
;; position (a vector, y = 0).
(defun in-saber-arc-p (epos reach)
  (let* ((to (linalg:sub epos (player-ground))) (d (linalg:norm to)))
    (and (< d reach) (> (linalg:dot to *aimf*) (* d -0.5)))))

(defun saber-strike ()
  ;; one swing's worth of damage across the frontal arc
  (dotimes (j *ntrooper*)
    (when (and (aref *talive* j) (in-saber-arc-p (aref *tpos* j) 2.9))
      (hit-trooper j 40.0)))
  (dotimes (j *natat*)
    (when (and (aref *aalive* j) (in-saber-arc-p (aref *apos* j) 3.7))
      (hit-atat j 20.0)))
  (when (and *valive* *vactive* (in-saber-arc-p *vpos* 3.1)) (hit-vader 20.0)))

(defun update-attack (dt)
  (when (> *swing-cd* 0.0) (setq *swing-cd* (- *swing-cd* dt)))
  (when (> *swing-t* 0.0) (setq *swing-t* (- *swing-t* dt)))
  (when (> *fire-cd* 0.0) (setq *fire-cd* (- *fire-cd* dt)))
  (let ((held (> *attack* 0.5)))
    (if (= *weapon* 0)
        ;; lightsaber: start a swing; deal its damage once at mid-arc
        (progn
          (when (and held (<= *swing-cd* 0.0))
            (setq *swing-t* 0.32)
            (setq *swing-cd* 0.40)
            (setq *swing-hit* nil))
          (when (and (> *swing-t* 0.0) (< *swing-t* 0.28) (not *swing-hit*))
            (setq *swing-hit* t)
            (saber-strike)))
        ;; blaster: automatic fire on a short cadence
        (when (and held (<= *fire-cd* 0.0))
          (setq *fire-cd* 0.16)
          (let ((muzzle
                 (linalg:add (player-chest)
                             (linalg:add (linalg:mul *aimf* 0.55)
                                         (linalg:mul *aimr* 0.16)))))
            (spawn-bolt muzzle *aimf* 46.0 0 12.0 1.0 0.35 1.0 0.45)
            (spawn-flash muzzle 0.06 0.3 0.5 1.0 0.5)))))
  (setq *attack-prev* (> *attack* 0.5)))

;; --- the playing-state step ---------------------------------------------------

(defun steer (dt)
  (let* ((fwd (- *in-f* *in-b*))
         (rgt (- *in-r* *in-l*))
         (n (if (or (= fwd 0.0) (= rgt 0.0)) 1.0 0.7071))
         (cy (cos *cam-yaw*))
         (sy (sin *cam-yaw*))
         (tx (* +run-speed+ n (- (* fwd cy) (* rgt sy))))
         (tz (* +run-speed+ n (+ (* fwd sy) (* rgt cy))))
         (acc (* dt 40.0)))
    ;; accelerate the horizontal velocity toward the steer target, capped by acc
    (setq *pvel*
          (linalg:from-list (list (+ (aref *pvel* 0)
                                     (max (- 0.0 acc)
                                          (min acc (- tx (aref *pvel* 0)))))
                                  (aref *pvel* 1)
                                  (+ (aref *pvel* 2)
                                     (max (- 0.0 acc)
                                          (min acc (- tz (aref *pvel* 2))))))
                            :element-type 'single-float))))

(defun jump-control (dt)
  ;; Space jumps off the ground; a real hop, gravity does the rest
  (let ((held (> *in-jump* 0.5)))
    (when (and held (not *jump-prev*) *grounded*)
      (setq *pvel*
            (linalg:from-list (list (aref *pvel* 0) +jump-v+ (aref *pvel* 2))
                              :element-type 'single-float))
      (setq *grounded* nil))
    (setq *jump-prev* held)))

(defun move-player (dt)
  ;; integrate gravity on the vertical component, then advance and clamp
  (let* ((vy (max -32.0 (- (aref *pvel* 1) (* +gravity+ dt))))
         (nx
          (max +field-min-x+
               (min +field-max-x+ (+ (aref *ppos* 0) (* (aref *pvel* 0) dt)))))
         (nz
          (max +field-min-z+
               (min +field-max-z+ (+ (aref *ppos* 2) (* (aref *pvel* 2) dt)))))
         (ny (+ (aref *ppos* 1) (* vy dt))))
    (when (<= ny 0.0) ; landed on the snow at y = 0
      (setq ny 0.0)
      (setq vy 0.0)
      (setq *grounded* t))
    (setq *pvel*
          (linalg:from-list (list (aref *pvel* 0) vy (aref *pvel* 2))
                            :element-type 'single-float))
    (setq *ppos*
          (linalg:from-list (list nx ny nz) :element-type 'single-float))))

(defun step-playing (dt)
  (when (> *inv-t* 0.0) (setq *inv-t* (- *inv-t* dt)))
  (steer dt)
  (jump-control dt)
  (move-player dt)
  (setq *pyaw* (- 0.0 *cam-yaw*)) ; face the aim
  (let ((sp
         (sqrt
          (+ (* (aref *pvel* 0) (aref *pvel* 0))
             (* (aref *pvel* 2) (aref *pvel* 2))))))
    (if (and *grounded* (> sp 0.4))
        (setq *run-phase* (+ *run-phase* (* sp 1.9 dt)))
        (setq *run-phase* 0.0)))
  (update-attack dt)
  (update-troopers dt)
  (update-atats dt)
  (update-vader dt)
  (when (> *hurt-flash* 0.0) (setq *hurt-flash* (- *hurt-flash* dt))))

(defun step-victory (dt)
  ;; the celebration: keep launching fireworks over the arena, and let you
  ;; hold your lightsaber raised (a slow idle sway through the run oscillator)
  (setq *weapon* 0)
  (setq *pyaw* (- 0.0 *cam-yaw*))
  (setq *run-phase* (+ *run-phase* (* 1.4 dt)))
  (setq *fw-launch* (- *fw-launch* dt))
  (when (<= *fw-launch* 0.0)
    (setq *fw-launch* (rand-range 0.16 0.32))
    (launch-firework) ; two bursts at a time for a fuller sky
    (launch-firework))
  (update-fireworks dt))

;; --- drawing the cast ---------------------------------------------------------

(defun emit-shadow (x z r)
  ;; a flattened ellipsoid reads as a soft round blob shadow, not a square
  (col 0.55 0.60 0.68)
  (emit-ellipsoid x 0.02 z r 0.006 r 0.0 10 3))

;; scratch for the leg's ankle hand-off and for Vader's cape sampler
(defvar *cp-x* 0.0)
(defvar *cp-y* 0.0)
(defvar *cp-z* 0.0)

;; --- the humanoid leg ------------------------------------------------------
;;
;; Shared by all three figures: hip ball, tapered thigh, knee ball, tapered
;; shin, ankle, and a boot the caller draws. `sw` is the walk cycle's -1..1
;; swing for this leg, so the knee leads the ankle and the foot lifts as it
;; comes forward -- the thing a rigid pair of posts sliding fore and aft can
;; never do. Colours are whatever the caller latched, except the boot.
(defun humanoid-leg (lz sw hipy scale)
  (let* ((kx (* 0.11 sw scale))
         (ky (* hipy 0.53))
         (ax (* 0.24 sw scale))
         (ay (+ (* hipy 0.135) (max 0.0 (* 0.055 sw scale)))))
    ;; both segments are bracketed -- hip ball above, knee ball between, boot
    ;; below -- so their end discs are inside solid geometry. Dropping them
    ;; halves the leg's triangles for nothing you could ever see.
    (limb-caps nil)
    (part-joint 0.0 hipy lz (* 0.125 scale))
    (taper (* 0.098 scale))
    (part-limb 0.0 hipy lz kx ky lz (* 0.115 scale))
    (part-joint kx ky lz (* 0.098 scale))
    (taper (* 0.078 scale))
    (part-limb kx ky lz ax ay lz (* 0.096 scale))
    (limb-caps t)
    ;; no ankle ball either: the boot swallows it, and a sphere nobody can see
    ;; is the most expensive kind of detail there is.
    ;; the ankle position is left in these two globals for the boot
    (setq *cp-x* ax *cp-y* ay)))

;; You, in Hoth (Echo Base) gear: tan jacket over dark trousers and boots,
;; the field backpack, the knit cap with its snow goggles pushed up on it,
;; holding either the glowing blue lightsaber or the blaster. ~1.72 tall.
(defun emit-player (tm)
  (emit-shadow (aref *ppos* 0) (aref *ppos* 2) 0.36)
  ;; you are always the closest figure on screen, so never let the distance
  ;; heuristic coarsen you
  (setq *lod* 2)
  (set-origin (aref *ppos* 0) (aref *ppos* 1) (aref *ppos* 2) *pyaw*)
  (let* ((sw (sin *run-phase*)) (arm (* -0.13 sw)))
    ;; legs: dark olive trousers over the articulated frame, then the boots
    (col 0.44 0.41 0.31)
    (humanoid-leg -0.105 sw 0.80 1.0)
    (let ((bx *cp-x*) (by *cp-y*))
      (col 0.15 0.12 0.10)
      (part-rbox (+ bx 0.025) (- by 0.045) -0.105 0.095 0.062 0.083 0.035))
    (col 0.44 0.41 0.31)
    (humanoid-leg 0.105 (- 0.0 sw) 0.80 1.0)
    (let ((bx *cp-x*) (by *cp-y*))
      (col 0.15 0.12 0.10)
      (part-rbox (+ bx 0.025) (- by 0.045) 0.105 0.095 0.062 0.083 0.035))
    ;; hips and the utility belt
    (col 0.42 0.38 0.29)
    (part-rbox 0.0 0.84 0.0 0.152 0.105 0.125 0.055)
    (col 0.26 0.22 0.17)
    (part-rbox 0.0 0.75 0.0 0.158 0.042 0.130 0.028)
    (metal 0.55 0.47 0.28)
    (part-rbox 0.15 0.75 0.0 0.022 0.032 0.040 0.012) ; buckle
    (col 0.22 0.19 0.15)                              ; holster
    (part-rbox 0.02 0.70 -0.145 0.045 0.075 0.035 0.022)
    ;; the tan quilted jacket: a chest and a slightly narrower waist, so the
    ;; torso has a taper instead of being one carton
    (col 0.80 0.70 0.52)
    (part-rbox 0.0 1.14 0.0 0.185 0.135 0.140 0.075)
    (part-rbox 0.0 0.97 0.0 0.163 0.115 0.125 0.065)
    (col 0.62 0.53 0.38) ; the vest panel
    (part-rbox 0.075 1.08 0.0 0.115 0.155 0.118 0.055)
    (col 0.86 0.78 0.62) ; the fur collar
    (part-cyl 0.0 1.29 0.0 0.112 0.048 10)
    ;; the Hoth field pack, its straps and a canteen
    (col 0.33 0.31 0.27)
    (part-rbox -0.185 1.06 0.0 0.075 0.185 0.145 0.055)
    (col 0.24 0.22 0.19)
    (part-rbox -0.06 1.14 -0.105 0.115 0.075 0.026 0.013)
    (part-rbox -0.06 1.14 0.105 0.115 0.075 0.026 0.013)
    (metal 0.42 0.44 0.46)
    (part-limb -0.27 1.02 0.085 -0.27 1.16 0.085 0.048)
    ;; shoulders and the off arm (the weapon arm is drawn by emit-arm-to)
    (col 0.78 0.68 0.50)
    (part-joint 0.0 1.215 -0.178 0.076)
    (part-joint 0.0 1.215 0.178 0.076)
    (col 0.74 0.64 0.47)
    (taper 0.047)
    (part-limb 0.0 1.215 0.192 (* 0.6 arm) 1.02 0.212 0.056)
    (part-joint (* 0.6 arm) 1.02 0.212 0.048)
    (taper 0.041)
    (part-limb (* 0.6 arm) 1.02 0.212 arm 0.855 0.218 0.047)
    (col 0.18 0.15 0.12)
    (part-ellipsoid arm 0.833 0.218 0.048 0.052 0.048 7 4)
    ;; neck and head
    (col 0.78 0.60 0.48)
    (part-cyl 0.0 1.335 0.0 0.05 0.05)
    (col 0.86 0.70 0.58)
    (part-ellipsoid 0.012 1.455 0.0 0.092 0.103 0.092 10 6)
    (part-ellipsoid 0.088 1.445 0.0 0.026 0.022 0.024 6 4) ; nose
    (col 0.34 0.27 0.21)
    (part-rbox 0.078 1.492 0.0 0.022 0.016 0.070 0.008) ; brow
    ;; the knit cap, its band, and the snow goggles pushed up onto it
    (col 0.70 0.62 0.46)
    (part-ellipsoid 0.0 1.545 0.0 0.108 0.075 0.108 10 5)
    (col 0.50 0.43 0.33)
    (part-cyl 0.0 1.528 0.0 0.112 0.020 10)
    (metal 0.20 0.20 0.22)
    (part-limb 0.055 1.560 -0.100 0.055 1.560 0.100 0.030)
    (col 0.30 0.42 0.48)
    (part-ellipsoid 0.082 1.560 -0.048 0.020 0.026 0.030 6 4)
    (part-ellipsoid 0.082 1.560 0.048 0.020 0.026 0.030 6 4))
  (if (= *weapon* 0) (emit-player-saber tm) (emit-player-blaster)))

;; The weapon arm, reaching from the weapon-side shoulder to the (animated)
;; weapon hand. It is a real two-segment arm: the elbow sits off the straight
;; shoulder-to-hand line, dropped and pushed outboard, so a raised weapon bends
;; the arm the way an arm bends instead of running one rigid pole from the
;; shoulder to the fist. Everything here is WORLD space -- the hand is already
;; solved in the aim frame by the caller, not in the figure's local frame.
(defun emit-arm-to (hx hy hz r g b)
  (let* ((arrx (- 0.0 (sin *cam-yaw*))) ; aim-right
         (arrz (cos *cam-yaw*))
         (sx (+ (aref *ppos* 0) (* arrx -0.18))) ; weapon-side shoulder
         (sz (+ (aref *ppos* 2) (* arrz -0.18)))
         (sy (+ (aref *ppos* 1) 1.16))
         ;; the elbow: midway, sagging, and bowed out along the aim-right axis
         (ex (+ (* 0.5 (+ sx hx)) (* arrx -0.07)))
         (ez (+ (* 0.5 (+ sz hz)) (* arrz -0.07)))
         (ey (- (* 0.5 (+ sy hy)) 0.085)))
    (col r g b)
    (limb-caps nil) ; shoulder ball, elbow ball, then a fist
    (emit-ellipsoid sx sy sz 0.070 0.070 0.070 0.0 (segs 6) (segs 3))
    (taper 0.042)
    (emit-limb sx sy sz ex ey ez 0.052)
    (emit-ellipsoid ex ey ez 0.044 0.044 0.044 0.0 (segs 6) (segs 3))
    (taper 0.039)
    (emit-limb ex ey ez hx hy hz 0.045)
    (limb-caps t)))

;; blade-glow inflation: pad the thin cross-axes more than the long axis
(defun glowh (h) (if (< h 0.2) (+ h 0.045) (+ h 0.03)))
(defun glowh2 (h) (if (< h 0.2) (+ h 0.09) (+ h 0.055)))

(defun emit-player-blaster ()
  ;; a compact grey blaster pistol, held out along the aim: a stepped barrel
  ;; with a muzzle collar, a receiver block, a sight rib and a raked grip
  (let* ((theta *cam-yaw*)
         (fwx (cos theta))
         (fwz (sin theta))
         (arx (- 0.0 (sin theta)))
         (arz (cos theta))
         (hx (+ (aref *ppos* 0) (* fwx 0.40) (* arx -0.15)))
         (hz (+ (aref *ppos* 2) (* fwz 0.40) (* arz -0.15)))
         (hy (+ (aref *ppos* 1) 0.98)))
    (emit-arm-to hx hy hz 0.74 0.64 0.47)
    (col 0.15 0.12 0.10)
    (emit-ellipsoid hx hy hz 0.052 0.055 0.052 0.0 (segs 8) (segs 5)) ; fist
    ;; the pistol's blocks are centimetres across -- plain soft-edged boxes,
    ;; never rounded ones; only the round parts of it are round
    (soften 0.5)
    (emit-box (+ hx (* fwx 0.03)) (+ hy 0.015) (+ hz (* fwz 0.03)) 0.075 0.045
              0.032 *pyaw*)
    (metal 0.20 0.21 0.24) ; barrel
    (taper 0.026)
    (emit-limb (+ hx (* fwx 0.07)) (+ hy 0.022) (+ hz (* fwz 0.07))
               (+ hx (* fwx 0.25)) (+ hy 0.022) (+ hz (* fwz 0.25)) 0.030)
    (metal 0.34 0.35 0.38) ; muzzle collar
    (emit-limb (+ hx (* fwx 0.23)) (+ hy 0.022) (+ hz (* fwz 0.23))
               (+ hx (* fwx 0.26)) (+ hy 0.022) (+ hz (* fwz 0.26)) 0.037)
    (col 0.11 0.11 0.13) ; sight rib
    (soften 0.5)
    (emit-box (+ hx (* fwx 0.10)) (+ hy 0.062) (+ hz (* fwz 0.10)) 0.055 0.012
              0.010 *pyaw*)
    (col 0.13 0.13 0.15) ; grip
    (soften 0.5)
    (emit-box (- hx (* fwx 0.015)) (- hy 0.075) (- hz (* fwz 0.015)) 0.030 0.062
              0.028 *pyaw*)))

;; The lightsaber is stored as a generic oriented box (center + half-extents +
;; yaw) so both the opaque core pass and the additive bloom pass can redraw it,
;; whether it is held vertical (idle) or swept horizontal (mid-slash).
(defvar *sab-cx* 0.0)
(defvar *sab-cy* 0.0)
(defvar *sab-cz* 0.0)
(defvar *sab-hx* 0.03)
(defvar *sab-hy* 0.6)
(defvar *sab-hz* 0.03)
(defvar *sab-yaw* 0.0)
(defvar *sab-vis* nil)

(defun emit-hilt (x y z)
  (col 0.16 0.13 0.11) ; fist
  (emit-ellipsoid x y z 0.05 0.05 0.05 0.0 8 5)
  (metal 0.76 0.77 0.81) ; metal hilt -- a round rod
  (emit-cylinder x (+ y 0.05) z 0.028 0.055 0.0 8))

(defun emit-player-blade ()
  ;; the stored blade: a blue core with a white-hot inner line -- a true
  ;; cylinder (hx and hz always match), not a slab
  (glow-col 0.55 0.78 1.0)
  (emit-cylinder *sab-cx* *sab-cy* *sab-cz* *sab-hx* *sab-hy* *sab-yaw* 8)
  (glow-col 0.92 0.97 1.0)
  (emit-cylinder *sab-cx* *sab-cy* *sab-cz* (max 0.012 (- *sab-hx* 0.016))
                 (max 0.012 (- *sab-hy* 0.016)) *sab-yaw* 8))

(defun emit-player-saber (tm)
  (let* ((theta *cam-yaw*)
         (fwx (cos theta))
         (fwz (sin theta))
         (arx (- 0.0 (sin theta)))
         (arz (cos theta)))
    (if (> *swing-t* 0.0)
        ;; a diagonal downward slash: the blade stays upright while the hand
        ;; arcs from upper-right to lower-left across the front, reaching out at
        ;; mid-swing -- the arm tracks it, so the whole strike reads cleanly
        (let* ((p (- 1.0 (/ *swing-t* 0.32)))
               (side (- 0.44 (* 0.88 p))) ; right -> left
               (reachf (+ 0.34 (* 0.24 (sin (* p +pi+)))))
               (hh (+ (aref *ppos* 1) (- 1.16 (* 0.40 p)))) ; high -> low
               (hx (+ (aref *ppos* 0) (* fwx reachf) (* arx side)))
               (hz (+ (aref *ppos* 2) (* fwz reachf) (* arz side)))
               (half 0.6))
          (emit-arm-to hx hh hz 0.74 0.64 0.47)
          (emit-hilt hx hh hz)
          (setq *sab-cx* hx *sab-cy* (+ hh 0.12 half) *sab-cz* hz *sab-hx* 0.032
                *sab-hy* half *sab-hz* 0.032 *sab-yaw* 0.0 *sab-vis* t)
          (emit-player-blade))
        ;; idle: held upright in the weapon hand
        (let* ((hx (+ (aref *ppos* 0) (* fwx 0.40) (* arx -0.18)))
               (hz (+ (aref *ppos* 2) (* fwz 0.40) (* arz -0.18)))
               (hh (+ (aref *ppos* 1) 0.95))
               (half 0.62))
          (emit-arm-to hx (+ hh 0.08) hz 0.74 0.64 0.47)
          (emit-hilt hx hh hz)
          (setq *sab-cx* hx *sab-cy* (+ hh 0.14 half) *sab-cz* hz *sab-hx* 0.032
                *sab-hy* half *sab-hz* 0.032 *sab-yaw* 0.0 *sab-vis* t)
          (emit-player-blade)))))

(defun emit-player-saber-glow ()
  (when *sab-vis*
    (glow-col 0.28 0.5 1.0)
    (emit-cylinder *sab-cx* *sab-cy* *sab-cz* (glowh *sab-hx*) (glowh *sab-hy*)
                   *sab-yaw* 8)
    (glow-col 0.12 0.28 0.85)
    (emit-cylinder *sab-cx* *sab-cy* *sab-cz* (glowh2 *sab-hx*)
                   (glowh2 *sab-hy*) *sab-yaw* 8)))

;; A stormtrooper: white armour over a black bodysuit. The armour is a SET OF
;; SEPARATE ROUNDED PLATES riding on the black limbs underneath -- chest, abdo,
;; shoulder bells, biceps, forearms, thigh and shin guards, boots -- because
;; that gap between plate and suit is the whole look; a single white cylinder
;; per limb reads as a robot. The helmet is built from the features people
;; actually recognise it by: the dome, the brow band, the two eye lenses, the
;; frown grille and the tube stripes on the cheeks. ~1.68 tall.
(defun emit-trooper (i tm)
  (setq *hit-tint* (min 1.0 (* (aref *thit* i) 6.0))) ; red when just struck
  (let ((p (aref *tpos* i)))
    (set-lod (aref p 0) (aref p 2) 1.68)
    (if (aref *talive* i)
        (let* ((sw (sin (aref *tstep* i))) (arm (* -0.11 sw)))
          (set-origin (aref p 0) 0.0 (aref p 2) (aref *tyaw* i))
          (emit-shadow (aref p 0) (aref p 2) 0.34)
          ;; the black bodysuit legs, then the white plates over them
          (col 0.11 0.11 0.13)
          (humanoid-leg -0.10 sw 0.78 0.95)
          (let ((ax *cp-x*) (ay *cp-y*))
            (col 0.93 0.94 0.97)
            (part-rbox (* 0.06 sw) 0.58 -0.10 0.088 0.13 0.088 0.045) ; thigh
            (part-rbox (* 0.18 sw) 0.28 -0.10 0.082 0.13 0.082 0.040) ; shin
            (part-rbox (+ ax 0.03) (- ay 0.035) -0.10 0.098 0.062 0.088 0.032))
          (col 0.11 0.11 0.13)
          (humanoid-leg 0.10 (- 0.0 sw) 0.78 0.95)
          (let ((ax *cp-x*) (ay *cp-y*))
            (col 0.93 0.94 0.97)
            (part-rbox (* -0.06 sw) 0.58 0.10 0.088 0.13 0.088 0.045)
            (part-rbox (* -0.18 sw) 0.28 0.10 0.082 0.13 0.082 0.040)
            (part-rbox (+ ax 0.03) (- ay 0.035) 0.10 0.098 0.062 0.088 0.032))
          ;; hip block and the black belt with its side pouches
          (col 0.90 0.91 0.95)
          (part-rbox 0.0 0.82 0.0 0.142 0.085 0.112 0.05)
          (col 0.09 0.09 0.11)
          (part-rbox 0.0 0.735 0.0 0.148 0.038 0.118 0.022)
          (part-rbox 0.02 0.735 -0.115 0.045 0.050 0.030 0.018)
          (part-rbox 0.02 0.735 0.115 0.045 0.050 0.030 0.018)
          ;; the chest and abdominal plates, and the back plate behind them
          (col 0.94 0.95 0.98)
          (part-rbox 0.005 1.08 0.0 0.158 0.115 0.115 0.06)
          (col 0.88 0.89 0.93)
          (part-rbox 0.02 0.935 0.0 0.140 0.055 0.108 0.035)
          (col 0.90 0.91 0.95)
          (part-rbox -0.06 1.05 0.0 0.095 0.145 0.108 0.05)
          (col 0.11 0.11 0.13) ; the chest control panel
          (part-rbox 0.15 1.115 -0.045 0.020 0.030 0.038 0.008)
          (col 0.70 0.20 0.18)
          (part-rbox 0.15 1.115 0.030 0.020 0.014 0.020 0.006)
          ;; shoulder bells, black under-arms and the white arm plates
          (col 0.94 0.95 0.98)
          (part-rbox 0.0 1.205 -0.185 0.078 0.062 0.085 0.045)
          (part-rbox 0.0 1.205 0.185 0.078 0.062 0.085 0.045)
          (col 0.11 0.11 0.13)
          (taper 0.048)
          (part-limb 0.0 1.19 -0.20 (* 0.5 arm) 1.00 -0.215 0.058)
          (part-joint (* 0.5 arm) 1.00 -0.215 0.050)
          (taper 0.042)
          (part-limb (* 0.5 arm) 1.00 -0.215 arm 0.845 -0.222 0.050)
          (taper 0.048)
          (part-limb 0.0 1.19 0.20 (* -0.5 arm) 1.00 0.215 0.058)
          (part-joint (* -0.5 arm) 1.00 0.215 0.050)
          (taper 0.042)
          (part-limb (* -0.5 arm) 1.00 0.215 (- 0.0 arm) 0.845 0.222 0.050)
          (col 0.93 0.94 0.97)
          (part-rbox (* 0.25 arm) 1.115 -0.208 0.062 0.060 0.062 0.030)
          (part-rbox (* -0.25 arm) 1.115 0.208 0.062 0.060 0.062 0.030)
          (part-rbox (* 0.8 arm) 0.925 -0.218 0.058 0.058 0.058 0.028)
          (part-rbox (* -0.8 arm) 0.925 0.218 0.058 0.058 0.058 0.028)
          (col 0.10 0.10 0.12) ; gloves
          (part-ellipsoid arm 0.815 -0.222 0.048 0.052 0.048 7 4)
          (part-ellipsoid (- 0.0 arm) 0.815 0.222 0.048 0.052 0.048 7 4)
          ;; the black neck seal
          (col 0.10 0.10 0.12)
          (part-cyl 0.0 1.245 0.0 0.062 0.048)
          ;; the helmet
          (metal 0.95 0.96 0.99)
          (part-ellipsoid -0.005 1.375 0.0 0.098 0.105 0.100 10 6)
          (part-rbox 0.062 1.360 0.0 0.048 0.082 0.086 0.030) ; faceplate
          (col 0.09 0.09 0.11)                                ; brow band
          (part-rbox 0.090 1.408 0.0 0.032 0.020 0.080 0.008)
          (part-rbox 0.098 1.372 -0.042 0.026 0.026 0.028 0.008) ; eye lenses
          (part-rbox 0.098 1.372 0.042 0.026 0.026 0.028 0.008)
          (part-rbox 0.096 1.312 0.0 0.028 0.024 0.040 0.010) ; frown grille
          (metal 0.20 0.20 0.22)                              ; cheek tubes
          (taper 0.008)
          (part-limb 0.086 1.330 -0.062 0.062 1.296 -0.075 0.011)
          (taper 0.008)
          (part-limb 0.086 1.330 0.062 0.062 1.296 0.075 0.011)
          (col 0.88 0.89 0.93) ; ear vents
          (part-rbox 0.010 1.345 -0.095 0.038 0.040 0.014 0.010)
          (part-rbox 0.010 1.345 0.095 0.038 0.040 0.014 0.010)
          ;; the E-11 held across the body: barrel, receiver, folding stock,
          ;; magazine and the scope rail
          (metal 0.13 0.13 0.15)
          (part-rbox 0.20 0.905 -0.20 0.075 0.038 0.030 0.014)
          (metal 0.18 0.18 0.21)
          (taper 0.020)
          (part-limb 0.27 0.912 -0.20 0.46 0.912 -0.20 0.024)
          (col 0.10 0.10 0.12)
          (part-rbox 0.19 0.845 -0.20 0.026 0.045 0.024 0.010) ; magazine
          (part-rbox 0.08 0.905 -0.20 0.060 0.024 0.022 0.010) ; stock
          (metal 0.22 0.22 0.25)
          (part-rbox 0.24 0.955 -0.20 0.048 0.014 0.014 0.006)) ; scope rail
        ;; a fallen trooper: face down in the snow, one arm flung out and the
        ;; helmet rolled clear of the body
        (progn
          (set-origin (aref p 0) 0.0 (aref p 2) (aref *tyaw* i))
          (emit-shadow (aref p 0) (aref p 2) 0.36)
          (col 0.11 0.11 0.13)
          (taper 0.075)
          (part-limb -0.10 0.10 -0.06 -0.44 0.07 -0.11 0.088)
          (taper 0.075)
          (part-limb -0.10 0.10 0.06 -0.42 0.07 0.14 0.088)
          (col 0.88 0.90 0.94)
          (part-rbox -0.02 0.10 0.0 0.20 0.095 0.135 0.06)
          (col 0.11 0.11 0.13)
          (taper 0.048)
          (part-limb 0.10 0.11 -0.14 0.34 0.06 -0.30 0.055)
          (metal 0.92 0.93 0.97)
          (part-ellipsoid 0.34 0.095 0.10 0.095 0.095 0.098 8 5)
          (col 0.09 0.09 0.11)
          (part-rbox 0.40 0.11 0.14 0.030 0.026 0.036 0.008))))
  (setq *hit-tint* 0.0))

;; --- the AT-AT ------------------------------------------------------------
;;
;; The walker used to be four vertical posts under a slab, which reads as a
;; table on legs from any angle. A real AT-AT's whole silhouette is its GAIT:
;; each leg is two long segments meeting at a knee that stands well forward of
;; the line from hip to foot, and the foot is a broad pad that stays flat on the
;; snow. So the leg is solved rather than drawn -- the foot is animated
;; (swinging fore and aft, lifting only while it travels forward), and the knee
;; is placed by the two-link inverse kinematics that the fixed thigh and shin
;; lengths force. That is what makes it stride instead of slide.

;; The two segments are only a little longer than the hip-to-foot distance, so
;; the knee stands proud of the leg line without the deep insect crouch a bigger
;; excess would give.
(defconstant +atat-thigh+ 1.56)
(defconstant +atat-shin+ 1.40)
(defconstant +atat-hip-y+ 3.15)

(defun atat-leg (fx fz th walking)
  (let* ((sw (if walking (* 0.44 (sin th)) 0.0))
         ;; lift only on the forward half of the stroke, so the planted foot
         ;; never skates
         (lift (if walking (max 0.0 (* 0.26 (cos th))) 0.0))
         (ankx (+ fx 0.10 sw))
         (anky (+ 0.44 lift))
         (dx (- ankx fx))
         (dy (- anky +atat-hip-y+))
         (d
          (min (* 0.995 (+ +atat-thigh+ +atat-shin+))
               (max 0.3 (sqrt (+ (* dx dx) (* dy dy))))))
         (ux (/ dx d))
         (uy (/ dy d))
         ;; cosine rule: how far along hip->ankle the knee's foot-point lies,
         ;; and how far it stands off that line
         (a
          (/ (+ (* d d)
                (- (* +atat-thigh+ +atat-thigh+) (* +atat-shin+ +atat-shin+)))
             (* 2.0 d)))
         (hh (sqrt (max 0.0 (- (* +atat-thigh+ +atat-thigh+) (* a a)))))
         ;; the offset direction, chosen to put the knee FORWARD (+x local)
         (kx (+ fx (* a ux) (* hh (- 0.0 uy))))
         (ky (+ +atat-hip-y+ (* a uy) (* hh ux))))
    ;; hip housing and ball
    (metal 0.56 0.58 0.61)
    (part-rbox fx +atat-hip-y+ fz 0.34 0.30 0.34 0.14)
    (metal 0.40 0.42 0.45)
    (part-joint fx +atat-hip-y+ fz 0.28)
    ;; thigh, knee, shin, ankle -- each segment runs ball to ball, so no caps
    (metal 0.58 0.60 0.63)
    (limb-caps nil)
    (taper 0.23)
    (part-limb fx +atat-hip-y+ fz kx ky fz 0.31)
    (metal 0.40 0.42 0.45)
    (part-joint kx ky fz 0.26)
    (metal 0.58 0.60 0.63)
    (taper 0.19)
    (part-limb kx ky fz ankx anky fz 0.24)
    (limb-caps t)
    (metal 0.40 0.42 0.45)
    (part-joint ankx anky fz 0.20)
    ;; the broad foot pad
    (metal 0.50 0.52 0.55)
    (part-rbox (+ ankx 0.05) (+ lift 0.16) fz 0.40 0.15 0.33 0.11)))

(defun emit-atat (i tm)
  (setq *hit-tint* (min 1.0 (* (aref *ahit* i) 6.0))) ; red when just struck
  (let* ((p (aref *apos* i))
         (dx (- (aref *ppos* 0) (aref p 0)))
         (dz (- (aref *ppos* 2) (aref p 2)))
         (far2 (+ (* dx dx) (* dz dz)))) ; squared horizontal distance to you
    (set-lod (aref p 0) (aref p 2) 5.5)
    (if (aref *aalive* i)
        (let* ((yaw (aref *ayaw* i)) (walking (> far2 170.0)) (th (* tm 2.1)))
          (set-origin (aref p 0) 0.0 (aref p 2) yaw)
          ;; the four legs, on the diagonal gait a quadruped actually uses:
          ;; front-left with rear-right, front-right with rear-left
          (atat-leg 1.05 -0.92 th walking)
          (atat-leg -1.05 0.92 th walking)
          (atat-leg 1.05 0.92 (+ th +pi+) walking)
          (atat-leg -1.05 -0.92 (+ th +pi+) walking)
          ;; the hull: a belly the legs hang from, the main armoured box, a
          ;; dorsal ridge and the rear engine block with its two thrusters
          (metal 0.54 0.56 0.59)
          (part-rbox 0.0 3.28 0.0 1.30 0.24 0.82 0.16)
          (metal 0.64 0.66 0.69)
          (part-rbox 0.0 3.88 0.0 1.55 0.55 0.98 0.26)
          (metal 0.58 0.60 0.63)
          (part-rbox -0.15 4.48 0.0 1.12 0.13 0.60 0.09)
          (metal 0.46 0.48 0.51)
          (part-rbox 0.10 3.80 -1.00 1.05 0.30 0.07 0.05)
          (part-rbox 0.10 3.80 1.00 1.05 0.30 0.07 0.05)
          (metal 0.44 0.46 0.49)
          (part-rbox -1.64 3.82 0.0 0.22 0.42 0.72 0.13)
          (metal 0.26 0.27 0.29)
          (part-limb -1.80 3.82 -0.36 -2.00 3.82 -0.36 0.17)
          (part-limb -1.80 3.82 0.36 -2.00 3.82 0.36 0.17)
          ;; the neck: a narrow trunk that slopes DOWN and forward out of the
          ;; hull's chest, with two armour rings, so the head hangs clear
          ;; below the hull line the way the real machine's does -- run it
          ;; level with the hull instead and the whole thing reads as a dog
          (metal 0.50 0.52 0.55)
          (taper 0.28)
          (part-limb 1.42 3.86 0.0 2.14 3.52 0.0 0.42)
          (metal 0.60 0.62 0.65)
          (part-limb 1.60 3.78 0.0 1.70 3.73 0.0 0.40)
          (part-limb 1.92 3.63 0.0 2.02 3.58 0.0 0.34)
          ;; the head: a blunt armoured box with a dark visor band, a jaw, two
          ;; temple cannons and the heavy chin blasters
          (metal 0.66 0.68 0.71)
          (part-rbox 2.54 3.40 0.0 0.42 0.40 0.50 0.13)
          (col 0.13 0.14 0.16)
          (part-rbox 2.94 3.52 0.0 0.07 0.12 0.40 0.04)
          (metal 0.58 0.60 0.63)
          (part-rbox 2.60 3.02 0.0 0.34 0.13 0.38 0.07)
          (metal 0.44 0.46 0.49)
          (part-rbox 2.30 3.60 -0.56 0.16 0.14 0.12 0.05)
          (part-rbox 2.30 3.60 0.56 0.16 0.14 0.12 0.05)
          (metal 0.22 0.22 0.24)
          (part-limb 2.42 3.60 -0.56 2.90 3.60 -0.56 0.055)
          (part-limb 2.42 3.60 0.56 2.90 3.60 0.56 0.055)
          (metal 0.20 0.20 0.22)
          (taper 0.058)
          (part-limb 2.70 3.08 -0.20 3.34 3.04 -0.20 0.085)
          (taper 0.058)
          (part-limb 2.70 3.08 0.20 3.34 3.04 0.20 0.085))
        ;; a smoking wreck: the hull slumped into the snow with its neck bent
        ;; under it, and a column of soot still lifting off the engine block
        (let* ((k (min 1.0 (/ (aref *awreck* i) 1.2))) (drop (* 0.9 k)))
          (set-origin (aref p 0) 0.0 (aref p 2) (aref *ayaw* i))
          (metal 0.34 0.34 0.35)
          (part-rbox 0.0 (- 1.5 drop) 0.0 1.5 0.55 0.95 0.24)
          (metal 0.28 0.28 0.30)
          (part-rbox -1.5 (- 1.4 drop) 0.0 0.24 0.40 0.70 0.13)
          (taper 0.30)
          (part-limb 1.3 (- 1.4 drop) 0.0 2.0 (- 0.6 (* 0.3 k)) 0.0 0.44)
          (col 0.22 0.22 0.24)
          (part-rbox 2.35 (- 0.62 (* 0.3 k)) 0.0 0.40 0.36 0.46 0.12)
          ;; the collapsed legs, splayed where they folded
          (metal 0.38 0.40 0.43)
          (taper 0.16)
          (part-limb 1.0 (- 1.2 drop) -0.9 1.9 0.22 -1.5 0.24)
          (taper 0.16)
          (part-limb -1.0 (- 1.2 drop) 0.9 -1.9 0.22 1.5 0.24)
          (taper 0.16)
          (part-limb 1.0 (- 1.2 drop) 0.9 1.7 0.22 1.6 0.24)
          (taper 0.16)
          (part-limb -1.0 (- 1.2 drop) -0.9 -1.8 0.22 -1.4 0.24)
          ;; soot, thinning as it climbs
          (col 0.30 0.30 0.32)
          (part-ellipsoid -1.4 (+ 2.1 (* 0.8 k)) 0.0 0.55 0.40 0.50 7 4)
          (col 0.42 0.42 0.44)
          (part-ellipsoid -1.1 (+ 3.0 (* 1.4 k)) 0.2 0.42 0.34 0.40 6 3))))
  (setq *hit-tint* 0.0))

;; Vader's blade, stored as a generic oriented box like yours.
(defvar *vsab-cx* 0.0)
(defvar *vsab-cy* 0.0)
(defvar *vsab-cz* 0.0)
(defvar *vsab-hx* 0.03)
(defvar *vsab-hy* 0.66)
(defvar *vsab-hz* 0.03)
(defvar *vsab-yaw* 0.0)
(defvar *vsab-vis* nil)

;; --- the cape --------------------------------------------------------------
;;
;; A cape is the one part of the boss that a box can never stand in for: its
;; whole character is that it is a SURFACE -- it wraps the shoulders, widens
;; towards the hem, and hangs in folds. So it is sampled from a parametric
;; patch, u running from the collar (0) to the hem (1) and v across the back
;; (-1 to 1). The half-angle it subtends and its distance from the body axis
;; both grow with u, which is the flare; a sine in v adds the standing folds,
;; and a slow drift in the phase makes them breathe as he walks.
;;
;; Normals come from finite differences of the same function, so the folds
;; catch the light instead of being painted-on stripes. The patch is emitted in
;; the figure's local frame like everything else.
(defconstant +cape-u+ 7) ; bands from collar to hem
(defconstant +cape-v+ 9) ; panels across the back

(defun cape-set (u v ph)
  (let* ((spread (+ 0.80 (* 0.45 u))) ; half-angle around the body
         (ang (* v spread))
         (fold (* 0.05 u (sin (+ (* 4.0 v) ph))))
         (rad (+ 0.21 (* 0.26 u u) fold))
         ;; the top edge follows the shoulder line down towards the front
         ;; instead of running level, or the collar reads as a shelf
         (yy (- (- 1.58 (* 0.13 v v)) (* 1.40 u))))
    (setq *cp-x* (- 0.0 (* rad (cos ang))) *cp-y* yy *cp-z* (* rad (sin ang)))))

(defun cape-vertex (u v ph)
  (cape-set u v ph)
  (let ((px *cp-x*) (py *cp-y*) (pz *cp-z*))
    (cape-set (min 1.0 (+ u 0.03)) v ph)
    (let ((ax (- *cp-x* px)) (ay (- *cp-y* py)) (az (- *cp-z* pz)))
      (cape-set u (min 1.0 (+ v 0.03)) ph)
      (let* ((bx (- *cp-x* px))
             (by (- *cp-y* py))
             (bz (- *cp-z* pz))
             (nx (- (* ay bz) (* az by)))
             (ny (- (* az bx) (* ax bz)))
             (nz (- (* ax by) (* ay bx)))
             (nl (max 0.000001 (sqrt (+ (* nx nx) (* ny ny) (* nz nz)))))
             ;; face the normal AWAY from the body axis -- the cross product's
             ;; sign flips as v crosses the back's centre line
             (sgn (if (< (+ (* nx px) (* nz pz)) 0.0) (- 0.0 1.0) 1.0))
             (mx (* sgn (/ nx nl)))
             (my (* sgn (/ ny nl)))
             (mz (* sgn (/ nz nl))))
        (emit-vertex (lwx px pz) (lwy py) (lwz px pz)
                     (+ (* *oc* mx) (* *os* mz)) my
                     (- (* *oc* mz) (* *os* mx)))))))

(defun emit-cape (ph)
  (dotimes (j +cape-u+)
    (let ((u0 (/ (float j) (float +cape-u+)))
          (u1 (/ (float (+ j 1)) (float +cape-u+))))
      (dotimes (i +cape-v+)
        (let ((v0 (- (* 2.0 (/ (float i) (float +cape-v+))) 1.0))
              (v1 (- (* 2.0 (/ (float (+ i 1)) (float +cape-v+))) 1.0)))
          (cape-vertex u0 v0 ph)
          (cape-vertex u1 v0 ph)
          (cape-vertex u1 v1 ph)
          (cape-vertex u0 v0 ph)
          (cape-vertex u1 v1 ph)
          (cape-vertex u0 v1 ph))))))

;; Vader: all black armour under a heavy cape, the domed helmet with its
;; flared mask, the chest control box, a red lightsaber. Tall (~1.9) and broad.
;; Only shown once he engages -- dormant until the walkers fall.
(defun emit-vader (tm)
  (setq *vsab-vis* nil)
  (setq *hit-tint* (min 1.0 (* *vhitf* 6.0))) ; red when just struck
  (when (and *valive* *vactive*)
    (let ((vdx (aref *vpos* 0)) (vdz (aref *vpos* 2)))
      (set-lod vdx vdz 1.95)
      (set-origin vdx 0.0 vdz *vyaw*)
      (emit-shadow vdx vdz 0.46)
      ;; legs and boots, under the cape
      (col 0.07 0.07 0.08)
      (humanoid-leg -0.135 0.0 0.90 1.10)
      (col 0.05 0.05 0.06)
      (part-rbox 0.045 0.075 -0.135 0.115 0.075 0.105 0.045)
      (col 0.07 0.07 0.08)
      (humanoid-leg 0.135 0.0 0.90 1.10)
      (col 0.05 0.05 0.06)
      (part-rbox 0.045 0.075 0.135 0.115 0.075 0.105 0.045)
      ;; hips, the wide belt and its side boxes
      (col 0.08 0.08 0.09)
      (part-rbox 0.0 0.96 0.0 0.185 0.10 0.145 0.07)
      (col 0.11 0.11 0.12)
      (part-rbox 0.0 0.88 0.0 0.195 0.048 0.152 0.026)
      (metal 0.34 0.29 0.16)
      (part-rbox 0.14 0.88 -0.095 0.042 0.048 0.042 0.016)
      (part-rbox 0.14 0.88 0.095 0.042 0.048 0.042 0.016)
      (part-rbox 0.16 0.88 0.0 0.030 0.036 0.038 0.014)
      ;; torso: a broad armoured chest over a narrower midriff
      (col 0.08 0.08 0.09)
      (part-rbox 0.0 1.36 0.0 0.195 0.185 0.155 0.085)
      (part-rbox 0.0 1.12 0.0 0.165 0.115 0.135 0.065)
      ;; the chest control box and its indicator lights
      (metal 0.14 0.14 0.16)
      (part-rbox 0.16 1.32 0.0 0.055 0.115 0.105 0.024)
      (glow-col 1.0 0.2 0.18)
      (part-ellipsoid 0.215 1.375 -0.045 0.018 0.018 0.018 6 3)
      (part-ellipsoid 0.215 1.300 -0.045 0.016 0.016 0.016 6 3)
      (glow-col 0.2 0.85 0.3)
      (part-ellipsoid 0.215 1.338 0.040 0.015 0.015 0.015 6 3)
      (metal 0.18 0.18 0.20) ; the shoulder-strap clasps
      (part-limb -0.02 1.55 -0.085 0.13 1.20 -0.075 0.022)
      (part-limb -0.02 1.55 0.085 0.13 1.20 0.075 0.022)
      ;; shoulder mantles, arms and gloved fists
      (metal 0.10 0.10 0.12)
      (part-rbox 0.0 1.545 -0.215 0.105 0.055 0.115 0.05)
      (part-rbox 0.0 1.545 0.215 0.105 0.055 0.115 0.05)
      (col 0.07 0.07 0.08)
      (taper 0.062)
      (part-limb 0.0 1.50 -0.235 0.05 1.24 -0.255 0.078)
      (part-joint 0.05 1.24 -0.255 0.064)
      (taper 0.055)
      (part-limb 0.05 1.24 -0.255 0.02 1.03 -0.262 0.064)
      (col 0.03 0.03 0.04)
      (part-ellipsoid 0.02 1.005 -0.262 0.058 0.062 0.058 7 4)
      ;; neck, and the standing collar behind it -- the cape hangs off this,
      ;; and its two raked panels are as much of the silhouette as the helmet
      (col 0.06 0.06 0.07)
      (part-cyl 0.0 1.655 0.0 0.078 0.055)
      (col 0.05 0.05 0.06)
      (part-rbox -0.115 1.660 -0.105 0.055 0.115 0.075 0.028)
      (part-rbox -0.115 1.660 0.105 0.055 0.115 0.075 0.028)
      (part-rbox -0.145 1.630 0.0 0.045 0.090 0.115 0.030)
      ;; the helmet: the dome, the flared skirt that drops over the neck (a
      ;; cone widening downward -- that flare IS the silhouette), the raked
      ;; face mask, the eye lenses, the mouth grille and its ribs
      ;; a shade lighter than the cloth around it, and fully polished: black
      ;; armour against a black cape is legible only by its highlight
      (metal 0.11 0.11 0.13)
      (part-ellipsoid -0.012 1.815 0.0 0.122 0.128 0.126 10 6)
      (taper 0.152)
      (part-limb -0.012 1.815 0.0 -0.012 1.640 0.0 0.118)
      (metal 0.14 0.14 0.16) ; the raked face plate
      (part-rbox 0.088 1.800 0.0 0.062 0.105 0.100 0.030)
      (part-rbox 0.070 1.690 0.0 0.070 0.055 0.078 0.026)
      (col 0.02 0.02 0.03) ; the eye lenses
      (part-rbox 0.140 1.828 -0.050 0.022 0.030 0.034 0.010)
      (part-rbox 0.140 1.828 0.050 0.022 0.030 0.034 0.010)
      (col 0.05 0.05 0.06) ; the brow ridge between them
      (part-rbox 0.142 1.872 0.0 0.020 0.020 0.088 0.008)
      (col 0.13 0.13 0.14) ; the mouth grille
      (part-rbox 0.140 1.712 0.0 0.028 0.042 0.052 0.014)
      (metal 0.24 0.24 0.26)
      (part-limb 0.166 1.752 -0.030 0.166 1.674 -0.030 0.008)
      (part-limb 0.166 1.752 0.0 0.166 1.674 0.0 0.008)
      (part-limb 0.166 1.752 0.030 0.166 1.674 0.030 0.008)
      ;; the cape last, so it is drawn over the shoulders it hangs from
      (col 0.045 0.045 0.055)
      (emit-cape (* tm 1.4))
      ;; the red blade -- swept horizontally during a melee strike, like yours
      (let* ((theta (- 0.0 *vyaw*))
             (fwx (cos theta))
             (fwz (sin theta))
             (arx (- 0.0 (sin theta)))
             (arz (cos theta)))
        (if (> *vswing* 0.0)
            ;; the same upright diagonal slash as yours
            (let* ((p (- 1.0 (/ *vswing* 0.45)))
                   (side (- 0.46 (* 0.92 p)))
                   (reachf (+ 0.38 (* 0.24 (sin (* p +pi+)))))
                   (hh (+ 1.22 (* -0.42 p)))
                   (hx (+ vdx (* fwx reachf) (* arx side)))
                   (hz (+ vdz (* fwz reachf) (* arz side)))
                   (half 0.66))
              (metal 0.18 0.18 0.20)
              (emit-cylinder hx hh hz 0.028 0.06 0.0 8)
              (setq *vsab-cx* hx *vsab-cy* (+ hh 0.12 half) *vsab-cz* hz
                    *vsab-hx* 0.032 *vsab-hy* half *vsab-hz* 0.032 *vsab-yaw*
                    0.0 *vsab-vis* t)
              (emit-vader-blade))
            (let* ((hx (+ vdx (* fwx 0.44) (* arx 0.14)))
                   (hz (+ vdz (* fwz 0.44) (* arz 0.14)))
                   (hh 1.06)
                   (half 0.66))
              (metal 0.18 0.18 0.20)
              (emit-cylinder hx hh hz 0.028 0.09 0.0 8)
              (setq *vsab-cx* hx *vsab-cy* (+ hh 0.12 half) *vsab-cz* hz
                    *vsab-hx* 0.032 *vsab-hy* half *vsab-hz* 0.032 *vsab-yaw*
                    0.0 *vsab-vis* t)
              (emit-vader-blade))))))
  (setq *hit-tint* 0.0))

(defun emit-vader-blade ()
  (glow-col 1.0 0.24 0.20)
  (emit-cylinder *vsab-cx* *vsab-cy* *vsab-cz* *vsab-hx* *vsab-hy* *vsab-yaw* 8)
  (glow-col 1.0 0.82 0.80)
  (emit-cylinder *vsab-cx* *vsab-cy* *vsab-cz* (max 0.012 (- *vsab-hx* 0.016))
                 (max 0.012 (- *vsab-hy* 0.016)) *vsab-yaw* 8))

(defun emit-vader-glow ()
  (when *vsab-vis*
    (glow-col 1.0 0.16 0.12)
    (emit-cylinder *vsab-cx* *vsab-cy* *vsab-cz* (glowh *vsab-hx*)
                   (glowh *vsab-hy*) *vsab-yaw* 8)
    (glow-col 0.55 0.05 0.04)
    (emit-cylinder *vsab-cx* *vsab-cy* *vsab-cz* (glowh2 *vsab-hx*)
                   (glowh2 *vsab-hy*) *vsab-yaw* 8)))

;; round sparks (a low-segment sphere, cheap enough for a per-frame particle)
;; instead of glowing cubes.
(defun emit-firework-cores ()
  ;; the solid, self-lit spark -- drawn in the OPAQUE pass so it keeps its own
  ;; vivid color against the bright sky (an additive-only spark washes out white)
  (dotimes (i +nfw+)
    (when (> (aref *fwt* i) 0.0)
      (let ((p (aref *fwpos* i)))
        (glow-col (aref *fwr* i) (aref *fwg* i) (aref *fwb* i))
        (emit-ellipsoid (aref p 0) (aref p 1) (aref p 2) 0.14 0.14 0.14 0.0 6
                        3)))))

(defun emit-fireworks ()
  ;; an additive halo around each spark, fading as it dies -> the bloom
  (dotimes (i +nfw+)
    (when (> (aref *fwt* i) 0.0)
      (let ((k (/ (aref *fwt* i) (aref *fwmax* i))) (p (aref *fwpos* i)))
        (glow-col (* 0.7 k (aref *fwr* i)) (* 0.7 k (aref *fwg* i))
                  (* 0.7 k (aref *fwb* i)))
        (emit-ellipsoid (aref p 0) (aref p 1) (aref p 2) 0.26 0.26 0.26 0.0 6
                        3)))))

;; a round beam instead of a box -- reads as a glowing capsule (per the
;; README) rather than a spinning rectangular slab.
(defun emit-bolt-core (i)
  (let* ((v (aref *bvel* i))
         (p (aref *bpos* i))
         (yaw (atan2 (- 0.0 (aref v 2)) (aref v 0)))
         (sz (aref *bsz* i)))
    (glow-col (aref *br* i) (aref *bg* i) (aref *bb* i))
    (emit-cyl-beam (aref p 0) (aref p 1) (aref p 2) (* 0.30 sz) (* 0.05 sz) yaw
                   6)))

(defun emit-bolt-glow (i)
  (let* ((v (aref *bvel* i))
         (p (aref *bpos* i))
         (yaw (atan2 (- 0.0 (aref v 2)) (aref v 0)))
         (sz (aref *bsz* i)))
    (glow-col (* 0.5 (aref *br* i)) (* 0.5 (aref *bg* i)) (* 0.5 (aref *bb* i)))
    (emit-cyl-beam (aref p 0) (aref p 1) (aref p 2) (* 0.42 sz) (* 0.13 sz) yaw
                   6)))

(defun emit-flash (i)
  (let ((k (/ (aref *ft* i) (aref *fttl* i))) (p (aref *fpos* i)))
    (glow-col (* k (aref *fr* i)) (* k (aref *fg* i)) (* k (aref *fb* i)))
    (let ((r (* (aref *fsz* i) (+ 0.4 (* 0.6 (- 1.0 k))))))
      (emit-ellipsoid (aref p 0) (aref p 1) (aref p 2) r r r 0.0 6 3))))

;; --- the frame ----------------------------------------------------------------

(defun draw (tm)
  (let ((w (canvas-width)) (h (canvas-height)))
    (gl:viewport 0 0 (floor w) (floor h)))
  (gl:clear-color 0.74 0.83 0.93 1.0)
  (gl:clear (+ gl:+color-buffer-bit+ gl:+depth-buffer-bit+))
  (gl:use-program *prog*)
  (upload-vp *u-vp*)
  (gl:uniform3f *u-eye* (aref *eye* 0) (aref *eye* 1) (aref *eye* 2))

  ;; --- opaque pass: bodies, walkers, bolt cores, blade cores --------------
  (setq *v* *static-verts*)
  (emit-player tm)
  (dotimes (i *ntrooper*) (emit-trooper i tm))
  (dotimes (i *natat*) (emit-atat i tm))
  (emit-vader tm)
  (dotimes (i +nbolt+) (when (aref *balive* i) (emit-bolt-core i)))
  (emit-firework-cores)
  (let ((opaque-end *v*))
    ;; --- bloom pass: additive blade shells, bolt halos, flashes -----------
    (emit-player-saber-glow)
    (emit-vader-glow)
    (dotimes (i +nbolt+) (when (aref *balive* i) (emit-bolt-glow i)))
    (dotimes (i +nflash+) (when (> (aref *ft* i) 0.0) (emit-flash i)))
    (emit-fireworks)
    (gl:bind-buffer gl:+array-buffer+ *buf*)
    (gl-upload-vertices (* *static-verts* 11) (* (- *v* *static-verts*) 11))
    (gl:bind-vertex-array *vao*)
    ;; opaque geometry writes depth as usual
    (gl:draw-arrays gl:+triangles+ 0 opaque-end)
    ;; the glow accumulates additively and does not write depth
    (gl:enable gl:+blend+)
    (gl:blend-func gl:+one+ gl:+one+)
    (gl:depth-mask nil)
    (gl:draw-arrays gl:+triangles+ opaque-end (- *v* opaque-end))
    (gl:depth-mask t)
    (gl:disable gl:+blend+)))

(defun reset-game ()
  (setq *ppos* #f(0.0 0.0 0.0))
  (setq *pvel* #f(0.0 0.0 0.0))
  (setq *grounded* t)
  (setq *in-jump* 0.0)
  (setq *jump-prev* nil)
  (setq *inv-t* 0.0)
  (setq *php* +player-max-hp+)
  (setq *weapon* 1) ; start armed with the blaster
  (setq *attack* 0.0)
  (setq *swing-t* 0.0)
  (setq *swing-cd* 0.0)
  (setq *fire-cd* 0.0)
  (setq *hurt-flash* 0.0)
  (setq *state* 0)
  (setq *state-t* 0.0)
  (setq *cam-yaw* +cam-yaw-0+)
  (setq *cam-pitch* +cam-pitch-0+)
  (setq *cam-dist* +cam-dist-0+)
  (setq *cam* #f(0.0 0.0 0.0))
  (dotimes (i +nbolt+) (setf (aref *balive* i) nil))
  (dotimes (i +nflash+) (setf (aref *ft* i) 0.0))
  (dotimes (i +nfw+) (setf (aref *fwt* i) 0.0))
  (setq *fw-launch* 0.0)
  (reset-troopers)
  (reset-atats)
  (reset-vader))

(defun frame (tm)
  (when *pending-reset*
    (setq *pending-reset* nil)
    (reset-game))
  (let ((dt (min 0.05 (max 0.0 (- tm *last-tm*)))))
    (setq *last-tm* tm)
    (setq *aspect* (/ (canvas-width) (canvas-height)))
    (setq *state-t* (+ *state-t* dt))
    (update-aim)
    (cond ((= *state* 0) (step-playing dt)) ((= *state* 1) (step-victory dt)))
    (update-bolts dt)
    (update-flashes dt)
    (update-enemy-flashes dt)
    (update-camera dt)
    (draw tm)))

;; --- HUD taps -----------------------------------------------------------------

(defun get-state () *state*)
(defun get-weapon () *weapon*)
(defun get-hp () ; 0..100 percentage, for the HP bar
  (floor (* 100.0 (/ *php* +player-max-hp+))))
(defun get-troopers () (troopers-alive))
(defun get-walkers () (atats-alive))
(defun boss-active () (if (and *vactive* *valive*) 1 0)) ; hide the bar once he falls
(defun boss-hp ()                                        ; 0..100, for the boss bar
  (if *valive* (floor (* 100.0 (/ *vhp* +vader-hp+))) 0))
(defun get-hurt () *hurt-flash*)
(defun get-px () (aref *ppos* 0))
(defun get-py () (aref *ppos* 1))
(defun get-pz () (aref *ppos* 2))

;; --- boot ---------------------------------------------------------------------
;; Runs inside _initialize, after the page has created the WebGL2 context: build
;; the pipeline, parse the roster and bake the snow field.

(setup-gl)
(parse-troopers)
(parse-atats)
(bake-static)
(reset-game)

(rontolisp:wasm-export 'frame :params '(:float) :returns :void)
(rontolisp:wasm-export 'set-key
                       :as "setKey"
                       :params '(:int :int)
                       :returns :void)
(rontolisp:wasm-export 'set-attack
                       :as "setAttack"
                       :params '(:int)
                       :returns :void)
(rontolisp:wasm-export 'switch-weapon
                       :as "switchWeapon"
                       :params '()
                       :returns :void)
(rontolisp:wasm-export 'orbit :params '(:float :float) :returns :void)
(rontolisp:wasm-export 'zoom :params '(:float) :returns :void)
(rontolisp:wasm-export 'restart :params '() :returns :void)
(rontolisp:wasm-export 'get-state :as "getState" :params '() :returns :int)
(rontolisp:wasm-export 'get-weapon :as "getWeapon" :params '() :returns :int)
(rontolisp:wasm-export 'get-hp :as "getHp" :params '() :returns :int)
(rontolisp:wasm-export 'get-troopers
                       :as "getTroopers"
                       :params '()
                       :returns :int)
(rontolisp:wasm-export 'get-walkers :as "getWalkers" :params '() :returns :int)
(rontolisp:wasm-export 'boss-active :as "bossActive" :params '() :returns :int)
(rontolisp:wasm-export 'boss-hp :as "bossHp" :params '() :returns :int)
(rontolisp:wasm-export 'get-hurt :as "getHurt" :params '() :returns :float)
(rontolisp:wasm-export 'get-px :as "getPx" :params '() :returns :float)
(rontolisp:wasm-export 'get-py :as "getPy" :params '() :returns :float)
(rontolisp:wasm-export 'get-pz :as "getPz" :params '() :returns :float)


---

# FILE: references/examples/browser/webgl-battlefront/build.sh

#!/usr/bin/env bash
# Recompile battlefront.lisp to a browser-loadable WebAssembly reactor.
# --no-wasi drops all WASI imports, so the module's only imports are the host
# functions battlefront.lisp declares with rontolisp:wasm-import ("gl",
# "canvas" and "math") plus the shared WebGL2 package; --optimize tree-shakes
# the runtime so only the reachable functions ship.
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling battlefront.lisp -> battlefront.wasm"
java -jar "$jar" "$here/battlefront.lisp" -o "$here/battlefront.wasm" --no-wasi --optimize

# The page imports the generated ../webgl-common/gl-imports.js, so the served
# root is examples/browser rather than this directory.
echo "done. Serve the examples/browser directory over http, e.g.:"
echo "  jwebserver -p 8000 --directory \"$(dirname "$here")\""
echo "then open http://localhost:8000/webgl-battlefront/"


---

# FILE: references/examples/browser/webgl-battlefront/index.html



  
    
    
    battlefront.lisp — a one-arena snow-battle skirmish, in Lisp
    
  
  
    

    

    


    

    

      
Vader

      

      
deflects blaster fire — use the lightsaber (F)

    


    

      

        

          You · HP
          
100%

        

        

          BLASTER
        

      

    


    

    
    

    

      

        
HOTH

        

          Move with WASD, aim with the mouse,
          click to attack.

          F swaps the lightsaber and the blaster — Vader's blade deflects
          blaster bolts,
so cut the boss down with your own.
        

        
      

    


    

      

      

      

      

        
        
        
      

    


    

      

        
THE EMPIRE FALLS

        

        
      

    


    

      

        
GAME OVER

        
Your HP hit zero — the garrison holds the field.
Regroup and try again.

        
      

    


    

    
  



---

# FILE: references/examples/browser/webgl-common/README.md

# webgl-common — the shared `gl` package for the WebGL demos

Every `examples/webgl-*` demo used to repeat the same block of host-import
directives for the WebGL2 API, the same WebGL enum constants and the same
shader-compilation helpers — and every page repeated the matching block of
JavaScript bindings. That boundary is now declared once, in `gl.wit`: `gl.lisp`
binds it for the Lisp side, `gl-imports.js` is generated from it for the page
side. A demo splices the package in at compile time with

```lisp
(require :gl "../webgl-common/gl.lisp")
```

and then talks to WebGL through the package:

```lisp
(let ((program (gl:build-program +vertex-shader-source+ +fragment-shader-source+)))
  (gl:use-program program)
  (gl:enable gl:+depth-test+)
  ...
  (gl:draw-arrays gl:+triangles+ 0 36))
```

## What's in here

| File            | Purpose                                                                |
| --------------- | ---------------------------------------------------------------------- |
| `gl.wit`        | The boundary itself: `interface gl` (29 WebGL2 functions) + `interface ui` (`fail`). |
| `gl.lisp`       | The `gl` package: two `rontolisp:wit-import` directives, the enum constants, the shader helpers. |
| `gl-imports.js` | The page's half, GENERATED from `gl.wit` — do not edit.                |

## The boundary is `gl.wit`

`gl.lisp` no longer spells the WebGL2 API out. Two directives bind the two
interfaces of `gl.wit`:

```lisp
(rontolisp:wit-import "gl.wit" :interface "local:webgl/gl")
(rontolisp:wit-import "gl.wit" :interface "local:webgl/ui")
```

Each WIT function lowers into exactly the Preview 1 host import a hand-written
`rontolisp:wasm-import` would have declared, so the compiled module is
unchanged. The directives bind into the *current* package rather than naming
one, so under `(in-package gl)` the bindings land beside the constants and
helpers below — which is why the `defpackage` stays hand-written: it has to
export the constants and helpers too, and no directive knows about those.

The page's import object is generated from the same file
(`gl-imports.js`, regenerated by `GlImportObjectTest`), and each page spreads it
in:

```js
import { glImports, uiImports } from "../webgl-common/gl-imports.js";

const imports = {
  gl: { ...glImports({ gl: gl2, handles, addHandle, str, retStr }),
        setVertex: (i, x, y) => { /* the demo's own staging entries */ } },
  ui: uiImports({ ui, str }),
};
```

So the two sides of the boundary cannot drift: a page can no longer provide a
field the module does not import, or spell one differently, without `gl.wit`
saying so.

## What it exports

- **The WebGL2 API** — `gl:create-shader`, `gl:shader-source`, ...
  `gl:draw-arrays`: the 29 functions of `gl.wit`'s `gl` interface, the union of
  every literal WebGL2 entry the demos use. GL objects cross the boundary as
  `s32` handles into a table the page keeps (named through `gl.wit`'s `shader`,
  `program`, `buffer`, `vertex-array` and `uniform-location` aliases, which are
  plain aliases of `s32`); strings (GLSL source, uniform names, info logs) cross
  as `string`.
- **The WebGL enum constants** — `gl:+vertex-shader+`, `gl:+float+`,
  `gl:+color-buffer-bit+`, ... — hand-written in `gl.lisp`.
- **Shader helpers** — `(gl:make-shader type source)` compiles one shader and
  `(gl:build-program vs fs)` links a whole program, both reporting compile/link
  errors through `gl.wit`'s `ui` interface (`fail`, which shows the page's error
  box and throws). Also hand-written in `gl.lisp`.

Each demo keeps its own page-specific staging imports (`setVertex`, `setFloat`,
...) declared with `rontolisp:wasm-import` next to its own code — those are not
part of the WebGL2 API, their shapes differ per demo, and they deliberately stay
off the WIT.

## A note on the names

A WIT label is ONE name: it becomes the Lisp name verbatim (`create-shader` →
`gl:create-shader`) and, camelCased, the host import field (`createShader`) —
which is what the WebGL2 API already calls itself. The old directives could give
each side a different name with `:as`; a WIT label cannot. Four functions were
therefore renamed to their WebGL spellings, and code written against the old
names needs updating:

| Old (`gl.lisp` before) | Now                     |
| ---------------------- | ----------------------- |
| `gl:shader-compiled-p` | `gl:get-shader-parameter` |
| `gl:shader-info-log`   | `gl:get-shader-info-log`  |
| `gl:program-linked-p`  | `gl:get-program-parameter` |
| `gl:program-info-log`  | `gl:get-program-info-log`  |

The names are the WebGL ones rather than Lisp-idiomatic predicates because
`get-shader-parameter` cannot also be spelled `shader-compiled-p` without giving
the boundary two names to keep in sync. A package qualifier never leaks into the
page's import object.

## Why importing the union is free

The demos compile with `--optimize`, and the tree-shaker drops unused host
imports. So although `gl.wit` declares every entry any demo needs, each compiled
`.wasm` imports only what that demo reaches — between 26 and 44 functions,
staging and canvas entries included.

One nuance: a program that takes functions as values — e.g. through the spliced
`linalg` library — keeps the same-arity import wrappers reachable through the
`funcall` dispatcher, so a couple of entries the demo never calls can survive
the shake. No page has to know: every page provides the whole generated union by
spreading `glImports`, so a surviving entry is already there.

`examples/browser/webgl-triangle` deliberately does not use this package: it is
the smallest complete `rontolisp:wasm-import` program, and staying a single
self-contained file is the point.


---

# FILE: references/examples/browser/webgl-common/gl-imports.js

// GENERATED from gl.wit -- do not edit.
//
// The host side of the WebGL2 boundary the examples/browser/webgl-* demos are
// written against: one entry per function of gl.wit, which is the same file
// gl.lisp binds with rontolisp:wit-import. Both halves of every name, and the
// handle/string plumbing around it, are derived from that one declaration, so
// the page can no longer provide a field the module does not import (or spell
// one differently) without the WIT saying so.
//
// Each factory below takes the page's own host plumbing and returns a plain
// object of plain functions, so a page spreads it into its import object and
// adds its own demo-specific staging entries beside it:
//
//   gl: { ...glImports({ gl, handles, addHandle, str, retStr }),
//         setVertex: (i, x, y) => { ... } },
//
// A later property wins, so a page that needs a different implementation of a
// generated entry can simply restate it after the spread.
//
// Regenerate with:
//   ./mvnw -Drontolisp.gl.fix=true -Dtest=GlImportObjectTest#fixGlImports test

/** The `gl` import module, one entry per gl.wit function. */
export function glImports({ gl, addHandle, handles, str, retStr }) {
  return {
    createShader: (kind) => addHandle(gl.createShader(kind)),
    shaderSource: (shader, source, sourceLen) =>
      gl.shaderSource(handles[shader], str(source, sourceLen)),
    compileShader: (shader) => gl.compileShader(handles[shader]),
    getShaderParameter: (shader, pname) => gl.getShaderParameter(handles[shader], pname),
    getShaderInfoLog: (shader) => retStr(gl.getShaderInfoLog(handles[shader]) ?? ""),
    createProgram: () => addHandle(gl.createProgram()),
    attachShader: (program, shader) => gl.attachShader(handles[program], handles[shader]),
    linkProgram: (program) => gl.linkProgram(handles[program]),
    getProgramParameter: (program, pname) => gl.getProgramParameter(handles[program], pname),
    getProgramInfoLog: (program) => retStr(gl.getProgramInfoLog(handles[program]) ?? ""),
    useProgram: (program) => gl.useProgram(handles[program]),
    getUniformLocation: (program, name, nameLen) =>
      addHandle(gl.getUniformLocation(handles[program], str(name, nameLen))),
    uniform1f: (location, x) => gl.uniform1f(handles[location], x),
    uniform3f: (location, x, y, z) => gl.uniform3f(handles[location], x, y, z),
    enable: (cap) => gl.enable(cap),
    disable: (cap) => gl.disable(cap),
    depthMask: (flag) => gl.depthMask(!!flag),
    blendFunc: (src, dst) => gl.blendFunc(src, dst),
    createBuffer: () => addHandle(gl.createBuffer()),
    bindBuffer: (target, buffer) => gl.bindBuffer(target, handles[buffer]),
    bufferData: (target, size, usage) => gl.bufferData(target, size, usage),
    createVertexArray: () => addHandle(gl.createVertexArray()),
    bindVertexArray: (array) => gl.bindVertexArray(handles[array]),
    enableVertexAttribArray: (index) => gl.enableVertexAttribArray(index),
    vertexAttribPointer: (index, size, kind, normalized, stride, offset) =>
      gl.vertexAttribPointer(index, size, kind, !!normalized, stride, offset),
    viewport: (x, y, width, height) => gl.viewport(x, y, width, height),
    clearColor: (red, green, blue, alpha) => gl.clearColor(red, green, blue, alpha),
    clear: (mask) => gl.clear(mask),
    drawArrays: (mode, first, count) => gl.drawArrays(mode, first, count),
  };
}

/** The `ui` import module, one entry per gl.wit function. */
export function uiImports({ ui, str }) {
  return {
    fail: (message, messageLen) => ui.fail(str(message, messageLen)),
  };
}


---

# FILE: references/examples/browser/webgl-common/gl.lisp

;;;; gl.lisp -- the shared WebGL2 host boundary for the examples/webgl-* demos.
;;;;
;;;; Every browser demo used to repeat the same block of host-import directives,
;;;; WebGL enum constants and shader-compilation helpers. This file factors that
;;;; block into a `gl` package: a demo splices it in at compile time with
;;;;
;;;;   (require :gl "../webgl-common/gl.lisp")
;;;;
;;;; and then calls (gl:create-shader ...), reads gl:+float+, or just builds a
;;;; whole pipeline with (gl:build-program vs fs).
;;;;
;;;; The boundary itself is not written here: it is `gl.wit`, and the two
;;;; rontolisp:wit-import directives below bind it. Each WIT function lowers into
;;;; exactly the host import a hand-written rontolisp:wasm-import would have
;;;; declared, so the compiled module is unchanged -- but the page's import
;;;; object is GENERATED from the same file (gl-imports.js), so the Lisp side and
;;;; the JavaScript side can no longer drift apart. See gl.wit for the type
;;;; conventions (GL objects cross as s32 handles into a table the page keeps;
;;;; GLSL sources, uniform names and info logs cross as strings).
;;;;
;;;; Imports the demo never calls are dropped by --optimize (the tree-shaker
;;;; removes unused host imports), so binding the full WebGL2 union below does
;;;; not grow any page's module: each page only imports what its own demo
;;;; actually reaches. Each demo still declares its own staging imports
;;;; (setVertex, setFloat, ...) with rontolisp:wasm-import next to its own code --
;;;; those are page-specific by design and deliberately stay off the WIT.
;;;;
;;;; The directives bind into the CURRENT package rather than naming one, so the
;;;; bindings land in `gl` beside the constants and helpers below: under
;;;; (in-package gl) each WIT label canonicalizes to gl:label (or gl::fail for
;;;; the unexported fail helper), which is what call sites resolve to. The
;;;; defpackage stays hand-written for the same reason -- it has to export the
;;;; constants and helpers too, which no directive knows about.

(provide :gl)

(defpackage gl
  (:use cl)
  (:export create-shader shader-source compile-shader get-shader-parameter
           get-shader-info-log create-program attach-shader link-program
           get-program-parameter get-program-info-log use-program
           get-uniform-location uniform1f uniform3f enable disable depth-mask
           blend-func create-buffer bind-buffer buffer-data create-vertex-array
           bind-vertex-array enable-vertex-attrib-array vertex-attrib-pointer
           viewport clear-color clear draw-arrays make-shader build-program
           +vertex-shader+ +fragment-shader+ +compile-status+ +link-status+
           +array-buffer+ +static-draw+ +dynamic-draw+ +float+ +blend+
           +depth-test+ +one+ +color-buffer-bit+ +depth-buffer-bit+ +points+
           +triangles+))

(in-package gl)

;; --- the WebGL2 API, and the page's fatal-error reporting ----------------------
;; Fatal-error reporting for the shader helpers below shows the page's error box
;; (and stops the program by throwing on the JavaScript side). It is internal to
;; this package -- demos report their own errors through their own imports.

(rontolisp:wit-import "gl.wit" :interface "local:webgl/gl")
(rontolisp:wit-import "gl.wit" :interface "local:webgl/ui")

;; --- WebGL constants -----------------------------------------------------------
;; The numeric enum values from the WebGL specification.

(defconstant +vertex-shader+ 35633)   ; 0x8B31
(defconstant +fragment-shader+ 35632) ; 0x8B30
(defconstant +compile-status+ 35713)  ; 0x8B81
(defconstant +link-status+ 35714)     ; 0x8B82
(defconstant +array-buffer+ 34962)    ; 0x8892
(defconstant +static-draw+ 35044)     ; 0x88E4
(defconstant +dynamic-draw+ 35048)    ; 0x88E8
(defconstant +float+ 5126)            ; 0x1406
(defconstant +blend+ 3042)            ; 0x0BE2
(defconstant +depth-test+ 2929)       ; 0x0B71
(defconstant +one+ 1)
(defconstant +color-buffer-bit+ 16384) ; 0x4000
(defconstant +depth-buffer-bit+ 256)   ; 0x0100
(defconstant +points+ 0)
(defconstant +triangles+ 4)

;; --- shader helpers --------------------------------------------------------------

(defun make-shader (type source)
  ;; Compile one shader, failing loudly with the driver's info log.
  (let ((shader (create-shader type)))
    (shader-source shader source)
    (compile-shader shader)
    (unless (get-shader-parameter shader +compile-status+)
      (fail (get-shader-info-log shader)))
    shader))

(defun build-program (vs-source fs-source)
  ;; Compile both shaders and link them into a program (not yet in use).
  (let ((program (create-program)))
    (attach-shader program (make-shader +vertex-shader+ vs-source))
    (attach-shader program (make-shader +fragment-shader+ fs-source))
    (link-program program)
    (unless (get-program-parameter program +link-status+)
      (fail (get-program-info-log program)))
    program))

(in-package cl-user)


---

# FILE: references/examples/browser/webgl-common/gl.wit

// The WebGL2 boundary the examples/browser/webgl-* demos are written against.
//
// This file is the contract. gl.lisp binds it with rontolisp:wit-import, which
// lowers each function into the Preview 1 host import the page provides, and
// gl-imports.js -- the page's import object -- is GENERATED from it. So the two
// sides of the boundary are spelled once, here, instead of once in Lisp and
// again in every index.html.
//
// A WIT label is ONE name: it becomes the Lisp name verbatim (create-shader ->
// gl:create-shader) and, camelCased, the host import field (createShader) --
// which is what the WebGL2 API already calls itself. That is why the names here
// are the WebGL ones rather than Lisp-idiomatic predicates: get-shader-parameter
// cannot also be spelled shader-compiled-p without giving the boundary two names
// to keep in sync.
//
// Everything crosses inside the Preview 1 import boundary: s32, f32, bool and
// string. A GL object (shader, program, buffer, VAO, uniform location) crosses
// as an s32 handle into a table the page keeps -- named through the aliases
// below, which are plain aliases of s32 and lower identically, but tell a reader
// (and the JS generator) which integers are handles and which are values.
package local:webgl;

/// The subset of the WebGL2 rendering context the demos drive from Lisp. A page
/// implements it in one line per function over its own WebGL2 context.
interface gl {
  /// A compiled shader, held in the page's object table.
  type shader = s32;
  /// A linked shader program, held in the page's object table.
  type program = s32;
  /// A buffer object, held in the page's object table.
  type buffer = s32;
  /// A vertex array object, held in the page's object table.
  type vertex-array = s32;
  /// A uniform's location within a program, held in the page's object table.
  type uniform-location = s32;

  /// Create an empty shader of the given kind (gl:+vertex-shader+ or
  /// gl:+fragment-shader+).
  create-shader: func(kind: s32) -> shader;
  /// Replace the shader's GLSL source.
  shader-source: func(shader: shader, source: string);
  /// Compile the shader's current source.
  compile-shader: func(shader: shader);
  /// Read one compile-time parameter of the shader (gl:+compile-status+).
  get-shader-parameter: func(shader: shader, pname: s32) -> bool;
  /// The shader's compile log, empty when it compiled cleanly.
  get-shader-info-log: func(shader: shader) -> string;
  /// Create an empty program.
  create-program: func() -> program;
  /// Add a compiled shader to the program.
  attach-shader: func(program: program, shader: shader);
  /// Link the program's attached shaders.
  link-program: func(program: program);
  /// Read one link-time parameter of the program (gl:+link-status+).
  get-program-parameter: func(program: program, pname: s32) -> bool;
  /// The program's link log, empty when it linked cleanly.
  get-program-info-log: func(program: program) -> string;
  /// Make the program current for subsequent draw calls.
  use-program: func(program: program);
  /// Look one of the program's uniforms up by name.
  get-uniform-location: func(program: program, name: string) -> uniform-location;
  /// Set a float uniform.
  uniform1f: func(location: uniform-location, x: f32);
  /// Set a vec3 uniform.
  uniform3f: func(location: uniform-location, x: f32, y: f32, z: f32);
  /// Turn a capability on (gl:+depth-test+, gl:+blend+).
  enable: func(cap: s32);
  /// Turn a capability off.
  disable: func(cap: s32);
  /// Enable or disable writing to the depth buffer.
  depth-mask: func(flag: bool);
  /// Set the source and destination blend factors.
  blend-func: func(src: s32, dst: s32);
  /// Create an empty buffer.
  create-buffer: func() -> buffer;
  /// Bind a buffer to a target (gl:+array-buffer+).
  bind-buffer: func(target: s32, buffer: buffer);
  /// Size the bound buffer's store, without initialising it.
  buffer-data: func(target: s32, size: s32, usage: s32);
  /// Create an empty vertex array object.
  create-vertex-array: func() -> vertex-array;
  /// Bind a vertex array object.
  bind-vertex-array: func(array: vertex-array);
  /// Enable a vertex attribute slot.
  enable-vertex-attrib-array: func(index: s32);
  /// Describe how the bound buffer feeds a vertex attribute slot.
  vertex-attrib-pointer: func(index: s32, size: s32, kind: s32, normalized: bool, stride: s32, offset: s32);
  /// Set the viewport, in device pixels.
  viewport: func(x: s32, y: s32, width: s32, height: s32);
  /// Set the colour the next clear writes.
  clear-color: func(red: f32, green: f32, blue: f32, alpha: f32);
  /// Clear the named buffers (gl:+color-buffer-bit+, gl:+depth-buffer-bit+).
  clear: func(mask: s32);
  /// Draw the bound vertex array.
  draw-arrays: func(mode: s32, first: s32, count: s32);
}

/// The page's error reporting. Not part of WebGL: this is how gl.lisp's shader
/// helpers surface a compile or link failure, which is fatal to the demo.
interface ui {
  /// Show the message in the page's error box and stop the program. A page
  /// implements this by throwing, so the call never returns.
  fail: func(message: string);
}


---

# FILE: references/examples/browser/webgl-cube/README.md

# cube.lisp — hello 3D: a rotating cube, matrices and all, driven from Lisp

The middle step between [`webgl-triangle/`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-triangle) (the hello
world) and [`webgl-galaxy/`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-galaxy) (a full pipeline). This one adds
the parts every real 3D program needs — a vertex buffer, a depth test, and
4x4 matrix math — and keeps them all in Lisp: the perspective projection, the
rotation matrices and their products are computed in Lisp every frame.

**Live demo:**  (this
directory is published as a subpath of the GitHub Pages site by
`.github/workflows/pages.yaml`).

## What's in here

| File         | Purpose                                                          |
| ------------ | ---------------------------------------------------------------- |
| `cube.lisp`  | Everything: GLSL shaders, cube geometry, mat4 math, the frame.    |
| `index.html` | The host page: one-line WebGL2 bindings + the animation loop.     |
| `cube.wasm`  | The compiled `--no-wasi` reactor (checked in).                    |
| `build.sh`   | Recompiles `cube.lisp` to `cube.wasm`.                            |

The WebGL2 API boundary itself (the WIT interface, the enum constants and the
shader helpers) lives in the shared `gl` package,
[`../webgl-common/gl.lisp`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-common), spliced in at compile time by
`(require :gl "../webgl-common/gl.lisp")`. The page's matching WebGL2 bindings
are generated from the same `gl.wit`, imported from
`../webgl-common/gl-imports.js`.

## How it works

Beyond the triangle's ten imports, the cube needs a uniform, a buffer,
attributes and a depth test — still one line of JavaScript each. Bulk floats
are the interesting part: neither the 216 floats
of cube geometry nor the 16 of the model-view-projection matrix can cross the
boundary as one WASM value, so the page keeps a small staging `Float32Array`
and three non-WebGL imports move data through it:

```lisp
(rontolisp:wasm-import 'set-float :from "gl" :as "setFloat"
                       :params '(:int :float) :returns :void)
(rontolisp:wasm-import 'gl-buffer-data-floats :from "gl" :as "bufferDataFloats"
                       :params '(:int :int :int) :returns :void)
(rontolisp:wasm-import 'gl-uniform-matrix4fv :from "gl" :as "uniformMatrix4fv"
                       :params '(:int) :returns :void)
```

```js
setFloat: (i, v) => { staging[i] = v; },
bufferDataFloats: (target, count, usage) =>
  gl.bufferData(target, staging.subarray(0, count), usage),
uniformMatrix4fv: (loc) =>
  gl.uniformMatrix4fv(handles[loc], false, staging.subarray(0, 16)),
```

The Lisp side owns everything that means anything:

- **Geometry.** The cube is eight corner lists and six `(quad-indices color)`
  face lists; `push-face` walks them and emits 36 interleaved vertices
  (position + color) through `set-float`, uploaded once at setup.
- **Matrix math.** `mat4-mul`, `mat4-perspective`, `mat4-rotation-x/y` and
  `mat4-translation` operate on 16-element arrays in column-major order (the
  OpenGL convention). `tan` for the projection is `sin`/`cos`, computed in Lisp
  by the built-ins — `cube.wasm` imports no `math` module at all.
- **The frame.** Every tick, `frame` multiplies
  `projection * translation * rotation-y * rotation-x`, writes the result
  through `set-float`, points the `uMvp` uniform at it, clears color + depth
  and draws 36 vertices.

Setup runs at load time (the top-level `(setup-gl)` inside `_initialize()`),
so the page only instantiates the module, calls `_initialize()`, and drives
`frame(t)` from `requestAnimationFrame`.

## Building and running

```bash
# from the repo root, once:
./mvnw clean package

# recompile the .wasm after editing cube.lisp:
examples/browser/webgl-cube/build.sh

# serve and open (any static file server works). The page imports the generated
# ../webgl-common/gl-imports.js, so serve examples/browser, not this directory:
jwebserver -p 8000 --directory "$PWD/examples/browser"
open http://localhost:8000/webgl-cube/
```

The page needs a browser with WebAssembly GC support (Chrome 119+,
Firefox 120+, Safari 18.2+, Edge 119+).

## Notes

- The module is compiled with `--no-wasi`, so its *only* imports are the
  host functions the program reaches — the import object is the whole
  embedding API. With `--optimize` the shipped `cube.wasm` imports 26
  functions: 20 of the shared `gl` package's 29 WebGL2 entries, its
  `ui.fail`, the three staging entries above and two canvas metrics. The nine
  `gl` entries the cube never calls (`uniform3f`, the VAO pair, `viewport`, ...)
  are tree-shaken away.
- Shader compile/link errors are reported by the shared `gl:build-program`
  (`gl:get-shader-parameter` / `gl:get-shader-info-log`, whose info log crosses
  back as a `string` result, shown through the `fail` entry of `gl.wit`'s `ui`
  interface).
- On the interpreter and JVM backends the `rontolisp:wasm-import` directives
  define stubs that signal an error when called, and the shared `gl` package's
  WIT-imported entries dispatch through a provider nothing binds (signaling
  `rontolisp:wit-error`), so this program is WASM-only by nature (there is no
  host to draw with elsewhere).


---

# FILE: references/examples/browser/webgl-cube/build.sh

#!/usr/bin/env bash
# Recompile cube.lisp to a browser-loadable WebAssembly reactor.
# --no-wasi drops all WASI imports, so the module's only imports are the host
# functions cube.lisp declares with rontolisp:wasm-import ("gl", "canvas" and
# "math"); --optimize tree-shakes the runtime so only the reachable functions
# ship.
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling cube.lisp -> cube.wasm"
java -jar "$jar" "$here/cube.lisp" -o "$here/cube.wasm" --no-wasi --optimize

# The page imports the generated ../webgl-common/gl-imports.js, so the served
# root is examples/browser rather than this directory.
echo "done. Serve the examples/browser directory over http, e.g.:"
echo "  jwebserver -p 8000 --directory \"$(dirname "$here")\""
echo "then open http://localhost:8000/webgl-cube/"


---

# FILE: references/examples/browser/webgl-cube/cube.lisp

;;;; cube.lisp -- hello 3D: a rotating cube, matrices and all, driven from Lisp.
;;;;
;;;; The middle step between ../webgl-triangle/ (hello world) and
;;;; ../webgl-galaxy/ (a full pipeline). This one adds the parts every real 3D
;;;; program needs -- a vertex buffer, a depth test, and 4x4 matrix math -- and
;;;; keeps them all in Lisp: the perspective projection, the rotation matrices
;;;; and their products are computed here every frame and cross the boundary as
;;;; 16 floats into a small staging array the page keeps.
;;;;
;;;; Compiled ahead of time to a --no-wasi reactor (build.sh). The page
;;;; instantiates the module and calls _initialize() -- which uploads the cube
;;;; geometry -- then calls the exported `frame` once per animation tick.

;; --- the host boundary ------------------------------------------------------
;; The WebGL2 API itself -- the wasm-import directives, the enum constants and
;; the shader helpers -- lives in the shared gl package
;; (../webgl-common/gl.lisp), spliced in here at compile time; --optimize
;; drops the entries this demo never calls. Only the imports specific to this
;; page stay below. GL objects cross the boundary as :int handles into a table
;; the page keeps; the GLSL source crosses as :string.

(require :gl "../webgl-common/gl.lisp")

;; Bulk floats (vertex data, matrices) cannot cross the boundary one WASM value
;; at a time, so the page keeps one small Float32Array. set-float writes one
;; slot; gl-buffer-data-floats uploads the first COUNT floats as buffer data;
;; gl-uniform-matrix4fv hands the first 16 to a mat4 uniform. These three are
;; the only imports that are not literal WebGL2 API entries.
(rontolisp:wasm-import 'set-float
                       :from "gl"
                       :as "setFloat"
                       :params '(:int :float)
                       :returns :void)
(rontolisp:wasm-import 'gl-buffer-data-floats
                       :from "gl"
                       :as "bufferDataFloats"
                       :params '(:int :int :int)
                       :returns :void)
(rontolisp:wasm-import 'gl-uniform-matrix4fv
                       :from "gl"
                       :as "uniformMatrix4fv"
                       :params '(:int)
                       :returns :void)

;; Canvas metrics (the backing store is fixed, read once for the aspect ratio).
(rontolisp:wasm-import 'canvas-width
                       :from "canvas"
                       :as "width"
                       :params '()
                       :returns :float)
(rontolisp:wasm-import 'canvas-height
                       :from "canvas"
                       :as "height"
                       :params '()
                       :returns :float)

;; The WASM backend has no transcendental built-ins, so borrow the host's:
;; these two lines are literally Math.sin / Math.cos on the JavaScript side.
(rontolisp:wasm-import 'sin :from "math" :params '(:float) :returns :float)
(rontolisp:wasm-import 'cos :from "math" :params '(:float) :returns :float)

;; --- shaders ----------------------------------------------------------------

(defconstant +vertex-shader-source+
  "#version 300 es
layout(location=0) in vec3 aPos;
layout(location=1) in vec3 aColor;
uniform mat4 uMvp;   // model-view-projection, computed in Lisp every frame
out vec3 vColor;
void main() {
  gl_Position = uMvp * vec4(aPos, 1.0);
  vColor = aColor;
}")

(defconstant +fragment-shader-source+
  "#version 300 es
precision mediump float;
in vec3 vColor;
out vec4 color;
void main() {
  color = vec4(vColor, 1.0);
}")

;; --- 4x4 matrix math --------------------------------------------------------
;; Column-major (the OpenGL convention): element (row, col) lives at
;; index (+ row (* col 4)).

(defconstant +pi+ 3.141592653589793)

(defun mat4-zero () (make-array 16))

(defun mat4-mul (a b)
  (let ((out (mat4-zero)))
    (dotimes (col 4)
      (dotimes (row 4)
        (let ((sum 0.0))
          (dotimes (k 4)
            (setq sum
             (+ sum (* (aref a (+ row (* k 4))) (aref b (+ k (* col 4)))))))
          (setf (aref out (+ row (* col 4))) sum))))
    out))

(defun mat4-perspective (fovy aspect near far)
  (let* ((half (* 0.5 fovy))
         ;; tan = sin/cos, with sin and cos borrowed from JavaScript's Math
         (f (/ (cos half) (sin half)))
         (nf (/ 1.0 (- near far)))
         (m (mat4-zero)))
    (dotimes (i 16) (setf (aref m i) 0.0))
    (setf (aref m 0) (/ f aspect))
    (setf (aref m 5) f)
    (setf (aref m 10) (* (+ far near) nf))
    (setf (aref m 11) -1.0)
    (setf (aref m 14) (* 2.0 far near nf))
    m))

(defun mat4-identity ()
  (let ((m (mat4-zero)))
    (dotimes (i 16) (setf (aref m i) 0.0))
    (setf (aref m 0) 1.0)
    (setf (aref m 5) 1.0)
    (setf (aref m 10) 1.0)
    (setf (aref m 15) 1.0)
    m))

(defun mat4-translation (x y z)
  (let ((m (mat4-identity)))
    (setf (aref m 12) x)
    (setf (aref m 13) y)
    (setf (aref m 14) z)
    m))

(defun mat4-rotation-x (angle)
  (let ((m (mat4-identity)) (c (cos angle)) (s (sin angle)))
    (setf (aref m 5) c)
    (setf (aref m 6) s)
    (setf (aref m 9) (- 0.0 s))
    (setf (aref m 10) c)
    m))

(defun mat4-rotation-y (angle)
  (let ((m (mat4-identity)) (c (cos angle)) (s (sin angle)))
    (setf (aref m 0) c)
    (setf (aref m 2) (- 0.0 s))
    (setf (aref m 8) s)
    (setf (aref m 10) c)
    m))

;; --- the cube ---------------------------------------------------------------
;; Eight corners, six colored faces; each face quad becomes two triangles, so
;; the vertex buffer holds 36 vertices x (position + color) = 216 floats.

(defconstant +corners+
  '((-0.5 -0.5 -0.5) (0.5 -0.5 -0.5) (0.5 0.5 -0.5) (-0.5 0.5 -0.5)
    (-0.5 -0.5 0.5) (0.5 -0.5 0.5) (0.5 0.5 0.5) (-0.5 0.5 0.5)))

;; Each face: the corner indices of its quad (counter-clockwise seen from
;; outside) and its color.
(defconstant +faces+
  '(((4 5 6 7) (0.94 0.42 0.48)) ; front  (+z) rose
    ((1 0 3 2) (0.42 0.72 0.94)) ; back   (-z) sky
    ((5 1 2 6) (0.55 0.48 0.94)) ; right  (+x) violet
    ((0 4 7 3) (0.44 0.88 0.72)) ; left   (-x) mint
    ((7 6 2 3) (0.93 0.90 0.72)) ; top    (+y) cream
    ((0 1 5 4) (0.36 0.40 0.62)) ; bottom (-y) slate
    ))

(defvar *float-index* 0) ; write cursor into the staging array

(defun push-float (v)
  (set-float *float-index* v)
  (setq *float-index* (+ *float-index* 1)))

(defun push-vertex (corner color)
  (dolist (v corner) (push-float v))
  (dolist (v color) (push-float v)))

(defun push-face (face)
  (let* ((quad (car face))
         (color (car (cdr face)))
         (c0 (nth (nth 0 quad) +corners+))
         (c1 (nth (nth 1 quad) +corners+))
         (c2 (nth (nth 2 quad) +corners+))
         (c3 (nth (nth 3 quad) +corners+)))
    ;; the quad c0-c1-c2-c3 as two triangles
    (push-vertex c0 color)
    (push-vertex c1 color)
    (push-vertex c2 color)
    (push-vertex c0 color)
    (push-vertex c2 color)
    (push-vertex c3 color)))

;; --- setup ------------------------------------------------------------------

(defvar *u-mvp* 0)        ; uniform location handle for uMvp
(defvar *projection* nil) ; fixed: the canvas size does not change

(defun setup-gl ()
  (let ((program
         (gl:build-program +vertex-shader-source+ +fragment-shader-source+)))
    (gl:use-program program)
    (setq *u-mvp* (gl:get-uniform-location program "uMvp"))
    (gl:enable gl:+depth-test+)
    ;; fill the staging array with the 216 floats of cube geometry and upload
    (gl:bind-buffer gl:+array-buffer+ (gl:create-buffer))
    (setq *float-index* 0)
    (dolist (face +faces+) (push-face face))
    (gl-buffer-data-floats gl:+array-buffer+ *float-index* gl:+static-draw+)
    ;; interleaved layout: vec3 position + vec3 color = 24 bytes per vertex
    (gl:enable-vertex-attrib-array 0)
    (gl:vertex-attrib-pointer 0 3 gl:+float+ nil 24 0)
    (gl:enable-vertex-attrib-array 1)
    (gl:vertex-attrib-pointer 1 3 gl:+float+ nil 24 12)
    (setq *projection*
          (mat4-perspective (/ +pi+ 4.0) (/ (canvas-width) (canvas-height)) 0.1
                            100.0))))

;; --- the frame --------------------------------------------------------------

(defun frame (tm)
  (let* ((model (mat4-mul (mat4-rotation-y tm) (mat4-rotation-x (* tm 0.7))))
         (view (mat4-translation 0.0 0.0 -2.6))
         (mvp (mat4-mul *projection* (mat4-mul view model))))
    (dotimes (i 16) (set-float i (aref mvp i)))
    (gl-uniform-matrix4fv *u-mvp*)
    (gl:clear-color 0.05 0.06 0.1 1.0)
    (gl:clear (+ gl:+color-buffer-bit+ gl:+depth-buffer-bit+))
    (gl:draw-arrays gl:+triangles+ 0 36)))

;; Build the pipeline and upload the geometry at load time: this runs inside
;; _initialize, after the page has created the WebGL2 context.
(setup-gl)

(rontolisp:wasm-export 'frame :params '(:float) :returns :void)


---

# FILE: references/examples/browser/webgl-cube/index.html



  
    
    
    cube.lisp — hello 3D: a rotating cube driven from Lisp
    
  
  
    

      
cube.lisp → WebGL

      

        Hello 3D: this cube is drawn by a Lisp program compiled to WebAssembly.
        The perspective projection and the rotation matrices are computed
        in Lisp every frame (4×4 matrix multiplication and all) and cross
        into WebGL across a boundary declared once in WIT — from which
        both the Lisp bindings and this page's one-line JavaScript ones are
        generated.
      

      
      

        source ·
        first step: webgl-triangle ·
        next step: webgl-galaxy
      

      
    


    
  



---

# FILE: references/examples/browser/webgl-galaxy/README.md

# galaxy.lisp — a spiral galaxy: the WebGL pipeline driven from Lisp

The `rontolisp:wasm-import` showcase: a Lisp program that *calls into the
browser*. Not just the physics — the whole WebGL2 pipeline runs from Lisp
compiled to WebAssembly. The GLSL shader sources live in the Lisp file as string
constants; Lisp compiles and links them, sets up the vertex buffer, attributes
and blending, and issues every clear and draw call. JavaScript supplies the
WebGL2 API as one-line bindings generated from the shared `gl.wit`, plus the page
UI — no rendering logic of its own.

**Live demo:** 

## What's in here

| File          | Purpose                                                            |
| ------------- | ------------------------------------------------------------------ |
| `galaxy.lisp` | Everything: GLSL shaders, pipeline setup, orbits, per-star drawing. |
| `index.html`  | The host page: one-line WebGL2 bindings + the HUD.                 |
| `galaxy.wasm` | The compiled `--no-wasi` reactor (checked in).                      |
| `build.sh`    | Recompiles `galaxy.lisp` to `galaxy.wasm`.                         |

The WebGL2 API boundary itself (the WIT interface, the enum constants and the
shader helpers) lives in the shared `gl` package,
[`../webgl-common/gl.lisp`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-common), spliced in at compile time by
`(require :gl "../webgl-common/gl.lisp")`.

## How the boundary works

`galaxy.lisp` declares 7 host functions of its own — the staging pair, the
canvas metrics, and two `math` entries that no longer reach the module (see the
notes):

```lisp
(rontolisp:wasm-import 'set-vertex :from "gl" :as "setVertex"
                       :params '(:int :float :float :float :float) :returns :void)
(rontolisp:wasm-import 'canvas-width :from "canvas" :as "width"
                       :params '() :returns :float)
;; ... bufferSubData, canvas height, devicePixelRatio, sin, cos
```

The literal WebGL2 entries are not declared here at all: they come from the
shared `gl` package, which binds them (and the `fail` error reporter) from
`../webgl-common/gl.wit` with two `rontolisp:wit-import` directives. The page's
matching import object is generated from that same file, so the JavaScript side
is a spread plus this demo's own staging one-liners over a handle table:

```js
import { glImports, uiImports } from "../webgl-common/gl-imports.js";

const imports = {
  gl: {
    ...glImports({ gl: gl2, handles, addHandle, str, retStr }),
    bufferSubData: (target, off, count) => gl2.bufferSubData(target, off, staging, 0, count),
    setVertex: (i, x, y, hue, size) => { /* fill the staging array */ },
  },
  ui: uiImports({ ui, str }),
  canvas: { width: () => canvas.width, /* ... */ },
};
```

Every type crosses the boundary somewhere in this example — spelled with the
`:int`/`:float`/`:bool`/`:string` designators in this demo's own directives, and
as `s32`/`f32`/`bool`/`string` in `gl.wit` for the shared entries:

- `:int` — GL enums, sizes, and the handles for shaders/programs/buffers.
- `:float` — star positions, uniforms, canvas metrics.
- `:bool` — `vertexAttribPointer`'s `normalized` flag, compile/link status.
- `:string` *parameter* — the GLSL source and uniform names travel from Lisp
  to `shaderSource`/`getUniformLocation` as `(ptr,len)` into the module's
  exported linear memory.
- `:string` *result* — on a shader compile error, `getShaderInfoLog` writes
  the log back through the exported `__ronto_alloc` allocator and the shared
  `gl:make-shader` hands it to the imported `fail` to show the page's error
  box.

Startup order is simple: the page creates the WebGL2 context, instantiates the
module, and calls `_initialize()`, which runs the top-level `(setup-gl)` — so the
shaders compile and the pipeline is configured *from Lisp* before the first
frame. Each `requestAnimationFrame` tick then calls `exports.frame(t)`: Lisp
sets the viewport, clears, computes every star's position on its slowly
precessing ellipse, stages it, uploads and draws. At 16,000 stars and 60 fps
that is several million Lisp-to-JavaScript calls per second — the counter in the
corner keeps score.

The two staging imports (`setVertex`, `bufferSubData`) are the only ones that
are not literal WebGL2 entries: per-star floats cannot cross into GPU memory
one call at a time, so the page keeps a single `Float32Array` that Lisp fills
and uploads. That array and the handle table are the host's entire state.

There is no randomness: a `--no-wasi` reactor has no entropy source, so the
stars are scattered with low-discrepancy sequences (the golden angle for orbit
phase, `sqrt 2 - 1` for radius). One subtlety worth stealing: the radius
sequence must *not* be built on the golden ratio, because the golden angle is
`2*pi*(1-phi)` — the two sequences would be exactly correlated and every star
would land on a single curve.

## Building and running

```bash
# from the repo root, once:
./mvnw clean package

# recompile the .wasm after editing galaxy.lisp:
examples/browser/webgl-galaxy/build.sh

# serve and open (any static file server works). The page imports the generated
# ../webgl-common/gl-imports.js, so serve examples/browser, not this directory:
jwebserver -p 8000 --directory "$PWD/examples/browser"
open http://localhost:8000/webgl-galaxy/
```

The page needs a browser with WebAssembly GC support (Chrome 119+,
Firefox 120+, Safari 18.2+, Edge 119+).

## Notes

- The module is compiled with `--no-wasi`, so its *only* imports are host
  functions — the import object is the whole embedding API. The shipped
  `galaxy.wasm` imports 32 of them: 26 of the shared `gl` package's 29 WebGL2
  entries, its `ui.fail`, and 5 of the 7 `galaxy.lisp` declares. (`math.sin` and
  `math.cos` are not among them — the WASM backend compiles `sin`/`cos`
  natively, so those two declarations are unreferenced and `--optimize` drops
  them.)
- `--optimize` tree-shakes the runtime down to a few KB, of which the GLSL
  sources are about a third.
- On the interpreter and JVM backends the `rontolisp:wasm-import` directives
  define stubs that signal an error when called, and the shared `gl` package's
  WIT-imported entries dispatch through a provider nothing binds (signaling
  `rontolisp:wit-error`), so this program is WASM-only by nature (there is no
  host to draw with elsewhere).


---

# FILE: references/examples/browser/webgl-galaxy/build.sh

#!/usr/bin/env bash
# Recompile galaxy.lisp to a browser-loadable WebAssembly reactor.
# --no-wasi drops all WASI imports, so the module's only imports are the host
# functions galaxy.lisp declares with rontolisp:wasm-import ("gl" and "math");
# --optimize tree-shakes the runtime so only the reachable functions ship.
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling galaxy.lisp -> galaxy.wasm"
java -jar "$jar" "$here/galaxy.lisp" -o "$here/galaxy.wasm" --no-wasi --optimize

# The page imports the generated ../webgl-common/gl-imports.js, so the served
# root is examples/browser rather than this directory.
echo "done. Serve the examples/browser directory over http, e.g.:"
echo "  jwebserver -p 8000 --directory \"$(dirname "$here")\""
echo "then open http://localhost:8000/webgl-galaxy/"


---

# FILE: references/examples/browser/webgl-galaxy/galaxy.lisp

;;;; galaxy.lisp -- a spiral galaxy: the simulation AND the WebGL pipeline,
;;;; all driven from Lisp.
;;;;
;;;; This is the rontolisp:wasm-import showcase. The page (index.html) does not
;;;; know how to render a galaxy: it only exposes the WebGL2 API one function at
;;;; a time (a handle table resolves :int handles to GL objects, every binding
;;;; is one line of JavaScript). Lisp compiles the shaders -- the GLSL source
;;;; lives in this file as string constants -- links the program, sets up the
;;;; vertex buffer and blending, and issues every draw call. JavaScript keeps
;;;; only the UI (canvas sizing, the star-count select, the HUD meters) and the
;;;; final vertex upload.
;;;;
;;;; Compiled ahead of time to a --no-wasi reactor (build.sh), so the module
;;;; imports nothing but the host functions declared below and instantiates in
;;;; any wasm-GC-capable browser.
;;;;
;;;; The galaxy is the classic density-wave toy model: every star follows a
;;;; fixed ellipse, and the ellipses' orientations twist with radius, so the
;;;; crowded parts of neighbouring orbits line up into spiral arms. Nothing is
;;;; random: star i is scattered with the golden angle, so the disc looks
;;;; even without an entropy source (a --no-wasi reactor has none).

;; --- the host boundary ------------------------------------------------------
;;
;; The WebGL2 API itself -- the wasm-import directives, the enum constants and
;; the shader helpers -- lives in the shared gl package
;; (../webgl-common/gl.lisp), spliced in here at compile time; --optimize
;; drops the entries this demo never calls. Only the imports specific to this
;; page stay below. GL objects (shaders, programs, buffers, uniform locations)
;; cross the boundary as :int handles into a table the page keeps; strings
;; (GLSL source, uniform names, info logs) cross as :string.

(require :gl "../webgl-common/gl.lisp")

;; The vertex staging path: floats cannot be written into GPU memory across the
;; boundary one call at a time, so the page keeps one Float32Array. set-vertex
;; writes one star's (x y hue size) record into it; gl-buffer-sub-data uploads
;; the first COUNT floats of it to the bound buffer. These two are the only
;; imports that are not literal WebGL2 API entries.
(rontolisp:wasm-import 'set-vertex
                       :from "gl"
                       :as "setVertex"
                       :params '(:int :float :float :float :float)
                       :returns :void)
(rontolisp:wasm-import 'gl-buffer-sub-data
                       :from "gl"
                       :as "bufferSubData"
                       :params '(:int :int :int)
                       :returns :void)

;; Canvas metrics, owned by the page (it resizes the backing store on window
;; resize; Lisp reads the result every frame and sets the viewport itself).
(rontolisp:wasm-import 'canvas-width
                       :from "canvas"
                       :as "width"
                       :params '()
                       :returns :float)
(rontolisp:wasm-import 'canvas-height
                       :from "canvas"
                       :as "height"
                       :params '()
                       :returns :float)
(rontolisp:wasm-import 'device-pixel-ratio
                       :from "canvas"
                       :as "devicePixelRatio"
                       :params '()
                       :returns :float)

;; The WASM backend has no transcendental built-ins, so borrow the host's:
;; these two lines are literally Math.sin / Math.cos on the JavaScript side.
(rontolisp:wasm-import 'sin :from "math" :params '(:float) :returns :float)
(rontolisp:wasm-import 'cos :from "math" :params '(:float) :returns :float)

;; --- shaders ----------------------------------------------------------------
;; The GLSL lives here, in Lisp, and reaches the GPU through the imported
;; gl:shader-source (a :string parameter crossing the boundary as (ptr,len)
;; into this module's linear memory).

(defconstant +vertex-shader-source+
  "#version 300 es
layout(location=0) in vec2 aPos;    // clip-space position from Lisp
layout(location=1) in float aHue;   // 0..1 hue from Lisp
layout(location=2) in float aSize;  // point size from Lisp
uniform float uDpr;
out float vHue;
void main() {
  gl_Position = vec4(aPos, 0.0, 1.0);
  gl_PointSize = aSize * uDpr;
  vHue = aHue;
}")

(defconstant +fragment-shader-source+
  "#version 300 es
precision mediump float;
in float vHue;
out vec4 color;
// hue = 0 at the core, 1 at the rim: warm white -> ice blue -> violet,
// the classic stellar-population gradient.
vec3 tint(float h) {
  vec3 core = vec3(1.00, 0.93, 0.78);
  vec3 mid  = vec3(0.62, 0.78, 1.00);
  vec3 rim  = vec3(0.66, 0.47, 1.00);
  return h < 0.5 ? mix(core, mid, h * 2.0) : mix(mid, rim, h * 2.0 - 1.0);
}
void main() {
  // a soft round sprite: bright center, glow falloff, additive blending
  float d = length(gl_PointCoord - 0.5) * 2.0;
  float a = exp(-3.2 * d * d) * (1.0 - smoothstep(0.8, 1.0, d));
  color = vec4(tint(vHue) * a, a);
}")

;; --- GL pipeline setup ------------------------------------------------------

(defvar *u-dpr* 0) ; uniform location handle for uDpr

(defun setup-gl ()
  (let ((program
         (gl:build-program +vertex-shader-source+ +fragment-shader-source+)))
    (gl:use-program program)
    (setq *u-dpr* (gl:get-uniform-location program "uDpr"))
    ;; additive blending: overlapping stars glow
    (gl:enable gl:+blend+)
    (gl:blend-func gl:+one+ gl:+one+)
    ;; one interleaved vertex buffer: x, y, hue, size = 16 bytes per star
    (gl:bind-vertex-array (gl:create-vertex-array))
    (gl:bind-buffer gl:+array-buffer+ (gl:create-buffer))
    (gl:enable-vertex-attrib-array 0)
    (gl:vertex-attrib-pointer 0 2 gl:+float+ nil 16 0)
    (gl:enable-vertex-attrib-array 1)
    (gl:vertex-attrib-pointer 1 1 gl:+float+ nil 16 8)
    (gl:enable-vertex-attrib-array 2)
    (gl:vertex-attrib-pointer 2 1 gl:+float+ nil 16 12)))

;; --- the galaxy -------------------------------------------------------------

;; Per-star orbit parameters, filled by `init`.
(defvar *n* 0)
(defvar *radius* nil) ; semi-major axis of the orbit
(defvar *phase* nil)  ; where on the orbit the star starts
(defvar *speed* nil)  ; angular speed (inner orbits run faster)
(defvar *tilt* nil)   ; orientation of the ellipse

;; The golden angle scatters the stars' orbit phases evenly; sqrt(2)-1
;; scatters their radii. (The radius sequence must not be built on the golden
;; ratio: the golden angle is 2*pi*(1-phi), so frac(i*phi) would be an exact
;; function of the phase and every star would land on one curve.) The two
;; low-discrepancy sequences are independent, so the disc fills smoothly
;; without an entropy source.
(defconstant +golden-angle+ 2.399963229728653)

(defconstant +radius-step+ 0.414213562373095)

;; How strongly the ellipse orientation twists with radius; this is what
;; winds the orbits into spiral arms.
(defconstant +twist+ 5.4)

;; The fractional part of X (X non-negative).
(defun frac (x) (- x (floor x)))

(defun init (n)
  (setq *n* n)
  (setq *radius* (make-array n))
  (setq *phase* (make-array n))
  (setq *speed* (make-array n))
  (setq *tilt* (make-array n))
  ;; size the GPU buffer (and the page's staging array) for n stars
  (gl:buffer-data gl:+array-buffer+ (* n 16) gl:+dynamic-draw+)
  (dotimes (i n)
    (let* ((u (frac (* (+ i 1) +radius-step+)))
           ;; sqrt biases the stars toward the bright core
           (r (+ 0.03 (* 0.95 (sqrt u))))
           ;; a third sequence breaks up the residual lattice at the rim
           (jitter (* 0.5 (frac (* (+ i 1) 0.754877666)))))
      (setf (aref *radius* i) r)
      (setf (aref *phase* i) (+ (* i +golden-angle+) jitter))
      (setf (aref *speed* i) (/ 0.5 (+ 0.15 r)))
      (setf (aref *tilt* i) (* r +twist+)))))

(defun frame (tm)
  (let* ((w (canvas-width)) (h (canvas-height)) (aspect (/ w h)))
    (gl:viewport 0 0 (floor w) (floor h))
    (gl:uniform1f *u-dpr* (device-pixel-ratio))
    (gl:clear-color 0.012 0.016 0.045 1.0)
    (gl:clear gl:+color-buffer-bit+)
    (dotimes (i *n*)
      (let* ((r (aref *radius* i))
             (theta (+ (aref *phase* i) (* tm (aref *speed* i))))
             ;; the star's position on its (axis-aligned) ellipse
             (ex (* r (cos theta)))
             (ey (* r 0.55 (sin theta)))
             ;; rotate the ellipse: per-radius twist plus a slow global spin
             (rot (+ (aref *tilt* i) (* tm 0.04)))
             (c (cos rot))
             (s (sin rot))
             (x (- (* ex c) (* ey s)))
             (y (+ (* ex s) (* ey c)))
             ;; the fragment shader maps 0 -> warm core white, 1 -> violet rim
             (hue (+ r (* 0.1 (frac (* i 0.754877666)))))
             (size (+ 1.8 (* 3.4 (- 1.0 r) (- 1.0 r)))))
        (set-vertex i (/ x aspect) y hue size)))
    (gl-buffer-sub-data gl:+array-buffer+ 0 (* *n* 4))
    (gl:draw-arrays gl:+points+ 0 *n*)))

;; Build the pipeline at load time: this runs inside _initialize, after the
;; page has created the WebGL2 context and instantiated the module.
(setup-gl)

(rontolisp:wasm-export 'init :params '(:int) :returns :void)
(rontolisp:wasm-export 'frame :params '(:float) :returns :void)


---

# FILE: references/examples/browser/webgl-galaxy/index.html



  
    
    
    galaxy.lisp — a spiral galaxy computed in Lisp, drawn by WebGL
    
    
    
    
  
  
    

    

      
galaxy.lisp → WebGL

      

        The whole WebGL pipeline runs in Lisp, compiled to WebAssembly:
        the GLSL shaders live in the Lisp source, and Lisp compiles, links, buffers
        and issues every draw call across a boundary declared once in WIT.
        JavaScript is just one-line bindings — generated from that WIT — and this HUD.
      

    


    

    

    

    
  



---

# FILE: references/examples/browser/webgl-heat3d/README.md

# heat3d.lisp — a rank-3 array diffusing heat, drawn by WebGL

This example is the rank-3 array showcase in the browser: the whole state of
the page is **one rank-3 `(n n n)` `make-array`**. Every frame, Lisp deposits
heat at two orbiting sources with three-subscript `(setf (aref grid i j k))`,
runs one explicit diffusion step over the lattice (insulated walls, mild
global cooling), normalizes the colors with the rank-generic
`(linalg:amax grid)`, reports `(linalg:sum grid)` to the HUD, and projects
every voxel to a screen-space point itself. JavaScript is the same host
boundary as [`webgl-galaxy`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-galaxy): one-line WebGL2 bindings over a
handle table, plus the HUD — no simulation or rendering logic of its own.

It is the browser companion of [`examples/ml/heat3d.lisp`](../../ml/heat3d.lisp), the
console version whose exact rational arithmetic conserves the total heat as
*exactly* 1000. Here the voxels hold floats instead: the simulation runs
forever, and exact ratio denominators would grow without bound (and overflow
the WASM backend's i31 fixnums within a few steps).

**Live demo:**  (this
directory is published as a subpath of the GitHub Pages site by
`.github/workflows/pages.yaml`).

## What's in here

| File          | Purpose                                                                 |
| ------------- | ----------------------------------------------------------------------- |
| `heat3d.lisp` | Everything: the rank-3 simulation, GLSL shaders, projection, draw calls. |
| `index.html`  | The host page: one-line WebGL2 bindings + the HUD.                       |
| `heat3d.wasm` | The compiled `--no-wasi` reactor (checked in).                           |
| `build.sh`    | Recompiles `heat3d.lisp` to `heat3d.wasm`.                               |

The WebGL2 API boundary itself (the WIT interface, the enum constants and the
shader helpers) lives in the shared `gl` package,
[`../webgl-common/gl.lisp`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-common), spliced in at compile time by
`(require :gl "../webgl-common/gl.lisp")`. The page's matching WebGL2 bindings
are generated from the same `gl.wit`, imported from
`../webgl-common/gl-imports.js`.

## The rank-3 array is the program

Everything interesting happens against one array (and its double buffer):

```lisp
(setq *grid* (make-array (list n n n) :initial-element 0.0))

;; three-subscript reads and writes, all six lattice neighbours:
(setf (aref out i j k) (* +cool+ (+ c (* +alpha+ acc))))

;; rank-generic linalg reductions over the whole rank-3 grid, every frame:
(linalg:amax *grid*)   ; normalizes the color scale
(linalg:sum *grid*)    ; the HUD's "total heat" meter
```

The flat row-major data order is also exactly the order the voxels are
written into the vertex buffer, so vertex `v` is the voxel at
`(array-row-major-index grid i j k) = v`.

The rest is the same shape as `webgl-galaxy`: the GLSL shader sources live in
the Lisp file, Lisp compiles and links them at `_initialize` time, and each
`requestAnimationFrame` tick calls `exports.frame(t)` — Lisp injects,
diffuses, rotates and perspective-projects every voxel, stages it with
`set-vertex`, uploads with `gl-buffer-sub-data` and draws with
`gl:draw-arrays` (additive blending, so no depth sorting). The HUD's "total
heat" meter polls the exported `totalHeat` (a `rontolisp:wasm-export ... :as`
alias for `total-heat`). At the 20×20×20 setting that is 8,000 voxels
simulated and projected in Lisp per frame.

## Building and running

```bash
# from the repo root, once:
./mvnw clean package

# recompile the .wasm after editing heat3d.lisp:
examples/browser/webgl-heat3d/build.sh

# serve and open (any static file server works). The page imports the generated
# ../webgl-common/gl-imports.js, so serve examples/browser, not this directory:
jwebserver -p 8000 --directory "$PWD/examples/browser"
open http://localhost:8000/webgl-heat3d/
```

The page needs a browser with WebAssembly GC support (Chrome 119+,
Firefox 120+, Safari 18.2+, Edge 119+).

## Notes

- The module is compiled with `--no-wasi`, so its *only* imports are the host
  functions declared in `heat3d.lisp` and the shared `gl` package — the
  import object is the whole embedding API. `--optimize` tree-shakes the
  runtime, including the unused parts of the spliced `linalg` library and most
  `gl` entries this demo never calls; the shipped `heat3d.wasm` imports 36
  functions. Two of them are entries `heat3d.lisp` never calls: because this
  program takes functions as values (through the spliced `linalg` library), the
  `funcall` dispatcher keeps the same-arity import wrappers reachable, so
  `disable` and `depthMask` survive the shake. The page provides them either
  way — it spreads the whole generated `glImports` union.
- The `.wasm` is several times the galaxy's, because the array runtime and the
  reachable `linalg` definitions ship with it.
- On the interpreter and JVM backends the `rontolisp:wasm-import` directives
  define stubs that signal an error when called, and the shared `gl` package's
  WIT-imported entries dispatch through a provider nothing binds (signaling
  `rontolisp:wit-error`), so this program is WASM-only by nature (there is no
  host to draw with elsewhere) — run
  [`examples/ml/heat3d.lisp`](../../ml/heat3d.lisp) for the cross-backend console
  version.


---

# FILE: references/examples/browser/webgl-heat3d/build.sh

#!/usr/bin/env bash
# Recompile heat3d.lisp to a browser-loadable WebAssembly reactor.
# --no-wasi drops all WASI imports, so the module's only imports are the host
# functions heat3d.lisp declares with rontolisp:wasm-import ("gl", "canvas",
# "math" and "ui"); --optimize tree-shakes the runtime so only the reachable
# functions (including the spliced linalg definitions it uses) ship.
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling heat3d.lisp -> heat3d.wasm"
java -jar "$jar" "$here/heat3d.lisp" -o "$here/heat3d.wasm" --no-wasi --optimize

# The page imports the generated ../webgl-common/gl-imports.js, so the served
# root is examples/browser rather than this directory.
echo "done. Serve the examples/browser directory over http, e.g.:"
echo "  jwebserver -p 8000 --directory \"$(dirname "$here")\""
echo "then open http://localhost:8000/webgl-heat3d/"


---

# FILE: references/examples/browser/webgl-heat3d/heat3d.lisp

;;;; heat3d.lisp -- heat diffusion in a rank-3 voxel grid, simulated in Lisp
;;;; and rendered as a glowing WebGL point cloud.
;;;;
;;;; This is the browser companion of examples/ml/heat3d.lisp and the rank-3
;;;; array showcase: the whole simulation state is ONE rank-3 (n n n)
;;;; make-array. Every frame Lisp injects heat at two orbiting sources
;;;; ((setf (aref grid i j k)) with three subscripts), runs one explicit
;;;; diffusion step over the lattice, normalizes the colors with the
;;;; rank-generic (linalg:amax grid), reports (linalg:sum grid) to the HUD,
;;;; and projects every voxel to a screen-space point itself -- the page only
;;;; exposes the WebGL2 API one line at a time (see webgl-galaxy, whose host
;;;; boundary this reuses verbatim).
;;;;
;;;; Unlike the exact-rational console example, the voxels hold floats: the
;;;; simulation runs forever, and exact ratio denominators would grow without
;;;; bound (and overflow the WASM backend's i31 fixnums within a few steps).
;;;;
;;;; Compiled ahead of time to a --no-wasi reactor (build.sh), so the module
;;;; imports nothing but the host functions declared below and instantiates
;;;; in any wasm-GC-capable browser.

;; --- the host boundary ------------------------------------------------------
;;
;; The WebGL2 API itself -- the wasm-import directives, the enum constants and
;; the shader helpers -- lives in the shared gl package
;; (../webgl-common/gl.lisp), spliced in here at compile time; --optimize
;; drops the entries this demo never calls. Only the imports specific to this
;; page stay below. GL objects cross as :int handles into a table the page
;; keeps; strings (GLSL source, info logs) cross as :string.

(require :gl "../webgl-common/gl.lisp")

;; The vertex staging path (see webgl-galaxy): per-voxel floats cannot cross
;; into GPU memory one call at a time, so the page keeps one Float32Array
;; that set-vertex fills and gl-buffer-sub-data uploads.
(rontolisp:wasm-import 'set-vertex
                       :from "gl"
                       :as "setVertex"
                       :params '(:int :float :float :float :float)
                       :returns :void)
(rontolisp:wasm-import 'gl-buffer-sub-data
                       :from "gl"
                       :as "bufferSubData"
                       :params '(:int :int :int)
                       :returns :void)

;; Canvas metrics, owned by the page.
(rontolisp:wasm-import 'canvas-width
                       :from "canvas"
                       :as "width"
                       :params '()
                       :returns :float)
(rontolisp:wasm-import 'canvas-height
                       :from "canvas"
                       :as "height"
                       :params '()
                       :returns :float)
(rontolisp:wasm-import 'device-pixel-ratio
                       :from "canvas"
                       :as "devicePixelRatio"
                       :params '()
                       :returns :float)

;; The WASM backend has no transcendental built-ins, so borrow the host's.
(rontolisp:wasm-import 'sin :from "math" :params '(:float) :returns :float)
(rontolisp:wasm-import 'cos :from "math" :params '(:float) :returns :float)

;; --- shaders ----------------------------------------------------------------

(defconstant +vertex-shader-source+
  "#version 300 es
layout(location=0) in vec2 aPos;    // clip-space position from Lisp
layout(location=1) in float aHeat;  // 0..1 normalized heat from Lisp
layout(location=2) in float aSize;  // point size from Lisp
uniform float uDpr;
out float vHeat;
void main() {
  gl_Position = vec4(aPos, 0.0, 1.0);
  gl_PointSize = aSize * uDpr;
  vHeat = aHeat;
}")

(defconstant +fragment-shader-source+
  "#version 300 es
precision mediump float;
in float vHeat;
out vec4 color;
// heat 0 -> faint lattice blue, 0.5 -> ember orange, 1 -> white heat.
vec3 tint(float h) {
  vec3 cold = vec3(0.10, 0.17, 0.48);
  vec3 warm = vec3(1.00, 0.45, 0.12);
  vec3 hot  = vec3(1.00, 0.97, 0.88);
  return h < 0.5 ? mix(cold, warm, h * 2.0) : mix(warm, hot, h * 2.0 - 1.0);
}
void main() {
  // a soft round sprite; cold voxels keep a faint glow so the lattice shows
  float d = length(gl_PointCoord - 0.5) * 2.0;
  float a = exp(-3.0 * d * d) * (1.0 - smoothstep(0.8, 1.0, d));
  float glow = 0.08 + 0.92 * vHeat;
  color = vec4(tint(vHeat) * a * glow, a * glow);
}")

;; --- GL pipeline setup ------------------------------------------------------

(defvar *u-dpr* 0) ; uniform location handle for uDpr

(defun setup-gl ()
  (let ((program
         (gl:build-program +vertex-shader-source+ +fragment-shader-source+)))
    (gl:use-program program)
    (setq *u-dpr* (gl:get-uniform-location program "uDpr"))
    ;; additive blending: overlapping voxels glow, no depth sorting needed
    (gl:enable gl:+blend+)
    (gl:blend-func gl:+one+ gl:+one+)
    ;; one interleaved vertex buffer: x, y, heat, size = 16 bytes per voxel
    (gl:bind-vertex-array (gl:create-vertex-array))
    (gl:bind-buffer gl:+array-buffer+ (gl:create-buffer))
    (gl:enable-vertex-attrib-array 0)
    (gl:vertex-attrib-pointer 0 2 gl:+float+ nil 16 0)
    (gl:enable-vertex-attrib-array 1)
    (gl:vertex-attrib-pointer 1 1 gl:+float+ nil 16 8)
    (gl:enable-vertex-attrib-array 2)
    (gl:vertex-attrib-pointer 2 1 gl:+float+ nil 16 12)))

;; --- the simulation ---------------------------------------------------------
;;
;; The whole state is one rank-3 array (plus its double buffer). Element
;; (i j k) is the heat of the voxel at lattice position (i j k); the flat
;; row-major data order is also exactly the order the voxels are written
;; into the vertex buffer, so vertex v is the voxel at
;; (array-row-major-index grid i j k) = v.

(defvar *n* 0)      ; lattice side
(defvar *grid* nil) ; rank-3 (n n n) array of floats
(defvar *next* nil) ; the double buffer

(defconstant +alpha+ 0.16) ; diffusion rate (stable: alpha <= 1/6)
(defconstant +cool+ 0.988) ; per-step global cooling

(defun init (n)
  (setq *n* n)
  (setq *grid* (make-array (list n n n) :initial-element 0.0))
  (setq *next* (make-array (list n n n) :initial-element 0.0))
  ;; size the GPU buffer (and the page's staging array) for n^3 voxels
  (gl:buffer-data gl:+array-buffer+ (* n n n 16) gl:+dynamic-draw+))

(defun add-heat (fi fj fk amount)
  ;; Deposits heat at the voxel containing the (float) lattice point.
  (let ((i (floor fi)) (j (floor fj)) (k (floor fk)))
    (setf (aref *grid* i j k) (+ (aref *grid* i j k) amount))))

(defun inject (tm)
  ;; Two counter-rotating heat sources orbit inside the cube. Their orbit
  ;; radius keeps the (floored) indices strictly inside the lattice, so no
  ;; bounds clamping is needed.
  (let* ((mid (* 0.5 (- *n* 1))) (r (- mid 1.5)))
    (add-heat (+ mid (* r (cos (* tm 1.1)))) (+ mid (* 0.6 r (sin (* tm 0.7))))
              (+ mid (* r (sin (* tm 1.1)))) 900.0)
    (add-heat (+ mid (* 0.7 r (cos (* tm -0.6)))) (+ mid (* r (sin (* tm 0.5))))
              (+ mid (* 0.7 r (sin (* tm -0.6)))) 600.0)))

(defun diffuse ()
  ;; One explicit Euler step with insulated boundaries into the double
  ;; buffer, exactly as in examples/ml/heat3d.lisp -- three-subscript aref all
  ;; the way -- plus a mild global cooling so the sources and the walls reach
  ;; a moving equilibrium.
  (let ((n *n*) (g *grid*) (out *next*))
    (dotimes (i n)
      (dotimes (j n)
        (dotimes (k n)
          (let ((c (aref g i j k)) (acc 0.0))
            (when (> i 0) (setq acc (+ acc (- (aref g (- i 1) j k) c))))
            (when (< i (- n 1)) (setq acc (+ acc (- (aref g (+ i 1) j k) c))))
            (when (> j 0) (setq acc (+ acc (- (aref g i (- j 1) k) c))))
            (when (< j (- n 1)) (setq acc (+ acc (- (aref g i (+ j 1) k) c))))
            (when (> k 0) (setq acc (+ acc (- (aref g i j (- k 1)) c))))
            (when (< k (- n 1)) (setq acc (+ acc (- (aref g i j (+ k 1)) c))))
            (setf (aref out i j k) (* +cool+ (+ c (* +alpha+ acc))))))))
    (setq *grid* out)
    (setq *next* g)))

;; The HUD polls this: the rank-generic linalg reduction over the rank-3 grid.
(defun total-heat () (linalg:sum *grid*))

;; --- rendering --------------------------------------------------------------

(defun frame (tm)
  (inject tm)
  (diffuse)
  (let* ((w (canvas-width))
         (h (canvas-height))
         (aspect (/ w h))
         (n *n*)
         (mid (* 0.5 (- n 1)))
         (scale (/ 1.6 n)) ; lattice -> model units (cube ~[-0.8, 0.8]^3)
         ;; a slow spin around Y plus a gently wobbling tilt around X
         (ry (* tm 0.4))
         (cy (cos ry))
         (sy (sin ry))
         (rx (+ 0.45 (* 0.15 (sin (* tm 0.31)))))
         (cx (cos rx))
         (sx (sin rx))
         ;; colors are normalized by the hottest voxel right now (the
         ;; rank-generic linalg:amax over the whole rank-3 grid); the sqrt
         ;; below tone-maps the ratio so mid heats stay visible next to the
         ;; freshly injected source voxels
         (top (linalg:amax *grid*))
         (norm (if (> top 0.0) (/ 1.0 top) 0.0))
         (v 0))
    (gl:viewport 0 0 (floor w) (floor h))
    (gl:uniform1f *u-dpr* (device-pixel-ratio))
    (gl:clear-color 0.012 0.016 0.045 1.0)
    (gl:clear gl:+color-buffer-bit+)
    (dotimes (i n)
      (dotimes (j n)
        (dotimes (k n)
          (let* ((heat (sqrt (* norm (aref *grid* i j k))))
                 (x0 (* scale (- i mid)))
                 (y0 (* scale (- j mid)))
                 (z0 (* scale (- k mid)))
                 ;; rotate around Y ...
                 (x1 (+ (* x0 cy) (* z0 sy)))
                 (z1 (- (* z0 cy) (* x0 sy)))
                 ;; ... then tilt around X ...
                 (y2 (- (* y0 cx) (* z1 sx)))
                 (z2 (+ (* y0 sx) (* z1 cx)))
                 ;; ... and a simple perspective divide
                 (persp (/ 1.0 (+ 3.0 z2)))
                 (px (* x1 persp 2.1))
                 (py (* y2 persp 2.1))
                 (size (* persp (+ 4.0 (* 36.0 heat)))))
            (set-vertex v (/ px aspect) py heat size)
            (setq v (+ v 1))))))
    (gl-buffer-sub-data gl:+array-buffer+ 0 (* v 4))
    (gl:draw-arrays gl:+points+ 0 v)))

;; Build the pipeline at load time: this runs inside _initialize, after the
;; page has created the WebGL2 context and instantiated the module.
(setup-gl)

(rontolisp:wasm-export 'init :params '(:int) :returns :void)
(rontolisp:wasm-export 'frame :params '(:float) :returns :void)
(rontolisp:wasm-export 'total-heat :as "totalHeat" :params '() :returns :float)


---

# FILE: references/examples/browser/webgl-heat3d/index.html



  
    
    
    heat3d.lisp — a rank-3 array diffusing heat in Lisp, drawn by WebGL
    
    
    
    
  
  
    

    

      
heat3d.lisp → WebGL

      

        The state of this page is one rank-3 Lisp array: every frame,
        Lisp deposits heat at two orbiting sources with three-subscript
        (setf (aref grid i j k)), diffuses it across the lattice,
        normalizes the colors with the rank-generic linalg:amax, and
        projects every voxel itself. JavaScript is one-line WebGL bindings and
        this HUD.
      

    


    

    

    

    
  



---

# FILE: references/examples/browser/webgl-platformer/README.md

# webgl-platformer — a one-stage 3D platformer, in Lisp

Run with WASD (W runs
forward, into the screen), jump with Space, restart with
R; drag to orbit the camera, scroll to zoom (steering is
camera-relative, so W stays "into the screen" from any angle).
Stomp the walkers, grab the coins, cross the pits and reach the flag pole in
front of the castle.

Everything that makes it a game lives in `platformer.lisp`, compiled
ahead of time to WebAssembly:

- **Physics** — gravity, variable jump height (release Space
  early for a short hop), a jump buffer and coyote time, ground/air
  acceleration.
- **Collision** — classic per-axis AABB resolution against the level
  blocks; the same block list drives both the collision arrays and the
  baked level mesh.
- **Enemies** — patrolling walkers; land on one while falling and it is
  squashed (with a bounce), touch it any other way and you are back at the
  start.
- **Coins, the goal and the HUD state** — pickups, the flag-pole trigger,
  the run clock and the fall counter, all polled by the page through
  exported functions (`getCoins`, `getState`, `getTime`, ...).
- **The camera and every triangle** — the orbiting follow camera (drag and
  scroll arrive through the exported `orbit`/`zoom`, as in
  [`../webgl-robot-arm/`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-robot-arm)), its look-at and
  perspective matrices, and the whole world tessellated from yaw-rotated
  boxes each frame: the level and scenery are baked once at load time, the
  robot explorer, the walkers and the spinning coins are re-emitted every
  frame after them in the same vertex buffer.

The JavaScript side is the same one-line WebGL2 host boundary as the other
`webgl-*` demos — bindings generated from the shared `gl.wit` (see
[`../webgl-common/`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-common)) — plus keyboard
forwarding — the page only maps key events to small integers; held state,
buffering and coyote time are Lisp's business — and the HUD.

**Live demo:**  (this
directory is published as a subpath of the GitHub Pages site by
`.github/workflows/pages.yaml`).

## Building

```bash
./build.sh          # platformer.lisp -> platformer.wasm (--no-wasi --optimize)
# the page imports the generated ../webgl-common/gl-imports.js, so serve
# examples/browser rather than this directory:
jwebserver -p 8000 --directory ..
# open http://localhost:8000/webgl-platformer/
```

`--no-wasi` makes the module a reactor whose only imports are host functions
(`gl`, `ui`, `canvas`, `math`) — the ones `platformer.lisp` declares with
`rontolisp:wasm-import` plus the WebGL2 entries the shared `gl` package binds
from `../webgl-common/gl.wit`; `--optimize` tree-shakes the runtime and most
unused entries of that package, leaving `platformer.wasm` importing 34
functions. The page instantiates the module, calls `_initialize()` (which
compiles the shaders, parses the stage and bakes its mesh — from Lisp), then
calls the exported `frame` once per animation tick.

Open the DevTools console and poke the game directly: the module's exports
are on `window.lisp` (`lisp.getPx()`, `lisp.restart()`, ...).


---

# FILE: references/examples/browser/webgl-platformer/build.sh

#!/usr/bin/env bash
# Recompile platformer.lisp to a browser-loadable WebAssembly reactor.
# --no-wasi drops all WASI imports, so the module's only imports are the host
# functions platformer.lisp declares with rontolisp:wasm-import ("gl",
# "canvas" and "math"); --optimize tree-shakes the runtime so only the
# reachable functions ship.
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling platformer.lisp -> platformer.wasm"
java -jar "$jar" "$here/platformer.lisp" -o "$here/platformer.wasm" --no-wasi --optimize

# The page imports the generated ../webgl-common/gl-imports.js, so the served
# root is examples/browser rather than this directory.
echo "done. Serve the examples/browser directory over http, e.g.:"
echo "  jwebserver -p 8000 --directory \"$(dirname "$here")\""
echo "then open http://localhost:8000/webgl-platformer/"


---

# FILE: references/examples/browser/webgl-platformer/index.html



  
    
    
    platformer.lisp — a one-stage 3D platformer, in Lisp
    
  
  
    

    

      
platformer.lisp → WebGL

      

        A one-stage 3D platformer whose every rule lives in Lisp compiled to
        WebAssembly: the physics (jump buffering, coyote time and all), the
        per-axis AABB collision, the enemy patrols and the stomp rule, the
        follow camera and each of the rotated boxes it is built from.
        JavaScript is one-line WebGL bindings, this HUD and the keyboard.
      

    


    

    

    

    

    
  



---

# FILE: references/examples/browser/webgl-platformer/platformer.lisp

;;;; platformer.lisp -- a one-stage 3D platformer, entirely in Lisp.
;;;;
;;;; Run with W/A/S/D (W is forward, into the screen), jump with Space,
;;;; reach the flag pole. Everything that makes it a game lives here: the
;;;; physics (gravity, jump buffering, coyote time, variable jump height),
;;;; the per-axis AABB collision resolution against the level geometry, the
;;;; enemy patrols and the stomp-or-die rule, the coin pickups, the goal
;;;; trigger, the follow camera with its look-at/perspective matrices, and
;;;; every triangle of the world -- the level blocks, the scenery, the
;;;; little robot explorer (antenna, visor and all), the enemies and the
;;;; spinning coins are tessellated from rotated boxes each frame.
;;;; JavaScript is the same one-line WebGL2 host boundary as the other
;;;; webgl-* demos, plus keyboard forwarding and the HUD.
;;;;
;;;; Compiled ahead of time to a --no-wasi reactor (build.sh), so the module
;;;; imports nothing but the host functions declared below and instantiates
;;;; in any wasm-GC-capable browser.

;; --- the host boundary ------------------------------------------------------
;;
;; The WebGL2 API itself -- the wasm-import directives, the enum constants and
;; the shader helpers -- lives in the shared gl package
;; (../webgl-common/gl.lisp), spliced in here at compile time; --optimize
;; drops the entries this demo never calls. Only the imports specific to this
;; page stay below. GL objects cross as :int handles into a table the page
;; keeps; strings (GLSL source, info logs) cross as :string.

(require :gl "../webgl-common/gl.lisp")

;; The bulk-float staging path (see webgl-robot-arm): per-vertex floats cannot
;; cross into GPU memory one WASM value at a time, so the page keeps one
;; Float32Array of 9-float vertices (position, normal, color) plus a 16-float
;; scratch for the mat4 uniform. Colors are constant per box, so set-color
;; latches the current color and set-vertex stages position + normal + that
;; color (functions cross the WASM boundary with at most 7 parameters).
(rontolisp:wasm-import 'set-color
                       :from "gl"
                       :as "setColor"
                       :params '(:float :float :float)
                       :returns :void)
(rontolisp:wasm-import 'set-vertex
                       :from "gl"
                       :as "setVertex"
                       :params '(:int :float :float :float :float :float :float)
                       :returns :void)
(rontolisp:wasm-import 'gl-upload-vertices
                       :from "gl"
                       :as "uploadVertices"
                       :params '(:int :int)
                       :returns :void)
(rontolisp:wasm-import 'set-float
                       :from "gl"
                       :as "setFloat"
                       :params '(:int :float)
                       :returns :void)
(rontolisp:wasm-import 'gl-uniform-matrix4fv
                       :from "gl"
                       :as "uniformMatrix4fv"
                       :params '(:int)
                       :returns :void)

;; Canvas metrics, owned by the page.
(rontolisp:wasm-import 'canvas-width
                       :from "canvas"
                       :as "width"
                       :params '()
                       :returns :float)
(rontolisp:wasm-import 'canvas-height
                       :from "canvas"
                       :as "height"
                       :params '()
                       :returns :float)

;; The WASM backend has no transcendental built-ins, so borrow the host's.
(rontolisp:wasm-import 'sin :from "math" :params '(:float) :returns :float)
(rontolisp:wasm-import 'cos :from "math" :params '(:float) :returns :float)
(rontolisp:wasm-import 'atan2
                       :from "math"
                       :params '(:float :float)
                       :returns :float)

(defconstant +pi+ 3.141592653589793)
(defconstant +two-pi+ 6.283185307179586)

;; --- shaders ----------------------------------------------------------------
;;
;; One program: lit triangles under a warm daylight key, a sky-tinted
;; hemisphere ambient and a distance fog that fades the far scenery into the
;; sky color.

(defconstant +solid-vs+
  "#version 300 es
layout(location=0) in vec3 aPos;     // world-space position from Lisp
layout(location=1) in vec3 aNormal;  // world-space normal from Lisp
layout(location=2) in vec3 aColor;
uniform mat4 uVP;                    // view-projection, computed in Lisp
out vec3 vN;
out vec3 vC;
out vec3 vW;
void main() {
  gl_Position = uVP * vec4(aPos, 1.0);
  vN = aNormal;
  vC = aColor;
  vW = aPos;
}")

(defconstant +solid-fs+
  "#version 300 es
precision mediump float;
in vec3 vN;
in vec3 vC;
in vec3 vW;
uniform vec3 uEye;
out vec4 color;
void main() {
  vec3 n = normalize(vN);
  vec3 l = normalize(vec3(0.45, 0.80, 0.35));
  float diff = max(dot(n, l), 0.0);
  float amb  = 0.42 + 0.16 * n.y;           // hemisphere: tops brighter
  vec3 lit = vC * (amb + 0.62 * diff);
  vec3 sky = vec3(0.52, 0.74, 0.98);
  float fog = smoothstep(18.0, 55.0, distance(uEye, vW));
  color = vec4(mix(lit, sky, fog), 1.0);
}")

;; --- 4x4 matrix math ----------------------------------------------------------
;; Column-major flat 16-element arrays (the OpenGL convention, as in
;; webgl-cube): element (row, col) lives at index (+ row (* col 4)).

(defun mat4-zero () (make-array 16 :initial-element 0.0))

(defun mat4-mul (a b)
  (let ((out (mat4-zero)))
    (dotimes (col 4)
      (dotimes (row 4)
        (let ((sum 0.0))
          (dotimes (k 4)
            (setq sum
             (+ sum (* (aref a (+ row (* k 4))) (aref b (+ k (* col 4)))))))
          (setf (aref out (+ row (* col 4))) sum))))
    out))

(defconstant +half-fov+ 0.39269908169872414) ; pi/8: half the 45-degree fov

(defun mat4-perspective (aspect near far)
  (let* ((f (/ (cos +half-fov+) (sin +half-fov+)))
         (nf (/ 1.0 (- near far)))
         (m (mat4-zero)))
    (setf (aref m 0) (/ f aspect))
    (setf (aref m 5) f)
    (setf (aref m 10) (* (+ far near) nf))
    (setf (aref m 11) -1.0)
    (setf (aref m 14) (* 2.0 far near nf))
    m))

;; --- the follow camera --------------------------------------------------------
;;
;; The eye orbits a smoothed copy of the player's position: drag gestures
;; arrive through the exported `orbit` (yaw/pitch deltas) and `zoom`, as in
;; webgl-robot-arm. At the default yaw the camera sits at -x looking down
;; the course, so W runs toward the flag in the distance -- and steering is
;; camera-relative, so W stays "into the screen" from any angle.

(defvar *camx* 0.0) ; the smoothed follow point
(defvar *camy* 0.0)
(defvar *camz* 0.0)
;; The default orbit reproduces the original fixed follow camera: 7.2 back,
;; 3.3 up, looking down the course. A restart puts the orbit back here.
(defconstant +cam-yaw-0+ 0.0) ; 0 looks along +x, down the course
(defconstant +cam-pitch-0+ 0.43)
(defconstant +cam-dist-0+ 7.9)

(defvar *cam-yaw* +cam-yaw-0+)
(defvar *cam-pitch* +cam-pitch-0+)
(defvar *cam-dist* +cam-dist-0+)
(defvar *eyex* 0.0)
(defvar *eyey* 0.0)
(defvar *eyez* 0.0)
(defvar *aspect* 1.0)
(defvar *vp* nil) ; the current view-projection matrix

(defun build-view (tx ty tz)
  ;; The look-at view matrix for eye -> target, straight into column-major
  ;; slots: rows are camera right / up / -forward.
  (let* ((fx (- tx *eyex*))
         (fy (- ty *eyey*))
         (fz (- tz *eyez*))
         (fl (sqrt (+ (* fx fx) (* fy fy) (* fz fz))))
         (nfx (/ fx fl))
         (nfy (/ fy fl))
         (nfz (/ fz fl))
         ;; right = normalize(cross(forward, world-up))
         (rx0 (- 0.0 nfz))
         (rz0 nfx)
         (rl (sqrt (+ (* rx0 rx0) (* rz0 rz0))))
         (rx (/ rx0 rl))
         (rz (/ rz0 rl))
         ;; up = cross(right, forward), with right.y = 0
         (ux (- 0.0 (* rz nfy)))
         (uy (- (* rz nfx) (* rx nfz)))
         (uz (* rx nfy))
         (v (mat4-zero)))
    (setf (aref v 0) rx)
    (setf (aref v 4) 0.0)
    (setf (aref v 8) rz)
    (setf (aref v 12) (- 0.0 (+ (* rx *eyex*) (* rz *eyez*))))
    (setf (aref v 1) ux)
    (setf (aref v 5) uy)
    (setf (aref v 9) uz)
    (setf (aref v 13) (- 0.0 (+ (* ux *eyex*) (* uy *eyey*) (* uz *eyez*))))
    (setf (aref v 2) (- 0.0 nfx))
    (setf (aref v 6) (- 0.0 nfy))
    (setf (aref v 10) (- 0.0 nfz))
    (setf (aref v 14) (+ (* nfx *eyex*) (* nfy *eyey*) (* nfz *eyez*)))
    (setf (aref v 15) 1.0)
    v))

(defun orbit (dx dy)
  ;; Exported: drag deltas, normalized by the canvas height.
  (setq *cam-yaw* (- *cam-yaw* (* 3.4 dx)))
  (let ((p (+ *cam-pitch* (* 2.6 dy))))
    (setq *cam-pitch* (max 0.12 (min 1.35 p)))))

(defun zoom (dz)
  ;; Exported: scroll-wheel deltas.
  (setq *cam-dist* (max 4.0 (min 14.0 (+ *cam-dist* dz)))))

(defun update-camera (dt)
  ;; Ease the follow point toward the player (never below the horizon while
  ;; falling into a pit), then place the eye on its orbit around it.
  (let ((k (min 1.0 (* 5.0 dt))) (ty (max *py* -0.5)))
    (setq *camx* (+ *camx* (* k (- *px* *camx*))))
    (setq *camy* (+ *camy* (* k (- ty *camy*))))
    (setq *camz* (+ *camz* (* k (- *pz* *camz*)))))
  (let ((cp (cos *cam-pitch*))
        (sp (sin *cam-pitch*))
        (cy (cos *cam-yaw*))
        (sy (sin *cam-yaw*)))
    (setq *eyex* (- *camx* (* *cam-dist* cp cy)))
    (setq *eyey* (+ *camy* (* *cam-dist* sp)))
    (setq *eyez* (- *camz* (* *cam-dist* cp sy)))
    (setq *vp*
          (mat4-mul (mat4-perspective *aspect* 0.1 90.0)
                    (build-view (+ *camx* (* 1.5 cy)) (+ *camy* 1.0)
                                (+ *camz* (* 1.5 sy)))))))

;; --- GL pipeline setup --------------------------------------------------------

(defvar *prog* 0)
(defvar *u-vp* 0)
(defvar *u-eye* 0)
(defvar *vao* 0)
(defvar *buf* 0)

(defconstant +max-verts+ 8192) ; lit-triangle vertex capacity

(defun setup-gl ()
  (setq *prog* (gl:build-program +solid-vs+ +solid-fs+))
  (setq *u-vp* (gl:get-uniform-location *prog* "uVP"))
  (setq *u-eye* (gl:get-uniform-location *prog* "uEye"))
  (gl:enable gl:+depth-test+)
  ;; one VAO: position + normal + color, 36 bytes per vertex
  (setq *vao* (gl:create-vertex-array))
  (gl:bind-vertex-array *vao*)
  (setq *buf* (gl:create-buffer))
  (gl:bind-buffer gl:+array-buffer+ *buf*)
  (gl:buffer-data gl:+array-buffer+ (* +max-verts+ 36) gl:+dynamic-draw+)
  (gl:enable-vertex-attrib-array 0)
  (gl:vertex-attrib-pointer 0 3 gl:+float+ nil 36 0)
  (gl:enable-vertex-attrib-array 1)
  (gl:vertex-attrib-pointer 1 3 gl:+float+ nil 36 12)
  (gl:enable-vertex-attrib-array 2)
  (gl:vertex-attrib-pointer 2 3 gl:+float+ nil 36 24))

;; --- box tessellation -----------------------------------------------------------
;;
;; Everything in the world is a yaw-rotated box. emit-box writes the 8 world
;; corners into scratch arrays, then stamps the 6 faces (12 triangles) with
;; face normals rotated by the same yaw. Corner index bits: bit 0 = +x,
;; bit 1 = +y, bit 2 = +z in local space.

(defvar *v* 0)            ; vertex write cursor
(defvar *static-verts* 0) ; level + scenery, uploaded once

(defvar *cwx* (make-array 8 :initial-element 0.0))
(defvar *cwy* (make-array 8 :initial-element 0.0))
(defvar *cwz* (make-array 8 :initial-element 0.0))

(defun emit-v (x y z nx ny nz)
  ;; stages one vertex with the color latched by the last set-color call
  (set-vertex *v* x y z nx ny nz)
  (setq *v* (+ *v* 1)))

(defun emit-face (a b c d nx ny nz)
  ;; one box face (two triangles) from the scratch corners
  (emit-v (aref *cwx* a) (aref *cwy* a) (aref *cwz* a) nx ny nz)
  (emit-v (aref *cwx* b) (aref *cwy* b) (aref *cwz* b) nx ny nz)
  (emit-v (aref *cwx* c) (aref *cwy* c) (aref *cwz* c) nx ny nz)
  (emit-v (aref *cwx* a) (aref *cwy* a) (aref *cwz* a) nx ny nz)
  (emit-v (aref *cwx* c) (aref *cwy* c) (aref *cwz* c) nx ny nz)
  (emit-v (aref *cwx* d) (aref *cwy* d) (aref *cwz* d) nx ny nz))

(defun emit-box (cx cy cz hx hy hz yaw)
  ;; a box centered at c with half extents h, rotated around y by yaw;
  ;; local +x rotates to world (cos yaw, 0, -sin yaw)
  (let ((c (cos yaw)) (s (sin yaw)))
    (dotimes (i 8)
      (let ((lx (if (= (logand i 1) 1) hx (- 0.0 hx)))
            (ly (if (= (logand i 2) 2) hy (- 0.0 hy)))
            (lz (if (= (logand i 4) 4) hz (- 0.0 hz))))
        (setf (aref *cwx* i) (+ cx (* lx c) (* lz s)))
        (setf (aref *cwy* i) (+ cy ly))
        (setf (aref *cwz* i) (+ cz (- (* lz c) (* lx s))))))
    (emit-face 4 5 7 6 s 0.0 c)                 ; +z
    (emit-face 1 0 2 3 (- 0.0 s) 0.0 (- 0.0 c)) ; -z
    (emit-face 5 1 3 7 c 0.0 (- 0.0 s))         ; +x
    (emit-face 0 4 6 2 (- 0.0 c) 0.0 s)         ; -x
    (emit-face 6 7 3 2 0.0 1.0 0.0)             ; +y
    (emit-face 0 1 5 4 0.0 -1.0 0.0)))          ; -y

;; A local frame for composite figures (the player, the enemies, the coins):
;; set-origin latches a world position + yaw, and part emits one box given in
;; that frame's local coordinates.
(defvar *ox* 0.0)
(defvar *oy* 0.0)
(defvar *oz* 0.0)
(defvar *oyaw* 0.0)
(defvar *oc* 1.0)
(defvar *os* 0.0)

(defun set-origin (x y z yaw)
  (setq *ox* x)
  (setq *oy* y)
  (setq *oz* z)
  (setq *oyaw* yaw)
  (setq *oc* (cos yaw))
  (setq *os* (sin yaw)))

(defun part (lx ly lz hx hy hz)
  (emit-box (+ *ox* (* lx *oc*) (* lz *os*)) (+ *oy* ly)
            (+ *oz* (- (* lz *oc*) (* lx *os*))) hx hy hz *oyaw*))

;; --- the stage ------------------------------------------------------------------
;;
;; One course along +x: grass runs broken by two pits, a pipe, brick
;; platforms, a staircase and the goal flag on the far ground. Each solid
;; block is (x0 y0 z0 x1 y1 z1 r g b); the same list drives both the baked
;; static mesh and the collision arrays. Scenery blocks render identically
;; but nothing collides with them.

(defconstant +solids+
  '((-4.0 -1.0 -3.5 14.0 0.0 3.5 0.38 0.70 0.34)  ; grass run A
    (-4.0 -2.2 -3.5 14.0 -1.0 3.5 0.47 0.34 0.24) ; its dirt band
    (7.0 1.2 0.6 9.5 1.8 2.6 0.76 0.47 0.29)      ; brick platform
    (16.0 -1.0 -3.5 31.0 0.0 3.5 0.38 0.70 0.34)  ; grass run B
    (16.0 -2.2 -3.5 31.0 -1.0 3.5 0.47 0.34 0.24)
    (20.0 0.0 -2.6 21.6 1.3 -1.0 0.22 0.62 0.30)      ; the pipe body
    (19.85 1.3 -2.75 21.75 1.75 -0.85 0.26 0.70 0.34) ; the pipe lip
    (23.0 1.6 -3.0 26.0 2.2 -0.5 0.76 0.47 0.29)      ; high bricks
    (32.0 0.5 -0.7 33.4 1.1 0.9 0.64 0.62 0.60)       ; stone step, pit B
    (34.0 -1.0 -3.5 48.0 0.0 3.5 0.38 0.70 0.34)      ; grass run C
    (34.0 -2.2 -3.5 48.0 -1.0 3.5 0.47 0.34 0.24)
    (49.0 0.4 -0.8 50.4 1.0 0.8 0.64 0.62 0.60)  ; stone step, pit C
    (51.0 -1.0 -3.5 68.0 0.0 3.5 0.38 0.70 0.34) ; goal ground
    (51.0 -2.2 -3.5 68.0 -1.0 3.5 0.47 0.34 0.24)
    (55.0 0.0 -2.2 59.0 0.7 0.2 0.82 0.66 0.40) ; the staircase
    (56.0 0.7 -2.2 59.0 1.4 0.2 0.82 0.66 0.40)
    (57.0 1.4 -2.2 59.0 2.1 0.2 0.82 0.66 0.40)
    (58.0 2.1 -2.2 59.0 2.8 0.2 0.82 0.66 0.40)
    (61.4 0.0 -1.6 62.6 0.5 -0.4 0.64 0.62 0.60)   ; flag pole base
    (64.6 0.0 -3.4 67.6 2.4 -0.9 0.86 0.83 0.78)   ; castle keep
    (65.6 2.4 -2.7 66.6 3.4 -1.6 0.86 0.83 0.78))) ; castle tower

(defconstant +scenery+
  '((61.95 0.5 -1.03 62.05 4.4 -0.97 0.80 0.83 0.88)   ; the flag pole
    (61.97 4.4 -1.05 62.03 4.52 -0.95 1.00 0.83 0.25)  ; its finial
    (61.99 3.65 -0.95 62.01 4.15 -0.25 0.24 0.70 0.34) ; the flag, facing the camera
    (64.55 0.0 -2.5 64.65 1.1 -1.7 0.35 0.25 0.18)     ; castle door
    (65.9 3.4 -2.17 65.98 4.1 -2.13 0.80 0.83 0.88)    ; castle banner pole
    (65.92 3.75 -2.13 65.96 4.1 -1.61 0.90 0.30 0.24)  ; castle banner
    (6.0 -1.0 -13.0 18.0 2.6 -7.5 0.30 0.58 0.31)      ; hills, left bank
    (24.0 -1.0 -15.0 42.0 4.2 -9.0 0.27 0.53 0.30)
    (46.0 -1.0 -13.0 60.0 2.2 -7.8 0.30 0.58 0.31)
    (10.0 -1.0 7.5 22.0 3.0 12.5 0.28 0.55 0.30) ; hills, right bank
    (30.0 -1.0 8.0 44.0 2.4 13.0 0.30 0.58 0.31)
    (52.0 -1.0 7.5 64.0 3.4 12.5 0.28 0.55 0.30)
    (76.0 -1.0 -16.0 100.0 8.0 16.0 0.26 0.50 0.29) ; the far range ahead
    (2.0 5.6 -11.0 6.0 6.6 -9.6 0.99 0.99 0.99)     ; clouds
    (20.0 6.8 -13.0 25.5 7.9 -11.4 0.99 0.99 0.99)
    (37.0 5.9 -11.5 41.5 6.9 -10.1 0.99 0.99 0.99)
    (54.0 6.7 -13.5 59.0 7.8 -11.9 0.99 0.99 0.99)
    (14.0 6.2 8.5 18.5 7.2 10.2 0.99 0.99 0.99)
    (44.0 7.0 9.0 48.5 8.0 10.8 0.99 0.99 0.99)))

;; The collision arrays, parsed once from +solids+.
(defvar *nsolid* 0)
(defvar *sx0* nil)
(defvar *sy0* nil)
(defvar *sz0* nil)
(defvar *sx1* nil)
(defvar *sy1* nil)
(defvar *sz1* nil)

(defun parse-solids ()
  (setq *nsolid* (length +solids+))
  (setq *sx0* (make-array *nsolid* :initial-element 0.0))
  (setq *sy0* (make-array *nsolid* :initial-element 0.0))
  (setq *sz0* (make-array *nsolid* :initial-element 0.0))
  (setq *sx1* (make-array *nsolid* :initial-element 0.0))
  (setq *sy1* (make-array *nsolid* :initial-element 0.0))
  (setq *sz1* (make-array *nsolid* :initial-element 0.0))
  (let ((i 0))
    (dolist (b +solids+)
      (setf (aref *sx0* i) (nth 0 b))
      (setf (aref *sy0* i) (nth 1 b))
      (setf (aref *sz0* i) (nth 2 b))
      (setf (aref *sx1* i) (nth 3 b))
      (setf (aref *sy1* i) (nth 4 b))
      (setf (aref *sz1* i) (nth 5 b))
      (setq i (+ i 1)))))

(defun emit-block (b)
  ;; one static block: an axis-aligned box from its corner list entry
  (set-color (nth 6 b) (nth 7 b) (nth 8 b))
  (emit-box (* 0.5 (+ (nth 0 b) (nth 3 b))) (* 0.5 (+ (nth 1 b) (nth 4 b)))
            (* 0.5 (+ (nth 2 b) (nth 5 b))) (* 0.5 (- (nth 3 b) (nth 0 b)))
            (* 0.5 (- (nth 4 b) (nth 1 b))) (* 0.5 (- (nth 5 b) (nth 2 b)))
            0.0))

(defun bake-static ()
  ;; the level and the scenery, tessellated once into the front of the buffer
  (setq *v* 0)
  (dolist (b +solids+) (emit-block b))
  (dolist (b +scenery+) (emit-block b))
  (setq *static-verts* *v*)
  (gl:bind-buffer gl:+array-buffer+ *buf*)
  (gl-upload-vertices 0 (* *static-verts* 9)))

;; --- coins ----------------------------------------------------------------------

(defconstant +coins+
  '((7.6 2.4 1.6) (8.4 2.4 1.6) (9.2 2.4 1.6)       ; over the bricks
    (14.6 1.1 0.0) (15.2 1.5 0.0) (15.8 1.1 0.0)    ; the arc over pit A
    (23.6 2.9 -1.7) (24.5 2.9 -1.7) (25.4 2.9 -1.7) ; on the high bricks
    (41.0 0.7 0.5) (42.0 0.7 0.5) (43.0 0.7 0.5)    ; the run-C row
    (49.2 1.8 0.0) (50.2 1.8 0.0)))                 ; the arc over pit C

(defvar *ncoin* 0)
(defvar *coinx* nil)
(defvar *coiny* nil)
(defvar *coinz* nil)
(defvar *ctaken* nil) ; t once collected
(defvar *coins* 0)    ; the HUD counter

(defun parse-coins ()
  (setq *ncoin* (length +coins+))
  (setq *coinx* (make-array *ncoin* :initial-element 0.0))
  (setq *coiny* (make-array *ncoin* :initial-element 0.0))
  (setq *coinz* (make-array *ncoin* :initial-element 0.0))
  (setq *ctaken* (make-array *ncoin* :initial-element nil))
  (let ((i 0))
    (dolist (c +coins+)
      (setf (aref *coinx* i) (nth 0 c))
      (setf (aref *coiny* i) (nth 1 c))
      (setf (aref *coinz* i) (nth 2 c))
      (setq i (+ i 1)))))

;; --- enemies --------------------------------------------------------------------
;;
;; Each entry is (x z minx maxx): a patroller shuffling along x on flat
;; ground (y 0), turning at its bounds. Stomp it from above to squash it;
;; touch it any other way and you lose a life.

(defconstant +enemy-list+
  '((18.0 1.2 17.2 28.5) (36.0 -1.5 35.0 46.5) (43.0 1.8 36.5 47.0)
    (53.0 1.6 51.8 54.4)))

(defconstant +enemy-speed+ 1.4)

(defvar *nenemy* 0)
(defvar *ex* nil)
(defvar *ez* nil)
(defvar *eminx* nil)
(defvar *emaxx* nil)
(defvar *edir* nil)    ; +1.0 / -1.0
(defvar *ealive* nil)  ; t while walking
(defvar *esquash* nil) ; squashed-remains countdown

(defun parse-enemies ()
  (setq *nenemy* (length +enemy-list+))
  (setq *ex* (make-array *nenemy* :initial-element 0.0))
  (setq *ez* (make-array *nenemy* :initial-element 0.0))
  (setq *eminx* (make-array *nenemy* :initial-element 0.0))
  (setq *emaxx* (make-array *nenemy* :initial-element 0.0))
  (setq *edir* (make-array *nenemy* :initial-element 1.0))
  (setq *ealive* (make-array *nenemy* :initial-element t))
  (setq *esquash* (make-array *nenemy* :initial-element 0.0))
  (let ((i 0))
    (dolist (e +enemy-list+)
      (setf (aref *ex* i) (nth 0 e))
      (setf (aref *ez* i) (nth 1 e))
      (setf (aref *eminx* i) (nth 2 e))
      (setf (aref *emaxx* i) (nth 3 e))
      (setf (aref *edir* i) 1.0)
      (setf (aref *ealive* i) t)
      (setf (aref *esquash* i) 0.0)
      (setq i (+ i 1)))))

(defun update-enemies (dt)
  (dotimes (i *nenemy*)
    (if (aref *ealive* i)
        (let ((x (+ (aref *ex* i) (* (aref *edir* i) +enemy-speed+ dt))))
          (when (> x (aref *emaxx* i))
            (setq x (aref *emaxx* i))
            (setf (aref *edir* i) -1.0))
          (when (< x (aref *eminx* i))
            (setq x (aref *eminx* i))
            (setf (aref *edir* i) 1.0))
          (setf (aref *ex* i) x))
        (when (> (aref *esquash* i) 0.0)
          (setf (aref *esquash* i) (- (aref *esquash* i) dt))))))

;; --- the player -----------------------------------------------------------------
;;
;; Position is the feet; the collision volume is the AABB
;; [p - (hx, 0, hz), p + (hx, height, hz)].

(defconstant +p-hx+ 0.30)
(defconstant +p-hz+ 0.30)
(defconstant +p-h+ 0.95) ; standing height
(defconstant +run-speed+ 4.6)
(defconstant +gravity+ 26.0)
(defconstant +jump-v+ 10.0) ; apex ~1.9, just over two blocks
(defconstant +spawn-x+ 0.0)
(defconstant +spawn-y+ 0.5)
(defconstant +spawn-z+ 0.0)
(defconstant +goal-x+ 62.0) ; the flag pole
(defconstant +goal-z+ -1.0)

(defvar *px* 0.0)
(defvar *py* 0.0)
(defvar *pz* 0.0)
(defvar *vx* 0.0)
(defvar *vy* 0.0)
(defvar *vz* 0.0)
(defvar *grounded* nil)
(defvar *coyote* 0.0)    ; grace after running off an edge
(defvar *jump-buf* 0.0)  ; grace before landing
(defvar *yaw* 0.0)       ; facing; 0 looks along +x
(defvar *run-phase* 0.0) ; the run-cycle oscillator

;; game state: 0 playing, 1 dying, 2 course clear
(defvar *state* 0)
(defvar *state-t* 0.0) ; time in the current state
(defvar *deaths* 0)
(defvar *start-tm* -1.0) ; wall time when the run started
(defvar *elapsed* 0.0)   ; frozen at the moment of clearing
(defvar *last-tm* 0.0)
(defvar *pending-reset* nil)

;; keyboard state, forwarded by the page: 1.0 while held
(defvar *in-l* 0.0)    ; A
(defvar *in-r* 0.0)    ; D
(defvar *in-f* 0.0)    ; W (away from the camera)
(defvar *in-b* 0.0)    ; S (toward the camera)
(defvar *in-jump* 0.0) ; Space
(defvar *jump-prev* nil)

(defun set-key (code down)
  ;; Exported: 0 = A, 1 = D, 2 = W, 3 = S, 4 = Space; down is 1 or 0.
  (let ((v (if (= down 1) 1.0 0.0)))
    (cond ((= code 0) (setq *in-l* v))
          ((= code 1) (setq *in-r* v))
          ((= code 2) (setq *in-f* v))
          ((= code 3) (setq *in-b* v))
          ((= code 4) (setq *in-jump* v)))))

(defun restart ()
  ;; Exported: back to the start, coins restored, enemies alive, clock zero.
  (setq *pending-reset* t))

(defun respawn ()
  (setq *px* +spawn-x+)
  (setq *py* +spawn-y+)
  (setq *pz* +spawn-z+)
  (setq *vx* 0.0)
  (setq *vy* 0.0)
  (setq *vz* 0.0)
  (setq *yaw* 0.0)
  (setq *grounded* nil)
  (setq *coyote* 0.0)
  (setq *jump-buf* 0.0)
  (setq *state* 0)
  (setq *state-t* 0.0)
  (setq *camx* *px*)
  (setq *camy* *py*)
  (setq *camz* *pz*))

(defun reset-game (tm)
  (respawn)
  (setq *cam-yaw* +cam-yaw-0+)
  (setq *cam-pitch* +cam-pitch-0+)
  (setq *cam-dist* +cam-dist-0+)
  (setq *coins* 0)
  (setq *deaths* 0)
  (setq *start-tm* tm)
  (setq *elapsed* 0.0)
  (dotimes (i *ncoin*) (setf (aref *ctaken* i) nil))
  (dotimes (i *nenemy*)
    (setf (aref *ex* i) (nth 0 (nth i +enemy-list+)))
    (setf (aref *edir* i) 1.0)
    (setf (aref *ealive* i) t)
    (setf (aref *esquash* i) 0.0)))

(defun die ()
  (setq *state* 1)
  (setq *state-t* 0.0)
  (setq *deaths* (+ *deaths* 1))
  (setq *vy* 8.0)) ; the little farewell hop

;; --- collision -------------------------------------------------------------------
;;
;; Classic per-axis resolution: integrate one axis, then push the player's
;; AABB out of any solid it entered, killing that axis' velocity. The y pass
;; also decides groundedness.

(defun overlap-p (i)
  (and (> (+ *px* +p-hx+) (aref *sx0* i)) (< (- *px* +p-hx+) (aref *sx1* i))
       (> (+ *py* +p-h+) (aref *sy0* i)) (< *py* (aref *sy1* i))
       (> (+ *pz* +p-hz+) (aref *sz0* i)) (< (- *pz* +p-hz+) (aref *sz1* i))))

(defun resolve-x ()
  (dotimes (i *nsolid*)
    (when (overlap-p i)
      (if (> *vx* 0.0)
          (setq *px* (- (aref *sx0* i) +p-hx+))
          (setq *px* (+ (aref *sx1* i) +p-hx+)))
      (setq *vx* 0.0))))

(defun resolve-z ()
  (dotimes (i *nsolid*)
    (when (overlap-p i)
      (if (> *vz* 0.0)
          (setq *pz* (- (aref *sz0* i) +p-hz+))
          (setq *pz* (+ (aref *sz1* i) +p-hz+)))
      (setq *vz* 0.0))))

(defun resolve-y ()
  (dotimes (i *nsolid*)
    (when (overlap-p i)
      (if (<= *vy* 0.0)
          (progn
            (setq *py* (aref *sy1* i))
            (setq *grounded* t))
          (setq *py* (- (aref *sy0* i) +p-h+)))
      (setq *vy* 0.0))))

(defun move-player (dt)
  (setq *px* (+ *px* (* *vx* dt)))
  (resolve-x)
  (setq *pz* (+ *pz* (* *vz* dt)))
  (resolve-z)
  (setq *grounded* nil)
  (setq *py* (+ *py* (* *vy* dt)))
  (resolve-y))

;; --- the playing-state step -----------------------------------------------------

(defun steer (dt)
  ;; Accelerate toward the held direction; the ground grips harder than air.
  ;; Steering is camera-relative: W runs away from the camera, A/D strafe
  ;; across it, whatever the current orbit yaw.
  (let* ((f (- *in-f* *in-b*))
         (r (- *in-r* *in-l*))
         ;; keep diagonals at running speed
         (n (if (or (= f 0.0) (= r 0.0)) 1.0 0.7071))
         (cy (cos *cam-yaw*))
         (sy (sin *cam-yaw*))
         (tx (* +run-speed+ n (- (* f cy) (* r sy))))
         (tz (* +run-speed+ n (+ (* f sy) (* r cy))))
         (acc (* dt (if *grounded* 34.0 16.0))))
    (let ((dx (- tx *vx*)) (dz (- tz *vz*)))
      (setq *vx* (+ *vx* (max (- 0.0 acc) (min acc dx))))
      (setq *vz* (+ *vz* (max (- 0.0 acc) (min acc dz)))))))

(defun update-facing (dt)
  ;; Turn toward the velocity, the short way around.
  (let ((sp (+ (* *vx* *vx*) (* *vz* *vz*))))
    (when (> sp 0.09)
      (let ((d (- (atan2 (- 0.0 *vz*) *vx*) *yaw*)))
        (while (> d +pi+) (setq d (- d +two-pi+)))
        (while (< d (- 0.0 +pi+)) (setq d (+ d +two-pi+)))
        (setq *yaw* (+ *yaw* (* d (min 1.0 (* 14.0 dt)))))))))

(defun jump-control (dt)
  ;; Buffered, coyote-timed, variable-height jumping.
  (let ((held (> *in-jump* 0.5)))
    (when (and held (not *jump-prev*)) (setq *jump-buf* 0.12))
    (setq *jump-prev* held)
    (when (> *jump-buf* 0.0) (setq *jump-buf* (- *jump-buf* dt)))
    (if *grounded*
        (setq *coyote* 0.10)
        (when (> *coyote* 0.0) (setq *coyote* (- *coyote* dt))))
    (when (and (> *jump-buf* 0.0) (or *grounded* (> *coyote* 0.0)))
      (setq *vy* +jump-v+)
      (setq *grounded* nil)
      (setq *coyote* 0.0)
      (setq *jump-buf* 0.0))
    ;; release early for a shorter hop
    (when (and (not held) (> *vy* 3.5)) (setq *vy* 3.5))))

(defun check-coins ()
  (dotimes (i *ncoin*)
    (unless (aref *ctaken* i)
      (let ((dx (- (aref *coinx* i) *px*))
            (dy (- (aref *coiny* i) (+ *py* 0.55)))
            (dz (- (aref *coinz* i) *pz*)))
        (when (< (+ (* dx dx) (* dy dy) (* dz dz)) 0.5)
          (setf (aref *ctaken* i) t)
          (setq *coins* (+ *coins* 1)))))))

(defun check-enemies ()
  (dotimes (i *nenemy*)
    (when (aref *ealive* i)
      (let ((dx (- *px* (aref *ex* i))) (dz (- *pz* (aref *ez* i))))
        (when (and (< (abs dx) (+ +p-hx+ 0.27)) (< (abs dz) (+ +p-hz+ 0.25))
                   (> (+ *py* +p-h+) 0.05) (< *py* 0.58))
          (if (and (< *vy* -0.5) (> *py* 0.12))
              (progn ; the stomp
                (setf (aref *ealive* i) nil)
                (setf (aref *esquash* i) 0.6)
                (setq *vy* 8.0))
              (die)))))))

(defun check-goal ()
  (when (and (> *px* (- +goal-x+ 0.95)) (< *px* (+ +goal-x+ 0.95))
             (> *pz* (- +goal-z+ 1.3)) (< *pz* (+ +goal-z+ 1.3)) (< *py* 4.4))
    (setq *state* 2)
    (setq *state-t* 0.0)
    (setq *elapsed* (- *last-tm* *start-tm*))))

(defun step-playing (dt)
  (steer dt)
  (jump-control dt)
  (setq *vy* (max -22.0 (- *vy* (* +gravity+ dt))))
  (move-player dt)
  (update-facing dt)
  ;; the run cycle swings the limbs while grounded and moving
  (let ((sp (sqrt (+ (* *vx* *vx*) (* *vz* *vz*)))))
    (if (and *grounded* (> sp 0.4))
        (setq *run-phase* (+ *run-phase* (* sp 3.2 dt)))
        (setq *run-phase* 0.0)))
  (check-coins)
  (check-enemies)
  (check-goal)
  (when (< *py* -7.0)
    (die)
    (setq *vy* 0.0))) ; already falling: no farewell hop

(defun step-dying (dt)
  ;; a spin and a fall, free of the world, then back to the start
  (setq *vy* (- *vy* (* +gravity+ dt)))
  (setq *py* (+ *py* (* *vy* dt)))
  (setq *yaw* (+ *yaw* (* 12.0 dt)))
  (when (> *state-t* 1.4) (respawn)))

(defun step-clear (dt)
  ;; land, stop and face the camera for the bow
  (setq *vx* (* *vx* (max 0.0 (- 1.0 (* 8.0 dt)))))
  (setq *vz* 0.0)
  (setq *vy* (max -22.0 (- *vy* (* +gravity+ dt))))
  (move-player dt)
  (let ((d (- (atan2 (sin *cam-yaw*) (- 0.0 (cos *cam-yaw*))) *yaw*)))
    (while (> d +pi+) (setq d (- d +two-pi+)))
    (while (< d (- 0.0 +pi+)) (setq d (+ d +two-pi+)))
    (setq *yaw* (+ *yaw* (* d (min 1.0 (* 8.0 dt)))))))

;; --- drawing the cast -----------------------------------------------------------

(defun emit-shadow ()
  ;; a soft dark pad on the highest solid top under the feet, scaled down
  ;; with altitude -- the landing aid every platformer owes its player
  (let ((top -99.0))
    (dotimes (i *nsolid*)
      (when (and (> (+ *px* 0.2) (aref *sx0* i)) (< (- *px* 0.2) (aref *sx1* i))
                 (> (+ *pz* 0.2) (aref *sz0* i)) (< (- *pz* 0.2) (aref *sz1* i))
                 (<= (aref *sy1* i) (+ *py* 0.05)) (> (aref *sy1* i) top))
        (setq top (aref *sy1* i))))
    (when (> top -50.0)
      (let ((k (max 0.3 (- 1.0 (* 0.16 (- *py* top))))))
        (set-color 0.16 0.28 0.16)
        (emit-box *px* (+ top 0.02) *pz* (* 0.30 k) 0.008 (* 0.30 k) 0.0)))))

(defun emit-player (tm)
  ;; the robot explorer: teal chassis, white head with a wraparound visor,
  ;; an antenna and an orange field pack -- built from a dozen rotated boxes
  (set-origin *px* *py* *pz* *yaw*)
  (let* ((swing (if *grounded* (sin *run-phase*) 0.6))
         (leg (* 0.11 swing))
         (arm (if (= *state* 2) 0.0 (* -0.10 swing)))
         (army (if (= *state* 2) (+ 0.72 (* 0.02 (sin (* tm 6.0)))) 0.55)))
    ;; legs (charcoal), swinging front-to-back on the run cycle
    (set-color 0.23 0.25 0.31)
    (part leg 0.17 -0.10 0.08 0.17 0.07)
    (part (- 0.0 leg) 0.17 0.10 0.08 0.17 0.07)
    ;; the teal torso and its lighter chest plate
    (set-color 0.13 0.62 0.58)
    (part 0.0 0.50 0.0 0.17 0.19 0.14)
    (set-color 0.72 0.88 0.85)
    (part 0.15 0.52 0.0 0.03 0.10 0.08)
    ;; arms (charcoal) -- raised overhead once the course is clear
    (set-color 0.23 0.25 0.31)
    (part arm army -0.21 0.05 0.14 0.05)
    (part (- 0.0 arm) army 0.21 0.05 0.14 0.05)
    ;; the orange field pack on the back
    (set-color 0.95 0.55 0.16)
    (part -0.20 0.56 0.0 0.05 0.12 0.10)
    ;; the white head with its dark wraparound visor
    (set-color 0.90 0.93 0.95)
    (part 0.0 0.84 0.0 0.13 0.115 0.12)
    (set-color 0.09 0.11 0.17)
    (part 0.115 0.86 0.0 0.025 0.05 0.095)
    ;; the antenna: a thin stalk and its glowing tip
    (set-color 0.55 0.58 0.62)
    (part 0.0 1.0 0.0 0.012 0.05 0.012)
    (set-color 1.0 0.62 0.20)
    (part 0.0 1.07 0.0 0.032 0.032 0.032)))

(defun emit-enemy (i tm)
  (let ((yaw (if (> (aref *edir* i) 0.0) 0.0 +pi+)))
    (set-origin (aref *ex* i) 0.0 (aref *ez* i) yaw)
    (if (aref *ealive* i)
        (let ((sw (sin (+ (* tm 9.0) (* 1.7 i)))))
          ;; a plum-colored walker: shuffling feet, one wide cyclops eye
          ;; and a pair of stubby horns
          (set-color 0.56 0.34 0.66)
          (part 0.0 0.32 0.0 0.27 0.27 0.25)
          (set-color 0.30 0.18 0.38)
          (part (* 0.09 sw) 0.05 -0.13 0.09 0.05 0.08)
          (part (* -0.09 sw) 0.05 0.13 0.09 0.05 0.08)
          (part 0.0 0.62 -0.14 0.035 0.06 0.035)
          (part 0.0 0.62 0.14 0.035 0.06 0.035)
          (set-color 0.97 0.95 0.90)
          (part 0.25 0.40 0.0 0.028 0.065 0.11)
          (set-color 0.12 0.10 0.10)
          (part 0.272 0.39 0.0 0.012 0.032 0.032))
        (when (> (aref *esquash* i) 0.0)
          ;; freshly stomped: a fading pancake
          (set-color 0.44 0.27 0.52)
          (part 0.0 0.05 0.0 0.30 0.05 0.28)))))

(defun emit-coin (i tm)
  (unless (aref *ctaken* i)
    (set-origin (aref *coinx* i)
                (+ (aref *coiny* i) (* 0.07 (sin (+ (* tm 2.4) (* 0.9 i)))))
                (aref *coinz* i) (* tm 3.0))
    (set-color 1.0 0.82 0.25)
    (part 0.0 0.0 0.0 0.16 0.16 0.03)
    (set-color 1.0 0.92 0.55)
    (part 0.0 0.0 0.0 0.09 0.09 0.036)))

;; --- the frame --------------------------------------------------------------------

(defun draw (tm)
  (let ((w (canvas-width)) (h (canvas-height)))
    (gl:viewport 0 0 (floor w) (floor h)))
  (gl:clear-color 0.52 0.74 0.98 1.0)
  (gl:clear (+ gl:+color-buffer-bit+ gl:+depth-buffer-bit+))
  (gl:use-program *prog*)
  (dotimes (i 16) (set-float i (aref *vp* i)))
  (gl-uniform-matrix4fv *u-vp*)
  (gl:uniform3f *u-eye* *eyex* *eyey* *eyez*)
  ;; the dynamic cast goes in the buffer right after the baked level
  (setq *v* *static-verts*)
  (when (= *state* 0) (emit-shadow))
  (emit-player tm)
  (dotimes (i *nenemy*) (emit-enemy i tm))
  (dotimes (i *ncoin*) (emit-coin i tm))
  (gl:bind-buffer gl:+array-buffer+ *buf*)
  (gl-upload-vertices (* *static-verts* 9) (* (- *v* *static-verts*) 9))
  (gl:bind-vertex-array *vao*)
  (gl:draw-arrays gl:+triangles+ 0 *v*))

(defun frame (tm)
  (when (or *pending-reset* (< *start-tm* 0.0))
    (setq *pending-reset* nil)
    (reset-game tm))
  (let ((dt (min 0.05 (max 0.0 (- tm *last-tm*)))))
    (setq *last-tm* tm)
    (setq *aspect* (/ (canvas-width) (canvas-height)))
    (setq *state-t* (+ *state-t* dt))
    (cond ((= *state* 0) (step-playing dt))
          ((= *state* 1) (step-dying dt))
          (t (step-clear dt)))
    (update-enemies dt)
    (update-camera dt)
    (draw tm)))

;; --- HUD taps ---------------------------------------------------------------------

(defun get-coins () *coins*)
(defun coin-total () *ncoin*)
(defun get-deaths () *deaths*)
(defun get-state () *state*)
;; the player's position, for tests and external instrumentation
(defun get-px () *px*)
(defun get-py () *py*)
(defun get-pz () *pz*)
(defun get-time ()
  (if (= *state* 2)
      *elapsed*
      (if (< *start-tm* 0.0) 0.0 (- *last-tm* *start-tm*))))

;; --- boot --------------------------------------------------------------------------
;; Runs inside _initialize, after the page has created the WebGL2 context:
;; build the pipeline, parse the stage and bake its mesh. The clock starts on
;; the first frame.

(setup-gl)
(parse-solids)
(parse-coins)
(parse-enemies)
(bake-static)
(respawn)

(rontolisp:wasm-export 'frame :params '(:float) :returns :void)
(rontolisp:wasm-export 'set-key
                       :as "setKey"
                       :params '(:int :int)
                       :returns :void)
(rontolisp:wasm-export 'orbit :params '(:float :float) :returns :void)
(rontolisp:wasm-export 'zoom :params '(:float) :returns :void)
(rontolisp:wasm-export 'restart :params '() :returns :void)
(rontolisp:wasm-export 'get-coins :as "getCoins" :params '() :returns :int)
(rontolisp:wasm-export 'coin-total :as "coinTotal" :params '() :returns :int)
(rontolisp:wasm-export 'get-deaths :as "getDeaths" :params '() :returns :int)
(rontolisp:wasm-export 'get-state :as "getState" :params '() :returns :int)
(rontolisp:wasm-export 'get-time :as "getTime" :params '() :returns :float)
(rontolisp:wasm-export 'get-px :as "getPx" :params '() :returns :float)
(rontolisp:wasm-export 'get-py :as "getPy" :params '() :returns :float)
(rontolisp:wasm-export 'get-pz :as "getPz" :params '() :returns :float)


---

# FILE: references/examples/browser/webgl-robot-arm/README.md

# robot-arm.lisp — 3D inverse kinematics with minimum-jerk motion, in Lisp

Click anywhere on the page: the arm reaches for that point in 3D, the
three-finger gripper opening for the flight and closing on arrival. Drag to
orbit, scroll to zoom. The controller and the renderer both run in Lisp,
compiled to WebAssembly.

**Live demo:** 

## Three IK solvers, switchable live from the HUD

- **Jacobian with damped least squares** (the default): every joint is a ball
  joint contributing three columns `a_k x (p_tip - p_i)` to the position
  Jacobian `J`, and each iteration solves `dθ = J^T (J J^T + λ²I)^-1 e` — `J J^T`
  built with `linalg:matmul`/`transpose`, the 3×3 system solved exactly by
  `linalg:solve`. The classic numerical IK of robotics, and real matrix
  computation every frame.
- **[FABRIK](http://www.andreasaristidou.com/FABRIK.html)**, the geometric
  contrast: pin the hand to the target and walk the chain back to the base, pin
  the base and walk forward, repeat — no matrices, just distance
  re-normalization.
- **The analytic closed form + forward kinematics**, the industrial-robot
  contrast: base yaw from `atan2`, the elbow from the law of cosines, exact in
  one shot. Closed forms only exist for specific structures, so the chain is
  split into two rigid groups and the extra joints ride frozen — you can *see*
  that limitation in the pose. Joint positions then come from a chain of 4×4
  homogeneous transforms combined with `linalg:matmul`.

Toggle between them mid-move and the poses differ: DLS spreads the motion
across all joints, FABRIK drags the chain geometrically, and the analytic arm
moves like a rigid two-segment machine.

## The rest of the program

- **Minimum-jerk interpolation** — the commanded hand position travels along
  `s(u) = 10u³ - 15u⁴ + 6u⁵`, the unique 5th-order polynomial with zero velocity
  and acceleration at both ends (Flash & Hogan 1985, the classic model of human
  reaching). Clicks are clamped into the reachable shell, and a click mid-flight
  restarts the profile from the currently commanded point, so the hand never
  jumps.
- **The gripper** — a palm and three two-phalanx fingers 120° apart. The solved
  chain includes one rigid "tool" link from the wrist to the grasp point, so the
  solver picks the approach direction and pins the point *between the
  fingertips* — not the wrist — to your click.
- **The renderer** — Lisp tessellates the machine every frame (tapered cylinders
  for links and fingers, spheres for joints, cones for the RGB = XYZ axis
  arrows), computes world-space normals and the click-ray unprojection, and
  draws two passes: lit triangles, then additive glow sprites that read depth
  but do not write it. `VP = P × V` is one `linalg:matmul`, transposed into
  WebGL's column-major order as it stages the 16 floats.

JavaScript is the same host boundary as [`webgl-galaxy`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-galaxy): one-line
WebGL2 bindings over a handle table, two staging `Float32Array`s, pointer
gestures forwarded as exported-function calls, and the HUD — no kinematics, no
matrices, no rendering logic of its own.

## What's in here

| File | Purpose |
| --- | --- |
| `robot-arm.lisp` | The program: trajectory, gripper, tessellator, camera, shaders |
| `ik-jacobian.lisp` | Solver 1: damped-least-squares Jacobian IK |
| `ik-fabrik.lisp` | Solver 2: FABRIK, the geometric method (no matrices) |
| `ik-analytic.lisp` | Solver 3: the closed form + matrix forward kinematics |
| `index.html` | The host page: WebGL2 bindings, pointer gestures, the HUD |
| `robot-arm.wasm` | The compiled `--no-wasi` reactor (checked in) |
| `build.sh` | Recompiles `robot-arm.lisp` |

Each solver lives in its own file, pulled in by literal top-level
`(load "ik-....lisp")` forms the compiler splices in **at compile time** (paths
resolve relative to the loading file) — the same mechanism the
[Minesweeper example](https://github.com/making/rontolisp/blob/develop/examples/browser/minesweeper) uses to share its core between the browser
and Swing builds. The WebGL2 boundary comes in the same way from the shared `gl`
package, [`../webgl-common/gl.lisp`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-common); the page's matching
bindings are generated from the same `gl.wit`.

## The controller is three small functions

```lisp
;; the minimum-jerk position profile: zero velocity/acceleration at both ends
(defun min-jerk (u)
  (* u u u (+ 10.0 (* u (+ -15.0 (* u 6.0))))))

;; each frame: advance the commanded hand position along the profile ...
(setq *tx* (+ *sx* (* s (- *gx* *sx*))))

;; ... and solve the chain onto it -- the damped-least-squares step is one
;; linalg expression: dtheta = J^T (J J^T + lambda^2 I)^-1 e
(let ((a (linalg:matmul jac (linalg:transpose jac))))
  (dotimes (k 3) (setf (aref a k k) (+ (aref a k k) +dls-lambda2+)))
  (linalg:matmul (linalg:transpose jac) (linalg:solve a e)))
```

A click arrives as clip coordinates; Lisp turns it into a ray from the eye and
intersects it with the plane through the orbit centre facing the camera, so
"click where you see" works from any viewpoint. The links selector (3-6)
re-initializes the chain, every link count tapering to the same total reach.

## Building and running

```bash
./mvnw clean package                        # from the repo root, once
examples/browser/webgl-robot-arm/build.sh   # recompile the .wasm

# the page imports ../webgl-common/gl-imports.js, so serve examples/browser:
jwebserver -p 8000 --directory "$PWD/examples/browser"
open http://localhost:8000/webgl-robot-arm/
```

Needs a browser with WebAssembly GC (Chrome 119+, Firefox 120+, Safari 18.2+).

## Notes

- `--no-wasi` means the module's *only* imports are the host functions declared
  in the program and the shared `gl` package — the import object is the whole
  embedding API. `--optimize` tree-shakes the runtime down to what is reached;
  the shipped module imports 44 functions, the most of any of these demos.
- Input is delivered through exported functions (`pointer`, `orbit`, `zoom`),
  not imports: the page classifies the gesture and pushes it in, and the camera
  itself lives in Lisp. Exports in, imports out — the module never polls the
  host.
- Functions cross the WASM boundary with at most 7 parameters, so vertex colors
  ride a latched `set-color` and tube radii a latched `set-radii`.
- The axis arrows and the pedestal never move, so they are tessellated once into
  the front of the vertex buffer at `init`.
- On the interpreter and JVM the `wasm-import` directives define stubs that
  signal, so this program is WASM-only by nature — there is no host to draw with
  elsewhere.


---

# FILE: references/examples/browser/webgl-robot-arm/build.sh

#!/usr/bin/env bash
# Recompile robot-arm.lisp to a browser-loadable WebAssembly reactor.
# --no-wasi drops all WASI imports, so the module's only imports are the host
# functions robot-arm.lisp declares with rontolisp:wasm-import ("gl", "canvas",
# "math" and "ui"); --optimize tree-shakes the runtime so only the reachable
# functions ship.
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling robot-arm.lisp -> robot-arm.wasm"
java -jar "$jar" "$here/robot-arm.lisp" -o "$here/robot-arm.wasm" --no-wasi --optimize

# The page imports the generated ../webgl-common/gl-imports.js, so the served
# root is examples/browser rather than this directory.
echo "done. Serve the examples/browser directory over http, e.g.:"
echo "  jwebserver -p 8000 --directory \"$(dirname "$here")\""
echo "then open http://localhost:8000/webgl-robot-arm/"


---

# FILE: references/examples/browser/webgl-robot-arm/ik-analytic.lisp

;;;; ik-analytic.lisp -- the analytic (closed-form) method + forward
;;;; kinematics through homogeneous transform matrices.
;;;;
;;;; The textbook 2R solution: base yaw from atan2, the elbow angle from
;;;; the law of cosines, the shoulder from atan2 -- no iteration, one
;;;; exact solution per frame. Closed forms only exist for specific
;;;; structures, so init splits the chain into two rigid groups at the
;;;; joint that best balances their lengths (*split*, *ga*, *gb*) and the
;;;; extra joints ride frozen inside them. The joint positions then come
;;;; from FORWARD kinematics: a chain of 4x4 homogeneous transforms
;;;; yaw(q0) . rotz(q1) . transx(s) [. rotz(q2) . transx(s)] combined
;;;; with linalg:matmul, each joint read off the translation column.
;;;;
;;;; Spliced into robot-arm.lisp at compile time by its literal top-level
;;;; (load "ik-analytic.lisp"); shares the joint arrays, the commanded
;;;; target and the elbow split computed there.

(defun mat-eye4 ()
  (let ((m (linalg:full '(4 4) 0.0)))
    (dotimes (k 4) (setf (aref m k k) 1.0))
    m))

(defun mat-yaw (q)
  ;; local +x -> the horizontal direction (sin q, 0, cos q); +y stays up
  (let ((m (mat-eye4)) (c (cos q)) (s (sin q)))
    (setf (aref m 0 0) s)
    (setf (aref m 2 0) c)
    (setf (aref m 0 2) (- 0.0 c))
    (setf (aref m 2 2) s)
    m))

(defun mat-rot-z (q)
  ;; rotates local +x toward +y: the shoulder / elbow pitch in the arm plane
  (let ((m (mat-eye4)) (c (cos q)) (s (sin q)))
    (setf (aref m 0 0) c)
    (setf (aref m 0 1) (- 0.0 s))
    (setf (aref m 1 0) s)
    (setf (aref m 1 1) c)
    m))

(defun mat-trans-x (d)
  (let ((m (mat-eye4)))
    (setf (aref m 0 3) d)
    m))

(defun %fk-place (m i)
  ;; joint i = the translation column of the accumulated transform
  (setf (aref *jx* i) (aref m 0 3))
  (setf (aref *jy* i) (aref m 1 3))
  (setf (aref *jz* i) (aref m 2 3)))

(defun solve-ik-analytic ()
  ;; Closed-form angles (elbow-up branch). The commanded point is clamped
  ;; into the two-segment annulus, so a target the frozen-joint structure
  ;; cannot fold to is reached as closely as the geometry allows (the
  ;; HUD's "to target" shows the residual).
  (let* ((tip (+ *links* 1))
         (q0 (atan2 *tx* *tz*))
         (r (sqrt (+ (* *tx* *tx*) (* *tz* *tz*))))
         (h *ty*)
         (d (sqrt (+ (* r r) (* h h))))
         (la *ga*)
         (lb *gb*)
         (dmin (+ (if (> la lb) (- la lb) (- lb la)) 0.001))
         (dmax (- (+ la lb) 0.001))
         (dc (cond ((< d dmin) dmin) ((> d dmax) dmax) (t d)))
         (ce (/ (- (* dc dc) (* la la) (* lb lb)) (* 2.0 la lb)))
         (q2 (- 0.0 (acos (cond ((> ce 1.0) 1.0) ((< ce -1.0) -1.0) (t ce)))))
         (q1 (- (atan2 h r) (atan2 (* lb (sin q2)) (+ la (* lb (cos q2)))))))
    ;; FORWARD kinematics: walk the transform chain and place every joint.
    (let ((t1 (linalg:matmul (mat-yaw q0) (mat-rot-z q1))) (s 0.0))
      (setf (aref *jx* 0) 0.0)
      (setf (aref *jy* 0) 0.0)
      (setf (aref *jz* 0) 0.0)
      (do ((i 1 (+ i 1)))
          ((> i *split*))
        (setq s (+ s (aref *len* (- i 1))))
        (%fk-place (linalg:matmul t1 (mat-trans-x s)) i))
      (let ((t2
             (linalg:matmul t1
                            (linalg:matmul (mat-trans-x la) (mat-rot-z q2)))))
        (setq s 0.0)
        (do ((i (+ *split* 1) (+ i 1)))
            ((> i tip))
          (setq s (+ s (aref *len* (- i 1))))
          (%fk-place (linalg:matmul t2 (mat-trans-x s)) i))))))


---

# FILE: references/examples/browser/webgl-robot-arm/ik-fabrik.lisp

;;;; ik-fabrik.lisp -- FABRIK (Forward And Backward Reaching Inverse
;;;; Kinematics, Aristidou & Lasenby 2011), the geometric method.
;;;;
;;;; No matrices at all: alternately pin the grasp point to the commanded
;;;; target and walk the chain back to the base re-normalizing each link's
;;;; length, then pin the base and walk forward. The target is always
;;;; inside the workspace (set-goal clamps it), so a few sweeps converge
;;;; far below a pixel.
;;;;
;;;; Spliced into robot-arm.lisp at compile time by its literal top-level
;;;; (load "ik-fabrik.lisp"); shares the joint arrays, the commanded
;;;; target and the `place` helper defined there.

(defun solve-ik-fabrik ()
  (let ((tip (+ *links* 1)))
    (dotimes (pass 8)
      ;; backward: grasp point -> base
      (setf (aref *jx* tip) *tx*)
      (setf (aref *jy* tip) *ty*)
      (setf (aref *jz* tip) *tz*)
      (do ((i (- tip 1) (- i 1)))
          ((< i 0))
        (place i (+ i 1) (aref *len* i)))
      ;; forward: base -> grasp point
      (setf (aref *jx* 0) 0.0)
      (setf (aref *jy* 0) 0.0)
      (setf (aref *jz* 0) 0.0)
      (dotimes (i tip) (place (+ i 1) i (aref *len* i))))))


---

# FILE: references/examples/browser/webgl-robot-arm/ik-jacobian.lisp

;;;; ik-jacobian.lisp -- the Jacobian method with damped least squares
;;;; (Wampler 1986), the numerical IK used across robotics.
;;;;
;;;; Every joint is a ball joint contributing three columns a_k x (p_tip -
;;;; p_i) to the 3 x 3(n+1) position Jacobian J, and each iteration solves
;;;; the damped normal equations dtheta = J^T (J J^T + lambda^2 I)^-1 e --
;;;; J J^T built with linalg:matmul / linalg:transpose, the 3x3 system
;;;; solved exactly by linalg:solve (Gaussian elimination) -- then applies
;;;; the rotations linearized and re-normalizes the link lengths.
;;;;
;;;; Spliced into robot-arm.lisp at compile time by its literal top-level
;;;; (load "ik-jacobian.lisp"); shares the joint arrays, the commanded
;;;; target and the `place` helper defined there.

(defconstant +dls-lambda2+ 0.05) ; damping^2: keeps J J^T well-conditioned
(defconstant +dls-max-err+ 0.35) ; clamp per-iteration error (stabilizes)

(defun solve-ik-jacobian ()
  ;; Damped-least-squares Jacobian IK. Each iteration: build the position
  ;; Jacobian about the current pose, solve the damped normal equations for
  ;; the joint rotations that move the grasp point toward the target, apply
  ;; them (linearized), and re-normalize the link lengths.
  (let* ((tip (+ *links* 1)) (m (* 3 tip))) ; 3 rotation DOFs per joint 0..*links*
    (dotimes (iter 10)
      (let ((ex (- *tx* (aref *jx* tip)))
            (ey (- *ty* (aref *jy* tip)))
            (ez (- *tz* (aref *jz* tip))))
        ;; clamp the error so early iterations stay in the linear regime
        (let ((d (sqrt (+ (* ex ex) (* ey ey) (* ez ez)))))
          (when (> d +dls-max-err+)
            (let ((r (/ +dls-max-err+ d)))
              (setq ex (* ex r))
              (setq ey (* ey r))
              (setq ez (* ez r)))))
        ;; J: column 3i+k is e_k x (p_tip - p_i), the tip velocity from
        ;; spinning joint i about the world axis e_k
        (let ((jac (make-array (list 3 m) :initial-element 0.0))
              (e (make-array 3 :initial-element 0.0)))
          (dotimes (i tip)
            (let ((px (- (aref *jx* tip) (aref *jx* i)))
                  (py (- (aref *jy* tip) (aref *jy* i)))
                  (pz (- (aref *jz* tip) (aref *jz* i)))
                  (c (* 3 i)))
              (setf (aref jac 1 c) (- 0.0 pz)) ; e_x x p
              (setf (aref jac 2 c) py)
              (setf (aref jac 0 (+ c 1)) pz) ; e_y x p
              (setf (aref jac 2 (+ c 1)) (- 0.0 px))
              (setf (aref jac 0 (+ c 2)) (- 0.0 py)) ; e_z x p
              (setf (aref jac 1 (+ c 2)) px)))
          (setf (aref e 0) ex)
          (setf (aref e 1) ey)
          (setf (aref e 2) ez)
          ;; dtheta = J^T (J J^T + lambda^2 I)^-1 e  -- the 3x3 damped
          ;; normal equations, solved exactly by linalg
          (let ((a (linalg:matmul jac (linalg:transpose jac))))
            (dotimes (k 3) (setf (aref a k k) (+ (aref a k k) +dls-lambda2+)))
            (let ((dth
                   (linalg:matmul (linalg:transpose jac) (linalg:solve a e))))
              ;; apply: joint i's rotation w moves every downstream joint
              ;; j by w x (p_j - p_i), the linearized rotation
              (dotimes (i tip)
                (let ((wx (aref dth (* 3 i)))
                      (wy (aref dth (+ (* 3 i) 1)))
                      (wz (aref dth (+ (* 3 i) 2))))
                  (do ((jj (+ i 1) (+ jj 1)))
                      ((> jj tip))
                    (let ((qx (- (aref *jx* jj) (aref *jx* i)))
                          (qy (- (aref *jy* jj) (aref *jy* i)))
                          (qz (- (aref *jz* jj) (aref *jz* i))))
                      (setf (aref *jx* jj)
                            (+ (aref *jx* jj) (- (* wy qz) (* wz qy))))
                      (setf (aref *jy* jj)
                            (+ (aref *jy* jj) (- (* wz qx) (* wx qz))))
                      (setf (aref *jz* jj)
                            (+ (aref *jz* jj) (- (* wx qy) (* wy qx))))))))))
          ;; linearized rotations stretch the links; re-normalize them
          (dotimes (i tip) (place (+ i 1) i (aref *len* i))))))))


---

# FILE: references/examples/browser/webgl-robot-arm/index.html



  
    
    
    robot-arm.lisp — 3D inverse kinematics with minimum-jerk motion, in Lisp
    
    
    
    
  
  
    

    

      
robot-arm.lisp → WebGL

      

        Click to reach, drag to orbit, scroll to zoom. Lisp unprojects
        the click into 3D, glides the hand there along a minimum-jerk
        trajectory (the classic model of smooth human reaching), solves the
        joints each frame with damped-least-squares Jacobian IK
        (linalg:matmul / solve; or FABRIK / analytic 2R + matrix forward
        kinematics — toggle below right) — gripper
        open in flight, closing on the target on arrival — and tessellates
        every cylinder and sphere, camera matrices included. JavaScript is
        one-line WebGL bindings, pointer gestures and this HUD.
      

    


    

    

    

    
  



---

# FILE: references/examples/browser/webgl-robot-arm/robot-arm.lisp

;;;; robot-arm.lisp -- a 3-D robot arm that reaches for wherever you click,
;;;; solved by FABRIK inverse kinematics and animated along a minimum-jerk
;;;; trajectory, entirely in Lisp.
;;;;
;;;; Drag to orbit the camera, scroll to zoom, click (or tap) to set the
;;;; target: the page forwards pointer gestures through the exported
;;;; `pointer` / `orbit` / `zoom` functions. Every frame Lisp advances the
;;;; commanded hand position along the minimum-jerk profile
;;;; 10u^3 - 15u^4 + 6u^5 (zero velocity and acceleration at both ends --
;;;; the classic model of smooth human reaching motion), solves the joint
;;;; positions with damped-least-squares Jacobian iterations (or 3-D FABRIK
;;;; sweeps, or the closed-form 2R solution -- the HUD toggles the solver;
;;;; one file per solver, spliced in by compile-time load), and renders
;;;; the arm as lit
;;;; cylinders and spheres it tessellates itself -- plus the RGB = XYZ axis
;;;; arrows at the origin and a glowing sprite pass for the target ring and
;;;; the hand's trail. The camera matrices are rank-2 (4 4) arrays combined
;;;; with linalg:matmul; the click-ray unprojection and the lighting setup
;;;; are computed here too. JavaScript is the same one-line WebGL2 host
;;;; boundary as webgl-galaxy and webgl-heat3d, plus pointer-gesture
;;;; forwarding and the HUD.
;;;;
;;;; Compiled ahead of time to a --no-wasi reactor (build.sh), so the module
;;;; imports nothing but the host functions declared below and instantiates
;;;; in any wasm-GC-capable browser.

;; --- the host boundary ------------------------------------------------------
;;
;; The WebGL2 API itself -- the wasm-import directives, the enum constants and
;; the shader helpers -- lives in the shared gl package
;; (../webgl-common/gl.lisp), spliced in here at compile time; --optimize
;; drops the entries this demo never calls. Only the imports specific to this
;; page stay below. GL objects cross as :int handles into a table the page
;; keeps; strings (GLSL source, info logs) cross as :string.

(require :gl "../webgl-common/gl.lisp")

;; The bulk-float staging path (see webgl-galaxy / webgl-cube): per-vertex
;; floats cannot cross into GPU memory one WASM value at a time, so the page
;; keeps two Float32Arrays -- one for the lit-triangle vertices (position,
;; normal, color = 9 floats), one for the glow sprites (position, tone,
;; size = 5 floats) -- plus a 16-float scratch for mat4 uniforms. Colors are
;; constant per mesh, so set-color latches the current color and set-vertex
;; stages position + normal + that color (functions cross the WASM boundary
;; with at most 7 parameters).
(rontolisp:wasm-import 'set-color
                       :from "gl"
                       :as "setColor"
                       :params '(:float :float :float)
                       :returns :void)
(rontolisp:wasm-import 'set-vertex
                       :from "gl"
                       :as "setVertex"
                       :params '(:int :float :float :float :float :float :float)
                       :returns :void)
(rontolisp:wasm-import 'set-sprite
                       :from "gl"
                       :as "setSprite"
                       :params '(:int :float :float :float :float :float)
                       :returns :void)
(rontolisp:wasm-import 'gl-upload-vertices
                       :from "gl"
                       :as "uploadVertices"
                       :params '(:int :int)
                       :returns :void)
(rontolisp:wasm-import 'gl-upload-sprites
                       :from "gl"
                       :as "uploadSprites"
                       :params '(:int)
                       :returns :void)
(rontolisp:wasm-import 'set-float
                       :from "gl"
                       :as "setFloat"
                       :params '(:int :float)
                       :returns :void)
(rontolisp:wasm-import 'gl-uniform-matrix4fv
                       :from "gl"
                       :as "uniformMatrix4fv"
                       :params '(:int)
                       :returns :void)

;; Canvas metrics, owned by the page.
(rontolisp:wasm-import 'canvas-width
                       :from "canvas"
                       :as "width"
                       :params '()
                       :returns :float)
(rontolisp:wasm-import 'canvas-height
                       :from "canvas"
                       :as "height"
                       :params '()
                       :returns :float)
(rontolisp:wasm-import 'device-pixel-ratio
                       :from "canvas"
                       :as "devicePixelRatio"
                       :params '()
                       :returns :float)

;; The WASM backend has no transcendental built-ins, so borrow the host's.
(rontolisp:wasm-import 'sin :from "math" :params '(:float) :returns :float)
(rontolisp:wasm-import 'cos :from "math" :params '(:float) :returns :float)
(rontolisp:wasm-import 'atan2
                       :from "math"
                       :params '(:float :float)
                       :returns :float)
(rontolisp:wasm-import 'acos :from "math" :params '(:float) :returns :float)

(defconstant +pi+ 3.141592653589793)
(defconstant +two-pi+ 6.283185307179586)

;; --- shaders ----------------------------------------------------------------
;;
;; Two programs: lit triangles for the solid machine (arm, joints, pedestal,
;; axis arrows) and additive point sprites for the glow (target ring, trail).

(defconstant +solid-vs+
  "#version 300 es
layout(location=0) in vec3 aPos;     // world-space position from Lisp
layout(location=1) in vec3 aNormal;  // world-space normal from Lisp
layout(location=2) in vec3 aColor;
uniform mat4 uVP;                    // view-projection, computed in Lisp
out vec3 vN;
out vec3 vC;
out vec3 vW;
void main() {
  gl_Position = uVP * vec4(aPos, 1.0);
  vN = aNormal;
  vC = aColor;
  vW = aPos;
}")

(defconstant +solid-fs+
  "#version 300 es
precision mediump float;
in vec3 vN;
in vec3 vC;
in vec3 vW;
uniform vec3 uEye;
out vec4 color;
void main() {
  // one key light + a hemisphere ambient + blinn-phong spec + a cool rim
  vec3 n = normalize(vN);
  vec3 l = normalize(vec3(0.5, 0.85, 0.35));
  vec3 e = normalize(uEye - vW);
  vec3 h = normalize(l + e);
  float diff = max(dot(n, l), 0.0);
  float amb  = 0.30 + 0.14 * n.y;
  float spec = pow(max(dot(n, h), 0.0), 48.0) * 0.45;
  float rim  = pow(1.0 - max(dot(n, e), 0.0), 3.0) * 0.18;
  color = vec4(vC * (amb + 0.75 * diff) + vec3(spec) + vec3(0.35, 0.55, 0.9) * rim, 1.0);
}")

(defconstant +sprite-vs+
  "#version 300 es
layout(location=0) in vec3 aPos;    // world-space position from Lisp
layout(location=1) in float aTone;  // 0..1 palette position from Lisp
layout(location=2) in float aSize;
uniform mat4 uVP;
uniform float uDpr;
out float vTone;
void main() {
  vec4 p = uVP * vec4(aPos, 1.0);
  gl_Position = p;
  gl_PointSize = aSize * uDpr / max(p.w, 0.1);  // perspective-sized sprites
  vTone = aTone;
}")

(defconstant +sprite-fs+
  "#version 300 es
precision mediump float;
in float vTone;
out vec4 color;
// tone ~0.5 -> trail cyan, ~0.7 -> ember, 1 -> white-hot target.
vec3 tint(float h) {
  vec3 steel = vec3(0.30, 0.44, 0.85);
  vec3 cyan  = vec3(0.30, 0.95, 0.85);
  vec3 ember = vec3(1.00, 0.55, 0.18);
  vec3 hot   = vec3(1.00, 0.97, 0.90);
  return h < 0.45 ? mix(steel, cyan, h / 0.45)
       : h < 0.80 ? mix(cyan, ember, (h - 0.45) / 0.35)
       :            mix(ember, hot, (h - 0.80) / 0.20);
}
void main() {
  // a soft round sprite under additive blending
  float d = length(gl_PointCoord - 0.5) * 2.0;
  float a = exp(-3.0 * d * d) * (1.0 - smoothstep(0.8, 1.0, d));
  float glow = 0.22 + 0.78 * vTone;
  color = vec4(tint(vTone) * a * glow, a * glow);
}")

;; --- 4x4 matrix math: rank-2 arrays + the linalg package ----------------------
;;
;; The matrices are ordinary rank-2 (4 4) arrays in textbook (row, col)
;; convention, and VP = P x V is one linalg:matmul; upload-vp transposes
;; into the column-major order WebGL expects as it stages the floats.

(defconstant +half-fov+ 0.39269908169872414) ; pi/8: half the 45-degree fov

(defun mat4-perspective (aspect near far)
  (let* ((f (/ (cos +half-fov+) (sin +half-fov+)))
         (nf (/ 1.0 (- near far)))
         (m (linalg:full '(4 4) 0.0)))
    (setf (aref m 0 0) (/ f aspect))
    (setf (aref m 1 1) f)
    (setf (aref m 2 2) (* (+ far near) nf))
    (setf (aref m 2 3) (* 2.0 far near nf))
    (setf (aref m 3 2) -1.0)
    m))

;; --- the orbit camera ---------------------------------------------------------
;;
;; The eye circles the point +ctr+ at *radius*; drag gestures arrive through
;; the exported `orbit` (yaw/pitch deltas) and `zoom`. The camera basis
;; (*rx*.. right, *ux*.. up, *fx*.. forward) doubles as the click-ray
;; unprojector and the billboard frame for the target ring.

(defconstant +ctr-x+ 0.0)
(defconstant +ctr-y+ 0.42)
(defconstant +ctr-z+ 0.0)

(defvar *yaw* 0.65)
(defvar *pitch* 0.34)
(defvar *radius* 2.9)
(defvar *aspect* 1.0)
(defvar *ex* 0.0)
(defvar *ey* 0.0)
(defvar *ez* 0.0)
(defvar *fx* 0.0) ; forward (eye -> centre)
(defvar *fy* 0.0)
(defvar *fz* -1.0)
(defvar *rx* 1.0) ; right
(defvar *ry* 0.0)
(defvar *rz* 0.0)
(defvar *ux* 0.0) ; up
(defvar *uy* 1.0)
(defvar *uz* 0.0)
(defvar *vp* nil) ; the current view-projection matrix

(defun orbit (dx dy)
  ;; Exported: drag deltas, normalized by the canvas height.
  (setq *yaw* (- *yaw* (* 3.4 dx)))
  (let ((p (+ *pitch* (* 2.6 dy))))
    (setq *pitch* (cond ((< p -0.45) -0.45) ((> p 1.45) 1.45) (t p)))))

(defun zoom (dz)
  ;; Exported: scroll-wheel deltas.
  (let ((r (+ *radius* dz)))
    (setq *radius* (cond ((< r 1.6) 1.6) ((> r 5.5) 5.5) (t r)))))

(defun update-camera ()
  (let ((cp (cos *pitch*)) (sp (sin *pitch*)))
    (setq *ex* (+ +ctr-x+ (* *radius* cp (sin *yaw*))))
    (setq *ey* (+ +ctr-y+ (* *radius* sp)))
    (setq *ez* (+ +ctr-z+ (* *radius* cp (cos *yaw*)))))
  ;; forward = normalize(centre - eye)
  (let* ((fx (- +ctr-x+ *ex*))
         (fy (- +ctr-y+ *ey*))
         (fz (- +ctr-z+ *ez*))
         (fl (sqrt (+ (* fx fx) (* fy fy) (* fz fz)))))
    (setq *fx* (/ fx fl))
    (setq *fy* (/ fy fl))
    (setq *fz* (/ fz fl)))
  ;; right = normalize(cross(forward, world-up)); up = cross(right, forward)
  (let* ((rx (- 0.0 *fz*)) (rz *fx*) (rl (sqrt (+ (* rx rx) (* rz rz)))))
    (setq *rx* (/ rx rl))
    (setq *ry* 0.0)
    (setq *rz* (/ rz rl)))
  (setq *ux* (- (* *ry* *fz*) (* *rz* *fy*)))
  (setq *uy* (- (* *rz* *fx*) (* *rx* *fz*)))
  (setq *uz* (- (* *rx* *fy*) (* *ry* *fx*)))
  ;; the look-at view matrix (rows = camera right / up / -forward),
  ;; then VP = P x V as a rank-2 matrix product
  (let ((v (linalg:full '(4 4) 0.0)))
    (setf (aref v 0 0) *rx*)
    (setf (aref v 0 1) *ry*)
    (setf (aref v 0 2) *rz*)
    (setf (aref v 0 3) (- 0.0 (+ (* *rx* *ex*) (* *ry* *ey*) (* *rz* *ez*))))
    (setf (aref v 1 0) *ux*)
    (setf (aref v 1 1) *uy*)
    (setf (aref v 1 2) *uz*)
    (setf (aref v 1 3) (- 0.0 (+ (* *ux* *ex*) (* *uy* *ey*) (* *uz* *ez*))))
    (setf (aref v 2 0) (- 0.0 *fx*))
    (setf (aref v 2 1) (- 0.0 *fy*))
    (setf (aref v 2 2) (- 0.0 *fz*))
    (setf (aref v 2 3) (+ (* *fx* *ex*) (* *fy* *ey*) (* *fz* *ez*)))
    (setf (aref v 3 3) 1.0)
    (setq *vp* (linalg:matmul (mat4-perspective *aspect* 0.1 20.0) v))))

;; --- GL pipeline setup ------------------------------------------------------

(defvar *prog-solid* 0)
(defvar *prog-sprite* 0)
(defvar *vao-solid* 0)
(defvar *vao-sprite* 0)
(defvar *buf-solid* 0)
(defvar *buf-sprite* 0)
(defvar *u-vp-solid* 0)
(defvar *u-eye* 0)
(defvar *u-vp-sprite* 0)
(defvar *u-dpr* 0)

(defconstant +max-verts+ 8192)   ; lit-triangle vertex capacity
(defconstant +max-sprites+ 2048) ; glow sprite capacity

(defun setup-gl ()
  (setq *prog-solid* (gl:build-program +solid-vs+ +solid-fs+))
  (setq *u-vp-solid* (gl:get-uniform-location *prog-solid* "uVP"))
  (setq *u-eye* (gl:get-uniform-location *prog-solid* "uEye"))
  (setq *prog-sprite* (gl:build-program +sprite-vs+ +sprite-fs+))
  (setq *u-vp-sprite* (gl:get-uniform-location *prog-sprite* "uVP"))
  (setq *u-dpr* (gl:get-uniform-location *prog-sprite* "uDpr"))
  (gl:enable gl:+depth-test+)
  ;; solid VAO: position + normal + color, 36 bytes per vertex
  (setq *vao-solid* (gl:create-vertex-array))
  (gl:bind-vertex-array *vao-solid*)
  (setq *buf-solid* (gl:create-buffer))
  (gl:bind-buffer gl:+array-buffer+ *buf-solid*)
  (gl:buffer-data gl:+array-buffer+ (* +max-verts+ 36) gl:+dynamic-draw+)
  (gl:enable-vertex-attrib-array 0)
  (gl:vertex-attrib-pointer 0 3 gl:+float+ nil 36 0)
  (gl:enable-vertex-attrib-array 1)
  (gl:vertex-attrib-pointer 1 3 gl:+float+ nil 36 12)
  (gl:enable-vertex-attrib-array 2)
  (gl:vertex-attrib-pointer 2 3 gl:+float+ nil 36 24)
  ;; sprite VAO: position + tone + size, 20 bytes per sprite
  (setq *vao-sprite* (gl:create-vertex-array))
  (gl:bind-vertex-array *vao-sprite*)
  (setq *buf-sprite* (gl:create-buffer))
  (gl:bind-buffer gl:+array-buffer+ *buf-sprite*)
  (gl:buffer-data gl:+array-buffer+ (* +max-sprites+ 20) gl:+dynamic-draw+)
  (gl:enable-vertex-attrib-array 0)
  (gl:vertex-attrib-pointer 0 3 gl:+float+ nil 20 0)
  (gl:enable-vertex-attrib-array 1)
  (gl:vertex-attrib-pointer 1 1 gl:+float+ nil 20 12)
  (gl:enable-vertex-attrib-array 2)
  (gl:vertex-attrib-pointer 2 1 gl:+float+ nil 20 16)
  (gl:blend-func gl:+one+ gl:+one+))

;; --- mesh emitters ------------------------------------------------------------
;;
;; The machine is tessellated in Lisp every frame: tapered tubes (cylinders
;; and arrow cones), discs and spheres, all as world-space triangles with
;; per-vertex normals. A 12-slot sin/cos table drives every ring; the unit
;; sphere is precomputed once into a rank-2 (verts 3) array.

(defconstant +seg+ 12) ; ring segments
(defvar *ctab* nil)    ; cos table, +seg+ entries
(defvar *stab* nil)

(defconstant +sph-stacks+ 5)
(defconstant +sph-slices+ 8)
(defconstant +sph-verts+ 240) ; +sph-stacks+ * +sph-slices+ * 6
(defvar *sph* nil)            ; rank-2 (vertex . xyz) unit sphere

(defvar *v* 0)            ; solid vertex write cursor
(defvar *s* 0)            ; sprite write cursor
(defvar *static-verts* 0) ; axes + pedestal, uploaded once

(defun emit-v (x y z nx ny nz)
  ;; stages one vertex with the color latched by the last set-color call
  (set-vertex *v* x y z nx ny nz)
  (setq *v* (+ *v* 1)))

(defun emit-s (x y z tone size)
  (set-sprite *s* x y z tone size)
  (setq *s* (+ *s* 1)))

;; A pair of unit vectors perpendicular to the given axis, into *pu*/*pv*.
(defvar *pux* 0.0)
(defvar *puy* 0.0)
(defvar *puz* 0.0)
(defvar *pvx* 0.0)
(defvar *pvy* 0.0)
(defvar *pvz* 0.0)

(defun perp-basis (tx ty tz)
  (let* ((ay (if (< ty 0.0) (- 0.0 ty) ty))
         ;; helper axis least aligned with t: world-up unless t is vertical
         (hx (if (< ay 0.9) 0.0 1.0))
         (hy (if (< ay 0.9) 1.0 0.0))
         ;; u = normalize(cross(t, h)) with h = (hx hy 0)
         (cx (- 0.0 (* tz hy)))
         (cy (* tz hx))
         (cz (- (* tx hy) (* ty hx)))
         (cl (sqrt (+ (* cx cx) (* cy cy) (* cz cz)))))
    (setq *pux* (/ cx cl))
    (setq *puy* (/ cy cl))
    (setq *puz* (/ cz cl))
    (setq *pvx* (- (* ty *puz*) (* tz *puy*)))
    (setq *pvy* (- (* tz *pux*) (* tx *puz*)))
    (setq *pvz* (- (* tx *puy*) (* ty *pux*)))))

;; Tube radii travel through globals (set-radii) rather than parameters,
;; keeping every function within the WASM backend's 7-parameter limit.
(defvar *r-a* 0.0) ; tube start / disc radius
(defvar *r-b* 0.0) ; tube end radius (0 = cone)

(defun set-radii (a b)
  (setq *r-a* a)
  (setq *r-b* b))

(defun emit-tube (x0 y0 z0 x1 y1 z1)
  ;; A tube from p0 (radius *r-a*) to p1 (radius *r-b*); *r-b* = 0 makes a
  ;; cone. Normals get the proper slant for the taper.
  (let* ((r0 *r-a*)
         (r1 *r-b*)
         (ax (- x1 x0))
         (ay (- y1 y0))
         (az (- z1 z0))
         (len (sqrt (+ (* ax ax) (* ay ay) (* az az))))
         (l (if (< len 0.000001) 0.000001 len))
         (tx (/ ax l))
         (ty (/ ay l))
         (tz (/ az l))
         (k (/ (- r0 r1) l)) ; radius change per unit length
         (nf (/ 1.0 (sqrt (+ 1.0 (* k k))))))
    (perp-basis tx ty tz)
    (let ((ux *pux*) (uy *puy*) (uz *puz*) (vx *pvx*) (vy *pvy*) (vz *pvz*))
      (dotimes (seg +seg+)
        (let* ((s2 (mod (+ seg 1) +seg+))
               (ca (aref *ctab* seg))
               (sa (aref *stab* seg))
               (cb2 (aref *ctab* s2))
               (sb2 (aref *stab* s2))
               ;; the two radial unit directions of this quad
               (dax (+ (* ca ux) (* sa vx)))
               (day (+ (* ca uy) (* sa vy)))
               (daz (+ (* ca uz) (* sa vz)))
               (dbx (+ (* cb2 ux) (* sb2 vx)))
               (dby (+ (* cb2 uy) (* sb2 vy)))
               (dbz (+ (* cb2 uz) (* sb2 vz)))
               ;; slanted normals: radial + k * axis, normalized
               (nax (* nf (+ dax (* k tx))))
               (nay (* nf (+ day (* k ty))))
               (naz (* nf (+ daz (* k tz))))
               (nbx (* nf (+ dbx (* k tx))))
               (nby (* nf (+ dby (* k ty))))
               (nbz (* nf (+ dbz (* k tz)))))
          (emit-v (+ x0 (* r0 dax)) (+ y0 (* r0 day)) (+ z0 (* r0 daz)) nax nay
                  naz)
          (emit-v (+ x0 (* r0 dbx)) (+ y0 (* r0 dby)) (+ z0 (* r0 dbz)) nbx nby
                  nbz)
          (emit-v (+ x1 (* r1 dbx)) (+ y1 (* r1 dby)) (+ z1 (* r1 dbz)) nbx nby
                  nbz)
          (emit-v (+ x0 (* r0 dax)) (+ y0 (* r0 day)) (+ z0 (* r0 daz)) nax nay
                  naz)
          (emit-v (+ x1 (* r1 dbx)) (+ y1 (* r1 dby)) (+ z1 (* r1 dbz)) nbx nby
                  nbz)
          (emit-v (+ x1 (* r1 dax)) (+ y1 (* r1 day)) (+ z1 (* r1 daz)) nax nay
                  naz))))))

(defun emit-disc (cx cy cz nx ny nz)
  ;; A filled circle of radius *r-a* at c with the given face normal.
  (perp-basis nx ny nz)
  (let ((r *r-a*)
        (ux *pux*)
        (uy *puy*)
        (uz *puz*)
        (vx *pvx*)
        (vy *pvy*)
        (vz *pvz*))
    (dotimes (seg +seg+)
      (let* ((s2 (mod (+ seg 1) +seg+))
             (ca (aref *ctab* seg))
             (sa (aref *stab* seg))
             (cb2 (aref *ctab* s2))
             (sb2 (aref *stab* s2)))
        (emit-v cx cy cz nx ny nz)
        (emit-v (+ cx (* r (+ (* ca ux) (* sa vx))))
                (+ cy (* r (+ (* ca uy) (* sa vy))))
                (+ cz (* r (+ (* ca uz) (* sa vz)))) nx ny nz)
        (emit-v (+ cx (* r (+ (* cb2 ux) (* sb2 vx))))
                (+ cy (* r (+ (* cb2 uy) (* sb2 vy))))
                (+ cz (* r (+ (* cb2 uz) (* sb2 vz)))) nx ny nz)))))

(defun emit-sphere (cx cy cz r)
  (dotimes (i +sph-verts+)
    (let ((nx (aref *sph* i 0)) (ny (aref *sph* i 1)) (nz (aref *sph* i 2)))
      (emit-v (+ cx (* r nx)) (+ cy (* r ny)) (+ cz (* r nz)) nx ny nz))))

(defun %sph-write (w b s)
  (let* ((th (/ (* +pi+ b) +sph-stacks+))
         (ph (/ (* +two-pi+ s) +sph-slices+))
         (sth (sin th)))
    (setf (aref *sph* w 0) (* sth (cos ph)))
    (setf (aref *sph* w 1) (cos th))
    (setf (aref *sph* w 2) (* sth (sin ph)))))

(defun build-tables ()
  (setq *ctab* (make-array +seg+ :initial-element 0.0))
  (setq *stab* (make-array +seg+ :initial-element 0.0))
  (dotimes (i +seg+)
    (let ((a (/ (* +two-pi+ i) +seg+)))
      (setf (aref *ctab* i) (cos a))
      (setf (aref *stab* i) (sin a))))
  ;; the unit sphere as a triangle list (quads; the pole slivers degenerate)
  (setq *sph* (make-array (list +sph-verts+ 3) :initial-element 0.0))
  (let ((w 0))
    (dotimes (b +sph-stacks+)
      (dotimes (s +sph-slices+)
        (%sph-write w b s)
        (%sph-write (+ w 1) (+ b 1) s)
        (%sph-write (+ w 2) (+ b 1) (+ s 1))
        (%sph-write (+ w 3) b s)
        (%sph-write (+ w 4) (+ b 1) (+ s 1))
        (%sph-write (+ w 5) b (+ s 1))
        (setq w (+ w 6))))))

;; --- the static scene: XYZ axis arrows + the pedestal -------------------------

(defun emit-arrow (dx dy dz cr cg cb)
  ;; A unit-axis arrow from the origin: shaft, cone base cap, cone tip.
  (set-color cr cg cb)
  (let ((s 0.52)    ; shaft length
        (tip 0.15)) ; cone length
    (set-radii 0.011 0.011)
    (emit-tube 0.0 0.0 0.0 (* dx s) (* dy s) (* dz s))
    (set-radii 0.034 0.0)
    (emit-disc (* dx s) (* dy s) (* dz s) (- 0.0 dx) (- 0.0 dy) (- 0.0 dz))
    (emit-tube (* dx s) (* dy s) (* dz s) (* dx (+ s tip)) (* dy (+ s tip))
               (* dz (+ s tip)))))

(defun emit-static-scene ()
  ;; the pedestal column under the base joint
  (set-color 0.20 0.23 0.31)
  (set-radii 0.13 0.095)
  (emit-tube 0.0 -0.30 0.0 0.0 -0.02 0.0)
  (set-color 0.16 0.18 0.25)
  (emit-disc 0.0 -0.30 0.0 0.0 -1.0 0.0)
  ;; the RGB = XYZ axis arrows at the origin
  (emit-arrow 1.0 0.0 0.0 0.90 0.25 0.31)  ; +X red
  (emit-arrow 0.0 1.0 0.0 0.30 0.82 0.42)  ; +Y green
  (emit-arrow 0.0 0.0 1.0 0.32 0.53 0.96)) ; +Z blue

;; --- the arm ------------------------------------------------------------------
;;
;; The base joint sits at the origin (where the axis arrows are). The chain
;; is *links* links whose lengths taper geometrically and always sum to
;; +reach+, so every link count has the same workspace.

(defconstant +reach+ 1.25)
(defconstant +tool+ 0.115) ; wrist -> grasp point (the gripper)
(defconstant +trail+ 90)

;; The gripper rides the chain as one extra "tool" link: FABRIK solves the
;; wrist AND the grasp point (the TCP, joint *links* + 1), so the fingers
;; close exactly on the clicked goal and the approach direction comes out
;; of the solver for free.
(defvar *links* 0)
(defvar *len* nil) ; rank-1 array: link lengths + the tool
(defvar *jx* nil)  ; joint positions, *links* + 2 entries;
(defvar *jy* nil)  ; joint 0 is the base, joint *links*
(defvar *jz* nil)  ; the wrist, joint *links*+1 the TCP

;; The minimum-jerk trajectory state: the commanded hand position
;; (*tx* *ty* *tz*) travels from (*sx* ..) to the goal (*gx* ..) over *dur*
;; seconds starting at *t0*.
(defvar *sx* 0.0)
(defvar *sy* 0.0)
(defvar *sz* 0.0)
(defvar *gx* 0.0)
(defvar *gy* 0.0)
(defvar *gz* 0.0)
(defvar *tx* 0.0)
(defvar *ty* 0.0)
(defvar *tz* 0.0)
(defvar *t0* -10.0)
(defvar *dur* 1.0)

;; The latest click, in clip coordinates; picked up by the next frame (which
;; knows the camera and the time).
(defvar *click-x* 0.0)
(defvar *click-y* 0.0)
(defvar *clicked* nil)

;; The hand's recent path, a ring buffer drawn as a fading trail.
(defvar *trail-x* nil)
(defvar *trail-y* nil)
(defvar *trail-z* nil)
(defvar *trail-head* 0)
(defvar *trail-count* 0)

(defun init (n)
  (build-tables)
  (setq *links* n)
  (setq *len* (make-array (+ n 1) :initial-element 0.0))
  (setq *jx* (make-array (+ n 2) :initial-element 0.0))
  (setq *jy* (make-array (+ n 2) :initial-element 0.0))
  (setq *jz* (make-array (+ n 2) :initial-element 0.0))
  ;; each link is 0.82x the previous, normalized to the fixed total reach;
  ;; the rigid tool link (wrist -> grasp point) rides at the end
  (let ((sum 0.0) (l 1.0))
    (dotimes (i n)
      (setf (aref *len* i) l)
      (setq sum (+ sum l))
      (setq l (* l 0.82)))
    (dotimes (i n) (setf (aref *len* i) (* (aref *len* i) (/ +reach+ sum)))))
  (setf (aref *len* n) +tool+)
  ;; the analytic solver's elbow: split the chain (tool included) into two
  ;; rigid groups at the joint that best balances their lengths
  (let ((total 0.0) (best 1) (bestd 999.0) (sa 0.0))
    (dotimes (i (+ n 1)) (setq total (+ total (aref *len* i))))
    (do ((k 1 (+ k 1)))
        ((> k (- n 1)))
      (setq sa (+ sa (aref *len* (- k 1))))
      (let ((dd (- (* 2.0 sa) total)))
        (when (< dd 0.0) (setq dd (- 0.0 dd)))
        (when (< dd bestd)
          (setq bestd dd)
          (setq best k))))
    (setq *split* best)
    (setq *ga* 0.0)
    (dotimes (i best) (setq *ga* (+ *ga* (aref *len* i))))
    (setq *gb* (- total *ga*)))
  ;; initial pose: straight up, grasp point at full extension
  (dotimes (i (+ n 1))
    (setf (aref *jy* (+ i 1)) (+ (aref *jy* i) (aref *len* i))))
  ;; idle until the first click: the trajectory is already at its goal
  (setq *tx* 0.0)
  (setq *ty* (aref *jy* (+ n 1)))
  (setq *tz* 0.0)
  (setq *sx* *tx*)
  (setq *sy* *ty*)
  (setq *sz* *tz*)
  (setq *gx* *tx*)
  (setq *gy* *ty*)
  (setq *gz* *tz*)
  (setq *t0* -10.0)
  (setq *dur* 1.0)
  (setq *trail-x* (make-array +trail+ :initial-element 0.0))
  (setq *trail-y* (make-array +trail+ :initial-element 0.0))
  (setq *trail-z* (make-array +trail+ :initial-element 0.0))
  (setq *trail-head* 0)
  (setq *trail-count* 0)
  ;; bake the axes and the pedestal into the front of the vertex buffer
  (setq *v* 0)
  (emit-static-scene)
  (setq *static-verts* *v*)
  (gl:bind-buffer gl:+array-buffer+ *buf-solid*)
  (gl-upload-vertices 0 (* *static-verts* 9)))

(defun pointer (cx cy)
  ;; Exported: the page calls this on every click/tap with clip-space coords.
  (setq *click-x* cx)
  (setq *click-y* cy)
  (setq *clicked* t))

;; --- the minimum-jerk trajectory ----------------------------------------------

(defun min-jerk (u)
  ;; The minimum-jerk position profile s(u) = 10u^3 - 15u^4 + 6u^5 for
  ;; u in [0, 1]: the unique 5th-order polynomial with zero velocity and
  ;; zero acceleration at both ends, i.e. the trajectory minimizing the
  ;; integral of squared jerk (Flash & Hogan 1985).
  (* u u u (+ 10.0 (* u (+ -15.0 (* u 6.0))))))

(defun set-goal (mx my mz tm)
  ;; Clamp the requested point into the sphere shell the arm can reach (and
  ;; above the pedestal), then restart the min-jerk clock. A click mid-flight
  ;; restarts the profile from the currently commanded point, so the hand
  ;; never jumps.
  (when (< my 0.05) (setq my 0.05))
  (let* ((d (sqrt (+ (* mx mx) (* my my) (* mz mz))))
         (lo (* 0.18 +reach+))
         (hi (* 0.985 (+ +reach+ +tool+))))
    (when (< d 0.0001)
      (setq my 1.0)
      (setq d 1.0))
    (let ((r (cond ((< d lo) (/ lo d)) ((> d hi) (/ hi d)) (t 1.0))))
      (setq *sx* *tx*)
      (setq *sy* *ty*)
      (setq *sz* *tz*)
      (setq *gx* (* mx r))
      (setq *gy* (* my r))
      (setq *gz* (* mz r))
      (setq *t0* tm)
      ;; duration grows with distance: quick nearby hops, ~1.5 s across
      (let* ((ex (- *gx* *sx*))
             (ey (- *gy* *sy*))
             (ez (- *gz* *sz*))
             (dist (sqrt (+ (* ex ex) (* ey ey) (* ez ez)))))
        (setq *dur* (+ 0.45 (* 0.55 dist)))))))

(defun set-goal-from-click (ccx ccy tm)
  ;; Unproject the click: a ray from the eye through the clip point,
  ;; intersected with the plane through the orbit centre facing the camera.
  ;; Everything needed (eye, camera basis, fov) is already in globals.
  (let* ((th (/ (sin +half-fov+) (cos +half-fov+)))
         (hx (* ccx th *aspect*))
         (hy (* ccy th))
         (dx (+ *fx* (* hx *rx*) (* hy *ux*)))
         (dy (+ *fy* (* hx *ry*) (* hy *uy*)))
         (dz (+ *fz* (* hx *rz*) (* hy *uz*)))
         (denom (+ (* dx *fx*) (* dy *fy*) (* dz *fz*)))
         (tt
          (/ (+ (* (- +ctr-x+ *ex*) *fx*) (* (- +ctr-y+ *ey*) *fy*)
                (* (- +ctr-z+ *ez*) *fz*)) denom)))
    (set-goal (+ *ex* (* tt dx)) (+ *ey* (* tt dy)) (+ *ez* (* tt dz)) tm)))

(defun update-target (tm)
  ;; Advance the commanded hand position along the min-jerk profile.
  (let ((u (/ (- tm *t0*) *dur*)))
    (cond ((>= u 1.0)
           (setq *tx* *gx*)
           (setq *ty* *gy*)
           (setq *tz* *gz*))
          ((<= u 0.0)
           (setq *tx* *sx*)
           (setq *ty* *sy*)
           (setq *tz* *sz*))
          (t (let ((s (min-jerk u)))
               (setq *tx* (+ *sx* (* s (- *gx* *sx*))))
               (setq *ty* (+ *sy* (* s (- *gy* *sy*))))
               (setq *tz* (+ *sz* (* s (- *gz* *sz*)))))))))

;; --- inverse kinematics (in 3-D): Jacobian DLS, FABRIK, or analytic ------------
;;
;; Three solvers over the same joint-position state, switchable from the
;; HUD. Each lives in its own file, spliced in here at compile time by the
;; literal top-level (load ...) below -- paths resolve relative to this
;; file, and the compiled .wasm sees the defuns natively:
;;
;;   ik-jacobian.lisp  0 = damped least squares over the position Jacobian
;;                         (linalg:matmul / transpose / solve, iterative)
;;   ik-fabrik.lisp    1 = FABRIK, the geometric method (no matrices)
;;   ik-analytic.lisp  2 = the closed form (atan2 + the law of cosines)
;;                         + forward kinematics through 4x4 homogeneous
;;                         transforms (linalg:matmul, exact, no iteration)

(defvar *solver* 0) ; 0 = jacobian, 1 = FABRIK, 2 = analytic

(defvar *split* 1) ; the analytic elbow: joint index
(defvar *ga* 0.0)  ; upper-group length (base -> elbow)
(defvar *gb* 0.0)  ; lower-group length (elbow -> grasp)

(defun set-solver (s)
  ;; Exported: the HUD's solver selector.
  (setq *solver* s))

(defun place (i from len)
  ;; Move joint i to distance len from joint `from`, preserving direction.
  ;; Shared by both iterative solvers (FABRIK sweeps, DLS re-normalization).
  (let* ((dx (- (aref *jx* i) (aref *jx* from)))
         (dy (- (aref *jy* i) (aref *jy* from)))
         (dz (- (aref *jz* i) (aref *jz* from)))
         (d (sqrt (+ (* dx dx) (* dy dy) (* dz dz))))
         (r (/ len (if (< d 0.000001) 0.000001 d))))
    (setf (aref *jx* i) (+ (aref *jx* from) (* dx r)))
    (setf (aref *jy* i) (+ (aref *jy* from) (* dy r)))
    (setf (aref *jz* i) (+ (aref *jz* from) (* dz r)))))

(load "ik-jacobian.lisp")
(load "ik-fabrik.lisp")
(load "ik-analytic.lisp")

(defun solve-ik ()
  (cond ((= *solver* 0) (solve-ik-jacobian))
        ((= *solver* 1) (solve-ik-fabrik))
        (t (solve-ik-analytic))))

;; The HUD polls this: how far the grasp point still is from the goal.
(defun ik-error ()
  (let* ((tip (+ *links* 1))
         (dx (- (aref *jx* tip) *gx*))
         (dy (- (aref *jy* tip) *gy*))
         (dz (- (aref *jz* tip) *gz*)))
    (sqrt (+ (* dx dx) (* dy dy) (* dz dz)))))

;; --- rendering ------------------------------------------------------------------

(defun link-radius (i)
  ;; the arm tapers from shoulder to wrist
  (* 0.052 (- 1.0 (* 0.55 (/ (* 1.0 i) *links*)))))

(defun push-trail ()
  (let ((tip (+ *links* 1)))
    (setf (aref *trail-x* *trail-head*) (aref *jx* tip))
    (setf (aref *trail-y* *trail-head*) (aref *jy* tip))
    (setf (aref *trail-z* *trail-head*) (aref *jz* tip)))
  (setq *trail-head* (mod (+ *trail-head* 1) +trail+))
  (when (< *trail-count* +trail+) (setq *trail-count* (+ *trail-count* 1))))

;; --- the gripper ---------------------------------------------------------------
;;
;; Three two-phalanx fingers, 120 degrees apart around the tool axis. The
;; grip state *grip* eases between 0 (open, while the hand is flying) and 1
;; (closed): the finger segments pivot from splayed-out angles to angles
;; that make the tips meet exactly at the grasp point, +tool+ ahead of the
;; wrist -- the very point FABRIK pins to the goal.

(defvar *grip* 1.0) ; 0 open .. 1 closed
(defvar *last-tm* 0.0)

(defun update-grip (tm)
  ;; open while the min-jerk flight is in progress, close on arrival;
  ;; eased with a time-based rate so the motion is frame-rate independent
  (let* ((dt0 (- tm *last-tm*))
         (dt (cond ((< dt0 0.0) 0.0) ((> dt0 0.05) 0.05) (t dt0)))
         (target (if (>= (/ (- tm *t0*) *dur*) 1.0) 1.0 0.0))
         (k (* dt (if (> target *grip*) 10.0 7.0)))
         (kk (if (> k 1.0) 1.0 k)))
    (setq *grip* (+ *grip* (* kk (- target *grip*))))
    (setq *last-tm* tm)))

(defun emit-gripper ()
  (let* ((n *links*)
         (wx (aref *jx* n)) ; the wrist
         (wy (aref *jy* n))
         (wz (aref *jz* n))
         ;; approach axis: the solved tool link, wrist -> grasp point
         (ax (/ (- (aref *jx* (+ n 1)) wx) +tool+))
         (ay (/ (- (aref *jy* (+ n 1)) wy) +tool+))
         (az (/ (- (aref *jz* (+ n 1)) wz) +tool+))
         ;; finger phalanx angles from the axis: splayed when open, curled
         ;; so the tips meet on the axis at +tool+ when closed
         (a1 (- 0.70 (* 0.58 *grip*)))
         (a2 (- 0.35 (* 1.15 *grip*)))
         (c1 (cos a1))
         (s1 (sin a1))
         (c2 (cos a2))
         (s2 (sin a2)))
    (perp-basis ax ay az)
    (let ((ux *pux*)
          (uy *puy*)
          (uz *puz*) ; capture: emit-tube reuses
          (vx *pvx*)
          (vy *pvy*)
          (vz *pvz*)) ; the perp-basis globals
      ;; the wrist ball and the palm
      (set-color 0.30 0.34 0.44)
      (emit-sphere wx wy wz (* 1.35 (link-radius n)))
      (set-radii 0.030 0.036)
      (emit-tube wx wy wz (+ wx (* 0.035 ax)) (+ wy (* 0.035 ay))
                 (+ wz (* 0.035 az)))
      ;; three fingers, at ring-table angles 0, 120, 240 degrees
      (dotimes (f 3)
        (let* ((ca (aref *ctab* (* f 4)))
               (sa (aref *stab* (* f 4)))
               (rkx (+ (* ca ux) (* sa vx)))
               (rky (+ (* ca uy) (* sa vy)))
               (rkz (+ (* ca uz) (* sa vz)))
               ;; phalanx directions: rotate the axis toward the radial
               (d1x (+ (* c1 ax) (* s1 rkx)))
               (d1y (+ (* c1 ay) (* s1 rky)))
               (d1z (+ (* c1 az) (* s1 rkz)))
               (d2x (+ (* c2 ax) (* s2 rkx)))
               (d2y (+ (* c2 ay) (* s2 rky)))
               (d2z (+ (* c2 az) (* s2 rkz)))
               ;; knuckle on the palm rim, then two phalanges
               (bx (+ wx (* 0.035 ax) (* 0.030 rkx)))
               (by (+ wy (* 0.035 ay) (* 0.030 rky)))
               (bz (+ wz (* 0.035 az) (* 0.030 rkz)))
               (kx (+ bx (* 0.05 d1x)))
               (ky (+ by (* 0.05 d1y)))
               (kz (+ bz (* 0.05 d1z)))
               (px (+ kx (* 0.05 d2x)))
               (py (+ ky (* 0.05 d2y)))
               (pz (+ kz (* 0.05 d2z))))
          (set-color 0.72 0.76 0.84)
          (set-radii 0.012 0.009)
          (emit-tube bx by bz kx ky kz)
          (set-radii 0.009 0.006)
          (emit-tube kx ky kz px py pz)
          (set-color 0.30 0.34 0.44)
          (emit-sphere kx ky kz 0.011)
          (set-color 1.0 0.52 0.16)
          (emit-sphere px py pz 0.008))))))

(defun emit-arm ()
  ;; tapered tube per link, dark sphere per joint, the gripper at the wrist
  (set-color 0.72 0.76 0.84)
  (dotimes (i *links*)
    (set-radii (link-radius i) (link-radius (+ i 1)))
    (emit-tube (aref *jx* i) (aref *jy* i) (aref *jz* i) (aref *jx* (+ i 1))
               (aref *jy* (+ i 1)) (aref *jz* (+ i 1))))
  (set-color 0.30 0.34 0.44)
  (dotimes (i *links*)
    (emit-sphere (aref *jx* i) (aref *jy* i) (aref *jz* i)
                 (* 1.45 (link-radius i))))
  (emit-gripper))

(defun emit-glow (tm)
  ;; the trail: the hand's recent path, fading with age
  (dotimes (m *trail-count*)
    (let* ((idx (mod (+ (- *trail-head* *trail-count*) m +trail+) +trail+))
           (age (/ (* 1.0 (+ m 1)) *trail-count*)) ; 0 oldest .. 1 newest
           (fade (* age age)))
      (emit-s (aref *trail-x* idx) (aref *trail-y* idx) (aref *trail-z* idx)
              (+ 0.50 (* 0.20 fade)) (+ 5.0 (* 12.0 fade)))))
  ;; the goal: a slowly turning, pulsing ring billboarded on the camera basis
  (let ((r (+ 0.045 (* 0.010 (sin (* tm 5.0))))))
    (dotimes (m 36)
      (let* ((a (+ (* tm 0.8) (/ (* +two-pi+ m) 36))) (ca (cos a)) (sa (sin a)))
        (emit-s (+ *gx* (* r (+ (* ca *rx*) (* sa *ux*))))
                (+ *gy* (* r (+ (* ca *ry*) (* sa *uy*))))
                (+ *gz* (* r (+ (* ca *rz*) (* sa *uz*)))) 1.0 9.0)))
    (emit-s *gx* *gy* *gz* 1.0 14.0)))

(defun upload-vp (loc)
  ;; WebGL wants column-major: element (row, col) lands at row + col*4
  (dotimes (c 4) (dotimes (r 4) (set-float (+ r (* c 4)) (aref *vp* r c))))
  (gl-uniform-matrix4fv loc))

(defun draw (tm)
  (update-camera)
  (let ((w (canvas-width)) (h (canvas-height)))
    (gl:viewport 0 0 (floor w) (floor h)))
  (gl:clear-color 0.012 0.016 0.045 1.0)
  (gl:clear (+ gl:+color-buffer-bit+ gl:+depth-buffer-bit+))
  ;; solid pass: the machine, lit and depth-tested
  (gl:disable gl:+blend+)
  (gl:depth-mask t)
  (gl:use-program *prog-solid*)
  (upload-vp *u-vp-solid*)
  (gl:uniform3f *u-eye* *ex* *ey* *ez*)
  (setq *v* *static-verts*)
  (emit-arm)
  (gl:bind-buffer gl:+array-buffer+ *buf-solid*)
  (gl-upload-vertices (* *static-verts* 9) (* (- *v* *static-verts*) 9))
  (gl:bind-vertex-array *vao-solid*)
  (gl:draw-arrays gl:+triangles+ 0 *v*)
  ;; glow pass: additive sprites that read depth but do not write it
  (gl:enable gl:+blend+)
  (gl:depth-mask nil)
  (gl:use-program *prog-sprite*)
  (upload-vp *u-vp-sprite*)
  (gl:uniform1f *u-dpr* (device-pixel-ratio))
  (setq *s* 0)
  (emit-glow tm)
  (gl:bind-buffer gl:+array-buffer+ *buf-sprite*)
  (gl-upload-sprites (* *s* 5))
  (gl:bind-vertex-array *vao-sprite*)
  (gl:draw-arrays gl:+points+ 0 *s*))

(defun frame (tm)
  (setq *aspect* (/ (canvas-width) (canvas-height)))
  ;; a pending click is unprojected with this frame's camera and time
  (when *clicked*
    (setq *clicked* nil)
    (set-goal-from-click *click-x* *click-y* tm))
  (update-target tm)
  (update-grip tm)
  (solve-ik)
  (push-trail)
  (draw tm))

;; Build the pipeline at load time: this runs inside _initialize, after the
;; page has created the WebGL2 context and instantiated the module.
(setup-gl)

(rontolisp:wasm-export 'init :params '(:int) :returns :void)
(rontolisp:wasm-export 'frame :params '(:float) :returns :void)
(rontolisp:wasm-export 'pointer :params '(:float :float) :returns :void)
(rontolisp:wasm-export 'orbit :params '(:float :float) :returns :void)
(rontolisp:wasm-export 'zoom :params '(:float) :returns :void)
(rontolisp:wasm-export 'set-solver
                       :as "setSolver"
                       :params '(:int)
                       :returns :void)
(rontolisp:wasm-export 'ik-error :as "ikError" :params '() :returns :float)


---

# FILE: references/examples/browser/webgl-triangle/README.md

# triangle.lisp — the WebGL hello world, driven from Lisp

The smallest complete `rontolisp:wasm-import` program: a Lisp program compiled
to WebAssembly that draws one colored triangle in the browser. If you want to
call JavaScript (or WebGL) from Lisp, start here; when you outgrow it,
[`webgl-cube/`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-cube) adds 3D (a vertex buffer, a depth test and
matrix math in Lisp), and [`webgl-galaxy/`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-galaxy) grows the same
idea into a full pipeline (uniforms, shader-error reporting, an animation
loop).

**Live demo:**  (this
directory is published as a subpath of the GitHub Pages site by
`.github/workflows/pages.yaml`).

## What's in here

| File            | Purpose                                                      |
| --------------- | ------------------------------------------------------------ |
| `triangle.lisp` | The whole program: GLSL shaders, pipeline setup, one draw.    |
| `index.html`    | The host page: ten one-line WebGL2 bindings.                  |
| `triangle.wasm` | The compiled `--no-wasi` reactor (checked in).                |
| `build.sh`      | Recompiles `triangle.lisp` to `triangle.wasm`.                |

## How it works

`triangle.lisp` declares ten host functions — the minimum slice of WebGL2
needed to compile two shaders, link them and draw:

```lisp
(rontolisp:wasm-import 'gl-create-shader :from "gl" :as "createShader"
                       :params '(:int) :returns :int)
(rontolisp:wasm-import 'gl-shader-source :from "gl" :as "shaderSource"
                       :params '(:int :string) :returns :void)
;; ... compileShader, createProgram, attachShader, linkProgram, useProgram,
;;     clearColor, clear, drawArrays
```

The GLSL sources are Lisp string constants; a `:string` parameter reaches the
host as a `(ptr, len)` pair into the module's exported linear memory. GL
objects (the shaders, the program) cross the boundary as `:int` handles into a
small table the page keeps.

The JavaScript side is the import object and nothing else:

```js
const handles = [];
const H = (obj) => handles.push(obj) - 1;
const str = (ptr, len) =>
  new TextDecoder().decode(new Uint8Array(lisp.memory.buffer, ptr, len));

const imports = {
  gl: {
    createShader: (type) => H(gl.createShader(type)),
    shaderSource: (sh, p, n) => gl.shaderSource(handles[sh], str(p, n)),
    // ... eight more one-liners
  },
};
const { instance: { exports: lisp } } = await WebAssembly.instantiate(bytes, imports);
lisp._initialize(); // runs the whole Lisp program: the triangle appears
```

There are no `rontolisp:wasm-export` directives and no frame loop: the whole
program is top-level forms, which a `--no-wasi` reactor runs inside
`_initialize()`. By the time that call returns, the triangle is on the canvas.

Two tricks keep it minimal:

- **No vertex buffer.** The vertex shader looks its corner positions and
  colors up by `gl_VertexID` (WebGL2 allows attributeless draws), so no
  buffer/attribute imports are needed at all.
- **No resize/viewport handling.** The canvas has a fixed backing store
  (`width="640" height="480"`), and a WebGL context's default viewport is the
  canvas size at creation.

## Building and running

```bash
# from the repo root, once:
./mvnw clean package

# recompile the .wasm after editing triangle.lisp:
examples/browser/webgl-triangle/build.sh

# serve and open (any static file server works):
jwebserver -p 8000 --directory "$PWD/examples/browser/webgl-triangle"
open http://localhost:8000/
```

The page needs a browser with WebAssembly GC support (Chrome 119+,
Firefox 120+, Safari 18.2+, Edge 119+).

## Notes

- The module is compiled with `--no-wasi`, so its *only* imports are the ten
  functions above — the import object is the whole embedding API.
- For brevity there is no shader-compile error check; see
  [`webgl-galaxy/`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-galaxy) for the error-reporting pattern
  (`getShaderParameter` / `getShaderInfoLog` as a `:string` result).
- On the interpreter and JVM backends the `rontolisp:wasm-import` directives
  define stubs that signal an error when called, so this program is
  WASM-only by nature (there is no host to draw with elsewhere).
- This demo deliberately stays on `rontolisp:wasm-import` and does *not* use
  the shared `gl` package or its `gl.wit`
  ([`../webgl-common/`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-common)), which every other `webgl-*` demo
  binds. Being one self-contained file — the program, its ten imports and the
  page — is the point here: a reader should see the whole boundary without
  following a `require` into another directory. Please do not retrofit it.


---

# FILE: references/examples/browser/webgl-triangle/build.sh

#!/usr/bin/env bash
# Recompile triangle.lisp to a browser-loadable WebAssembly reactor.
# --no-wasi drops all WASI imports, so the module's only imports are the ten
# "gl" host functions triangle.lisp declares with rontolisp:wasm-import;
# --optimize tree-shakes the runtime so only the reachable functions ship.
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling triangle.lisp -> triangle.wasm"
java -jar "$jar" "$here/triangle.lisp" -o "$here/triangle.wasm" --no-wasi --optimize

echo "done. Serve this directory over http, e.g.:"
echo "  jwebserver -p 8000 --directory \"$here\""
echo "then open http://localhost:8000/"


---

# FILE: references/examples/browser/webgl-triangle/index.html



  
    
    
    triangle.lisp — the WebGL hello world, driven from Lisp
    
  
  
    

      
triangle.lisp → WebGL

      

        The WebGL hello world: this triangle is drawn by a Lisp program compiled to
        WebAssembly. Lisp compiles the shaders (the GLSL lives in the Lisp source),
        links the program and issues the draw call through ten
        rontolisp:wasm-import host functions — one line of JavaScript each.
        There is no frame loop and no exports: the page calls
        _initialize() and the triangle is there.
      

      
      

        source ·
        next step: webgl-cube (3D: matrices in Lisp) ·
        then: webgl-galaxy (a full pipeline)
      

      
    


    
  



---

# FILE: references/examples/browser/webgl-triangle/triangle.lisp

;;;; triangle.lisp -- the WebGL hello world, driven from Lisp.
;;;;
;;;; The smallest complete rontolisp:wasm-import program: it draws one colored
;;;; triangle. There are no exports and no frame loop -- the whole program is
;;;; top-level forms, so the page only instantiates the module and calls
;;;; _initialize(); by the time that returns, the triangle is on the canvas.
;;;;
;;;; To stay minimal the triangle uses no vertex buffer at all: the vertex
;;;; shader looks its corner positions and colors up by gl_VertexID (WebGL2
;;;; allows attributeless draws), so the only GL work is compiling the two
;;;; shaders, linking, clearing and one draw call.
;;;;
;;;; Ten imported host functions, one line of JavaScript each. When you outgrow
;;;; this, ../webgl-galaxy/ is the full-pipeline version: vertex buffers,
;;;; uniforms, shader-error reporting and an animation loop.

;; --- the host boundary ------------------------------------------------------
;; :as maps the Lisp name to the JavaScript property; :from names the
;; import-object key. GL objects (shaders, the program) cross the boundary as
;; :int handles into a table the page keeps; the GLSL source crosses as
;; :string.

(rontolisp:wasm-import 'gl-create-shader
                       :from "gl"
                       :as "createShader"
                       :params '(:int)
                       :returns :int)
(rontolisp:wasm-import 'gl-shader-source
                       :from "gl"
                       :as "shaderSource"
                       :params '(:int :string)
                       :returns :void)
(rontolisp:wasm-import 'gl-compile-shader
                       :from "gl"
                       :as "compileShader"
                       :params '(:int)
                       :returns :void)
(rontolisp:wasm-import 'gl-create-program
                       :from "gl"
                       :as "createProgram"
                       :params '()
                       :returns :int)
(rontolisp:wasm-import 'gl-attach-shader
                       :from "gl"
                       :as "attachShader"
                       :params '(:int :int)
                       :returns :void)
(rontolisp:wasm-import 'gl-link-program
                       :from "gl"
                       :as "linkProgram"
                       :params '(:int)
                       :returns :void)
(rontolisp:wasm-import 'gl-use-program
                       :from "gl"
                       :as "useProgram"
                       :params '(:int)
                       :returns :void)
(rontolisp:wasm-import 'gl-clear-color
                       :from "gl"
                       :as "clearColor"
                       :params '(:float :float :float :float)
                       :returns :void)
(rontolisp:wasm-import 'gl-clear
                       :from "gl"
                       :as "clear"
                       :params '(:int)
                       :returns :void)
(rontolisp:wasm-import 'gl-draw-arrays
                       :from "gl"
                       :as "drawArrays"
                       :params '(:int :int :int)
                       :returns :void)

;; --- WebGL constants --------------------------------------------------------
;; The numeric enum values from the WebGL specification.

(defconstant +gl-vertex-shader+ 35633)    ; 0x8B31
(defconstant +gl-fragment-shader+ 35632)  ; 0x8B30
(defconstant +gl-color-buffer-bit+ 16384) ; 0x4000
(defconstant +gl-triangles+ 4)

;; --- shaders ----------------------------------------------------------------
;; The GLSL lives here, in Lisp, and reaches the GPU through the imported
;; gl-shader-source (a :string parameter crossing the boundary as (ptr,len)
;; into this module's linear memory).

(defconstant +vertex-shader-source+
  "#version 300 es
// the classic hello-world triangle: positions and colors baked into the
// shader, looked up by gl_VertexID -- no vertex buffer needed
const vec2 POSITION[3] = vec2[3](
  vec2( 0.0,   0.62),
  vec2(-0.65, -0.5),
  vec2( 0.65, -0.5)
);
const vec3 COLOR[3] = vec3[3](
  vec3(1.0, 0.25, 0.3),
  vec3(0.25, 1.0, 0.45),
  vec3(0.3, 0.45, 1.0)
);
out vec3 vColor;
void main() {
  gl_Position = vec4(POSITION[gl_VertexID], 0.0, 1.0);
  vColor = COLOR[gl_VertexID];
}")

(defconstant +fragment-shader-source+
  "#version 300 es
precision mediump float;
in vec3 vColor;
out vec4 color;
void main() {
  color = vec4(vColor, 1.0);
}")

;; --- the program ------------------------------------------------------------

(defun make-shader (type source)
  (let ((shader (gl-create-shader type)))
    (gl-shader-source shader source)
    (gl-compile-shader shader)
    shader))

(defun main ()
  (let ((program (gl-create-program)))
    (gl-attach-shader program
                      (make-shader +gl-vertex-shader+ +vertex-shader-source+))
    (gl-attach-shader program
     (make-shader +gl-fragment-shader+ +fragment-shader-source+))
    (gl-link-program program)
    (gl-use-program program)
    (gl-clear-color 0.07 0.08 0.12 1.0)
    (gl-clear +gl-color-buffer-bit+)
    (gl-draw-arrays +gl-triangles+ 0 3)))

;; Runs inside _initialize, after the page has created the WebGL2 context and
;; instantiated the module.
(main)


---

# FILE: references/examples/browser/wit-component/README.md

# A WebAssembly component in the browser

An interactive Mandelbrot / Julia explorer whose every pixel is computed by
[`fractal.lisp`](fractal.lisp), compiled to a **WebAssembly component** and
loaded by [`index.html`](index.html) with nothing else at all: no
`WebAssembly.instantiate`, no import object, no WASI shim, no `memory.buffer`,
no bump allocator, no `(ptr, len)` pair to decode.

Every other browser demo in this tree loads a raw **core module** and pays for
it in the page. [`rainbow`](https://github.com/making/rontolisp/blob/develop/examples/browser/rainbow) copies UTF-8 bytes into the module's
linear memory through its exported `__ronto_alloc` and decodes a returned
`(ptr, len)` pair by hand; [`wasm-browser`](https://github.com/making/rontolisp/blob/develop/examples/browser/wasm-browser) and
[`hiragana`](https://github.com/making/rontolisp/blob/develop/examples/browser/hiragana) ship an 8-function hand-written WASI shim; the
[`webgl-*`](https://github.com/making/rontolisp/blob/develop/examples/browser/webgl-galaxy) demos hand-write a JavaScript import object of
dozens of host functions to satisfy their `rontolisp:wasm-import` declarations.
This page's entire interface to the module is:

```js
const { mandelbrot, julia, palette, escapeTime, inSet } = await import("./dist/fractal.js");
const chars = mandelbrot(centerX, centerY, scale, cols, rows, maxIter);
```

That is the difference the component model makes: [`wit/fractal.wit`](wit/fractal.wit)
types the exports, the canonical ABI moves the strings across the boundary and
frees them after every call, and `jco` generates the JavaScript bindings by
reading that world back out of the `.wasm`.

## Build

Needs the rontolisp jar (built once from the repository root) and Node, for
`npx`. Everything else `build.sh` downloads on demand.

```bash
# from the repository root
./mvnw clean package -DskipTests

# here
./build.sh
```

`build.sh` runs two commands:

```bash
rontolisp fractal.lisp -o fractal.wasm --no-gc --component --optimize --emit-wit
npx -y @bytecodealliance/jco transpile fractal.wasm -o dist --base64-cutoff 1000000
```

`--base64-cutoff` inlines the core module into the generated JavaScript, so
`dist/fractal.js` is a single self-contained ES module with **zero `import`
statements** -- the page fetches nothing but that file.

## Run

ES modules need an http server (`file://` will not load them):

```bash
python3 -m http.server 8000
# open http://localhost:8000/
```

Hover the Mandelbrot set on the left and the Julia set of the point under the
cursor is drawn on the right, live; click to zoom in, shift-click to zoom out.
A 240x144 frame is one call and renders in ~20 ms in Chrome.

## What the page has to supply: nothing

The world has **no imports**, so there is nothing for the page to provide. That
is not a JavaScript convenience -- it is a property of the module: `--no-gc`
compiles a pure-compute reactor with no WASI, no GC and no runtime flags, and a
component with an empty import section instantiates against an empty world.

Two shapes in `fractal.lisp` are dictated by the boundary rather than by the
mathematics, and both are worth knowing before writing a world of your own:

- **A frame comes back as a string of palette characters, one per pixel.** A
  component that could return a `list` would not need the detour, but a
  rontolisp component's exports carry scalars and strings only: `string` is the
  widest channel available today. The module exports its own `palette`,
  so the encoding is declared in the world and hard-coded nowhere in the page.
- **Mandelbrot and Julia are separate exports** although one iteration loop
  serves both: a wasm-GC callable takes at most **seven parameters**, so a single
  `render(center-x, center-y, scale, cols, rows, max-iter, julia, cx, cy)` would
  compile under `--no-gc` and fail on the wasm-GC backend. Six parameters each
  keeps one source compiling on every backend -- which is what `examples.yaml`
  pins (`no-gc` and `wasm-component`).

## What does NOT work yet

This demo is `--no-gc` on purpose. A **wasm-GC** `--component` also loads and
computes in Chrome 149 -- wasm-GC, JSPI and the canonical ABI are all fine there
-- but it cannot **print** in a browser, for two upstream reasons that are worth
stating separately:

- **jco 1.25.2 half-emits the `future` runtime.** A printing component imports
  `wasi:cli/stdout`, whose `write-via-stream` returns a `future<...>` in WASI
  0.3. The transpiled bundle *references* `FutureReadableEnd`, `FutureEnd` and
  `FutureWritableEnd` and **defines none of them**, so the first write dies with
  `ReferenceError: FutureReadableEnd is not defined` -- on the import side,
  before any export is even lifted. It reproduces under every `--async-mode` /
  `--instantiation` combination. (This is distinct from the known gap where jco
  cannot *call* a stackful-async export at all.)
- **`@bytecodealliance/preview3-shim` has no browser build.** Its `exports` map
  has only a `node` condition and its code imports `node:worker_threads`, `net`,
  `fs/promises`, ... -- unlike `preview2-shim`, which ships `dist/browser/`. So a
  WASI 0.3 component in a browser needs a hand-written 0.3 shim (~90 lines, nine
  names jco destructures at module load).

Neither is a rontolisp bug, and neither can touch this demo: a non-printing
`--no-gc` component imports **nothing**, so there is no WASI, no `future` and no
shim in the picture. (A *printing* `--no-gc` component shares the wasm-GC
component's fate here since its print bridge moved to WASI 0.3: it imports
`wasi:cli/stdout@0.3.0` and its exports are async lifts, so keep browser-bound
programs print-free.) (Node is a *worse* host than Chrome here, incidentally:
Node 22.16 cannot import a wasm-GC component at all -- `TypeError:
WebAssembly.Suspending is not a constructor`.)

## The world is the contract

`rontolisp:wit-export` at the bottom of `fractal.lisp` names
[`wit/fractal.wit`](wit/fractal.wit). The compiler reads it, checks the five
exports it declares against the defuns -- name, arity, parameter and result types
-- and lowers each into the export directive it stands for. Rename a defun,
change an argument, return the wrong type, and the build fails with a compile
error naming the WIT line instead of a puzzle in the browser console. The check
runs on **every** backend, so even `rontolisp fractal.lisp` (the interpreter)
catches a drifted contract without compiling anything.

`--emit-wit` writes the component's own world next to the `.wasm` as
`fractal.wit`: the same five exports, under the package and world name a
component's type always has (`root:component` / `root`). It is the `.wit` without
the binary -- hand it to `wit-bindgen` or `jco types` and you get bindings for
any host language, with no `wasm-tools` introspection step.


---

# FILE: references/examples/browser/wit-component/build.sh

#!/usr/bin/env bash
# Compile fractal.lisp to a WebAssembly COMPONENT and transpile it to a browser
# ES module.
#
#   1. rontolisp: --no-gc --component --optimize --emit-wit
#        --no-gc      a plain MVP core module (no wasm-GC, no WASI, no flags),
#                     wrapped in a component
#        --component  the canonical ABI carries the strings across the boundary,
#                     so no page code ever touches memory or a pointer
#        --emit-wit   write the component's OWN world next to the .wasm; it is a
#                     regeneration of wit/fractal.wit (which the program declares
#                     it implements) with the component's own package/world name
#   2. jco transpile: read the types out of the .wasm and generate the JS
#      bindings the page imports. --base64-cutoff inlines the core module into
#      the .js, so dist/fractal.js is one self-contained file with no imports.
#
# Needs Node (for npx). Everything else is downloaded on demand by npx.
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling fractal.lisp -> fractal.wasm (component) + fractal.wit"
java -jar "$jar" "$here/fractal.lisp" -o "$here/fractal.wasm" \
  --no-gc --component --optimize --emit-wit

echo "transpiling fractal.wasm -> dist/fractal.js"
(cd "$here" && npx -y @bytecodealliance/jco transpile fractal.wasm \
  -o dist --base64-cutoff 1000000)

echo "done. Serve this directory over http, e.g.:"
echo "  python3 -m http.server 8000 --directory \"$here\""
echo "then open http://localhost:8000/"


---

# FILE: references/examples/browser/wit-component/fractal.lisp

;;;; fractal -- a Mandelbrot/Julia explorer that runs in a browser as a
;;;; WebAssembly COMPONENT: the page imports five functions and supplies nothing.
;;;;
;;;; Every other browser demo in this tree loads a raw core module, and the page
;;;; pays for it: rainbow.html hands its text to the module by calling the
;;;; exported bump allocator __ronto_alloc, copying UTF-8 bytes into
;;;; exports.memory.buffer, passing (ptr, len) and decoding the returned
;;;; (ptr, len) back out; the webgl demos hand-write a JavaScript import object
;;;; of dozens of host functions to satisfy their rontolisp:wasm-import
;;;; declarations. None of that is here. This program is compiled to a component:
;;;;
;;;;   rontolisp fractal.lisp -o fractal.wasm --no-gc --component --optimize --emit-wit
;;;;   npx -y @bytecodealliance/jco transpile fractal.wasm -o dist
;;;;
;;;; and the whole of index.html's interface to it is:
;;;;
;;;;   const { mandelbrot, julia, palette, escapeTime, inSet }
;;;;     = await import("./dist/fractal.js");
;;;;
;;;; No WebAssembly.instantiate, no import object, no memory, no allocator, no
;;;; WASI shim. The canonical ABI of the component model moves the strings across
;;;; and frees them, and jco generated those bindings by reading the world out of
;;;; the .wasm. Nothing is downloaded at run time either: jco inlines the core
;;;; module into a single self-contained ES module, so the page is a static file
;;;; any http server can serve.
;;;;
;;;; The world -- wit/fractal.wit -- is the contract, and rontolisp:wit-export at
;;;; the bottom says "this program implements it". The compiler reads the .wit,
;;;; checks the five exports it declares against the defuns below (name, arity,
;;;; parameter and result types) and lowers each into the export directive it
;;;; stands for. A drifted contract is a compile error naming the WIT line, not a
;;;; puzzle in the browser console.
;;;;
;;;; Two shapes here are dictated by the boundary rather than by the mathematics,
;;;; and both are worth knowing before writing a world of your own:
;;;;
;;;;   * The Mandelbrot and Julia renderers are separate exports, although one
;;;;     iteration loop serves both. A wasm-GC callable takes at most seven
;;;;     parameters, so a single render(center-x, center-y, scale, cols, rows,
;;;;     max-iter, julia, cx, cy) would compile under --no-gc and fail on the
;;;;     wasm-GC backend. Six parameters each keeps ONE source compiling on every
;;;;     backend, which is what examples.yaml pins.
;;;;
;;;;   * A frame comes back as a string of palette characters, one per pixel. A
;;;;     component that could return a list would not need the detour, but a
;;;;     rontolisp component's exports carry scalars and strings only: string is
;;;;     the widest channel available today.
;;;;
;;;; Everything stays inside the --no-gc subset -- floats, integers, string
;;;; literals, (concatenate 'string ...) and (subseq ...); no cons, list, hash or
;;;; I/O -- so the component needs no garbage collector, no WASI and no runtime
;;;; flags.

;;; The characters the renderers may return, darkest first. Index 0 means "inside
;;; the set"; indices 1..63 are the escape-time ramp. The page colors a pixel by
;;; the index of its character here, so this string IS the pixel encoding -- and
;;; it is an export, so the page never hard-codes it.
;;; WIT: palette: func() -> string
(defun palette ()
  "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz+-")

;;; The escape time of the orbit that starts at z = (zx, zy) with the constant
;;; c = (px, py): how many iterations of z <- z^2 + c it survives before |z| > 2
;;; (i.e. |z|^2 > 4), capped at max-iter. Mandelbrot starts z at the origin and
;;; takes c from the pixel; Julia starts z at the pixel and takes c from the
;;; caller. That one swap is the whole difference between the two sets, which is
;;; why both renderers iterate this same function.
(defun iterate (zx zy px py max-iter)
  (let ((i 0))
    (while (and (< i max-iter) (<= (+ (* zx zx) (* zy zy)) 4.0))
      (let ((zt (+ (- (* zx zx) (* zy zy)) px)))
        (setq zy (+ (* 2.0 (* zx zy)) py))
        (setq zx zt))
      (setq i (+ i 1)))
    i))

;;; One escape time as one palette character. The ramp is square-root shaped
;;; because escape times crowd into the low end: sqrt spreads the outer bands
;;; over the whole palette instead of leaving most of the 64 characters unused.
(defun shade (n max-iter)
  (if (>= n max-iter)
      (subseq (palette) 0 1)
      (let ((k (+ 1 (floor (* 63 (sqrt (/ n max-iter)))))))
        (subseq (palette) k (+ k 1)))))

;;; One row of a Mandelbrot / Julia grid, as a string of cols palette characters.
;;;
;;; A row is built separately from the frame on purpose. Growing one string
;;; character by character copies it every time, so accumulating a whole frame
;;; that way costs O((cols*rows)^2) byte copies -- ~600 million for a 240x144
;;; view. Per row it is ~8 million, and the frame renders in milliseconds.
(defun mandelbrot-row (y x0 step cols max-iter)
  (let ((row ""))
    (dotimes (c cols)
      (setq row
            (concatenate 'string row
             (shade (iterate 0.0 0.0 (+ x0 (* step c)) y max-iter) max-iter))))
    row))

(defun julia-row (y x0 step cols max-iter cx cy)
  (let ((row ""))
    (dotimes (c cols)
      (setq row
            (concatenate 'string row
             (shade (iterate (+ x0 (* step c)) y cx cy max-iter) max-iter))))
    row))

;;; Render the Mandelbrot set as a cols x rows grid, row-major, as a single string
;;; of cols * rows palette characters. scale is the width of the view in
;;; complex-plane units; the height follows from the grid's aspect ratio.
;;; WIT: mandelbrot: func(center-x: f64, center-y: f64, scale: f64, cols: s32, rows: s32, max-iter: s32) -> string
(defun mandelbrot (center-x center-y scale cols rows max-iter)
  (let ((step (/ scale cols)) (out ""))
    (let ((x0 (- center-x (/ scale 2.0)))
          (y0 (- center-y (/ (* step rows) 2.0))))
      (dotimes (r rows)
        (setq out
              (concatenate 'string out
               (mandelbrot-row (+ y0 (* step r)) x0 step cols max-iter)))))
    out))

;;; Render the Julia set of the constant cx + cy*i the same way, as a view of
;;; width scale centered on the origin.
;;; WIT: julia: func(cx: f64, cy: f64, scale: f64, cols: s32, rows: s32, max-iter: s32) -> string
(defun julia (cx cy scale cols rows max-iter)
  (let ((step (/ scale cols)) (out ""))
    (let ((x0 (- 0.0 (/ scale 2.0))) (y0 (- 0.0 (/ (* step rows) 2.0))))
      (dotimes (r rows)
        (setq out
              (concatenate 'string out
               (julia-row (+ y0 (* step r)) x0 step cols max-iter cx cy)))))
    out))

;;; The escape time of one point of the Mandelbrot set -- what the page shows for
;;; the point under the cursor.
;;; WIT: escape-time: func(x: f64, y: f64, max-iter: s32) -> s32
(defun escape-time (x y max-iter) (iterate 0.0 0.0 x y max-iter))

;;; Whether the point is in the Mandelbrot set: it survives every iteration.
;;; WIT: in-set: func(x: f64, y: f64, max-iter: s32) -> bool
(defun in-set (x y max-iter) (>= (escape-time x y max-iter) max-iter))

;;; Implement wit/fractal.wit -- the contract this program is checked against,
;;; and the exports it gets. Nothing else in this file is visible to the host.
(rontolisp:wit-export "wit/fractal.wit" :world fractal)


---

# FILE: references/examples/browser/wit-component/index.html



  
    
    
    rontolisp fractal explorer (WebAssembly component)
    
  
  
    
A fractal explorer, compiled from Lisp to a WebAssembly component

    

      Hover the Mandelbrot set on the left: the Julia set of the point under the
      cursor is drawn on the right, live. Click to zoom in, shift-click to zoom
      out. Every pixel of both images is computed by
      fractal.lisp, compiled to a ~2.5 KB WebAssembly component and transpiled with
      jco.
    


    

      This is what the page does to talk to the module — all of it:
    

    
const { mandelbrot, julia, palette, escapeTime, inSet } = await import("./dist/fractal.js");
const chars = mandelbrot(centerX, centerY, scale, cols, rows, maxIter);

    

      No WebAssembly.instantiate, no import object, no WASI shim, no
      memory.buffer, no __ronto_alloc, no
      (ptr, len) pair to decode. The
      WIT world types the five exports, the
      canonical ABI of the component model moves the strings across and frees
      them, and jco generated the bindings by reading that world out
      of the .wasm. Compare with the sibling
      rainbow demo, whose page copies UTF-8
      bytes into the module's linear memory through its bump allocator and decodes
      a returned (ptr, len) pair, or with
      webgl-galaxy, whose page hand-writes an
      import object of dozens of host functions.
    


    

      

        
        

          mandelbrot(center-x, center-y, scale, cols, rows, max-iter)
          — hover to pick c, click to zoom.
        

      

      

        
        

          julia(cx, cy, scale, cols, rows, max-iter) — the Julia
          set of the hovered c, redrawn on every mouse move.
        

      

    


    

      
      
      
    


    
(loading the component...)

    


    
  



---

# FILE: references/examples/browser/wit-component/wit/fractal.wit

package example:fractal;

/// An escape-time fractal explorer. This file is the contract, and the only
/// thing the browser page ever learns about the module: `jco transpile` reads
/// these types straight out of the compiled component and generates the
/// JavaScript bindings from them, so the page imports five functions and never
/// touches memory, a pointer or an import object.
///
/// Every function stays inside the type set a rontolisp component can carry
/// today (s32/s64/f64/bool/string), and inside the seven-parameter limit a
/// wasm-GC callable has -- so the same program also compiles on the wasm-GC
/// component backend, not just on the --no-gc one the page loads.
world fractal {
  /// Render the Mandelbrot set as a `cols` x `rows` grid, row-major, returned as
  /// a single string of exactly `cols * rows` characters -- one per pixel, taken
  /// from `palette`. `scale` is the width of the view in complex-plane units;
  /// the height follows from the grid's aspect ratio.
  export mandelbrot: func(center-x: f64, center-y: f64, scale: f64, cols: s32, rows: s32, max-iter: s32) -> string;

  /// Render the Julia set of the constant `cx + cy*i` the same way, as a view of
  /// width `scale` centered on the origin.
  export julia: func(cx: f64, cy: f64, scale: f64, cols: s32, rows: s32, max-iter: s32) -> string;

  /// The characters the two renderers may return, darkest first: index 0 means
  /// "inside the set", and the rest are an escape-time ramp. The host colors a
  /// pixel by the index of its character in this string, so the pixel encoding
  /// is declared here and nowhere in the page.
  export palette: func() -> string;

  /// The escape time of one point of the Mandelbrot set: how many iterations of
  /// z <- z^2 + c it survives before |z| > 2, capped at `max-iter`.
  export escape-time: func(x: f64, y: f64, max-iter: s32) -> s32;

  /// Whether the point is in the Mandelbrot set, i.e. whether it survives all
  /// `max-iter` iterations.
  export in-set: func(x: f64, y: f64, max-iter: s32) -> bool;
}


---

# FILE: references/examples/cloudflare-workers/README.md

# rontolisp on Cloudflare Workers

Common Lisp compiled to WebAssembly, running on Cloudflare Workers. Each
directory is a complete, independent Worker project: `./build.sh && npx wrangler
dev`, then `npx wrangler deploy`.

Two subjects — a greeting and a mini [httpbin](https://httpbin.org) — written
once with **no library** and then once in the idiom of each web library. The
point of the set is how differently the same endpoints come out, so the versions
share no code and are not meant to diff cleanly against each other. Two more
directories are about the **boundary** rather than the library:
[`dog-fetcher/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/dog-fetcher) and [`btc-ticker/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/btc-ticker) both call out over
HTTP, on the two shapes `--host-boundary` chooses between, and
[`dog-relay/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/dog-relay) is the streaming shape compiled `--reentrant`, serving
its requests overlapped on one instance.

Module sizes are measured rather than quoted here:
[`size-report/results/cloudflare-workers.md`](https://github.com/making/rontolisp/blob/develop/examples/size-report/results/cloudflare-workers.md).

| Directory | Written as | Host glue |
| --- | --- | --- |
| [`hello/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/hello) | **Start here.** Three `wasm-export`ed functions JavaScript calls directly. `--no-gc`, a plain MVP module with zero imports | 32 lines, no dependencies |
| [`hello-clack/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/hello-clack) | **Start here if you want Clack.** One application function and `clack:clackup` — the whole of [Clack](https://github.com/fukamachi/clack)'s API | GENERATED, and all of it: `src/index.js` is three lines |
| [`hello-tiny-routes/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/hello-tiny-routes) | [tiny-routes](https://github.com/jeko2000/tiny-routes): a route table composed with `define-routes`, threaded through middleware with `pipe`. Loaded as `tiny-routes/lite`, so no regex engine ships | GENERATED — the same file `hello-clack` gets, because the declarations are the same |
| [`hello-ningle/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/hello-ningle) | [ningle](https://github.com/fukamachi/ningle): routes assigned to a CLOS *object*, a bare string as a controller, an overridden `not-found` **method** | GENERATED — the same file `hello-clack` gets, because the declarations are the same |
| [`httpbin/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin) | **No library.** Five echo endpoints, 405, 404, `handler-case` — plus the reactor adapter written out by hand, so clack never ships | 54 lines, boundary included — the one hand-written host left here |
| [`httpbin-clack/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin-clack) | Plain Clack: an application *function*, a `cond` over `:path-info`, and one middleware — a function from application to application | GENERATED, and all of it: `src/index.js` is a `worker(module)` call |
| [`httpbin-clack-one-source/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin-clack-one-source) | **No `worker.lisp` at all**: `build.sh` compiles [`net/httpbin-clack.lisp`](../net/httpbin-clack.lisp) unchanged, the file that binds a socket locally. One source, four hosts | GENERATED — the same file `httpbin-clack` gets |
| [`httpbin-tiny-routes/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin-tiny-routes) | tiny-routes: route macros, a `/status/:code` template, declining, and middleware that reads the body, parses the query and sets the content type | GENERATED — the same file `httpbin-clack` gets |
| [`httpbin-ningle/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin-ningle) | ningle: routes assigned in a loop, a controller that returns a string and mutates `*response*`, a request that arrives already parsed, a regex rule that declines | GENERATED — the same file `httpbin-clack` gets |
| [`httpbin-component/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin-component) | The same `httpbin` source through the component model (`--component --no-wasi` + `jco transpile`) instead of raw linear memory | 37 lines + generated glue |
| [`dog-fetcher/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/dog-fetcher) | **Outgoing HTTP, streaming bodies.** A proxy over [dog.ceo](https://dog.ceo), routed with `tiny-routes/lite`. `rontolisp:fetch` is wasi:http and a reactor has no WASI, so the client is the host's own `fetch`, imported and bridged with JSPI | GENERATED, and all of it: `src/index.js` is three lines |
| [`btc-ticker/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/btc-ticker) | **The Worker with nothing in it.** One outgoing request, one JSON answer — on the DEFAULT boundary, where every body rides the envelope and the module imports exactly one function | GENERATED, and all of it: `src/index.js` is three lines |
| [`dog-relay/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/dog-relay) | **Overlapped calls, relayed bodies.** Forwards dog.ceo's own reply chunk by chunk, and is compiled `--reentrant` so concurrent requests overlap on ONE instance instead of queueing — the streaming boundary with a call id on every body import | GENERATED, and all of it: `src/index.js` is three lines |

## Which one should I copy?

- **`hello/`** if your Lisp is numbers and strings — by far the smallest thing
  that works, and it needs no runtime support at all.
- **`httpbin/`** for anything else without a library: the shortest path to the
  full language on Workers, with the thirty lines that put it on Cloudflare in
  the file rather than in a library.
- **`httpbin-clack/`** when the file should read like every other Clack program.
  The per-request cost is the same as `httpbin/`'s; what clack costs is module
  size and a little isolate startup, paid once.
- **`httpbin-clack-one-source/`** when the program already serves somewhere
  else. `:server :rontolisp` picks the transport from the compile target, so
  there is no edit between a local server and this Worker.
- **`httpbin-tiny-routes/`** when the routes deserve a library — templates,
  declining, middleware combinators — *provided* it is loaded as
  `tiny-routes/lite`. Full `"tiny-routes"` spells the same routes and ships
  cl-ppcre.
- **`httpbin-ningle/`** if you already write ningle. It is by an order of
  magnitude the largest and slowest to start of the four, and the reason is not
  ningle: it reads every request through the `lack-request` chain, which is also
  what lets its controllers ignore streams and JSON parsing entirely.
- **`btc-ticker/`** when the Worker fetches a document and answers a document —
  which is most of them, and which is what a build gets for saying nothing: the
  module imports one host function, nothing on the JavaScript side keeps state,
  and there is no cursor whose lifetime anyone has to get right.
- **`dog-fetcher/`** when a body is **binary, large, or relayed** — an image, an
  upload, an upstream reply to forward as it arrives. Those are the cases the
  envelope cannot serve (it carries a body as JSON text, so a non-UTF-8 byte does
  not survive, and it holds the whole thing in memory), so its `build.sh` ASKS
  for `--host-boundary=streaming`. It costs no more JavaScript: the glue writes
  the body imports too, so this `src/index.js` is also three lines. It is where
  the synchronous-Lisp/asynchronous-JavaScript seam is explained in full; both
  directories rely on it.
- **`dog-relay/`** when the Worker is I/O-bound and cannot afford a queue: each
  request is mostly time parked on an upstream, so `--reentrant` lets them
  overlap on one instance (six concurrent relays in about one round trip). It
  is dog-fetcher's boundary plus that flag; read dog-fetcher first.
- **`httpbin-component/`** answers a question rather than being a
  recommendation: *wouldn't the component model be simpler?* For the string
  marshalling, yes. Everywhere else, no.

## What is settled about Workers

All verified end to end under `npx wrangler dev` (workerd), not inferred:

- **wasm-GC runs.** rontolisp's default output is WebAssembly GC, and workerd is
  V8, which has had WasmGC on by default since Chrome 119. No flag, no setting.
- **wasm exception handling runs**, also with no flag — so `handler-case` can
  answer 500 from inside the Lisp. Under wasmtime the same module needs
  `-W exceptions=y`.
- **JSPI runs**, again with no flag and no compatibility-date opt-in, so a
  synchronous wasm import can be answered by an `async` JavaScript function —
  which is what lets a reactor, whose `rontolisp:fetch` is unavailable, make
  outgoing HTTP requests at all. [`dog-fetcher/`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/dog-fetcher) is that, and
  spells out what suspending costs: while a handler is parked the isolate is
  free, so concurrent requests have to be serialised by hand.
- **A Worker may not compile WebAssembly at run time.** `import module from
  "./x.wasm"` gives an already-compiled `WebAssembly.Module`; anything calling
  `WebAssembly.compile()` on bytes hangs at startup. That one fact shapes the
  `httpbin-component/` build.
- **An instance is per isolate, not per request.** Instantiate once and cache
  it, and remember Lisp globals then persist between that isolate's requests.
  Anything that must really persist belongs in KV, D1 or a Durable Object.
- **wasm-GC does not cover the boundary.** Inside the module everything is
  engine-managed, but WebAssembly has no string type, so a string crossing the
  boundary is UTF-8 bytes in **linear memory**, which the engine never traces.
  That is what the `__ronto_alloc_mark`/`_reset` bracket in
  `httpbin/src/index.js` is, and why `hello/` needs no such code.
  [Details](httpbin/README.md#two-heaps).

## Would the component model be simpler?

Only in one place, and it is a real one:

```js
// httpbin: linear memory, by hand (src/index.js)
const mark = exports.__ronto_alloc_mark();
const ptr = exports.__ronto_alloc(bytes.length);
new Uint8Array(exports.memory.buffer, ptr, bytes.length).set(bytes);
const [resultPtr, resultLen] = exports["handle-request"](ptr, bytes.length);
const result = decoder.decode(new Uint8Array(exports.memory.buffer.slice(resultPtr, resultPtr + resultLen)));
exports.__ronto_alloc_reset(mark);

// component: the canonical ABI does all of that
const result = lisp.handleRequest(input);
```

The costs, measured on the identical `httpbin/worker.lisp`: the Worker imports a
generated `.js` beside the `.wasm`, the build needs `@bytecodealliance/jco`, and
the top-level forms move from `_initialize` to instantiation. Two non-obvious
findings behind it:

1. **jco's default output does not run on Workers.** It calls
   `WebAssembly.compile()` on a base64 blob at module scope, and workerd rejects
   the module with `Top-level await in module is unsettled`; `--tla-compat`
   starts but hangs every request. The mode that works is
   `--instantiation sync`, where the glue asks *the host* for each compiled core
   module — exactly a `.wasm` import.
2. **`handler-case` needs `--bindgen-enable-wasm-exnref`**, or jco refuses the
   component outright.

`--no-wasi` is doing quiet work there. Without it a wasm-GC component is a
`wasi:cli/run` command by construction: it imports three WASI interfaces whether
or not the program does any I/O, ships two extra core modules, and its top-level
forms live in a `run` export jco cannot drive. With it the compiler emits a
**reactor component** that imports nothing, and the top level runs from the core
module's start section inside `instantiate`.

## Developing without Cloudflare

The Lisp in every one of these is an ordinary function, so the whole edit/run
loop happens locally: every directory with a handler has a `check.lisp` that
drives it on the interpreter, the JVM and wasmtime. `httpbin/`'s goes one step
further and asserts each answer with
[rove](https://github.com/making/rontolisp/blob/develop/examples/doc/en/guides/testing.md), so it exits non-zero when the handler
drifts — which is why that one needs rove's directories on `--system-path`.
`httpbin-clack-one-source/` needs none — its program IS
`../net/httpbin-clack.lisp`, so its loop is serving that file and `curl`; and
`dog-fetcher/` and `btc-ticker/` cannot have one, because their HTTP client is
an import only a Worker provides. Every other Lisp source here is pinned by
`examples/examples.yaml`:

```bash
./mvnw -Dtest=ExamplesE2eTest -DfailIfNoTests=false \
       -Drontolisp.examples=true -Drontolisp.examples.only=cloudflare test
```

## Deploying

**Eleven of the twelve are deployed to the real edge**, not only run under
`wrangler dev`, and every endpoint in the tables above was checked there with
`curl` — including the 405, the 404, the unparseable body, both `/status`
answers, and `dog-fetcher/`'s outgoing request, whose JSPI bridge needs nothing
on the edge that it did not need locally. `btc-ticker/` is the exception so far:
it has been driven end to end against the real bitFlyer API through its own
generated `worker()` on node 24 JSPI, which is the same code path workerd runs,
but not yet deployed.

Cloudflare budget-checks **Worker Startup Time** at deploy, and `wrangler
deploy` prints it when it has one to report. Going through `clack:clackup`
instead of calling the handler backend directly is what moves that number; the
per-request cost does not follow it. Module scope would be the nicer place to
pay it — that is where the deploy-time check can see it — but a Worker forbids
drawing entropy there, and the seed has to be in before `_initialize` runs the
Lisp top level. So every host here, hand-written or generated, instantiates on
the **first request** instead, and nothing checks the cost at deploy time.

**One gotcha, and it is not your code**: for several minutes after a Worker's
*first* deploy its fresh `*.workers.dev` hostname answers intermittently with
Cloudflare edge errors (`1042`, `1104`, a bare 404) while the route propagates.
The giveaway is that `wrangler tail` shows nothing for those requests. It settles
on its own; re-deploys do not show it.

## Building

Every `build.sh` needs the compiler jar, built once from the repository root
with `./mvnw clean package -DskipTests`. The `.wasm` files are build products
and are not checked in.

A `--no-wasi` build names, per primitive, every refusal its **load path** can
reach — the ones that would otherwise die inside `_initialize` as a bare
`RuntimeError: unreachable`, before any export exists. The line carries the call
chain and the way out (`__ronto_set_time` for the clock, `--host-random` for
entropy); a primitive only the *export* can reach stays quiet, because that one
signals at the call, where `src/index.js` sees it. None of these directories
prints one today.
[Details](https://github.com/making/rontolisp/blob/develop/examples/doc/en/guides/wasm-gc-module.md#what-the-build-tells-you-before-you-run-it).


---

# FILE: references/examples/cloudflare-workers/btc-ticker/README.md

# btc-ticker — the Worker with nothing in it

One outgoing request, one JSON answer: ask
[bitFlyer](https://lightning.bitflyer.com/docs?lang=en) what a bitcoin costs in
yen, and say so. It is the hello world of the **envelope boundary**
(the default; `build.sh` passes no `--host-boundary`), and the whole of its
JavaScript is:

```js
import module from "./worker.wasm";
import { worker } from "./worker.js";

export default worker(module);
```

`src/worker.js` is generated by `--emit-js-glue`;
[`src/index.js`](src/index.js) is those three lines, under a comment header.

```bash
./build.sh          # worker.lisp -> src/worker.wasm + src/worker.js
npx wrangler dev    # http://localhost:8787
npx wrangler deploy
```

```console
$ curl http://localhost:8787/
{"pair":"BTC/JPY","price":10125486.0}
```

## The boundary, and why there is nothing left to write

An HTTP reactor speaks one JSON envelope in each direction — `{method, target,
headers, body, scheme, remote-addr}` in, `{status, headers, body}` out. What
`--host-boundary` decides is whether a **body** rides inside that envelope or
streams beside it through imports of its own.

`envelope` says inside. Everything follows from that:

| | this Worker | [`../dog-fetcher`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/dog-fetcher) (`streaming`, the default) |
| --- | --- | --- |
| Module imports | `env.fetch` | `env.fetch`, `env.readResponseBody`, `env.readRequestBody`, `env.writeResponseBody` |
| Host-side state | none | the upstream reader, the request body, the response chunks, the cursor `lisp.drop` discards |
| `src/index.js` | 3 lines | 3 lines — the glue writes that half on both boundaries |
| A binary body | flattened | crosses exactly |
| A large body | copied | never doubles linear memory |
| A streamed upstream reply | buffered, then forwarded | forwarded chunk at a time |

**This is the DEFAULT boundary** — `build.sh` names no `--host-boundary` at all.
Most Workers read a document and answer one, and there the copy is unmeasurable
while the state it removes is where the bugs were. Ask for
`--host-boundary=streaming` when a body is **binary** (the envelope carries a
body as JSON text, so `ff fe 41` arrives as the seven bytes
`ef bf bd ef bf bd 41` with the `content-length` still saying three), **large**
(linear memory grows with it), or **relayed** from an upstream reply you would
rather forward a chunk at a time — which is [`../dog-fetcher`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/dog-fetcher),
and the five `httpbin*` Workers, all of which say so in their own `build.sh`.

**It is not a size decision.** The same source on the two boundaries lands within
about 1% either way — measured, both raw and gzipped, in
[`size-report/results/cloudflare-workers.md`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/size-report/results/cloudflare-workers.md),
and which way it falls is not stable across compiler changes. What changes is the
amount of *state* on the host side, and every defect this surface's reviews have
turned up was a state-lifetime bug — a reply cursor outliving its source, an
instance bound outside the critical section it is used in. An envelope host has
no cursor to outlive anything.

The host half is written for you either way — `--emit-js-glue` derives it on both
boundaries, so `../dog-fetcher`'s `src/index.js` is three lines too. What this
boundary removes is not JavaScript, it is STATE: there is no reader for the host
to own and no cursor whose lifetime it has to get right.

- **`env.fetch`** — `--host-fetch` states both directions of its envelope
  (rontolisp's own `FetchResponseShape`, pinned by `HostFetchLibraryTest`), so
  its host half is the same twenty lines in every program: call the platform
  `fetch`, answer `{status, headers, body}`, answer `{error}` on a throw. The
  generated file exports it as `defaultHost()`.
- **the reactor envelope** — mapping a `Request` onto it and a `Response` off it
  is transport work. The generated file exports that as `worker(module)`.

Both are **defaults, not replacements**. A host that wants its own supplies it,
entry by entry:

```js
import { worker, suspending } from "./worker.js";

export default worker(module, {
  // Which header carries the client address is the platform's business, so the
  // generated file does not guess. This is the Cloudflare answer.
  remoteAddr: (request) => request.headers.get("cf-connecting-ip"),
  // Laid over defaultHost()'s entries one at a time, so replacing env.fetch
  // does not cost you the rest of `env`.
  host: { env: { fetch: suspending(myOwnFetch) } },
});
```

Everything the streaming boundary is documented to cost is still paid here and
is still worth reading: the JSPI seam, the one-call-at-a-time queue, and the
re-entrancy the two of them create. `worker()` owns all of it —
[`../dog-fetcher`](../dog-fetcher/README.md#one-lisp-call-at-a-time) explains
what it is doing.

## Why the application hands over a future

`rontolisp:await` is legal only inside an `async-defun` / `async-lambda`, and
this Worker has to look *at* the fetched price to choose 200 or 502. So the
response is built in an `async-defun` and `app` — an ordinary Clack function —
returns its **future**, which every rontolisp transport resolves at its own
boundary (the reactor's dispatcher, the socket servers, `wasmtime serve`).

That is the same shape as [`../dog-fetcher`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/dog-fetcher), and there it is
not a choice: tiny-routes generates its route bodies as plain lambdas, where
`await` is a compile error. Here it is one, and the reason to make it is that
anything **wrapping** the application — Clack middleware, a router — would
otherwise be handed the future where it expects the response list.

Nothing stops an `async-defun` from *being* the application when nothing wraps
it (`(rontolisp:async-defun app (env) ... (rontolisp:await ...) ...)`, four
lines shorter, and what the
[Clack guide](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/doc/en/guides/clack.md) shows for a handler). This
directory takes the other one so that the only thing separating it from
`../dog-fetcher` is the boundary.

## The same worker.lisp on every backend

`:server :rontolisp` resolves the transport when the source is read for a
target, and `rontolisp:fetch` follows along — the JDK client, `wasi:http`,
`env.fetch`. All four, from the repo root:

```bash
JAR=target/rontolisp-0.1.0-SNAPSHOT-exec.jar
W=examples/cloudflare-workers/btc-ticker/worker.lisp

# 1. interpreter — a blocking server on :8080
java -jar $JAR $W

# 2. JVM class (keep the jar on the classpath)
java -jar $JAR $W -o BtcTicker.class && java -cp $JAR:. BtcTicker

# 3. WASI component under wasmtime serve
java -jar $JAR $W -o btc-ticker.wasm --component && \
  wasmtime serve -W gc=y -W exceptions=y -S cli=y -S tcp=y -S inherit-network=y btc-ticker.wasm

# 4. the Worker (this directory): build.sh + wrangler dev, as above
```

Off Cloudflare the reactor build is equally drivable from node, under
`node --experimental-wasm-jspi`: `worker()` is a standard fetch handler, so
`worker(module).fetch(new Request("http://x/"))` answers a `Response`. The flag
is needed because the generated `defaultHost()` marks `env.fetch` suspending —
workerd has JSPI unflagged, node does not. A host that would rather answer
synchronously overrides that entry (`worker(module, { host: { env: { fetch } } })`)
and needs no flag, which is what [`../dog-fetcher`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/dog-fetcher) does with its
hand-written half.

## What's in here

| File | Purpose |
| --- | --- |
| [`worker.lisp`](worker.lisp) | The whole program. This is what `build.sh` compiles. |
| [`src/index.js`](src/index.js) | Three lines. |
| [`src/worker.js`](src/worker.js) | The boundary AND the host, GENERATED by `--emit-js-glue` from worker.lisp's declarations. Do not edit; `./build.sh` rewrites it. |
| `src/worker.wasm` | A build product — run `./build.sh` first. |

Module sizes are measured rather than quoted here:
[`size-report/results/cloudflare-workers.md`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/size-report/results/cloudflare-workers.md).

## Limitations

The Worker sandbox and `--no-wasi` limitations of
[`../hello-clack`](../hello-clack/README.md#limitations) apply unchanged, plus
the ones this boundary chooses:

- **A body is a document, not a stream.** Every body is copied through the
  envelope, and a binary one is flattened one character per octet. A Worker that
  proxies uploads, serves images or forwards a streamed reply wants
  `--host-boundary=streaming` — [`../dog-fetcher`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/dog-fetcher) is that.
- **One in-flight request per isolate.** A handler waiting on bitFlyer returns
  to the event loop, and the module refuses a second entry with a trap; the
  generated queue is what turns that into serialisation rather than a failed
  request.
- **Started == settled.** The reactor's fetch future is settled the moment the
  call returns, so `await` never suspends and two fetches never overlap — and
  because the reply arrives whole, a transport failure is always the `fetch`
  call's, never a drain's.

## Rebuilding

```bash
./mvnw clean package -DskipTests   # from the repository root
examples/cloudflare-workers/btc-ticker/build.sh
```


---

# FILE: references/examples/cloudflare-workers/btc-ticker/build.sh

#!/usr/bin/env bash
# Compile worker.lisp to the .wasm module the Worker imports.
#
# --no-wasi: the Worker calls the exported entry point directly, so the module
#   needs no WASI imports -- it becomes a reactor (`_initialize`, not `_start`).
# --host-fetch: rontolisp:fetch is lowered onto the host's own fetch, imported
#   as env.fetch(request-json) -> response-json.
# No --host-boundary: the DEFAULT is `envelope`, and that is what this Worker
#   wants. Every body rides the envelope's own "body" key -- the request's, the
#   response's, and the reply of that fetch -- so there is no env.readRequestBody,
#   no env.writeResponseBody and no env.readResponseBody, and no host-side cursor
#   behind any of them: this module imports exactly ONE function. It pays a copy
#   per body and cannot carry binary, which is nothing for a few hundred bytes of
#   JSON. ../dog-fetcher asks for --host-boundary=streaming because its bodies are
#   not documents; the two are meant to be read together.
# --emit-js-glue: write src/worker.js beside the module. On this boundary both
#   remaining halves are fixed by the transport rather than chosen by the
#   program, so the glue writes BOTH -- the env.fetch host half and a
#   worker(module) that maps a Request onto the envelope and a Response off it.
#   src/index.js is then three lines. It is CHECKED IN and pinned by
#   HostGlueEmitterTest, so regenerate it here rather than editing it.
# --optimize=size: a Worker bundle has a size limit.
#
# The first run downloads clack into ~/.rontolisp/quicklisp; after that the
# build is offline.
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling worker.lisp -> src/worker.wasm + src/worker.js"
java -jar "$jar" "$here/worker.lisp" -o "$here/src/worker.wasm" \
  --no-wasi --host-fetch --optimize=size --emit-js-glue

ls -l "$here/src/worker.wasm" "$here/src/worker.js"
echo "done. Run it with:  npx wrangler dev"


---

# FILE: references/examples/cloudflare-workers/btc-ticker/package.json

{
  "name": "rontolisp-cloudflare-btc-ticker",
  "version": "0.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "build": "./build.sh",
    "dev": "wrangler dev",
    "deploy": "wrangler deploy",
    "tail": "wrangler tail"
  },
  "devDependencies": {
    "wrangler": "^4"
  }
}


---

# FILE: references/examples/cloudflare-workers/btc-ticker/src/index.js

// The whole Worker. `./worker.js` is generated from worker.lisp's own
// declarations (build.sh's --emit-js-glue), and on the ENVELOPE boundary it
// owns all of it: the import object, the (ptr, len) staging, the __ronto_alloc
// bracket, the JSPI wiring, the one-call queue, the env.fetch host half, and
// the Request -> envelope -> Response mapping. Nothing is left to say here.
//
// The other boundary is ../dog-fetcher: there the bodies stream through imports
// of their own -- and its src/index.js is these same three lines, because the
// readers behind those imports are the Request the glue is already holding and
// the Response it is already building. What the boundary changes is what happens
// to a body, not how much host it costs.

import module from "./worker.wasm";
import { worker } from "./worker.js";

export default worker(module);


---

# FILE: references/examples/cloudflare-workers/btc-ticker/src/worker.js

// GENERATED by rontolisp --emit-js-glue -- do not edit.
//
// The host half of this module's boundary, derived from the program's own
// declarations: one import-object entry per rontolisp:wasm-import, one entry
// point per rontolisp:wasm-export, and every piece of linear-memory plumbing
// between them -- the (ptr, len) pair a :string crosses as and the __ronto_alloc
// bracket around a call.
//
// What a declaration cannot state is what a host function DOES, so that is the
// one thing this file asks for: `host` is a plain function per import, keyed by
// import module and field, taking and answering ordinary JavaScript values --
// never a (ptr, len) pair.
//
//   import { instantiate, suspending } from "./worker.js";
//
//   const lisp = instantiate(module, {
//     env: {
//       fetch: (text) => text,   // or leave it to defaultHost()
//     },
//   });
//   await lisp.handleRequest(text);
//
// A host that suspends marks its own entries -- suspending(async (...) => ...)
// -- and every entry point above then answers a promise: the marked imports are
// wrapped in WebAssembly.Suspending, each entry point that can reach one is
// entered through WebAssembly.promising, and calls are serialised onto one
// promise chain, because a suspended module returns to the host's event loop
// and a re-entered export refuses with a trap rather than corrupting both
// calls. Host state that belongs to ONE such call is set inside that section:
//
//   await lisp.serially(async (entry) => { ...; return entry.handleRequest(...) });
//
// This module's boundary is the reactor envelope, so the Request/Response half
// is derivable too and `worker` below is it:
//
//   import module from "./worker.wasm";
//   import { worker } from "./worker.js";
//
//   export default worker(module);
//
// Regenerate it, never edit it: --emit-js-glue on the compile that wrote the
// .wasm beside it.

const encoder = new TextEncoder();
// ignoreBOM, because these octets are a VALUE and not a document: a
// leading U+FEFF is a character the other side chose, and the default
// decoder deletes it -- silently shortening a BOM-prefixed request body
// by one character while the content-length beside it still counts three.
const decoder = new TextDecoder("utf-8", { ignoreBOM: true });

const SUSPENDING = Symbol.for("rontolisp.suspending");

/**
 * Marks a host function as one that answers a promise, so the wasm stack parks
 * until it settles (JSPI). The wrapper is not free -- an import answering
 * SYNCHRONOUSLY through one still returns to the event loop -- so mark only the
 * entries this host really implements asynchronously.
 *
 * @param {Function} fn the host function
 * @returns {object} the marked entry, to be passed as the import
 */
export function suspending(fn) {
  return { [SUSPENDING]: fn };
}

/**
 * Instantiates the module against this host and returns its callable surface.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @param {object} host one function per import, keyed by module and field
 * @returns {object} `exports`, plus one entry point per rontolisp:wasm-export
 */
export function instantiate(module, host = {}) {
  let exports;

  // Every view is built where it is used, never held: growing the module's
  // memory DETACHES the buffer behind every view taken before the growth, and a
  // suspending host resumes after growths it never saw.
  const readString = (ptr, len) =>
    decoder.decode(new Uint8Array(exports.memory.buffer, ptr, len));

  // Bytes the HOST hands over live in the module's own bump allocator, and
  // cross as the (ptr, len) pair every memory-typed value is.
  const write = (octets) => {
    const ptr = exports.__ronto_alloc(octets.length);
    new Uint8Array(exports.memory.buffer, ptr, octets.length).set(octets);
    return [ptr, octets.length];
  };
  const writeString = (value) => write(encoder.encode(String(value)));

  // A host answers a value, or -- from an entry it marked suspending -- a promise
  // of one. Both shapes ride one expression, which is what lets this file drive a
  // synchronous host and a JSPI host without being written twice.
  const marked = new Set();
  const unmarked = (what) =>
    what +
    " answered a promise; wrap it in suspending() so the wasm stack parks" +
    " until it settles";
  const settle = (what, value, next) => {
    if (typeof value?.then === "function") {
      if (!marked.has(what)) {
        // Reported before it is thrown: this throw crosses back into wasm,
        // where a catch_all landing pad turns it into whatever the module
        // makes of a failed import.
        console.error(unmarked(what));
        throw new TypeError(unmarked(what));
      }
      return value.then(next);
    }
    return next(value);
  };

  // The entries the module really imports: --optimize shakes out an import the
  // program never calls, and a host should not have to know which survived.
  const linked = new Set(
    WebAssembly.Module.imports(module).map((i) => i.module + "." + i.name),
  );
  const bind = (moduleName, field, wrap) => {
    const key = moduleName + "." + field;
    const given = (host[moduleName] ?? {})[field];
    if (given == null) {
      if (!linked.has(key)) return undefined;
      throw new TypeError("host." + key + " is missing; this module imports it");
    }
    if (given[SUSPENDING] === undefined) {
      // An async function that was never marked is the one mistake worth
      // catching HERE, where the report is a plain stack and not whatever
      // the module makes of an import that threw.
      if (given.constructor?.name === "AsyncFunction") {
        throw new TypeError(unmarked("host." + key));
      }
      return wrap(key, given);
    }
    marked.add(key);
    return new WebAssembly.Suspending(wrap(key, given[SUSPENDING]));
  };

  const imports = {
    env: {
      // (:string) -> :string
      fetch: bind("env", "fetch", (what, call) => {
        return (p0, p0Len) =>
          settle(what, call(readString(p0, p0Len)), writeString);
      }),
    },
  };
  const instance = new WebAssembly.Instance(module, imports);
  exports = instance.exports;
  // Entropy, BEFORE the top level runs: a --no-wasi module imports none, so its
  // `random` starts from a constant and every instance would draw one sequence.
  // Seeding here also covers the draws a library makes while it LOADS. A Worker
  // forbids this in global scope, so instantiate on the first request.
  exports.__ronto_seed_random(
    new BigUint64Array(crypto.getRandomValues(new Uint8Array(8)).buffer)[0],
  );
  // And a clock, nanoseconds since the Unix epoch, for the same reason and
  // before the same line: until one is set the clock built-ins signal rather
  // than report 1970.
  exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
  exports._initialize();

  // The entry points a call chain can reach a suspending import from -- the
  // list the build prints -- entered through WebAssembly.promising exactly when
  // the host marked one. An unmarked host never pays for the promise.
  const suspends = marked.size !== 0;
  const entry$handleRequest = suspends
    ? WebAssembly.promising(exports["handle-request"])
    : exports["handle-request"];

  // One Lisp call at a time. A suspended call returns to the host's event loop,
  // and the module's own re-entry guard TRAPS a second entry rather than let two
  // calls share its allocator and its dynamic bindings -- so the queue is the
  // contract, not a nicety. `.then(work, work)` because one rejected call must
  // not wedge the chain behind it.
  let queue = Promise.resolve();
  const queued = (work) => {
    const done = queue.then(work, work);
    queue = done.then(
      () => {},
      () => {},
    );
    return done;
  };
  // A bare entry point only needs the queue when a host marked something: a
  // synchronous call cannot be interleaved, and paying a promise for it would
  // make every host asynchronous. `serially` below always takes it, because
  // the work it runs awaits and a second request WOULD land inside it.
  const serialised = (work) => (suspends ? queued(work) : work());

  // One call into the module: stage the arguments, enter, decode the result out
  // of the scratch it sits in, and only THEN pop the arena that scratch is in.
  // A promising entry answers a promise, so the tail rides `then`; a synchronous
  // one runs the same expression inline. `run` is how the call reaches the
  // module -- through the queue, or straight through when it is already inside
  // the one call the queue admits.
  const call = (run, entry, stage, decode) => {
    const work = () => {
      // The module's clock moves only when the host moves it, so move it per
      // call. Not a workaround for a frozen clock -- a Worker's own Date.now()
      // is frozen for the duration of a request as a timing-attack mitigation,
      // so a value that changes once per call is what the platform has.
      exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
      const mark = exports.__ronto_alloc_mark();
      const done = (raw) => {
        const value = decode(raw);
        exports.__ronto_alloc_reset(mark);
        return value;
      };
      // A TRAP skips the pop above and leaves the module's state half-written
      // with it, so a host that keeps serving instantiates again rather than
      // calling back into this instance.
      const raw = entry(...stage());
      return typeof raw?.then === "function" ? raw.then(done) : done(raw);
    };
    return run(work);
  };

  /** `handle-request` -- (:string) -> :string */
  const make$handleRequest =
    (run) =>
    (p0) => {
      return call(
        run,
        entry$handleRequest,
        () => {
          const a0 = writeString(p0);
          return [a0[0], a0[1]];
        },
        ([ptr, len]) => readString(ptr, len),
      );
    };


  // Host state that belongs to ONE call -- what the module pulls DURING it,
  // and what the call leaves behind -- is set and read inside `work`, which
  // runs in the same critical section: a suspended call returns to the event
  // loop, so setting it beside the call instead would let the next request
  // move it under this one. The entry points `work` is handed enter the module
  // directly, because the queue they would take is the one they are in.
  const inside = {
    handleRequest: make$handleRequest((work) => work()),
  };
  const serially = (work) => queued(() => work(inside));

  return {
    exports,
    handleRequest: make$handleRequest(serialised),
    serially,
  };
}

/**
 * The half of this boundary the module's own declarations FIX, ready to hand to
 * `instantiate` -- or to leave to `worker` below, which passes it for you. What
 * a host is still free to do is override it: whatever it supplies wins, entry by
 * entry.
 *
 * @returns {object} the import-object entries this file implements itself
 */
export function defaultHost() {
  return {
    env: {
      fetch: suspending(async (head) => {
        const request = JSON.parse(head);
        try {
          const response = await fetch(request.url, {
            method: request.method,
            headers: request.headers,
            body: request.body,
          });
          return JSON.stringify({
            status: response.status,
            headers: [...response.headers],
            body: await response.text(),
          });
        } catch (error) {
          // The error arm becomes a Lisp condition at the fetch CALL; throwing
          // here would trap the instance instead, and take the request with it.
          return JSON.stringify({ error: String(error) });
        }
      }),
    },
  };
}

/**
 * This module as a fetch handler -- the whole Worker, with nothing left over:
 *
 *   import module from "./worker.wasm";
 *   import { worker } from "./worker.js";
 *
 *   export default worker(module);
 *
 * A Request becomes the envelope the module's entry point takes, the head it
 * answers becomes a Response, and the instance is created on the FIRST REQUEST
 * (a Worker forbids drawing entropy in global scope) and retired if a call ever
 * traps.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @param {object} [options] `host` -- import entries, laid over defaultHost()'s
 *   one at a time; `remoteAddr` -- (request, env, ctx) => the client
 *   address, since which header carries it is the platform's business and not
 *   this file's (`(r) => r.headers.get("cf-connecting-ip")` on Cloudflare)
 * @returns {object} `{ fetch(request, env, ctx) }`
 */
export function worker(module, options = {}) {
  let instance = null;
  // Set when a call TRAPPED: that instance skipped its arena reset and its Lisp
  // state may be half-written, so nothing else may run on it. A Lisp ERROR is not
  // this -- the transport answers 500 itself and the instance is fine.
  let poisoned = false;
  const base = defaultHost();
  const given = options.host ?? {};
  const host = {};
  for (const key of new Set([...Object.keys(base), ...Object.keys(given)])) {
    host[key] = { ...(base[key] ?? {}), ...(given[key] ?? {}) };
  }
  const live = () => {
    if (poisoned) {
      instance = null;
      poisoned = false;
    }
    return (instance ??= instantiate(module, host));
  };

  // The request head. `target` stays RAW -- path and query still joined and
  // still percent-encoded -- because the shared normalizer on the other side
  // owns that split, and a pre-split path leaves the query string nil.
  const envelope = (request, octets, remoteAddr) => {
    const url = new URL(request.url);
    const headers = Object.fromEntries(request.headers);
    // A body with no content-length is a body the request parser does not read,
    // and a chunked request carries none -- so set it from the octets we have.
    if (octets?.length) headers["content-length"] = String(octets.length);
    const head = {
      method: request.method,
      target: url.pathname + url.search,
      headers,
      scheme: url.protocol.replace(":", ""),
    };
    if (octets?.length) head.body = decoder.decode(octets);
    if (remoteAddr != null) head["remote-addr"] = remoteAddr;
    return JSON.stringify(head);
  };

  return {
    // EVERYTHING is inside the try, not just the module call: reading an
    // aborted upload rejects, and `new Response` throws on a status or a
    // header an application is free to produce (0, 999, a newline in a
    // value). Outside it those escape as an unhandled rejection, which the
    // platform answers with its own error page and nothing in the log.
    async fetch(request, env, ctx) {
      let entered = false;
      try {
        const remoteAddr = await options.remoteAddr?.(request, env, ctx);
        const octets = request.body
          ? new Uint8Array(await request.arrayBuffer())
          : null;
        const input = envelope(request, octets, remoteAddr);
        const head = JSON.parse(
          await live().serially((lisp) => {
            // Re-read INSIDE the critical section: the instance was bound at
            // admission, and a call parked ahead of this one can poison it
            // before this one runs. Refusing is the whole point -- a
            // half-unwound instance answers wrong rather than failing, and
            // the module's own re-entry guard is cleared by the landing pad
            // on exactly the path that poisons it.
            if (poisoned) throw new Error("instance discarded by an earlier trap");
            entered = true;
            return lisp.handleRequest(input);
          }),
        );
        // Headers as an ARRAY of pairs, which keeps two Set-Cookie two.
        // An EMPTY body becomes null whichever shape it arrived in: 204/205/304
        // may only be constructed with a null body, and "" and a zero-length
        // Uint8Array are the same response as none.
        const body = head.body;
        return new Response(body?.length ? body : null, {
          status: head.status ?? 200,
          headers: head.headers ?? [],
        });
      } catch (error) {
        console.error("handle-request failed:", error);
        // Only a call that ENTERED the module can have left it half-written.
        // A mapping, or a Response the platform refused, says nothing about
        // the instance, and discarding it would cost the next request a
        // reinstantiation for someone else's bad header.
        if (entered) poisoned = true;
        return new Response("internal error\n", { status: 500 });
      }
    },
  };
}


---

# FILE: references/examples/cloudflare-workers/btc-ticker/worker.lisp

;;; The hello world of the ENVELOPE boundary: one outgoing request, one JSON
;;; answer. Ask bitFlyer what a bitcoin costs in yen, and say so.
;;;
;;; Every body here is a DOCUMENT -- a few hundred bytes of JSON in, a few
;;; dozen out -- so nothing is gained by taking one out of the envelope and
;;; streaming it, and build.sh says exactly that with --host-boundary=envelope.
;;; What that buys is not size (the two boundaries land within about 1% of each
;;; other, either way round) but the BOUNDARY: the module imports one host
;;; function and the host keeps no state at all, so both halves of it are fixed
;;; by the transport and src/index.js is three lines. ../dog-fetcher is the same
;;; program shape on the streaming boundary -- read the two together.
;;;
;;; :server :rontolisp picks the transport per target at read time, so THIS ONE
;;; SOURCE runs on every backend (the README has the commands).

(ql:quickload "clack")

;; bitFlyer answers {"product_code": "BTC_JPY", "ltp": 16000000.0, ...} --
;; `ltp' is the last traded price, which is what "the price" means here.
(rontolisp:async-defun ticker ()
  (let* ((res
          (rontolisp:await
           (rontolisp:fetch
            "https://api.bitflyer.com/v1/ticker?product_code=BTC_JPY")))
         (body (rontolisp:await (rontolisp:read-all (getf res :body)))))
    (if (eql (getf res :status) 200)
        (gethash "ltp" (rontolisp:json-parse body))
        nil)))

(defun json-response (status obj)
  (list status '(:content-type "application/json")
        (list (format nil "~a~%" (rontolisp:json-stringify obj)))))

;; Awaiting needs an async frame, so the response is built in one -- and the
;; application below just hands its FUTURE over, which the reactor transport
;; resolves at the boundary. ../dog-fetcher is shaped this way because
;; tiny-routes generates its route bodies as plain lambdas, where `await' is a
;; compile error; here it is a choice, and the reason to make it is that
;; anything WRAPPING the application (Clack middleware, a router) would
;; otherwise be handed the future where it expects the response list.
(rontolisp:async-defun ticker-response ()
  (let ((price (rontolisp:await (ticker))))
    (if price
        (json-response 200
         (rontolisp:plist-hash-table (list :pair "BTC/JPY" :price price)))
        (json-response 502
                       (rontolisp:plist-hash-table
                        (list :error "the bitFlyer API did not answer"))))))

;; A Clack application is a function of the request environment. This one
;; ignores it: there is one endpoint and it takes no arguments.
(defun app (env)
  (declare (ignore env))
  (ticker-response))

(clack:clackup #'app :server :rontolisp :port 8080 :use-thread nil)


---

# FILE: references/examples/cloudflare-workers/btc-ticker/wrangler.jsonc

{
  // https://developers.cloudflare.com/workers/wrangler/configuration/
  "name": "rontolisp-btc-ticker",
  "main": "src/index.js",
  "compatibility_date": "2026-01-01",

  // `import module from "./worker.wasm"` needs no configuration: .wasm is a
  // built-in module rule for ES-module Workers, and the file is compiled at
  // deploy time rather than on the request path.
  "observability": {
    "enabled": true
  }
}


---

# FILE: references/examples/cloudflare-workers/dog-fetcher/README.md

# dog-fetcher — a Worker that calls out

[`../../net/dog-fetcher.lisp`](../../net/dog-fetcher.lisp)'s proxy shape on
Cloudflare: every request asks [dog.ceo](https://dog.ceo) for a picture and
answers with JSON. It is the first Worker here that does **outgoing** HTTP, and
the client is `rontolisp:fetch` itself — the same `(rontolisp:await
(rontolisp:fetch ...))` that runs on the interpreter, the JVM and a `wasi:http`
component. A `--no-wasi` reactor imports no WASI, so `--host-fetch` lowers the
call onto what a Worker host can always provide: its own `fetch`.

Routes come from [tiny-routes](https://github.com/jeko2000/tiny-routes), loaded
as `tiny-routes/lite` exactly as in [`../hello-tiny-routes`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/hello-tiny-routes),
and the application is served with `:server :rontolisp` — the backend that
picks its transport **per target at read time** — so this one `worker.lisp`
runs on every backend, not only on Cloudflare (see
[the same worker.lisp on every backend](#the-same-workerlisp-on-every-backend)).

```bash
./build.sh          # worker.lisp -> src/worker.wasm + src/worker.js
npx wrangler dev    # http://localhost:8787
npx wrangler deploy
```

```console
$ curl http://localhost:8787/
{"dog":"https://images.dog.ceo/breeds/mix/annabelle0.jpg"}

$ curl http://localhost:8787/breed/husky
{"dog":"https://images.dog.ceo/breeds/husky/n02110185_7246.jpg","breed":"husky"}

$ curl -i http://localhost:8787/breed/unicorn
HTTP/1.1 404 Not Found
Content-Type: application/json

{"error":"no such breed"}

$ curl -i http://localhost:8787/anything
HTTP/1.1 404 Not Found
Content-Type: application/json

{"error":"no route for /anything"}
```

`/breed/123` is answered by a different mechanism: the route DECLINES a breed
that is not letters before it can reach a URL, so the catch-all answers it
rather than the upstream. A request that never completes is a 502 — the only
case that is not the upstream's own answer.

## The boundary

No hand-written import and no bespoke envelope: `--host-fetch` (build.sh)
injects two imports and lowers every `rontolisp:fetch` onto them —
`env.fetch(request-json) -> response-head-json` for the request and the reply's
head, and `env.readResponseBody(ptr, cap) -> i32` for the reply's body, pulled a
chunk at a time. Same options, same
`(:status  :headers  :body )` answer as every other backend.
Beside them the reactor transport declares two more,
`env.readRequestBody` / `env.writeResponseBody`, so the request and response
bodies leave the envelope the same way. **That is what
`--host-boundary=streaming` (the default) means**, and it is the whole difference
to [`../btc-ticker`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/btc-ticker), which declines it.

**And the JavaScript half is not hand-written either — none of it.**
`--emit-js-glue` (build.sh) writes [`src/worker.js`](src/worker.js) from the
same declarations the module was built from: the import object, the
`(ptr, len)` staging both ways, the `__ronto_alloc` bracket, the
`WebAssembly.Suspending` wrappers, the `WebAssembly.promising` entry, the
one-call-at-a-time queue, **all four host functions**, and the
`Request -> envelope -> Response` mapping over them. It is generated and checked
in, and pinned by `HostGlueEmitterTest`. [`src/index.js`](src/index.js) is:

```js
import module from "./worker.wasm";
import { worker } from "./worker.js";

export default worker(module);
```

The four host functions look like the one thing a declaration cannot state —
what a host *does* — and for a caller of `instantiate` they are. Inside
`worker()` they are not: the octets a request body comes from are the `Request`
it is already holding, the response chunks are the `Response` it is already
building, and the reply body is the `fetch` its own `defaultHost()` just made.
A host that wants any of them back supplies it, laid over the derived entry:

```js
export default worker(module, {
  remoteAddr: (request) => request.headers.get("cf-connecting-ip"),
  host: { env: { fetch: suspending(myOwnFetch) } },
});
```

Four things are worth reading twice:

- **A wasm import is a synchronous call and `fetch` is a promise.** JSPI
  (`WebAssembly.Suspending` on the import, `WebAssembly.promising` on the
  export) is what joins them: the whole wasm stack parks until the promise
  settles and resumes with the result. **workerd runs it with no flag and no
  compatibility date opt-in** — verified under `wrangler dev` and on the
  deployed edge. A suspending import may only be called on a stack entered
  through `promising`, so `_initialize` must never reach one; the build prints
  exactly this obligation, and would print a warning line naming any fetch its
  load path reaches. Which entries actually suspend is the HOST's answer, not
  the module's: the generated file marks the two that answer promises and leaves
  the other two plain calls, because the wrapper is not free — an import that
  answers *synchronously* through one still parks the stack and returns to the
  event loop.
- **Awaiting still reads the same.** On the reactor the future `fetch` returns
  is settled the moment the call returns (the stack was parked for the round
  trip to the headers), so `await` never suspends and two fetches never
  overlap — `dog-image` is an ordinary `async-defun`, and the route bodies
  (synchronous tiny-routes functions, where `await` is not legal) simply return
  its FUTURE: the reactor transport resolves a future-valued response at its
  boundary.
- **The body is not in the head.** `env.fetch` answers status and headers; the
  octets come through `env.readResponseBody` as `read-all` asks for them, so a
  large reply never becomes a JSON string and a binary one crosses as the
  octets it is — three bytes `ff fe 41` arrive as three. The envelope boundary
  cannot do that (it would make them seven), which is the reason to be on this
  one. What it costs in exchange is stated below: a failure *during* the body
  surfaces at the drain, and only one reply body is live at a time.
- **A `:string` result is host-written bytes.** The host allocates with the
  module's exported `__ronto_alloc` and returns `[ptr, len]`; the per-request
  arena reset frees it along with everything else. Nothing may be kept across
  the `await` — growing the module's memory detaches `memory.buffer`. All of
  that is `src/worker.js`'s job: a host function that replaces one of the
  derived entries takes and answers plain JavaScript values and never sees a
  pointer.

## One Lisp call at a time

Everywhere else in these examples the Lisp call is synchronous and an isolate
cannot interleave requests inside it. Suspending changes that: a handler waiting
on dog.ceo returns control to the event loop, and a second request would enter
the same module — the same globals, and the same bump allocator whose
mark/reset bracket assumes it is alone. The module refuses that entry with a
trap (the compiled export carries a re-entry guard), so a host that forgets the
queue sees a failed request, not silently corrupted answers. The generated glue
owns that queue AND everything that has to sit inside it: the request body and
the response chunks belong to the one call that is running, and a suspended
handler would otherwise let the next request move them under it. It also
re-checks, inside that section, whether the instance an earlier parked call has
since trapped — because binding the instance happens when a request is admitted,
and the trap happens later.

The cost is real and measured: eight concurrent `GET /` under `wrangler dev`
complete about 250 ms apart rather than together. On the deployed edge the same
eight finish within 0.27–0.98 s, because the queue is per isolate and Cloudflare
is free to use more than one. It buys correctness, and it costs the isolate
nothing else — everything that is not this module keeps running while a handler
waits.

## The same worker.lisp on every backend

`:server :rontolisp` resolves the transport when the source is read for a
target: a real socket on the interpreter/JVM, `wasi:http` under `--component`,
and the host-driven `handle-request` export on a `--no-wasi` reactor (the
`#+rontolisp-reactor` leg). `rontolisp:fetch` follows along — the JDK client,
`wasi:http/client`, `env.fetch`. All four, from the repo root:

```bash
JAR=target/rontolisp-0.1.0-SNAPSHOT-exec.jar
W=examples/cloudflare-workers/dog-fetcher/worker.lisp

# 1. interpreter — a blocking server on :8080
java -jar $JAR $W

# 2. JVM class (keep the jar on the classpath)
java -jar $JAR $W -o DogFetcher.class && java -cp $JAR:. DogFetcher

# 3. WASI component under wasmtime serve (the socket flags: clack's own
#    socket leg keeps wasi:sockets in the import surface)
java -jar $JAR $W -o dog-fetcher.wasm --component && \
  wasmtime serve -W gc=y -W exceptions=y -S cli=y -S tcp=y -S inherit-network=y dog-fetcher.wasm

# 4. the Worker (this directory): build.sh + wrangler dev, as above
```

Off Cloudflare the reactor build is equally drivable from plain node: the
module's client is `env.fetch`, and node has no JSPI, so a node host answers
*synchronously* — which the boundary equally allows. The sibling
`../../net/dog-fetcher.lisp` (the `rontolisp:http-handler` spelling of the same
program) compiles to the same reactor shape with no edit at all.

## What's in here

| File | Purpose |
| --- | --- |
| [`worker.lisp`](worker.lisp) | The whole program. This is what `build.sh` compiles. |
| [`src/index.js`](src/index.js) | Three lines over the generated `worker()`. |
| [`src/worker.js`](src/worker.js) | The boundary, GENERATED by `--emit-js-glue` from worker.lisp's declarations. Do not edit; `./build.sh` rewrites it. |
| `src/worker.wasm` | A build product — run `./build.sh` first. |

## Limitations

The Worker sandbox and `--no-wasi` limitations of
[`../hello-clack`](../hello-clack/README.md#limitations) apply unchanged, plus:

- **One in-flight request per isolate**, as above. Overlapping them is what
  `--reentrant` buys — the module then owns its per-call state, and the body
  imports carry a call id — and [`../dog-relay`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/dog-relay) is this boundary
  with that flag; this directory deliberately stays serialised, as the
  controlled comparison with `../btc-ticker`.
- **Started == settled, and settled means the HEAD.** The reactor's fetch
  future is settled at creation (the host call blocked the stack until the
  headers), so two fetches never overlap and a transport failure *before* the
  head signals at the `fetch` call rather than at `await` — the documented
  degenerate-async shape of the Preview 1 backend. A failure *during* the body
  signals at the drain instead, as on every other backend.
- **One live reply body.** The host has one read cursor, moved by each
  `env.fetch`, so starting the next fetch before draining the previous reply
  makes that drain signal rather than answer the new reply's octets.

## Rebuilding

```bash
./mvnw clean package -DskipTests   # from the repository root
examples/cloudflare-workers/dog-fetcher/build.sh
```


---

# FILE: references/examples/cloudflare-workers/dog-fetcher/build.sh

#!/usr/bin/env bash
# Compile worker.lisp to the .wasm module the Worker imports.
#
# --no-wasi: the Worker calls the exported entry point directly, so the module
#   needs no WASI imports -- it becomes a reactor (`_initialize`, not `_start`).
# --host-fetch: rontolisp:fetch is lowered onto the host's own fetch, imported
#   as env.fetch(request-json) -> response-head-json.
# --host-boundary=streaming: the bodies leave the JSON envelope and cross as
#   octets through imports of their own -- env.readRequestBody,
#   env.writeResponseBody, and env.readResponseBody for the reply of that fetch.
#   ASKED FOR, because the default is `envelope`: this Worker relays an upstream
#   reply and wants it forwarded a chunk at a time rather than held whole, and
#   the envelope would carry a body as JSON text, which no binary survives.
#   ../btc-ticker is the same program shape on the default boundary.
# --emit-js-glue: write src/worker.js beside the module -- the import object,
#   the linear-memory plumbing, the Suspending/promising wiring and the one-call
#   queue, all derived from the same declarations the module was built from.
#   src/index.js is then only what a declaration cannot say: what each host
#   function does. It is CHECKED IN and pinned by HostGlueEmitterTest, so
#   regenerate it here rather than editing it.
# --optimize=size: a Worker bundle has a size limit; tiny-routes/lite is what
#   keeps cl-ppcre out of what the tree-shaker has to keep.
#
# The first run downloads clack/lack/tiny-routes into ~/.rontolisp/quicklisp;
# after that the build is offline.
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling worker.lisp -> src/worker.wasm + src/worker.js"
java -jar "$jar" "$here/worker.lisp" -o "$here/src/worker.wasm" \
  --no-wasi --host-fetch --host-boundary=streaming --optimize=size --emit-js-glue

ls -l "$here/src/worker.wasm" "$here/src/worker.js"
echo "done. Run it with:  npx wrangler dev"


---

# FILE: references/examples/cloudflare-workers/dog-fetcher/package.json

{
  "name": "rontolisp-cloudflare-dog-fetcher",
  "version": "0.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "build": "./build.sh",
    "dev": "wrangler dev",
    "deploy": "wrangler deploy",
    "tail": "wrangler tail"
  },
  "devDependencies": {
    "wrangler": "^4"
  }
}


---

# FILE: references/examples/cloudflare-workers/dog-fetcher/src/index.js

// The whole Worker. `./worker.js` is generated from worker.lisp's own
// declarations (build.sh's --emit-js-glue) and owns all of it: the import
// object, the (ptr, len) staging, the __ronto_alloc bracket, the JSPI wiring,
// the one-call-at-a-time queue, the four host functions this boundary declares
// -- env.fetch and the three body imports -- and the Request -> envelope ->
// Response mapping.
//
// That the STREAMING boundary needs no more of a host than ../btc-ticker's
// envelope one is the point of the pair: the bodies here cross as octets
// through imports of their own, and where they come from is the Request this
// file already has and the Response it is already building, so the glue writes
// that too. What the boundary buys is in the README: a binary body crossing
// exactly, a large one never doubling linear memory, an upstream reply
// forwarded a chunk at a time.
//
// To take any of it over, hand `worker` a host: whatever it supplies is laid
// over the derived entries one at a time.
//
//   import { worker, suspending } from "./worker.js";
//   export default worker(module, {
//     remoteAddr: (request) => request.headers.get("cf-connecting-ip"),
//     host: { env: { fetch: suspending(myOwnFetch) } },
//   });

import module from "./worker.wasm";
import { worker } from "./worker.js";

export default worker(module);


---

# FILE: references/examples/cloudflare-workers/dog-fetcher/src/worker.js

// GENERATED by rontolisp --emit-js-glue -- do not edit.
//
// The host half of this module's boundary, derived from the program's own
// declarations: one import-object entry per rontolisp:wasm-import, one entry
// point per rontolisp:wasm-export, and every piece of linear-memory plumbing
// between them -- the (ptr, len) pair a :string crosses as, the __ronto_alloc
// bracket around a call, and the read(2) cursor a :bytes result is pulled
// through.
//
// What a declaration cannot state is what a host function DOES, so that is the
// one thing this file asks for: `host` is a plain function per import, keyed by
// import module and field, taking and answering ordinary JavaScript values --
// never a (ptr, len) pair, and, where an entry answers `chunk` below, a
// Uint8Array or a string with null for the end of them.
//
//   import { instantiate, suspending } from "./worker.js";
//
//   const lisp = instantiate(module, {
//     env: {
//       fetch: (text) => text,   // or leave it to defaultHost()
//       readResponseBody: () => chunk,   // or leave it to defaultHost()
//       readRequestBody: () => chunk,   // or leave it to worker()
//       writeResponseBody: (chunk) => {},   // or leave it to worker()
//     },
//   });
//   await lisp.handleRequest(text);
//
// A host that suspends marks its own entries -- suspending(async (...) => ...)
// -- and every entry point above then answers a promise: the marked imports are
// wrapped in WebAssembly.Suspending, each entry point that can reach one is
// entered through WebAssembly.promising, and calls are serialised onto one
// promise chain, because a suspended module returns to the host's event loop
// and a re-entered export refuses with a trap rather than corrupting both
// calls. Host state that belongs to ONE such call is set inside that section:
//
//   await lisp.serially(async (entry) => { ...; return entry.handleRequest(...) });
//
// This module's boundary is the reactor envelope, so the Request/Response half
// is derivable too and `worker` below is it:
//
//   import module from "./worker.wasm";
//   import { worker } from "./worker.js";
//
//   export default worker(module);
//
// Regenerate it, never edit it: --emit-js-glue on the compile that wrote the
// .wasm beside it.

const encoder = new TextEncoder();
// ignoreBOM, because these octets are a VALUE and not a document: a
// leading U+FEFF is a character the other side chose, and the default
// decoder deletes it -- silently shortening a BOM-prefixed request body
// by one character while the content-length beside it still counts three.
const decoder = new TextDecoder("utf-8", { ignoreBOM: true });

const SUSPENDING = Symbol.for("rontolisp.suspending");

/**
 * Marks a host function as one that answers a promise, so the wasm stack parks
 * until it settles (JSPI). The wrapper is not free -- an import answering
 * SYNCHRONOUSLY through one still returns to the event loop -- so mark only the
 * entries this host really implements asynchronously.
 *
 * @param {Function} fn the host function
 * @returns {object} the marked entry, to be passed as the import
 */
export function suspending(fn) {
  return { [SUSPENDING]: fn };
}

/**
 * Instantiates the module against this host and returns its callable surface.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @param {object} host one function per import, keyed by module and field
 * @returns {object} `exports`, plus one entry point per rontolisp:wasm-export
 */
export function instantiate(module, host = {}) {
  let exports;

  // Every view is built where it is used, never held: growing the module's
  // memory DETACHES the buffer behind every view taken before the growth, and a
  // suspending host resumes after growths it never saw.
  const readString = (ptr, len) =>
    decoder.decode(new Uint8Array(exports.memory.buffer, ptr, len));

  // Octets, COPIED: the module pops the staging behind the pointer the moment
  // the call returns, so a chunk not taken by then is one the host never gets.
  const readBytes = (ptr, len) =>
    new Uint8Array(exports.memory.buffer.slice(ptr, ptr + len));

  // Bytes the HOST hands over live in the module's own bump allocator, and
  // cross as the (ptr, len) pair every memory-typed value is.
  const write = (octets) => {
    const ptr = exports.__ronto_alloc(octets.length);
    new Uint8Array(exports.memory.buffer, ptr, octets.length).set(octets);
    return [ptr, octets.length];
  };
  const writeString = (value) => write(encoder.encode(String(value)));
  const octets = (chunk) =>
    typeof chunk === "string" ? encoder.encode(chunk) : chunk;

  // A host answers a value, or -- from an entry it marked suspending -- a promise
  // of one. Both shapes ride one expression, which is what lets this file drive a
  // synchronous host and a JSPI host without being written twice.
  const marked = new Set();
  const unmarked = (what) =>
    what +
    " answered a promise; wrap it in suspending() so the wasm stack parks" +
    " until it settles";
  const settle = (what, value, next) => {
    if (typeof value?.then === "function") {
      if (!marked.has(what)) {
        // Reported before it is thrown: this throw crosses back into wasm,
        // where a catch_all landing pad turns it into whatever the module
        // makes of a failed import.
        console.error(unmarked(what));
        throw new TypeError(unmarked(what));
      }
      return value.then(next);
    }
    return next(value);
  };

  // The entries the module really imports: --optimize shakes out an import the
  // program never calls, and a host should not have to know which survived.
  const linked = new Set(
    WebAssembly.Module.imports(module).map((i) => i.module + "." + i.name),
  );
  const bind = (moduleName, field, wrap) => {
    const key = moduleName + "." + field;
    const given = (host[moduleName] ?? {})[field];
    if (given == null) {
      if (!linked.has(key)) return undefined;
      throw new TypeError("host." + key + " is missing; this module imports it");
    }
    if (given[SUSPENDING] === undefined) {
      // An async function that was never marked is the one mistake worth
      // catching HERE, where the report is a plain stack and not whatever
      // the module makes of an import that threw.
      if (given.constructor?.name === "AsyncFunction") {
        throw new TypeError(unmarked("host." + key));
      }
      return wrap(key, given);
    }
    marked.add(key);
    return new WebAssembly.Suspending(wrap(key, given[SUSPENDING]));
  };

  // A :bytes RESULT is the read(2) shape: the MODULE owns the buffer and asks
  // for up to `cap` octets, so what a host answers is the next CHUNK and this
  // holds whatever did not fit. That remainder is the read side's only state,
  // and it is why a host supplies chunks rather than a reader -- which source
  // they come from (a ReadableStream, a Uint8Array) is all that is left to it.
  const readers = new Map();
  const reader = (what, source) => {
    let rest = null;
    let from = null;
    // A body the module did not drain belongs to the call that could have and
    // to no other, so every cursor is dropped at the next entry below -- and a
    // host whose SOURCE moves inside one call (a new upstream reply opened by
    // another import) drops this one itself with lisp.drop(key), because what
    // did not fit is held here and nothing else can see the source move.
    readers.set(what, () => {
      rest = null;
      from = null;
    });
    const drain = (ptr, cap) => {
      const n = Math.min(cap, rest.length);
      new Uint8Array(exports.memory.buffer, ptr, n).set(rest.subarray(0, n));
      rest = rest.subarray(n);
      return n;
    };
    // A read that FAILS answers a NEGATIVE count. Throwing would trap the
    // instance; the count is an error channel the module turns into a Lisp
    // condition where the octets are consumed, which is where every other
    // backend reports a transfer that broke mid-body.
    const failed = (error) => {
      console.error(what + " failed:", error);
      return -1;
    };
    return (args, ptr, cap) => {
      // The remainder belongs to the arguments that asked for it: a source
      // selected by argument must not be served the previous one's octets.
      const key = JSON.stringify(args);
      if (from !== key) {
        rest = null;
        from = key;
      }
      if (rest !== null && rest.length !== 0) return drain(ptr, cap);
      try {
        const answer = settle(what, source(...args), (chunk) => {
          rest = chunk == null ? new Uint8Array(0) : octets(chunk);
          return rest.length === 0 ? 0 : drain(ptr, cap);
        });
        return typeof answer?.then === "function"
          ? answer.then(undefined, failed)
          : answer;
      } catch (error) {
        return failed(error);
      }
    };
  };

  // What a read import left over, thrown away on demand. A host calls it when
  // the SOURCE behind that import moves under it INSIDE one call -- a new
  // upstream reply, say -- since the remainder is held above and nothing else
  // can see the source move. With no argument it drops every one of them.
  const drop = (key) =>
    key === undefined ? readers.forEach((f) => f()) : readers.get(key)?.();

  const imports = {
    env: {
      // (:string) -> :string
      fetch: bind("env", "fetch", (what, call) => {
        return (p0, p0Len) =>
          settle(what, call(readString(p0, p0Len)), writeString);
      }),
      // () -> :bytes
      readResponseBody: bind("env", "readResponseBody", (what, call) => {
        const read = reader(what, call);
        return (ptr, cap) => read([], ptr, cap);
      }),
      // () -> :bytes
      readRequestBody: bind("env", "readRequestBody", (what, call) => {
        const read = reader(what, call);
        return (ptr, cap) => read([], ptr, cap);
      }),
      // (:bytes) -> :void
      writeResponseBody: bind("env", "writeResponseBody", (what, call) => {
        return (p0, p0Len) =>
          settle(what, call(readBytes(p0, p0Len)), () => undefined);
      }),
    },
  };
  const instance = new WebAssembly.Instance(module, imports);
  exports = instance.exports;
  // Entropy, BEFORE the top level runs: a --no-wasi module imports none, so its
  // `random` starts from a constant and every instance would draw one sequence.
  // Seeding here also covers the draws a library makes while it LOADS. A Worker
  // forbids this in global scope, so instantiate on the first request.
  exports.__ronto_seed_random(
    new BigUint64Array(crypto.getRandomValues(new Uint8Array(8)).buffer)[0],
  );
  // And a clock, nanoseconds since the Unix epoch, for the same reason and
  // before the same line: until one is set the clock built-ins signal rather
  // than report 1970.
  exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
  exports._initialize();

  // The entry points a call chain can reach a suspending import from -- the
  // list the build prints -- entered through WebAssembly.promising exactly when
  // the host marked one. An unmarked host never pays for the promise.
  const suspends = marked.size !== 0;
  const entry$handleRequest = suspends
    ? WebAssembly.promising(exports["handle-request"])
    : exports["handle-request"];

  // One Lisp call at a time. A suspended call returns to the host's event loop,
  // and the module's own re-entry guard TRAPS a second entry rather than let two
  // calls share its allocator and its dynamic bindings -- so the queue is the
  // contract, not a nicety. `.then(work, work)` because one rejected call must
  // not wedge the chain behind it.
  let queue = Promise.resolve();
  const queued = (work) => {
    const done = queue.then(work, work);
    queue = done.then(
      () => {},
      () => {},
    );
    return done;
  };
  // A bare entry point only needs the queue when a host marked something: a
  // synchronous call cannot be interleaved, and paying a promise for it would
  // make every host asynchronous. `serially` below always takes it, because
  // the work it runs awaits and a second request WOULD land inside it.
  const serialised = (work) => (suspends ? queued(work) : work());

  // One call into the module: stage the arguments, enter, decode the result out
  // of the scratch it sits in, and only THEN pop the arena that scratch is in.
  // A promising entry answers a promise, so the tail rides `then`; a synchronous
  // one runs the same expression inline. `run` is how the call reaches the
  // module -- through the queue, or straight through when it is already inside
  // the one call the queue admits.
  const call = (run, entry, stage, decode) => {
    const work = () => {
      readers.forEach((drop) => drop());
      // The module's clock moves only when the host moves it, so move it per
      // call. Not a workaround for a frozen clock -- a Worker's own Date.now()
      // is frozen for the duration of a request as a timing-attack mitigation,
      // so a value that changes once per call is what the platform has.
      exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
      const mark = exports.__ronto_alloc_mark();
      const done = (raw) => {
        const value = decode(raw);
        exports.__ronto_alloc_reset(mark);
        return value;
      };
      // A TRAP skips the pop above and leaves the module's state half-written
      // with it, so a host that keeps serving instantiates again rather than
      // calling back into this instance.
      const raw = entry(...stage());
      return typeof raw?.then === "function" ? raw.then(done) : done(raw);
    };
    return run(work);
  };

  /** `handle-request` -- (:string) -> :string */
  const make$handleRequest =
    (run) =>
    (p0) => {
      return call(
        run,
        entry$handleRequest,
        () => {
          const a0 = writeString(p0);
          return [a0[0], a0[1]];
        },
        ([ptr, len]) => readString(ptr, len),
      );
    };


  // Host state that belongs to ONE call -- what the module pulls DURING it,
  // and what the call leaves behind -- is set and read inside `work`, which
  // runs in the same critical section: a suspended call returns to the event
  // loop, so setting it beside the call instead would let the next request
  // move it under this one. The entry points `work` is handed enter the module
  // directly, because the queue they would take is the one they are in.
  const inside = {
    handleRequest: make$handleRequest((work) => work()),
  };
  const serially = (work) => queued(() => work(inside));

  return {
    exports,
    handleRequest: make$handleRequest(serialised),
    drop,
    serially,
  };
}

/**
 * The half of this boundary the module's own declarations FIX, ready to hand to
 * `instantiate` -- or to leave to `worker` below, which passes it for you. What
 * a host is still free to do is override it: whatever it supplies wins, entry by
 * entry.
 *
 * @param {Function} lisp a thunk answering the instantiated object, or null
 *   before there is one. Only the reply-body cursor needs it: a second fetch
 *   inside ONE call REPLACES the reply this file is reading, and the octets
 *   the glue is still holding belong to a reply nobody may read again --
 *   which only this side can see.
 * @returns {object} the import-object entries this file implements itself
 */
export function defaultHost(lisp) {
  // The reply this file is currently reading. The generated cursor holds what
  // of a chunk did not fit; this is only WHERE the octets come from.
  let upstream = null;
  return {
    env: {
      fetch: suspending(async (head) => {
        const request = JSON.parse(head);
        upstream = null;
        lisp?.()?.drop("env.readResponseBody");
        try {
          const response = await fetch(request.url, {
            method: request.method,
            headers: request.headers,
            body: request.body,
          });
          // The reader IS the body; the module pulls it after this returns.
          upstream = response.body ? response.body.getReader() : null;
          return JSON.stringify({
            status: response.status,
            headers: [...response.headers],
          });
        } catch (error) {
          // The error arm becomes a Lisp condition at the fetch CALL; throwing
          // here would trap the instance instead, and take the request with it.
          return JSON.stringify({ error: String(error) });
        }
      }),
      readResponseBody: suspending(
        // The next chunk of the reply the last fetch opened, null at the end
        // of it. Reading a ReadableStream is asynchronous, so this one really
        // does suspend; a read that THROWS becomes the negative count the
        // module signals at the drain, which the glue answers on our behalf.
        async () => {
          if (!upstream) return null;
          const { value, done } = await upstream.read();
          if (done) {
            upstream = null;
            return null;
          }
          return value;
        },
      ),
    },
  };
}

/**
 * This module as a fetch handler -- the whole Worker, with nothing left over:
 *
 *   import module from "./worker.wasm";
 *   import { worker } from "./worker.js";
 *
 *   export default worker(module);
 *
 * A Request becomes the envelope the module's entry point takes, the head it
 * answers becomes a Response, and the instance is created on the FIRST REQUEST
 * (a Worker forbids drawing entropy in global scope) and retired if a call ever
 * traps.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @param {object} [options] `host` -- import entries, laid over defaultHost()'s
 *   one at a time; `remoteAddr` -- (request, env, ctx) => the client
 *   address, since which header carries it is the platform's business and not
 *   this file's (`(r) => r.headers.get("cf-connecting-ip")` on Cloudflare)
 * @returns {object} `{ fetch(request, env, ctx) }`
 */
export function worker(module, options = {}) {
  let instance = null;
  // Set when a call TRAPPED: that instance skipped its arena reset and its Lisp
  // state may be half-written, so nothing else may run on it. A Lisp ERROR is not
  // this -- the transport answers 500 itself and the instance is fine.
  let poisoned = false;
  // The request body the module pulls, and the response body coming back the
  // same way. Both belong to the ONE call running below, which is where they
  // are set.
  let requestBody = null;
  let responseChunks = [];
  const collected = () => {
    const all = new Uint8Array(
      responseChunks.reduce((n, chunk) => n + chunk.length, 0),
    );
    let at = 0;
    for (const chunk of responseChunks) {
      all.set(chunk, at);
      at += chunk.length;
    }
    return all;
  };
  const base = defaultHost(() => instance);
  base.env = {
    ...(base.env ?? {}),
    readRequestBody: () => {
      // Handed over ONCE: a chunk source that never answers null is one the
      // module pulls forever.
      const chunk = requestBody;
      requestBody = null;
      return chunk;
    },
    writeResponseBody: (chunk) => responseChunks.push(chunk),
  };
  const given = options.host ?? {};
  const host = {};
  for (const key of new Set([...Object.keys(base), ...Object.keys(given)])) {
    host[key] = { ...(base[key] ?? {}), ...(given[key] ?? {}) };
  }
  const live = () => {
    if (poisoned) {
      instance = null;
      poisoned = false;
    }
    return (instance ??= instantiate(module, host));
  };

  // The request head. `target` stays RAW -- path and query still joined and
  // still percent-encoded -- because the shared normalizer on the other side
  // owns that split, and a pre-split path leaves the query string nil.
  const envelope = (request, octets, remoteAddr) => {
    const url = new URL(request.url);
    const headers = Object.fromEntries(request.headers);
    // A body with no content-length is a body the request parser does not read,
    // and a chunked request carries none -- so set it from the octets we have.
    if (octets?.length) headers["content-length"] = String(octets.length);
    const head = {
      method: request.method,
      target: url.pathname + url.search,
      headers,
      scheme: url.protocol.replace(":", ""),
    };
    if (remoteAddr != null) head["remote-addr"] = remoteAddr;
    return JSON.stringify(head);
  };

  return {
    // EVERYTHING is inside the try, not just the module call: reading an
    // aborted upload rejects, and `new Response` throws on a status or a
    // header an application is free to produce (0, 999, a newline in a
    // value). Outside it those escape as an unhandled rejection, which the
    // platform answers with its own error page and nothing in the log.
    async fetch(request, env, ctx) {
      let entered = false;
      try {
        const remoteAddr = await options.remoteAddr?.(request, env, ctx);
        const octets = request.body
          ? new Uint8Array(await request.arrayBuffer())
          : null;
        const input = envelope(request, octets, remoteAddr);
        const head = JSON.parse(
          await live().serially((lisp) => {
            // Re-read INSIDE the critical section: the instance was bound at
            // admission, and a call parked ahead of this one can poison it
            // before this one runs. Refusing is the whole point -- a
            // half-unwound instance answers wrong rather than failing, and
            // the module's own re-entry guard is cleared by the landing pad
            // on exactly the path that poisons it.
            if (poisoned) throw new Error("instance discarded by an earlier trap");
            // Per-call state, set HERE and not beside the call: a suspended
            // handler returns to the event loop, so the next request would
            // otherwise move it under this one.
            requestBody = octets;
            responseChunks = [];
            entered = true;
            return lisp.handleRequest(input);
          }),
        );
        // Headers as an ARRAY of pairs, which keeps two Set-Cookie two.
        // An EMPTY body becomes null whichever shape it arrived in: 204/205/304
        // may only be constructed with a null body, and "" and a zero-length
        // Uint8Array are the same response as none.
        const body = head.body ?? collected();
        return new Response(body?.length ? body : null, {
          status: head.status ?? 200,
          headers: head.headers ?? [],
        });
      } catch (error) {
        console.error("handle-request failed:", error);
        // Only a call that ENTERED the module can have left it half-written.
        // A mapping, or a Response the platform refused, says nothing about
        // the instance, and discarding it would cost the next request a
        // reinstantiation for someone else's bad header.
        if (entered) poisoned = true;
        return new Response("internal error\n", { status: 500 });
      }
    },
  };
}


---

# FILE: references/examples/cloudflare-workers/dog-fetcher/worker.lisp

;;; A Worker that itself makes an outgoing HTTP request -- the proxy shape of
;;; ../../net/dog-fetcher.lisp, routed with tiny-routes/lite. The client is
;;; rontolisp:fetch, THE SAME (await (fetch ...)) that runs on the interpreter,
;;; the JVM and a wasi:http component: --host-fetch lowers it onto the Worker
;;; runtime's own fetch, imported as env.fetch and suspended through JSPI
;;; (src/index.js), so the source no longer spells its own transport.
;;;
;;; :server :rontolisp picks the transport per target at read time, so THIS
;;; ONE SOURCE runs on every backend (the README has the commands):
;;;   interpreter / JVM  -- a real socket on :8080, fetch over the JDK client
;;;   --component        -- wasmtime serve, fetch over wasi:http
;;;   --no-wasi --host-fetch (build.sh) -- the Worker: the host calls the
;;;                         synthesized handle-request export, fetch is env.fetch

(ql:quickload '("clack" "tiny-routes/lite"))

(defun json-response (status obj)
  (list status '(:content-type "application/json")
        (list (format nil "~a~%" (rontolisp:json-stringify obj)))))

(defun error-response (status message)
  (json-response status (rontolisp:plist-hash-table (list :error message))))

;; One upstream round trip: dog.ceo answers {"message": ,
;; "status": "success"|"error"}. Awaiting needs an async-defun, and a future
;; settles to ONE value, so the (url status) pair rides a list -- status 0
;; means nothing came back at all (the transport error fetch signals at await),
;; which is the only case the routes answer 502 for.
(rontolisp:async-defun dog-image (path)
  (handler-case (let* ((res
                        (rontolisp:await
                         (rontolisp:fetch
                          (concatenate 'string "https://dog.ceo/api" path))))
                       (status (getf res :status))
                       (body
                        (rontolisp:await (rontolisp:read-all (getf res :body))))
                       (answer (rontolisp:json-parse body)))
                  (list (and (eql status 200)
                             (equal (gethash "status" answer) "success")
                             (gethash "message" answer)) status))
    (error () (list nil 0))))

;; The route bodies below are synchronous (tiny-routes composes plain
;; functions), so they cannot await -- they return the async-defun's FUTURE as
;; the response, and the reactor transport resolves it at the boundary.
(rontolisp:async-defun random-dog-response ()
  (let ((dog (car (rontolisp:await (dog-image "/breeds/image/random")))))
    (if dog
        (json-response 200 (rontolisp:plist-hash-table (list :dog dog)))
        (error-response 502 "the dog API did not answer"))))

(rontolisp:async-defun breed-response (breed)
  (let* ((result
          (rontolisp:await
           (dog-image (format nil "/breed/~a/images/random" breed))))
         (dog (car result))
         (status (car (cdr result))))
    (cond (dog (json-response 200
                (rontolisp:plist-hash-table (list :breed breed :dog dog))))
          ((eql status 0) (error-response 502 "the dog API did not answer"))
          (t (error-response 404 "no such breed")))))

;;; --- the routes --------------------------------------------------------------

;; A breed reaches the upstream inside a URL, so it is checked first. nil
;; DECLINES the route, which drops the request into the catch-all 404.
(defun valid-breed (breed)
  (and (plusp (length breed))
       (every (lambda (c) (or (alpha-char-p c) (eql c #\-))) breed) breed))

(tiny:define-routes *routes*
  (tiny:define-get "/" () (random-dog-response))
  (tiny:define-get "/breed/:breed" (req)
    (let ((breed
           (valid-breed (string-downcase (tiny:path-parameter req :breed)))))
      (when breed (breed-response breed))))
  (tiny:define-any "*" (req)
    (error-response 404 (format nil "no route for ~a" (tiny:path-info req)))))

(clack:clackup *routes* :server :rontolisp :port 8080 :use-thread nil)


---

# FILE: references/examples/cloudflare-workers/dog-fetcher/wrangler.jsonc

{
  // https://developers.cloudflare.com/workers/wrangler/configuration/
  "name": "rontolisp-dog-fetcher",
  "main": "src/index.js",
  "compatibility_date": "2026-01-01",

  // `import module from "./worker.wasm"` needs no configuration: .wasm is a
  // built-in module rule for ES-module Workers, and the file is compiled at
  // deploy time rather than on the request path.
  "observability": {
    "enabled": true
  }
}


---

# FILE: references/examples/cloudflare-workers/dog-relay/README.md

# dog-relay — a Worker that relays, many at a time

Every request is forwarded to [dog.ceo](https://dog.ceo) and the reply —
status, content type and body — is streamed back to the client **as it
arrives**, a chunk at a time. Where [`../dog-fetcher`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/dog-fetcher) parses
the upstream's answer and builds its own JSON, this one hands the answer
through; and where dog-fetcher serves **one request at a time**, this one is
compiled `--reentrant` and serves them **overlapped on one instance**.

Routes come from [tiny-routes](https://github.com/jeko2000/tiny-routes)
(`tiny-routes/lite`), the application is served with `:server :rontolisp`, and
this one `worker.lisp` runs on every backend (see
[the same worker.lisp on every backend](#the-same-workerlisp-on-every-backend)).

```bash
./build.sh          # worker.lisp -> src/worker.wasm + src/worker.js
npx wrangler dev    # http://localhost:8787
npx wrangler deploy
```

```console
$ curl http://localhost:8787/                 # dog.ceo's /breeds/list/all, relayed
{"message":{"affenpinscher":[],"african":["wild"],"airedale":[], ...

$ curl http://localhost:8787/breed/hound      # /breed/hound/images -- ~50 KB, relayed as it streams
{"message":["https://images.dog.ceo/breeds/hound-basset/n02088238_10005.jpg", ...

$ curl -i http://localhost:8787/breed/unicorn # the upstream's own 404, relayed
HTTP/1.1 404 Not Found
Content-Type: application/json

{"status":"error","message":"Breed not found (main breed does not exist)","code":404}

$ curl -i http://localhost:8787/anything      # ours
HTTP/1.1 404 Not Found
Content-Type: application/json

{"error":"no route for /anything"}
```

## Two flags, and why this Worker asks for both

`build.sh` passes `--host-boundary=streaming --reentrant` on top of
dog-fetcher's `--no-wasi --host-fetch`.

**`--host-boundary=streaming`, because the reply is relayed.** The handler
answers the fetch reply's `:body` STREAM as its own response body and never
reads it. On the streaming boundary the transport pulls each chunk through
`env.readResponseBody` and pushes it out through `env.writeResponseBody` the
moment it has it, so a 50 KB breed listing is on its way to the client while
the rest of it is still on the wire and never exists whole in linear memory.
The default `envelope` boundary would drain the reply into the response head
first.

**`--reentrant`, because a relay is parked time.** A request here is one
upstream round trip and almost no CPU. Serialised — what every module that can
suspend does by default, and what dog-fetcher does — N concurrent clients each
wait for the N-1 relays ahead of them. `--reentrant` makes the module own its
per-call state and drops the re-entry guard, so the Worker runtime may start a
call while another is parked on dog.ceo. Measured with plain node against the
real upstream: six concurrent relays complete in about one round trip
(≈ 400 ms) against ≈ 1.6 s summed.

The two compose only because the streaming body protocol carries a **call
identity** under `--reentrant`: every body import leads with an `:int` id —
`env.readRequestBody(id, ptr, cap)`, `env.writeResponseBody(id, ptr, len)`,
`env.readResponseBody(id, ptr, cap)` — so each pull and push names the relay it
belongs to instead of sharing one host-side cursor. The request's id rides the
envelope's `"call-id"` key and is minted per request by the generated
`worker()`; a fetch reply's id is its own (`"body-id"` in the reply head),
minted per fetch by the generated `defaultHost()`, so replies drain
independently and nothing is superseded.

**And still nothing is hand-written.** `--emit-js-glue` writes
[`src/worker.js`](src/worker.js) from the same declarations: no queue, per-call
body state keyed by id, per-reply readers, and the same
`Request -> envelope -> Response` mapping. It is generated, checked in and
pinned by `HostGlueEmitterTest`. [`src/index.js`](src/index.js) is:

```js
import module from "./worker.wasm";
import { worker } from "./worker.js";

export default worker(module);
```

## What overlaps, and what does not

`--reentrant` buys **I/O overlap, never CPU parallelism**: one wasm stack runs
at a time, and what overlaps is the time calls spend parked on the upstream.
Inside one call nothing changes either — a fetch future is settled the moment
`fetch` returns, exactly as in dog-fetcher — so `relay` is an ordinary
`async-defun` and the route bodies return its FUTURE for the transport to
resolve. Only the host's view is different: it may enter `handle-request` again
while a previous call is suspended, and the module keeps every call's dynamic
bindings and every call's body cursors apart.

What the relay forwards is the reply's TEXT: `rontolisp:fetch`'s `:body` is a
character stream on every backend, so a binary upstream reply (an image) does
not survive the round trip byte for byte. A binary body a Worker *reads*
(`:raw-body`) or *answers itself* (an `(unsigned-byte 8)` vector) does cross
exactly on this boundary; a fetched one relayed as-is does not, yet.

## The same worker.lisp on every backend

`:server :rontolisp` resolves the transport when the source is read for a
target — a real socket on the interpreter/JVM, `wasi:http` under `--component`,
the host-driven `handle-request` export on a `--no-wasi` reactor — and
`rontolisp:fetch` follows along. From the repo root:

```bash
JAR=target/rontolisp-0.1.0-SNAPSHOT-exec.jar
W=examples/cloudflare-workers/dog-relay/worker.lisp

# 1. interpreter -- a blocking server on :8080
java -jar $JAR $W

# 2. JVM class (keep the jar on the classpath)
java -jar $JAR $W -o DogRelay.class && java -cp $JAR:. DogRelay

# 3. WASI component under wasmtime serve
java -jar $JAR $W -o dog-relay.wasm --component && \
  wasmtime serve -W gc=y -W exceptions=y -S cli=y -S tcp=y -S inherit-network=y dog-relay.wasm

# 4. the Worker (this directory): build.sh + wrangler dev, as above
```

The reactor build is equally drivable from plain node 24 with
`--experimental-wasm-jspi`: `worker(module)` from `src/worker.js` is the whole
host, and `Promise.all` over its `fetch` is the overlap.

## What's in here

| File | Purpose |
| --- | --- |
| [`worker.lisp`](worker.lisp) | The whole program. This is what `build.sh` compiles. |
| [`src/index.js`](src/index.js) | Three lines over the generated `worker()`. |
| [`src/worker.js`](src/worker.js) | The boundary, GENERATED by `--emit-js-glue` from worker.lisp's declarations. Do not edit; `./build.sh` rewrites it. |
| `src/worker.wasm` | A build product — run `./build.sh` first. |

## Limitations

The Worker sandbox and `--no-wasi` limitations of
[`../hello-clack`](../hello-clack/README.md#limitations) apply unchanged, plus:

- **A relayed fetch reply is text**, as above; a binary upstream body does not
  cross byte-exact.
- **Overlap is per isolate and per parked time.** Two calls never run Lisp at
  the same moment; a CPU-bound handler gains nothing from `--reentrant`.


---

# FILE: references/examples/cloudflare-workers/dog-relay/build.sh

#!/usr/bin/env bash
# Compile worker.lisp to the .wasm module the Worker imports.
#
# --no-wasi: the Worker calls the exported entry point directly, so the module
#   needs no WASI imports -- it becomes a reactor (`_initialize`, not `_start`).
# --host-fetch: rontolisp:fetch is lowered onto the host's own fetch, imported
#   as env.fetch(request-json) -> response-head-json.
# --host-boundary=streaming: the bodies leave the JSON envelope and cross as
#   octets through imports of their own -- env.readRequestBody,
#   env.writeResponseBody, and env.readResponseBody for the reply of that fetch.
#   ASKED FOR, because the default is `envelope`: this Worker relays an upstream
#   reply and forwards it a chunk at a time rather than holding it whole.
# --reentrant: let the Worker runtime OVERLAP calls into the one instance. A
#   relay is one upstream round trip of parked time per request; serialised,
#   N concurrent clients would each wait for the N-1 relays ahead of them. The
#   module then owns its per-call state, and every body import above carries a
#   leading call id (the envelope's "call-id", the fetch reply's "body-id"), so
#   each pull and push names the relay it belongs to. ../dog-fetcher is the same
#   boundary WITHOUT the flag: its calls go through the generated queue.
# --emit-js-glue: write src/worker.js beside the module -- the import object,
#   the linear-memory plumbing, the Suspending/promising wiring, the per-call
#   body state keyed by id, all derived from the same declarations the module was
#   built from. src/index.js is then three lines. It is CHECKED IN and pinned by
#   HostGlueEmitterTest, so regenerate it here rather than editing it.
# --optimize=size: a Worker bundle has a size limit; tiny-routes/lite is what
#   keeps cl-ppcre out of what the tree-shaker has to keep.
#
# The first run downloads clack/lack/tiny-routes into ~/.rontolisp/quicklisp;
# after that the build is offline.
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling worker.lisp -> src/worker.wasm + src/worker.js"
java -jar "$jar" "$here/worker.lisp" -o "$here/src/worker.wasm" \
  --no-wasi --host-fetch --host-boundary=streaming --reentrant --optimize=size --emit-js-glue

ls -l "$here/src/worker.wasm" "$here/src/worker.js"
echo "done. Run it with:  npx wrangler dev"


---

# FILE: references/examples/cloudflare-workers/dog-relay/package.json

{
  "name": "rontolisp-cloudflare-dog-relay",
  "version": "0.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "build": "./build.sh",
    "dev": "wrangler dev",
    "deploy": "wrangler deploy",
    "tail": "wrangler tail"
  },
  "devDependencies": {
    "wrangler": "^4"
  }
}


---

# FILE: references/examples/cloudflare-workers/dog-relay/src/index.js

// The whole Worker. `./worker.js` is generated from worker.lisp's own
// declarations (build.sh's --emit-js-glue) and owns all of it: the import
// object, the (ptr, len) staging, the __ronto_alloc bracket, the JSPI wiring,
// the four host functions this boundary declares -- env.fetch and the three
// body imports -- and the Request -> envelope -> Response mapping.
//
// What differs from ../dog-fetcher/src/index.js is nothing here and everything
// in the generated file: this module was compiled --reentrant, so worker()
// has NO one-call-at-a-time queue. Requests overlap on the one instance, and
// the body state that used to be "the current call's" is a map keyed by the
// call id worker() mints per request and the envelope carries -- every pull
// of a request body, every response chunk and every chunk of a relayed reply
// names the call it belongs to.
//
// To take any of it over, hand `worker` a host: whatever it supplies is laid
// over the derived entries one at a time.
//
//   import { worker, suspending } from "./worker.js";
//   export default worker(module, {
//     remoteAddr: (request) => request.headers.get("cf-connecting-ip"),
//     host: { env: { fetch: suspending(myOwnFetch) } },
//   });

import module from "./worker.wasm";
import { worker } from "./worker.js";

export default worker(module);


---

# FILE: references/examples/cloudflare-workers/dog-relay/src/worker.js

// GENERATED by rontolisp --emit-js-glue -- do not edit.
//
// The host half of this module's boundary, derived from the program's own
// declarations: one import-object entry per rontolisp:wasm-import, one entry
// point per rontolisp:wasm-export, and every piece of linear-memory plumbing
// between them -- the (ptr, len) pair a :string crosses as, the __ronto_alloc
// bracket around a call, and the read(2) cursor a :bytes result is pulled
// through.
//
// What a declaration cannot state is what a host function DOES, so that is the
// one thing this file asks for: `host` is a plain function per import, keyed by
// import module and field, taking and answering ordinary JavaScript values --
// never a (ptr, len) pair, and, where an entry answers `chunk` below, a
// Uint8Array or a string with null for the end of them.
//
//   import { instantiate, suspending } from "./worker.js";
//
//   const lisp = instantiate(module, {
//     env: {
//       fetch: (text) => text,   // or leave it to defaultHost()
//       readResponseBody: (number) => chunk,   // or leave it to defaultHost()
//       readRequestBody: (number) => chunk,   // or leave it to worker()
//       writeResponseBody: (number, chunk2) => {},   // or leave it to worker()
//     },
//   });
//   await lisp.handleRequest(text);
//
// A host that suspends marks its own entries -- suspending(async (...) => ...)
// -- and every entry point above then answers a promise: the marked imports are
// wrapped in WebAssembly.Suspending and each entry point that can reach one is
// entered through WebAssembly.promising. This module was compiled --reentrant:
// it owns its per-call state, so calls are NOT serialised -- overlap them
// freely (what overlaps is the parked time; one stack still runs at a time).
// A read import's remainder is keyed by its arguments only, so two overlapped
// calls pulling one source through IDENTICAL arguments are the host's own
// hazard to serialise.
//
// This module's boundary is the reactor envelope, so the Request/Response half
// is derivable too and `worker` below is it:
//
//   import module from "./worker.wasm";
//   import { worker } from "./worker.js";
//
//   export default worker(module);
//
// Regenerate it, never edit it: --emit-js-glue on the compile that wrote the
// .wasm beside it.

const encoder = new TextEncoder();
// ignoreBOM, because these octets are a VALUE and not a document: a
// leading U+FEFF is a character the other side chose, and the default
// decoder deletes it -- silently shortening a BOM-prefixed request body
// by one character while the content-length beside it still counts three.
const decoder = new TextDecoder("utf-8", { ignoreBOM: true });

const SUSPENDING = Symbol.for("rontolisp.suspending");

/**
 * Marks a host function as one that answers a promise, so the wasm stack parks
 * until it settles (JSPI). The wrapper is not free -- an import answering
 * SYNCHRONOUSLY through one still returns to the event loop -- so mark only the
 * entries this host really implements asynchronously.
 *
 * @param {Function} fn the host function
 * @returns {object} the marked entry, to be passed as the import
 */
export function suspending(fn) {
  return { [SUSPENDING]: fn };
}

/**
 * Instantiates the module against this host and returns its callable surface.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @param {object} host one function per import, keyed by module and field
 * @returns {object} `exports`, plus one entry point per rontolisp:wasm-export
 */
export function instantiate(module, host = {}) {
  let exports;

  // Every view is built where it is used, never held: growing the module's
  // memory DETACHES the buffer behind every view taken before the growth, and a
  // suspending host resumes after growths it never saw.
  const readString = (ptr, len) =>
    decoder.decode(new Uint8Array(exports.memory.buffer, ptr, len));

  // Octets, COPIED: the module pops the staging behind the pointer the moment
  // the call returns, so a chunk not taken by then is one the host never gets.
  const readBytes = (ptr, len) =>
    new Uint8Array(exports.memory.buffer.slice(ptr, ptr + len));

  // Bytes the HOST hands over live in the module's own bump allocator, and
  // cross as the (ptr, len) pair every memory-typed value is.
  const write = (octets) => {
    const ptr = exports.__ronto_alloc(octets.length);
    new Uint8Array(exports.memory.buffer, ptr, octets.length).set(octets);
    return [ptr, octets.length];
  };
  const writeString = (value) => write(encoder.encode(String(value)));
  // An import's text answer lives in a park block until the MODULE has copied
  // it out -- which it frees itself with __ronto_park_free.
  const writeParkString = (value) => {
    const octets = encoder.encode(String(value));
    const ptr = exports.__ronto_park_alloc(octets.length);
    new Uint8Array(exports.memory.buffer, ptr, octets.length).set(octets);
    return [ptr, octets.length];
  };
  const octets = (chunk) =>
    typeof chunk === "string" ? encoder.encode(chunk) : chunk;

  // A host answers a value, or -- from an entry it marked suspending -- a promise
  // of one. Both shapes ride one expression, which is what lets this file drive a
  // synchronous host and a JSPI host without being written twice.
  const marked = new Set();
  const unmarked = (what) =>
    what +
    " answered a promise; wrap it in suspending() so the wasm stack parks" +
    " until it settles";
  const settle = (what, value, next) => {
    if (typeof value?.then === "function") {
      if (!marked.has(what)) {
        // Reported before it is thrown: this throw crosses back into wasm,
        // where a catch_all landing pad turns it into whatever the module
        // makes of a failed import.
        console.error(unmarked(what));
        throw new TypeError(unmarked(what));
      }
      return value.then(next);
    }
    return next(value);
  };

  // The entries the module really imports: --optimize shakes out an import the
  // program never calls, and a host should not have to know which survived.
  const linked = new Set(
    WebAssembly.Module.imports(module).map((i) => i.module + "." + i.name),
  );
  const bind = (moduleName, field, wrap) => {
    const key = moduleName + "." + field;
    const given = (host[moduleName] ?? {})[field];
    if (given == null) {
      if (!linked.has(key)) return undefined;
      throw new TypeError("host." + key + " is missing; this module imports it");
    }
    if (given[SUSPENDING] === undefined) {
      // An async function that was never marked is the one mistake worth
      // catching HERE, where the report is a plain stack and not whatever
      // the module makes of an import that threw.
      if (given.constructor?.name === "AsyncFunction") {
        throw new TypeError(unmarked("host." + key));
      }
      return wrap(key, given);
    }
    marked.add(key);
    return new WebAssembly.Suspending(wrap(key, given[SUSPENDING]));
  };

  // A :bytes RESULT is the read(2) shape: the MODULE owns the buffer and asks
  // for up to `cap` octets, so what a host answers is the next CHUNK and this
  // holds whatever did not fit. Calls OVERLAP (--reentrant), so the remainders
  // are KEYED by the arguments that asked for them -- per call/reply id on the
  // id-carrying body protocol -- and each overlapped pull keeps its own; two
  // overlapped calls pulling one source through IDENTICAL arguments are the
  // host's own hazard to serialise. A remainder is dropped with the end of its
  // stream, never at the next entry: another call may be mid-pull.
  const readers = new Map();
  const reader = (what, source) => {
    const rests = new Map();
    readers.set(what, () => rests.clear());
    const drain = (key, rest, ptr, cap) => {
      const n = Math.min(cap, rest.length);
      new Uint8Array(exports.memory.buffer, ptr, n).set(rest.subarray(0, n));
      const left = rest.subarray(n);
      if (left.length === 0) rests.delete(key);
      else rests.set(key, left);
      return n;
    };
    // A read that FAILS answers a NEGATIVE count. Throwing would trap the
    // instance; the count is an error channel the module turns into a Lisp
    // condition where the octets are consumed.
    const failed = (error) => {
      console.error(what + " failed:", error);
      return -1;
    };
    return (args, ptr, cap) => {
      const key = JSON.stringify(args);
      const rest = rests.get(key);
      if (rest !== undefined) return drain(key, rest, ptr, cap);
      try {
        const answer = settle(what, source(...args), (chunk) => {
          if (chunk == null) return 0;
          const value = octets(chunk);
          return value.length === 0 ? 0 : drain(key, value, ptr, cap);
        });
        return typeof answer?.then === "function"
          ? answer.then(undefined, failed)
          : answer;
      } catch (error) {
        return failed(error);
      }
    };
  };

  // What a read import left over, thrown away on demand -- with no argument,
  // every remainder of every import.
  const drop = (key) =>
    key === undefined ? readers.forEach((f) => f()) : readers.get(key)?.();

  const imports = {
    env: {
      // (:string) -> :string
      fetch: bind("env", "fetch", (what, call) => {
        return (p0, p0Len) =>
          settle(what, call(readString(p0, p0Len)), writeParkString);
      }),
      // (:s32) -> :bytes
      readResponseBody: bind("env", "readResponseBody", (what, call) => {
        const read = reader(what, call);
        return (p0, ptr, cap) => read([p0], ptr, cap);
      }),
      // (:s32) -> :bytes
      readRequestBody: bind("env", "readRequestBody", (what, call) => {
        const read = reader(what, call);
        return (p0, ptr, cap) => read([p0], ptr, cap);
      }),
      // (:s32, :bytes) -> :void
      writeResponseBody: bind("env", "writeResponseBody", (what, call) => {
        return (p0, p1, p1Len) =>
          settle(what, call(p0, readBytes(p1, p1Len)), () => undefined);
      }),
    },
  };
  const instance = new WebAssembly.Instance(module, imports);
  exports = instance.exports;
  // Entropy, BEFORE the top level runs: a --no-wasi module imports none, so its
  // `random` starts from a constant and every instance would draw one sequence.
  // Seeding here also covers the draws a library makes while it LOADS. A Worker
  // forbids this in global scope, so instantiate on the first request.
  exports.__ronto_seed_random(
    new BigUint64Array(crypto.getRandomValues(new Uint8Array(8)).buffer)[0],
  );
  // And a clock, nanoseconds since the Unix epoch, for the same reason and
  // before the same line: until one is set the clock built-ins signal rather
  // than report 1970.
  exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
  exports._initialize();

  // The entry points a call chain can reach a suspending import from -- the
  // list the build prints -- entered through WebAssembly.promising exactly when
  // the host marked one. An unmarked host never pays for the promise.
  const suspends = marked.size !== 0;
  // --reentrant: NO serialisation queue. The module owns its per-call state
  // (task-scoped dynamic bindings, park-block staging), so overlapped calls
  // into one instance are the point of the build.
  const entry$handleRequest = suspends
    ? WebAssembly.promising(exports["handle-request"])
    : exports["handle-request"];

  // One call into the module (--reentrant, so calls may OVERLAP). The
  // argument staging is popped SYNCHRONOUSLY the moment the entry call
  // starts -- the wrapper boxes its parameters before its first suspension,
  // and by decode time another overlapped call may hold staging of its own
  // above the mark (the reset clamps to the module's park floor, so a park
  // block carved meanwhile survives the pop). Anything that must outlive
  // this synchronous window crosses in park blocks instead: `reserve`d
  // receive buffers, and the module's own :string/:s-expr results, which
  // `decode` frees with __ronto_park_free.
  const call = (run, entry, stage, decode) => {
    const work = () => {
      // The module's clock moves only when the host moves it, so move it per
      // call. Not a workaround for a frozen clock -- a Worker's own Date.now()
      // is frozen for the duration of a request as a timing-attack mitigation,
      // so a value that changes once per call is what the platform has.
      exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
      const mark = exports.__ronto_alloc_mark();
      const raw = entry(...stage());
      exports.__ronto_alloc_reset(mark);
      return typeof raw?.then === "function" ? raw.then(decode) : decode(raw);
    };
    return run(work);
  };

  /** `handle-request` -- (:string) -> :string */
  const make$handleRequest =
    (run) =>
    (p0) => {
      return call(
        run,
        entry$handleRequest,
        () => {
          const a0 = writeString(p0);
          return [a0[0], a0[1]];
        },
        ([ptr, len]) => {
          const value = readString(ptr, len);
          exports.__ronto_park_free(ptr);
          return value;
        },
      );
    };


  return {
    exports,
    handleRequest: make$handleRequest((work) => work()),
    drop,
  };
}

/**
 * The half of this boundary the module's own declarations FIX, ready to hand to
 * `instantiate` -- or to leave to `worker` below, which passes it for you. What
 * a host is still free to do is override it: whatever it supplies wins, entry by
 * entry.
 *
 * @returns {object} the import-object entries this file implements itself
 */
export function defaultHost() {
  // One reader per reply, keyed by the id the head carries back: overlapped
  // calls -- and a second fetch inside one call -- each drain their own. A
  // drained reader is dropped; one nobody drains stays until the platform
  // reclaims its stream.
  let replySerial = 0;
  const upstreams = new Map();
  return {
    env: {
      fetch: suspending(async (head) => {
        const request = JSON.parse(head);
        try {
          const response = await fetch(request.url, {
            method: request.method,
            headers: request.headers,
            body: request.body,
          });
          const id = ++replySerial;
          // The reader IS the body; the module pulls it BY THIS ID afterwards.
          if (response.body) upstreams.set(id, response.body.getReader());
          return JSON.stringify({
            status: response.status,
            headers: [...response.headers],
            "body-id": id,
          });
        } catch (error) {
          // The error arm becomes a Lisp condition at the fetch CALL; throwing
          // here would trap the instance instead, and take the request with it.
          return JSON.stringify({ error: String(error) });
        }
      }),
      readResponseBody: suspending(
        // The next chunk of the reply the id names, null at the end of it --
        // and for an id whose reply is already drained or was never opened.
        // Reading a ReadableStream is asynchronous, so this one really does
        // suspend; a read that THROWS becomes the negative count the module
        // signals at the drain, which the glue answers on our behalf.
        async (id) => {
          const upstream = upstreams.get(id);
          if (!upstream) return null;
          const { value, done } = await upstream.read();
          if (done) {
            upstreams.delete(id);
            return null;
          }
          return value;
        },
      ),
    },
  };
}

/**
 * This module as a fetch handler -- the whole Worker, with nothing left over:
 *
 *   import module from "./worker.wasm";
 *   import { worker } from "./worker.js";
 *
 *   export default worker(module);
 *
 * A Request becomes the envelope the module's entry point takes, the head it
 * answers becomes a Response, and the instance is created on the FIRST REQUEST
 * (a Worker forbids drawing entropy in global scope) and retired if a call ever
 * traps.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @param {object} [options] `host` -- import entries, laid over defaultHost()'s
 *   one at a time; `remoteAddr` -- (request, env, ctx) => the client
 *   address, since which header carries it is the platform's business and not
 *   this file's (`(r) => r.headers.get("cf-connecting-ip")` on Cloudflare)
 * @returns {object} `{ fetch(request, env, ctx) }`
 */
export function worker(module, options = {}) {
  let instance = null;
  // Set when a call TRAPPED: that instance skipped its arena reset and its Lisp
  // state may be half-written, so nothing else may run on it. A Lisp ERROR is not
  // this -- the transport answers 500 itself and the instance is fine.
  let poisoned = false;
  // The request bodies the module pulls and the response bodies coming back,
  // keyed by the call id worker() mints per request: overlapped calls each
  // pull their own and collect their own.
  let callSerial = 0;
  const requestBodies = new Map();
  const responseChunks = new Map();
  const collected = (id) => {
    const chunks = responseChunks.get(id) ?? [];
    const all = new Uint8Array(
      chunks.reduce((n, chunk) => n + chunk.length, 0),
    );
    let at = 0;
    for (const chunk of chunks) {
      all.set(chunk, at);
      at += chunk.length;
    }
    return all;
  };
  const base = defaultHost();
  base.env = {
    ...(base.env ?? {}),
    readRequestBody: (id) => {
      // Handed over ONCE per call: a chunk source that never answers null is
      // one the module pulls forever.
      const chunk = requestBodies.get(id) ?? null;
      if (chunk) requestBodies.set(id, null);
      return chunk;
    },
    writeResponseBody: (id, chunk) => responseChunks.get(id)?.push(chunk),
  };
  const given = options.host ?? {};
  const host = {};
  for (const key of new Set([...Object.keys(base), ...Object.keys(given)])) {
    host[key] = { ...(base[key] ?? {}), ...(given[key] ?? {}) };
  }
  const live = () => {
    if (poisoned) {
      instance = null;
      poisoned = false;
    }
    return (instance ??= instantiate(module, host));
  };

  // The request head. `target` stays RAW -- path and query still joined and
  // still percent-encoded -- because the shared normalizer on the other side
  // owns that split, and a pre-split path leaves the query string nil.
  const envelope = (request, octets, remoteAddr, callId) => {
    const url = new URL(request.url);
    const headers = Object.fromEntries(request.headers);
    // A body with no content-length is a body the request parser does not read,
    // and a chunked request carries none -- so set it from the octets we have.
    if (octets?.length) headers["content-length"] = String(octets.length);
    const head = {
      method: request.method,
      target: url.pathname + url.search,
      headers,
      scheme: url.protocol.replace(":", ""),
      "call-id": callId,
    };
    if (remoteAddr != null) head["remote-addr"] = remoteAddr;
    return JSON.stringify(head);
  };

  return {
    // EVERYTHING is inside the try, not just the module call: reading an
    // aborted upload rejects, and `new Response` throws on a status or a
    // header an application is free to produce (0, 999, a newline in a
    // value). Outside it those escape as an unhandled rejection, which the
    // platform answers with its own error page and nothing in the log.
    async fetch(request, env, ctx) {
      let entered = false;
      let callId = 0;
      try {
        const remoteAddr = await options.remoteAddr?.(request, env, ctx);
        const octets = request.body
          ? new Uint8Array(await request.arrayBuffer())
          : null;
        callId = ++callSerial;
        const input = envelope(request, octets, remoteAddr, callId);
        requestBodies.set(callId, octets);
        responseChunks.set(callId, []);
        entered = true;
        const head = JSON.parse(await live().handleRequest(input));
        // Headers as an ARRAY of pairs, which keeps two Set-Cookie two.
        // An EMPTY body becomes null whichever shape it arrived in: 204/205/304
        // may only be constructed with a null body, and "" and a zero-length
        // Uint8Array are the same response as none.
        const body = head.body ?? collected(callId);
        return new Response(body?.length ? body : null, {
          status: head.status ?? 200,
          headers: head.headers ?? [],
        });
      } catch (error) {
        console.error("handle-request failed:", error);
        // Only a call that ENTERED the module can have left it half-written.
        // A mapping, or a Response the platform refused, says nothing about
        // the instance, and discarding it would cost the next request a
        // reinstantiation for someone else's bad header.
        if (entered) poisoned = true;
        return new Response("internal error\n", { status: 500 });
      }
      finally {
        // The call's body state goes with the call, on every path -- the
        // Response above reads the chunks before this runs.
        requestBodies.delete(callId);
        responseChunks.delete(callId);
      }
    },
  };
}


---

# FILE: references/examples/cloudflare-workers/dog-relay/worker.lisp

;;; A Worker that RELAYS: every request is forwarded to dog.ceo and the reply
;;; -- status, content type and body -- is streamed back to the client as it
;;; arrives, a chunk at a time. ../dog-fetcher parses the upstream's answer and
;;; builds its own; this one hands it through. Two things about that shape
;;; decide the build (build.sh):
;;;
;;;   --host-boundary=streaming  the reply body is RELAYED: the transport
;;;                              forwards each chunk the moment it is pulled,
;;;                              so a 50 KB breed listing never exists whole in
;;;                              linear memory. The default envelope would hold
;;;                              it before answering.
;;;   --reentrant                a relay is one upstream round trip of parked
;;;                              time and almost no CPU, so serialising the
;;;                              calls costs the whole width: N concurrent
;;;                              clients would each wait for the N-1 relays
;;;                              ahead of them. --reentrant lets them overlap
;;;                              on ONE instance; every body import then
;;;                              carries a call id, so each pull names the
;;;                              relay it belongs to.
;;;
;;; The client is rontolisp:fetch, THE SAME (await (fetch ...)) that runs on
;;; the interpreter, the JVM and a wasi:http component; --host-fetch lowers it
;;; onto the Worker runtime's own fetch. :server :rontolisp picks the transport
;;; per target at read time, so THIS ONE SOURCE runs on every backend (the
;;; README has the commands).

(ql:quickload '("clack" "tiny-routes/lite"))

(defun json-response (status obj)
  (list status '(:content-type "application/json")
        (list (format nil "~a~%" (rontolisp:json-stringify obj)))))

(defun error-response (status message)
  (json-response status (rontolisp:plist-hash-table (list :error message))))

;; The relay. The reply's :body is a STREAM and it is answered AS the response
;; body: nothing here reads it -- the transport pulls it chunk by chunk
;; (env.readResponseBody) and pushes each chunk out (env.writeResponseBody), so
;; the upstream's answer is on its way to the client while the rest of it is
;; still on the wire. The upstream's status and content type pass through with
;; it; a transport error before the head is the one case answered here.
(rontolisp:async-defun relay (path)
  (handler-case (let ((res
                       (rontolisp:await
                        (rontolisp:fetch
                         (concatenate 'string "https://dog.ceo/api" path)))))
                  (list (getf res :status)
                        (list :content-type
                              (or (cdr
                                   (assoc "content-type" (getf res :headers)
                                          :test #'string-equal))
                                  "application/octet-stream"))
                        (getf res :body)))
    (error () (error-response 502 "the dog API did not answer"))))

;;; --- the routes --------------------------------------------------------------

;; A breed reaches the upstream inside a URL, so it is checked first. nil
;; DECLINES the route, which drops the request into the catch-all 404.
(defun valid-breed (breed)
  (and (plusp (length breed))
       (every (lambda (c) (or (alpha-char-p c) (eql c #\-))) breed) breed))

;; The route bodies are synchronous (tiny-routes composes plain functions), so
;; they return the async-defun's FUTURE and the reactor transport resolves it
;; at the boundary.
(tiny:define-routes *routes*
  (tiny:define-get "/" () (relay "/breeds/list/all"))
  (tiny:define-get "/breed/:breed" (req)
    (let ((breed
           (valid-breed (string-downcase (tiny:path-parameter req :breed)))))
      (when breed (relay (format nil "/breed/~a/images" breed)))))
  (tiny:define-any "*" (req)
    (error-response 404 (format nil "no route for ~a" (tiny:path-info req)))))

(clack:clackup *routes* :server :rontolisp :port 8080 :use-thread nil)


---

# FILE: references/examples/cloudflare-workers/dog-relay/wrangler.jsonc

{
  // https://developers.cloudflare.com/workers/wrangler/configuration/
  "name": "rontolisp-dog-relay",
  "main": "src/index.js",
  "compatibility_date": "2026-01-01",

  // `import module from "./worker.wasm"` needs no configuration: .wasm is a
  // built-in module rule for ES-module Workers, and the file is compiled at
  // deploy time rather than on the request path.
  "observability": {
    "enabled": true
  }
}


---

# FILE: references/examples/cloudflare-workers/hello-clack/README.md

# hello-clack — a Clack application on Cloudflare Workers

The smallest thing that is still a real
[Clack](https://github.com/fukamachi/clack) application: `ql:quickload`, one
`defun`, `clack:clackup`.

```bash
./build.sh          # worker.lisp -> src/worker.wasm
npx wrangler dev    # http://localhost:8787
npx wrangler deploy
```

```console
$ curl http://localhost:8787/
Hello from Clack on Cloudflare Workers!
GET /
```

## The whole program

```lisp
(ql:quickload '("clack" "clack-handler-reactor"))

(defun app (env)
  (list 200 '(:content-type "text/plain; charset=utf-8")
        (list
         (format nil "Hello from Clack on Cloudflare Workers!~%~a ~a~%"
                 (getf env :request-method) (getf env :path-info)))))

(clack:clackup #'app :server :reactor :use-thread nil)
```

That is the whole of Clack's API: an application is a **function** of the
environment plist returning the `(status headers body)` list, and a middleware
is a function from application to application ([`../httpbin-clack`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin-clack)
has one). There is no Worker-specific code here, so `app` runs on hunchentoot,
on woo, under `wasmtime serve` and on the JVM, unchanged.

`:reactor` is a built-in handler backend that is **host-driven on every
backend**: its `run` stores the application and returns, and what replaces the
socket is one WASM export, `handle-request` (a JSON request string in, a JSON
response string out). You do not declare it: `rontolisp:wasm-export` needs a
literal name at compile time, which a `clackup` call has none to give, so the
compiler synthesizes it from a marker the backend leaves behind. The
[Clack guide](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/doc/en/guides/clack.md) has the full story.

A Worker does not strictly need this designator — `:server :rontolisp` picks
each target's own transport, which is how
[`../httpbin-clack-one-source`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin-clack-one-source) deploys a socket
server unchanged. What `:reactor` adds is "host-driven *everywhere*", which is
what lets [`check.lisp`](check.lisp) drive the whole Worker on the interpreter.

**`:use-thread nil`** is a property of the other backends rather than
boilerplate: it is already the default on WASM, but the interpreter and the JVM
have threads and `clackup` would otherwise store the application on one of them,
racing the next form.

## Developing without Cloudflare

`clack.handler.reactor:dispatch` is an ordinary function of a JSON string, and
exactly what the synthesized export calls, so the whole Worker runs on every
backend — which [`examples/examples.yaml`](../../examples.yaml) pins:

```bash
rontolisp check.lisp
rontolisp check.lisp -o Check.class && java -cp rontolisp-0.1.0-SNAPSHOT-exec.jar:. Check
rontolisp check.lisp -o check.wasm && wasmtime run -W gc -W exceptions=y check.wasm
```

The two lines before the first `-->` are upstream clack's own banner and debug
notice. On a Worker (`--no-wasi`) standard output is a **sink**, so those bytes
are discarded rather than trapping the instance; locally they are not. Pass
`:silent t :debug nil` to quiet them.

## The JavaScript half is not hand-written either

`--emit-js-glue` (`build.sh`) writes [`src/worker.js`](src/worker.js) from
`worker.lisp`'s own declarations: instantiation, the entropy and the clock a
`--no-wasi` module cannot draw for itself, the `(ptr, len)` staging around the
entry point, and the `Request -> envelope -> Response` mapping.
[`src/index.js`](src/index.js) is then:

```js
import module from "./worker.wasm";
import { worker } from "./worker.js";

export default worker(module);
```

There is no import object in it because there are no imports. On the DEFAULT
boundary every body rides the envelope's own `body` key, so this module asks the
host for **nothing at all** — and the mapping around it is transport work, fixed
by the envelope rather than chosen by the program, which is what makes it
derivable. [`../httpbin-clack`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin-clack) is the same application on
`--host-boundary=streaming`, where the bodies cross as octets through imports of
their own, and its `src/index.js` is the same call: the glue writes those too.

The generated file is **checked in** and pinned by `HostGlueEmitterTest`, so
regenerate it with `./build.sh` rather than editing it — and the three
`hello-*` directories carry the same one, because they declare the same
boundary.

## What's in here

| File | Purpose |
| --- | --- |
| [`worker.lisp`](worker.lisp) | The whole program — three forms. This is what `build.sh` compiles. |
| [`check.lisp`](check.lisp) | Drives it with no Cloudflare in sight, on any backend. |
| [`src/index.js`](src/index.js) | Three lines over the generated `worker()`. |
| [`src/worker.js`](src/worker.js) | The boundary, GENERATED by `--emit-js-glue` from `worker.lisp`'s declarations. Do not edit; `./build.sh` rewrites it. |
| `src/worker.wasm` | A build product — run `./build.sh` first. |

The cost of "it is a real Clack application" is module size
([size report](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/size-report/results/cloudflare-workers.md)) and isolate
startup, both paid once; the per-request cost is the Lisp call plus the string
boundary. [`../hello`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/hello) is the cheaper shape if a program will only
ever run on a Worker.

## Limitations

The Worker sandbox and the `--no-wasi` build, exactly as in
[`../httpbin`](../httpbin/README.md#limitations): no standard input, no
filesystem, and no `rontolisp:fetch` unless the build asks for `--host-fetch`
(which is [`../btc-ticker`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/btc-ticker)). Printing is discarded rather than
trapping, `random` works on a generator the generated `src/worker.js` seeds from
`crypto`, and the clock is whatever it hands to
`__ronto_set_time`. A *runtime* `(ql:quickload ...)`
cannot work: the one at the top of `worker.lisp` is resolved at **compile** time
and inlined, which is why the first `./build.sh` needs network and later ones do
not.

## Rebuilding

```bash
./mvnw clean package -DskipTests   # from the repository root
examples/cloudflare-workers/hello-clack/build.sh
```


---

# FILE: references/examples/cloudflare-workers/hello-clack/build.sh

#!/usr/bin/env bash
# Compile worker.lisp -- clack, the handler backend, the application and the
# clackup call -- to the .wasm module the Worker imports.
#
# --no-wasi: the Worker calls the exported entry point directly and never runs
#   the module as a program, so it needs no WASI imports at all. It becomes a
#   reactor: nothing to shim on the JavaScript side, and `_initialize` instead
#   of `_start`. clackup's start-up banner is not a problem there -- standard
#   output is a sink, so the bytes are discarded.
# --optimize=size: a Worker bundle has a size limit, and the tree-shaker is what
#   keeps the module down -- only what the program reaches ships. The =size
#   level additionally declines the two speed-over-size wasm-GC emissions, the
#   right trade on a Worker: smaller and a slightly faster isolate startup, for
#   a per-request cost of single-digit microseconds.
# --emit-js-glue: write src/worker.js beside the module -- the host half of this
#   boundary, from the program's own declarations. This module imports NOTHING
#   (the default `envelope` boundary keeps every body inside the head), so the
#   whole of that half is fixed by the transport and the glue writes all of it,
#   the Request -> envelope -> Response mapping included. src/index.js is then
#   three lines. It is CHECKED IN and pinned by HostGlueEmitterTest, so
#   regenerate it here rather than editing it.
#
# The first run downloads clack/lack into ~/.rontolisp/quicklisp; after that the
# build is offline.
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling worker.lisp -> src/worker.wasm + src/worker.js"
java -jar "$jar" "$here/worker.lisp" -o "$here/src/worker.wasm" \
  --no-wasi --optimize=size --emit-js-glue

ls -l "$here/src/worker.wasm" "$here/src/worker.js"
echo "done. Run it with:  npx wrangler dev"


---

# FILE: references/examples/cloudflare-workers/hello-clack/check.lisp

;;; Drive worker.lisp without Cloudflare, on any backend.
;;;
;;; The Worker's entry point is a WASM export, but what sits under it is not:
;;; `dispatch` -- a JSON request string in, a JSON response string out, over the
;;; application clackup stored -- is an ordinary function of the handler
;;; backend, and exactly what the synthesized export calls.
;;;
;;; The two lines before the first --> are upstream clack's. On a Worker
;;; (--no-wasi) they go to a discarding stdout; here they do not.

(load "worker.lisp")

(defun try (target)
  (let ((request
         (rontolisp:json-stringify
          (rontolisp:plist-hash-table
           (list :method "GET"
                 :target target
                 :headers (rontolisp:plist-hash-table
                           (list :host "example.com")))))))
    (format t "~&--> ~a~%" target)
    (format t "<-- ~a~%" (clack.handler.reactor:dispatch request))))

(try "/")
(try "/anything")


---

# FILE: references/examples/cloudflare-workers/hello-clack/package.json

{
  "name": "rontolisp-cloudflare-hello-clack",
  "version": "0.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "build": "./build.sh",
    "dev": "wrangler dev",
    "deploy": "wrangler deploy",
    "tail": "wrangler tail"
  },
  "devDependencies": {
    "wrangler": "^4"
  }
}


---

# FILE: references/examples/cloudflare-workers/hello-clack/src/index.js

// The whole Worker. `./worker.js` is generated from worker.lisp's own
// declarations (build.sh's --emit-js-glue) and owns all of it: instantiation,
// the entropy and the clock a --no-wasi module cannot draw for itself, the
// (ptr, len) staging around the entry point, and the Request -> envelope ->
// Response mapping.
//
// There is no import object to write because there are no imports: on the
// DEFAULT (envelope) boundary every body rides the head, so this module asks
// the host for nothing at all. ../httpbin-clack is the same application on the
// streaming boundary, where the bodies cross as octets through imports of their
// own -- and its src/index.js is the same call, because the glue writes those
// too.
//
// BYTE-IDENTICAL in every hello-* directory here, and in the httpbin-* ones bar
// the client-address hook: how the Lisp half routes is not visible from
// JavaScript. A new sibling copies this file, it does not edit it.

import module from "./worker.wasm";
import { worker } from "./worker.js";

export default worker(module);


---

# FILE: references/examples/cloudflare-workers/hello-clack/src/worker.js

// GENERATED by rontolisp --emit-js-glue -- do not edit.
//
// The host half of this module's boundary, derived from the program's own
// declarations: one import-object entry per rontolisp:wasm-import, one entry
// point per rontolisp:wasm-export, and every piece of linear-memory plumbing
// between them -- the (ptr, len) pair a :string crosses as and the __ronto_alloc
// bracket around a call.
//
// What a declaration cannot state is what a host function DOES, so that is the
// one thing this file asks for: `host` is a plain function per import, keyed by
// import module and field, taking and answering ordinary JavaScript values --
// never a (ptr, len) pair.
//
//   import { instantiate } from "./worker.js";
//
//   const lisp = instantiate(module);
//   lisp.handleRequest(text);
//
// This module's boundary is the reactor envelope, so the Request/Response half
// is derivable too and `worker` below is it:
//
//   import module from "./worker.wasm";
//   import { worker } from "./worker.js";
//
//   export default worker(module);
//
// Regenerate it, never edit it: --emit-js-glue on the compile that wrote the
// .wasm beside it.

const encoder = new TextEncoder();
// ignoreBOM, because these octets are a VALUE and not a document: a
// leading U+FEFF is a character the other side chose, and the default
// decoder deletes it -- silently shortening a BOM-prefixed request body
// by one character while the content-length beside it still counts three.
const decoder = new TextDecoder("utf-8", { ignoreBOM: true });

const SUSPENDING = Symbol.for("rontolisp.suspending");

/**
 * Instantiates the module against this host and returns its callable surface.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @returns {object} `exports`, plus one entry point per rontolisp:wasm-export
 */
export function instantiate(module) {
  let exports;

  // Every view is built where it is used, never held: growing the module's
  // memory DETACHES the buffer behind every view taken before the growth, and a
  // suspending host resumes after growths it never saw.
  const readString = (ptr, len) =>
    decoder.decode(new Uint8Array(exports.memory.buffer, ptr, len));

  // Bytes the HOST hands over live in the module's own bump allocator, and
  // cross as the (ptr, len) pair every memory-typed value is.
  const write = (octets) => {
    const ptr = exports.__ronto_alloc(octets.length);
    new Uint8Array(exports.memory.buffer, ptr, octets.length).set(octets);
    return [ptr, octets.length];
  };
  const writeString = (value) => write(encoder.encode(String(value)));

  // This module imports nothing: instantiating it is the whole boundary.
  const imports = {};
  const instance = new WebAssembly.Instance(module, imports);
  exports = instance.exports;
  // Entropy, BEFORE the top level runs: a --no-wasi module imports none, so its
  // `random` starts from a constant and every instance would draw one sequence.
  // Seeding here also covers the draws a library makes while it LOADS. A Worker
  // forbids this in global scope, so instantiate on the first request.
  exports.__ronto_seed_random(
    new BigUint64Array(crypto.getRandomValues(new Uint8Array(8)).buffer)[0],
  );
  // And a clock, nanoseconds since the Unix epoch, for the same reason and
  // before the same line: until one is set the clock built-ins signal rather
  // than report 1970.
  exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
  exports._initialize();

  const entry$handleRequest = exports["handle-request"];

  // One call into the module: stage the arguments, enter, decode the result out
  // of the scratch it sits in, and only THEN pop the arena that scratch is in.
  // A promising entry answers a promise, so the tail rides `then`; a synchronous
  // one runs the same expression inline. `run` is how the call reaches the
  // module -- through the queue, or straight through when it is already inside
  // the one call the queue admits.
  const call = (run, entry, stage, decode) => {
    const work = () => {
      // The module's clock moves only when the host moves it, so move it per
      // call. Not a workaround for a frozen clock -- a Worker's own Date.now()
      // is frozen for the duration of a request as a timing-attack mitigation,
      // so a value that changes once per call is what the platform has.
      exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
      const mark = exports.__ronto_alloc_mark();
      const done = (raw) => {
        const value = decode(raw);
        exports.__ronto_alloc_reset(mark);
        return value;
      };
      // A TRAP skips the pop above and leaves the module's state half-written
      // with it, so a host that keeps serving instantiates again rather than
      // calling back into this instance.
      const raw = entry(...stage());
      return typeof raw?.then === "function" ? raw.then(done) : done(raw);
    };
    return run(work);
  };

  /** `handle-request` -- (:string) -> :string */
  const make$handleRequest =
    (run) =>
    (p0) => {
      return call(
        run,
        entry$handleRequest,
        () => {
          const a0 = writeString(p0);
          return [a0[0], a0[1]];
        },
        ([ptr, len]) => readString(ptr, len),
      );
    };


  return {
    exports,
    handleRequest: make$handleRequest((work) => work()),
  };
}

/**
 * This module as a fetch handler -- the whole Worker, with nothing left over:
 *
 *   import module from "./worker.wasm";
 *   import { worker } from "./worker.js";
 *
 *   export default worker(module);
 *
 * A Request becomes the envelope the module's entry point takes, the head it
 * answers becomes a Response, and the instance is created on the FIRST REQUEST
 * (a Worker forbids drawing entropy in global scope) and retired if a call ever
 * traps.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @param {object} [options] `remoteAddr` -- (request, env, ctx) => the client
 *   address, since which header carries it is the platform's business and not
 *   this file's (`(r) => r.headers.get("cf-connecting-ip")` on Cloudflare)
 * @returns {object} `{ fetch(request, env, ctx) }`
 */
export function worker(module, options = {}) {
  let instance = null;
  // Set when a call TRAPPED: that instance skipped its arena reset and its Lisp
  // state may be half-written, so nothing else may run on it. A Lisp ERROR is not
  // this -- the transport answers 500 itself and the instance is fine.
  let poisoned = false;
  const live = () => {
    if (poisoned) {
      instance = null;
      poisoned = false;
    }
    return (instance ??= instantiate(module));
  };

  // The request head. `target` stays RAW -- path and query still joined and
  // still percent-encoded -- because the shared normalizer on the other side
  // owns that split, and a pre-split path leaves the query string nil.
  const envelope = (request, octets, remoteAddr) => {
    const url = new URL(request.url);
    const headers = Object.fromEntries(request.headers);
    // A body with no content-length is a body the request parser does not read,
    // and a chunked request carries none -- so set it from the octets we have.
    if (octets?.length) headers["content-length"] = String(octets.length);
    const head = {
      method: request.method,
      target: url.pathname + url.search,
      headers,
      scheme: url.protocol.replace(":", ""),
    };
    if (octets?.length) head.body = decoder.decode(octets);
    if (remoteAddr != null) head["remote-addr"] = remoteAddr;
    return JSON.stringify(head);
  };

  return {
    // EVERYTHING is inside the try, not just the module call: reading an
    // aborted upload rejects, and `new Response` throws on a status or a
    // header an application is free to produce (0, 999, a newline in a
    // value). Outside it those escape as an unhandled rejection, which the
    // platform answers with its own error page and nothing in the log.
    async fetch(request, env, ctx) {
      let entered = false;
      try {
        const remoteAddr = await options.remoteAddr?.(request, env, ctx);
        const octets = request.body
          ? new Uint8Array(await request.arrayBuffer())
          : null;
        const input = envelope(request, octets, remoteAddr);
        entered = true;
        const head = JSON.parse(live().handleRequest(input));
        // Headers as an ARRAY of pairs, which keeps two Set-Cookie two.
        // An EMPTY body becomes null whichever shape it arrived in: 204/205/304
        // may only be constructed with a null body, and "" and a zero-length
        // Uint8Array are the same response as none.
        const body = head.body;
        return new Response(body?.length ? body : null, {
          status: head.status ?? 200,
          headers: head.headers ?? [],
        });
      } catch (error) {
        console.error("handle-request failed:", error);
        // Only a call that ENTERED the module can have left it half-written.
        // A mapping, or a Response the platform refused, says nothing about
        // the instance, and discarding it would cost the next request a
        // reinstantiation for someone else's bad header.
        if (entered) poisoned = true;
        return new Response("internal error\n", { status: 500 });
      }
    },
  };
}


---

# FILE: references/examples/cloudflare-workers/hello-clack/worker.lisp

;;; Clack, plain: an application is a FUNCTION of the environment plist that
;;; returns the (status headers body) list. That is the whole API -- clack has
;;; no router and no request object -- so the same `app` runs on hunchentoot,
;;; on woo, under `wasmtime serve` and on the JVM, unchanged.
;;;
;;; :server :reactor is the handler backend for a host that CALLS you instead
;;; of handing you a socket; the compiler synthesizes the export src/index.js
;;; calls. :use-thread nil keeps clackup in the foreground off WASM.

(ql:quickload '("clack" "clack-handler-reactor"))

(defun app (env)
  (list 200 '(:content-type "text/plain; charset=utf-8")
        (list
         (format nil "Hello from Clack on Cloudflare Workers!~%~a ~a~%"
                 (getf env :request-method) (getf env :path-info)))))

(clack:clackup #'app :server :reactor :use-thread nil)


---

# FILE: references/examples/cloudflare-workers/hello-clack/wrangler.jsonc

{
  // https://developers.cloudflare.com/workers/wrangler/configuration/
  "name": "rontolisp-hello-clack",
  "main": "src/index.js",
  "compatibility_date": "2026-01-01",

  // `import module from "./worker.wasm"` needs no configuration: .wasm is a
  // built-in module rule for ES-module Workers, and the file is compiled at
  // deploy time rather than on the request path.
  "observability": {
    "enabled": true
  }
}


---

# FILE: references/examples/cloudflare-workers/hello-ningle/README.md

# hello-ningle — a Worker whose application is an object

The same greeting as [`../hello-clack`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/hello-clack), written the way
[ningle](https://github.com/fukamachi/ningle) wants it: the application is not a
function at all but a CLOS **object** you hang routes on, and the 404 is a
method you override rather than a route you add.

```bash
./build.sh              # worker.lisp -> src/worker.wasm
npx wrangler dev        # or deploy it
rontolisp check.lisp    # drive the whole Worker locally, on any backend
```

## The whole program

```lisp
(ql:quickload '("clack" "clack-handler-reactor" "ningle"))

(defvar *app* (make-instance 'ningle:app))

(setf (ningle:route *app* "/")
      (format nil "Hello from ningle on Cloudflare Workers!~%"))

(setf (ningle:route *app* "/hello/:name")
      (lambda (params) (format nil "Hello, ~a!~%" (cdr (assoc :name params)))))

(defmethod ningle:not-found ((app ningle:app))
  (setf (lack.response:response-status ningle:*response*) 404)
  (format nil "no route for ~a~%"
          (lack.request:request-path-info ningle:*request*)))

(clack:clackup *app* :server :reactor :use-thread nil)
```

Four things make it ningle:

- **The application is an object, and a route is a `setf`.** There is no
  route-list form, so routes can be added from anywhere — another file, a
  function, run time.
- **A controller does not have to be a function.** The `/` route is a *string*;
  ningle answers a non-function controller as the response body.
- **A controller receives the parameters, not the environment.** The `:name`
  token binds into an alist keyed by the keyword; the request itself is in
  `ningle:*request*`, with `*response*` (mutable — that is how the 404 sets its
  status) and `*session*` beside it.
- **The 404 is an extension point.** `ningle:not-found` is a generic function on
  the application class, so answering "no rule matched" is a `defmethod` on a
  *library* generic.

There is no `defpackage` here, and that is ningle's own idiom: a thin framework
used through qualified names, exactly as its README shows. The tiny-routes
Worker needs one because `(:use :tiny-routes)` is what makes `define-get` and
`ok` unqualified; nothing here is used unqualified, so a package would earn
nothing.

`*app*` is still an ordinary Clack application, so it runs on hunchentoot, on
woo, under `wasmtime serve` and on the JVM; the
[`../hello-clack` README](../hello-clack/README.md) explains the `:server`
designator half. To serve it over a real socket, drop `clack-handler-reactor`
and use `:server :rontolisp` — that is what
[`examples/net/httpbin-ningle.lisp`](../../net/httpbin-ningle.lisp) does.

## What it costs

This is by an order of magnitude the largest of the four hello Workers
([size report](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/size-report/results/cloudflare-workers.md)) — **and
almost none of it is ningle.** The same build with ningle replaced by one
`lack.request:make-request` call is barely smaller, so ningle, its router
[myway](https://github.com/fukamachi/myway) and myway's `map-set` are a fifth of
the difference; the rest is the `lack-request` chain — `http-body`,
`fast-http`'s generated header and multipart state machines, `smart-buffer`,
`circular-streams`, `yason`, `trivial-mimes`, `quri`. tiny-routes never touches
it, because its request IS the Clack environment plist; ningle's `call` reads
`request-headers` / `-method` / `-path-info` / `-parameters` on every request.
There is no size opt-in to offer either, the way tiny-routes has one: myway
compiles every rule to a **cl-ppcre scanner**, so the regex engine is genuinely
reachable. It still fits the free plan's bundle limit with room to spare — a
cost, not a wall — but it is the reason to reach for `tiny-routes` when routing
is all you need.

## Developing without Cloudflare

The synthesized export calls `clack.handler.reactor:dispatch`, an ordinary
function, so the whole Worker — routes, the `not-found` method and all — runs on
every backend, which [`examples/examples.yaml`](../../examples.yaml) pins:

```bash
rontolisp check.lisp
rontolisp check.lisp -o Check.class && java -cp rontolisp-0.1.0-SNAPSHOT-exec.jar:. Check
rontolisp check.lisp -o check.wasm --optimize && wasmtime run -W gc -W exceptions=y check.wasm
```

```console
--> /
<-- {"body":"Hello from ningle on Cloudflare Workers!\n","headers":[],"status":200}
--> /hello/rontolisp
<-- {"body":"Hello, rontolisp!\n","headers":[],"status":200}
--> /anything
<-- {"body":"no route for /anything\n","headers":[],"status":404}
```

(Key order differs per backend — it follows hash-table iteration order — which is
why the manifest checks with `contains`.)

## What's in here

| File | Purpose |
| --- | --- |
| [`worker.lisp`](worker.lisp) | The whole program. This is what `build.sh` compiles. |
| [`check.lisp`](check.lisp) | Drives it with no Cloudflare in sight, on any backend. |
| [`src/index.js`](src/index.js) | Three lines over the generated `worker()`. |
| [`src/worker.js`](src/worker.js) | The boundary, GENERATED by `--emit-js-glue` from `worker.lisp`'s declarations. Byte-identical to `../hello-clack/src/worker.js`, because the declarations are. Do not edit; `./build.sh` rewrites it. |
| `src/worker.wasm` | A build product — run `./build.sh` first. |

## Limitations

The Worker sandbox and `--no-wasi` limitations of
[`../hello-clack`](../hello-clack/README.md#limitations) apply unchanged.

This directory used to open with a blockquote saying it did not run on
Cloudflare at all: `lack-request -> http-body -> fast-http -> smart-buffer` names
a temporary directory with a top-level `(random ...)` over
`uiop:default-temporary-directory`, and on a `--no-wasi` module both halves used
to trap inside `_initialize`, before any export existed. Nothing about ningle
changed; the reactor learned to answer them.

## Rebuilding

```bash
./mvnw clean package -DskipTests   # from the repository root
examples/cloudflare-workers/hello-ningle/build.sh
```


---

# FILE: references/examples/cloudflare-workers/hello-ningle/build.sh

#!/usr/bin/env bash
# Compile worker.lisp -- clack, ningle, the routes and the clackup call -- to
# the .wasm module the Worker imports.
#
# --no-wasi: the Worker calls the exported entry point directly and never runs
#   the module as a program, so it needs no WASI imports at all. It becomes a
#   reactor: nothing to shim on the JavaScript side, and `_initialize` instead
#   of `_start`.
# --optimize=size: a Worker bundle has a size limit, and the tree-shaker is what
#   keeps the module down. ningle has no size opt-in to offer the way
#   tiny-routes does: myway compiles every rule to a cl-ppcre scanner, so the
#   regex engine is genuinely reachable and the shaker is right to keep it.
# --emit-js-glue: write src/worker.js beside the module -- the host half of this
#   boundary, from the program's own declarations. This module imports NOTHING
#   (the default `envelope` boundary keeps every body inside the head), so the
#   whole of that half is fixed by the transport and the glue writes all of it,
#   the Request -> envelope -> Response mapping included. src/index.js is then
#   three lines. It is CHECKED IN and pinned by HostGlueEmitterTest, so
#   regenerate it here rather than editing it.
#
# The first run downloads clack/lack/ningle into ~/.rontolisp/quicklisp; after
# that the build is offline.
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling worker.lisp -> src/worker.wasm + src/worker.js"
java -jar "$jar" "$here/worker.lisp" -o "$here/src/worker.wasm" \
  --no-wasi --optimize=size --emit-js-glue

ls -l "$here/src/worker.wasm" "$here/src/worker.js"
echo "done. Run it with:  npx wrangler dev"


---

# FILE: references/examples/cloudflare-workers/hello-ningle/check.lisp

;;; Drive worker.lisp without Cloudflare, on any backend (../hello-clack/check.lisp
;;; has the notes). The middle probe binds ":name"; the last matches no rule and
;;; so reaches the not-found METHOD.

(load "worker.lisp")

(defun try (target)
  (let ((request
         (rontolisp:json-stringify
          (rontolisp:plist-hash-table
           (list :method "GET"
                 :target target
                 :headers (rontolisp:plist-hash-table
                           (list :host "example.com")))))))
    (format t "~&--> ~a~%" target)
    (format t "<-- ~a~%" (clack.handler.reactor:dispatch request))))

(try "/")
(try "/hello/rontolisp")
(try "/anything")


---

# FILE: references/examples/cloudflare-workers/hello-ningle/package.json

{
  "name": "rontolisp-cloudflare-hello-ningle",
  "version": "0.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "build": "./build.sh",
    "dev": "wrangler dev",
    "deploy": "wrangler deploy",
    "tail": "wrangler tail"
  },
  "devDependencies": {
    "wrangler": "^4"
  }
}


---

# FILE: references/examples/cloudflare-workers/hello-ningle/src/index.js

// The whole Worker. `./worker.js` is generated from worker.lisp's own
// declarations (build.sh's --emit-js-glue) and owns all of it: instantiation,
// the entropy and the clock a --no-wasi module cannot draw for itself, the
// (ptr, len) staging around the entry point, and the Request -> envelope ->
// Response mapping.
//
// There is no import object to write because there are no imports: on the
// DEFAULT (envelope) boundary every body rides the head, so this module asks
// the host for nothing at all. ../httpbin-clack is the same application on the
// streaming boundary, where the bodies cross as octets through imports of their
// own -- and its src/index.js is the same call, because the glue writes those
// too.
//
// BYTE-IDENTICAL in every hello-* directory here, and in the httpbin-* ones bar
// the client-address hook: how the Lisp half routes is not visible from
// JavaScript. A new sibling copies this file, it does not edit it.

import module from "./worker.wasm";
import { worker } from "./worker.js";

export default worker(module);


---

# FILE: references/examples/cloudflare-workers/hello-ningle/src/worker.js

// GENERATED by rontolisp --emit-js-glue -- do not edit.
//
// The host half of this module's boundary, derived from the program's own
// declarations: one import-object entry per rontolisp:wasm-import, one entry
// point per rontolisp:wasm-export, and every piece of linear-memory plumbing
// between them -- the (ptr, len) pair a :string crosses as and the __ronto_alloc
// bracket around a call.
//
// What a declaration cannot state is what a host function DOES, so that is the
// one thing this file asks for: `host` is a plain function per import, keyed by
// import module and field, taking and answering ordinary JavaScript values --
// never a (ptr, len) pair.
//
//   import { instantiate } from "./worker.js";
//
//   const lisp = instantiate(module);
//   lisp.handleRequest(text);
//
// This module's boundary is the reactor envelope, so the Request/Response half
// is derivable too and `worker` below is it:
//
//   import module from "./worker.wasm";
//   import { worker } from "./worker.js";
//
//   export default worker(module);
//
// Regenerate it, never edit it: --emit-js-glue on the compile that wrote the
// .wasm beside it.

const encoder = new TextEncoder();
// ignoreBOM, because these octets are a VALUE and not a document: a
// leading U+FEFF is a character the other side chose, and the default
// decoder deletes it -- silently shortening a BOM-prefixed request body
// by one character while the content-length beside it still counts three.
const decoder = new TextDecoder("utf-8", { ignoreBOM: true });

const SUSPENDING = Symbol.for("rontolisp.suspending");

/**
 * Instantiates the module against this host and returns its callable surface.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @returns {object} `exports`, plus one entry point per rontolisp:wasm-export
 */
export function instantiate(module) {
  let exports;

  // Every view is built where it is used, never held: growing the module's
  // memory DETACHES the buffer behind every view taken before the growth, and a
  // suspending host resumes after growths it never saw.
  const readString = (ptr, len) =>
    decoder.decode(new Uint8Array(exports.memory.buffer, ptr, len));

  // Bytes the HOST hands over live in the module's own bump allocator, and
  // cross as the (ptr, len) pair every memory-typed value is.
  const write = (octets) => {
    const ptr = exports.__ronto_alloc(octets.length);
    new Uint8Array(exports.memory.buffer, ptr, octets.length).set(octets);
    return [ptr, octets.length];
  };
  const writeString = (value) => write(encoder.encode(String(value)));

  // This module imports nothing: instantiating it is the whole boundary.
  const imports = {};
  const instance = new WebAssembly.Instance(module, imports);
  exports = instance.exports;
  // Entropy, BEFORE the top level runs: a --no-wasi module imports none, so its
  // `random` starts from a constant and every instance would draw one sequence.
  // Seeding here also covers the draws a library makes while it LOADS. A Worker
  // forbids this in global scope, so instantiate on the first request.
  exports.__ronto_seed_random(
    new BigUint64Array(crypto.getRandomValues(new Uint8Array(8)).buffer)[0],
  );
  // And a clock, nanoseconds since the Unix epoch, for the same reason and
  // before the same line: until one is set the clock built-ins signal rather
  // than report 1970.
  exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
  exports._initialize();

  const entry$handleRequest = exports["handle-request"];

  // One call into the module: stage the arguments, enter, decode the result out
  // of the scratch it sits in, and only THEN pop the arena that scratch is in.
  // A promising entry answers a promise, so the tail rides `then`; a synchronous
  // one runs the same expression inline. `run` is how the call reaches the
  // module -- through the queue, or straight through when it is already inside
  // the one call the queue admits.
  const call = (run, entry, stage, decode) => {
    const work = () => {
      // The module's clock moves only when the host moves it, so move it per
      // call. Not a workaround for a frozen clock -- a Worker's own Date.now()
      // is frozen for the duration of a request as a timing-attack mitigation,
      // so a value that changes once per call is what the platform has.
      exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
      const mark = exports.__ronto_alloc_mark();
      const done = (raw) => {
        const value = decode(raw);
        exports.__ronto_alloc_reset(mark);
        return value;
      };
      // A TRAP skips the pop above and leaves the module's state half-written
      // with it, so a host that keeps serving instantiates again rather than
      // calling back into this instance.
      const raw = entry(...stage());
      return typeof raw?.then === "function" ? raw.then(done) : done(raw);
    };
    return run(work);
  };

  /** `handle-request` -- (:string) -> :string */
  const make$handleRequest =
    (run) =>
    (p0) => {
      return call(
        run,
        entry$handleRequest,
        () => {
          const a0 = writeString(p0);
          return [a0[0], a0[1]];
        },
        ([ptr, len]) => readString(ptr, len),
      );
    };


  return {
    exports,
    handleRequest: make$handleRequest((work) => work()),
  };
}

/**
 * This module as a fetch handler -- the whole Worker, with nothing left over:
 *
 *   import module from "./worker.wasm";
 *   import { worker } from "./worker.js";
 *
 *   export default worker(module);
 *
 * A Request becomes the envelope the module's entry point takes, the head it
 * answers becomes a Response, and the instance is created on the FIRST REQUEST
 * (a Worker forbids drawing entropy in global scope) and retired if a call ever
 * traps.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @param {object} [options] `remoteAddr` -- (request, env, ctx) => the client
 *   address, since which header carries it is the platform's business and not
 *   this file's (`(r) => r.headers.get("cf-connecting-ip")` on Cloudflare)
 * @returns {object} `{ fetch(request, env, ctx) }`
 */
export function worker(module, options = {}) {
  let instance = null;
  // Set when a call TRAPPED: that instance skipped its arena reset and its Lisp
  // state may be half-written, so nothing else may run on it. A Lisp ERROR is not
  // this -- the transport answers 500 itself and the instance is fine.
  let poisoned = false;
  const live = () => {
    if (poisoned) {
      instance = null;
      poisoned = false;
    }
    return (instance ??= instantiate(module));
  };

  // The request head. `target` stays RAW -- path and query still joined and
  // still percent-encoded -- because the shared normalizer on the other side
  // owns that split, and a pre-split path leaves the query string nil.
  const envelope = (request, octets, remoteAddr) => {
    const url = new URL(request.url);
    const headers = Object.fromEntries(request.headers);
    // A body with no content-length is a body the request parser does not read,
    // and a chunked request carries none -- so set it from the octets we have.
    if (octets?.length) headers["content-length"] = String(octets.length);
    const head = {
      method: request.method,
      target: url.pathname + url.search,
      headers,
      scheme: url.protocol.replace(":", ""),
    };
    if (octets?.length) head.body = decoder.decode(octets);
    if (remoteAddr != null) head["remote-addr"] = remoteAddr;
    return JSON.stringify(head);
  };

  return {
    // EVERYTHING is inside the try, not just the module call: reading an
    // aborted upload rejects, and `new Response` throws on a status or a
    // header an application is free to produce (0, 999, a newline in a
    // value). Outside it those escape as an unhandled rejection, which the
    // platform answers with its own error page and nothing in the log.
    async fetch(request, env, ctx) {
      let entered = false;
      try {
        const remoteAddr = await options.remoteAddr?.(request, env, ctx);
        const octets = request.body
          ? new Uint8Array(await request.arrayBuffer())
          : null;
        const input = envelope(request, octets, remoteAddr);
        entered = true;
        const head = JSON.parse(live().handleRequest(input));
        // Headers as an ARRAY of pairs, which keeps two Set-Cookie two.
        // An EMPTY body becomes null whichever shape it arrived in: 204/205/304
        // may only be constructed with a null body, and "" and a zero-length
        // Uint8Array are the same response as none.
        const body = head.body;
        return new Response(body?.length ? body : null, {
          status: head.status ?? 200,
          headers: head.headers ?? [],
        });
      } catch (error) {
        console.error("handle-request failed:", error);
        // Only a call that ENTERED the module can have left it half-written.
        // A mapping, or a Response the platform refused, says nothing about
        // the instance, and discarding it would cost the next request a
        // reinstantiation for someone else's bad header.
        if (entered) poisoned = true;
        return new Response("internal error\n", { status: 500 });
      }
    },
  };
}


---

# FILE: references/examples/cloudflare-workers/hello-ningle/worker.lisp

;;; ningle: the application is a CLOS OBJECT and a route is an assignment, so
;;; routes can be added anywhere -- in a loop, from another file, at run time.
;;; A controller returns the BODY and says the rest by mutating ningle:*response*
;;; (the response object bound for this request); the 404 is ningle:not-found, a
;;; METHOD on the application class, not a route at the bottom of a list.

(ql:quickload '("clack" "clack-handler-reactor" "ningle"))

(defvar *app* (make-instance 'ningle:app))

;; A controller does not have to be a function: a bare value IS one.
(setf (ningle:route *app* "/")
      (format nil "Hello from ningle on Cloudflare Workers!~%"))

;; A :name token binds one path segment into the parameter alist.
(setf (ningle:route *app* "/hello/:name")
      (lambda (params) (format nil "Hello, ~a!~%" (cdr (assoc :name params)))))

(defmethod ningle:not-found ((app ningle:app))
  (setf (lack.response:response-status ningle:*response*) 404)
  (format nil "no route for ~a~%"
          (lack.request:request-path-info ningle:*request*)))

(clack:clackup *app* :server :reactor :use-thread nil)


---

# FILE: references/examples/cloudflare-workers/hello-ningle/wrangler.jsonc

{
  // https://developers.cloudflare.com/workers/wrangler/configuration/
  "name": "rontolisp-hello-ningle",
  "main": "src/index.js",
  "compatibility_date": "2026-01-01",

  // `import module from "./worker.wasm"` needs no configuration: .wasm is a
  // built-in module rule for ES-module Workers, and the file is compiled at
  // deploy time rather than on the request path.
  "observability": {
    "enabled": true
  }
}


---

# FILE: references/examples/cloudflare-workers/hello-tiny-routes/README.md

# hello-tiny-routes — a Worker composed out of routes

The same greeting as [`../hello-clack`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/hello-clack), written the way
[tiny-routes](https://github.com/jeko2000/tiny-routes) wants it: the application
is not a function you write but one the library **composes** — a route table,
threaded through middleware.

```bash
./build.sh          # worker.lisp -> src/worker.wasm
npx wrangler dev    # http://localhost:8787
npx wrangler deploy
```

```console
$ curl http://localhost:8787/
Hello from tiny-routes on Cloudflare Workers!

$ curl -i http://localhost:8787/anything
HTTP/1.1 404 Not Found
Content-Type: text/plain; charset=utf-8

no route for /anything
```

## The whole program

```lisp
(ql:quickload '("clack" "clack-handler-reactor" "tiny-routes/lite"))

(tiny:define-routes *routes*
  (tiny:define-get "/" ()
    (tiny:ok (format nil "Hello from tiny-routes on Cloudflare Workers!~%")))
  (tiny:define-get "/hello/:name" (req)
    (tiny:ok (format nil "Hello, ~a!~%" (tiny:path-parameter req :name))))
  (tiny:define-any "*" (req)
    (tiny:not-found (format nil "no route for ~a~%" (tiny:path-info req)))))

(defparameter *app*
  (tiny:pipe *routes*
             (tiny:wrap-response-content-type "text/plain; charset=utf-8")))

(clack:clackup *app* :server :reactor :use-thread nil)
```

Four things make it tiny-routes rather than a `cond` in disguise:

- **A route is a handler, and `nil` DECLINES.** `define-routes` is "try these in
  order, take the first non-`nil`", which is why the last route, `"*"`, is the
  404 and needs no special mechanism.
- **A path template binds parameters.** `"/hello/:name"` matches one segment and
  `path-parameter` reads it; `/hello` and `/hello/` do not match it, so they
  decline into the 404 like anything else.
- **Response constructors name the status.** `ok` and `not-found` build the
  Clack triple instead of spelling the number.
- **`pipe` threads the table through middleware.** `wrap-response-content-type`
  sets the header for every route at once, so no route sets one — and
  [`../httpbin-tiny-routes`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin-tiny-routes) uses the same seam to read
  the request body and parse the query string.

`tiny` is the library's own nickname, so there is no `defpackage` here either:
every name it contributes is reachable qualified, which keeps the library's
surface visible at each call site.

`*app*` is still an ordinary Clack application, so it runs on hunchentoot, on
woo, under `wasmtime serve` and on the JVM; the
[`../hello-clack` README](../hello-clack/README.md) explains the `:server`
designator half.

## `tiny-routes/lite`, and why it is on the `quickload` line

A tiny-routes path template compiles to a **cl-ppcre scanner at run time**, so
in a compiled module the whole regex engine is genuinely reachable and the
tree-shaker is right to keep it. `"tiny-routes/lite"` is the opt-in system: the
same source tree with the path-template matcher swapped for a ppcre-free one and
the `:cl-ppcre` dependency dropped with it. It accepts templates of literal
characters and `:name` tokens, matches them exactly as the full system does, and
**refuses at route-build time** on a regex metacharacter or a `:regex t`
template — which under `--no-wasi` means at `_initialize`, so it is a build-time
decision rather than a request-time surprise. Exact subset: the
[ASDF systems guide](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/doc/en/guides/asdf-systems.md); what it is worth
in bytes: the
[size report](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/size-report/results/cloudflare-workers.md).

## Developing without Cloudflare

As in [`../hello-clack`](../hello-clack/README.md#developing-without-cloudflare):
the synthesized export calls `clack.handler.reactor:dispatch`, an ordinary
function, so the whole Worker runs on every backend, which
[`examples/examples.yaml`](../../examples.yaml) pins:

```bash
rontolisp check.lisp
rontolisp check.lisp -o Check.class && java -cp rontolisp-0.1.0-SNAPSHOT-exec.jar:. Check
rontolisp check.lisp -o check.wasm --optimize && wasmtime run -W gc -W exceptions=y check.wasm
```

The first build downloads clack, lack and tiny-routes into
`~/.rontolisp/quicklisp`; after that everything is offline, because the
`ql:quickload` is resolved at **compile** time and inlined into the module.

## What's in here

| File | Purpose |
| --- | --- |
| [`worker.lisp`](worker.lisp) | The whole program. This is what `build.sh` compiles. |
| [`check.lisp`](check.lisp) | Drives it with no Cloudflare in sight, on any backend. |
| [`src/index.js`](src/index.js) | Three lines over the generated `worker()`. |
| [`src/worker.js`](src/worker.js) | The boundary, GENERATED by `--emit-js-glue` from `worker.lisp`'s declarations. Byte-identical to `../hello-clack/src/worker.js`, because the declarations are. Do not edit; `./build.sh` rewrites it. |
| `src/worker.wasm` | A build product — run `./build.sh` first. |

## Limitations

The Worker sandbox and `--no-wasi` limitations of
[`../hello-clack`](../hello-clack/README.md#limitations) apply unchanged.

## Rebuilding

```bash
./mvnw clean package -DskipTests   # from the repository root
examples/cloudflare-workers/hello-tiny-routes/build.sh
```


---

# FILE: references/examples/cloudflare-workers/hello-tiny-routes/build.sh

#!/usr/bin/env bash
# Compile worker.lisp -- clack, tiny-routes/lite, the routes and the clackup
# call -- to the .wasm module the Worker imports.
#
# --no-wasi: the Worker calls the exported entry point directly and never runs
#   the module as a program, so it needs no WASI imports at all. It becomes a
#   reactor: nothing to shim on the JavaScript side, and `_initialize` instead
#   of `_start`.
# --optimize=size: a Worker bundle has a size limit, and the tree-shaker is what
#   keeps the module down. Routing adds to what it has to keep, which is why
#   worker.lisp asks for "tiny-routes/lite": the full system reaches cl-ppcre at
#   RUN time, so the whole regex engine would ship with it.
# --emit-js-glue: write src/worker.js beside the module -- the host half of this
#   boundary, from the program's own declarations. This module imports NOTHING
#   (the default `envelope` boundary keeps every body inside the head), so the
#   whole of that half is fixed by the transport and the glue writes all of it,
#   the Request -> envelope -> Response mapping included. src/index.js is then
#   three lines. It is CHECKED IN and pinned by HostGlueEmitterTest, so
#   regenerate it here rather than editing it.
#
# The first run downloads clack/lack/tiny-routes into ~/.rontolisp/quicklisp;
# after that the build is offline.
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling worker.lisp -> src/worker.wasm + src/worker.js"
java -jar "$jar" "$here/worker.lisp" -o "$here/src/worker.wasm" \
  --no-wasi --optimize=size --emit-js-glue

ls -l "$here/src/worker.wasm" "$here/src/worker.js"
echo "done. Run it with:  npx wrangler dev"


---

# FILE: references/examples/cloudflare-workers/hello-tiny-routes/check.lisp

;;; Drive worker.lisp without Cloudflare, on any backend (../hello-clack/check.lisp
;;; has the notes). The middle probes bind ":name"; the last declines into the
;;; catch-all 404.

(load "worker.lisp")

(defun try (target)
  (let ((request
         (rontolisp:json-stringify
          (rontolisp:plist-hash-table
           (list :method "GET"
                 :target target
                 :headers (rontolisp:plist-hash-table
                           (list :host "example.com")))))))
    (format t "~&--> ~a~%" target)
    (format t "<-- ~a~%" (clack.handler.reactor:dispatch request))))

(try "/")
(try "/hello/rontolisp")
(try "/hello/world")
(try "/anything")


---

# FILE: references/examples/cloudflare-workers/hello-tiny-routes/package.json

{
  "name": "rontolisp-cloudflare-hello-tiny-routes",
  "version": "0.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "build": "./build.sh",
    "dev": "wrangler dev",
    "deploy": "wrangler deploy",
    "tail": "wrangler tail"
  },
  "devDependencies": {
    "wrangler": "^4"
  }
}


---

# FILE: references/examples/cloudflare-workers/hello-tiny-routes/src/index.js

// The whole Worker. `./worker.js` is generated from worker.lisp's own
// declarations (build.sh's --emit-js-glue) and owns all of it: instantiation,
// the entropy and the clock a --no-wasi module cannot draw for itself, the
// (ptr, len) staging around the entry point, and the Request -> envelope ->
// Response mapping.
//
// There is no import object to write because there are no imports: on the
// DEFAULT (envelope) boundary every body rides the head, so this module asks
// the host for nothing at all. ../httpbin-clack is the same application on the
// streaming boundary, where the bodies cross as octets through imports of their
// own -- and its src/index.js is the same call, because the glue writes those
// too.
//
// BYTE-IDENTICAL in every hello-* directory here, and in the httpbin-* ones bar
// the client-address hook: how the Lisp half routes is not visible from
// JavaScript. A new sibling copies this file, it does not edit it.

import module from "./worker.wasm";
import { worker } from "./worker.js";

export default worker(module);


---

# FILE: references/examples/cloudflare-workers/hello-tiny-routes/src/worker.js

// GENERATED by rontolisp --emit-js-glue -- do not edit.
//
// The host half of this module's boundary, derived from the program's own
// declarations: one import-object entry per rontolisp:wasm-import, one entry
// point per rontolisp:wasm-export, and every piece of linear-memory plumbing
// between them -- the (ptr, len) pair a :string crosses as and the __ronto_alloc
// bracket around a call.
//
// What a declaration cannot state is what a host function DOES, so that is the
// one thing this file asks for: `host` is a plain function per import, keyed by
// import module and field, taking and answering ordinary JavaScript values --
// never a (ptr, len) pair.
//
//   import { instantiate } from "./worker.js";
//
//   const lisp = instantiate(module);
//   lisp.handleRequest(text);
//
// This module's boundary is the reactor envelope, so the Request/Response half
// is derivable too and `worker` below is it:
//
//   import module from "./worker.wasm";
//   import { worker } from "./worker.js";
//
//   export default worker(module);
//
// Regenerate it, never edit it: --emit-js-glue on the compile that wrote the
// .wasm beside it.

const encoder = new TextEncoder();
// ignoreBOM, because these octets are a VALUE and not a document: a
// leading U+FEFF is a character the other side chose, and the default
// decoder deletes it -- silently shortening a BOM-prefixed request body
// by one character while the content-length beside it still counts three.
const decoder = new TextDecoder("utf-8", { ignoreBOM: true });

const SUSPENDING = Symbol.for("rontolisp.suspending");

/**
 * Instantiates the module against this host and returns its callable surface.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @returns {object} `exports`, plus one entry point per rontolisp:wasm-export
 */
export function instantiate(module) {
  let exports;

  // Every view is built where it is used, never held: growing the module's
  // memory DETACHES the buffer behind every view taken before the growth, and a
  // suspending host resumes after growths it never saw.
  const readString = (ptr, len) =>
    decoder.decode(new Uint8Array(exports.memory.buffer, ptr, len));

  // Bytes the HOST hands over live in the module's own bump allocator, and
  // cross as the (ptr, len) pair every memory-typed value is.
  const write = (octets) => {
    const ptr = exports.__ronto_alloc(octets.length);
    new Uint8Array(exports.memory.buffer, ptr, octets.length).set(octets);
    return [ptr, octets.length];
  };
  const writeString = (value) => write(encoder.encode(String(value)));

  // This module imports nothing: instantiating it is the whole boundary.
  const imports = {};
  const instance = new WebAssembly.Instance(module, imports);
  exports = instance.exports;
  // Entropy, BEFORE the top level runs: a --no-wasi module imports none, so its
  // `random` starts from a constant and every instance would draw one sequence.
  // Seeding here also covers the draws a library makes while it LOADS. A Worker
  // forbids this in global scope, so instantiate on the first request.
  exports.__ronto_seed_random(
    new BigUint64Array(crypto.getRandomValues(new Uint8Array(8)).buffer)[0],
  );
  // And a clock, nanoseconds since the Unix epoch, for the same reason and
  // before the same line: until one is set the clock built-ins signal rather
  // than report 1970.
  exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
  exports._initialize();

  const entry$handleRequest = exports["handle-request"];

  // One call into the module: stage the arguments, enter, decode the result out
  // of the scratch it sits in, and only THEN pop the arena that scratch is in.
  // A promising entry answers a promise, so the tail rides `then`; a synchronous
  // one runs the same expression inline. `run` is how the call reaches the
  // module -- through the queue, or straight through when it is already inside
  // the one call the queue admits.
  const call = (run, entry, stage, decode) => {
    const work = () => {
      // The module's clock moves only when the host moves it, so move it per
      // call. Not a workaround for a frozen clock -- a Worker's own Date.now()
      // is frozen for the duration of a request as a timing-attack mitigation,
      // so a value that changes once per call is what the platform has.
      exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
      const mark = exports.__ronto_alloc_mark();
      const done = (raw) => {
        const value = decode(raw);
        exports.__ronto_alloc_reset(mark);
        return value;
      };
      // A TRAP skips the pop above and leaves the module's state half-written
      // with it, so a host that keeps serving instantiates again rather than
      // calling back into this instance.
      const raw = entry(...stage());
      return typeof raw?.then === "function" ? raw.then(done) : done(raw);
    };
    return run(work);
  };

  /** `handle-request` -- (:string) -> :string */
  const make$handleRequest =
    (run) =>
    (p0) => {
      return call(
        run,
        entry$handleRequest,
        () => {
          const a0 = writeString(p0);
          return [a0[0], a0[1]];
        },
        ([ptr, len]) => readString(ptr, len),
      );
    };


  return {
    exports,
    handleRequest: make$handleRequest((work) => work()),
  };
}

/**
 * This module as a fetch handler -- the whole Worker, with nothing left over:
 *
 *   import module from "./worker.wasm";
 *   import { worker } from "./worker.js";
 *
 *   export default worker(module);
 *
 * A Request becomes the envelope the module's entry point takes, the head it
 * answers becomes a Response, and the instance is created on the FIRST REQUEST
 * (a Worker forbids drawing entropy in global scope) and retired if a call ever
 * traps.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @param {object} [options] `remoteAddr` -- (request, env, ctx) => the client
 *   address, since which header carries it is the platform's business and not
 *   this file's (`(r) => r.headers.get("cf-connecting-ip")` on Cloudflare)
 * @returns {object} `{ fetch(request, env, ctx) }`
 */
export function worker(module, options = {}) {
  let instance = null;
  // Set when a call TRAPPED: that instance skipped its arena reset and its Lisp
  // state may be half-written, so nothing else may run on it. A Lisp ERROR is not
  // this -- the transport answers 500 itself and the instance is fine.
  let poisoned = false;
  const live = () => {
    if (poisoned) {
      instance = null;
      poisoned = false;
    }
    return (instance ??= instantiate(module));
  };

  // The request head. `target` stays RAW -- path and query still joined and
  // still percent-encoded -- because the shared normalizer on the other side
  // owns that split, and a pre-split path leaves the query string nil.
  const envelope = (request, octets, remoteAddr) => {
    const url = new URL(request.url);
    const headers = Object.fromEntries(request.headers);
    // A body with no content-length is a body the request parser does not read,
    // and a chunked request carries none -- so set it from the octets we have.
    if (octets?.length) headers["content-length"] = String(octets.length);
    const head = {
      method: request.method,
      target: url.pathname + url.search,
      headers,
      scheme: url.protocol.replace(":", ""),
    };
    if (octets?.length) head.body = decoder.decode(octets);
    if (remoteAddr != null) head["remote-addr"] = remoteAddr;
    return JSON.stringify(head);
  };

  return {
    // EVERYTHING is inside the try, not just the module call: reading an
    // aborted upload rejects, and `new Response` throws on a status or a
    // header an application is free to produce (0, 999, a newline in a
    // value). Outside it those escape as an unhandled rejection, which the
    // platform answers with its own error page and nothing in the log.
    async fetch(request, env, ctx) {
      let entered = false;
      try {
        const remoteAddr = await options.remoteAddr?.(request, env, ctx);
        const octets = request.body
          ? new Uint8Array(await request.arrayBuffer())
          : null;
        const input = envelope(request, octets, remoteAddr);
        entered = true;
        const head = JSON.parse(live().handleRequest(input));
        // Headers as an ARRAY of pairs, which keeps two Set-Cookie two.
        // An EMPTY body becomes null whichever shape it arrived in: 204/205/304
        // may only be constructed with a null body, and "" and a zero-length
        // Uint8Array are the same response as none.
        const body = head.body;
        return new Response(body?.length ? body : null, {
          status: head.status ?? 200,
          headers: head.headers ?? [],
        });
      } catch (error) {
        console.error("handle-request failed:", error);
        // Only a call that ENTERED the module can have left it half-written.
        // A mapping, or a Response the platform refused, says nothing about
        // the instance, and discarding it would cost the next request a
        // reinstantiation for someone else's bad header.
        if (entered) poisoned = true;
        return new Response("internal error\n", { status: 500 });
      }
    },
  };
}


---

# FILE: references/examples/cloudflare-workers/hello-tiny-routes/worker.lisp

;;; tiny-routes: an application is COMPOSED. `define-routes` builds a handler
;;; that tries each route and takes the first non-nil answer, so returning nil
;;; DECLINES and "*" is the 404; `ok` / `not-found` name the status instead of
;;; spelling the triple; and `pipe` threads the whole table through middleware
;;; -- here the one that gives every response its content type. `tiny` is the
;;; library's own nickname, so nothing has to be imported to reach any of it.
;;;
;;; "tiny-routes/lite" is the opt-in system whose ppcre-free path-template
;;; matcher keeps the regex engine out of the module. It takes literal
;;; characters and :name tokens, and refuses a regex-shaped template.

(ql:quickload '("clack" "clack-handler-reactor" "tiny-routes/lite"))

(tiny:define-routes *routes*
  (tiny:define-get "/" ()
    (tiny:ok (format nil "Hello from tiny-routes on Cloudflare Workers!~%")))
  (tiny:define-get "/hello/:name" (req)
    (tiny:ok (format nil "Hello, ~a!~%" (tiny:path-parameter req :name))))
  (tiny:define-any "*" (req)
    (tiny:not-found (format nil "no route for ~a~%" (tiny:path-info req)))))

(defparameter *app*
  (tiny:pipe *routes*
             (tiny:wrap-response-content-type "text/plain; charset=utf-8")))

(clack:clackup *app* :server :reactor :use-thread nil)


---

# FILE: references/examples/cloudflare-workers/hello-tiny-routes/wrangler.jsonc

{
  // https://developers.cloudflare.com/workers/wrangler/configuration/
  "name": "rontolisp-hello-tiny-routes",
  "main": "src/index.js",
  "compatibility_date": "2026-01-01",

  // `import module from "./worker.wasm"` needs no configuration: .wasm is a
  // built-in module rule for ES-module Workers, and the file is compiled at
  // deploy time rather than on the request path.
  "observability": {
    "enabled": true
  }
}


---

# FILE: references/examples/cloudflare-workers/hello/README.md

# hello — the smallest rontolisp Worker

Three Lisp functions ([`worker.lisp`](worker.lisp)) that the Worker calls the
way it would call any JavaScript function. No library, no allocator, no WASI
shim: the compiled module imports **nothing**, and
[`src/index.js`](src/index.js) is the entire host side.

```bash
./build.sh          # worker.lisp -> src/worker.wasm
npx wrangler dev    # http://localhost:8787
npx wrangler deploy
```

```console
$ curl http://localhost:8787/
Hello from Lisp, compiled to WebAssembly!
$ curl 'http://localhost:8787/add?a=2&b=3'
2 + 3 = 5
$ curl 'http://localhost:8787/fib?n=20'
fib(20) = 6765
```

## The module

```lisp
(rontolisp:wasm-export 'add   :params '(:s32 :s32) :returns :s32)
(rontolisp:wasm-export 'fib   :params '(:s32)      :returns :s32)
(rontolisp:wasm-export 'greet :params '()          :returns :string)
```

Compiled with `--no-gc --optimize`, that is a plain MVP module — no wasm-GC, no
WASI, nothing to link against:

```console
$ node -e 'const m = new WebAssembly.Module(require("fs").readFileSync("src/worker.wasm"));
           console.log(WebAssembly.Module.imports(m), WebAssembly.Module.exports(m).map(e => e.name))'
[] [ 'add', 'fib', 'greet', 'memory', '__ronto_alloc', '__ronto_alloc_mark', '__ronto_alloc_reset' ]
```

so instantiating it is one line, once per isolate, and `lisp.add(2, 3)` returns
`5` — a `:s32` is an i32 on both sides:

```js
const lisp = new WebAssembly.Instance(module, {}).exports;
```

## Strings, and why there is no bookkeeping here

WebAssembly has no string type, so `greet` returns **two i32 values** — a
pointer into linear memory and a length — and the host decodes the bytes:

```js
const [ptr, len] = lisp.greet();
new TextDecoder().decode(new Uint8Array(lisp.memory.buffer, ptr, len));
```

Linear memory is not garbage collected, so in general those bytes are the
host's to reclaim. Note what is *absent* here: the module exports
`__ronto_alloc` and the arena pair `__ronto_alloc_mark`/`__ronto_alloc_reset`,
and this Worker never touches them. Nothing crosses the boundary *into* the
module, so the host never allocates and the returned string lands in a fixed
scratch area the next call reuses — measured, 150 000 calls to `add` + `fib` +
`greet` leave linear memory exactly where it started.

Pass a string *in* and that changes: see [`../httpbin`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin) and its
[Two heaps](../httpbin/README.md#two-heaps) section.

## The non-GC subset

`--no-gc` is what makes this module tiny and dependency-free, and it is
available because `worker.lisp` stays inside the
[numeric/string subset](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/doc/en/guides/wasm-nogc.md): integers, a string
literal, `dotimes`. Add a cons cell, a hash table or the JSON library and the
build needs the full language — `--no-wasi` instead, which is what `../httpbin`
does.

## Rebuilding

```bash
./mvnw clean package -DskipTests   # from the repository root
examples/cloudflare-workers/hello/build.sh
```


---

# FILE: references/examples/cloudflare-workers/hello/build.sh

#!/usr/bin/env bash
# Compile worker.lisp to the .wasm module the Worker imports.
#
# --no-gc: this program is inside the non-GC subset (integers and a string
#   literal, no cons cells or hash tables), so it compiles to a plain MVP module
#   -- no wasm-GC, no WASI, no imports at all.
# --optimize: the dead-code tree-shaker; only what the exports reach ships.
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling worker.lisp -> src/worker.wasm"
java -jar "$jar" "$here/worker.lisp" -o "$here/src/worker.wasm" --no-gc --optimize

ls -l "$here/src/worker.wasm"
echo "done. Run it with:  npx wrangler dev"


---

# FILE: references/examples/cloudflare-workers/hello/package.json

{
  "name": "rontolisp-cloudflare-hello",
  "version": "0.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "build": "./build.sh",
    "dev": "wrangler dev",
    "deploy": "wrangler deploy"
  },
  "devDependencies": {
    "wrangler": "^4"
  }
}


---

# FILE: references/examples/cloudflare-workers/hello/src/index.js

// index.js -- the whole Worker. There is nothing else in this directory's
// JavaScript: no WASI shim, no allocator, no bindings library.

import module from "./worker.wasm";

// The module imports nothing, so instantiating it is one synchronous line with
// an empty import object -- and it happens once per isolate, not per request.
//
// One line, unlike ../httpbin: this module exports no `_initialize` (--no-gc,
// and worker.lisp has no top-level forms, so there is nothing to run), and none
// of these three functions can trap and leave the instance unusable.
const lisp = new WebAssembly.Instance(module, {}).exports;

/** Decode a (pointer, length) result out of the module's linear memory. */
function readString([ptr, len]) {
  return new TextDecoder().decode(new Uint8Array(lisp.memory.buffer, ptr, len));
}

export default {
  async fetch(request) {
    const { pathname, searchParams } = new URL(request.url);
    const number = (name, fallback) => Number(searchParams.get(name) ?? fallback);

    switch (pathname) {
      case "/":
        // A `:string` result: two i32s naming bytes in linear memory.
        return text(readString(lisp.greet()));

      case "/add": {
        // A `:s32` result: JavaScript just gets a number back.
        const [a, b] = [number("a", 2), number("b", 3)];
        return text(`${a} + ${b} = ${lisp.add(a, b)}`);
      }

      case "/fib": {
        const n = number("n", 20);
        return text(`fib(${n}) = ${lisp.fib(n)}`);
      }

      default:
        return text(`no route for ${pathname}\n\ntry /, /add?a=2&b=3, /fib?n=20`, 404);
    }
  },
};

function text(body, status = 200) {
  return new Response(body + "\n", {
    status,
    headers: { "content-type": "text/plain; charset=utf-8" },
  });
}


---

# FILE: references/examples/cloudflare-workers/hello/worker.lisp

;;; No library at all: three Lisp functions JavaScript calls directly.
;;;
;;; rontolisp:wasm-export gives each one a host-callable signature. A :string
;;; comes back as a (pointer, length) pair into linear memory, which
;;; src/index.js decodes; nothing crosses INTO the module, so the host never
;;; allocates. build.sh compiles this with --no-gc -- a plain MVP module.

(rontolisp:wasm-export 'add :params '(:s32 :s32) :returns :s32)
(rontolisp:wasm-export 'fib :params '(:s32) :returns :s32)
(rontolisp:wasm-export 'greet :params '() :returns :string)

(defun add (a b) (+ a b))

(defun fib (n)
  "The nth Fibonacci number, computed iteratively."
  (let ((a 0) (b 1))
    (dotimes (i n)
      (let ((next (+ a b)))
        (setq a b)
        (setq b next)))
    a))

(defun greet () "Hello from Lisp, compiled to WebAssembly!")


---

# FILE: references/examples/cloudflare-workers/hello/wrangler.jsonc

{
  // https://developers.cloudflare.com/workers/wrangler/configuration/
  "name": "rontolisp-hello",
  "main": "src/index.js",
  "compatibility_date": "2026-01-01"
}


---

# FILE: references/examples/cloudflare-workers/httpbin-clack-one-source/README.md

# httpbin-clack-one-source — one Clack file, every host, this one included

There is **no `worker.lisp` in this directory**, and that is the point. The
program is [`net/httpbin-clack.lisp`](../../net/httpbin-clack.lisp) *itself* —
the file that serves these endpoints on the interpreter, on the JVM and under
`wasmtime serve` — compiled here unchanged for a host that calls an export
instead of handing over a socket.

```bash
./build.sh          # ../../net/httpbin-clack.lisp -> src/worker.wasm
npx wrangler dev    # http://localhost:8787
npx wrangler deploy
```

```console
$ curl 'http://localhost:8787/get?a=1&b=two'
{"args":{"a":"1","b":"two"},"headers":{"host":"localhost:8787",...},"method":"GET","path":"/get"}
```

## One source, four hosts

`:server :rontolisp` means "serve on **this target's** native inbound
transport", and the transport is chosen at *compile* time:

| Build | Transport | How it runs |
| --- | --- | --- |
| interpret / `-o App.class` | the program binds a socket | `rontolisp ../../net/httpbin-clack.lisp`, then `curl :8080` |
| `-o app.wasm --component` | the host owns the socket (wasi:http) | `wasmtime serve ... app.wasm` |
| `-o worker.wasm --no-wasi` | the host **calls** the module — a reactor | this directory: the generated `src/worker.js` calls `handle-request` |

The `clackup` line does not change between the rows: `:port 8080` applies where
the program owns the socket and is ignored where the host does, and
`:use-thread nil` keeps the interpreter and the JVM serving in the foreground.
Deploying to Cloudflare is not a port of the program; it is a compile flag.
WASM Preview 1 is the one host where `clackup` cannot serve — it has no incoming
TCP, so the program compiles and `clackup` fails at run time.

```bash
rontolisp ../../net/httpbin-clack.lisp                                  # :8080
rontolisp ../../net/httpbin-clack.lisp -o Serve.class && \
  java -cp rontolisp-0.1.0-SNAPSHOT-exec.jar:. Serve
rontolisp ../../net/httpbin-clack.lisp -o serve.wasm --component && \
  wasmtime serve -W gc=y -W exceptions=y -S cli=y -S tcp=y -S inherit-network=y serve.wasm
rontolisp ../../net/httpbin-clack.lisp -o src/worker.wasm --no-wasi --optimize=size
```

[`examples.yaml`](../../examples.yaml) pins the first three (a blocking server,
so the manifest builds it rather than running it); the fourth is this
directory's `build.sh`.

The most direct check that this is a Clack application is to serve it as one and
point the same `curl`s at both: nothing is recompiled and nothing edited between
them. As in `../httpbin`, the **order of keys inside a JSON object differs
between backends** — it follows hash-table iteration order.

## How it differs from `../httpbin-clack`

Only in the `:server` designator, and each answers a different question.

| | [`../httpbin-clack`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin-clack) | this |
| --- | --- | --- |
| The program | its own `worker.lisp` | `net/httpbin-clack.lisp`, not a copy |
| `:server` | `:reactor` — host-driven on *every* backend | `:rontolisp` — this *target*'s native transport |
| Developed locally by | [`check.lisp`](../httpbin-clack/check.lisp) calling `dispatch` on every backend | serving it for real and `curl` |
| Answers | "what does a Clack Worker look like?" | "how much does deploying one cost me in edits?" — none |

Everything else is shared and documented once, in `../httpbin-clack`: where the
[`handle-request` export comes from](../httpbin-clack/README.md#where-the-export-comes-from),
[middleware and request bodies](../httpbin-clack/README.md#middleware-lackbuilder-request-bodies),
[why `clackup` prints](../httpbin-clack/README.md#why-clackup-prints-and-why-that-is-fine-here),
and the [limitations](../httpbin-clack/README.md#limitations). Module sizes are
in the [size report](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/size-report/results/cloudflare-workers.md).

## What's in here

| File | Purpose |
| --- | --- |
| [`../../net/httpbin-clack.lisp`](../../net/httpbin-clack.lisp) | **The whole program** — not in this directory, deliberately |
| [`build.sh`](build.sh) | `--no-wasi --optimize=size` over that file |
| [`src/index.js`](src/index.js) | Three lines over the generated `worker()`, plus the one hook it leaves to a caller — which header carries the client address |
| [`src/worker.js`](src/worker.js) | The boundary, GENERATED by `--emit-js-glue` from [`net/httpbin-clack.lisp`](../../net/httpbin-clack.lisp)'s declarations. Byte-identical to `../httpbin-clack/src/worker.js`, because the declarations are. Do not edit; `./build.sh` rewrites it |
| `src/worker.wasm` | A build product — run `./build.sh` first |

## Rebuilding

```bash
./mvnw clean package -DskipTests   # from the repository root
examples/cloudflare-workers/httpbin-clack-one-source/build.sh
```


---

# FILE: references/examples/cloudflare-workers/httpbin-clack-one-source/build.sh

#!/usr/bin/env bash
# Compile the Worker's module. There is no worker.lisp in this directory and
# that is the point: the program is examples/net/httpbin-clack.lisp ITSELF --
# the same file that serves on the interpreter, on the JVM and under
# `wasmtime serve` -- compiled for a host that calls an export instead of
# handing over a socket. :server :rontolisp picks this transport at compile
# time (--no-wasi reads the handler backend in reactor shape), and the
# compiler synthesizes the `handle-request` export src/index.js calls.
#
# --no-wasi: the Worker calls the exported `handle-request` directly, it never
#   runs the module as a program, and the handler does no I/O -- so the module
#   needs no WASI imports at all. It becomes a reactor: nothing to shim on the
#   JavaScript side, and `_initialize` instead of `_start`.
# --optimize=size: a Worker bundle has a size limit, and the tree-shaker is
#   what keeps the module small -- only the functions the program actually
#   reaches end up in the output. It matters more here than in ../httpbin,
#   because the program quickloads the whole of clack. =size additionally
#   declines the two speed-over-size emissions: -11% raw / -14% gzip for a
#   per-request cost of a few microseconds, the right trade on a Worker.
#
# The first run downloads clack/lack into ~/.rontolisp/quicklisp; after that the
# build is offline.
# --host-boundary=streaming: this Worker ECHOES request bodies, so they must
#   cross as octets rather than as JSON text in the envelope -- which is what
#   the generated src/worker.js feeds through env.readRequestBody /
#   env.writeResponseBody, and what lets a BINARY body come back exactly. Asked
#   for, because the default is `envelope` (see ../btc-ticker), where a body
#   rides the head instead.
# --emit-js-glue: write src/worker.js beside the module -- the host half of this
#   boundary, from the program's own declarations: the import object, the
#   (ptr, len) staging, the __ronto_alloc bracket, the two body imports above --
#   fed from the Request it is already holding and the Response it is already
#   building -- and the Request -> envelope -> Response mapping over them. That
#   half is derivable on THIS boundary too, so src/index.js is a worker(module)
#   call. It is CHECKED IN and pinned by HostGlueEmitterTest, so regenerate it
#   here rather than editing it.
#
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling ../../net/httpbin-clack.lisp -> src/worker.wasm + src/worker.js"
java -jar "$jar" "$repo_root/examples/net/httpbin-clack.lisp" \
  -o "$here/src/worker.wasm" --no-wasi --host-boundary=streaming --optimize=size \
  --emit-js-glue

ls -l "$here/src/worker.wasm" "$here/src/worker.js"
echo "done. Run it with:  npx wrangler dev"


---

# FILE: references/examples/cloudflare-workers/httpbin-clack-one-source/package.json

{
  "name": "rontolisp-cloudflare-httpbin-clack-one-source",
  "version": "0.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "build": "./build.sh",
    "dev": "wrangler dev",
    "deploy": "wrangler deploy",
    "tail": "wrangler tail"
  },
  "devDependencies": {
    "wrangler": "^4"
  }
}


---

# FILE: references/examples/cloudflare-workers/httpbin-clack-one-source/src/index.js

// The whole Worker. `./worker.js` is generated from worker.lisp's own
// declarations (build.sh's --emit-js-glue) and owns all of it: the import
// object, the (ptr, len) staging, the __ronto_alloc bracket, the two body
// imports this boundary declares -- fed from the Request it is already holding
// and the Response it is already building -- and the Request -> envelope ->
// Response mapping over them.
//
// That the STREAMING boundary needs no more of a host than ../hello-clack's
// envelope one is the point of the pair: what the boundary buys is a binary
// body crossing exactly and a large one never doubling linear memory, not a
// bigger host. ../httpbin is the one Worker here that still writes its own,
// because it declares `handle-request` by hand and the compile path recognises
// the SYNTHESIZED bridge -- read that directory's src/index.js for what this
// file would otherwise say.
//
// BYTE-IDENTICAL in every httpbin-* directory that goes through clackup. A new
// sibling copies this file, it does not edit it.

import module from "./worker.wasm";
import { worker } from "./worker.js";

export default worker(module, {
  // Clack's :remote-addr. Which header carries the client address is the
  // platform's business and not the glue's, so it is the one thing worker()
  // leaves to its caller; Cloudflare puts it here. There is no peer port to
  // report, so :remote-port stays nil.
  remoteAddr: (request) => request.headers.get("cf-connecting-ip"),
});


---

# FILE: references/examples/cloudflare-workers/httpbin-clack-one-source/src/worker.js

// GENERATED by rontolisp --emit-js-glue -- do not edit.
//
// The host half of this module's boundary, derived from the program's own
// declarations: one import-object entry per rontolisp:wasm-import, one entry
// point per rontolisp:wasm-export, and every piece of linear-memory plumbing
// between them -- the (ptr, len) pair a :string crosses as, the __ronto_alloc
// bracket around a call, and the read(2) cursor a :bytes result is pulled
// through.
//
// What a declaration cannot state is what a host function DOES, so that is the
// one thing this file asks for: `host` is a plain function per import, keyed by
// import module and field, taking and answering ordinary JavaScript values --
// never a (ptr, len) pair, and, where an entry answers `chunk` below, a
// Uint8Array or a string with null for the end of them.
//
//   import { instantiate, suspending } from "./worker.js";
//
//   const lisp = instantiate(module, {
//     env: {
//       readRequestBody: () => chunk,   // or leave it to worker()
//       writeResponseBody: (chunk) => {},   // or leave it to worker()
//     },
//   });
//   await lisp.handleRequest(text);
//
// A host that suspends marks its own entries -- suspending(async (...) => ...)
// -- and every entry point above then answers a promise: the marked imports are
// wrapped in WebAssembly.Suspending, each entry point that can reach one is
// entered through WebAssembly.promising, and calls are serialised onto one
// promise chain, because a suspended module returns to the host's event loop
// and a re-entered export refuses with a trap rather than corrupting both
// calls. Host state that belongs to ONE such call is set inside that section:
//
//   await lisp.serially(async (entry) => { ...; return entry.handleRequest(...) });
//
// This module's boundary is the reactor envelope, so the Request/Response half
// is derivable too and `worker` below is it:
//
//   import module from "./worker.wasm";
//   import { worker } from "./worker.js";
//
//   export default worker(module);
//
// Regenerate it, never edit it: --emit-js-glue on the compile that wrote the
// .wasm beside it.

const encoder = new TextEncoder();
// ignoreBOM, because these octets are a VALUE and not a document: a
// leading U+FEFF is a character the other side chose, and the default
// decoder deletes it -- silently shortening a BOM-prefixed request body
// by one character while the content-length beside it still counts three.
const decoder = new TextDecoder("utf-8", { ignoreBOM: true });

const SUSPENDING = Symbol.for("rontolisp.suspending");

/**
 * Marks a host function as one that answers a promise, so the wasm stack parks
 * until it settles (JSPI). The wrapper is not free -- an import answering
 * SYNCHRONOUSLY through one still returns to the event loop -- so mark only the
 * entries this host really implements asynchronously.
 *
 * @param {Function} fn the host function
 * @returns {object} the marked entry, to be passed as the import
 */
export function suspending(fn) {
  return { [SUSPENDING]: fn };
}

/**
 * Instantiates the module against this host and returns its callable surface.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @param {object} host one function per import, keyed by module and field
 * @returns {object} `exports`, plus one entry point per rontolisp:wasm-export
 */
export function instantiate(module, host = {}) {
  let exports;

  // Every view is built where it is used, never held: growing the module's
  // memory DETACHES the buffer behind every view taken before the growth, and a
  // suspending host resumes after growths it never saw.
  const readString = (ptr, len) =>
    decoder.decode(new Uint8Array(exports.memory.buffer, ptr, len));

  // Octets, COPIED: the module pops the staging behind the pointer the moment
  // the call returns, so a chunk not taken by then is one the host never gets.
  const readBytes = (ptr, len) =>
    new Uint8Array(exports.memory.buffer.slice(ptr, ptr + len));

  // Bytes the HOST hands over live in the module's own bump allocator, and
  // cross as the (ptr, len) pair every memory-typed value is.
  const write = (octets) => {
    const ptr = exports.__ronto_alloc(octets.length);
    new Uint8Array(exports.memory.buffer, ptr, octets.length).set(octets);
    return [ptr, octets.length];
  };
  const writeString = (value) => write(encoder.encode(String(value)));
  const octets = (chunk) =>
    typeof chunk === "string" ? encoder.encode(chunk) : chunk;

  // A host answers a value, or -- from an entry it marked suspending -- a promise
  // of one. Both shapes ride one expression, which is what lets this file drive a
  // synchronous host and a JSPI host without being written twice.
  const marked = new Set();
  const unmarked = (what) =>
    what +
    " answered a promise; wrap it in suspending() so the wasm stack parks" +
    " until it settles";
  const settle = (what, value, next) => {
    if (typeof value?.then === "function") {
      if (!marked.has(what)) {
        // Reported before it is thrown: this throw crosses back into wasm,
        // where a catch_all landing pad turns it into whatever the module
        // makes of a failed import.
        console.error(unmarked(what));
        throw new TypeError(unmarked(what));
      }
      return value.then(next);
    }
    return next(value);
  };

  // The entries the module really imports: --optimize shakes out an import the
  // program never calls, and a host should not have to know which survived.
  const linked = new Set(
    WebAssembly.Module.imports(module).map((i) => i.module + "." + i.name),
  );
  const bind = (moduleName, field, wrap) => {
    const key = moduleName + "." + field;
    const given = (host[moduleName] ?? {})[field];
    if (given == null) {
      if (!linked.has(key)) return undefined;
      throw new TypeError("host." + key + " is missing; this module imports it");
    }
    if (given[SUSPENDING] === undefined) {
      // An async function that was never marked is the one mistake worth
      // catching HERE, where the report is a plain stack and not whatever
      // the module makes of an import that threw.
      if (given.constructor?.name === "AsyncFunction") {
        throw new TypeError(unmarked("host." + key));
      }
      return wrap(key, given);
    }
    marked.add(key);
    return new WebAssembly.Suspending(wrap(key, given[SUSPENDING]));
  };

  // A :bytes RESULT is the read(2) shape: the MODULE owns the buffer and asks
  // for up to `cap` octets, so what a host answers is the next CHUNK and this
  // holds whatever did not fit. That remainder is the read side's only state,
  // and it is why a host supplies chunks rather than a reader -- which source
  // they come from (a ReadableStream, a Uint8Array) is all that is left to it.
  const readers = new Map();
  const reader = (what, source) => {
    let rest = null;
    let from = null;
    // A body the module did not drain belongs to the call that could have and
    // to no other, so every cursor is dropped at the next entry below -- and a
    // host whose SOURCE moves inside one call (a new upstream reply opened by
    // another import) drops this one itself with lisp.drop(key), because what
    // did not fit is held here and nothing else can see the source move.
    readers.set(what, () => {
      rest = null;
      from = null;
    });
    const drain = (ptr, cap) => {
      const n = Math.min(cap, rest.length);
      new Uint8Array(exports.memory.buffer, ptr, n).set(rest.subarray(0, n));
      rest = rest.subarray(n);
      return n;
    };
    // A read that FAILS answers a NEGATIVE count. Throwing would trap the
    // instance; the count is an error channel the module turns into a Lisp
    // condition where the octets are consumed, which is where every other
    // backend reports a transfer that broke mid-body.
    const failed = (error) => {
      console.error(what + " failed:", error);
      return -1;
    };
    return (args, ptr, cap) => {
      // The remainder belongs to the arguments that asked for it: a source
      // selected by argument must not be served the previous one's octets.
      const key = JSON.stringify(args);
      if (from !== key) {
        rest = null;
        from = key;
      }
      if (rest !== null && rest.length !== 0) return drain(ptr, cap);
      try {
        const answer = settle(what, source(...args), (chunk) => {
          rest = chunk == null ? new Uint8Array(0) : octets(chunk);
          return rest.length === 0 ? 0 : drain(ptr, cap);
        });
        return typeof answer?.then === "function"
          ? answer.then(undefined, failed)
          : answer;
      } catch (error) {
        return failed(error);
      }
    };
  };

  // What a read import left over, thrown away on demand. A host calls it when
  // the SOURCE behind that import moves under it INSIDE one call -- a new
  // upstream reply, say -- since the remainder is held above and nothing else
  // can see the source move. With no argument it drops every one of them.
  const drop = (key) =>
    key === undefined ? readers.forEach((f) => f()) : readers.get(key)?.();

  const imports = {
    env: {
      // () -> :bytes
      readRequestBody: bind("env", "readRequestBody", (what, call) => {
        const read = reader(what, call);
        return (ptr, cap) => read([], ptr, cap);
      }),
      // (:bytes) -> :void
      writeResponseBody: bind("env", "writeResponseBody", (what, call) => {
        return (p0, p0Len) =>
          settle(what, call(readBytes(p0, p0Len)), () => undefined);
      }),
    },
  };
  const instance = new WebAssembly.Instance(module, imports);
  exports = instance.exports;
  // Entropy, BEFORE the top level runs: a --no-wasi module imports none, so its
  // `random` starts from a constant and every instance would draw one sequence.
  // Seeding here also covers the draws a library makes while it LOADS. A Worker
  // forbids this in global scope, so instantiate on the first request.
  exports.__ronto_seed_random(
    new BigUint64Array(crypto.getRandomValues(new Uint8Array(8)).buffer)[0],
  );
  // And a clock, nanoseconds since the Unix epoch, for the same reason and
  // before the same line: until one is set the clock built-ins signal rather
  // than report 1970.
  exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
  exports._initialize();

  // The entry points a call chain can reach a suspending import from -- the
  // list the build prints -- entered through WebAssembly.promising exactly when
  // the host marked one. An unmarked host never pays for the promise.
  const suspends = marked.size !== 0;
  const entry$handleRequest = suspends
    ? WebAssembly.promising(exports["handle-request"])
    : exports["handle-request"];

  // One Lisp call at a time. A suspended call returns to the host's event loop,
  // and the module's own re-entry guard TRAPS a second entry rather than let two
  // calls share its allocator and its dynamic bindings -- so the queue is the
  // contract, not a nicety. `.then(work, work)` because one rejected call must
  // not wedge the chain behind it.
  let queue = Promise.resolve();
  const queued = (work) => {
    const done = queue.then(work, work);
    queue = done.then(
      () => {},
      () => {},
    );
    return done;
  };
  // A bare entry point only needs the queue when a host marked something: a
  // synchronous call cannot be interleaved, and paying a promise for it would
  // make every host asynchronous. `serially` below always takes it, because
  // the work it runs awaits and a second request WOULD land inside it.
  const serialised = (work) => (suspends ? queued(work) : work());

  // One call into the module: stage the arguments, enter, decode the result out
  // of the scratch it sits in, and only THEN pop the arena that scratch is in.
  // A promising entry answers a promise, so the tail rides `then`; a synchronous
  // one runs the same expression inline. `run` is how the call reaches the
  // module -- through the queue, or straight through when it is already inside
  // the one call the queue admits.
  const call = (run, entry, stage, decode) => {
    const work = () => {
      readers.forEach((drop) => drop());
      // The module's clock moves only when the host moves it, so move it per
      // call. Not a workaround for a frozen clock -- a Worker's own Date.now()
      // is frozen for the duration of a request as a timing-attack mitigation,
      // so a value that changes once per call is what the platform has.
      exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
      const mark = exports.__ronto_alloc_mark();
      const done = (raw) => {
        const value = decode(raw);
        exports.__ronto_alloc_reset(mark);
        return value;
      };
      // A TRAP skips the pop above and leaves the module's state half-written
      // with it, so a host that keeps serving instantiates again rather than
      // calling back into this instance.
      const raw = entry(...stage());
      return typeof raw?.then === "function" ? raw.then(done) : done(raw);
    };
    return run(work);
  };

  /** `handle-request` -- (:string) -> :string */
  const make$handleRequest =
    (run) =>
    (p0) => {
      return call(
        run,
        entry$handleRequest,
        () => {
          const a0 = writeString(p0);
          return [a0[0], a0[1]];
        },
        ([ptr, len]) => readString(ptr, len),
      );
    };


  // Host state that belongs to ONE call -- what the module pulls DURING it,
  // and what the call leaves behind -- is set and read inside `work`, which
  // runs in the same critical section: a suspended call returns to the event
  // loop, so setting it beside the call instead would let the next request
  // move it under this one. The entry points `work` is handed enter the module
  // directly, because the queue they would take is the one they are in.
  const inside = {
    handleRequest: make$handleRequest((work) => work()),
  };
  const serially = (work) => queued(() => work(inside));

  return {
    exports,
    handleRequest: make$handleRequest(serialised),
    drop,
    serially,
  };
}

/**
 * This module as a fetch handler -- the whole Worker, with nothing left over:
 *
 *   import module from "./worker.wasm";
 *   import { worker } from "./worker.js";
 *
 *   export default worker(module);
 *
 * A Request becomes the envelope the module's entry point takes, the head it
 * answers becomes a Response, and the instance is created on the FIRST REQUEST
 * (a Worker forbids drawing entropy in global scope) and retired if a call ever
 * traps.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @param {object} [options] `host` -- one plain function per import, keyed by
 *   module and field, as `instantiate` takes it; `remoteAddr` -- (request, env, ctx) => the client
 *   address, since which header carries it is the platform's business and not
 *   this file's (`(r) => r.headers.get("cf-connecting-ip")` on Cloudflare)
 * @returns {object} `{ fetch(request, env, ctx) }`
 */
export function worker(module, options = {}) {
  let instance = null;
  // Set when a call TRAPPED: that instance skipped its arena reset and its Lisp
  // state may be half-written, so nothing else may run on it. A Lisp ERROR is not
  // this -- the transport answers 500 itself and the instance is fine.
  let poisoned = false;
  // The request body the module pulls, and the response body coming back the
  // same way. Both belong to the ONE call running below, which is where they
  // are set.
  let requestBody = null;
  let responseChunks = [];
  const collected = () => {
    const all = new Uint8Array(
      responseChunks.reduce((n, chunk) => n + chunk.length, 0),
    );
    let at = 0;
    for (const chunk of responseChunks) {
      all.set(chunk, at);
      at += chunk.length;
    }
    return all;
  };
  const base = {};
  base.env = {
    ...(base.env ?? {}),
    readRequestBody: () => {
      // Handed over ONCE: a chunk source that never answers null is one the
      // module pulls forever.
      const chunk = requestBody;
      requestBody = null;
      return chunk;
    },
    writeResponseBody: (chunk) => responseChunks.push(chunk),
  };
  const given = options.host ?? {};
  const host = {};
  for (const key of new Set([...Object.keys(base), ...Object.keys(given)])) {
    host[key] = { ...(base[key] ?? {}), ...(given[key] ?? {}) };
  }
  const live = () => {
    if (poisoned) {
      instance = null;
      poisoned = false;
    }
    return (instance ??= instantiate(module, host));
  };

  // The request head. `target` stays RAW -- path and query still joined and
  // still percent-encoded -- because the shared normalizer on the other side
  // owns that split, and a pre-split path leaves the query string nil.
  const envelope = (request, octets, remoteAddr) => {
    const url = new URL(request.url);
    const headers = Object.fromEntries(request.headers);
    // A body with no content-length is a body the request parser does not read,
    // and a chunked request carries none -- so set it from the octets we have.
    if (octets?.length) headers["content-length"] = String(octets.length);
    const head = {
      method: request.method,
      target: url.pathname + url.search,
      headers,
      scheme: url.protocol.replace(":", ""),
    };
    if (remoteAddr != null) head["remote-addr"] = remoteAddr;
    return JSON.stringify(head);
  };

  return {
    // EVERYTHING is inside the try, not just the module call: reading an
    // aborted upload rejects, and `new Response` throws on a status or a
    // header an application is free to produce (0, 999, a newline in a
    // value). Outside it those escape as an unhandled rejection, which the
    // platform answers with its own error page and nothing in the log.
    async fetch(request, env, ctx) {
      let entered = false;
      try {
        const remoteAddr = await options.remoteAddr?.(request, env, ctx);
        const octets = request.body
          ? new Uint8Array(await request.arrayBuffer())
          : null;
        const input = envelope(request, octets, remoteAddr);
        const head = JSON.parse(
          await live().serially((lisp) => {
            // Re-read INSIDE the critical section: the instance was bound at
            // admission, and a call parked ahead of this one can poison it
            // before this one runs. Refusing is the whole point -- a
            // half-unwound instance answers wrong rather than failing, and
            // the module's own re-entry guard is cleared by the landing pad
            // on exactly the path that poisons it.
            if (poisoned) throw new Error("instance discarded by an earlier trap");
            // Per-call state, set HERE and not beside the call: a suspended
            // handler returns to the event loop, so the next request would
            // otherwise move it under this one.
            requestBody = octets;
            responseChunks = [];
            entered = true;
            return lisp.handleRequest(input);
          }),
        );
        // Headers as an ARRAY of pairs, which keeps two Set-Cookie two.
        // An EMPTY body becomes null whichever shape it arrived in: 204/205/304
        // may only be constructed with a null body, and "" and a zero-length
        // Uint8Array are the same response as none.
        const body = head.body ?? collected();
        return new Response(body?.length ? body : null, {
          status: head.status ?? 200,
          headers: head.headers ?? [],
        });
      } catch (error) {
        console.error("handle-request failed:", error);
        // Only a call that ENTERED the module can have left it half-written.
        // A mapping, or a Response the platform refused, says nothing about
        // the instance, and discarding it would cost the next request a
        // reinstantiation for someone else's bad header.
        if (entered) poisoned = true;
        return new Response("internal error\n", { status: 500 });
      }
    },
  };
}


---

# FILE: references/examples/cloudflare-workers/httpbin-clack-one-source/wrangler.jsonc

{
  // https://developers.cloudflare.com/workers/wrangler/configuration/
  "name": "rontolisp-httpbin-clack-one-source",
  "main": "src/index.js",
  "compatibility_date": "2026-01-01",

  // `import module from "./worker.wasm"` needs no configuration: .wasm is a
  // built-in module rule for ES-module Workers, and the file is compiled at
  // deploy time rather than on the request path.
  "observability": {
    "enabled": true
  }
}


---

# FILE: references/examples/cloudflare-workers/httpbin-clack/README.md

# httpbin-clack — the same endpoints as a plain Clack application

The five echo endpoints of [`../httpbin`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin), written with nothing but
[Clack](https://github.com/fukamachi/clack): an application **function**, a
`cond` over `:path-info` (clack has no router), and one **middleware** — which
in Clack is just a function from application to application:

```lisp
(defun wrap-json (app)
  (lambda (env)
    (let ((response (funcall app env)))
      (list* (first response)
             (list* :content-type "application/json" (second response))
             (cddr response)))))

(clack:clackup (wrap-json #'app) :server :reactor :use-thread nil)
```

Nothing in `worker.lisp` mentions Cloudflare, or a Worker, or an export.

```bash
./build.sh          # worker.lisp -> src/worker.wasm
npx wrangler dev    # http://localhost:8787
npx wrangler deploy
```

```console
$ curl 'http://localhost:8787/get?a=1&b=two'
{"args":{"a":"1","b":"two"},"headers":{"host":"localhost:8787",...},"method":"GET","path":"/get"}

$ curl -X POST -d '{"name":"rontolisp"}' http://localhost:8787/post
{"args":{},...,"data":"{\"name\":\"rontolisp\"}","json":{"name":"rontolisp"}}
```

A wrong method answers 405 with the one it wanted, an unknown path 404, and a
body that does not parse leaves `"json": null`.

## What's in here

| File | Purpose |
| --- | --- |
| [`worker.lisp`](worker.lisp) | **The whole program**: quickload, the endpoints, the middleware, `clackup` |
| [`check.lisp`](check.lisp) | The same handler driven without Cloudflare, on every backend |
| [`build.sh`](build.sh) | `--no-wasi --optimize=size` over `worker.lisp` |
| [`src/index.js`](src/index.js) | Three lines over the generated `worker()`, plus the one hook it leaves to a caller — which header carries the client address |
| [`src/worker.js`](src/worker.js) | The boundary, GENERATED by `--emit-js-glue` from `worker.lisp`'s declarations. Do not edit; `./build.sh` rewrites it |
| `src/worker.wasm` | A build product — run `./build.sh` first |

## What this answers that `../httpbin` does not

| | `../httpbin` | this |
| --- | --- | --- |
| How it reaches the Worker | thirty hand-written lines and an explicit `wasm-export` | `clackup`, with the export synthesized |
| Reads as | a Worker program that happens to speak Clack | **every other Clack program** |
| clack in the module | none | what the tree-shaker keeps of clack and lack |

The trade is honest: you pay for clack and lack to be in the module
([size report](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/size-report/results/cloudflare-workers.md)) so that the
file reads like an ordinary Clack program. Per request you pay nothing — when
this directory switched from a hand-written export to `clackup`, the module grew
9% and `_initialize` roughly doubled, while warm `GET` and `POST` did not move
at all. Everything clack costs on a reactor is module size and startup, and
startup is paid once per isolate.

## The backend is a handler backend, not example code

Nothing here writes an adapter. `clack-handler-reactor` is a built-in Clack
handler backend, and `:server :reactor` means **host-driven on every backend**:
its `run` stores the application where a socket backend would bind a listener.
That is what lets [`check.lisp`](check.lisp) drive this Worker — through the same
`dispatch` the export calls — on the interpreter and the JVM as well.

[`../httpbin-clack-one-source`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin-clack-one-source) is the other
designator: `:server :rontolisp` serves on whatever the compile *target*'s
native transport is, which is what lets one file be a socket server locally and
a Worker here without an edit. Both store into the same reactor machinery, so
the two cannot drift.

### Where the export comes from

`clackup` applies the backend's `run`, and a reactor owns no socket, so `run`
stores the application and returns. What replaces the socket is a WASM export —
and `rontolisp:wasm-export` needs a **literal** name at compile time, which a
program whose whole Worker half is a `clackup` call cannot give. So `run`
carries a marker and the compiler answers it, appending the equivalent of

```lisp
(defun %reactor-dispatch (json) (rontolisp::%http-reactor-dispatch json))
(rontolisp:wasm-export '%reactor-dispatch :as "handle-request"
                       :params '(:string) :returns :string)
```

after the program. `%http-reactor-dispatch` is the **shared** reactor machinery
([`http-reactor.lisp`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/src/main/resources/am/ik/rontolisp/eval/http-reactor.lisp)),
and `clack.handler.reactor:dispatch` is a thin public name over the same
functions — which is why `check.lisp` exercises what the Worker exercises. It
converts nothing either: every backend meets in
[`http-server.lisp`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/src/main/resources/am/ik/rontolisp/eval/http-server.lisp)'s
two entry points, and the JSON envelope — with the
[two fields the JavaScript side must get right](../httpbin/README.md#the-envelope-and-two-fields-the-javascript-side-must-get-right)
— is `../httpbin`'s.

`:use-thread nil` is the one upstream keyword that matters beyond sockets: the
WASM backends are single-threaded so `clackup` already defaults to `nil` there,
but the interpreter and the JVM have threads and a script wants the foreground.

### And the JavaScript half is not hand-written either

`--emit-js-glue` (`build.sh`) writes [`src/worker.js`](src/worker.js) from
`worker.lisp`'s own declarations: the import object, the `(ptr, len)` staging,
the `__ronto_alloc` bracket, **both body imports** — fed from the `Request` it is
already holding and the `Response` it is already building — and the
`Request -> envelope -> Response` mapping over them.
[`src/index.js`](src/index.js) is:

```js
import module from "./worker.wasm";
import { worker } from "./worker.js";

export default worker(module, {
  remoteAddr: (request) => request.headers.get("cf-connecting-ip"),
});
```

The `remoteAddr` hook is the one thing left to a caller, and deliberately: which
header carries the client address is the platform's business rather than the
boundary's. It becomes Clack's `:remote-addr`.

That the STREAMING boundary needs no more of a host than
[`../hello-clack`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/hello-clack)'s envelope one is the point of the pair — what
`--host-boundary=streaming` buys is a binary body crossing exactly and a large
one never doubling linear memory, not a bigger host.
[`../httpbin`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin) is the one Worker here that still writes its own: it
declares `handle-request` by hand, and the compile path recognises the
*synthesized* bridge, so no `worker()` is emitted for it.

The generated file is **checked in** and pinned by `HostGlueEmitterTest`, so
regenerate it with `./build.sh` rather than editing it — and all four
`httpbin-*` directories that go through `clackup` carry the same one.

## Developing it without Cloudflare

`dispatch` is an ordinary function, so the whole Worker runs locally on every
backend the compiler has:

```bash
rontolisp check.lisp                                  # interpreter
rontolisp check.lisp -o Check.class && java -cp rontolisp-0.1.0-SNAPSHOT-exec.jar:. Check
rontolisp check.lisp -o check.wasm && wasmtime run -W gc -W exceptions=y check.wasm
```

[`examples.yaml`](../../examples.yaml) pins all three. The report on standard
error for the unparseable-body probe is lack's `backtrace` middleware, which
prints even for an error the application catches; on the Worker it goes to a
discarding sink and the `200` with `"json": null` still comes back.

## Middleware, `lack:builder`, request bodies

`wrap-json` above is Clack's whole composition story at one middleware. Several
of them compose with `lack:builder`, and `clackup`'s own default `backtrace`
middleware is already in this module and active, so "middleware works" is not
hypothetical. Neither is the body/params stack: `(ql:quickload "lack-request")`
loads on a `--no-wasi` build, and a `lack:builder` around this application reads
a query string and a urlencoded POST body the way it does anywhere else.

That took a fix rather than a discovery, and it is the shape every heavy library
has. The chain `lack-request -> http-body -> fast-http -> smart-buffer` ends at
one upstream form — smart-buffer names its temporary directory with a top-level
`(random ...)` over `uiop:default-temporary-directory` — and on a `--no-wasi`
module both halves used to be `unreachable` stubs, firing at *load* time inside
`_initialize` before any export existed. Both are answered now: the module
carries its own `random` generator (seeded from `crypto` by the generated
`src/worker.js`), and
`getenv` reports the empty environment a reactor really has.
`lack-middleware-session` followed the same rule: it reads the clock while it
loads, so `src/worker.js` hands the time over through `__ronto_set_time`. Add
`--host-random` — the session id is `rontolisp:random-bytes`, which a fixed-seed
generator must not stand in for — and a `(:session)` builder serves a real
session cookie and recognises it on the next request.

**One caveat if you write your own `lack:builder` stack here**: a `--no-wasi`
build then prints a standing `WITH-OPEN-FILE is reachable` warning. It is a
false alarm — `builder` returns its composed application through `reduce`, so
the compiler can no longer see that what reaches `clackup` is a function rather
than a pathname, and `clackup`'s "the app is a file to load" branch stops being
provably dead. Applying a middleware directly, as `worker.lisp` does, keeps the
branch provably dead and the build quiet.

## Why `clackup` prints, and why that is fine here

`clackup` writes two lines unconditionally — its startup banner and
`clack.handler:run`'s debug NOTICE — and they are upstream clack's, not ours. On
a `--no-wasi` module there is no stdout to write them to. They used to **trap**
the instance at `_initialize`, which is why this example could not call
`clackup` at all; now standard output and error are a **sink** under
`--no-wasi`, so the bytes are simply discarded. That is a deliberate policy
rather than a patch for clack: a reactor host hands the module no file
descriptors, so discarding loses only the bytes, while the alternative was
killing the instance for a log line. Locally the two lines are visible on real
stdout; pass `:silent t :debug nil` to suppress them.

## Limitations

The Worker sandbox and `--no-wasi` limitations of
[`../httpbin`](../httpbin/README.md#limitations) apply unchanged. One more is
specific to the clack directories: a *runtime* `(ql:quickload ...)` cannot work
either — the form in the source is resolved at **compile** time and inlined into
the module, which is why the first `./build.sh` needs network and later ones do
not.

## Rebuilding

```bash
./mvnw clean package -DskipTests   # from the repository root
examples/cloudflare-workers/httpbin-clack/build.sh
```


---

# FILE: references/examples/cloudflare-workers/httpbin-clack/build.sh

#!/usr/bin/env bash
# Compile worker.lisp -- clack, the five echo endpoints, the middleware and the
# clackup call -- to the .wasm module the Worker imports.
#
# --no-wasi: the Worker calls the exported `handle-request` directly, it never
#   runs the module as a program, and the handler does no I/O -- so the module
#   needs no WASI imports at all. It becomes a reactor: nothing to shim on the
#   JavaScript side, and `_initialize` instead of `_start`.
# --optimize=size: a Worker bundle has a size limit, and the tree-shaker is what
#   keeps the module small. It matters more here than in ../httpbin, because the
#   program quickloads the whole of clack.
#
# The first run downloads clack/lack into ~/.rontolisp/quicklisp; after that the
# build is offline.
# --host-boundary=streaming: this Worker ECHOES request bodies, so they must
#   cross as octets rather than as JSON text in the envelope -- which is what
#   the generated src/worker.js feeds through env.readRequestBody /
#   env.writeResponseBody, and what lets a BINARY body come back exactly. Asked
#   for, because the default is `envelope` (see ../btc-ticker), where a body
#   rides the head instead.
# --emit-js-glue: write src/worker.js beside the module -- the host half of this
#   boundary, from the program's own declarations: the import object, the
#   (ptr, len) staging, the __ronto_alloc bracket, the two body imports above --
#   fed from the Request it is already holding and the Response it is already
#   building -- and the Request -> envelope -> Response mapping over them. That
#   half is derivable on THIS boundary too, so src/index.js is a worker(module)
#   call. It is CHECKED IN and pinned by HostGlueEmitterTest, so regenerate it
#   here rather than editing it.
#
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling worker.lisp -> src/worker.wasm + src/worker.js"
java -jar "$jar" "$here/worker.lisp" -o "$here/src/worker.wasm" \
  --no-wasi --host-boundary=streaming --optimize=size --emit-js-glue

ls -l "$here/src/worker.wasm" "$here/src/worker.js"
echo "done. Run it with:  npx wrangler dev"


---

# FILE: references/examples/cloudflare-workers/httpbin-clack/check.lisp

;;; Drive worker.lisp's handler without Cloudflare, on any backend: the
;;; synthesized export calls `dispatch`, an ordinary function of the handler
;;; backend, over the application clackup stored.
;;;
;;; Two kinds of noise are upstream clack's, and both are gone on a Worker,
;;; where stdout and *error-output* are a sink: the clackup banner before the
;;; first -->, and the request dump from lack's default backtrace middleware,
;;; which prints even for an error the application CATCHES -- the unparseable
;;; body is still answered 200 with "json":null.

(load "worker.lisp")

(defun try (request-plist)
  (let ((request
         (rontolisp:json-stringify (rontolisp:plist-hash-table request-plist))))
    (format t "~&--> ~a~%" request)
    (format t "<-- ~a~%" (clack.handler.reactor:dispatch request))))

(defun headers (&rest plist) (rontolisp:plist-hash-table plist))

(defun json-headers (body)
  (headers :host "example.com"
           :content-type "application/json"
           :content-length (princ-to-string (length body))))

;; GET /get with a query string -- the "?" split and the percent-decoding are
;; %http-make-env's, over the RAW target the envelope carries.
(try
 (list :method "GET"
       :target "/%67et?a=1&b=two"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (headers :host "example.com" :accept "application/json")
       :body ""))

;; POST /post with a JSON body -- "data" is the raw text, "json" the parsed
;; value, read off clack's buffered :raw-body stream.
(try
 (list :method "POST"
       :target "/post"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (json-headers "{\"name\":\"rontolisp\"}")
       :body "{\"name\":\"rontolisp\"}"))

;; A body that does not parse -- "json" is null and the answer is still 200,
;; which is what the real httpbin does.
(try
 (list :method "POST"
       :target "/post"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (json-headers "{not json")
       :body "{not json"))

;; The wrong method for an endpoint -- 405, naming the one it wanted.
(try
 (list :method "GET"
       :target "/post"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (headers :host "example.com")
       :body ""))

;; An unknown path -- 404.
(try
 (list :method "GET"
       :target "/nope"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (headers :host "example.com")
       :body ""))


---

# FILE: references/examples/cloudflare-workers/httpbin-clack/package.json

{
  "name": "rontolisp-cloudflare-httpbin-clack",
  "version": "0.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "build": "./build.sh",
    "dev": "wrangler dev",
    "deploy": "wrangler deploy",
    "tail": "wrangler tail"
  },
  "devDependencies": {
    "wrangler": "^4"
  }
}


---

# FILE: references/examples/cloudflare-workers/httpbin-clack/src/index.js

// The whole Worker. `./worker.js` is generated from worker.lisp's own
// declarations (build.sh's --emit-js-glue) and owns all of it: the import
// object, the (ptr, len) staging, the __ronto_alloc bracket, the two body
// imports this boundary declares -- fed from the Request it is already holding
// and the Response it is already building -- and the Request -> envelope ->
// Response mapping over them.
//
// That the STREAMING boundary needs no more of a host than ../hello-clack's
// envelope one is the point of the pair: what the boundary buys is a binary
// body crossing exactly and a large one never doubling linear memory, not a
// bigger host. ../httpbin is the one Worker here that still writes its own,
// because it declares `handle-request` by hand and the compile path recognises
// the SYNTHESIZED bridge -- read that directory's src/index.js for what this
// file would otherwise say.
//
// BYTE-IDENTICAL in every httpbin-* directory that goes through clackup. A new
// sibling copies this file, it does not edit it.

import module from "./worker.wasm";
import { worker } from "./worker.js";

export default worker(module, {
  // Clack's :remote-addr. Which header carries the client address is the
  // platform's business and not the glue's, so it is the one thing worker()
  // leaves to its caller; Cloudflare puts it here. There is no peer port to
  // report, so :remote-port stays nil.
  remoteAddr: (request) => request.headers.get("cf-connecting-ip"),
});


---

# FILE: references/examples/cloudflare-workers/httpbin-clack/src/worker.js

// GENERATED by rontolisp --emit-js-glue -- do not edit.
//
// The host half of this module's boundary, derived from the program's own
// declarations: one import-object entry per rontolisp:wasm-import, one entry
// point per rontolisp:wasm-export, and every piece of linear-memory plumbing
// between them -- the (ptr, len) pair a :string crosses as, the __ronto_alloc
// bracket around a call, and the read(2) cursor a :bytes result is pulled
// through.
//
// What a declaration cannot state is what a host function DOES, so that is the
// one thing this file asks for: `host` is a plain function per import, keyed by
// import module and field, taking and answering ordinary JavaScript values --
// never a (ptr, len) pair, and, where an entry answers `chunk` below, a
// Uint8Array or a string with null for the end of them.
//
//   import { instantiate, suspending } from "./worker.js";
//
//   const lisp = instantiate(module, {
//     env: {
//       readRequestBody: () => chunk,   // or leave it to worker()
//       writeResponseBody: (chunk) => {},   // or leave it to worker()
//     },
//   });
//   await lisp.handleRequest(text);
//
// A host that suspends marks its own entries -- suspending(async (...) => ...)
// -- and every entry point above then answers a promise: the marked imports are
// wrapped in WebAssembly.Suspending, each entry point that can reach one is
// entered through WebAssembly.promising, and calls are serialised onto one
// promise chain, because a suspended module returns to the host's event loop
// and a re-entered export refuses with a trap rather than corrupting both
// calls. Host state that belongs to ONE such call is set inside that section:
//
//   await lisp.serially(async (entry) => { ...; return entry.handleRequest(...) });
//
// This module's boundary is the reactor envelope, so the Request/Response half
// is derivable too and `worker` below is it:
//
//   import module from "./worker.wasm";
//   import { worker } from "./worker.js";
//
//   export default worker(module);
//
// Regenerate it, never edit it: --emit-js-glue on the compile that wrote the
// .wasm beside it.

const encoder = new TextEncoder();
// ignoreBOM, because these octets are a VALUE and not a document: a
// leading U+FEFF is a character the other side chose, and the default
// decoder deletes it -- silently shortening a BOM-prefixed request body
// by one character while the content-length beside it still counts three.
const decoder = new TextDecoder("utf-8", { ignoreBOM: true });

const SUSPENDING = Symbol.for("rontolisp.suspending");

/**
 * Marks a host function as one that answers a promise, so the wasm stack parks
 * until it settles (JSPI). The wrapper is not free -- an import answering
 * SYNCHRONOUSLY through one still returns to the event loop -- so mark only the
 * entries this host really implements asynchronously.
 *
 * @param {Function} fn the host function
 * @returns {object} the marked entry, to be passed as the import
 */
export function suspending(fn) {
  return { [SUSPENDING]: fn };
}

/**
 * Instantiates the module against this host and returns its callable surface.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @param {object} host one function per import, keyed by module and field
 * @returns {object} `exports`, plus one entry point per rontolisp:wasm-export
 */
export function instantiate(module, host = {}) {
  let exports;

  // Every view is built where it is used, never held: growing the module's
  // memory DETACHES the buffer behind every view taken before the growth, and a
  // suspending host resumes after growths it never saw.
  const readString = (ptr, len) =>
    decoder.decode(new Uint8Array(exports.memory.buffer, ptr, len));

  // Octets, COPIED: the module pops the staging behind the pointer the moment
  // the call returns, so a chunk not taken by then is one the host never gets.
  const readBytes = (ptr, len) =>
    new Uint8Array(exports.memory.buffer.slice(ptr, ptr + len));

  // Bytes the HOST hands over live in the module's own bump allocator, and
  // cross as the (ptr, len) pair every memory-typed value is.
  const write = (octets) => {
    const ptr = exports.__ronto_alloc(octets.length);
    new Uint8Array(exports.memory.buffer, ptr, octets.length).set(octets);
    return [ptr, octets.length];
  };
  const writeString = (value) => write(encoder.encode(String(value)));
  const octets = (chunk) =>
    typeof chunk === "string" ? encoder.encode(chunk) : chunk;

  // A host answers a value, or -- from an entry it marked suspending -- a promise
  // of one. Both shapes ride one expression, which is what lets this file drive a
  // synchronous host and a JSPI host without being written twice.
  const marked = new Set();
  const unmarked = (what) =>
    what +
    " answered a promise; wrap it in suspending() so the wasm stack parks" +
    " until it settles";
  const settle = (what, value, next) => {
    if (typeof value?.then === "function") {
      if (!marked.has(what)) {
        // Reported before it is thrown: this throw crosses back into wasm,
        // where a catch_all landing pad turns it into whatever the module
        // makes of a failed import.
        console.error(unmarked(what));
        throw new TypeError(unmarked(what));
      }
      return value.then(next);
    }
    return next(value);
  };

  // The entries the module really imports: --optimize shakes out an import the
  // program never calls, and a host should not have to know which survived.
  const linked = new Set(
    WebAssembly.Module.imports(module).map((i) => i.module + "." + i.name),
  );
  const bind = (moduleName, field, wrap) => {
    const key = moduleName + "." + field;
    const given = (host[moduleName] ?? {})[field];
    if (given == null) {
      if (!linked.has(key)) return undefined;
      throw new TypeError("host." + key + " is missing; this module imports it");
    }
    if (given[SUSPENDING] === undefined) {
      // An async function that was never marked is the one mistake worth
      // catching HERE, where the report is a plain stack and not whatever
      // the module makes of an import that threw.
      if (given.constructor?.name === "AsyncFunction") {
        throw new TypeError(unmarked("host." + key));
      }
      return wrap(key, given);
    }
    marked.add(key);
    return new WebAssembly.Suspending(wrap(key, given[SUSPENDING]));
  };

  // A :bytes RESULT is the read(2) shape: the MODULE owns the buffer and asks
  // for up to `cap` octets, so what a host answers is the next CHUNK and this
  // holds whatever did not fit. That remainder is the read side's only state,
  // and it is why a host supplies chunks rather than a reader -- which source
  // they come from (a ReadableStream, a Uint8Array) is all that is left to it.
  const readers = new Map();
  const reader = (what, source) => {
    let rest = null;
    let from = null;
    // A body the module did not drain belongs to the call that could have and
    // to no other, so every cursor is dropped at the next entry below -- and a
    // host whose SOURCE moves inside one call (a new upstream reply opened by
    // another import) drops this one itself with lisp.drop(key), because what
    // did not fit is held here and nothing else can see the source move.
    readers.set(what, () => {
      rest = null;
      from = null;
    });
    const drain = (ptr, cap) => {
      const n = Math.min(cap, rest.length);
      new Uint8Array(exports.memory.buffer, ptr, n).set(rest.subarray(0, n));
      rest = rest.subarray(n);
      return n;
    };
    // A read that FAILS answers a NEGATIVE count. Throwing would trap the
    // instance; the count is an error channel the module turns into a Lisp
    // condition where the octets are consumed, which is where every other
    // backend reports a transfer that broke mid-body.
    const failed = (error) => {
      console.error(what + " failed:", error);
      return -1;
    };
    return (args, ptr, cap) => {
      // The remainder belongs to the arguments that asked for it: a source
      // selected by argument must not be served the previous one's octets.
      const key = JSON.stringify(args);
      if (from !== key) {
        rest = null;
        from = key;
      }
      if (rest !== null && rest.length !== 0) return drain(ptr, cap);
      try {
        const answer = settle(what, source(...args), (chunk) => {
          rest = chunk == null ? new Uint8Array(0) : octets(chunk);
          return rest.length === 0 ? 0 : drain(ptr, cap);
        });
        return typeof answer?.then === "function"
          ? answer.then(undefined, failed)
          : answer;
      } catch (error) {
        return failed(error);
      }
    };
  };

  // What a read import left over, thrown away on demand. A host calls it when
  // the SOURCE behind that import moves under it INSIDE one call -- a new
  // upstream reply, say -- since the remainder is held above and nothing else
  // can see the source move. With no argument it drops every one of them.
  const drop = (key) =>
    key === undefined ? readers.forEach((f) => f()) : readers.get(key)?.();

  const imports = {
    env: {
      // () -> :bytes
      readRequestBody: bind("env", "readRequestBody", (what, call) => {
        const read = reader(what, call);
        return (ptr, cap) => read([], ptr, cap);
      }),
      // (:bytes) -> :void
      writeResponseBody: bind("env", "writeResponseBody", (what, call) => {
        return (p0, p0Len) =>
          settle(what, call(readBytes(p0, p0Len)), () => undefined);
      }),
    },
  };
  const instance = new WebAssembly.Instance(module, imports);
  exports = instance.exports;
  // Entropy, BEFORE the top level runs: a --no-wasi module imports none, so its
  // `random` starts from a constant and every instance would draw one sequence.
  // Seeding here also covers the draws a library makes while it LOADS. A Worker
  // forbids this in global scope, so instantiate on the first request.
  exports.__ronto_seed_random(
    new BigUint64Array(crypto.getRandomValues(new Uint8Array(8)).buffer)[0],
  );
  // And a clock, nanoseconds since the Unix epoch, for the same reason and
  // before the same line: until one is set the clock built-ins signal rather
  // than report 1970.
  exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
  exports._initialize();

  // The entry points a call chain can reach a suspending import from -- the
  // list the build prints -- entered through WebAssembly.promising exactly when
  // the host marked one. An unmarked host never pays for the promise.
  const suspends = marked.size !== 0;
  const entry$handleRequest = suspends
    ? WebAssembly.promising(exports["handle-request"])
    : exports["handle-request"];

  // One Lisp call at a time. A suspended call returns to the host's event loop,
  // and the module's own re-entry guard TRAPS a second entry rather than let two
  // calls share its allocator and its dynamic bindings -- so the queue is the
  // contract, not a nicety. `.then(work, work)` because one rejected call must
  // not wedge the chain behind it.
  let queue = Promise.resolve();
  const queued = (work) => {
    const done = queue.then(work, work);
    queue = done.then(
      () => {},
      () => {},
    );
    return done;
  };
  // A bare entry point only needs the queue when a host marked something: a
  // synchronous call cannot be interleaved, and paying a promise for it would
  // make every host asynchronous. `serially` below always takes it, because
  // the work it runs awaits and a second request WOULD land inside it.
  const serialised = (work) => (suspends ? queued(work) : work());

  // One call into the module: stage the arguments, enter, decode the result out
  // of the scratch it sits in, and only THEN pop the arena that scratch is in.
  // A promising entry answers a promise, so the tail rides `then`; a synchronous
  // one runs the same expression inline. `run` is how the call reaches the
  // module -- through the queue, or straight through when it is already inside
  // the one call the queue admits.
  const call = (run, entry, stage, decode) => {
    const work = () => {
      readers.forEach((drop) => drop());
      // The module's clock moves only when the host moves it, so move it per
      // call. Not a workaround for a frozen clock -- a Worker's own Date.now()
      // is frozen for the duration of a request as a timing-attack mitigation,
      // so a value that changes once per call is what the platform has.
      exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
      const mark = exports.__ronto_alloc_mark();
      const done = (raw) => {
        const value = decode(raw);
        exports.__ronto_alloc_reset(mark);
        return value;
      };
      // A TRAP skips the pop above and leaves the module's state half-written
      // with it, so a host that keeps serving instantiates again rather than
      // calling back into this instance.
      const raw = entry(...stage());
      return typeof raw?.then === "function" ? raw.then(done) : done(raw);
    };
    return run(work);
  };

  /** `handle-request` -- (:string) -> :string */
  const make$handleRequest =
    (run) =>
    (p0) => {
      return call(
        run,
        entry$handleRequest,
        () => {
          const a0 = writeString(p0);
          return [a0[0], a0[1]];
        },
        ([ptr, len]) => readString(ptr, len),
      );
    };


  // Host state that belongs to ONE call -- what the module pulls DURING it,
  // and what the call leaves behind -- is set and read inside `work`, which
  // runs in the same critical section: a suspended call returns to the event
  // loop, so setting it beside the call instead would let the next request
  // move it under this one. The entry points `work` is handed enter the module
  // directly, because the queue they would take is the one they are in.
  const inside = {
    handleRequest: make$handleRequest((work) => work()),
  };
  const serially = (work) => queued(() => work(inside));

  return {
    exports,
    handleRequest: make$handleRequest(serialised),
    drop,
    serially,
  };
}

/**
 * This module as a fetch handler -- the whole Worker, with nothing left over:
 *
 *   import module from "./worker.wasm";
 *   import { worker } from "./worker.js";
 *
 *   export default worker(module);
 *
 * A Request becomes the envelope the module's entry point takes, the head it
 * answers becomes a Response, and the instance is created on the FIRST REQUEST
 * (a Worker forbids drawing entropy in global scope) and retired if a call ever
 * traps.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @param {object} [options] `host` -- one plain function per import, keyed by
 *   module and field, as `instantiate` takes it; `remoteAddr` -- (request, env, ctx) => the client
 *   address, since which header carries it is the platform's business and not
 *   this file's (`(r) => r.headers.get("cf-connecting-ip")` on Cloudflare)
 * @returns {object} `{ fetch(request, env, ctx) }`
 */
export function worker(module, options = {}) {
  let instance = null;
  // Set when a call TRAPPED: that instance skipped its arena reset and its Lisp
  // state may be half-written, so nothing else may run on it. A Lisp ERROR is not
  // this -- the transport answers 500 itself and the instance is fine.
  let poisoned = false;
  // The request body the module pulls, and the response body coming back the
  // same way. Both belong to the ONE call running below, which is where they
  // are set.
  let requestBody = null;
  let responseChunks = [];
  const collected = () => {
    const all = new Uint8Array(
      responseChunks.reduce((n, chunk) => n + chunk.length, 0),
    );
    let at = 0;
    for (const chunk of responseChunks) {
      all.set(chunk, at);
      at += chunk.length;
    }
    return all;
  };
  const base = {};
  base.env = {
    ...(base.env ?? {}),
    readRequestBody: () => {
      // Handed over ONCE: a chunk source that never answers null is one the
      // module pulls forever.
      const chunk = requestBody;
      requestBody = null;
      return chunk;
    },
    writeResponseBody: (chunk) => responseChunks.push(chunk),
  };
  const given = options.host ?? {};
  const host = {};
  for (const key of new Set([...Object.keys(base), ...Object.keys(given)])) {
    host[key] = { ...(base[key] ?? {}), ...(given[key] ?? {}) };
  }
  const live = () => {
    if (poisoned) {
      instance = null;
      poisoned = false;
    }
    return (instance ??= instantiate(module, host));
  };

  // The request head. `target` stays RAW -- path and query still joined and
  // still percent-encoded -- because the shared normalizer on the other side
  // owns that split, and a pre-split path leaves the query string nil.
  const envelope = (request, octets, remoteAddr) => {
    const url = new URL(request.url);
    const headers = Object.fromEntries(request.headers);
    // A body with no content-length is a body the request parser does not read,
    // and a chunked request carries none -- so set it from the octets we have.
    if (octets?.length) headers["content-length"] = String(octets.length);
    const head = {
      method: request.method,
      target: url.pathname + url.search,
      headers,
      scheme: url.protocol.replace(":", ""),
    };
    if (remoteAddr != null) head["remote-addr"] = remoteAddr;
    return JSON.stringify(head);
  };

  return {
    // EVERYTHING is inside the try, not just the module call: reading an
    // aborted upload rejects, and `new Response` throws on a status or a
    // header an application is free to produce (0, 999, a newline in a
    // value). Outside it those escape as an unhandled rejection, which the
    // platform answers with its own error page and nothing in the log.
    async fetch(request, env, ctx) {
      let entered = false;
      try {
        const remoteAddr = await options.remoteAddr?.(request, env, ctx);
        const octets = request.body
          ? new Uint8Array(await request.arrayBuffer())
          : null;
        const input = envelope(request, octets, remoteAddr);
        const head = JSON.parse(
          await live().serially((lisp) => {
            // Re-read INSIDE the critical section: the instance was bound at
            // admission, and a call parked ahead of this one can poison it
            // before this one runs. Refusing is the whole point -- a
            // half-unwound instance answers wrong rather than failing, and
            // the module's own re-entry guard is cleared by the landing pad
            // on exactly the path that poisons it.
            if (poisoned) throw new Error("instance discarded by an earlier trap");
            // Per-call state, set HERE and not beside the call: a suspended
            // handler returns to the event loop, so the next request would
            // otherwise move it under this one.
            requestBody = octets;
            responseChunks = [];
            entered = true;
            return lisp.handleRequest(input);
          }),
        );
        // Headers as an ARRAY of pairs, which keeps two Set-Cookie two.
        // An EMPTY body becomes null whichever shape it arrived in: 204/205/304
        // may only be constructed with a null body, and "" and a zero-length
        // Uint8Array are the same response as none.
        const body = head.body ?? collected();
        return new Response(body?.length ? body : null, {
          status: head.status ?? 200,
          headers: head.headers ?? [],
        });
      } catch (error) {
        console.error("handle-request failed:", error);
        // Only a call that ENTERED the module can have left it half-written.
        // A mapping, or a Response the platform refused, says nothing about
        // the instance, and discarding it would cost the next request a
        // reinstantiation for someone else's bad header.
        if (entered) poisoned = true;
        return new Response("internal error\n", { status: 500 });
      }
    },
  };
}


---

# FILE: references/examples/cloudflare-workers/httpbin-clack/worker.lisp

;;; Clack, plain: a mini httpbin as an application FUNCTION -- the environment
;;; plist in, the (status headers body) list out -- and one MIDDLEWARE, which in
;;; Clack is just a function from application to application. Clack has no
;;; router, so dispatch is a `cond` over :path-info. Nothing here mentions
;;; Cloudflare: the same `app` runs on hunchentoot, on woo, under
;;; `wasmtime serve` and on the JVM.
;;;
;;; :server :reactor is the handler backend for a host that CALLS you instead of
;;; handing you a socket; the compiler synthesizes the handle-request export
;;; src/index.js calls, because rontolisp:wasm-export needs a literal name at
;;; compile time and a clackup call has none to give.

(ql:quickload '("clack" "clack-handler-reactor"))

;;; --- the endpoints -----------------------------------------------------------

;; clack's :raw-body is a synchronous stream, and nil when there is no body.
(defun read-body (stream)
  (if (null stream)
      ""
      (with-output-to-string (out)
        (do ((ch (read-char stream nil nil) (read-char stream nil nil)))
            ((null ch))
          (write-char ch out)))))

;; Parse the body as JSON when it looks like one, and fall back to null when it
;; does not parse -- which is what the real httpbin does.
(defun body-json (body)
  (if (and (stringp body) (> (length body) 0)
           (or (eql (char body 0) #\{) (eql (char body 0) #\[)))
      (handler-case (rontolisp:json-parse body) (error () 'null))
      'null))

(defun json-body (object)
  (list (format nil "~a~%" (rontolisp:json-stringify object))))

;; plist-hash-table and alist-hash-table give json-stringify the string-keyed
;; hash tables it serializes as objects (:method becomes "method"; an empty
;; query still renders {}), and the env :headers already is one.
(defun echo (env with-body)
  (let ((info
         (rontolisp:plist-hash-table
          (list :args (rontolisp:alist-hash-table
                       (rontolisp:query-params (getf env :query-string)))
                :headers (getf env :headers)
                :method (symbol-name (getf env :request-method))
                :path (getf env :path-info)))))
    (when with-body
      (let ((body (read-body (getf env :raw-body))))
        (setf (gethash "data" info) body)
        (setf (gethash "json" info) (body-json body))))
    (list 200 nil (json-body info))))

;; :request-method is an interned keyword, so the check is eq.
(defun endpoint (env method with-body)
  (if (eq (getf env :request-method) method)
      (echo env with-body)
      (list 405 nil
            (json-body
             (rontolisp:plist-hash-table
              (list :error "method not allowed"
                    :allowed (symbol-name method)))))))

;;; --- the application ---------------------------------------------------------

;; :path-info carries the decoded path only -- the query string arrives
;; separately -- so the comparisons are exact.
(defun app (env)
  (let ((path (getf env :path-info)))
    (cond ((string= path "/get") (endpoint env :GET nil))
          ((string= path "/post") (endpoint env :POST t))
          ((string= path "/put") (endpoint env :PUT t))
          ((string= path "/patch") (endpoint env :PATCH t))
          ((string= path "/delete") (endpoint env :DELETE t))
          (t (list 404 nil
                   (json-body
                    (rontolisp:plist-hash-table
                     (list :error "not found" :path path))))))))

;; A middleware takes an application and returns one, which is why no endpoint
;; above sets a header. Several of them compose with lack:builder.
(defun wrap-json (app)
  (lambda (env)
    (let ((response (funcall app env)))
      (list* (first response)
             (list* :content-type "application/json" (second response))
             (cddr response)))))

(clack:clackup (wrap-json #'app) :server :reactor :use-thread nil)


---

# FILE: references/examples/cloudflare-workers/httpbin-clack/wrangler.jsonc

{
  // https://developers.cloudflare.com/workers/wrangler/configuration/
  "name": "rontolisp-httpbin-clack",
  "main": "src/index.js",
  "compatibility_date": "2026-01-01",

  // `import module from "./worker.wasm"` needs no configuration: .wasm is a
  // built-in module rule for ES-module Workers, and the file is compiled at
  // deploy time rather than on the request path.
  "observability": {
    "enabled": true
  }
}


---

# FILE: references/examples/cloudflare-workers/httpbin-component/README.md

# httpbin-component — the same handler, through the component model

This directory compiles **[`../httpbin/worker.lisp`](../httpbin/worker.lisp)** —
the same file, not a copy — as a WebAssembly **reactor component**
(`--component --no-wasi`) and runs it on Cloudflare Workers through
`jco transpile`. Same routes, same responses; a different way of crossing the
boundary.

```bash
./build.sh          # ../httpbin/worker.lisp -> worker.wasm -> src/dist/
npx wrangler dev    # http://localhost:8787
```

## What it buys

The canonical ABI marshals the strings, so the call is a call:

```js
const result = lisp.handleRequest(input);   // string in, string out
```

Compare [`../httpbin/src/index.js`](../httpbin/src/index.js), whose
`handleRequest` allocates, writes bytes into linear memory, reads a `[ptr, len]`
pair back out and pops a bump-allocator arena to do the same thing. That whole
function disappears here. It is the entire benefit, and it is a genuine one.

## What it costs

| | [`../httpbin`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin) | this directory |
| --- | --- | --- |
| Files the Worker imports | 1 × `.wasm` | 1 × `.wasm` + a generated `worker.js` ([size report](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/size-report/results/cloudflare-workers.md)) |
| Build tools | the rontolisp compiler | + `@bytecodealliance/jco` |
| WASI imports to satisfy | none | none |
| Top-level `defparameter` | works, via `_initialize` | works, at instantiation |
| Both bodies | out of band, as octets through `env.readRequestBody` / `env.writeResponseBody` | inside the envelopes' `"body"` keys |

That last row is the one place the shared source diverges, and it is the
`(not rontolisp-component)` half of the guard on the two imports in
`worker.lisp`: a component's host functions cross the canonical ABI rather than
a core import, so there is no byte-shaped import here to carry a body through in
either direction. This build therefore pays what
[`../httpbin`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin#neither-body-is-in-the-envelope) stopped paying — each
body JSON-escaped into its head, and no way to carry a binary one.

`--no-wasi` is what makes the right column read that way: it asks for a
component that **imports nothing** (`wasm-tools component wit` shows not a
single `import` line), so the generated glue's `ImportObject` type is literally
empty and `src/index.js` instantiates with `{}`. The Lisp top level runs from
the core module's *start section* inside `instantiate`. Without the flag, the
same build imports three WASI interfaces that must be stubbed by hand, ships two
extra (empty) core modules, and cannot run its top-level forms at all, because
they live in a `wasi:cli/run` export jco cannot drive.

## The two things that are not obvious

**1. jco's default output does not run on Workers.** It calls
`WebAssembly.compile()` on an inlined base64 blob at module scope, and a Worker
may not compile WebAssembly at run time — workerd rejects the module with
`Uncaught Error: Top-level await in module is unsettled.` `--tla-compat` moves
that into an awaited `$init` promise, which is worse: the Worker starts and then
every request hangs. The mode that works is **`--instantiation sync`**, where
the glue asks the host for each already-compiled core module through a
`getCoreModule(path)` callback. A `.wasm` import is exactly that:

```js
import core0 from "./dist/worker.core.wasm";
// ...
instantiate(() => core0, {});
```

`-b 0` goes with it, to stop jco inlining a small core module as base64 rather
than emitting the file we need to import.

**2. `handler-case` needs `--bindgen-enable-wasm-exnref`.** Without it jco
refuses the component before generating anything (`exceptions proposal not
enabled`).

## When this path is actually clean

When the program fits the `--no-gc` subset. [`../hello`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/hello) has no
component build of its own, but try it: `--no-gc --component` is a tiny
component that imports *nothing*, so there is no exnref flag and the glue has no
dependencies:

```bash
JAR=../../../target/rontolisp-0.1.0-SNAPSHOT-exec.jar
java -jar $JAR ../hello/worker.lisp -o worker-no-gc.wasm --no-gc --component --optimize
npx -y @bytecodealliance/jco transpile worker-no-gc.wasm -o worker-no-gc-dist
```

```console
$ node --input-type=module -e '
    import { add, fib, greet } from "./worker-no-gc-dist/worker-no-gc.js";
    console.log(add(2, 3), fib(20), greet());'
5 6765 Hello from Lisp, compiled to WebAssembly!
```

(That plain transpile is enough for Node. For a Worker, add
`--instantiation sync -b 0` and hand the core module over as above.)

Even there it is generated JavaScript standing in for about ten lines of
hand-written glue — which is the honest summary of this whole directory.

## Rebuilding

```bash
./mvnw clean package -DskipTests   # from the repository root
examples/cloudflare-workers/httpbin-component/build.sh
```

`build.sh` prints the core-module file names it produced. A reactor component
has exactly one; if a rebuild ever prints more, the program stopped being
import-free (or the flag went missing) and `src/index.js` needs to hear about it.


---

# FILE: references/examples/cloudflare-workers/httpbin-component/build.sh

#!/usr/bin/env bash
# Build ../httpbin/worker.lisp -- the SAME Lisp source -- as a WebAssembly
# component, and transpile it for Cloudflare Workers with jco.
#
# Workers has no native component-model support, so `jco transpile` lowers the
# component to a core module plus JavaScript glue. Three flags make that glue
# runnable inside a Worker:
#
#   --instantiation sync       the glue does not compile wasm itself; it asks the
#                              host for each already-compiled core module. Workers
#                              forbids runtime WebAssembly compilation, so this is
#                              the only mode that works -- the default glue calls
#                              WebAssembly.compile() and hangs at startup.
#   -b 0                       never inline a core module as base64; emit it as a
#                              .wasm file the Worker can `import`.
#   --bindgen-enable-wasm-exnref   accept the exception-handling instructions
#                              `handler-case` compiles into.
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling ../httpbin/worker.lisp -> worker.wasm (reactor component)"
# --optimize=size: same trade as the sibling builds -- smaller core modules for
# a per-request cost of a few microseconds.
# --no-wasi: worker.lisp does no I/O, so ask for the REACTOR component -- it
# imports NOTHING (no WASI stubs to hand-write on the JavaScript side) and its
# top-level forms run at instantiation, exactly like ../httpbin's _initialize.
java -jar "$jar" "$here/../httpbin/worker.lisp" -o "$here/worker.wasm" --component --no-wasi --optimize=size

echo "transpiling worker.wasm -> src/dist/"
rm -rf "$here/src/dist"
npx -y @bytecodealliance/jco transpile "$here/worker.wasm" -o "$here/src/dist" \
  --instantiation sync -b 0 --bindgen-enable-wasm-exnref

echo
echo "core modules src/index.js must import (a reactor component has ONE):"
ls -1 "$here/src/dist"/*.wasm | xargs -n1 basename
echo "done. Run it with:  npx wrangler dev"


---

# FILE: references/examples/cloudflare-workers/httpbin-component/package.json

{
  "name": "rontolisp-cloudflare-httpbin-component",
  "version": "0.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "build": "./build.sh",
    "dev": "wrangler dev",
    "deploy": "wrangler deploy"
  },
  "devDependencies": {
    "@bytecodealliance/jco": "^1",
    "wrangler": "^4"
  }
}


---

# FILE: references/examples/cloudflare-workers/httpbin-component/src/index.js

// index.js -- the same handler as ../httpbin, reached through the component
// model. The one line worth comparing is the `handleRequest` call below: plain
// JavaScript strings in and out. What that costs is in ../README.md.

import core0 from "./dist/worker.core.wasm";
import { instantiate } from "./dist/worker.js";

// jco's glue asks the host for each core module by the file name it emitted.
// A --no-wasi reactor component has exactly ONE -- the whole program -- and
// nothing to import: the second argument really is the empty object (the
// generated .d.ts says `interface ImportObject {}`).
//
// The Lisp top level runs inside `instantiate` (the core module's start
// section), so a `defparameter` in worker.lisp is already assigned before the
// first request -- the reactor counterpart of ../httpbin calling _initialize.
//
// At module scope, so the cost is isolate startup rather than request work.
const getCoreModule = () => core0;
let lisp = instantiate(getCoreModule, {});

// The envelope is worker.lisp's, not this directory's, so this function is
// ../../httpbin/src/index.js's unchanged -- the component model moves the
// boundary, not the contract. Its two load-bearing fields (the RAW target and
// the forwarded content-length) are commented there.
/** The raw facts the Lisp side turns into the Clack environment. */
async function requestToJson(request) {
  const url = new URL(request.url);
  const hasBody = request.method !== "GET" && request.method !== "HEAD";
  const body = hasBody ? await request.text() : "";
  const headers = Object.fromEntries(request.headers);
  if (body) headers["content-length"] = String(new TextEncoder().encode(body).length);

  return JSON.stringify({
    method: request.method,
    target: url.pathname + url.search,
    scheme: url.protocol.replace(":", ""),
    "remote-addr": request.headers.get("cf-connecting-ip"),
    headers,
    body,
  });
}

export default {
  async fetch(request) {
    const input = await requestToJson(request);
    let reply;
    try {
      reply = JSON.parse(lisp.handleRequest(input));
    } catch (error) {
      lisp = instantiate(getCoreModule, {}); // retire the trapped one
      console.error("handleRequest failed:", error);
      return new Response("internal error\n", { status: 500 });
    }
    // An ARRAY of [name, value] pairs, so a repeated Set-Cookie survives.
    return new Response(reply.body ?? "", {
      status: reply.status ?? 200,
      headers: reply.headers ?? [],
    });
  },
};


---

# FILE: references/examples/cloudflare-workers/httpbin-component/wrangler.jsonc

{
  // https://developers.cloudflare.com/workers/wrangler/configuration/
  "name": "rontolisp-httpbin-component",
  "main": "src/index.js",
  "compatibility_date": "2026-01-01"
}


---

# FILE: references/examples/cloudflare-workers/httpbin-ningle/README.md

# httpbin-ningle — the same endpoints on an application object

The endpoints of [`../httpbin-clack`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin-clack) written the way
[ningle](https://github.com/fukamachi/ningle) wants them: the application is a
CLOS **object** you assign routes to, and the request arrives already decoded.

```bash
./build.sh          # worker.lisp -> src/worker.wasm
npx wrangler dev    # http://localhost:8787
npx wrangler deploy
rontolisp check.lisp    # drive the whole Worker locally, on any backend
```

```console
$ curl 'http://localhost:8787/get?a=1&b=two'
{"method":"GET","path":"/get","args":{"a":"1","b":"two"},"form":{},"headers":{...}}

$ curl -H 'content-type: application/json' -d '{"name":"rontolisp"}' \
       http://localhost:8787/post
{"method":"POST","path":"/post","args":{},"form":{"name":"rontolisp"},...}

$ curl http://localhost:8787/status/418
418

$ curl http://localhost:8787/status/teapot     # not three digits -> no rule
{"error":"not found","path":"/status/teapot"}  # matches -> not-found
```

## The four things that make it ningle

### A route is an assignment, so routes can be a loop

There is no enclosing route-list form. `(setf (ningle:route app url) controller)`
mutates the application object, which means routes can be added from another
file, from a function, or at run time — and that the five echo endpoints are
written once:

```lisp
(dolist (endpoint '(("/get" . :GET) ("/post" . :POST) ("/put" . :PUT)
                    ("/patch" . :PATCH) ("/delete" . :DELETE)))
  (let ((path (car endpoint)) (allowed (cdr endpoint)))
    (setf (ningle:route *app* path :method allowed) #'echo)
    (setf (ningle:route *app* path :method :ANY) ...)))   ; the 405
```

Rules are tried in the order they were assigned, so the second rule of each pair
is a per-path fallback: the method that path answers goes to `echo`, anything
else to a 405 naming it. That leaves `ningle:not-found` with only the answer it
is really for.

### A controller returns the body; `*response*` carries the rest

The `(status headers body)` triple never appears in this file. A controller
returns a **string** — ningle makes that the body — and says everything else by
mutating `ningle:*response*`, the response object bound for the current request:

```lisp
(defun respond-json (object)
  (setf (lack.response:response-headers ningle:*response*)
        (list :content-type "application/json"))
  (format nil "~a~%" (rontolisp:json-stringify object)))

(defun set-status (code)
  (setf (lack.response:response-status ningle:*response*) code))
```

### The request arrives decoded, so there is one echo controller

A controller receives the **parameters**, not the Clack environment — and by the
time it runs, `lack/request` has decoded the query string *and parsed the request
body*. `args` is the query string, `form` is the parsed body (for the JSON post
above, the JSON object itself), and the alist ningle hands the controller is
those two appended.

So nothing here reads a stream or parses JSON, and the five echo endpoints share
one controller. That is also the visible difference from the neighbouring
documents, and not a cosmetic one: they answer `data` (the raw body) and `json`
(their own parse of it) because they read `:raw-body` themselves. Here the parse
already happened.

### Declining means not matching — so `/status` is a regex rule

Returning `nil` from a ningle controller is **not** a decline; it answers an
empty body. The only way a route declines is by not matching, so the status
endpoint is written as a rule that cannot match a bad code — myway's other rule
spelling, a regex instead of a template, whose capture groups arrive as
`:captures`:

```lisp
(setf (ningle:route *app* "/status/([0-9]{3})" :regexp t)
      (lambda (params)
        (let ((code (parse-integer (first (cdr (assoc :captures params))))))
          (set-status code)
          (respond-text code))))
```

A `"/status/:code"` template would match `/status/teapot` as happily and leave
the controller holding a request it has no good answer for. This way
`/status/teapot` matches no rule at all and reaches `ningle:not-found`, a generic
function on the application class — ningle's own extension point, and the 404 is
an *override* of it rather than a route at the bottom of a list.

## The endpoints

| | |
| --- | --- |
| `GET /get` | echo the request: `method`, `path`, `args`, `form`, `headers` |
| `POST /post`, `PUT /put`, `PATCH /patch`, `DELETE /delete` | the same, with the parsed body in `form` |
| any of those, wrong method | 405 from that path's `:ANY` rule, naming the method that works |
| `ANY /anything` | echo, whatever the method — `:ANY` used as itself rather than as a fallback |
| `GET /status/NNN` | answer with that status; a code that is not three digits matches no rule |
| anything else | 404 from the overridden `ningle:not-found` |

## What it costs

This is by an order of magnitude the largest and slowest to start of the four
httpbin Workers ([size
report](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/size-report/results/cloudflare-workers.md)) — **and almost none
of that is ningle.** It is the `lack-request` chain the section above is about:
getting the request decoded before the controller runs means `http-body`,
`fast-http`'s generated header and multipart state machines, `smart-buffer`,
`circular-streams`, `yason`, `trivial-mimes` and `quri` all ship and all run.
tiny-routes never touches any of it. There is no size opt-in to offer either:
myway compiles every rule to a **cl-ppcre scanner**, so the regex engine is
genuinely reachable. It still fits the free plan's bundle limit with room to
spare — a cost, not a wall — but it is the reason to reach for `tiny-routes`
when routing is all you need.

## Developing without Cloudflare

As in [`../httpbin-clack`](../httpbin-clack/README.md): the synthesized export
calls `clack.handler.reactor:dispatch`, an ordinary function, so the whole
Worker — routes, the `not-found` method and all — runs on every backend, which
[`examples/examples.yaml`](../../examples.yaml) pins:

```bash
rontolisp check.lisp
rontolisp check.lisp -o Check.class && java -cp rontolisp-0.1.0-SNAPSHOT-exec.jar:. Check
rontolisp check.lisp -o check.wasm --optimize && wasmtime run -W gc -W exceptions=y check.wasm
```

(Key order inside a JSON object differs per backend — it follows hash-table
iteration order — which is why the manifest checks with `contains`.)

To serve the same model over a real socket, drop `clack-handler-reactor` and use
`:server :rontolisp` — that is what
[`examples/net/httpbin-ningle.lisp`](../../net/httpbin-ningle.lisp) does.

## What's in here

| File | Purpose |
| --- | --- |
| [`worker.lisp`](worker.lisp) | **The whole program**: quickload, the routes, the `not-found` method, `clackup` |
| [`check.lisp`](check.lisp) | Drives it with no Cloudflare in sight — the local edit/run loop |
| [`src/index.js`](src/index.js) | Three lines over the generated `worker()`, plus the one hook it leaves to a caller — which header carries the client address |
| [`src/worker.js`](src/worker.js) | The boundary, GENERATED by `--emit-js-glue` from `worker.lisp`'s declarations. Byte-identical to `../httpbin-clack/src/worker.js`, because the declarations are. Do not edit; `./build.sh` rewrites it |
| `src/worker.wasm` | A build product — run `./build.sh` first |

## Limitations

The Worker sandbox and `--no-wasi` limitations of
[`../httpbin-clack`](../httpbin-clack/README.md#limitations) apply unchanged.


---

# FILE: references/examples/cloudflare-workers/httpbin-ningle/build.sh

#!/usr/bin/env bash
# Compile worker.lisp -- clack, ningle, the routes and the clackup call -- to
# the .wasm module the Worker imports.
#
# --no-wasi: the Worker calls the exported `handle-request` directly, it never
#   runs the module as a program, and the handler does no I/O -- so the module
#   needs no WASI imports at all. It becomes a reactor: nothing to shim on the
#   JavaScript side, and `_initialize` instead of `_start`.
# --optimize=size: a Worker bundle has a size limit, and the tree-shaker is what
#   keeps the module down. ningle has no size opt-in to offer the way
#   tiny-routes does: myway compiles every rule to a cl-ppcre scanner, so the
#   regex engine is genuinely reachable and the shaker is right to keep it.
#
# The first run downloads clack/lack/ningle into ~/.rontolisp/quicklisp; after
# that the build is offline.
# --host-boundary=streaming: this Worker ECHOES request bodies, so they must
#   cross as octets rather than as JSON text in the envelope -- which is what
#   the generated src/worker.js feeds through env.readRequestBody /
#   env.writeResponseBody, and what lets a BINARY body come back exactly. Asked
#   for, because the default is `envelope` (see ../btc-ticker), where a body
#   rides the head instead.
# --emit-js-glue: write src/worker.js beside the module -- the host half of this
#   boundary, from the program's own declarations: the import object, the
#   (ptr, len) staging, the __ronto_alloc bracket, the two body imports above --
#   fed from the Request it is already holding and the Response it is already
#   building -- and the Request -> envelope -> Response mapping over them. That
#   half is derivable on THIS boundary too, so src/index.js is a worker(module)
#   call. It is CHECKED IN and pinned by HostGlueEmitterTest, so regenerate it
#   here rather than editing it.
#
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling worker.lisp -> src/worker.wasm + src/worker.js"
java -jar "$jar" "$here/worker.lisp" -o "$here/src/worker.wasm" \
  --no-wasi --host-boundary=streaming --optimize=size --emit-js-glue

ls -l "$here/src/worker.wasm" "$here/src/worker.js"
echo "done. Run it with:  npx wrangler dev"


---

# FILE: references/examples/cloudflare-workers/httpbin-ningle/check.lisp

;;; Drive worker.lisp's handler without Cloudflare, on any backend
;;; (../httpbin-clack/check.lisp has the notes). The probes are chosen for the
;;; four ningle mechanisms worker.lisp is built on: the parsed body arriving as
;;; `form`, the :ANY fallback route answering 405, the regex rule declining a
;;; non-numeric status, and ningle:not-found answering the 404.

(load "worker.lisp")

(defun try (request-plist)
  (let ((request
         (rontolisp:json-stringify (rontolisp:plist-hash-table request-plist))))
    (format t "~&--> ~a~%" request)
    (format t "<-- ~a~%" (clack.handler.reactor:dispatch request))))

(defun headers (&rest plist) (rontolisp:plist-hash-table plist))

(defun json-headers (body)
  (headers :host "example.com"
           :content-type "application/json"
           :content-length (princ-to-string (length body))))

;; GET /get with a query string -- it comes back as "args".
(try
 (list :method "GET"
       :target "/get?a=1&b=two"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (headers :host "example.com" :accept "application/json")
       :body ""))

;; POST /post with a JSON body -- it comes back as "form", already parsed:
;; lack/request did that before the controller ran.
(try
 (list :method "POST"
       :target "/post"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (json-headers "{\"name\":\"rontolisp\"}")
       :body "{\"name\":\"rontolisp\"}"))

;; A form-encoded body reaches the same field by the same route.
(try
 (list :method "PUT"
       :target "/put"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (headers :host "example.com"
                         :content-type "application/x-www-form-urlencoded"
                         :content-length "9")
       :body "name=lisp"))

;; The wrong method for an endpoint -- the method rule does not match, the :ANY
;; rule assigned right after it does, and that one answers 405.
(try
 (list :method "GET"
       :target "/post"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (headers :host "example.com")
       :body ""))

;; /anything answers whatever method it is asked with.
(try
 (list :method "DELETE"
       :target "/anything"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (headers :host "example.com")
       :body ""))

;; An unknown path -- ningle:not-found, the overridden method.
(try
 (list :method "GET"
       :target "/nope"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (headers :host "example.com")
       :body ""))

;; The regex rule -- "418" is three digits, so it matches and :captures binds.
(try
 (list :method "GET"
       :target "/status/418"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (headers :host "example.com")
       :body ""))

;; "teapot" is not, so no rule matches at all and not-found answers the 404 --
;; no controller had to decide anything.
(try
 (list :method "GET"
       :target "/status/teapot"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (headers :host "example.com")
       :body ""))


---

# FILE: references/examples/cloudflare-workers/httpbin-ningle/package.json

{
  "name": "rontolisp-cloudflare-httpbin-ningle",
  "version": "0.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "build": "./build.sh",
    "dev": "wrangler dev",
    "deploy": "wrangler deploy",
    "tail": "wrangler tail"
  },
  "devDependencies": {
    "wrangler": "^4"
  }
}


---

# FILE: references/examples/cloudflare-workers/httpbin-ningle/src/index.js

// The whole Worker. `./worker.js` is generated from worker.lisp's own
// declarations (build.sh's --emit-js-glue) and owns all of it: the import
// object, the (ptr, len) staging, the __ronto_alloc bracket, the two body
// imports this boundary declares -- fed from the Request it is already holding
// and the Response it is already building -- and the Request -> envelope ->
// Response mapping over them.
//
// That the STREAMING boundary needs no more of a host than ../hello-clack's
// envelope one is the point of the pair: what the boundary buys is a binary
// body crossing exactly and a large one never doubling linear memory, not a
// bigger host. ../httpbin is the one Worker here that still writes its own,
// because it declares `handle-request` by hand and the compile path recognises
// the SYNTHESIZED bridge -- read that directory's src/index.js for what this
// file would otherwise say.
//
// BYTE-IDENTICAL in every httpbin-* directory that goes through clackup. A new
// sibling copies this file, it does not edit it.

import module from "./worker.wasm";
import { worker } from "./worker.js";

export default worker(module, {
  // Clack's :remote-addr. Which header carries the client address is the
  // platform's business and not the glue's, so it is the one thing worker()
  // leaves to its caller; Cloudflare puts it here. There is no peer port to
  // report, so :remote-port stays nil.
  remoteAddr: (request) => request.headers.get("cf-connecting-ip"),
});


---

# FILE: references/examples/cloudflare-workers/httpbin-ningle/src/worker.js

// GENERATED by rontolisp --emit-js-glue -- do not edit.
//
// The host half of this module's boundary, derived from the program's own
// declarations: one import-object entry per rontolisp:wasm-import, one entry
// point per rontolisp:wasm-export, and every piece of linear-memory plumbing
// between them -- the (ptr, len) pair a :string crosses as, the __ronto_alloc
// bracket around a call, and the read(2) cursor a :bytes result is pulled
// through.
//
// What a declaration cannot state is what a host function DOES, so that is the
// one thing this file asks for: `host` is a plain function per import, keyed by
// import module and field, taking and answering ordinary JavaScript values --
// never a (ptr, len) pair, and, where an entry answers `chunk` below, a
// Uint8Array or a string with null for the end of them.
//
//   import { instantiate, suspending } from "./worker.js";
//
//   const lisp = instantiate(module, {
//     env: {
//       readRequestBody: () => chunk,   // or leave it to worker()
//       writeResponseBody: (chunk) => {},   // or leave it to worker()
//     },
//   });
//   await lisp.handleRequest(text);
//
// A host that suspends marks its own entries -- suspending(async (...) => ...)
// -- and every entry point above then answers a promise: the marked imports are
// wrapped in WebAssembly.Suspending, each entry point that can reach one is
// entered through WebAssembly.promising, and calls are serialised onto one
// promise chain, because a suspended module returns to the host's event loop
// and a re-entered export refuses with a trap rather than corrupting both
// calls. Host state that belongs to ONE such call is set inside that section:
//
//   await lisp.serially(async (entry) => { ...; return entry.handleRequest(...) });
//
// This module's boundary is the reactor envelope, so the Request/Response half
// is derivable too and `worker` below is it:
//
//   import module from "./worker.wasm";
//   import { worker } from "./worker.js";
//
//   export default worker(module);
//
// Regenerate it, never edit it: --emit-js-glue on the compile that wrote the
// .wasm beside it.

const encoder = new TextEncoder();
// ignoreBOM, because these octets are a VALUE and not a document: a
// leading U+FEFF is a character the other side chose, and the default
// decoder deletes it -- silently shortening a BOM-prefixed request body
// by one character while the content-length beside it still counts three.
const decoder = new TextDecoder("utf-8", { ignoreBOM: true });

const SUSPENDING = Symbol.for("rontolisp.suspending");

/**
 * Marks a host function as one that answers a promise, so the wasm stack parks
 * until it settles (JSPI). The wrapper is not free -- an import answering
 * SYNCHRONOUSLY through one still returns to the event loop -- so mark only the
 * entries this host really implements asynchronously.
 *
 * @param {Function} fn the host function
 * @returns {object} the marked entry, to be passed as the import
 */
export function suspending(fn) {
  return { [SUSPENDING]: fn };
}

/**
 * Instantiates the module against this host and returns its callable surface.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @param {object} host one function per import, keyed by module and field
 * @returns {object} `exports`, plus one entry point per rontolisp:wasm-export
 */
export function instantiate(module, host = {}) {
  let exports;

  // Every view is built where it is used, never held: growing the module's
  // memory DETACHES the buffer behind every view taken before the growth, and a
  // suspending host resumes after growths it never saw.
  const readString = (ptr, len) =>
    decoder.decode(new Uint8Array(exports.memory.buffer, ptr, len));

  // Octets, COPIED: the module pops the staging behind the pointer the moment
  // the call returns, so a chunk not taken by then is one the host never gets.
  const readBytes = (ptr, len) =>
    new Uint8Array(exports.memory.buffer.slice(ptr, ptr + len));

  // Bytes the HOST hands over live in the module's own bump allocator, and
  // cross as the (ptr, len) pair every memory-typed value is.
  const write = (octets) => {
    const ptr = exports.__ronto_alloc(octets.length);
    new Uint8Array(exports.memory.buffer, ptr, octets.length).set(octets);
    return [ptr, octets.length];
  };
  const writeString = (value) => write(encoder.encode(String(value)));
  const octets = (chunk) =>
    typeof chunk === "string" ? encoder.encode(chunk) : chunk;

  // A host answers a value, or -- from an entry it marked suspending -- a promise
  // of one. Both shapes ride one expression, which is what lets this file drive a
  // synchronous host and a JSPI host without being written twice.
  const marked = new Set();
  const unmarked = (what) =>
    what +
    " answered a promise; wrap it in suspending() so the wasm stack parks" +
    " until it settles";
  const settle = (what, value, next) => {
    if (typeof value?.then === "function") {
      if (!marked.has(what)) {
        // Reported before it is thrown: this throw crosses back into wasm,
        // where a catch_all landing pad turns it into whatever the module
        // makes of a failed import.
        console.error(unmarked(what));
        throw new TypeError(unmarked(what));
      }
      return value.then(next);
    }
    return next(value);
  };

  // The entries the module really imports: --optimize shakes out an import the
  // program never calls, and a host should not have to know which survived.
  const linked = new Set(
    WebAssembly.Module.imports(module).map((i) => i.module + "." + i.name),
  );
  const bind = (moduleName, field, wrap) => {
    const key = moduleName + "." + field;
    const given = (host[moduleName] ?? {})[field];
    if (given == null) {
      if (!linked.has(key)) return undefined;
      throw new TypeError("host." + key + " is missing; this module imports it");
    }
    if (given[SUSPENDING] === undefined) {
      // An async function that was never marked is the one mistake worth
      // catching HERE, where the report is a plain stack and not whatever
      // the module makes of an import that threw.
      if (given.constructor?.name === "AsyncFunction") {
        throw new TypeError(unmarked("host." + key));
      }
      return wrap(key, given);
    }
    marked.add(key);
    return new WebAssembly.Suspending(wrap(key, given[SUSPENDING]));
  };

  // A :bytes RESULT is the read(2) shape: the MODULE owns the buffer and asks
  // for up to `cap` octets, so what a host answers is the next CHUNK and this
  // holds whatever did not fit. That remainder is the read side's only state,
  // and it is why a host supplies chunks rather than a reader -- which source
  // they come from (a ReadableStream, a Uint8Array) is all that is left to it.
  const readers = new Map();
  const reader = (what, source) => {
    let rest = null;
    let from = null;
    // A body the module did not drain belongs to the call that could have and
    // to no other, so every cursor is dropped at the next entry below -- and a
    // host whose SOURCE moves inside one call (a new upstream reply opened by
    // another import) drops this one itself with lisp.drop(key), because what
    // did not fit is held here and nothing else can see the source move.
    readers.set(what, () => {
      rest = null;
      from = null;
    });
    const drain = (ptr, cap) => {
      const n = Math.min(cap, rest.length);
      new Uint8Array(exports.memory.buffer, ptr, n).set(rest.subarray(0, n));
      rest = rest.subarray(n);
      return n;
    };
    // A read that FAILS answers a NEGATIVE count. Throwing would trap the
    // instance; the count is an error channel the module turns into a Lisp
    // condition where the octets are consumed, which is where every other
    // backend reports a transfer that broke mid-body.
    const failed = (error) => {
      console.error(what + " failed:", error);
      return -1;
    };
    return (args, ptr, cap) => {
      // The remainder belongs to the arguments that asked for it: a source
      // selected by argument must not be served the previous one's octets.
      const key = JSON.stringify(args);
      if (from !== key) {
        rest = null;
        from = key;
      }
      if (rest !== null && rest.length !== 0) return drain(ptr, cap);
      try {
        const answer = settle(what, source(...args), (chunk) => {
          rest = chunk == null ? new Uint8Array(0) : octets(chunk);
          return rest.length === 0 ? 0 : drain(ptr, cap);
        });
        return typeof answer?.then === "function"
          ? answer.then(undefined, failed)
          : answer;
      } catch (error) {
        return failed(error);
      }
    };
  };

  // What a read import left over, thrown away on demand. A host calls it when
  // the SOURCE behind that import moves under it INSIDE one call -- a new
  // upstream reply, say -- since the remainder is held above and nothing else
  // can see the source move. With no argument it drops every one of them.
  const drop = (key) =>
    key === undefined ? readers.forEach((f) => f()) : readers.get(key)?.();

  const imports = {
    env: {
      // () -> :bytes
      readRequestBody: bind("env", "readRequestBody", (what, call) => {
        const read = reader(what, call);
        return (ptr, cap) => read([], ptr, cap);
      }),
      // (:bytes) -> :void
      writeResponseBody: bind("env", "writeResponseBody", (what, call) => {
        return (p0, p0Len) =>
          settle(what, call(readBytes(p0, p0Len)), () => undefined);
      }),
    },
  };
  const instance = new WebAssembly.Instance(module, imports);
  exports = instance.exports;
  // Entropy, BEFORE the top level runs: a --no-wasi module imports none, so its
  // `random` starts from a constant and every instance would draw one sequence.
  // Seeding here also covers the draws a library makes while it LOADS. A Worker
  // forbids this in global scope, so instantiate on the first request.
  exports.__ronto_seed_random(
    new BigUint64Array(crypto.getRandomValues(new Uint8Array(8)).buffer)[0],
  );
  // And a clock, nanoseconds since the Unix epoch, for the same reason and
  // before the same line: until one is set the clock built-ins signal rather
  // than report 1970.
  exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
  exports._initialize();

  // The entry points a call chain can reach a suspending import from -- the
  // list the build prints -- entered through WebAssembly.promising exactly when
  // the host marked one. An unmarked host never pays for the promise.
  const suspends = marked.size !== 0;
  const entry$handleRequest = suspends
    ? WebAssembly.promising(exports["handle-request"])
    : exports["handle-request"];

  // One Lisp call at a time. A suspended call returns to the host's event loop,
  // and the module's own re-entry guard TRAPS a second entry rather than let two
  // calls share its allocator and its dynamic bindings -- so the queue is the
  // contract, not a nicety. `.then(work, work)` because one rejected call must
  // not wedge the chain behind it.
  let queue = Promise.resolve();
  const queued = (work) => {
    const done = queue.then(work, work);
    queue = done.then(
      () => {},
      () => {},
    );
    return done;
  };
  // A bare entry point only needs the queue when a host marked something: a
  // synchronous call cannot be interleaved, and paying a promise for it would
  // make every host asynchronous. `serially` below always takes it, because
  // the work it runs awaits and a second request WOULD land inside it.
  const serialised = (work) => (suspends ? queued(work) : work());

  // One call into the module: stage the arguments, enter, decode the result out
  // of the scratch it sits in, and only THEN pop the arena that scratch is in.
  // A promising entry answers a promise, so the tail rides `then`; a synchronous
  // one runs the same expression inline. `run` is how the call reaches the
  // module -- through the queue, or straight through when it is already inside
  // the one call the queue admits.
  const call = (run, entry, stage, decode) => {
    const work = () => {
      readers.forEach((drop) => drop());
      // The module's clock moves only when the host moves it, so move it per
      // call. Not a workaround for a frozen clock -- a Worker's own Date.now()
      // is frozen for the duration of a request as a timing-attack mitigation,
      // so a value that changes once per call is what the platform has.
      exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
      const mark = exports.__ronto_alloc_mark();
      const done = (raw) => {
        const value = decode(raw);
        exports.__ronto_alloc_reset(mark);
        return value;
      };
      // A TRAP skips the pop above and leaves the module's state half-written
      // with it, so a host that keeps serving instantiates again rather than
      // calling back into this instance.
      const raw = entry(...stage());
      return typeof raw?.then === "function" ? raw.then(done) : done(raw);
    };
    return run(work);
  };

  /** `handle-request` -- (:string) -> :string */
  const make$handleRequest =
    (run) =>
    (p0) => {
      return call(
        run,
        entry$handleRequest,
        () => {
          const a0 = writeString(p0);
          return [a0[0], a0[1]];
        },
        ([ptr, len]) => readString(ptr, len),
      );
    };


  // Host state that belongs to ONE call -- what the module pulls DURING it,
  // and what the call leaves behind -- is set and read inside `work`, which
  // runs in the same critical section: a suspended call returns to the event
  // loop, so setting it beside the call instead would let the next request
  // move it under this one. The entry points `work` is handed enter the module
  // directly, because the queue they would take is the one they are in.
  const inside = {
    handleRequest: make$handleRequest((work) => work()),
  };
  const serially = (work) => queued(() => work(inside));

  return {
    exports,
    handleRequest: make$handleRequest(serialised),
    drop,
    serially,
  };
}

/**
 * This module as a fetch handler -- the whole Worker, with nothing left over:
 *
 *   import module from "./worker.wasm";
 *   import { worker } from "./worker.js";
 *
 *   export default worker(module);
 *
 * A Request becomes the envelope the module's entry point takes, the head it
 * answers becomes a Response, and the instance is created on the FIRST REQUEST
 * (a Worker forbids drawing entropy in global scope) and retired if a call ever
 * traps.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @param {object} [options] `host` -- one plain function per import, keyed by
 *   module and field, as `instantiate` takes it; `remoteAddr` -- (request, env, ctx) => the client
 *   address, since which header carries it is the platform's business and not
 *   this file's (`(r) => r.headers.get("cf-connecting-ip")` on Cloudflare)
 * @returns {object} `{ fetch(request, env, ctx) }`
 */
export function worker(module, options = {}) {
  let instance = null;
  // Set when a call TRAPPED: that instance skipped its arena reset and its Lisp
  // state may be half-written, so nothing else may run on it. A Lisp ERROR is not
  // this -- the transport answers 500 itself and the instance is fine.
  let poisoned = false;
  // The request body the module pulls, and the response body coming back the
  // same way. Both belong to the ONE call running below, which is where they
  // are set.
  let requestBody = null;
  let responseChunks = [];
  const collected = () => {
    const all = new Uint8Array(
      responseChunks.reduce((n, chunk) => n + chunk.length, 0),
    );
    let at = 0;
    for (const chunk of responseChunks) {
      all.set(chunk, at);
      at += chunk.length;
    }
    return all;
  };
  const base = {};
  base.env = {
    ...(base.env ?? {}),
    readRequestBody: () => {
      // Handed over ONCE: a chunk source that never answers null is one the
      // module pulls forever.
      const chunk = requestBody;
      requestBody = null;
      return chunk;
    },
    writeResponseBody: (chunk) => responseChunks.push(chunk),
  };
  const given = options.host ?? {};
  const host = {};
  for (const key of new Set([...Object.keys(base), ...Object.keys(given)])) {
    host[key] = { ...(base[key] ?? {}), ...(given[key] ?? {}) };
  }
  const live = () => {
    if (poisoned) {
      instance = null;
      poisoned = false;
    }
    return (instance ??= instantiate(module, host));
  };

  // The request head. `target` stays RAW -- path and query still joined and
  // still percent-encoded -- because the shared normalizer on the other side
  // owns that split, and a pre-split path leaves the query string nil.
  const envelope = (request, octets, remoteAddr) => {
    const url = new URL(request.url);
    const headers = Object.fromEntries(request.headers);
    // A body with no content-length is a body the request parser does not read,
    // and a chunked request carries none -- so set it from the octets we have.
    if (octets?.length) headers["content-length"] = String(octets.length);
    const head = {
      method: request.method,
      target: url.pathname + url.search,
      headers,
      scheme: url.protocol.replace(":", ""),
    };
    if (remoteAddr != null) head["remote-addr"] = remoteAddr;
    return JSON.stringify(head);
  };

  return {
    // EVERYTHING is inside the try, not just the module call: reading an
    // aborted upload rejects, and `new Response` throws on a status or a
    // header an application is free to produce (0, 999, a newline in a
    // value). Outside it those escape as an unhandled rejection, which the
    // platform answers with its own error page and nothing in the log.
    async fetch(request, env, ctx) {
      let entered = false;
      try {
        const remoteAddr = await options.remoteAddr?.(request, env, ctx);
        const octets = request.body
          ? new Uint8Array(await request.arrayBuffer())
          : null;
        const input = envelope(request, octets, remoteAddr);
        const head = JSON.parse(
          await live().serially((lisp) => {
            // Re-read INSIDE the critical section: the instance was bound at
            // admission, and a call parked ahead of this one can poison it
            // before this one runs. Refusing is the whole point -- a
            // half-unwound instance answers wrong rather than failing, and
            // the module's own re-entry guard is cleared by the landing pad
            // on exactly the path that poisons it.
            if (poisoned) throw new Error("instance discarded by an earlier trap");
            // Per-call state, set HERE and not beside the call: a suspended
            // handler returns to the event loop, so the next request would
            // otherwise move it under this one.
            requestBody = octets;
            responseChunks = [];
            entered = true;
            return lisp.handleRequest(input);
          }),
        );
        // Headers as an ARRAY of pairs, which keeps two Set-Cookie two.
        // An EMPTY body becomes null whichever shape it arrived in: 204/205/304
        // may only be constructed with a null body, and "" and a zero-length
        // Uint8Array are the same response as none.
        const body = head.body ?? collected();
        return new Response(body?.length ? body : null, {
          status: head.status ?? 200,
          headers: head.headers ?? [],
        });
      } catch (error) {
        console.error("handle-request failed:", error);
        // Only a call that ENTERED the module can have left it half-written.
        // A mapping, or a Response the platform refused, says nothing about
        // the instance, and discarding it would cost the next request a
        // reinstantiation for someone else's bad header.
        if (entered) poisoned = true;
        return new Response("internal error\n", { status: 500 });
      }
    },
  };
}


---

# FILE: references/examples/cloudflare-workers/httpbin-ningle/worker.lisp

;;; ningle: a mini httpbin on an application OBJECT. Four things make it ningle
;;; rather than a second route table:
;;;
;;;   * a route is an ASSIGNMENT, so the five echo endpoints are a loop;
;;;   * a controller returns the BODY and says the rest by mutating
;;;     ningle:*response* -- the (status headers body) triple never appears;
;;;   * a controller receives the PARAMETERS, and by then lack/request has
;;;     decoded the query string and PARSED the body, so nothing here reads a
;;;     stream or parses JSON;
;;;   * declining means NOT MATCHING (returning nil answers an empty body), so
;;;     every miss lands on ningle:not-found, a METHOD on the application class.

(ql:quickload '("clack" "clack-handler-reactor" "ningle"))

(defvar *app* (make-instance 'ningle:app))

;;; --- answering ---------------------------------------------------------------

(defun respond-json (object)
  (setf (lack.response:response-headers ningle:*response*)
        (list :content-type "application/json"))
  (format nil "~a~%" (rontolisp:json-stringify object)))

(defun respond-text (text)
  (setf (lack.response:response-headers ningle:*response*)
        (list :content-type "text/plain; charset=utf-8"))
  (format nil "~a~%" text))

(defun set-status (code)
  (setf (lack.response:response-status ningle:*response*) code))

;;; --- the echo endpoint -------------------------------------------------------

;; ONE controller for every echo endpoint: per method there is nothing left to
;; do. `args` is the query string and `form` the parsed body -- for a JSON post
;; that is the JSON object itself -- and the alist ningle hands the controller
;; is those two appended, which is why this one ignores it.
(defun echo (params)
  (declare (ignore params))
  (let ((request ningle:*request*))
    (respond-json
     (rontolisp:plist-hash-table
      (list :method (symbol-name (lack.request:request-method request))
            :path (lack.request:request-path-info request)
            :args (rontolisp:alist-hash-table
                   (lack.request:request-query-parameters request))
            :form (rontolisp:alist-hash-table
                   (lack.request:request-body-parameters request))
            :headers (lack.request:request-headers request))))))

;;; --- the routes --------------------------------------------------------------

;; Rules are tried in the order they were assigned, so each path gets two: the
;; ONE method it answers, then :ANY for the 405. That leaves not-found with only
;; the answer it is really for.
(dolist (endpoint
         '(("/get" . :GET) ("/post" . :POST) ("/put" . :PUT) ("/patch" . :PATCH)
           ("/delete" . :DELETE)))
  (let ((path (car endpoint)) (allowed (cdr endpoint)))
    (setf (ningle:route *app* path :method allowed) #'echo)
    (setf (ningle:route *app* path :method :ANY)
          (lambda (params)
            (declare (ignore params))
            (set-status 405)
            (respond-json
             (rontolisp:plist-hash-table
              (list :error "method not allowed"
                    :allowed (symbol-name allowed))))))))

;; :ANY used as itself rather than as a fallback.
(setf (ningle:route *app* "/anything" :method :ANY) #'echo)

;; myway's other rule spelling: a REGEX, whose capture groups arrive as
;; :captures. It fits because a code that is not three digits then matches no
;; rule at all -- where a "/status/:code" template would match "/status/teapot"
;; and leave the controller with nothing good to answer.
(setf (ningle:route *app* "/status/([0-9]{3})" :regexp t)
      (lambda (params)
        (let ((code (parse-integer (first (cdr (assoc :captures params))))))
          (set-status code)
          (respond-text code))))

(defmethod ningle:not-found ((app ningle:app))
  (set-status 404)
  (respond-json
   (rontolisp:plist-hash-table
    (list :error "not found"
          :path (lack.request:request-path-info ningle:*request*)))))

(clack:clackup *app* :server :reactor :use-thread nil)


---

# FILE: references/examples/cloudflare-workers/httpbin-ningle/wrangler.jsonc

{
  // https://developers.cloudflare.com/workers/wrangler/configuration/
  "name": "rontolisp-httpbin-ningle",
  "main": "src/index.js",
  "compatibility_date": "2026-01-01",

  // `import module from "./worker.wasm"` needs no configuration: .wasm is a
  // built-in module rule for ES-module Workers, and the file is compiled at
  // deploy time rather than on the request path.
  "observability": {
    "enabled": true
  }
}


---

# FILE: references/examples/cloudflare-workers/httpbin-tiny-routes/README.md

# httpbin-tiny-routes — the same endpoints, composed

The five echo endpoints of [`../httpbin-clack`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin-clack) written the
way [tiny-routes](https://github.com/jeko2000/tiny-routes) wants them: route
macros instead of a `cond`, a `/status/:code` **path template**, the
route-decline protocol, and **middleware** for everything a handler would
otherwise do itself.

```bash
./build.sh          # worker.lisp -> src/worker.wasm
npx wrangler dev    # http://localhost:8787
npx wrangler deploy
```

```console
$ curl 'http://localhost:8787/get?a=1&b=two'
{"args":{"b":"two","a":"1"},"headers":{...},"method":"GET","path":"/get"}

$ curl http://localhost:8787/status/418
418

$ curl http://localhost:8787/status/teapot     # :code must parse -> the route
{"error":"not found","path":"/status/teapot"}  # declines into the 404
```

## Middleware does the work

No echo handler reads a stream, parses a query string or sets a header. `pipe`
threads the route table through the library's own middleware:

```lisp
(defparameter *app*
  (tiny:pipe *routes* (tiny:wrap-request-body) (tiny:wrap-query-parameters)))
```

so `(tiny:request-body req)` is the raw body as a string and
`(tiny:request-get req :query-parameters)` is the parsed query — and each route
group gets its content type the same way:

```lisp
(tiny:pipe *json-routes* (tiny:wrap-response-content-type "application/json"))
```

`/status/:code` answers `text/plain`, so it is a group of its own; it is also
the one route that names a status, which `tiny:make-response` takes.

`tiny` is the library's own nickname, so there is no `defpackage` in the file:
every name tiny-routes contributes is reachable qualified.

## Declining is the whole error story

Each echo endpoint is declared with the macro for the **one** method it answers,
so a wrong method claims nothing and falls through — and the single catch-all at
the bottom decides both of httpbin's error answers: a path that is one of the
five gives the 405 (naming the method that would have worked), any other path
the 404. One route, not one per endpoint.

`/status/:code` declines too, on a `:code` that is not a number, and the
catch-all has no entry for it — which is exactly the 404 httpbin answers for
`/status/teapot`.

`PATCH` has no macro in tiny-routes. Matching the method is the whole of what
those add over `define-any`, and the matcher is exported, so that one route is
spelled the way the macros expand:

```lisp
(tiny:wrap-request-matches-method
 (tiny:define-any "/patch" (req) (echo req t)) :patch)
```

| | |
| --- | --- |
| `GET /get` | echo the request: `args`, `headers`, `method`, `path` |
| `POST /post`, `PUT /put`, `PATCH /patch`, `DELETE /delete` | the same, plus `data` (the raw body) and `json` (its parsed value) |
| any of those, wrong method | 405 from the catch-all, naming the method that works |
| `GET /status/NNN` | answer with that status; a non-numeric `:code` declines |
| anything else | 404 from the catch-all |

`args` comes from tiny-routes' `parse-query-parameters`, which splits on `&`
and `=` and does not percent-decode — the library's own behaviour, and the one
visible difference from the neighbouring documents.

## `tiny-routes/lite`

The opt-in system: the same library with the path-template matcher swapped for a
ppcre-free one, described in
[`../hello-tiny-routes`](../hello-tiny-routes/README.md#tiny-routeslite-and-why-it-is-on-the-quickload-line).
This is where the choice is measured — the
[size report](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/size-report/results/cloudflare-workers.md) carries both
rows, built from this `worker.lisp` with only the `ql:quickload` line changed
(they answer the same probes byte for byte).

## Developing without Cloudflare

As in [`../httpbin-clack`](../httpbin-clack/README.md): the synthesized export
calls `clack.handler.reactor:dispatch`, an ordinary function, so the whole
Worker — routes included — runs on every backend:

```bash
rontolisp check.lisp
rontolisp check.lisp -o Check.class && java -cp rontolisp-0.1.0-SNAPSHOT-exec.jar:. Check
rontolisp check.lisp -o check.wasm --optimize && wasmtime run -W gc -W exceptions=y check.wasm
```

The first build downloads clack, lack and tiny-routes into
`~/.rontolisp/quicklisp`; after that everything is offline, because the
`ql:quickload` is resolved at **compile** time and inlined into the module.

## What's in here

| File | Purpose |
| --- | --- |
| [`worker.lisp`](worker.lisp) | **The whole program**: quickload, the handlers, the routes, the middleware, `clackup` |
| [`check.lisp`](check.lisp) | Drives it with no Cloudflare in sight — the local edit/run loop |
| [`src/index.js`](src/index.js) | Three lines over the generated `worker()`, plus the one hook it leaves to a caller — which header carries the client address |
| [`src/worker.js`](src/worker.js) | The boundary, GENERATED by `--emit-js-glue` from `worker.lisp`'s declarations. Byte-identical to `../httpbin-clack/src/worker.js`, because the declarations are. Do not edit; `./build.sh` rewrites it |
| `src/worker.wasm` | A build product — run `./build.sh` first |

## Limitations

The Worker sandbox and `--no-wasi` limitations of
[`../httpbin-clack`](../httpbin-clack/README.md#limitations) apply unchanged.
One more is this directory's own: the lite matcher refuses regex-shaped
templates at route-build time.


---

# FILE: references/examples/cloudflare-workers/httpbin-tiny-routes/build.sh

#!/usr/bin/env bash
# Compile worker.lisp -- clack, tiny-routes/lite, the routes, the middleware and
# the clackup call -- to the .wasm module the Worker imports.
#
# --no-wasi: the Worker calls the exported `handle-request` directly, it never
#   runs the module as a program, and the handler does no I/O -- so the module
#   needs no WASI imports at all. It becomes a reactor: nothing to shim on the
#   JavaScript side, and `_initialize` instead of `_start`.
# --optimize=size: a Worker bundle has a size limit, and the tree-shaker is what
#   keeps the module small. It matters most in THIS example: the routes go
#   through tiny-routes/lite, whose ppcre-free path-template matcher is what
#   keeps the whole cl-ppcre engine out (the full "tiny-routes" spells the same
#   routes and ships it -- both rows are in the size report).
#
# The first run downloads clack/lack/tiny-routes into ~/.rontolisp/quicklisp;
# after that the build is offline.
# --host-boundary=streaming: this Worker ECHOES request bodies, so they must
#   cross as octets rather than as JSON text in the envelope -- which is what
#   the generated src/worker.js feeds through env.readRequestBody /
#   env.writeResponseBody, and what lets a BINARY body come back exactly. Asked
#   for, because the default is `envelope` (see ../btc-ticker), where a body
#   rides the head instead.
# --emit-js-glue: write src/worker.js beside the module -- the host half of this
#   boundary, from the program's own declarations: the import object, the
#   (ptr, len) staging, the __ronto_alloc bracket, the two body imports above --
#   fed from the Request it is already holding and the Response it is already
#   building -- and the Request -> envelope -> Response mapping over them. That
#   half is derivable on THIS boundary too, so src/index.js is a worker(module)
#   call. It is CHECKED IN and pinned by HostGlueEmitterTest, so regenerate it
#   here rather than editing it.
#
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling worker.lisp -> src/worker.wasm + src/worker.js"
java -jar "$jar" "$here/worker.lisp" -o "$here/src/worker.wasm" \
  --no-wasi --host-boundary=streaming --optimize=size --emit-js-glue

ls -l "$here/src/worker.wasm" "$here/src/worker.js"
echo "done. Run it with:  npx wrangler dev"


---

# FILE: references/examples/cloudflare-workers/httpbin-tiny-routes/check.lisp

;;; Drive worker.lisp's handler without Cloudflare, on any backend
;;; (../httpbin-clack/check.lisp has the notes). The last two probes are the
;;; routed additions: the /status/:code template binding a parameter, and a
;;; non-numeric :code making that route DECLINE into the catch-all 404.

(load "worker.lisp")

(defun try (request-plist)
  (let ((request
         (rontolisp:json-stringify (rontolisp:plist-hash-table request-plist))))
    (format t "~&--> ~a~%" request)
    (format t "<-- ~a~%" (clack.handler.reactor:dispatch request))))

(defun headers (&rest plist) (rontolisp:plist-hash-table plist))

(defun json-headers (body)
  (headers :host "example.com"
           :content-type "application/json"
           :content-length (princ-to-string (length body))))

;; GET /get with a query string.
(try
 (list :method "GET"
       :target "/get?a=1&b=two"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (headers :host "example.com" :accept "application/json")
       :body ""))

;; POST /post with a JSON body -- "data" is the raw text, "json" the parsed
;; value, read off clack's :raw-body stream.
(try
 (list :method "POST"
       :target "/post"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (json-headers "{\"name\":\"rontolisp\"}")
       :body "{\"name\":\"rontolisp\"}"))

;; The wrong method for an endpoint -- the method-specific route declines and
;; the define-any right after it answers 405.
(try
 (list :method "GET"
       :target "/post"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (headers :host "example.com")
       :body ""))

;; An unknown path -- the catch-all 404.
(try
 (list :method "GET"
       :target "/nope"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (headers :host "example.com")
       :body ""))

;; The /status/:code path template -- :code binds "418".
(try
 (list :method "GET"
       :target "/status/418"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (headers :host "example.com")
       :body ""))

;; A :code that does not parse -- the route declines, 404.
(try
 (list :method "GET"
       :target "/status/teapot"
       :scheme "https"
       :remote-addr "203.0.113.7"
       :headers (headers :host "example.com")
       :body ""))


---

# FILE: references/examples/cloudflare-workers/httpbin-tiny-routes/package.json

{
  "name": "rontolisp-cloudflare-httpbin-tiny-routes",
  "version": "0.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "build": "./build.sh",
    "dev": "wrangler dev",
    "deploy": "wrangler deploy",
    "tail": "wrangler tail"
  },
  "devDependencies": {
    "wrangler": "^4"
  }
}


---

# FILE: references/examples/cloudflare-workers/httpbin-tiny-routes/src/index.js

// The whole Worker. `./worker.js` is generated from worker.lisp's own
// declarations (build.sh's --emit-js-glue) and owns all of it: the import
// object, the (ptr, len) staging, the __ronto_alloc bracket, the two body
// imports this boundary declares -- fed from the Request it is already holding
// and the Response it is already building -- and the Request -> envelope ->
// Response mapping over them.
//
// That the STREAMING boundary needs no more of a host than ../hello-clack's
// envelope one is the point of the pair: what the boundary buys is a binary
// body crossing exactly and a large one never doubling linear memory, not a
// bigger host. ../httpbin is the one Worker here that still writes its own,
// because it declares `handle-request` by hand and the compile path recognises
// the SYNTHESIZED bridge -- read that directory's src/index.js for what this
// file would otherwise say.
//
// BYTE-IDENTICAL in every httpbin-* directory that goes through clackup. A new
// sibling copies this file, it does not edit it.

import module from "./worker.wasm";
import { worker } from "./worker.js";

export default worker(module, {
  // Clack's :remote-addr. Which header carries the client address is the
  // platform's business and not the glue's, so it is the one thing worker()
  // leaves to its caller; Cloudflare puts it here. There is no peer port to
  // report, so :remote-port stays nil.
  remoteAddr: (request) => request.headers.get("cf-connecting-ip"),
});


---

# FILE: references/examples/cloudflare-workers/httpbin-tiny-routes/src/worker.js

// GENERATED by rontolisp --emit-js-glue -- do not edit.
//
// The host half of this module's boundary, derived from the program's own
// declarations: one import-object entry per rontolisp:wasm-import, one entry
// point per rontolisp:wasm-export, and every piece of linear-memory plumbing
// between them -- the (ptr, len) pair a :string crosses as, the __ronto_alloc
// bracket around a call, and the read(2) cursor a :bytes result is pulled
// through.
//
// What a declaration cannot state is what a host function DOES, so that is the
// one thing this file asks for: `host` is a plain function per import, keyed by
// import module and field, taking and answering ordinary JavaScript values --
// never a (ptr, len) pair, and, where an entry answers `chunk` below, a
// Uint8Array or a string with null for the end of them.
//
//   import { instantiate, suspending } from "./worker.js";
//
//   const lisp = instantiate(module, {
//     env: {
//       readRequestBody: () => chunk,   // or leave it to worker()
//       writeResponseBody: (chunk) => {},   // or leave it to worker()
//     },
//   });
//   await lisp.handleRequest(text);
//
// A host that suspends marks its own entries -- suspending(async (...) => ...)
// -- and every entry point above then answers a promise: the marked imports are
// wrapped in WebAssembly.Suspending, each entry point that can reach one is
// entered through WebAssembly.promising, and calls are serialised onto one
// promise chain, because a suspended module returns to the host's event loop
// and a re-entered export refuses with a trap rather than corrupting both
// calls. Host state that belongs to ONE such call is set inside that section:
//
//   await lisp.serially(async (entry) => { ...; return entry.handleRequest(...) });
//
// This module's boundary is the reactor envelope, so the Request/Response half
// is derivable too and `worker` below is it:
//
//   import module from "./worker.wasm";
//   import { worker } from "./worker.js";
//
//   export default worker(module);
//
// Regenerate it, never edit it: --emit-js-glue on the compile that wrote the
// .wasm beside it.

const encoder = new TextEncoder();
// ignoreBOM, because these octets are a VALUE and not a document: a
// leading U+FEFF is a character the other side chose, and the default
// decoder deletes it -- silently shortening a BOM-prefixed request body
// by one character while the content-length beside it still counts three.
const decoder = new TextDecoder("utf-8", { ignoreBOM: true });

const SUSPENDING = Symbol.for("rontolisp.suspending");

/**
 * Marks a host function as one that answers a promise, so the wasm stack parks
 * until it settles (JSPI). The wrapper is not free -- an import answering
 * SYNCHRONOUSLY through one still returns to the event loop -- so mark only the
 * entries this host really implements asynchronously.
 *
 * @param {Function} fn the host function
 * @returns {object} the marked entry, to be passed as the import
 */
export function suspending(fn) {
  return { [SUSPENDING]: fn };
}

/**
 * Instantiates the module against this host and returns its callable surface.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @param {object} host one function per import, keyed by module and field
 * @returns {object} `exports`, plus one entry point per rontolisp:wasm-export
 */
export function instantiate(module, host = {}) {
  let exports;

  // Every view is built where it is used, never held: growing the module's
  // memory DETACHES the buffer behind every view taken before the growth, and a
  // suspending host resumes after growths it never saw.
  const readString = (ptr, len) =>
    decoder.decode(new Uint8Array(exports.memory.buffer, ptr, len));

  // Octets, COPIED: the module pops the staging behind the pointer the moment
  // the call returns, so a chunk not taken by then is one the host never gets.
  const readBytes = (ptr, len) =>
    new Uint8Array(exports.memory.buffer.slice(ptr, ptr + len));

  // Bytes the HOST hands over live in the module's own bump allocator, and
  // cross as the (ptr, len) pair every memory-typed value is.
  const write = (octets) => {
    const ptr = exports.__ronto_alloc(octets.length);
    new Uint8Array(exports.memory.buffer, ptr, octets.length).set(octets);
    return [ptr, octets.length];
  };
  const writeString = (value) => write(encoder.encode(String(value)));
  const octets = (chunk) =>
    typeof chunk === "string" ? encoder.encode(chunk) : chunk;

  // A host answers a value, or -- from an entry it marked suspending -- a promise
  // of one. Both shapes ride one expression, which is what lets this file drive a
  // synchronous host and a JSPI host without being written twice.
  const marked = new Set();
  const unmarked = (what) =>
    what +
    " answered a promise; wrap it in suspending() so the wasm stack parks" +
    " until it settles";
  const settle = (what, value, next) => {
    if (typeof value?.then === "function") {
      if (!marked.has(what)) {
        // Reported before it is thrown: this throw crosses back into wasm,
        // where a catch_all landing pad turns it into whatever the module
        // makes of a failed import.
        console.error(unmarked(what));
        throw new TypeError(unmarked(what));
      }
      return value.then(next);
    }
    return next(value);
  };

  // The entries the module really imports: --optimize shakes out an import the
  // program never calls, and a host should not have to know which survived.
  const linked = new Set(
    WebAssembly.Module.imports(module).map((i) => i.module + "." + i.name),
  );
  const bind = (moduleName, field, wrap) => {
    const key = moduleName + "." + field;
    const given = (host[moduleName] ?? {})[field];
    if (given == null) {
      if (!linked.has(key)) return undefined;
      throw new TypeError("host." + key + " is missing; this module imports it");
    }
    if (given[SUSPENDING] === undefined) {
      // An async function that was never marked is the one mistake worth
      // catching HERE, where the report is a plain stack and not whatever
      // the module makes of an import that threw.
      if (given.constructor?.name === "AsyncFunction") {
        throw new TypeError(unmarked("host." + key));
      }
      return wrap(key, given);
    }
    marked.add(key);
    return new WebAssembly.Suspending(wrap(key, given[SUSPENDING]));
  };

  // A :bytes RESULT is the read(2) shape: the MODULE owns the buffer and asks
  // for up to `cap` octets, so what a host answers is the next CHUNK and this
  // holds whatever did not fit. That remainder is the read side's only state,
  // and it is why a host supplies chunks rather than a reader -- which source
  // they come from (a ReadableStream, a Uint8Array) is all that is left to it.
  const readers = new Map();
  const reader = (what, source) => {
    let rest = null;
    let from = null;
    // A body the module did not drain belongs to the call that could have and
    // to no other, so every cursor is dropped at the next entry below -- and a
    // host whose SOURCE moves inside one call (a new upstream reply opened by
    // another import) drops this one itself with lisp.drop(key), because what
    // did not fit is held here and nothing else can see the source move.
    readers.set(what, () => {
      rest = null;
      from = null;
    });
    const drain = (ptr, cap) => {
      const n = Math.min(cap, rest.length);
      new Uint8Array(exports.memory.buffer, ptr, n).set(rest.subarray(0, n));
      rest = rest.subarray(n);
      return n;
    };
    // A read that FAILS answers a NEGATIVE count. Throwing would trap the
    // instance; the count is an error channel the module turns into a Lisp
    // condition where the octets are consumed, which is where every other
    // backend reports a transfer that broke mid-body.
    const failed = (error) => {
      console.error(what + " failed:", error);
      return -1;
    };
    return (args, ptr, cap) => {
      // The remainder belongs to the arguments that asked for it: a source
      // selected by argument must not be served the previous one's octets.
      const key = JSON.stringify(args);
      if (from !== key) {
        rest = null;
        from = key;
      }
      if (rest !== null && rest.length !== 0) return drain(ptr, cap);
      try {
        const answer = settle(what, source(...args), (chunk) => {
          rest = chunk == null ? new Uint8Array(0) : octets(chunk);
          return rest.length === 0 ? 0 : drain(ptr, cap);
        });
        return typeof answer?.then === "function"
          ? answer.then(undefined, failed)
          : answer;
      } catch (error) {
        return failed(error);
      }
    };
  };

  // What a read import left over, thrown away on demand. A host calls it when
  // the SOURCE behind that import moves under it INSIDE one call -- a new
  // upstream reply, say -- since the remainder is held above and nothing else
  // can see the source move. With no argument it drops every one of them.
  const drop = (key) =>
    key === undefined ? readers.forEach((f) => f()) : readers.get(key)?.();

  const imports = {
    env: {
      // () -> :bytes
      readRequestBody: bind("env", "readRequestBody", (what, call) => {
        const read = reader(what, call);
        return (ptr, cap) => read([], ptr, cap);
      }),
      // (:bytes) -> :void
      writeResponseBody: bind("env", "writeResponseBody", (what, call) => {
        return (p0, p0Len) =>
          settle(what, call(readBytes(p0, p0Len)), () => undefined);
      }),
    },
  };
  const instance = new WebAssembly.Instance(module, imports);
  exports = instance.exports;
  // Entropy, BEFORE the top level runs: a --no-wasi module imports none, so its
  // `random` starts from a constant and every instance would draw one sequence.
  // Seeding here also covers the draws a library makes while it LOADS. A Worker
  // forbids this in global scope, so instantiate on the first request.
  exports.__ronto_seed_random(
    new BigUint64Array(crypto.getRandomValues(new Uint8Array(8)).buffer)[0],
  );
  // And a clock, nanoseconds since the Unix epoch, for the same reason and
  // before the same line: until one is set the clock built-ins signal rather
  // than report 1970.
  exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
  exports._initialize();

  // The entry points a call chain can reach a suspending import from -- the
  // list the build prints -- entered through WebAssembly.promising exactly when
  // the host marked one. An unmarked host never pays for the promise.
  const suspends = marked.size !== 0;
  const entry$handleRequest = suspends
    ? WebAssembly.promising(exports["handle-request"])
    : exports["handle-request"];

  // One Lisp call at a time. A suspended call returns to the host's event loop,
  // and the module's own re-entry guard TRAPS a second entry rather than let two
  // calls share its allocator and its dynamic bindings -- so the queue is the
  // contract, not a nicety. `.then(work, work)` because one rejected call must
  // not wedge the chain behind it.
  let queue = Promise.resolve();
  const queued = (work) => {
    const done = queue.then(work, work);
    queue = done.then(
      () => {},
      () => {},
    );
    return done;
  };
  // A bare entry point only needs the queue when a host marked something: a
  // synchronous call cannot be interleaved, and paying a promise for it would
  // make every host asynchronous. `serially` below always takes it, because
  // the work it runs awaits and a second request WOULD land inside it.
  const serialised = (work) => (suspends ? queued(work) : work());

  // One call into the module: stage the arguments, enter, decode the result out
  // of the scratch it sits in, and only THEN pop the arena that scratch is in.
  // A promising entry answers a promise, so the tail rides `then`; a synchronous
  // one runs the same expression inline. `run` is how the call reaches the
  // module -- through the queue, or straight through when it is already inside
  // the one call the queue admits.
  const call = (run, entry, stage, decode) => {
    const work = () => {
      readers.forEach((drop) => drop());
      // The module's clock moves only when the host moves it, so move it per
      // call. Not a workaround for a frozen clock -- a Worker's own Date.now()
      // is frozen for the duration of a request as a timing-attack mitigation,
      // so a value that changes once per call is what the platform has.
      exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
      const mark = exports.__ronto_alloc_mark();
      const done = (raw) => {
        const value = decode(raw);
        exports.__ronto_alloc_reset(mark);
        return value;
      };
      // A TRAP skips the pop above and leaves the module's state half-written
      // with it, so a host that keeps serving instantiates again rather than
      // calling back into this instance.
      const raw = entry(...stage());
      return typeof raw?.then === "function" ? raw.then(done) : done(raw);
    };
    return run(work);
  };

  /** `handle-request` -- (:string) -> :string */
  const make$handleRequest =
    (run) =>
    (p0) => {
      return call(
        run,
        entry$handleRequest,
        () => {
          const a0 = writeString(p0);
          return [a0[0], a0[1]];
        },
        ([ptr, len]) => readString(ptr, len),
      );
    };


  // Host state that belongs to ONE call -- what the module pulls DURING it,
  // and what the call leaves behind -- is set and read inside `work`, which
  // runs in the same critical section: a suspended call returns to the event
  // loop, so setting it beside the call instead would let the next request
  // move it under this one. The entry points `work` is handed enter the module
  // directly, because the queue they would take is the one they are in.
  const inside = {
    handleRequest: make$handleRequest((work) => work()),
  };
  const serially = (work) => queued(() => work(inside));

  return {
    exports,
    handleRequest: make$handleRequest(serialised),
    drop,
    serially,
  };
}

/**
 * This module as a fetch handler -- the whole Worker, with nothing left over:
 *
 *   import module from "./worker.wasm";
 *   import { worker } from "./worker.js";
 *
 *   export default worker(module);
 *
 * A Request becomes the envelope the module's entry point takes, the head it
 * answers becomes a Response, and the instance is created on the FIRST REQUEST
 * (a Worker forbids drawing entropy in global scope) and retired if a call ever
 * traps.
 *
 * @param {WebAssembly.Module} module the compiled module
 * @param {object} [options] `host` -- one plain function per import, keyed by
 *   module and field, as `instantiate` takes it; `remoteAddr` -- (request, env, ctx) => the client
 *   address, since which header carries it is the platform's business and not
 *   this file's (`(r) => r.headers.get("cf-connecting-ip")` on Cloudflare)
 * @returns {object} `{ fetch(request, env, ctx) }`
 */
export function worker(module, options = {}) {
  let instance = null;
  // Set when a call TRAPPED: that instance skipped its arena reset and its Lisp
  // state may be half-written, so nothing else may run on it. A Lisp ERROR is not
  // this -- the transport answers 500 itself and the instance is fine.
  let poisoned = false;
  // The request body the module pulls, and the response body coming back the
  // same way. Both belong to the ONE call running below, which is where they
  // are set.
  let requestBody = null;
  let responseChunks = [];
  const collected = () => {
    const all = new Uint8Array(
      responseChunks.reduce((n, chunk) => n + chunk.length, 0),
    );
    let at = 0;
    for (const chunk of responseChunks) {
      all.set(chunk, at);
      at += chunk.length;
    }
    return all;
  };
  const base = {};
  base.env = {
    ...(base.env ?? {}),
    readRequestBody: () => {
      // Handed over ONCE: a chunk source that never answers null is one the
      // module pulls forever.
      const chunk = requestBody;
      requestBody = null;
      return chunk;
    },
    writeResponseBody: (chunk) => responseChunks.push(chunk),
  };
  const given = options.host ?? {};
  const host = {};
  for (const key of new Set([...Object.keys(base), ...Object.keys(given)])) {
    host[key] = { ...(base[key] ?? {}), ...(given[key] ?? {}) };
  }
  const live = () => {
    if (poisoned) {
      instance = null;
      poisoned = false;
    }
    return (instance ??= instantiate(module, host));
  };

  // The request head. `target` stays RAW -- path and query still joined and
  // still percent-encoded -- because the shared normalizer on the other side
  // owns that split, and a pre-split path leaves the query string nil.
  const envelope = (request, octets, remoteAddr) => {
    const url = new URL(request.url);
    const headers = Object.fromEntries(request.headers);
    // A body with no content-length is a body the request parser does not read,
    // and a chunked request carries none -- so set it from the octets we have.
    if (octets?.length) headers["content-length"] = String(octets.length);
    const head = {
      method: request.method,
      target: url.pathname + url.search,
      headers,
      scheme: url.protocol.replace(":", ""),
    };
    if (remoteAddr != null) head["remote-addr"] = remoteAddr;
    return JSON.stringify(head);
  };

  return {
    // EVERYTHING is inside the try, not just the module call: reading an
    // aborted upload rejects, and `new Response` throws on a status or a
    // header an application is free to produce (0, 999, a newline in a
    // value). Outside it those escape as an unhandled rejection, which the
    // platform answers with its own error page and nothing in the log.
    async fetch(request, env, ctx) {
      let entered = false;
      try {
        const remoteAddr = await options.remoteAddr?.(request, env, ctx);
        const octets = request.body
          ? new Uint8Array(await request.arrayBuffer())
          : null;
        const input = envelope(request, octets, remoteAddr);
        const head = JSON.parse(
          await live().serially((lisp) => {
            // Re-read INSIDE the critical section: the instance was bound at
            // admission, and a call parked ahead of this one can poison it
            // before this one runs. Refusing is the whole point -- a
            // half-unwound instance answers wrong rather than failing, and
            // the module's own re-entry guard is cleared by the landing pad
            // on exactly the path that poisons it.
            if (poisoned) throw new Error("instance discarded by an earlier trap");
            // Per-call state, set HERE and not beside the call: a suspended
            // handler returns to the event loop, so the next request would
            // otherwise move it under this one.
            requestBody = octets;
            responseChunks = [];
            entered = true;
            return lisp.handleRequest(input);
          }),
        );
        // Headers as an ARRAY of pairs, which keeps two Set-Cookie two.
        // An EMPTY body becomes null whichever shape it arrived in: 204/205/304
        // may only be constructed with a null body, and "" and a zero-length
        // Uint8Array are the same response as none.
        const body = head.body ?? collected();
        return new Response(body?.length ? body : null, {
          status: head.status ?? 200,
          headers: head.headers ?? [],
        });
      } catch (error) {
        console.error("handle-request failed:", error);
        // Only a call that ENTERED the module can have left it half-written.
        // A mapping, or a Response the platform refused, says nothing about
        // the instance, and discarding it would cost the next request a
        // reinstantiation for someone else's bad header.
        if (entered) poisoned = true;
        return new Response("internal error\n", { status: 500 });
      }
    },
  };
}


---

# FILE: references/examples/cloudflare-workers/httpbin-tiny-routes/worker.lisp

;;; tiny-routes: a mini httpbin COMPOSED out of routes and middleware. `pipe`
;;; threads the route table through wrap-request-body (the raw body, read) and
;;; wrap-query-parameters (the query string, parsed), and the JSON group through
;;; wrap-response-content-type -- so the echo handlers neither drain a stream nor
;;; set a header. A route answers or returns nil to DECLINE, which is how a wrong
;;; method reaches the catch-all and how /status/:code refuses a bad code.
;;; `tiny` is the library's own nickname, so nothing has to be imported.
;;;
;;; "tiny-routes/lite" is the opt-in system whose ppcre-free path-template
;;; matcher keeps the regex engine out of the module.

(ql:quickload '("clack" "clack-handler-reactor" "tiny-routes/lite"))

;;; --- the handlers ------------------------------------------------------------

(defun body-json (body)
  (if (and (stringp body) (> (length body) 0)
           (or (eql (char body 0) #\{) (eql (char body 0) #\[)))
      (handler-case (rontolisp:json-parse body) (error () 'null))
      'null))

(defun json (object) (format nil "~a~%" (rontolisp:json-stringify object)))

;; The echo document. Everything it reports is already on the request, so this
;; only shapes JSON: plist-hash-table turns tiny-routes' query plist and this
;; plist into the string-keyed hash tables json-stringify renders as objects.
(defun echo (req with-body)
  (let ((info
         (rontolisp:plist-hash-table
          (list :args (rontolisp:plist-hash-table
                       (tiny:request-get req :query-parameters))
                :headers (tiny:request-headers req)
                :method (symbol-name (tiny:request-method req))
                :path (tiny:path-info req)))))
    (when with-body
      (let ((body (tiny:request-body req "")))
        (setf (gethash "data" info) body)
        (setf (gethash "json" info) (body-json body))))
    (tiny:ok (json info))))

;; Nothing claimed the request. The table is not a second dispatch: it is what
;; tells a known path that DECLINED on its method (405, naming the one that
;; works) from a path no route has (404).
(defparameter *endpoints*
  '(("/get" . :GET) ("/post" . :POST) ("/put" . :PUT) ("/patch" . :PATCH)
    ("/delete" . :DELETE)))

(defun no-route (req)
  (let* ((path (tiny:path-info req))
         (allowed (cdr (assoc path *endpoints* :test #'string=))))
    (if allowed
        (tiny:method-not-allowed
         (json
          (rontolisp:plist-hash-table
           (list :error "method not allowed" :allowed (symbol-name allowed)))))
        (tiny:not-found
         (json
          (rontolisp:plist-hash-table (list :error "not found" :path path)))))))

;;; --- the routes --------------------------------------------------------------

;; Each route answers the one method it names and declines every other, so the
;; catch-all at the bottom is reached by both a wrong method and an unknown path.
(tiny:define-routes *json-routes*
  (tiny:define-get "/get" (req) (echo req nil))
  (tiny:define-post "/post" (req) (echo req t))
  (tiny:define-put "/put" (req) (echo req t))
  ;; tiny-routes has no define-patch; matching the method is all the other
  ;; macros add over define-any, and that matcher is exported.
  (tiny:wrap-request-matches-method
   (tiny:define-any "/patch" (req) (echo req t)) :patch)
  (tiny:define-delete "/delete" (req) (echo req t))
  (tiny:define-any "*" (req) (no-route req)))

;; The one endpoint that does not answer JSON, so it is its own group with its
;; own content type. A :code that is not a number declines.
(defparameter *status-route*
  (tiny:pipe (tiny:define-get "/status/:code" (req)
               (let ((code
                      (parse-integer (tiny:path-parameter req :code)
                                     :junk-allowed t)))
                 (when code
                   (tiny:make-response :status code
                                       :body (format nil "~a~%" code)))))
             (tiny:wrap-response-content-type "text/plain; charset=utf-8")))

(tiny:define-routes *routes*
  *status-route*
  (tiny:pipe *json-routes*
             (tiny:wrap-response-content-type "application/json")))

(defparameter *app*
  (tiny:pipe *routes* (tiny:wrap-request-body) (tiny:wrap-query-parameters)))

(clack:clackup *app* :server :reactor :use-thread nil)


---

# FILE: references/examples/cloudflare-workers/httpbin-tiny-routes/wrangler.jsonc

{
  // https://developers.cloudflare.com/workers/wrangler/configuration/
  "name": "rontolisp-httpbin-tiny-routes",
  "main": "src/index.js",
  "compatibility_date": "2026-01-01",

  // `import module from "./worker.wasm"` needs no configuration: .wasm is a
  // built-in module rule for ES-module Workers, and the file is compiled at
  // deploy time rather than on the request path.
  "observability": {
    "enabled": true
  }
}


---

# FILE: references/examples/cloudflare-workers/httpbin/README.md

# httpbin — a mini httpbin on Cloudflare Workers, with no library

The five echo endpoints of [httpbin.org](https://httpbin.org)
([`worker.lisp`](worker.lisp)), compiled to WebAssembly and served from a
Worker — with the adapter that puts it there **written out by hand** instead of
installed by a library, so nothing but the program ships.

```bash
./build.sh          # worker.lisp -> src/worker.wasm
npx wrangler dev    # http://localhost:8787
npx wrangler deploy
```

```console
$ curl 'http://localhost:8787/get?a=1&b=two'
{"args":{"a":"1","b":"two"},"headers":{"host":"localhost:8787",...},"method":"GET","path":"/get"}

$ curl -X POST -d '{"name":"rontolisp"}' http://localhost:8787/post
{"args":{},...,"data":"{\"name\":\"rontolisp\"}","json":{"name":"rontolisp"}}
```

## The endpoints

`GET /get` echoes `args`, `headers`, `method` and `path`; `POST /post`,
`PUT /put`, `PATCH /patch` and `DELETE /delete` add `data` (the raw body) and
`json` (its parsed value). A wrong method answers **405** with the one it
wanted, an unknown path **404**, and a body that does not parse leaves
`"json": null` — the real httpbin's behaviour.

```bash
curl -X POST -d '{not json'  http://localhost:8787/post   # "json":null
curl         http://localhost:8787/post                   # 405 {"allowed":"POST",...}
curl         http://localhost:8787/nope                   # 404
```

## What's in here

| File | Purpose |
| --- | --- |
| [`worker.lisp`](worker.lisp) | **The whole program**: the endpoints plus the reactor adapter |
| [`check.lisp`](check.lisp) | Drives it with no Cloudflare in sight — the local edit/run loop, and a [rove](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/doc/en/guides/testing.md) suite: every answer it expects is an assertion, and the run's exit code is the verdict |
| [`src/index.js`](src/index.js) | The whole Worker: `Request` -> JSON -> Lisp -> JSON -> `Response`. The one **hand-written** host left in these directories — see below |
| `src/worker.wasm` | A build product — run `./build.sh` first |

## The interface is one exported function

A Worker hands over a request JavaScript has already parsed rather than a
socket, so there is no server to run. `worker.lisp` declares a host-callable
export instead:

```lisp
(rontolisp:wasm-export 'handle-request :params '(:string) :returns :string)
```

As in [`../hello`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/hello) a `:string` crosses as UTF-8 bytes in linear
memory, but here one crosses *in*, which brings the allocator into the picture:
the module exports `memory`, `__ronto_alloc`, the arena pair
`__ronto_alloc_mark`/`__ronto_alloc_reset`, and
`handle-request(ptr, len) -> [ptr, len]`.

The adapter under the endpoints is what `clack:clackup :server :reactor` would
have installed, in about thirty lines — and it converts nothing itself.
rontolisp's server protocol *is* Clack's, so there is one implementation of it
([`http-server.lisp`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/src/main/resources/am/ik/rontolisp/eval/http-server.lisp)),
and the adapter calls the two entry points every transport meets in:

```lisp
(rontolisp::%http-make-env raw)          ; positional raw tuple -> the Clack environment
(rontolisp::%http-normalize-response r)  ; whatever the handler returned -> (status header-alist body-string)
```

The percent-decoding, the `?` split, the header lowercasing, the `Host` split,
the `content-length` parsing, the `:raw-body` stream and the response normalizer
therefore come for free and **cannot drift from what a served request sees**.
All that is left to write is the JSON envelope. Hand the same job to a library
instead and you get [`../httpbin-clack`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin-clack), which declares no
export at all: `wasm-export` needs a **literal** name at compile time, so the
compiler synthesizes one from a marker the handler backend leaves behind.

## The envelope, and two fields the JavaScript side must get right

The request and response are JSON because the other side is JavaScript. Out —
this is the request **head**; the body does not ride in it, see the section
after this one:

```json
{ "method": "GET", "target": "/path?a=1", "headers": {"host": "..."},
  "scheme": "https", "remote-addr": "203.0.113.7" }
```

back — the response **head**, whose `"body"` key is likewise absent whenever the
body crossed out of band (which is the normal case here; see the section after
this one):

```json
{ "status": 200, "headers": [["content-type", "application/json"]] }
```

This is the envelope the built-in `clack-handler-reactor` backend speaks, and
the reason every other Worker here gets its host **generated**: mapping a
`Request` onto it and a `Response` off it is transport work, fixed by the
envelope rather than chosen by the program, so `--emit-js-glue` writes it (see
[`../httpbin-clack`](../httpbin-clack/README.md#and-the-javascript-half-is-not-hand-written-either)).
This directory is the exception, and on purpose: it declares
`(rontolisp:wasm-export 'handle-request ...)` **by hand**, the compile path
recognises the *synthesized* bridge rather than the export name, and no
`worker()` is emitted for a hand-written one. Which is a fair thing for the "no
library, boundary included" example to be — what the generated file would say is
written out here instead. Two fields are load-bearing, and both fail quietly:

- **Pass the raw target** (`url.pathname + url.search`) as one string, not a
  pre-split path plus a query object. `%http-make-env` does the `?` split and
  the percent-decoding itself; send a pre-split path and the application gets a
  `:query-string` of `nil`.
- **Forward `content-length`.** `%http-make-env` reads it off the header table
  and body parsing returns nothing without it — while a chunked request carries
  none. `src/index.js` sets it from the bytes it just read rather than copying
  the incoming header.

On the way out, response `headers` are an **array of pairs, not an object**:
`%http-normalize-response` answers an alist in which a name may repeat (two
cookies, two `Set-Cookie` headers). An object would collapse the duplicates.

## Neither body is in the envelope

They cross the other way instead, through the two imports this module has:

```js
readRequestBody(ptr, cap) -> n   // write up to cap octets at ptr, answer how
                                 // many; 0 is end of stream
writeResponseBody(ptr, len)      // take these octets, they are the next chunk
```

Note the direction flip. Going *in*, the module owns the buffer and hands it
over per call, reusing one for every chunk of every request; going *out*, the
octets are the module's own and the host must copy them before the call returns.
Both say the same thing — the caller owns the memory, so JavaScript may never
hold on to a pointer — and the write import answers nothing, because a host
cannot short-read a write.

Two things follow that a JSON string could not give: a **binary** body crosses
exactly in either direction (the `:string` boundary decodes UTF-8 and does not
validate, so arbitrary octets come back as garbage code points), and a large
upload or download costs the module **no linear memory** — the envelope used to
hold the body several times over, and a 256 KiB `POST` the handler drops now
leaves `memory.buffer.byteLength` exactly where it was.

Both imports are declared `:async t`, which says the host *may* suspend.
`src/index.js` answers synchronously — it reads the body first, then calls in,
and collects the response chunks as they arrive — which the declaration allows
and which needs nothing from the platform. A host that instead wraps an import in
`WebAssembly.Suspending` streams the upload straight from `request.body`'s
reader, or writes the response into a stream that applies backpressure, at the
price of entering `handle-request` through `WebAssembly.promising` and
serialising its calls (see [`../dog-fetcher`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/dog-fetcher), which does that
for its outgoing `fetch`).

The guard on both imports in `worker.lisp` is `#+rontolisp-body-imports` — the
feature a build carries exactly where those two imports exist, which is a
`--no-wasi` wasm-GC core module built with **`--host-boundary=streaming`** —
which this directory's `build.sh` asks for, the default being the in-band
envelope. Every other way of running this same file lacks them, and the `#-`
half answers all of them with the envelope's own `"body"` key:

- **the interpreter and the JVM** — `check.lisp` drives `handle-request` as an
  ordinary function, where there is no JavaScript at all.
- **a plain WASI command module** — its host is `wasmtime run`, which satisfies
  no `env.*` import; a declared-but-unprovided one makes the module refuse to
  instantiate.
- **a component** — [`../httpbin-component`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin-component) builds this
  same file that way, and a component's host functions cross the canonical ABI
  rather than a core import.
- **the DEFAULT boundary** — `envelope`, the in-band body. That is what
  [`../btc-ticker`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/btc-ticker) is, and it is why this directory's `build.sh`
  has to ASK for `--host-boundary=streaming`: these endpoints echo arbitrary
  request bodies, so the octets have to stay octets. It is also the reason the
  guard names the imports rather than the targets that lack them — a flag turns
  them off too.

One source, four boundaries.

## Nothing to shim: `--no-wasi`

The Worker calls the export rather than running the module, and the handler does
no I/O, so `build.sh` compiles with `--no-wasi` and the module imports nothing:

```console
$ node -e 'const m = new WebAssembly.Module(require("fs").readFileSync("src/worker.wasm"));
           console.log(WebAssembly.Module.imports(m))'
[]
```

Instantiating is `new WebAssembly.Instance(module, {})` — synchronous, empty
import object, no WASI shim in the project. `--no-wasi` also makes the module a
*reactor*, so its top-level forms run under `_initialize` instead of `_start`.

`--optimize=size` is not optional either: a Worker bundle has a
[size limit](https://developers.cloudflare.com/workers/platform/limits/), and
the tree-shaker is what keeps the module small for no behaviour difference.

## Two heaps

The single thing most worth understanding before writing a handler of your own.
A rontolisp module has **two** memories:

| | The Lisp heap | Linear memory |
| --- | --- | --- |
| What lives there | every cons cell, hash table, instance and Lisp string, the Clack environment and the reply included | *only* the bytes of a string crossing the boundary, plus static data |
| Managed by | **the engine** — it is wasm-GC, and nothing in `src/index.js` touches it | **you.** The engine never traces it, so nothing there is freed on its own |
| Grows with | nothing, over time | every argument you write in, forever, unless you reclaim it |

The boundary is in the second column because **WebAssembly has no string type**.
A `:string` can only cross as UTF-8 bytes at a (pointer, length), and those
bytes are outside the GC's world by construction.

So the Lisp side of a request costs nothing to clean up and the *envelope* costs
a bracket: `__ronto_alloc` is a bump pointer with no `free`, and
`__ronto_alloc_mark`/`__ronto_alloc_reset` snapshot and restore its top. That
matters because the instance is **resident** — one isolate serves many requests.
Measured over 44,000 requests from Node: with the bracket, linear memory stays
flat; without it, it grows without bound.

Two rules come with a manual arena, both observed in `handleRequest`: read the
returned bytes out **before** resetting, and only reset to a mark taken before
everything still live. Resetting is otherwise safe even when the call interned
new symbols — the floor is the interned-symbol pool's high-water mark. The
bracket is also why `handleRequest` is deliberately **synchronous**: an isolate
interleaves requests only at `await` points, so a call with no `await` cannot
have another request's allocation land inside the bracket.

[`../hello`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/hello) is the same boundary with the arena absent — nothing
crosses *into* that module. [`../httpbin-component`](https://github.com/making/rontolisp/blob/develop/examples/cloudflare-workers/httpbin-component)
replaces this whole section with the canonical ABI, at
[a different set of costs](../README.md#would-the-component-model-be-simpler).

## Errors stay in the Lisp

Workers' engine supports the WebAssembly exception-handling proposal with no
flag, and rontolisp compiles `handler-case` into it automatically (under
wasmtime the same module needs `-W exceptions=y`). Two places use it: `body-json`
falls back to `null` when the body does not parse, and `handle-request` wraps the
whole adapter so any other Lisp error answers 500 and the instance keeps
serving. That is not optional on a reactor — an uncaught Lisp error is a **trap**,
which takes the instance down. `src/index.js` still catches whatever escapes —
that would be a real trap — and drops the instance, since a trapped instance's
Lisp heap cannot be trusted afterwards.

## Developing the handler without Cloudflare

`handle-request` is an ordinary function of a string, adapter included, so the
whole loop can happen on the interpreter — and identically on the other
backends, which is what keeps the handler honest. `check.lisp` prints each
exchange *and* asserts it with rove, so a drifted answer exits non-zero instead
of quietly no longer looking right; rove is vendored in this repository, hence
the three directories on `--system-path`:

```bash
SP=src/test/resources/rove:src/test/resources/dissect:src/test/resources/cl-ppcre
rontolisp check.lisp --system-path $SP
rontolisp check.lisp --system-path $SP -o Check.class && java -cp . Check
rontolisp check.lisp --system-path $SP -o check.wasm --optimize && \
  wasmtime run -W gc -W exceptions=y check.wasm
```

Expect one difference when comparing the printed exchanges: the **order of keys
inside a JSON object differs between backends**, because it follows hash-table
iteration order. The values are identical — which is why the assertions read
the reply as parsed data rather than as text.

## Limitations

All of these are the Worker sandbox or the `--no-wasi` build, not rontolisp:

- **No standard input.** A reactor has no environment and no files, so
  `uiop:getenv` and `probe-file` answer nothing; `read-line` traps, because it
  is not true that input ended. Pass what the handler needs through the envelope.
- **The clock is the one JavaScript sets**, through the exported
  `__ronto_set_time` — before `_initialize` so a library timestamping at load
  sees it, and again per request so it advances. It does not tick *inside* a
  request (neither does a Worker's `Date.now()`), so `(sleep n)` signals. Until a
  host sets it, the clock built-ins signal a catchable error rather than
  reporting 1970.
- **`random` works; `rontolisp:random-bytes` does not.** The module carries its
  own generator, seeded from `crypto.getRandomValues` through
  `__ronto_seed_random`. That is unpredictable per isolate but not
  cryptographically strong, so the API promising entropy keeps signalling — mint
  tokens with `crypto.randomUUID()` and pass them in. Compiling with
  `--host-random` draws every number from the isolate instead, at the price of
  one import.
- **Printing is discarded, not trapped.** `print` and `format t` reach a sink
  under `--no-wasi`. Log from `src/index.js` with `console.log`, which reaches
  `npx wrangler tail`.
- **No filesystem.** `with-open-file` and `open` compile to call-time error
  stubs and a runtime `load` cannot work. A compile-time `(load "...")` is fine
  — it is inlined before the module reaches Cloudflare.
- **No outgoing HTTP from the Lisp.** `rontolisp:fetch` needs a WASI HTTP host;
  call JavaScript's `fetch()` in `src/index.js` and pass the result in.
- **Repeated query parameters collapse.** `args` is built with
  `alist-hash-table`, so `?a=1&a=2` reports only `"a":"1"`. Nothing is lost on
  the way in — `rontolisp:query-params` answers both pairs.

## Rebuilding

```bash
./mvnw clean package -DskipTests   # from the repository root
examples/cloudflare-workers/httpbin/build.sh
```


---

# FILE: references/examples/cloudflare-workers/httpbin/build.sh

#!/usr/bin/env bash
# Compile worker.lisp to the .wasm module the Worker imports.
#
# --no-wasi: the Worker calls the exported `handle-request` directly, it never
#   runs the module as a program, and the handler does no I/O -- so the module
#   needs no WASI imports at all. It becomes a reactor: nothing to shim on the
#   JavaScript side, and `_initialize` instead of `_start`.
# --optimize=size: a Worker bundle has a size limit, and the tree-shaker is what
#   keeps the module small -- only what the program reaches ships. =size
#   additionally declines the two speed-over-size wasm-GC emissions, which weigh
#   more on this library-free module than on the clack builds, for a
#   per-request cost of a few microseconds.
# --host-boundary=streaming: this Worker ECHOES request bodies, so they must
#   cross as octets rather than as JSON text in the envelope -- which is what
#   src/index.js feeds through env.readRequestBody / env.writeResponseBody, and
#   what lets a BINARY body come back exactly. Asked for, because the default is
#   `envelope` (see ../btc-ticker), where a body rides the head instead.
#
set -euo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
repo_root="$(cd "$here/../../.." && pwd)"

jar="$repo_root/target/rontolisp-0.1.0-SNAPSHOT-exec.jar"
if [[ ! -f "$jar" ]]; then
  echo "JAR not found: $jar" >&2
  echo "Build it first from the repo root: ./mvnw clean package" >&2
  exit 1
fi

echo "compiling worker.lisp -> src/worker.wasm"
java -jar "$jar" "$here/worker.lisp" -o "$here/src/worker.wasm" --no-wasi --host-boundary=streaming --optimize=size

ls -l "$here/src/worker.wasm"
echo "done. Run it with:  npx wrangler dev"


---

# FILE: references/examples/cloudflare-workers/httpbin/check.lisp

;;; Drive worker.lisp's handler without Cloudflare, on any backend:
;;; `handle-request` is an ordinary function of a string, adapter included.
;;;
;;; The requests below are the envelope src/index.js builds out of a real
;;; `Request`: the RAW target rather than a pre-split path and query object, and
;;; a content-length for anything with a body. Both fail quietly if you get them
;;; wrong -- so what each probe expects back is written down as rove assertions
;;; rather than as printed lines, and the file ends by making its verdict the
;;; process exit code. A handler that stops answering them fails the run instead
;;; of scrolling past.
;;;
;;; rove is loaded with asdf, so pass the directories holding its .asd files
;;; (rove, dissect and cl-ppcre, all vendored in this repository) with
;;; --system-path; outside this repository (ql:quickload "rove") fetches the
;;; same sources. See the Testing guide: doc/en/guides/testing.md
;;;
;;;   SP=src/test/resources/rove:src/test/resources/dissect:src/test/resources/cl-ppcre
;;;   rontolisp check.lisp --system-path $SP
;;;   rontolisp check.lisp --system-path $SP -o Check.class && java -cp . Check
;;;   rontolisp check.lisp --system-path $SP -o check.wasm --optimize && \
;;;     wasmtime run -W gc -W exceptions=y check.wasm

(asdf:load-system :rove)
(use-package :rove)
;; rove colors its report for a terminal; a checked pipeline wants plain text.
(setf *enable-colors* nil)

(load "worker.lisp")

(defun headers (&rest plist) (rontolisp:plist-hash-table plist))

(defun json-headers (body)
  (headers :host "example.com"
           :content-type "application/json"
           :content-length (princ-to-string (length body))))

;;; One exchange: build the envelope, call the handler, print both halves (this
;;; is still the local edit/run loop, and seeing the JSON is the point of it),
;;; and answer the PARSED reply -- so the assertions below read it as data
;;; instead of matching substrings, and nothing has to escape a quote.
(defun probe (request-plist)
  (let* ((request
          (rontolisp:json-stringify (rontolisp:plist-hash-table request-plist)))
         (response (handle-request request)))
    (format t "--> ~a~%" request)
    (format t "<-- ~a~%~%" response)
    (rontolisp:json-parse response)))

;;; The echo document, which travels as a JSON string inside the reply.
(defun reply-body (reply) (rontolisp:json-parse (gethash "body" reply)))

;;; --- the exchanges ----------------------------------------------------------
;;; Most requests are driven BEFORE the suite: arrange first, then assert, so
;;; the local edit/run loop's printed JSON stays in one block above the report.
;;; The unparseable-body probe is the exception and runs INSIDE its test --
;;; read-body's "json": null fallback is a handler-case, and a handler-case
;;; nested in rove's failure recorder (a handler-bind around each test body) is
;;; exactly the shape that has to keep working.

;; GET /get with a query string. The target arrives raw -- path and query still
;; joined -- and the environment's :query-string is what becomes "args".
(defparameter *query-reply*
  (probe
   (list :method "GET"
         :target "/get?a=1&b=two"
         :scheme "https"
         :remote-addr "203.0.113.7"
         :headers (headers :host "example.com" :accept "application/json")
         :body "")))

;; A percent-encoded path: %http-make-env decodes it, so :path-info -- and the
;; "path" the echo document reports -- is the decoded form.
(defparameter *encoded-path-reply*
  (probe
   (list :method "GET"
         :target "/%67et"
         :scheme "https"
         :remote-addr "203.0.113.7"
         :headers (headers :host "example.com")
         :body "")))

;; POST /post with a JSON body -- "data" is the raw text, "json" the parsed
;; value. The body reaches the application as clack's :raw-body, a synchronous
;; bivalent stream that read-body drains with read-char.
(defparameter *json-body-reply*
  (probe
   (list :method "POST"
         :target "/post"
         :scheme "https"
         :remote-addr "203.0.113.7"
         :headers (json-headers "{\"name\":\"rontolisp\"}")
         :body "{\"name\":\"rontolisp\"}")))

;; The wrong method for an endpoint -- 405.
(defparameter *wrong-method-reply*
  (probe
   (list :method "GET"
         :target "/post"
         :scheme "https"
         :remote-addr "203.0.113.7"
         :headers (headers :host "example.com")
         :body "")))

;; An unknown path -- 404.
(defparameter *unknown-path-reply*
  (probe
   (list :method "GET"
         :target "/nope"
         :scheme "https"
         :remote-addr "203.0.113.7"
         :headers (headers :host "example.com")
         :body "")))

;;; --- what each one must answer ----------------------------------------------

(deftest get-with-a-query-string
  (let ((body (reply-body *query-reply*)))
    (ok (= (gethash "status" *query-reply*) 200))
    (testing "the target arrives RAW, so %http-make-env owns the ? split"
      (ok (string= (gethash "path" body) "/get"))
      (ok (string= (gethash "a" (gethash "args" body)) "1"))
      (ok (string= (gethash "b" (gethash "args" body)) "two")))
    (testing "the request headers reach the application, lowercased"
      (ok (string= (gethash "host" (gethash "headers" body)) "example.com"))
      (ok (string= (gethash "method" body) "GET")))
    (testing "the response headers cross as an ARRAY of pairs, not an object"
      ;; An object would collapse a repeated name -- two cookies mean two
      ;; Set-Cookie headers.
      (let ((pair (aref (gethash "headers" *query-reply*) 0)))
        (ok (string= (aref pair 0) "content-type"))
        (ok (string= (aref pair 1) "application/json"))))))

(deftest a-percent-encoded-path
  (ok (string= (gethash "path" (reply-body *encoded-path-reply*)) "/get")))

(deftest post-with-a-json-body
  (let ((body (reply-body *json-body-reply*)))
    (ok (string= (gethash "data" body) "{\"name\":\"rontolisp\"}"))
    (ok (string= (gethash "name" (gethash "json" body)) "rontolisp"))))

;; POST /post with a body that does not parse -- "json" falls back to null,
;; which is `handler-case` doing its work, inside rove's own handler-bind.
(deftest post-with-a-body-that-does-not-parse
  (let* ((reply
          (probe
           (list :method "POST"
                 :target "/post"
                 :scheme "https"
                 :remote-addr "203.0.113.7"
                 :headers (json-headers "{not json")
                 :body "{not json")))
         (body (reply-body reply)))
    (ok (eq (gethash "json" body) 'null))
    (testing "the raw text still comes back untouched"
      (ok (string= (gethash "data" body) "{not json")))))

(deftest the-wrong-method-for-an-endpoint
  (ok (= (gethash "status" *wrong-method-reply*) 405))
  (ok (string= (gethash "allowed" (reply-body *wrong-method-reply*)) "POST")))

(deftest an-unknown-path
  (ok (= (gethash "status" *unknown-path-reply*) 404))
  (ok (string= (gethash "path" (reply-body *unknown-path-reply*)) "/nope")))

;;; Loading this file runs its suite (rove's file-driven entry point), and the
;;; exit code is the verdict -- so a handler that drifts breaks the build rather
;;; than the deployment.
(uiop:quit (if (run-suite *package*) 0 1))


---

# FILE: references/examples/cloudflare-workers/httpbin/package.json

{
  "name": "rontolisp-cloudflare-httpbin",
  "version": "0.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "build": "./build.sh",
    "dev": "wrangler dev",
    "deploy": "wrangler deploy",
    "tail": "wrangler tail"
  },
  "devDependencies": {
    "wrangler": "^4"
  }
}


---

# FILE: references/examples/cloudflare-workers/httpbin/src/index.js

// index.js -- the whole Worker: Request -> JSON head + body octets -> Lisp ->
// JSON head + body octets -> Response.
//
// BYTE-IDENTICAL in every httpbin-* directory that drives the module directly
// (httpbin-component has its own generated glue instead), and that is the
// point: how the Lisp half is written is not visible from JavaScript. A new
// sibling copies this file rather than editing it.
//
// The envelope is documented in ../../httpbin/README.md; two of its fields are
// easy to get wrong, and requestToHead below says which.

import module from "./worker.wasm";

const encoder = new TextEncoder();
const decoder = new TextDecoder();

// Instantiated on the FIRST REQUEST, not at module scope. Module scope would be
// the nicer place -- `wrangler deploy` reports and budget-checks what happens
// there as "Worker Startup Time" -- but a Worker forbids GENERATING RANDOM
// VALUES in the global scope, and the seed below has to be handed over before
// `_initialize` runs the Lisp top level. So the cost lands on one request
// instead, and nothing checks it at deploy time.
let lisp = null;
const lispInstance = () => (lisp ??= instantiate());

// The request body the module is about to pull, and how much of it has already
// crossed. Module-level rather than passed in, because the module asks for it
// from inside its own call -- see readRequestBody below.
const NO_BODY = new Uint8Array(0);
let body = NO_BODY;
let bodyOffset = 0;

// And the response body coming back the same way, chunk by chunk, in order.
// Reset per request.
let responseChunks = [];

function responseBody() {
  const all = new Uint8Array(responseChunks.reduce((n, c) => n + c.length, 0));
  let at = 0;
  for (const chunk of responseChunks) {
    all.set(chunk, at);
    at += chunk.length;
  }
  return all;
}

// `_initialize` is where the Lisp program's top-level forms run. ../../hello has
// no such entry point at all -- it is --no-gc with nothing to initialise --
// which is why it needs none of this.
function instantiate() {
  let instance;
  const env = {
    // The request body, out of band. The module owns the buffer and hands it
    // over per call -- write up to `cap` octets at `ptr` and answer how many,
    // 0 for end of stream -- so nothing here may hold on to the pointer, and
    // the body never has to exist as one JSON-escaped string inside the
    // envelope. A binary upload crosses exactly for the same reason.
    //
    // Synchronous, which is one of the two hosts the module accepts: it
    // declares the import `:async t`, so a host may equally wrap this in
    // `WebAssembly.Suspending` and pull straight from `request.body`'s reader
    // -- at the price of entering `handle-request` through
    // `WebAssembly.promising` and serialising the calls, because a suspended
    // module can be re-entered (see ../../dog-fetcher for that shape).
    readRequestBody(ptr, cap) {
      const n = Math.min(cap, body.length - bodyOffset);
      if (n <= 0) return 0;
      new Uint8Array(instance.exports.memory.buffer, ptr, n).set(
        body.subarray(bodyOffset, bodyOffset + n),
      );
      bodyOffset += n;
      return n;
    },
    // The response body, out of band and the other way round: take these
    // octets, they are the next chunk. COPY now -- the module pops the staging
    // behind the pointer the moment this returns, and reuses it for the next
    // chunk. A body that never becomes a JSON string is also a body that can be
    // BINARY, and one a large or streamed response never holds twice.
    //
    // Declared `:async t` like the reader, so this may equally be a
    // `WebAssembly.Suspending` that awaits a `TransformStream` writer -- which
    // is how a Worker would apply real backpressure -- at the same price the
    // reader's note names.
    writeResponseBody(ptr, len) {
      responseChunks.push(
        new Uint8Array(instance.exports.memory.buffer.slice(ptr, ptr + len)),
      );
    },
  };
  instance = new WebAssembly.Instance(module, { env });
  // Hand the module real entropy before its top level runs: a --no-wasi build
  // imports nothing else, so its `random` starts from a constant and every
  // isolate would otherwise draw the same sequence. Seeding here -- BEFORE
  // _initialize -- also covers the load-time draws inside quickloaded
  // libraries.
  instance.exports.__ronto_seed_random(
    new BigUint64Array(crypto.getRandomValues(new Uint8Array(8)).buffer)[0],
  );
  // And a clock, the same way and for the same reason: a --no-wasi build
  // imports none, so its time is whatever a host writes through
  // __ronto_set_time (nanoseconds since the Unix epoch). Before _initialize,
  // so a library that timestamps while it LOADS sees one -- unset, the clock
  // built-ins signal rather than report 1970.
  instance.exports.__ronto_set_time(BigInt(Date.now()) * 1000000n);
  instance.exports._initialize();
  return instance.exports;
}

// Synchronous on purpose: a Worker isolate interleaves concurrent requests only
// at `await` points, so nothing else can allocate inside the bracket -- and
// nothing else can take the body cursor above, which is the same guarantee the
// module's own re-entry guard makes on the other side.
function handleRequest(lisp, input) {
  // The module's clock moves only when we move it, so move it per request. That
  // is not a workaround for a frozen clock -- a Worker's own `Date.now()` is
  // frozen for the duration of a request as a timing-attack mitigation, so a
  // value that changes exactly once per request is what this platform has.
  lisp.__ronto_set_time(BigInt(Date.now()) * 1000000n);
  const bytes = encoder.encode(input);
  const mark = lisp.__ronto_alloc_mark();

  const ptr = lisp.__ronto_alloc(bytes.length);
  new Uint8Array(lisp.memory.buffer, ptr, bytes.length).set(bytes);

  const [resultPtr, resultLen] = lisp["handle-request"](ptr, bytes.length);

  // Copy out before resetting: the result sits in the scratch the reset frees.
  const result = decoder.decode(
    new Uint8Array(lisp.memory.buffer.slice(resultPtr, resultPtr + resultLen)),
  );

  lisp.__ronto_alloc_reset(mark);
  return result;
}

/** The raw facts the Lisp side turns into the Clack environment. */
function requestToHead(request, bodyLength) {
  const url = new URL(request.url);
  const headers = Object.fromEntries(request.headers);

  // Forward content-length. `%http-make-env` reads :content-length off the
  // header table, and lack/request's body parsing returns nothing without it --
  // while a request that arrived chunked has no content-length header at all.
  // We are holding the body, so set it from the octets we actually have.
  if (bodyLength) headers["content-length"] = String(bodyLength);

  return JSON.stringify({
    method: request.method,
    // The RAW target -- path and query still joined, still percent-encoded.
    // `%http-make-env` does the "?" split and the decoding itself, and
    // :path-info / :query-string have to come from it for a Clack application
    // to see what Clack promises. A pre-split path leaves :query-string nil.
    target: url.pathname + url.search,
    scheme: url.protocol.replace(":", ""),
    // Clack's :remote-addr. Cloudflare puts the client IP here; there is no
    // peer port to report, so :remote-port stays nil.
    "remote-addr": request.headers.get("cf-connecting-ip"),
    headers,
  });
}

export default {
  async fetch(request) {
    // `request.body` is null exactly when there is none, so a GET reads
    // nothing. This is the one place a synchronous host has to buffer: the
    // module pulls the octets from inside its own call, and this side cannot
    // await there without JSPI.
    body = request.body ? new Uint8Array(await request.arrayBuffer()) : NO_BODY;
    bodyOffset = 0;
    responseChunks = [];
    const input = requestToHead(request, body.length);
    let reply;
    try {
      reply = JSON.parse(handleRequest(lispInstance(), input));
    } catch (error) {
      // The Lisp side answers 500 for Lisp errors itself, so reaching here
      // means a WASM trap -- which skipped the arena reset and may have left
      // Lisp state half-written. Replace the instance rather than keep serving.
      lisp = instantiate();
      console.error("handle-request failed:", error);
      return new Response("internal error\n", { status: 500 });
    }
    // The headers arrive as an ARRAY of [name, value] pairs, not an object:
    // that is what keeps a Clack application's two Set-Cookie headers two
    // headers instead of one.
    //
    // A "body" key is present only when the body did NOT cross out of band --
    // absent and empty are different responses, which is why the Lisp side
    // drops the key rather than emptying it. When it IS there it WINS: the 500
    // a failing handler answers rides the head, so the chunks taken before it
    // are discarded rather than prepended to it.
    return new Response(reply.body ?? responseBody(), {
      status: reply.status ?? 200,
      headers: reply.headers ?? [],
    });
  },
};


---

# FILE: references/examples/cloudflare-workers/httpbin/worker.lisp

;;; No library: a mini httpbin (https://httpbin.org) whose Worker adapter is
;;; written out under it, so clack never loads and only the program ships.
;;;
;;; A Worker hands over a request JavaScript has already parsed rather than a
;;; socket, so there is no server to run -- the module exports ONE function,
;;;
;;;   handle-request : JSON request string -> JSON response string
;;;
;;; which src/index.js calls. The adapter converts nothing itself:
;;; rontolisp::%http-make-env and %http-normalize-response are the entry points
;;; every SERVED request also goes through, so the "?" split, the
;;; percent-decoding, the header table and the buffered :raw-body come for free.
;;; All that is left to write is the JSON envelope.
;;;
;;; Nothing here does I/O, which is what lets build.sh compile with --no-wasi:
;;; the only thing the module imports is the request body below.

(rontolisp:wasm-export 'handle-request :params '(:string) :returns :string)

;;; The body does NOT ride the envelope. The host hands it over through a
;;; byte-shaped import instead -- (ptr, cap) in, "how many octets I wrote" out,
;;; 0 for end of stream -- writing into ONE buffer the module keeps and reuses
;;; for every chunk of every request. Two things follow that a JSON string
;;; cannot give: a BINARY body crosses exactly (the string boundary's decoder is
;;; non-validating), and a large upload no longer costs linear memory
;;; proportional to its own size. :async t says the host MAY suspend while it
;;; reads -- a WebAssembly.Suspending wrapper over a ReadableStream reader --
;;; and src/index.js answers synchronously, which the declaration allows.
;;;
;;; The guard names the thing itself: #+rontolisp-body-imports is present
;;; exactly where these two imports exist -- a --no-wasi wasm-GC core module
;;; built with --host-boundary=streaming, which build.sh ASKS for because the
;;; default is the in-band envelope and these endpoints echo arbitrary bodies.
;;; Every other way of running THIS FILE lacks them, each for its own reason,
;;; and the one feature covers all of them: check.lisp drives handle-request as
;;; an ordinary function on the interpreter and the JVM; a plain WASI command
;;; module's host is `wasmtime run`, which satisfies no env.* import (a
;;; declared-but-unprovided one makes it refuse to instantiate);
;;; ../httpbin-component builds this file as a component, whose host functions
;;; cross the canonical ABI instead of a core import; and --host-boundary=envelope
;;; asks for the in-band body on purpose. Every one of them keeps the envelope's
;;; own "body" key, which is what the #- half below answers with. Same
;;; endpoints, one source, every host.
#+rontolisp-body-imports
(rontolisp:wasm-import '%read-request-body
                       :from "env"
                       :as "readRequestBody"
                       :params '()
                       :returns :bytes
                       :async t)

;; The request body, in whichever shape it arrived: OCTETS pulled through the
;; import, or the envelope's own string where there is no import. The Gray body
;; stream below is bivalent over octets and takes either, so the pulled body is
;; never decoded and encoded again. The two rontolisp:: names below are the
;; transport's own, like %http-make-env: what this file writes out by hand is
;; the ENVELOPE, and neither the reused buffer nor the drain that empties it is
;; envelope work.
#+rontolisp-body-imports
(defun %body-source (req)
  (declare (ignore req))
  (rontolisp::%http-reactor-body-octets
   (lambda ()
     (let ((buf (rontolisp::%http-reactor-buffer 65536)))
       (rontolisp::%http-reactor-chunk buf (%read-request-body buf))))))

#-rontolisp-body-imports (defun %body-source (req) (gethash "body" req))

;;; The response body leaves the envelope the same way, through the mirror
;;; import: (ptr, len) OUT, "take these octets, they are the next chunk". No
;;; result -- a host cannot short-read a write, and a chunk it has not taken by
;;; the time the call returns is one it never gets, since the module reuses the
;;; memory behind it. Note the direction flip: a chunk crossing out is a :bytes
;;; PARAMETER where one crossing in is a :bytes RESULT, which is the same rule
;;; -- the caller owns the memory -- applied both ways.
#+rontolisp-body-imports
(rontolisp:wasm-import '%write-response-body
                       :from "env"
                       :as "writeResponseBody"
                       :params '(:bytes)
                       :returns :void
                       :async t)

;; The SINK. The encode is the transport's own name again: a text chunk becomes
;; UTF-8, and an (unsigned-byte 8) body is already the octets it means.
#+rontolisp-body-imports
(defun %body-sink (chunk)
  (%write-response-body (rontolisp::%http-reactor-octets chunk)))

;; T when the body was taken out of band. Every other build answers NIL and
;; never names the transport's writer at all, so nothing of it is spliced there.
#+rontolisp-body-imports
(defun %write-body (body)
  (rontolisp::%http-reactor-write (function %body-sink) body)
  t)

#-rontolisp-body-imports
(defun %write-body (body)
  (declare (ignore body))
  nil)

;;; --- the endpoints -----------------------------------------------------------

(defun read-body (stream)
  (if (null stream)
      ""
      (with-output-to-string (out)
        (do ((ch (read-char stream nil nil) (read-char stream nil nil)))
            ((null ch))
          (write-char ch out)))))

;; Parse the body as JSON when it looks like one, and fall back to null when it
;; does not parse -- which is what the real httpbin does.
(defun body-json (body)
  (if (and (stringp body) (> (length body) 0)
           (or (eql (char body 0) #\{) (eql (char body 0) #\[)))
      (handler-case (rontolisp:json-parse body) (error () 'null))
      'null))

(defun json-response (status object)
  (list status '(:content-type "application/json")
        (list (format nil "~a~%" (rontolisp:json-stringify object)))))

;; plist-hash-table and alist-hash-table give json-stringify the string-keyed
;; hash tables it serializes as objects (:method becomes "method"; an empty
;; query still renders {}), and the env :headers already is one.
(defun echo (env with-body)
  (let ((info
         (rontolisp:plist-hash-table
          (list :args (rontolisp:alist-hash-table
                       (rontolisp:query-params (getf env :query-string)))
                :headers (getf env :headers)
                :method (symbol-name (getf env :request-method))
                :path (getf env :path-info)))))
    (when with-body
      (let ((body (read-body (getf env :raw-body))))
        (setf (gethash "data" info) body)
        (setf (gethash "json" info) (body-json body))))
    (json-response 200 info)))

;; :request-method is an interned keyword, so the check is eq.
(defun endpoint (env method with-body)
  (if (eq (getf env :request-method) method)
      (echo env with-body)
      (json-response 405
                     (rontolisp:plist-hash-table
                      (list :error "method not allowed"
                            :allowed (symbol-name method))))))

;; :path-info carries the decoded path only -- the query string arrives
;; separately -- so the comparisons are exact.
(defun dispatch (env)
  (let ((path (getf env :path-info)))
    (cond ((string= path "/get") (endpoint env :GET nil))
          ((string= path "/post") (endpoint env :POST t))
          ((string= path "/put") (endpoint env :PUT t))
          ((string= path "/patch") (endpoint env :PATCH t))
          ((string= path "/delete") (endpoint env :DELETE t))
          (t (json-response 404
                            (rontolisp:plist-hash-table
                             (list :error "not found" :path path)))))))

;;; --- the reactor adapter -----------------------------------------------------
;;; What `clack:clackup :server :reactor` would install. Nothing above knows it
;;; exists.

;; The headers JSON object -> the ((name . value) ...) alist the raw tuple wants.
(defun %header-alist (table)
  (if (null table)
      nil
      (let ((out nil))
        (maphash (lambda (name value) (setq out (cons (cons name value) out)))
                 table)
        (nreverse out))))

;; The response header alist -> a JSON ARRAY of [name, value]: a name may
;; repeat, and two cookies mean two Set-Cookie headers.
(defun %header-pairs (alist)
  (let ((out nil))
    (dolist (pair alist) (setq out (cons (list (car pair) (cdr pair)) out)))
    (nreverse out)))

;; The positional tuple %http-make-env consumes. "target" is RAW -- path and
;; query still joined and still encoded, because %http-make-env owns that split
;; -- and the Host header supplies :server-name / :server-port, so the two
;; placeholders below never win when the host sends one. The body is DRAINED
;; here rather than streamed: the endpoints echo it whole, and buffering it is
;; the one thing Clack's :raw-body would do anyway.
(defun %request-tuple (req)
  (list (or (gethash "method" req) "GET") (or (gethash "target" req) "/")
        (%header-alist (gethash "headers" req))
        (rontolisp::%http-body-stream (%body-source req)) "HTTP/1.1"
        (gethash "scheme" req) "localhost" 80 (gethash "remote-addr" req) nil))

;; With a SINK the body crosses out of band and the "body" key is ABSENT -- not
;; empty: a host has to be able to tell "the body crossed" from "the body is the
;; empty string". The chunks cross BEFORE this head, because the head is the
;; return value, so a head that carries the key WINS over anything already
;; written -- which is what makes the 500 below recoverable rather than a
;; corrupt response.
(defun %envelope (status headers body out-of-band)
  (let* ((sent (and out-of-band (%write-body body)))
         (head (list :status status :headers (%header-pairs headers))))
    (rontolisp:json-stringify
     (rontolisp:plist-hash-table
      (if sent head (append head (list :body body)))))))

;; The host's entry point. It CATCHES: on a reactor an uncaught Lisp error is a
;; trap that takes the whole instance down, so answer 500 and keep serving --
;; in band, whether or not the body had a sink.
(defun handle-request (request-json)
  (handler-case (let* ((req (rontolisp:json-parse request-json))
                       (env (rontolisp::%http-make-env (%request-tuple req)))
                       (triple
                        (rontolisp::%http-normalize-response (dispatch env))))
                  (%envelope (car triple) (cadr triple) (caddr triple) t))
    (error (e)
      (%envelope 500 (list (cons "content-type" "application/json"))
                 (format nil "~a~%"
                         (rontolisp:json-stringify
                          (rontolisp:plist-hash-table
                           (list :error (format nil "~a" e))))) nil))))


---

# FILE: references/examples/cloudflare-workers/httpbin/wrangler.jsonc

{
  // https://developers.cloudflare.com/workers/wrangler/configuration/
  "name": "rontolisp-httpbin",
  "main": "src/index.js",
  "compatibility_date": "2026-01-01",

  // `import module from "./worker.wasm"` needs no configuration: .wasm is a
  // built-in module rule for ES-module Workers, and the file is compiled at
  // deploy time rather than on the request path.
  "observability": {
    "enabled": true
  }
}


---

# FILE: references/examples/console/calc.lisp

;;;; A tiny expression interpreter in rontolisp
;;;; Evaluates a small prefix arithmetic language, represented as ordinary
;;;; s-expressions, with a hand-written recursive evaluator over an association
;;;; list environment. Constant expressions are also cross-checked against the
;;;; built-in `eval`. Uses recursion, cond/case, alists and eval -> runs on all
;;;; three backends (interpreter / JVM / WASM).
;;;;
;;;; Run:
;;;;   java -jar target/rontolisp-0.1.0-SNAPSHOT-exec.jar examples/console/calc.lisp
;;;;   java -jar ...-exec.jar examples/console/calc.lisp -o Calc.class && java Calc
;;;;   java -jar ...-exec.jar examples/console/calc.lisp -o calc.wasm && wasmtime run -W gc calc.wasm

;;; Look a variable up in the environment (an alist of (symbol . value)).
;;; Symbols compare by eql, so plain `assoc` is enough -- no :test needed.
(defun lookup (sym env)
  (let ((pair (assoc sym env)))
    (if pair (cdr pair) (error "unbound variable: ~a" sym))))

;;; Evaluate `expr` in `env`. The language is: numbers, variables (symbols), and
;;; binary applications (op a b) where op is one of + - * mod.
(defun my-eval (expr env)
  (cond ((numberp expr) expr)
        ((symbolp expr) (lookup expr env))
        ((consp expr)
         (let ((op (car expr))
               (a (my-eval (cadr expr) env))
               (b (my-eval (caddr expr) env)))
           (case op
             ((+) (+ a b))
             ((-) (- a b))
             ((*) (* a b))
             ((mod) (mod a b))
             (t (error "unknown operator: ~a" op)))))
        (t (error "cannot evaluate: ~a" expr))))

;;; An environment binding x and y.
(defparameter *env* (list (cons 'x 10) (cons 'y 3)))

(format t "Evaluating with env x=10, y=3:~%")
(dolist (p '((+ 1 (* 2 3)) (- (* x x) (* y y)) (mod (+ x y) 5)))
  (format t "  ~a => ~a~%" p (my-eval p *env*)))

(format t "~%Cross-check against the built-in eval (constant expressions):~%")
(dolist (p '((+ 1 (* 2 3)) (- 100 (* 7 8)) (* (+ 1 2) (+ 3 4))))
  (format t "  ~a : my-eval=~a  eval=~a~%" p (my-eval p nil) (eval p)))


---

# FILE: references/examples/console/contact-book.lisp

;;;; Contact book using defstruct in rontolisp
;;;; Demonstrates defstruct (tagged-list representation), hash tables,
;;;; setf accessors, and format directives. Runs on all three backends.
;;;;
;;;; Run:
;;;;   rontolisp examples/console/contact-book.lisp
;;;;   rontolisp examples/console/contact-book.lisp -o ContactBook.class && java ContactBook
;;;;   rontolisp examples/console/contact-book.lisp -o contact-book.wasm && wasmtime run -W gc contact-book.wasm

(defstruct contact name email phone notes)

(defun make-book ()
  "Create an empty contact book (hash table keyed by name)."
  (make-hash-table))

(defun add-contact (book &key name email phone notes)
  "Add a contact to the book. Returns the contact."
  (let ((c (make-contact :name name :email email :phone phone :notes notes)))
    (setf (gethash name book) c)
    c))

(defun find-contact (book name)
  "Look up a contact by name."
  (gethash name book))

(defun update-email (book name new-email)
  "Update a contact's email using (setf contact-email)."
  (let ((c (find-contact book name)))
    (if c
        (progn
          (setf (contact-email c) new-email)
          t)
        nil)))

(defun list-contacts (book)
  "Return all contacts in the book as a list."
  (let ((result nil))
    (maphash (lambda (name contact) (push contact result)) book)
    result))

;;; Build a sample contact book
(let ((book (make-book)))
  (add-contact book
               :name "Alice"
               :email "alice@example.com"
               :phone "555-0101"
               :notes "Developer")
  (add-contact book
               :name "Bob"
               :email "bob@example.com"
               :phone "555-0102"
               :notes "Designer")
  (add-contact book
               :name "Carol"
               :email "carol@example.com"
               :phone "555-0103"
               :notes "Manager")

  (format t "Contact Book (~d entries):~%" (hash-table-count book))
  (format t "~%All contacts:~%")
  (dolist (c (list-contacts book))
    (format t "  ~a | ~a | ~a | ~a~%" (contact-name c) (contact-email c)
            (contact-phone c) (contact-notes c)))

  ;; Update an email
  (update-email book "Alice" "alice@newdomain.com")
  (format t "~%After updating Alice's email:~%")
  (let ((alice (find-contact book "Alice")))
    (format t "  ~a -> ~a~%" (contact-name alice) (contact-email alice)))

  ;; Search by notes
  (format t "~%Contacts with 'Developer' in notes:~%")
  (dolist (c (list-contacts book))
    (when (string= (contact-notes c) "Developer")
      (format t "  ~a (~a)~%" (contact-name c) (contact-email c)))))


---

# FILE: references/examples/console/error-handling.lisp

;;;; Error handling: typed conditions, handler-case and unwind-protect.
;;;; A tiny bank account that signals a typed `insufficient-funds` error:
;;;; `define-condition` with slots, `:reader` accessors and a `:report` lambda
;;;; (the message an uncaught error prints); `(error 'type :initarg value ...)`
;;;; designators; `handler-case` dispatching by class hierarchy with a
;;;; `:no-error` clause; `ignore-errors`; `unwind-protect` running its cleanup
;;;; on the error path (the audit log records refused withdrawals too); a
;;;; non-fatal `signal` that returns nil when no handler is established; and
;;;; `typecase` / `with-slots` over condition objects.
;;;;
;;;; Runs on every backend except --no-gc; the WASM output uses the wasm
;;;; exception-handling proposal, so wasmtime (37+) needs `-W exceptions=y`.
;;;;
;;;; Run:
;;;;   rontolisp examples/console/error-handling.lisp
;;;;   rontolisp examples/console/error-handling.lisp -o Bank.class && java Bank
;;;;   rontolisp examples/console/error-handling.lisp -o bank.wasm && wasmtime run -W gc -W exceptions=y bank.wasm

;;; The condition hierarchy: error > account-error > insufficient-funds.
;;; account-error exists so a handler can catch every account problem at once.
(define-condition account-error (error) ())

(define-condition insufficient-funds (account-error)
  ((requested :initarg :requested :reader requested-amount)
   (balance :initarg :balance :reader available-balance))
  ;; The :report lambda renders the message an UNCAUGHT error would print,
  ;; e.g. "Error: cannot withdraw 200: only 70 available".
  (:report
   (lambda (c s)
     (format s "cannot withdraw ~a: only ~a available" (requested-amount c)
             (available-balance c)))))

(defvar *balance* 100)
(defvar *audit-log* nil)

(defun withdraw (amount)
  "Withdraw AMOUNT or signal a typed insufficient-funds error."
  (when (> amount *balance*)
    (error 'insufficient-funds :requested amount :balance *balance*))
  (setq *balance* (- *balance* amount))
  *balance*)

(defun audited-withdraw (amount)
  "Record every attempt -- unwind-protect runs the cleanup on success AND
when `withdraw` signals, so refused withdrawals reach the audit log too."
  (unwind-protect (withdraw amount)
    (setq *audit-log* (cons amount *audit-log*))))

(defun try-withdraw (amount)
  "handler-case: catch by the concrete type and read its slots; :no-error
runs on normal completion with the protected form's value."
  (format t "withdraw ~a:~%" amount)
  (handler-case (audited-withdraw amount)
    (insufficient-funds (e)
      (format t "  refused: wanted ~a but only ~a available~%"
              (requested-amount e) (available-balance e)))
    (:no-error (balance) (format t "  ok, new balance ~a~%" balance))))

(format t "Error handling: conditions + handler-case + unwind-protect~%~%")

(try-withdraw 30)  ; succeeds
(try-withdraw 200) ; refused, but still audited
(try-withdraw 60)  ; succeeds again

;; ignore-errors = handler-case sugar: nil instead of an unwind.
(format t "ignore-errors on withdraw 999 -> ~a~%"
        (ignore-errors (audited-withdraw 999)))

;; The cleanups ran on every path: the refused 200 and 999 are logged too.
(format t "audit log (newest first): ~a~%" *audit-log*)
(format t "balance is still ~a~%~%" *balance*)

;; A clause naming the PARENT type catches the subtype too.
(format t "caught by the parent type -> ~a~%"
        (handler-case (error 'insufficient-funds :requested 1 :balance 0)
          (account-error (e) :caught-as-account-error)))

;; signal is non-fatal: no established handler means it just returns nil...
(format t "signal without a handler -> ~a~%" (signal "balance is getting low"))

;; ...but handler-case catches a raised signal like any other condition.
(format t "signal with a handler    -> ~a~%"
        (handler-case (progn
                        (signal "balance is getting low")
                        :not-reached)
          (condition (c) :noticed)))

;; Condition objects are ordinary values: typecase dispatches on their class
;; hierarchy and with-slots reads their slots (no signaling involved).
(let ((c (make-condition 'insufficient-funds :requested 5 :balance 1)))
  (format t "typecase classifies it as ~a~%"
          (typecase c
            (warning 'some-warning)
            (account-error 'an-account-error)
            (error 'some-other-error)))
  (with-slots (requested balance) c
    (format t "with-slots reads: requested=~a balance=~a~%" requested balance)))


---

# FILE: references/examples/console/hanoi.lisp

;;;; Tower of Hanoi in rontolisp
;;;; Classic recursive puzzle solver: move N disks from source to destination
;;;; using an auxiliary peg, printing each move. Demonstrates recursion,
;;;; conditional logic, and list accumulation.
;;;; Runs on all three backends (interpreter / JVM / WASM).
;;;;
;;;; Run:
;;;;   rontolisp examples/console/hanoi.lisp
;;;;   rontolisp examples/console/hanoi.lisp -o Hanoi.class && java Hanoi
;;;;   rontolisp examples/console/hanoi.lisp -o hanoi.wasm && wasmtime run -W gc hanoi.wasm

(defun take (n lst)
  "Return the first N elements of LST."
  (if (or (<= n 0) (null lst)) nil (cons (car lst) (take (1- n) (cdr lst)))))

(defun count-moves (n)
  "Return the number of moves for N disks: 2^n - 1."
  (1- (expt 2 n)))

(defun hanoi (n source destination auxiliary)
  "Move N disks from SOURCE to DESTINATION via AUXILIARY, printing each move."
  (if (= n 1)
      (format t "  Move disk 1 from ~a to ~a~%" source destination)
      (progn
        (hanoi (1- n) source auxiliary destination)
        (format t "  Move disk ~d from ~a to ~a~%" n source destination)
        (hanoi (1- n) auxiliary destination source))))

(defun hanoi-moves (n source destination auxiliary)
  "Move N disks, returning a list of (from to) pairs instead of printing."
  (if (= n 1)
      (list (list source destination))
      (append (hanoi-moves (1- n) source auxiliary destination)
              (list (list source destination))
              (hanoi-moves (1- n) auxiliary destination source))))

(format t "Tower of Hanoi (3 disks):~%")
(hanoi 3 "A" "B" "C")
(format t "~%Total moves: ~d (expected: ~d = 2^3 - 1)~%"
        (length (hanoi-moves 3 "A" "B" "C")) (count-moves 3))

(format t "~%Tower of Hanoi (4 disks) — first 5 moves:~%")
(let ((move-list (hanoi-moves 4 "α" "β" "γ")))
  (format t "Total: ~d moves (expected: ~d)~%" (length move-list)
          (count-moves 4))
  (dolist (m (take 5 move-list)) (format t "  ~a -> ~a~%" (car m) (cadr m)))
  (when (> (length move-list) 5)
    (format t "  ... (~d more)~%" (- (length move-list) 5))))


---

# FILE: references/examples/console/l-system.lisp

;;;; L-system (Lindenmayer system) fractal generator in rontolisp
;;;; Demonstrates string rewriting systems and hash-table rule dispatch.
;;;; Runs on all three backends (interpreter / JVM / WASM).
;;;;
;;;; Run:
;;;;   rontolisp examples/console/l-system.lisp
;;;;   rontolisp examples/console/l-system.lisp -o LSystem.class && java LSystem
;;;;   rontolisp examples/console/l-system.lisp -o l-system.wasm && wasmtime run -W gc l-system.wasm

(defun make-rule-table (&rest pairs)
  "Build a hash table from alternating key-value pairs."
  (let ((table (make-hash-table)))
    (dotimes (i (/ (length pairs) 2))
      (setf (gethash (nth (* i 2) pairs) table) (nth (1+ (* i 2)) pairs)))
    table))

(defun l-system-step (current rules)
  "Apply one iteration of L-system rules to CURRENT string."
  (let ((next ""))
    (dotimes (i (length current))
      (let ((ch (char current i)))
        (setq next
              (concatenate 'string next
                           (or (gethash ch rules) (format nil "~a" ch))))))
    next))

(defun l-system-string (axiom rules iterations)
  "Generate the L-system string after N iterations."
  (let ((current axiom))
    (dotimes (_ iterations) (setq current (l-system-step current rules)))
    current))

;;; Count occurrences of each character in a string
(defun char-counts (s)
  "Return a hash table of character -> count for string S."
  (let ((counts (make-hash-table)))
    (dotimes (i (length s))
      (setf (gethash (char s i) counts)
            (1+ (or (gethash (char s i) counts) 0))))
    counts))

(format t "L-system String Rewriting~%~%")

;;; Sierpinski triangle: F->FX, X->XFX
(let ((rules (make-rule-table #\F "FX" #\X "XFX")))
  (format t "Sierpinski triangle (F->FX, X->XFX):~%")
  (let ((axiom "F-X"))
    (dotimes (n 5)
      (let ((s (l-system-string axiom rules n)))
        (format t "  ~d: length=~6d  ~a~%" n (length s) s))))

  (format t "~%Character distribution at iteration 6:~%")
  (let ((s (l-system-string "F-X" rules 6)))
    (format t "  Total length: ~d~%" (length s))
    (let ((counts (char-counts s)))
      (maphash (lambda (ch count) (format t "  ~a: ~d~%" ch count)) counts))))

(format t "~%")

;;; Koch curve: F->F+F-F+F
(let ((rules (make-rule-table #\F "F+F-F+F")))
  (format t "Koch curve (F->F+F-F+F):~%")
  (dotimes (n 4)
    (let ((s (l-system-string "F" rules n)))
      (format t "  ~d: length=~6d  ~a~%" n (length s) s)))

  (format t "~%At iteration 3:~%")
  (let ((s (l-system-string "F" rules 3)))
    (format t "  Length: ~d characters~%" (length s))
    (let ((plus 0) (minus 0))
      (dotimes (i (length s))
        (when (char= (char s i) #\+) (setq plus (1+ plus)))
        (when (char= (char s i) #\-) (setq minus (1+ minus))))
      (format t "  Plus signs: ~d~%" plus)
      (format t "  Minus signs: ~d~%" minus))))

(format t "~%")

;;; Dragon curve: X->X+YF+, Y->-FX-Y
(let ((rules (make-rule-table #\X "X+YF+" #\Y "-FX-Y")))
  (format t "Dragon curve (X->X+YF+, Y->-FX-Y):~%")
  (dotimes (n 8)
    (let ((s (l-system-string "FX" rules n)))
      (format t "  ~d: length=~6d~%" n (length s))))

  (format t "~%At iteration 12:~%")
  (let ((s (l-system-string "FX" rules 12)))
    (format t "  Length: ~d characters~%" (length s))
    (let ((counts (char-counts s)))
      (maphash (lambda (ch count) (format t "  ~a: ~d~%" ch count)) counts))))


---

# FILE: references/examples/console/life-core.lisp

;;;; life-core.lisp -- Conway's Game of Life, rendering-free core.
;;;;
;;;; Shared by life.lisp (console) and life-gui.lisp (Swing). A 30x24 toroidal
;;;; world is a 2-D array of cells (1 = alive, 0 = dead); the world wraps at every
;;;; edge. Plain integer arithmetic and O(1) aref. `life-seed` returns a fresh grid
;;;; stamped with several classic patterns (glider, LWSS, blinker, toad, beacon,
;;;; pulsar); the drivers decide how to display successive generations.

(defparameter *rows* 24)
(defparameter *cols* 30)

;;; A fresh rows x cols grid, all dead.
(defun make-grid (rows cols) (make-array (list rows cols) :initial-element 0))

;;; The cell at (r, c), wrapping both coordinates onto the torus.
(defun cell-at (grid r c rows cols) (aref grid (mod r rows) (mod c cols)))

;;; How many of the eight neighbours of (r, c) are alive.
(defun live-neighbors (grid r c rows cols)
  (let ((sum 0) (dr -1))
    (while (<= dr 1)
      (let ((dc -1))
        (while (<= dc 1)
          (unless (and (= dr 0) (= dc 0))
            (setq sum (+ sum (cell-at grid (+ r dr) (+ c dc) rows cols))))
          (setq dc (+ dc 1))))
      (setq dr (+ dr 1)))
    sum))

;;; The next state of cell (r, c) under Conway's B3/S23 rule.
(defun next-cell (grid r c rows cols)
  (let ((n (live-neighbors grid r c rows cols))
        (alive (= (cell-at grid r c rows cols) 1)))
    (if alive (if (or (= n 2) (= n 3)) 1 0) (if (= n 3) 1 0))))

;;; Advance the whole grid one generation, returning a new grid.
(defun next-gen (grid rows cols)
  (let ((new (make-grid rows cols)) (r 0))
    (while (< r rows)
      (let ((c 0))
        (while (< c cols)
          (setf (aref new r c) (next-cell grid r c rows cols))
          (setq c (+ c 1))))
      (setq r (+ r 1)))
    new))

;;; Total number of live cells in the grid.
(defun population (grid rows cols)
  (let ((sum 0) (r 0))
    (while (< r rows)
      (let ((c 0))
        (while (< c cols)
          (setq sum (+ sum (aref grid r c)))
          (setq c (+ c 1))))
      (setq r (+ r 1)))
    sum))

;;; Stamp a list of (row col) offsets into the grid, rooted at (top, left).
(defun stamp (grid top left coords)
  (dolist (rc coords)
    (setf (aref grid (+ top (car rc)) (+ left (car (cdr rc)))) 1)))

;;; A fresh world seeded with several classic patterns.
(defun life-seed ()
  (let ((grid (make-grid *rows* *cols*)))
    ;; A glider, crawling down-right from the top-left corner.
    (stamp grid 1 1 '((0 1) (1 2) (2 0) (2 1) (2 2)))
    ;; A lightweight spaceship (LWSS), gliding left along the top band.
    (stamp grid 2 18 '((0 1) (0 4) (1 0) (2 0) (2 4) (3 0) (3 1) (3 2) (3 3)))
    ;; A vertical blinker (period 2).
    (stamp grid 10 4 '((0 0) (1 0) (2 0)))
    ;; A toad (period 2).
    (stamp grid 14 4 '((0 1) (0 2) (0 3) (1 0) (1 1) (1 2)))
    ;; A beacon (period 2).
    (stamp grid 18 4 '((0 0) (0 1) (1 0) (2 3) (3 2) (3 3)))
    ;; A pulsar (period 3), centred on the right side.
    (stamp grid 5 14
           '((0 2) (0 3) (0 4) (0 8) (0 9) (0 10) (2 0) (2 5) (2 7) (2 12) (3 0)
             (3 5) (3 7) (3 12) (4 0) (4 5) (4 7) (4 12) (5 2) (5 3) (5 4) (5 8)
             (5 9) (5 10) (7 2) (7 3) (7 4) (7 8) (7 9) (7 10) (8 0) (8 5) (8 7)
             (8 12) (9 0) (9 5) (9 7) (9 12) (10 0) (10 5) (10 7) (10 12) (12 2)
             (12 3) (12 4) (12 8) (12 9) (12 10)))
    grid))


---

# FILE: references/examples/console/life.lisp

;;;; Conway's Game of Life in rontolisp -- console front-end.
;;;;
;;;; The simulation lives in life-core.lisp; this file only loads it and prints a
;;;; handful of generations as ASCII. life-gui.lisp renders the same core in a
;;;; Swing window (JVM only). The load resolves relative to this file, so
;;;; it runs from anywhere and on all three backends.
;;;;
;;;; Run:
;;;;   java -jar target/rontolisp-0.1.0-SNAPSHOT-exec.jar examples/console/life.lisp
;;;;   java -jar ...-exec.jar examples/console/life.lisp -o life.wasm && wasmtime run -W gc life.wasm

(load "life-core.lisp")

;;; Render a grid as ASCII ('#' alive, '.' dead).
(defun print-grid (grid rows cols)
  (let ((r 0))
    (while (< r rows)
      (let ((c 0))
        (while (< c cols)
          (princ (if (= (aref grid r c) 1) "#" "."))
          (setq c (+ c 1))))
      (terpri)
      (setq r (+ r 1)))))

(let ((g (life-seed)) (gen 0))
  (while (<= gen 6)
    (format t "Generation ~d (population ~d):~%" gen
            (population g *rows* *cols*))
    (print-grid g *rows* *cols*)
    (terpri)
    (setq g (next-gen g *rows* *cols*))
    (setq gen (+ gen 1))))


---

# FILE: references/examples/console/line-numbers.lisp

;;;; Line-numbering file tool (like `cat -n`) in rontolisp
;;;; Writes a small sample text file, reads it back line by line, produces a
;;;; line-numbered copy, and reports line and character counts. Uses only
;;;; with-open-file, read-line, write-line, length and `format nil` -> runs on
;;;; all three backends (interpreter / JVM / WASM).
;;;;
;;;; NOTE: the WASM backend needs a preopened directory for file access, and
;;;; with-open-file rides the exception-handling proposal (wasmtime 37+):
;;;;   wasmtime run -W gc -W exceptions=y --dir . line-numbers.wasm
;;;;
;;;; Run:
;;;;   java -jar target/rontolisp-0.1.0-SNAPSHOT-exec.jar examples/console/line-numbers.lisp
;;;;   java -jar ...-exec.jar examples/console/line-numbers.lisp -o LineNumbers.class && java LineNumbers
;;;;   java -jar ...-exec.jar examples/console/line-numbers.lisp -o ln.wasm && wasmtime run -W gc -W exceptions=y --dir . ln.wasm

(defparameter *src* "poem.txt")
(defparameter *dst* "poem-numbered.txt")

;;; Create a small input file so the example is self-contained.
(with-open-file (out *src* :direction :output)
  (write-line "the quick brown fox" out)
  (write-line "jumps over" out)
  (write-line "the lazy dog" out))

;;; Read every line from `in`, write it to `out` prefixed with a right-aligned
;;; line number, and return (line-count . char-count). `format` only writes to
;;; t/nil, so we build each numbered line with `format nil` and `write-line` it.
(defun number-file (in out)
  (let ((n 0) (chars 0) (line (read-line in)))
    (while line
      (setq n (+ n 1))
      (setq chars (+ chars (length line)))
      (write-line (format nil "~4d  ~a" n line) out)
      (setq line (read-line in)))
    (cons n chars)))

(let ((counts nil))
  (with-open-file (in *src*)
    (with-open-file (out *dst* :direction :output)
      (setq counts (number-file in out))))
  (format t "Wrote ~d lines (~d characters) to ~a~%~%" (car counts) (cdr counts)
          *dst*)
  (format t "Contents of ~a:~%" *dst*)
  (with-open-file (in *dst*)
    (let ((line (read-line in)))
      (while line
        (princ line)
        (terpri)
        (setq line (read-line in))))))


---

# FILE: references/examples/console/mandelbrot-nogc.lisp

;;;; Mandelbrot set as ASCII art -- non-GC (--no-gc) edition
;;;; The companion to examples/console/mandelbrot.lisp. That version prints to stdout;
;;;; this one RETURNS the rendered grid as a string, because --no-gc compiles a
;;;; pure-compute reactor with no WASI imports and no I/O (it runs on any
;;;; MVP-class WebAssembly runtime with NO `-W gc`).
;;;;
;;;; Everything here stays within the --no-gc subset: floating-point arithmetic,
;;;; nested loops (dotimes/while/setq), cond, string literals and
;;;; (concatenate 'string ...). No cons, list, symbol, hash or I/O.
;;;;
;;;; The export is not described here: mandelbrot_component.wit is, and
;;;; rontolisp:wit-export at the bottom says "this program implements that world".
;;;; The compiler reads the .wit, checks the export it declares against the defun
;;;; below -- name, arity, parameter and result types -- and lowers it into the
;;;; export directive it stands for, so one directive serves both builds and a
;;;; drifted contract is a compile error naming the WIT line.
;;;;
;;;; 1. A plain MVP core module. A string crosses a core boundary as a
;;;;    (pointer, length) pair into the module's exported linear memory, so the
;;;;    host reads it out itself (the async IIFE lets `node -e` use await, which a
;;;;    bare top-level script cannot):
;;;;
;;;;      rontolisp examples/console/mandelbrot-nogc.lisp --no-gc --optimize -o mandelbrot.wasm
;;;;      node -e '(async () => {
;;;;        const ex = (await WebAssembly.instantiate(
;;;;          require("fs").readFileSync("mandelbrot.wasm"), {})).instance.exports;
;;;;        const [ptr, len] = ex.mandelbrot(-2.5, 1.0, -1.2, 1.2, 70, 30, 30);
;;;;        process.stdout.write(
;;;;          Buffer.from(new Uint8Array(ex.memory.buffer, ptr, len)).toString());
;;;;      })()'
;;;;
;;;; 2. A component. The canonical ABI carries the string across and frees it, so
;;;;    the host writes no memory code at all, and no runtime flags:
;;;;
;;;;      rontolisp examples/console/mandelbrot-nogc.lisp --no-gc --component --optimize \
;;;;        --emit-wit -o mandelbrot_component.wasm
;;;;      wasmtime run --invoke 'mandelbrot(-2.5, 1.0, -1.2, 1.2, 70, 30, 30)' \
;;;;        mandelbrot_component.wasm
;;;;
;;;;    (wasmtime prints the RETURNED value, so the art comes back as one escaped
;;;;    string literal rather than rendered; a real host gets the string itself.)
;;;;
;;;; Neither route is this example's discovery. The same relief inside a page,
;;;; through jco-generated bindings, is examples/browser/wit-component/; the
;;;; :string-PARAMETER half of the story -- and what --emit-wit does and does not
;;;; prove -- is examples/count-vowels/. What is worth seeing here is that the two
;;;; builds above come from one unchanged program and one world.

;;; Escape time for the complex point (cx, cy): the number of iterations of
;;; z <- z^2 + c before |z| > 2 (i.e. |z|^2 > 4), capped at `max-iter`.
(defun escape-time (cx cy max-iter)
  (let ((x 0.0) (y 0.0) (i 0))
    (while (and (< i max-iter) (<= (+ (* x x) (* y y)) 4.0))
      (let ((xt (+ (- (* x x) (* y y)) cx)))
        (setq y (+ (* 2.0 (* x y)) cy))
        (setq x xt))
      (setq i (+ i 1)))
    i))

;;; Map an escape time to a single-character shading string.
(defun shade (i max-iter)
  (cond ((>= i max-iter) "#") ((>= i 10) "+") ((>= i 5) ".") (t " ")))

;;; Render the region [x0,x1] x [y0,y1] as a cols x rows grid, accumulating the
;;; characters (and a newline per row) into a single string that is returned.
(defun mandelbrot (x0 x1 y0 y1 cols rows max-iter)
  (let ((dx (/ (- x1 x0) cols)) (dy (/ (- y1 y0) rows)) (out ""))
    (dotimes (r rows)
      (let ((cy (+ y0 (* dy r))))
        (dotimes (c cols)
          (setq out
                (concatenate 'string out
                 (shade (escape-time (+ x0 (* dx c)) cy max-iter) max-iter))))
        (setq out
              (concatenate 'string out
                           "
"))))
    out))

;;; Implement mandelbrot_component.wit, whose world declares
;;;   export mandelbrot: func(x0: f64, ..., max-iter: s32) -> string;
;;; -- seven scalar inputs and a string out, the contract this program is checked
;;; against and the export it gets.
(rontolisp:wit-export "mandelbrot_component.wit")


---

# FILE: references/examples/console/mandelbrot.lisp

;;;; Mandelbrot set as ASCII art in rontolisp
;;;; Renders the Mandelbrot set to the terminal using only floating-point
;;;; arithmetic and nested loops -- no transcendental functions -- so it runs
;;;; identically on all three backends (interpreter / JVM / WASM). The iteration
;;;; cap is threaded through as an argument, which is also idiomatic CL; the
;;;; JVM/WASM compilers can equally read a global special variable from inside a
;;;; function body.
;;;;
;;;; Run:
;;;;   java -jar target/rontolisp-0.1.0-SNAPSHOT-exec.jar examples/console/mandelbrot.lisp
;;;;   java -jar ...-exec.jar examples/console/mandelbrot.lisp -o Prog.class && java Prog
;;;;   java -jar ...-exec.jar examples/console/mandelbrot.lisp -o mandelbrot.wasm && wasmtime run -W gc mandelbrot.wasm

;;; Escape time for the complex point (cx, cy): the number of iterations of
;;; z <- z^2 + c before |z| > 2 (i.e. |z|^2 > 4), capped at `max-iter`.
(defun escape-time (cx cy max-iter)
  (let ((x 0.0) (y 0.0) (i 0))
    (while (and (< i max-iter) (<= (+ (* x x) (* y y)) 4.0))
      (let ((xt (+ (- (* x x) (* y y)) cx)))
        (setq y (+ (* 2.0 (* x y)) cy))
        (setq x xt))
      (setq i (+ i 1)))
    i))

;;; Map an escape time to a shading character: dense for points that stay
;;; bounded ("inside"), sparse for points that escape quickly.
(defun shade (i max-iter)
  (cond ((>= i max-iter) "#") ((>= i 10) "+") ((>= i 5) ".") (t " ")))

;;; Render the region [x0,x1] x [y0,y1] as a cols x rows grid of characters.
(defun mandelbrot (x0 x1 y0 y1 cols rows max-iter)
  (let ((dx (/ (- x1 x0) cols)) (dy (/ (- y1 y0) rows)))
    (dotimes (r rows)
      (let ((cy (+ y0 (* dy r))))
        (dotimes (c cols)
          (princ (shade (escape-time (+ x0 (* dx c)) cy max-iter) max-iter)))
        (terpri)))))

(defparameter *max-iter* 30)
(format t "Mandelbrot set (~d iterations):~%" *max-iter*)
(mandelbrot -2.5 1.0 -1.2 1.2 70 30 *max-iter*)


---

# FILE: references/examples/console/mandelbrot_component.wit

package root:component;

world root {
  export mandelbrot: func(x0: f64, x1: f64, y0: f64, y1: f64, cols: s32, rows: s32, max-iter: s32) -> string;
}


---

# FILE: references/examples/console/nqueens.lisp

;;;; N-Queens solver in rontolisp
;;;; Enumerates every solution to the N-Queens problem and prints one board,
;;;; using plain recursion and backtracking over lists. Written functionally
;;;; (each function takes the board size `n` as an argument) so it compiles on
;;;; all three backends -- the JVM/WASM compilers cannot yet read a global
;;;; special variable from inside a function body. Integer/list only.
;;;;
;;;; Run:
;;;;   java -jar target/rontolisp-0.1.0-SNAPSHOT-exec.jar examples/console/nqueens.lisp
;;;;   java -jar ...-exec.jar examples/console/nqueens.lisp -o Prog.class && java Prog
;;;;   java -jar ...-exec.jar examples/console/nqueens.lisp -o nqueens.wasm && wasmtime run -W gc nqueens.wasm

;;; A partial placement is a list of column positions, one per already-placed
;;; row, most recent row at the head. The queen `d` rows back sits at (nth (- d 1)
;;; cols). Can a new queen go in `col` without being attacked by one in `cols`?
(defun safep (col cols)
  (let ((ok t) (d 1) (rest cols))
    (while rest
      (let ((c (car rest)))
        (when (or (= c col) (= (abs (- c col)) d)) (setq ok nil)))
      (setq d (+ d 1))
      (setq rest (cdr rest)))
    ok))

;;; All complete boards reachable from `cols` (which already has `row` queens),
;;; as a list of solutions; each solution is a list of column positions by row.
(defun solve (n row cols)
  (if (= row n)
      (list (reverse cols))
      (let ((acc nil) (col 0))
        (while (< col n)
          (when (safep col cols)
            (setq acc (append acc (solve n (+ row 1) (cons col cols)))))
          (setq col (+ col 1)))
        acc)))

(defun queens (n) (solve n 0 nil))

;;; Print a board given a list of column positions, one per row.
(defun print-board (n cols)
  (dolist (c cols)
    (dotimes (i n) (princ (if (= i c) "Q " ". ")))
    (terpri)))

(defparameter *n* 6)
(format t "Solving ~d-Queens...~%" *n*)
(let ((sols (queens *n*)))
  (format t "Total solutions: ~d~%~%" (length sols))
  (format t "First solution found:~%")
  (print-board *n* (car sols)))


---

# FILE: references/examples/console/parse-numbers.lisp

;;;; Numeric-column + character-classification demo in rontolisp
;;;; Writes a small data file, reads it back line by line, turns each line into
;;;; an integer with parse-integer, and reports the count, sum, min and max.
;;;; Then classifies the characters of a string with the character predicates.
;;;; Uses with-open-file, read-line, parse-integer, char, alpha-char-p,
;;;; digit-char-p -> runs on all three backends (interpreter / JVM / WASM).
;;;;
;;;; NOTE: the WASM backend needs a preopened directory for file access, and
;;;; with-open-file rides the exception-handling proposal (wasmtime 37+):
;;;;   wasmtime run -W gc -W exceptions=y --dir . parse-numbers.wasm
;;;;
;;;; Run:
;;;;   java -jar target/rontolisp-0.1.0-SNAPSHOT-exec.jar examples/console/parse-numbers.lisp
;;;;   java -jar ...-exec.jar examples/console/parse-numbers.lisp -o ParseNumbers.class && java ParseNumbers
;;;;   java -jar ...-exec.jar examples/console/parse-numbers.lisp -o pn.wasm && wasmtime run -W gc -W exceptions=y --dir . pn.wasm

(defparameter *data* "numbers.txt")

;;; Create a small input file so the example is self-contained.
(with-open-file (out *data* :direction :output)
  (write-line "10" out)
  (write-line "25" out)
  (write-line "3" out)
  (write-line "42" out)
  (write-line "17" out))

;;; Fold the integers in `in` (one per line) into (count sum min max).
(defun summarize (in)
  (let ((count 0) (sum 0) (lo nil) (hi nil) (line (read-line in)))
    (while line
      (let ((n (parse-integer line)))
        (setq count (+ count 1))
        (setq sum (+ sum n))
        (when (or (null lo) (< n lo)) (setq lo n))
        (when (or (null hi) (> n hi)) (setq hi n)))
      (setq line (read-line in)))
    (list count sum lo hi)))

(with-open-file (in *data*)
  (let ((stats (summarize in)))
    (format t "count=~d sum=~d min=~d max=~d~%" (first stats) (second stats)
            (third stats) (fourth stats))))

;;; Character classification: count the letters and digits in a string by
;;; indexing it with `char` and testing each character.
(defun count-kinds (s)
  (let ((i 0) (n (length s)) (letters 0) (digits 0))
    (while (< i n)
      (let ((c (char s i)))
        (when (alpha-char-p c) (setq letters (+ letters 1)))
        (when (digit-char-p c) (setq digits (+ digits 1))))
      (setq i (+ i 1)))
    (list letters digits)))

(let ((kinds (count-kinds "rontolisp 0.1 (2026)")))
  (format t "letters=~d digits=~d~%" (first kinds) (second kinds)))


---

# FILE: references/examples/console/roman.lisp

;;;; Roman numeral encoder and decoder in rontolisp
;;;; Demonstrates association lists, string concatenation, recursion -- and an
;;;; example that CHECKS ITSELF: the 1..3999 round-trip is a rove assertion, so
;;;; a broken encoder fails the run instead of printing a line nobody reads.
;;;; Runs on all three backends (interpreter / JVM / WASM).
;;;;
;;;; rove is loaded with asdf, so pass the directories holding its .asd files
;;;; (rove, dissect and cl-ppcre, all vendored in this repository) with
;;;; --system-path; outside this repository (ql:quickload "rove") fetches the
;;;; same sources instead. The compile paths splice the system in at compile
;;;; time, so the produced class / module is self-contained. See the Testing
;;;; guide: doc/en/guides/testing.md
;;;;
;;;; Run:
;;;;   SP=src/test/resources/rove:src/test/resources/dissect:src/test/resources/cl-ppcre
;;;;   rontolisp examples/console/roman.lisp --system-path $SP
;;;;   rontolisp examples/console/roman.lisp --system-path $SP -o Roman.class && java Roman
;;;;   rontolisp examples/console/roman.lisp --system-path $SP -o roman.wasm && \
;;;;     wasmtime run -W gc -W exceptions=y roman.wasm

(asdf:load-system :rove)
(use-package :rove)
;; rove colors its report for a terminal; a checked pipeline wants plain text.
(setf *enable-colors* nil)

;;; Mapping of integer values to Roman numeral strings, sorted descending.
(defparameter *roman-values*
  (list (cons 1000 "M") (cons 900 "CM") (cons 500 "D") (cons 400 "CD")
        (cons 100 "C") (cons 90 "XC") (cons 50 "L") (cons 40 "XL") (cons 10 "X")
        (cons 9 "IX") (cons 5 "V") (cons 4 "IV") (cons 1 "I")))

;;; Mapping of Roman numeral characters to integer values.
(defparameter *roman-char-table*
  (list (cons #\M 1000) (cons #\D 500) (cons #\C 100) (cons #\L 50)
        (cons #\X 10) (cons #\V 5) (cons #\I 1)))

(defun integer-to-roman (n)
  "Convert an integer (1-3999) to a Roman numeral string."
  (when (or (< n 1) (> n 3999))
    (error "INTEGER-TO-ROMAN: ~d is out of range (1-3999)" n))
  (let ((result ""))
    (dolist (pair *roman-values*)
      (let ((value (car pair)) (numeral (cdr pair)))
        (while (>= n value)
          (setq result (concatenate 'string result numeral))
          (setq n (- n value)))))
    result))

(defun roman-to-integer (s)
  "Convert a Roman numeral string to an integer.
   Algorithm: scan right-to-left; add if current >= last, else subtract."
  (let ((result 0) (last 0) (i (1- (length s))))
    (while (>= i 0)
      (let ((ch (char-upcase (char s i))))
        (setq i (1- i))
        (let ((current (cdr (assoc ch *roman-char-table*))))
          (when (null current)
            (error "ROMAN-TO-INTEGER: invalid character ~a" ch))
          (if (>= current last)
              (setq result (+ result current))
              (setq result (- result current)))
          (setq last current))))
    result))

;;; --- the demonstration ------------------------------------------------------

(format t "Integer -> Roman:~%")
(dolist (n '(1 4 9 14 42 99 399 400 999 1999 2024 3999))
  (format t "  ~4d -> ~a~%" n (integer-to-roman n)))

(format t "~%Roman -> Integer:~%")
(dolist (s '("I" "IV" "IX" "XIV" "XLII" "XCIX" "CMXCIX" "MCMXCIX" "MMXXIV"))
  (format t "  ~-6s -> ~a~%" s (roman-to-integer s)))

(terpri)

;;; --- the assertions ---------------------------------------------------------

(deftest encoding
  (testing "the subtractive pairs and the extremes"
    (ok (string= (integer-to-roman 1) "I"))
    (ok (string= (integer-to-roman 4) "IV"))
    (ok (string= (integer-to-roman 400) "CD"))
    (ok (string= (integer-to-roman 3999) "MMMCMXCIX")))
  (testing "out of range"
    (ok (signals (integer-to-roman 0)))
    (ok (signals (integer-to-roman 4000)))))

(deftest decoding
  (testing "a smaller numeral before a larger one subtracts"
    (ok (= (roman-to-integer "MCMXCIX") 1999))
    (ok (= (roman-to-integer "XLII") 42)))
  (testing "lowercase is accepted" (ok (= (roman-to-integer "mmxxiv") 2024)))
  (testing "an unknown character is an error"
    (ok (signals (roman-to-integer "MMZ")))))

;;; The whole point of the pair: every value in range survives the trip out and
;;; back. Reported as ONE assertion carrying the first counter-example, so a
;;; broken encoder names the value it broke on instead of printing 3999 lines.
(deftest round-trip
  (testing "every integer 1..3999 encodes and decodes back to itself"
    (let ((mismatch nil))
      (dotimes (i 3999)
        (let ((n (1+ i)))
          (when (and (null mismatch)
                     (/= n (roman-to-integer (integer-to-roman n))))
            (setq mismatch n))))
      (ok (null mismatch)
          (if mismatch
              (format nil "first mismatch at ~d" mismatch)
              "all 3999 round-trips")))))

;;; Loading this file runs its suite (rove's file-driven entry point), and the
;;; exit code is the verdict -- which is what makes a broken example fail a
;;; build instead of scrolling past.
(uiop:quit (if (run-suite *package*) 0 1))


---

# FILE: references/examples/console/sieve.lisp

;;;; Sieve of Eratosthenes in rontolisp
;;;; Uses a boolean array as the sieve, demonstrating make-array, aref,
;;;; (setf (aref ...)), and list accumulation.
;;;; Pure list/array/number code -> runs on all three backends.
;;;;
;;;; Run:
;;;;   rontolisp examples/console/sieve.lisp
;;;;   rontolisp examples/console/sieve.lisp -o Sieve.class && java Sieve
;;;;   rontolisp examples/console/sieve.lisp -o sieve.wasm && wasmtime run -W gc sieve.wasm

(defun sieve-of-eratosthenes (limit)
  "Return a list of all primes up to LIMIT using the Sieve of Eratosthenes."
  (let ((sieve (make-array (1+ limit) :initial-element t)) (primes nil))
    (dotimes (i (1+ limit) (reverse primes))
      (when (and (>= i 2) (aref sieve i))
        (push i primes)
        ;; Mark all multiples of i starting from i*i
        (let ((j (* i i)))
          (while (<= j limit)
            (setf (aref sieve j) nil)
            (setq j (+ j i))))))))

(defun take (n lst)
  "Return the first N elements of LST."
  (if (or (<= n 0) (null lst)) nil (cons (car lst) (take (1- n) (cdr lst)))))

(defun prime-factorize (n)
  "Return the prime factorization of N as a list of (prime . exponent) pairs."
  (let ((primes (sieve-of-eratosthenes (ceiling (sqrt n))))
        (factors nil)
        (remaining n))
    (dolist (p primes)
      (let ((count 0))
        (while (= (mod remaining p) 0)
          (setq remaining (/ remaining p))
          (setq count (1+ count)))
        (when (> count 0) (push (cons p count) factors))))
    (when (> remaining 1) (push (cons remaining 1) factors))
    (reverse factors)))

(format t "Primes up to 50:~%  ~a~%" (sieve-of-eratosthenes 50))
(format t "~%First 20 primes:~%  ~a~%" (take 20 (sieve-of-eratosthenes 80)))
(format t "~%Prime factorizations:~%")
(dolist (n '(60 100 315 256)) (format t "  ~4d = ~a~%" n (prime-factorize n)))


---

# FILE: references/examples/console/sorting.lisp

;;;; Sorting algorithms in rontolisp
;;;; Hand-written quicksort and merge sort over number lists, each parameterized
;;;; by a comparator passed as a first-class function, and cross-checked against
;;;; the built-in `sort`. Pure list/number code -> runs on all three backends
;;;; (interpreter / JVM / WASM).
;;;;
;;;; Run:
;;;;   java -jar target/rontolisp-0.1.0-SNAPSHOT-exec.jar examples/console/sorting.lisp
;;;;   java -jar ...-exec.jar examples/console/sorting.lisp -o Sorting.class && java Sorting
;;;;   java -jar ...-exec.jar examples/console/sorting.lisp -o sorting.wasm && wasmtime run -W gc sorting.wasm

;;; Quicksort: pivot on the first element, recurse on the two partitions.
;;; `less` is a two-argument comparator (e.g. #'< for ascending order).
(defun quicksort (less lst)
  (if (null lst)
      nil
      (let* ((pivot (car lst))
             (rest (cdr lst))
             (smaller (remove-if-not (lambda (x) (funcall less x pivot)) rest))
             (larger (remove-if (lambda (x) (funcall less x pivot)) rest)))
        (append (quicksort less smaller) (list pivot)
                (quicksort less larger)))))

;;; Merge two already-sorted lists under `less`.
(defun merge2 (less a b)
  (cond ((null a) b)
        ((null b) a)
        ((funcall less (car a) (car b)) (cons (car a) (merge2 less (cdr a) b)))
        (t (cons (car b) (merge2 less a (cdr b))))))

;;; Front `n` elements of a list.
(defun take (n lst)
  (if (or (= n 0) (null lst)) nil (cons (car lst) (take (- n 1) (cdr lst)))))

;;; Merge sort: split in half (using ash for portable integer halving), sort
;;; each half, then merge.
(defun merge-sort (less lst)
  (let ((n (length lst)))
    (if (<= n 1)
        lst
        (let* ((half (ash n -1))
               (front (take half lst))
               (back (nthcdr half lst)))
          (merge2 less (merge-sort less front) (merge-sort less back))))))

(defparameter *data* '(5 -3 8 -1 2 -7 4 0 6))

(format t "input:               ~a~%" *data*)
(format t "quicksort   (<):     ~a~%" (quicksort #'< *data*))
(format t "merge-sort  (<):     ~a~%" (merge-sort #'< *data*))
(format t "built-in sort (<):   ~a~%" (sort (copy-list *data*) #'<))
(format t "quicksort |x| desc:  ~a~%"
        (quicksort (lambda (a b) (> (abs a) (abs b))) *data*))


---

# FILE: references/examples/console/word-frequency.lisp

;;;; Word frequency counter in rontolisp
;;;; Demonstrates hash tables, string manipulation, sorting, and
;;;; higher-order functions. Runs on all three backends.
;;;;
;;;; Run:
;;;;   rontolisp examples/console/word-frequency.lisp
;;;;   rontolisp examples/console/word-frequency.lisp -o WordFreq.class && java WordFreq
;;;;   rontolisp examples/console/word-frequency.lisp -o word-frequency.wasm && wasmtime run -W gc word-frequency.wasm

(defun take (n lst)
  "Return the first N elements of LST."
  (if (or (<= n 0) (null lst)) nil (cons (car lst) (take (1- n) (cdr lst)))))

(defun string-lessp (a b)
  "Lexicographic string comparison using char<."
  (let ((len-a (length a)) (len-b (length b)) (j 0) (result nil) (decided nil))
    ;; Compare character by character
    (while (and (not decided) (< j len-a) (< j len-b))
      (when (char< (char a j) (char b j))
        (setq result t)
        (setq decided t))
      (when (and (not decided) (char< (char b j) (char a j)))
        (setq result nil)
        (setq decided t))
      (setq j (1+ j)))
    ;; If all compared characters are equal, shorter string is less
    (when (not decided) (setq result (< len-a len-b)))
    result))

(defun word-frequency (text)
  "Count word occurrences in TEXT, returning a hash table of word -> count."
  (let ((freq (make-hash-table)) (len (length text)) (pos 0) (done nil))
    (while (and (not done) (< pos len))
      ;; Skip non-alpha characters
      (while (and (< pos len) (not (alpha-char-p (char text pos))))
        (setq pos (1+ pos)))
      (when (>= pos len) (setq done t))
      (when (not done)
        ;; Read word start
        (let ((start pos))
          (while (and (< pos len) (alpha-char-p (char text pos)))
            (setq pos (1+ pos)))
          (let ((word (string-downcase (subseq text start pos))))
            (setf (gethash word freq) (1+ (or (gethash word freq) 0)))))))
    freq))

(defun top-words (freq-table &optional (n 10))
  "Return the N most frequent words from FREQ-TABLE as a sorted list of (word . count) pairs."
  (let ((pairs nil))
    (maphash (lambda (word count) (push (cons word count) pairs)) freq-table)
    (setq pairs (sort pairs (lambda (a b) (> (cdr a) (cdr b)))))
    (take n pairs)))

(defparameter *sample-text*
  "To be or not to be that is the question
Whether 'tis nobler in the mind to suffer
The slings and arrows of outrageous fortune
Or to take arms against a sea of troubles
And by opposing end them")

(format t "Word frequency analysis:~%~%")
(let ((freq (word-frequency *sample-text*)))
  (format t "Total unique words: ~d~%" (hash-table-count freq))
  (format t "~%Top 10 words:~%")
  (let ((rank 1))
    (dolist (pair (top-words freq 10))
      (format t "  ~2d. ~-12s ~d~%" rank (car pair) (cdr pair))
      (setq rank (1+ rank))))
  (format t "~%Words appearing exactly once:~%")
  (let ((singletons nil))
    (maphash (lambda (word count) (when (= count 1) (push word singletons)))
             freq)
    (format t "  ~a~%" (sort (copy-list singletons) #'string-lessp))))


---

# FILE: references/examples/count-vowels/README.md

# count-vowels -- sharing a string with a host through Wasm memory

The rontolisp counterpart of the classic "share a string through Wasm memory"
host tutorial: a module receives a string by pointer and returns the vowel
count. Here it is written in Lisp ([`count-vowels.lisp`](count-vowels.lisp)) and
driven from Node and from a pure-Java [Endive](https://endive.run) host
([`CountVowels.java`](src/main/java/CountVowels.java)).

WebAssembly understands only integers and floats, so a string crosses as a
`(pointer, length)` pair of raw UTF-8 bytes in the module's linear memory. That
boundary's type is not written in the Lisp but in WIT
([`count_vowels_component.wit`](count_vowels_component.wit)):

```wit
package root:component;

world root {
  export count-vowels: func(s: string) -> s32;
}
```

and the Lisp says only *I implement that world*:

```lisp
(rontolisp:wit-export "count_vowels_component.wit")
```

The compiler reads the world, checks every export against the program's
`defun`s — name, arity, parameter and result types — and lowers each into the
export it stands for. Nothing states the signature twice, so nothing can drift:

```console
count_vowels_component.wit:4: export 'count-vowels' declares 1 parameter(s), but (defun count-vowels ...) takes 2
```

`__ronto_alloc` is a bump allocator with no `dealloc`, so the interesting
question is **who reclaims that memory** in a host that keeps one instance alive
and calls it in a loop. The same source answers it two ways:

| Build | Who frees | Host work |
|---|---|---|
| [`--no-gc`](#1---no-gc-the-host-pops-the-heap) | the host, with `__ronto_alloc_mark`/`_reset` | alloc, write, call, reset |
| [`--no-gc --component`](#2-component-model-the-canonical-abi-does-everything) | the canonical ABI + `post-return`, every call | none — pass a JS string |

`count-vowels` is a pure loop over characters, so it fits the
[non-GC subset](https://github.com/making/rontolisp/blob/develop/examples/doc/en/guides/wasm-nogc.md#eligible-subset) and both builds
are `--no-gc`: a sub-kilobyte module that runs on **any** engine. A function
needing the full language compiles to a wasm-GC core module instead
(`--no-wasi`); the memory boundary is exactly case 1's, since the engine's GC
collects the Lisp values but never the host's input buffer.

## Build the two modules

```bash
JAR=../../target/rontolisp-0.1.0-SNAPSHOT-exec.jar   # ./mvnw clean package first

java -jar $JAR count-vowels.lisp -o count_vowels.wasm           --no-gc --optimize
java -jar $JAR count-vowels.lisp -o count_vowels_component.wasm --no-gc --component --optimize --emit-wit
```

## 1. `--no-gc`: the host pops the heap

A plain MVP module — no wasm-GC, no WASI imports, so any engine instantiates it
with an empty import object. It exports:

```
count-vowels       : (i32 ptr, i32 len) -> i32      the vowel count
__ronto_alloc      : (i32 size)         -> i32 ptr  bump allocator
__ronto_alloc_mark : ()                 -> i32 mark heap-top snapshot (arena API)
__ronto_alloc_reset: (i32 mark)         -> ()       restore the heap top (arena API)
memory                                              the linear memory to write into
```

Snapshot the heap top *before* allocating the input, pop back to it *after*
reading the result: the input buffer and everything the call allocated
internally are reclaimed, so a resident instance stays flat.

```bash
node -e '(async () => {
  const ex = (await WebAssembly.instantiate(
    require("fs").readFileSync("count_vowels.wasm"), {})).instance.exports;
  const enc = new TextEncoder();
  const countVowels = (s) => {
    const b = enc.encode(s);
    const mark = ex.__ronto_alloc_mark();          // snapshot BEFORE allocating
    const ptr = ex.__ronto_alloc(b.length);
    new Uint8Array(ex.memory.buffer, ptr, b.length).set(b);
    const n = ex["count-vowels"](ptr, b.length);   // scalar result, read out here
    ex.__ronto_alloc_reset(mark);                  // pop input + call scratch
    return n;
  };
  console.log("\"Hello, World!\" has", countVowels("Hello, World!"), "vowels");
  const before = ex.memory.buffer.byteLength;
  for (let i = 0; i < 100000; i++) countVowels("Hello, World! " + i);
  console.log("resident 100000 calls: memory", before, "->", ex.memory.buffer.byteLength);
})()'
# "Hello, World!" has 3 vowels
# resident 100000 calls: memory 65536 -> 65536
```

Drop the two arena lines and the same loop grows linear memory without bound. Two caveats — the
arena is a manual stack, not a collector:

- Only reset to a mark taken **before** everything still live.
- For a `:string`-**returning** export, read the returned bytes out **before**
  resetting; resetting first frees the string.

### The same thing from Java (Endive)

This directory is a self-contained Maven project depending on
`run.endive:runtime`. Build the compiler jar once from the repository root, then
one Maven command compiles the module, the host, and runs it:

```bash
./mvnw -q clean package -DskipTests    # from the repository root
mvn -q compile exec:java               # here; `compile` also builds count_vowels.wasm
# "Hello, World!" has 3 vowels
# resident 100000 calls: memory 1 -> 1 pages
```

[`CountVowels.java`](src/main/java/CountVowels.java) is the Node host line for
line, and `instance.memory().pages()` stays constant across the loop. Pass a
different word with `-Dexec.args=Programming` (a single token — `exec:java`
splits on whitespace).

## 2. Component model: the canonical ABI does everything

With `--no-gc --component` the output is a component whose export is typed
`func(s: string) -> s32`. The canonical string ABI lowers the host's string into
the module's memory through a `cabi_realloc` the compiler emits, and the
generated `post-return` resets the bump heap after every call — so the host does
no allocation, no writing into `memory`, no arena bookkeeping:

```bash
npx -y @bytecodealliance/jco transpile count_vowels_component.wasm -o dist

node --input-type=module -e '
import { countVowels } from "./dist/count_vowels_component.js";
console.log("\"Hello, World!\" has", countVowels("Hello, World!"), "vowels");
'
# "Hello, World!" has 3 vowels
```

The kebab-case export surfaces in JS as `countVowels`. `wasmtime` runs it with
no flags at all:

```bash
wasmtime run --invoke 'count-vowels("Hello, World!")' count_vowels_component.wasm
# 3
```

`--emit-wit` prints the component's own type back out, and here the file comes
back byte-for-byte unchanged, parameter name `s` included:

```bash
git diff --exit-code count_vowels_component.wit && echo "the component IS the world"
```

Be precise about what that proves. The export line *cannot* come out disagreeing
with the world: the world produced the export directive, which produced the
component's function type, which is what gets printed back. The diff is a
regression check on rontolisp's type mapping, not on this program — what catches
a drifted program is `wit-export`, and it already fired at compile time.

What makes the round trip *byte*-exact is `--no-gc`: an adapter-free reactor
imports nothing, so the component's whole type is the one export. Drop it and
the same source builds a wasm-GC component whose real type runs to ~150 lines —
ten `wasi:*` imports and `export wasi:cli/run` wrapped around the same
`count-vowels`. Those imports are the half a hand-written world never states,
and `--emit-wit` is the only thing that reports them.

`jco transpile` read the types straight out of the `.wasm`, but the `.wit` is
that same contract **without the binary**: hand it to anyone generating bindings
from WIT — a wit-bindgen host embedding, or `jco types` for just the TypeScript
signatures — with no introspection step.

Endive cannot run this one yet (WASIp2 / the component model is ongoing work
there), so the Java host stays on the `--no-gc` core module.

## The Lisp is portable

`count-vowels` is an ordinary pure function, so the same source runs on the
interpreter, the JVM and the wasm-GC backend as a normal program. The
`wit-export` directive exports nothing outside a WASM build, but it is never
inert: every backend still checks the program against the world, so a plain
`java -jar $JAR count-vowels.lisp` catches a drifted `.wit` without compiling
anything.


---

# FILE: references/examples/count-vowels/count-vowels.lisp

;;;; count-vowels -- share a string with a host through WebAssembly memory
;;;;
;;;; The rontolisp counterpart of the classic "share a string through Wasm
;;;; memory" host tutorial (see Endive's memory guide,
;;;; https://endive.run/docs/core/memory), where a count_vowels.wasm receives a
;;;; string by pointer and returns the vowel count. Here the module is written in
;;;; Lisp instead, against a WIT world that types the export
;;;; `count-vowels: func(s: string) -> s32`.
;;;;
;;;; Wasm only speaks integers and floats, so a string crosses the boundary as a
;;;; (pointer, length) pair of raw UTF-8 bytes in the module's linear memory. The
;;;; module therefore exports its `memory` plus a bump allocator
;;;; `__ronto_alloc(size)`: the host reserves space, writes the bytes there, then
;;;; calls `count-vowels(ptr, len)`. This is exactly the alloc / writeString /
;;;; call flow of the tutorial.
;;;;
;;;; A bump allocator never frees, so the real question is who reclaims that
;;;; memory in a host that keeps one instance alive and calls it in a loop. The
;;;; same source answers it two ways, depending on how it is compiled --
;;;; README.md drives both from Node:
;;;;
;;;;   --no-gc --optimize              the host pops the bump heap itself, with
;;;;                                   the arena API __ronto_alloc_mark/_reset
;;;;   --no-gc --component --optimize  a component: the canonical ABI passes the
;;;;                                   string and post-return frees everything --
;;;;                                   the host writes no memory code at all
;;;;
;;;; The export is not described here at all: count_vowels_component.wit is, and
;;;; `rontolisp:wit-export` at the bottom of this file says "this program
;;;; implements that world". The compiler reads the .wit, checks every export it
;;;; declares against the defuns below -- name, arity, parameter and result types
;;;; -- and lowers each one into the export directive it stands for. A drifted
;;;; contract is a compile error naming the WIT file and line instead of a
;;;; wasmtime --invoke failure, and `--emit-wit` regenerates the very file it was
;;;; handed. The world names the export count-vowels, in lower-kebab-case as a
;;;; component-model export name must be, which lets one directive serve both
;;;; builds.

;;; A character is its code point everywhere (in --no-gc a character simply IS
;;; its i64 code, so char= is an ordinary numeric comparison). Test both cases so
;;; the count is case-insensitive.
(defun vowelp (c)
  (or (char= c #\a) (char= c #\e) (char= c #\i) (char= c #\o) (char= c #\u)
      (char= c #\A) (char= c #\E) (char= c #\I) (char= c #\O) (char= c #\U)))

;;; Count the vowels in s by walking it character by character. Pure compute:
;;; no cons, list, hash or I/O, so it stays inside the --no-gc subset.
(defun count-vowels (s)
  (let ((n 0))
    (dotimes (i (length s)) (when (vowelp (char s i)) (setq n (+ n 1))))
    n))

;;; Implement count_vowels_component.wit, whose world declares
;;;   export count-vowels: func(s: string) -> s32;
;;; -- the contract this program is checked against, and the exports it gets.
(rontolisp:wit-export "count_vowels_component.wit")


---

# FILE: references/examples/count-vowels/count_vowels_component.wit

package root:component;

world root {
  export count-vowels: func(s: string) -> s32;
}


---

# FILE: references/examples/count-vowels/pom.xml



	4.0.0

	am.ik.rontolisp.examples
	count-vowels
	0.1.0-SNAPSHOT
	jar

	rontolisp count-vowels host (Endive)
	
		A pure-Java Endive host that shares a string with a rontolisp Wasm module through
		linear memory (the counterpart of Endive/Chicory's "share data" tutorial):
		CountVowels drives the --no-gc module with the host arena API.
	

	
		17
		UTF-8
		1.0.1
		CountVowels
		
		${project.basedir}/../../target/rontolisp-0.1.0-SNAPSHOT-exec.jar
	

	
		
		
			run.endive
			runtime
			${endive.version}
		
	

	
		
			
				org.codehaus.mojo
				exec-maven-plugin
				3.5.0
				
					
					
						compile-wasm-nogc
						generate-resources
						
							exec
						
						
							java
							
								-jar
								${rontolisp.jar}
								${project.basedir}/count-vowels.lisp
								-o
								${project.basedir}/count_vowels.wasm
								--no-gc
								--optimize
							
						
					
				
				
				
					${exec.mainClass}
				
			
		
	




---

# FILE: references/examples/count-vowels/src/main/java/CountVowels.java

// The rontolisp equivalent of Endive's "sharing data through memory" tutorial
// (Endive is the successor to Chicory; see https://endive.run). Instead of a Rust
// count_vowels.wasm, the module here is compiled from count-vowels.lisp with
// --no-gc (a plain MVP module: no wasm-GC, no WASI imports), so the pure-Java
// Endive runtime instantiates it with no import object at all.
//
// Wasm only understands integers and floats, so we pass the string across the
// boundary as a pointer/length pair of raw UTF-8 bytes in the module's linear
// memory. The module exports its `memory` and a bump allocator
// `__ronto_alloc(size)` for exactly this: reserve space, write the bytes, then
// call `count-vowels(ptr, len)`.
//
// A bump allocator never frees, so every call is bracketed with the module's
// arena API -- `__ronto_alloc_mark()` snapshots the heap top BEFORE the input
// buffer is allocated, `__ronto_alloc_reset(mark)` pops back to it AFTER the
// result has been read. That reclaims the input buffer and everything the call
// allocated internally, so a resident instance stays flat forever: the loop below
// runs 100000 calls and asserts memory().pages() never grows.
//
// Build the compiler jar once (from the repo root), then run this host with Maven:
//   ./mvnw -q clean package -DskipTests
//   mvn -q compile exec:java -Dexec.args="Hello, World!"
// `mvn compile` also builds count_vowels.wasm from count-vowels.lisp (see pom.xml).

import run.endive.runtime.ExportFunction;
import run.endive.runtime.Instance;
import run.endive.runtime.Memory;
import run.endive.wasm.Parser;

import java.io.File;
import java.nio.charset.StandardCharsets;

public class CountVowels {

	// The module exports the host needs: the entry point, the bump allocator and the
	// arena API over it, plus the linear memory to write the input into.
	record Module(ExportFunction countVowels, ExportFunction alloc, ExportFunction mark, ExportFunction reset,
			Memory memory) {

		static Module of(Instance instance) {
			return new Module(instance.export("count-vowels"), instance.export("__ronto_alloc"),
					instance.export("__ronto_alloc_mark"), instance.export("__ronto_alloc_reset"), instance.memory());
		}

		// One call across the :string boundary, bracketed by the arena API.
		long countVowels(String message) {
			byte[] bytes = message.getBytes(StandardCharsets.UTF_8);

			// Snapshot the heap top BEFORE allocating the input buffer.
			long snapshot = this.mark.apply()[0];

			// Reserve {len} bytes of module memory and write the string into it. alloc
			// returns a pointer to that memory; the :string ABI expects the raw UTF-8
			// bytes at that pointer (no length header on the host side).
			int ptr = (int) this.alloc.apply(bytes.length)[0];
			this.memory.write(ptr, bytes);

			// Call count-vowels with the (pointer, length) pair. It reads the bytes back
			// out of linear memory and returns the count.
			long result = this.countVowels.apply(ptr, bytes.length)[0];

			// The result is a scalar and is already read out, so pop the input buffer
			// (and the call's internal string copy) off the bump heap.
			this.reset.apply(snapshot);
			return result;
		}
	}

	public static void main(String[] args) {
		String message = args.length > 0 ? args[0] : "Hello, World!";

		// Instantiate the module. --no-gc emits no imports, so no import object is
		// needed -- the same "instantiate, then call the exports" shape as the
		// Endive tutorial.
		Instance instance = Instance.builder(Parser.parse(new File("count_vowels.wasm"))).build();
		Module module = Module.of(instance);

		long result = module.countVowels(message);
		System.out.println("\"" + message + "\" has " + result + " vowels");
		if (message.equals("Hello, World!") && result != 3L) {
			throw new AssertionError("expected 3 vowels in \"Hello, World!\", got " + result);
		}

		// A resident instance: call 100000 times, each with a fresh input string. The
		// arena bracket inside countVowels() pops every call's allocation, so linear
		// memory never grows.
		int pagesBefore = module.memory().pages();
		for (int i = 0; i < 100_000; i++) {
			module.countVowels("Hello, World! " + i);
		}
		int pagesAfter = module.memory().pages();
		System.out.println("resident 100000 calls: memory " + pagesBefore + " -> " + pagesAfter + " pages");
		if (pagesAfter != pagesBefore) {
			throw new AssertionError("resident loop grew memory from " + pagesBefore + " to " + pagesAfter + " pages");
		}
	}

}


---

# FILE: references/examples/db/README.md

# db

Talking to PostgreSQL from rontolisp using the real upstream libraries
(cl-postgres, the low-level driver, and postmodern on top of it) rather than a
rontolisp-specific binding.

| Program | Description | Upstream |
| --- | --- | --- |
| [`postgres-hello.lisp`](postgres-hello.lisp) | connect and run two select queries |  |
| [`postgres-crud.lisp`](postgres-crud.lisp) | full CRUD cycle: prepared statements (`prepare-query` + `exec-prepared`), an alist row reader, all inside a rolled-back transaction so it is safe to re-run |  |
| [`postmodern-crud.lisp`](postmodern-crud.lisp) | the same cycle one layer up: `with-connection`, statements written as S-SQL s-expressions, `with-transaction`, `defprepared`, and the result formats (`:alists`, `:single`, `:column`) |  |
| [`postmodern-dao.lisp`](postmodern-dao.lisp) | the DAO layer on top: a `(:metaclass pomo:dao-class)` class as the table definition, `dao-table-definition`, `deftable`/`create-table`, and the CRUD cycle as `insert-dao` / `get-dao` / `update-dao` / `upsert-dao` / `select-dao` / `delete-dao` |  |
| [`postgres-web.lisp`](postgres-web.lisp) | notes app: PostgreSQL storage + `rontolisp:http-handler` + cl-who for HTML |  |
| [`bbs-api.lisp`](bbs-api.lisp) | bulletin-board REST API: JSON instead of HTML, routed with tiny-routes and served through Clack — paginated `GET`, a validated `POST`, a `DELETE` that tells 204 from 404, and one error document behind every failure |  |
| [`database-url.lisp`](database-url.lisp) | not a program: the `DATABASE_URL` parser the four above share, `(load)`ed by each of them | — |

## Setup

```bash
docker run --rm -p 54329:5432 -e POSTGRES_HOST_AUTH_METHOD=trust postgres:17-alpine
export DATABASE_URL=postgresql://postgres@127.0.0.1:54329/postgres
```

No program here holds a connection string. Each calls
`(database-url-parts (uiop:getenv "DATABASE_URL"))`, where `database-url-parts`
comes from [`database-url.lisp`](database-url.lisp), pulled in with a top-level
`(load ...)` and so spliced in at compile time — the parser compiles natively on
every backend rather than needing `--dynamic`. It takes a URL and knows nothing
about the environment, so the same five values could come from a config file
instead.

```
postgresql://user:password@host:port/database
```

`postgres://` is an alias. The password, the port (5432) and a trailing
`?sslmode=…` (recognised and dropped — `use-ssl` is not wired up here) are all
optional. `%XX` escapes in the user and password are decoded, so a password
holding an `@` travels as `%40`; a `+` stays a literal plus, this being URI
userinfo rather than a query string. Anything else — an unset variable, a
missing host, a non-numeric port — is an error naming what is wrong, never a
silent default.

## Interpreter / JVM

```bash
rontolisp examples/db/postgres-hello.lisp
rontolisp examples/db/postgres-crud.lisp
rontolisp examples/db/postmodern-crud.lisp
rontolisp examples/db/postmodern-dao.lisp
rontolisp examples/db/postgres-web.lisp   # then open http://127.0.0.1:8080
rontolisp examples/db/bbs-api.lisp        # then curl http://127.0.0.1:8080/api/v1/comments

rontolisp examples/db/postgres-hello.lisp -o Prog.class && java Prog
```

`ql:quickload` pulls md5, split-sequence, ironclad, cl-base64, cl-ppcre, uax-15,
alexandria, (for postmodern) s-sql, uiop and bordeaux-threads, (for the web
app) cl-who and (for the REST API) clack, lack and tiny-routes, downloading them
into `~/.rontolisp/quicklisp` on first run.

## WASM

TCP requires `--component` (WASI 0.3 sockets); Preview 1 has no host socket API.
`--env DATABASE_URL` is what puts the variable inside the component: wasmtime
passes no environment through unless asked. Forget it and the program stops on
`no database URL given` — except that an uncaught error on WASM surfaces as a
bare `unreachable` trap with the message lost, so on that backend the missing
flag looks like a crash rather than a diagnostic.

```bash
rontolisp examples/db/postgres-hello.lisp -o postgres-hello.wasm --component --optimize
wasmtime run -W gc=y -W exceptions=y -S tcp=y -S inherit-network=y --env DATABASE_URL postgres-hello.wasm
```

`postgres-crud.lisp` and `postmodern-crud.lisp` build and run the same way.
`postgres-web.lisp` runs under `wasmtime serve` and needs one more flag
(`-S cli=y`); `--env DATABASE_URL` reaches a served handler exactly like it
reaches the others.

```bash
rontolisp examples/db/postgres-web.lisp -o app.wasm --component --optimize
wasmtime serve -W gc=y -W exceptions=y -S cli=y -S tcp=y -S inherit-network=y --env DATABASE_URL app.wasm
```

`bbs-api.lisp` builds and serves with exactly the same two commands.

wasmCloud runs the same component under `wash dev` with
`wasm_proposals: [gc, exception-handling, component-model-async]`. There is
no `--env` flag there; `.wash/config.yaml` needs a top-level `workload:`
block instead (a sibling of `build:`/`dev:`, not nested under `dev:`):

```yaml
workload:
  environment:
    config:
      DATABASE_URL: postgresql://postgres@192.168.11.76:54329/postgres
```

`workload.environment.config` is inline values; `configFrom` / `secretFrom`
instead name entries of a top-level `configs:` / `secrets:` block, for values
that should not sit in the YAML in the clear. Either way this reaches
`uiop:getenv` unmodified — wash links the same `wasi:cli/environment@0.3.0`
interface `wasmtime serve --env` satisfies.

**Spin**
([canary build](https://github.com/spinframework/spin/releases/tag/canary),
4.1.0-pre0+) runs the same component with no flags at all and serves it on
`:3000`. Its sandbox is deny-by-default, so the manifest carries both the
environment and the database address:

```toml
spin_manifest_version = 2

[application]
name = "rontolisp-postgres-web"
version = "0.1.0"

[[trigger.http]]
route = "/..."
component = "notes"

[component.notes]
source = "app.wasm"
allowed_outbound_hosts = ["tcp://127.0.0.1:54329"]
environment = { DATABASE_URL = "postgresql://postgres@127.0.0.1:54329/postgres" }
```

```bash
rontolisp examples/db/postgres-web.lisp -o app.wasm --component --optimize
spin up
```

`allowed_outbound_hosts` takes `://:`, and the driver's
plain TCP connect is checked under the **`tcp`** scheme. Omit the entry and
every request 500s with the destination named in the log; omit `environment` and
you get the same bare `unreachable` trap `--env`-less wasmtime gives. Unlike
wash, Spin does not virtualize the loopback.

**A served component imports the environment interface itself.** The WASI 0.3
service world carries no `wasi:cli/environment`, so the preview1 bridge answers
the `environ_*` calls with a zero-entry environment — reading `DATABASE_URL`
there is not the bridge's job. A program that calls `uiop:getenv` instead binds
`wasi:cli/environment@0.3.0` directly and the component declares that import
(`wasm-tools component wit app.wasm` lists it, next to `wasi:sockets`), which is
what `wasmtime serve --env DATABASE_URL` then satisfies. A program that never
reads the environment declares nothing extra, so it still runs on any host that
provides only the service world.

## The bulletin-board API

`bbs-api.lisp` serves three endpoints under `/api/v1`:

| Request | Answer |
| --- | --- |
| `POST /comments` with `{"author": ..., "content": ...}` | `201` + the stored comment (`id`, `author`, `content`, `createdAt`) and a `Location` header |
| `GET /comments?page=1&limit=20&sort=newest` | `200` + `data` (the page) and `pagination` (`totalComments`, `totalPages`, `currentPage`, `limit`, `hasNextPage`, `hasPrevPage`) |
| `DELETE /comments/{id}` | `204`, or `404` when no such comment |

```bash
curl -X POST -H 'content-type: application/json' \
     -d '{"author":"alice","content":"hello"}' http://127.0.0.1:8080/api/v1/comments
curl 'http://127.0.0.1:8080/api/v1/comments?page=1&limit=20&sort=oldest'
curl -i -X DELETE http://127.0.0.1:8080/api/v1/comments/1
```

Every failure is the same document — `timestamp`, `status`, `error`, `message`,
`path` — so a client has one shape to parse: `400` for a body that is not a JSON
object with a non-empty `author` and `content`, or a `page`/`limit`/`sort` that
is not one of the values the contract allows (`limit` tops out at 100); `404`
for a comment or a path that is not there; `405`, with an `Allow` header, for a
known path taken with the wrong method; and `500` for anything the handlers did
not foresee, including the database being down.

Three things are worth reading the source for:

- **The sort direction is spliced into the SQL, everything else is a
  parameter.** An `ORDER BY` direction cannot be a placeholder, so `page-comments`
  builds the statement text — which is safe only because the caller has already
  turned `sort` into one of exactly two literals. `author`, `content`, the id and
  the page window all travel as `$n` parameters.
- **The connection is opened by a POST-MATCH middleware.** `wrap-connection` runs
  only once a route has claimed the request by method *and* path, so a request on
  its way to a 404 never opens one. Piping it in the ordinary way would.
- **The response shapes are CLOS classes.** `json-stringify` serializes a
  standard-object slot by slot in definition order, so the class *is* the schema;
  the `|createdAt|` spelling is what keeps the reader from upcasing a camelCase
  key into `createdat`.

## Notes

- **postmodern loads in its MOP build.** The query, transaction and
  prepared-statement layers `postmodern-crud.lisp` shows all work, and so does
  the DAO layer `postmodern-dao.lisp` shows (`:metaclass dao-class`, `get-dao` /
  `select-dao` / `insert-dao` / `upsert-dao`), running on the definition-time
  metaobject subset: DAO classes are top-level `defclass` forms with literal
  options.
- **SCRAM-SHA-256** completes on all backends well within PostgreSQL's default
  60-second `authentication_timeout`. The interpreter runs its 4096-round PBKDF2
  on a native kernel, so the handshake costs milliseconds there; the compiled
  backends run ironclad's own code and finish in seconds. `trust`, `password`
  and `md5` auth are unaffected.
- **TLS is interpreter/JVM only.** A connection that negotiates SSL cannot work
  under WASM; use plain TCP (the default `:no`).
- **wasmCloud addressing.** wash routes loopback to a per-workload virtual
  network, so the host in `DATABASE_URL` must be a non-loopback address there.
  `wasmtime serve` and Spin both use the real loopback, so `127.0.0.1` works.
- **Instance lifetime.** The program's top level runs once per instance, and
  every host draws that line differently. `wasmtime serve` keeps one instance
  for the whole run; wasmCloud gives each request a fresh instance — this is
  why `postgres-web.lisp` uses `create table if not exists` instead of dropping
  and recreating the table. Spin sits between the two: it reuses an instance
  for 128 requests and then retires it (`--max-instance-reuse-count`, and
  `--max-instance-concurrent-reuse-count` — 16 by default — lets that many
  requests share one instance at a time). Write the top level so that all three
  are fine: idempotent, and with durable state in the database rather than in a
  global.
- **Non-ASCII.** cl-who escapes characters above ASCII as numeric character
  references by default; bind `cl-who:*escape-char-p*` to a predicate matching
  only `<>&'"` to emit raw UTF-8 instead.


---

# FILE: references/examples/db/bbs-api.lisp

;; A bulletin-board REST API: PostgreSQL for the comments (the real cl-postgres
;; driver, as in postgres-crud.lisp) and tiny-routes for the routing, served
;; through Clack. Where postgres-web.lisp renders HTML for a browser, this one
;; answers JSON for a client, so the interesting part is not the storage but
;; what a REST contract needs on top of it: a validated query string, a status
;; code per outcome, and ONE error document behind every failure.
;;
;;   POST   /api/v1/comments        {"author": ..., "content": ...} -> 201 + the comment
;;   GET    /api/v1/comments        ?page=1&limit=20&sort=newest    -> 200 + data + pagination
;;   DELETE /api/v1/comments/:id                                    -> 204, or 404
;;
;; Anything else is an error document -- timestamp, status, error, message,
;; path -- with 400 for a request that does not parse, 404 for a comment (or a
;; path) that is not there, 405 for a known path taken with the wrong method,
;; and 500 for whatever the handler did not foresee.
;;
;; "tiny-routes/lite" is the ppcre-free opt-in system; the full "tiny-routes"
;; runs this file unchanged and costs a regex engine.
;;
;; It needs a server, and reads where to find it out of DATABASE_URL
;; (database-url.lisp beside this file parses the URL). The one-liner this
;; example is written against, and the URL that reaches it:
;;   docker run --rm -p 54329:5432 -e POSTGRES_HOST_AUTH_METHOD=trust postgres:17-alpine
;;   export DATABASE_URL=postgresql://postgres@127.0.0.1:54329/postgres
;;   rontolisp examples/db/bbs-api.lisp
;;
;; Runs on the interpreter and on the JVM backend (keep the rontolisp jar on
;; the classpath, it carries the runtime):
;;   rontolisp examples/db/bbs-api.lisp -o BbsApi.class && \
;;     java -cp target/rontolisp-0.1.0-SNAPSHOT-exec.jar:. BbsApi
;; And as a WASI component under wasmtime serve, which both serves the API and
;; lets the driver dial out -- the same flag set postgres-web.lisp needs, -S
;; cli=y included:
;;   rontolisp examples/db/bbs-api.lisp -o bbs-api.wasm --component --optimize
;;   wasmtime serve -W gc=y -W exceptions=y -S cli=y -S tcp=y -S inherit-network=y \
;;     --env DATABASE_URL bbs-api.wasm
;; See README.md for the wasmCloud and Spin manifests, and for why the
;; component reads DATABASE_URL through an interface of its own.
;;
;;   curl -X POST -H 'content-type: application/json' \
;;        -d '{"author":"alice","content":"hello"}' http://127.0.0.1:8080/api/v1/comments
;;   curl 'http://127.0.0.1:8080/api/v1/comments?page=1&limit=20&sort=oldest'
;;   curl -X DELETE -i http://127.0.0.1:8080/api/v1/comments/1

(ql:quickload '("clack" "tiny-routes/lite" "cl-postgres"))

(load "database-url.lisp")

;;; --- the documents -----------------------------------------------------------

;; The response shapes are CLOS classes, not hash tables: json-stringify
;; serializes a standard-object slot by slot IN DEFINITION ORDER, so the class
;; is the schema and the JSON comes out in the order written here. A slot name
;; is down-cased for the key unless it already holds a lower-case letter, which
;; is what the |...| escapes are for -- the reader would otherwise upcase
;; createdAt and the key would ship as "createdat".
(defclass comment ()
  ((id :initarg :id) (author :initarg :author) (content :initarg :content)
   (|createdAt| :initarg :created-at)))

(defclass pagination ()
  ((|totalComments| :initarg :total-comments)
   (|totalPages| :initarg :total-pages) (|currentPage| :initarg :current-page)
   (limit :initarg :limit) (|hasNextPage| :initarg :has-next-page)
   (|hasPrevPage| :initarg :has-prev-page)))

(defclass comment-page ()
  ((data :initarg :data) (pagination :initarg :pagination)))

(defclass api-error ()
  ((timestamp :initarg :timestamp) (status :initarg :status)
   (error :initarg :error) (message :initarg :message) (path :initarg :path)))

(defun json-body (object) (format nil "~a~%" (rontolisp:json-stringify object)))

;;; --- the error document ------------------------------------------------------

(defparameter *reasons*
  '((400 . "Bad Request") (404 . "Not Found") (405 . "Method Not Allowed")
    (500 . "Internal Server Error")))

(defun iso-8601-now ()
  "The current instant as 2025-05-15T10:30:00Z. The zone argument 0 asks for
   GMT, which is the one zone every backend agrees on."
  (multiple-value-bind (second minute hour date month year)
      (decode-universal-time (get-universal-time) 0)
    (format nil "~4,'0d-~2,'0d-~2,'0dT~2,'0d:~2,'0d:~2,'0dZ" year month date
            hour minute second)))

(defun problem (req status message)
  "The one error shape the whole API answers with. Everything that can go
   wrong comes through here, so a client never has to tell two failure
   documents apart."
  (tiny:make-response :status status
                      :body (json-body
                             (make-instance 'api-error
                              :timestamp (iso-8601-now)
                              :status status
                              :error (cdr (assoc status *reasons*))
                              :message message
                              :path (tiny:path-info req)))))

;;; --- storage -----------------------------------------------------------------

(defun connect ()
  "A connection to the database. One per request -- see wrap-connection below."
  (multiple-value-bind (database user password host port)
      (database-url-parts (uiop:getenv "DATABASE_URL"))
    (cl-postgres:open-database database user password host port)))

;; if not exists, not drop + create: this top level runs once per INSTANCE, and
;; how long an instance lives is the host's business -- wasmtime serve keeps one
;; for the whole run, while wasmCloud starts a fresh one per request. A drop
;; here would empty the board on every request there.
(let ((db (connect)))
  (unwind-protect (cl-postgres:exec-query db
                                          "create table if not exists comments (
                                             id bigserial primary key,
                                             author text not null,
                                             content text not null,
                                             created_at timestamptz not null default now())")
    (cl-postgres:close-database db)))

;; The four columns every endpoint returns, in the order the comment class
;; declares them. id is cast to text because the contract says the id is a
;; string, and the timestamp is formatted BY THE SERVER: a `to_char` keeps the
;; ISO-8601 rendering in one place and spares the program a calendar library.
(defparameter *comment-columns*
  "id::text, author, content,
   to_char(created_at at time zone 'UTC', 'YYYY-MM-DD\"T\"HH24:MI:SS\"Z\"')")

(defun row-comment (row)
  "One list-row-reader row as a comment instance."
  (make-instance 'comment
                 :id (first row)
                 :author (second row)
                 :content (third row)
                 :created-at (fourth row)))

(defun insert-comment (db author content)
  "One comment, through the extended query protocol: $1 and $2 are
   placeholders, so what the client sent stays data and never becomes SQL.
   `returning` hands back the row the server actually stored, id and timestamp
   included, which is what the 201 body has to carry."
  (cl-postgres:prepare-query db "insert-comment"
   (concatenate 'string
    "insert into comments (author, content) values ($1, $2) returning "
    *comment-columns*))
  (row-comment
   (first
    (cl-postgres:exec-prepared db "insert-comment" (list author content)
                               'cl-postgres:list-row-reader))))

(defun count-comments (db)
  (first
   (first
    (cl-postgres:exec-query db "select count(*)::int from comments"
                            'cl-postgres:list-row-reader))))

(defun page-comments (db limit offset direction)
  "One page, newest or oldest first. DIRECTION is spliced into the statement
   rather than passed as a parameter -- an ORDER BY direction cannot be one --
   and that is safe here only because it is never the client's text: the caller
   has already turned the sort parameter into one of exactly two literals.
   id breaks the tie so two comments of the same instant keep a total order."
  (cl-postgres:prepare-query db "page-comments"
                             (concatenate 'string "select " *comment-columns*
                                          " from comments order by created_at "
                                          direction ", id " direction
                                          " limit $1 offset $2"))
  (mapcar #'row-comment
          (cl-postgres:exec-prepared db "page-comments" (list limit offset)
                                     'cl-postgres:list-row-reader)))

(defun delete-comment (db id)
  "True when a row was deleted. `returning` is what tells 204 from 404: a
   delete that matched nothing reports no rows rather than an error."
  (cl-postgres:prepare-query db "delete-comment"
                             "delete from comments where id = $1 returning id")
  (and (cl-postgres:exec-prepared db "delete-comment" (list id)
                                  'cl-postgres:list-row-reader) t))

;;; --- reading the request -----------------------------------------------------

(defun positive-integer (text)
  "TEXT as a positive integer, or nil when it is not one. :junk-allowed stops
   parse-integer from signalling on its own terms, so the whole of TEXT has to
   be consumed for the number to count -- a 20x is a typo, not 20."
  (when (stringp text)
    (multiple-value-bind (n end) (parse-integer text :junk-allowed t)
      (when (and n (= end (length text)) (> n 0)) n))))

;; tiny-routes interns a query key with (intern key :keyword) and does NOT
;; upcase it, so ?page=2 arrives under the keyword whose name is the three
;; lower-case letters -- spelled :|page| here, because the reader would upcase
;; a bare :page and the lookup would miss. A path token is the other way round
;; (the matcher upcases it), which is why :commentId below needs no escape.
(defun query-param (req name default)
  (getf (tiny:request-get req :query-parameters) name default))

(defun json-object (text)
  "TEXT as a JSON object, or nil when it is not one. json-parse signals on
   malformed input, so this handler-case is what makes a broken body a 400
   rather than a 500."
  (when (and (stringp text) (> (length text) 0))
    (let ((value (handler-case (rontolisp:json-parse text) (error () nil))))
      (and (hash-table-p value) value))))

(defun non-empty-string-p (value) (and (stringp value) (string/= value "")))

;;; --- the handlers ------------------------------------------------------------

(defun create-comment (req)
  (let* ((body (json-object (tiny:request-body req "")))
         (author (and body (gethash "author" body)))
         (content (and body (gethash "content" body))))
    (cond
     ((null body) (problem req 400 "the request body is not a JSON object"))
     ((not (non-empty-string-p author))
      (problem req 400 "author is required and must be a non-empty string"))
     ((not (non-empty-string-p content))
      (problem req 400 "content is required and must be a non-empty string"))
     (t (let ((created
               (insert-comment (tiny:request-get req :db) author content)))
          ;; 201 + Location: the client learns where the comment it just
          ;; posted now lives.
          (tiny:created
           (concatenate 'string "/api/v1/comments/" (slot-value created 'id))
           (json-body created)))))))

(defun list-comments (req)
  (let* ((page-text (query-param req :|page| "1"))
         (limit-text (query-param req :|limit| "20"))
         (sort-text (query-param req :|sort| "newest"))
         (page (positive-integer page-text))
         (limit (positive-integer limit-text))
         ;; Two literals, chosen here and never taken from the request -- see
         ;; page-comments.
         (direction
          (cond ((string= sort-text "newest") "desc")
                ((string= sort-text "oldest") "asc"))))
    (cond ((null page)
           (problem req 400
            (format nil "page must be a positive integer, not ~s" page-text)))
          ((null limit)
           (problem req 400
            (format nil "limit must be a positive integer, not ~s" limit-text)))
          ((> limit 100)
           (problem req 400
                    (format nil "limit must be 100 or less, not ~a" limit)))
          ((null direction)
           (problem req 400
            (format nil "sort must be newest or oldest, not ~s" sort-text)))
          (t (let* ((db (tiny:request-get req :db))
                    (total (count-comments db))
                    (total-pages (ceiling total limit))
                    ;; A page past the end is not an error: it is an empty page,
                    ;; which is what a client walking the list has to handle
                    ;; anyway.
                    (rows
                     (page-comments db limit (* (- page 1) limit) direction)))
               (tiny:ok
                (json-body
                 (make-instance 'comment-page
                                ;; A vector, not a list: json-stringify renders nil as false,
                                ;; so an empty LIST of comments would ship as "data": false
                                ;; where the empty vector is the "data": [] a client expects.
                                :data (coerce rows 'vector)
                                :pagination
                                (make-instance 'pagination
                                 :total-comments total
                                 :total-pages total-pages
                                 :current-page page
                                 :limit limit
                                 :has-next-page (< page total-pages)
                                 :has-prev-page (> page 1))))))))))

(defun destroy-comment (req)
  (let* ((id-text (tiny:path-parameter req :commentId))
         (id (positive-integer id-text)))
    ;; An id that is not a number is not a bad request, it is a comment that
    ;; does not exist -- the same 404 a well-formed id nobody posted gets.
    (if (and id (delete-comment (tiny:request-get req :db) id))
        (tiny:no-content)
        (problem req 404 (format nil "no comment with id ~s" id-text)))))

(defun method-not-allowed (req allowed)
  "405 naming the methods the path does have, in the header the status is
   defined with."
  (tiny:clone-response
   (problem req 405 (format nil "~a accepts ~a" (tiny:path-info req) allowed))
   :headers (list :allow allowed)))

;;; --- middleware --------------------------------------------------------------

(defun wrap-connection (handler)
  "Middleware: one connection per request, handed to the handler as :db and
   closed even if a query signals. A connection is a single conversation with
   the server, and the interpreter/JVM servers run every request on its own
   thread, so requests that genuinely overlap stay isolated."
  (lambda (req)
    (let ((db (connect)))
      (unwind-protect (funcall handler (tiny:request-append req :db db))
        (cl-postgres:close-database db)))))

(defun wrap-errors (handler)
  "Middleware: whatever the handler did not foresee becomes the same error
   document as everything else. Without it a signalling handler is a dropped
   connection, which tells the client nothing."
  (lambda (req)
    (handler-case (funcall handler req)
      (error (c) (problem req 500 (format nil "~a" c))))))

(defun wrap-json-content-type (handler)
  "Middleware: label every response that HAS a body as JSON. That last clause
   is why this is not tiny:wrap-response-content-type -- a 204 carries no body
   and must not claim a type for it. A nil response is a route DECLINING, and
   passes through untouched so the next route still sees it."
  (lambda (req)
    (let ((res (funcall handler req)))
      (if (and res (tiny:response-body res))
          (tiny:clone-response res
                               :headers (append
                                         (list :content-type "application/json")
                                         (tiny:response-headers res)))
          res))))

;;; --- the routes --------------------------------------------------------------

;; Each route answers the one method it names and declines every other, so a
;; request that reaches the fallbacks below either used the wrong method on a
;; path that exists or asked for a path that does not.
(tiny:define-routes *comment-routes*
  (tiny:define-post "/api/v1/comments" (req) (create-comment req))
  (tiny:define-get "/api/v1/comments" (req) (list-comments req))
  (tiny:define-delete "/api/v1/comments/:commentId" (req)
    (destroy-comment req)))

(tiny:define-routes *fallback-routes*
  (tiny:define-any "/api/v1/comments" (req)
    (method-not-allowed req "GET, POST"))
  (tiny:define-any "/api/v1/comments/:commentId" (req)
    (method-not-allowed req "DELETE"))
  (tiny:define-any "*" (req)
    (problem req 404 (format nil "no resource at ~a" (tiny:path-info req)))))

;; wrap-post-match-middleware, not pipe: a POST-MATCH middleware runs only once
;; a route has claimed the request by method AND path. Piped in the ordinary
;; way, wrap-connection would open a connection for every request that merely
;; passes THROUGH this group on its way to a 404.
(defparameter *routes*
  (tiny:routes (tiny:pipe *comment-routes*
                          (tiny:wrap-post-match-middleware #'wrap-connection))
               *fallback-routes*))

;; pipe reads inside out: the routes are wrapped by wrap-errors first, so the
;; content type and the parsed request reach even a 500.
(defparameter *app*
  (tiny:pipe *routes* (wrap-errors) (wrap-json-content-type)
             (tiny:wrap-request-body) (tiny:wrap-query-parameters)))

(clack:clackup *app* :server :rontolisp :port 8080 :use-thread nil)


---

# FILE: references/examples/db/database-url.lisp

;; The connection details, out of a URL instead of out of the source: every
;; program in this directory calls (database-url-parts (uiop:getenv
;; "DATABASE_URL")). Reading the environment is the caller's business -- this
;; file only knows how to take a URL apart -- and a literal top-level (load
;; "database-url.lisp") is spliced in at compile time, so these defuns compile
;; natively on every backend.
;;
;;   export DATABASE_URL=postgresql://user:password@host:port/database
;;
;; postgres:// is accepted as an alias for postgresql://. The port is optional
;; (5432), so is a trailing query string -- an ?sslmode=..., say, which these
;; examples recognise and drop -- and the password is optional too, which is
;; what the trust-auth server in the README wants. %XX escapes in the user and
;; the password are decoded, so a password holding an @ or a : travels as %40 /
;; %3A. Note that + stays a literal plus: this is URI userinfo, not a query
;; string, which is why rontolisp:url-decode is the wrong tool here.

(defun db-url-hex-value (c)
  "The value of one hexadecimal digit, or nil if C is not one."
  (let ((code (char-code c)))
    (cond ((and (>= code 48) (<= code 57)) (- code 48))
          ((and (>= code 97) (<= code 102)) (- code 87))
          ((and (>= code 65) (<= code 70)) (- code 55))
          (t nil))))

(defun db-url-decode (s)
  "S with its %XX escapes turned back into characters. Anything that is not a
   well-formed escape -- a lone %, a % followed by non-hex -- is left as it
   stands."
  (let ((n (length s)) (out "") (i 0))
    (do ()
        ((>= i n) out)
      (let* ((escapep (and (char= (char s i) #\%) (< (+ i 2) n)))
             (hi (if escapep (db-url-hex-value (char s (+ i 1))) nil))
             (lo (if escapep (db-url-hex-value (char s (+ i 2))) nil))
             (escaped (and hi lo)))
        (setq out
              (concatenate 'string out
               (string (if escaped (code-char (+ (* 16 hi) lo)) (char s i)))))
        (setq i (+ i (if escaped 3 1)))))))

(defun db-url-error (reason url)
  "REASON, the offending URL, and a reminder of the shape one has."
  (error "~a: ~a~%Expected postgresql://user:password@host:port/database" reason
         url))

(defun db-url-part (s empty-reason url)
  "S, or an error naming URL when S is empty. Nothing in a connection URL is
   worth defaulting silently."
  (if (string= s "") (db-url-error empty-reason url) s))

(defun db-url-port (text url)
  "TEXT as a port number. :junk-allowed stops parse-integer from signalling on
   its own terms, so the whole of TEXT has to be consumed for the number to
   count -- a 5432x is a typo, not a port."
  (multiple-value-bind (port end) (parse-integer text :junk-allowed t)
    (if (and port (= end (length text)))
        port
        (db-url-error "the port is not a number" url))))

(defun database-url-parts (url)
  "Splits postgresql://user:password@host:port/database into five values --
   database, user, password, host, port -- which is exactly the argument order
   cl-postgres:open-database takes. The password is nil when URL carries none.
   URL is whatever the caller got hold of, nil included: an unset environment
   variable is an error here rather than a fallback address, because a default
   would be the hardcoded connection this file exists to remove."
  (when (or (null url) (string= url ""))
    (error
     "no database URL given.~%Expected postgresql://user:password@host:port/database"))
  (let ((mark (search "://" url)))
    (when (null mark) (db-url-error "not a URL" url))
    (let ((scheme (string-downcase (subseq url 0 mark))))
      (when (and (string/= scheme "postgresql") (string/= scheme "postgres"))
        (db-url-error
         (concatenate 'string "the scheme is " scheme "://, not postgresql://")
         url)))
    (let* ((body (subseq url (+ mark 3)))
           ;; A trailing ?sslmode=... and friends is not part of the authority.
           (query (position #\? body))
           (located (if query (subseq body 0 query) body))
           ;; The authority ends at the first / ; what follows is the database.
           (slash (position #\/ located))
           (authority (if slash (subseq located 0 slash) located))
           (database (if slash (subseq located (+ slash 1)) ""))
           ;; The LAST @ separates the userinfo, so an unescaped @ inside the
           ;; password cuts the string in the right place anyway.
           (at (position #\@ authority :from-end t))
           (userinfo (if at (subseq authority 0 at) ""))
           (hostport (if at (subseq authority (+ at 1)) authority))
           (user-end (position #\: userinfo))
           (user
            (db-url-decode (if user-end (subseq userinfo 0 user-end) userinfo)))
           (secret
            (if user-end (db-url-decode (subseq userinfo (+ user-end 1))) ""))
           (host-end (position #\: hostport :from-end t))
           (host (if host-end (subseq hostport 0 host-end) hostport))
           (port-text (if host-end (subseq hostport (+ host-end 1)) "5432"))
           (port (db-url-port port-text url)))
      (values (db-url-part database "no database in the URL" url)
              (db-url-part user "no user in the URL" url)
              ;; nil, not "": a server on trust auth wants no password at all.
              (if (string= secret "") nil secret)
              (db-url-part host "no host in the URL" url) port))))


---

# FILE: references/examples/db/postgres-crud.lisp

;; The whole CRUD cycle -- create, insert, select, update, delete -- against a
;; real PostgreSQL with the REAL cl-postgres (zlib, Marijn Haverbeke / Sabra
;; Crolleton -- Postmodern's low-level driver) loaded from its unmodified
;; upstream sources. Also two things postgres-hello.lisp does not show: a
;; parameterised statement through the extended query protocol
;; (prepare-query + exec-prepared), and a second row reader shape.
;;
;; It needs a server, and reads where to find it out of DATABASE_URL
;; (database-url.lisp beside this file parses the URL). The one-liner this
;; example is written against, and the URL that reaches it:
;;   docker run --rm -p 54329:5432 -e POSTGRES_HOST_AUTH_METHOD=trust postgres:17-alpine
;;   export DATABASE_URL=postgresql://postgres@127.0.0.1:54329/postgres
;;   rontolisp examples/db/postgres-crud.lisp
;;
;; Connecting, authenticating and the other backends work exactly as in
;; postgres-hello.lisp (see it, and the README beside it):
;;   rontolisp examples/db/postgres-crud.lisp -o Prog.class && java Prog
;;   rontolisp examples/db/postgres-crud.lisp -o postgres-crud.wasm --component --optimize
;;   wasmtime run -W gc=y -W exceptions=y -S tcp=y -S inherit-network=y --env DATABASE_URL postgres-crud.wasm

(ql:quickload "cl-postgres")

(load "database-url.lisp")

(defun fruits (conn)
  "Every row, as a list of (id name price) lists."
  (cl-postgres:exec-query conn "select id, name, price from fruits order by id"
                          'cl-postgres:list-row-reader))

(defun connect ()
  "A connection to the server DATABASE_URL names. The five values arrive in the
   order open-database takes them."
  (multiple-value-bind (database user password host port)
      (database-url-parts (uiop:getenv "DATABASE_URL"))
    (cl-postgres:open-database database user password host port)))

(let ((conn (connect)))
  ;; close-database in the cleanup, so the connection goes back even if a query
  ;; signals.
  (unwind-protect (progn
                    ;; One transaction, rolled back at the end: the example leaves the
                    ;; server exactly as it found it, so it can be re-run as often as you
                    ;; like -- and a run that dies halfway leaves nothing behind either.
                    (cl-postgres:exec-query conn "begin")

                    ;; CREATE. A statement with no result needs no row reader.
                    (cl-postgres:exec-query conn
                                            "create table fruits (id integer primary key, name text, price integer)")

                    ;; INSERT
                    (cl-postgres:exec-query conn
                                            "insert into fruits (id, name, price)
               values (1, 'apple', 120), (2, 'banana', 80), (3, 'cherry', 300)")
                    (format t "inserted: ~a~%" (fruits conn))

                    ;; READ. The row reader decides the shape: alist-row-reader labels each
                    ;; column with its name, where list-row-reader gives bare lists.
                    (format t "as alists: ~a~%"
                            (cl-postgres:exec-query conn
                             "select name, price from fruits where id = 2"
                             'cl-postgres:alist-row-reader))

                    ;; UPDATE
                    (cl-postgres:exec-query conn
                     "update fruits set price = 150 where name = 'apple'")
                    (format t "after update: ~a~%" (fruits conn))

                    ;; DELETE
                    (cl-postgres:exec-query conn
                     "delete from fruits where price > 200")
                    (format t "after delete: ~a~%" (fruits conn))

                    ;; A parameterised statement: prepared once on the server, then run with
                    ;; different arguments. $1 is the placeholder; the arguments are a list.
                    (cl-postgres:prepare-query conn "cheaper-than"
                     "select name from fruits where price < $1 order by name")
                    (format t "under 100: ~a~%"
                            (cl-postgres:exec-prepared conn "cheaper-than"
                             (list 100) 'cl-postgres:list-row-reader))
                    (format t "under 200: ~a~%"
                            (cl-postgres:exec-prepared conn "cheaper-than"
                             (list 200) 'cl-postgres:list-row-reader))

                    (cl-postgres:exec-query conn "rollback"))
    (cl-postgres:close-database conn)))


---

# FILE: references/examples/db/postgres-hello.lisp

;; Connects to a PostgreSQL server with the REAL cl-postgres (zlib, Marijn
;; Haverbeke / Sabra Crolleton -- Postmodern's low-level driver) loaded from its
;; unmodified upstream sources, and runs one query. Run with:
;;   rontolisp examples/db/postgres-hello.lisp
;;
;; It needs a server, and it reads where to find it out of DATABASE_URL
;; (database-url.lisp beside this file parses the URL). The one-liner this
;; example is written against, and the URL that reaches it:
;;   docker run --rm -p 54329:5432 -e POSTGRES_HOST_AUTH_METHOD=trust postgres:17-alpine
;;   export DATABASE_URL=postgresql://postgres@127.0.0.1:54329/postgres
;;
;; Runs on the interpreter and on the JVM backend:
;;   rontolisp examples/db/postgres-hello.lisp -o Prog.class && java Prog
;;
;; The driver loads through ql:quickload, which pulls md5, split-sequence,
;; ironclad, cl-base64, cl-ppcre, uax-15 and alexandria (downloaded once into
;; ~/.rontolisp/quicklisp). Compiling takes a couple of seconds, and a run is
;; well under a second on any backend, the interpreter included -- the library
;; tables uax-15 would otherwise build at load time are derived at compile time
;; and built only if something asks for one, and nothing here does.
;; trust, password, md5 and SCRAM-SHA-256 authentication all complete well
;; within the default 60-second authentication_timeout: a SCRAM connect costs
;; about a fifth of a second on every backend (the interpreter runs its
;; 4096-round PBKDF2 on a native kernel).
;;
;; On WASM the driver needs --component (TCP is WASI 0.3 sockets; Preview 1 has
;; no host socket API):
;;   rontolisp examples/db/postgres-hello.lisp -o postgres-hello.wasm --component --optimize
;;   wasmtime run -W gc=y -W exceptions=y -S tcp=y -S inherit-network=y --env DATABASE_URL postgres-hello.wasm
;; TLS (sslmode) is interpreter/JVM only; use plain TCP on WASM.

(ql:quickload "cl-postgres")

(load "database-url.lisp")

;; uiop:getenv reads the variable -- rontolisp's one spelling of that, ANSI CL
;; having none -- and the parser hands back the five values in the order
;; open-database wants them, which is what makes this a multiple-value-bind.
(multiple-value-bind (database user password host port)
    (database-url-parts (uiop:getenv "DATABASE_URL"))
  (let ((conn (cl-postgres:open-database database user password host port)))
    ;; A row reader turns the result rows into Lisp data; list-row-reader gives a
    ;; list of lists.
    (print
     (cl-postgres:exec-query conn "select 42, 'hello'"
                             'cl-postgres:list-row-reader))
    (print
     (cl-postgres:exec-query conn "select generate_series(1, 3) as n"
                             'cl-postgres:list-row-reader))
    (cl-postgres:close-database conn)))


---

# FILE: references/examples/db/postgres-web.lisp

;; A tiny web app: the two libraries of the other examples in this directory
;; put together. PostgreSQL keeps the notes (the REAL cl-postgres driver, as in
;; postgres-hello.lisp), cl-who renders the page (the real upstream (X)HTML
;; library, as in ../net/http-handler-cl-who.lisp) and rontolisp:http-handler
;; serves it. Two routes: GET / lists the notes and shows a form, POST /add
;; inserts one and redirects back.
;;
;; It needs a server, and reads where to find it out of DATABASE_URL
;; (database-url.lisp beside this file parses the URL). The one-liner this
;; example is written against, and the URL that reaches it:
;;   docker run --rm -p 54329:5432 -e POSTGRES_HOST_AUTH_METHOD=trust postgres:17-alpine
;;   export DATABASE_URL=postgresql://postgres@127.0.0.1:54329/postgres
;;   rontolisp examples/db/postgres-web.lisp
;; Then open http://127.0.0.1:8080 in a browser.
;;
;; Runs on the interpreter and on the JVM backend (keep the rontolisp jar on
;; the classpath, it carries the runtime):
;;   rontolisp examples/db/postgres-web.lisp -o App.class && \
;;     java -cp target/rontolisp-0.1.0-SNAPSHOT-exec.jar:. App
;; And as a WASI component under wasmtime serve, which both serves the page and
;; lets the driver dial out -- but only with -S cli=y in the flag set. Without
;; it the linker rejects the module with "resource implementation is missing",
;; which reads like the host has no sockets at all:
;;   rontolisp examples/db/postgres-web.lisp -o app.wasm --component --optimize
;;   wasmtime serve -W gc=y -W exceptions=y -S cli=y -S tcp=y -S inherit-network=y --env DATABASE_URL app.wasm
;; wasmCloud runs the same component (wash dev; its startup log lists 0.2
;; interfaces but it does provide wasi:sockets 0.3), and so does Spin (the
;; canary build, https://github.com/spinframework/spin/releases/tag/canary --
;; 4.1.0-pre0+), which needs no flags but does need the environment and the
;; database address spelled out in spin.toml -- its sandbox is deny-by-default:
;;   [component.notes]
;;   source = "app.wasm"
;;   allowed_outbound_hosts = ["tcp://127.0.0.1:54329"]
;;   environment = { DATABASE_URL = "postgresql://postgres@127.0.0.1:54329/postgres" }
;; See README.md for the whole manifest.
;; The component reaches the server over the network it is given, so DATABASE_URL
;; has to name an address that is reachable from inside it -- a container IP
;; under wasmtime-in-Docker, and under wash a NON-loopback address, because wash
;; routes loopback to a per-workload virtual network.
;;
;; --env DATABASE_URL is what puts the variable inside the component: the WASI
;; 0.3 service world carries no wasi:cli/environment, so a program that calls
;; uiop:getenv imports the interface itself and the served component declares it
;; (visible in `wasm-tools component wit app.wasm`). Drop the flag and this
;; program stops on "no database URL given" -- which an uncaught error on WASM
;; turns into a bare unreachable trap, so the missing flag reads as a crash.

(ql:quickload "cl-postgres")
(ql:quickload "cl-who")

(load "database-url.lisp")

(defun connect ()
  "A connection to the database. One per request: a connection is a single
   conversation with the server, and the interpreter/JVM servers run every
   request on its own thread with its own dynamic bindings, so requests that
   genuinely overlap stay isolated. (See the README for how each host holds
   up under a concurrent load test.)"
  (multiple-value-bind (database user password host port)
      (database-url-parts (uiop:getenv "DATABASE_URL"))
    (cl-postgres:open-database database user password host port)))

;; if not exists, not drop + create: this top level runs once per INSTANCE, and
;; how long an instance lives is the host's business -- wasmtime serve keeps one
;; for the whole run, while wasmCloud starts a fresh one per request. A drop
;; here would empty the table on every request there.
(let ((db (connect)))
  (unwind-protect (cl-postgres:exec-query db
                                          "create table if not exists notes (id serial primary key, body text)")
    (cl-postgres:close-database db)))

(defun all-notes (db)
  "Every note, newest first. A row reader decides the shape; list-row-reader
   gives one list per row, so each note is a one-element list."
  (cl-postgres:exec-query db "select body from notes order by id desc"
                          'cl-postgres:list-row-reader))

(defun add-note (db body)
  "One note, through the extended query protocol. $1 is a placeholder, so what
   the visitor typed stays data and never becomes SQL. The statement is
   prepared on this connection; the next request's connection starts fresh."
  (cl-postgres:prepare-query db "add-note"
                             "insert into notes (body) values ($1)")
  (cl-postgres:exec-prepared db "add-note" (list body)))

(defun page (db)
  "The whole site: the form, then the notes."
  (cl-who:with-html-output-to-string (s)
    (:html (:head (:title "Notes"))
           (:body (:h1 "Notes")
                  (:form :method "post"
                         :action "/add"
                         (:input :name "body" :size "40" :autofocus t)
                         (:button "Add"))
                  (:ul
                       ;; Markup inside a plain Lisp form needs an htm to go back into the
                       ;; markup DSL; esc escapes whatever the visitor typed.
                       (dolist (note (all-notes db))
                         (cl-who:htm (:li (cl-who:esc (first note))))))))))

;; async-defun because the env :raw-body is a stream: read-all drains it and
;; await waits for the whole thing. Draining comes first, and taking a
;; connection second, so none is held open while we wait on the client.
(rontolisp:async-defun handle (env)
  (let* ((form (rontolisp:await (rontolisp:read-all (getf env :raw-body))))
         (db (connect)))
    (unwind-protect (if (string= (getf env :path-info) "/add")
                        ;; What arrived is url-encoded, the same shape as a query string.
                        (let ((body (rontolisp:query-param form "body")))
                          (when (and body (string/= body ""))
                            (add-note db body))
                          ;; 303 + Location: back to the list, so a reload does not
                          ;; re-post; a two-element response is a bodyless one.
                          '(303 (:location "/")))
                        (list 200 '(:content-type "text/html; charset=utf-8")
                              (list (page db))))
      ;; Handed back even if a query signals.
      (cl-postgres:close-database db))))

;; Blocks and serves on port 8080.
(rontolisp:http-handler 'handle 8080)


---

# FILE: references/examples/db/postgres-web/spin.toml

spin_manifest_version = 2

[application]
name = "postgres-web"
version = "0.1.0"
description = "A rontolisp:http-handler served as a wasi:http/handler@0.3.0 component."

[[trigger.http]]
route = "/..."
component = "postgres-web"

[component.postgres-web]
source = "app.wasm"
allowed_outbound_hosts = ["tcp://127.0.0.1:54329"]
environment = { DATABASE_URL = "postgresql://postgres@127.0.0.1:54329/postgres" }

[component.postgres-web.build]
command = "rontolisp ../postgres-web.lisp -o app.wasm --component --optimize"
watch = ["../postgres-web.lisp"]


---

# FILE: references/examples/db/postmodern-crud.lisp

;; The same CRUD cycle as postgres-crud.lisp, one layer up: the REAL
;; postmodern (zlib, Marijn Haverbeke / Sabra Crolleton) loaded from its
;; unmodified upstream sources. Where postgres-crud.lisp opens a connection by
;; hand and writes SQL strings, everything here is a macro over the same
;; driver: with-connection manages the connection, S-SQL writes the statements
;; as s-expressions, with-transaction wraps the update, and defprepared turns a
;; parameterised statement into an ordinary function.
;;
;; It needs a server, and reads where to find it out of DATABASE_URL
;; (database-url.lisp beside this file parses the URL). The one-liner this
;; example is written against, and the URL that reaches it:
;;   docker run --rm -p 54329:5432 -e POSTGRES_HOST_AUTH_METHOD=trust postgres:17-alpine
;;   export DATABASE_URL=postgresql://postgres@127.0.0.1:54329/postgres
;;   rontolisp examples/db/postmodern-crud.lisp
;;
;; The other backends work exactly as in postgres-hello.lisp (see it, and the
;; README beside it):
;;   rontolisp examples/db/postmodern-crud.lisp -o Prog.class && java Prog
;;   rontolisp examples/db/postmodern-crud.lisp -o postmodern-crud.wasm --component --optimize
;;   wasmtime run -W gc=y -W exceptions=y -S tcp=y -S inherit-network=y --env DATABASE_URL postmodern-crud.wasm
;;
;; This file shows the query, transaction and prepared-statement layers. The
;; DAO layer on top of them (defclass with :metaclass dao-class, get-dao /
;; select-dao / insert-dao) is postmodern-dao.lisp beside this file.

(ql:quickload "postmodern")

(load "database-url.lisp")

(defun connection-spec ()
  "What DATABASE_URL says, in the shape with-connection wants: the four
   positional values, then :port. The same five values postgres-crud.lisp hands
   to open-database -- with-connection is a macro over that same driver."
  (multiple-value-bind (database user password host port)
      (database-url-parts (uiop:getenv "DATABASE_URL"))
    (list database user password host :port port)))

;; S-SQL turns an s-expression into an SQL string. When every value in the form
;; is a literal the whole translation happens while the program is being read,
;; and the statement reaches the server as a constant; when one is not, s-sql
;; assembles the string while the program runs. pomo:sql shows either.
(format t "sql: ~a~%"
        (pomo:sql (:select 'name :from 'fruits :where (:< 'price 100))))
(format t "sql: ~a~%"
        (pomo:sql (:insert-rows-into 'fruits :columns 'id :values '((1) (2)))))

;; The connection is bound for the whole body -- every query below finds it
;; through pomo:*database* -- and closed on the way out, however the body ends.
(pomo:with-connection (connection-spec)

  ;; CREATE. execute is query with no result; the leading drop makes the
  ;; example re-runnable.
  (pomo:execute (:drop-table :if-exists 'fruits))
  (pomo:execute
   (:create-table 'fruits
                  ((id :type integer :primary-key t) (name :type text)
                   (price :type integer))))

  ;; INSERT. :set takes alternating column and value forms.
  (pomo:execute (:insert-into 'fruits :set 'id 1 'name "apple" 'price 120))
  (pomo:execute (:insert-into 'fruits :set 'id 2 'name "banana" 'price 80))
  ;; A value that is not a literal -- here an expression, but a variable reads
  ;; the same -- is what makes s-sql assemble the statement while the program
  ;; runs rather than while it is read. pomo:sql shows the difference.
  (pomo:execute
   (:insert-into 'fruits :set 'id 3 'name "cherry" 'price (* 3 100)))

  ;; :insert-rows-into writes several rows at once, and is likewise assembled
  ;; at run time.
  (pomo:execute
   (:insert-rows-into 'fruits
                      :columns 'id 'name 'price
                      :values '((4 "durian" 900) (5 "elderberry" 60))))

  ;; READ. The result format is an argument: the default is a list of rows...
  (format t "rows: ~a~%"
          (pomo:query (:order-by (:select '* :from 'fruits) 'id)))
  ;; ...:alists labels each column with its name...
  (format t "alists: ~a~%"
   (pomo:query (:select 'name 'price :from 'fruits :where (:= 'id 2)) :alists))
  ;; ...and :single takes the one value out of a one-row, one-column result.
  (format t "count: ~a~%"
          (pomo:query (:select (:count '*) :from 'fruits) :single))

  ;; UPDATE, in a transaction: the body commits on a normal return and rolls
  ;; back if it signals.
  (pomo:with-transaction ()
    (pomo:execute (:update 'fruits :set 'price 150 :where (:= 'name "apple"))))
  (format t "after update: ~a~%"
          (pomo:query (:order-by (:select 'name 'price :from 'fruits) 'id)))

  ;; DELETE
  (pomo:execute (:delete-from 'fruits :where (:> 'price 200)))
  (format t "after delete: ~a~%"
          (pomo:query (:order-by (:select 'name :from 'fruits) 'name) :column))

  ;; A parameterised statement, prepared once on the server and then called
  ;; like any other function. $1 is the placeholder.
  (pomo:defprepared cheaper-than
    (:select 'name :from 'fruits :where (:< 'price '$1))
    :column)
  (format t "under 100: ~a~%" (cheaper-than 100))
  (format t "under 200: ~a~%" (cheaper-than 200)))

;; The table is left behind on purpose, so you can look at it with psql after
;; the run; the drop at the top is what makes the next run start clean.


---

# FILE: references/examples/db/postmodern-dao.lisp

;; The layer postmodern-crud.lisp stops short of: the DAO. A table row becomes
;; an ordinary CLOS instance -- the class IS the table definition. Writing
;; (:metaclass pomo:dao-class) runs postmodern's metaclass protocol when the
;; class is defined: every :col-type slot becomes a column, :keys names the
;; primary key, and the insert-dao / get-dao / update-dao / upsert-dao /
;; delete-dao / select-dao methods for exactly this class are built on the
;; spot.
;;
;; It needs a server, and reads where to find it out of DATABASE_URL
;; (database-url.lisp beside this file parses the URL). The one-liner this
;; example is written against, and the URL that reaches it:
;;   docker run --rm -p 54329:5432 -e POSTGRES_HOST_AUTH_METHOD=trust postgres:17-alpine
;;   export DATABASE_URL=postgresql://postgres@127.0.0.1:54329/postgres
;;   rontolisp examples/db/postmodern-dao.lisp
;;
;; The other backends work exactly as in postgres-hello.lisp (see it, and the
;; README beside it):
;;   rontolisp examples/db/postmodern-dao.lisp -o Prog.class && java Prog
;;   rontolisp examples/db/postmodern-dao.lisp -o postmodern-dao.wasm --component --optimize
;;   wasmtime run -W gc=y -W exceptions=y -S tcp=y -S inherit-network=y --env DATABASE_URL postmodern-dao.wasm

(ql:quickload "postmodern")

(load "database-url.lisp")

(defun connection-spec ()
  "What DATABASE_URL says, in the shape with-connection wants."
  (multiple-value-bind (database user password host port)
      (database-url-parts (uiop:getenv "DATABASE_URL"))
    (list database user password host :port port)))

;; The class definition is the whole schema: two columns, id the primary key.
;; Slots without :col-type would be ordinary (non-column) slots.
(defclass fruit ()
  ((id :col-type integer :initarg :id :accessor fruit-id)
   (name :col-type text :initarg :name :accessor fruit-name)
   (price :col-type integer :initarg :price :accessor fruit-price))
  (:metaclass pomo:dao-class)
  (:keys id))

;; The CREATE TABLE statement is derived from the class -- no server needed to
;; look at it.
(format t "ddl: ~a~%" (pomo:dao-table-definition 'fruit))

;; deftable records how to create the table; !dao-def says "from the DAO class
;; of the same name". create-table below replays it against the server.
(pomo:deftable fruit (pomo:!dao-def))

(pomo:with-connection (connection-spec)

  ;; The leading drop makes the example re-runnable.
  (pomo:execute (:drop-table :if-exists 'fruit))
  (pomo:create-table 'fruit)

  ;; CREATE: an instance goes in as a row.
  (pomo:insert-dao (make-instance 'fruit :id 1 :name "apple" :price 120))
  (pomo:insert-dao (make-instance 'fruit :id 2 :name "banana" :price 80))

  ;; READ: get-dao fetches by primary key and answers an instance.
  (let ((apple (pomo:get-dao 'fruit 1)))
    (format t "got: ~a at ~a~%" (fruit-name apple) (fruit-price apple))

    ;; UPDATE: change the instance, then write it back.
    (setf (fruit-price apple) 150)
    (pomo:update-dao apple))

  ;; UPSERT: update the row if the key exists, insert it otherwise. The second
  ;; value says which happened -- nil for an update, t for an insert.
  (multiple-value-bind (dao inserted-p)
      (pomo:upsert-dao (make-instance 'fruit :id 2 :name "blueberry" :price 90))
    (format t "upsert ~a: inserted-p ~a~%" (fruit-name dao) inserted-p))
  (multiple-value-bind (dao inserted-p)
      (pomo:upsert-dao (make-instance 'fruit :id 3 :name "cherry" :price 300))
    (format t "upsert ~a: inserted-p ~a~%" (fruit-name dao) inserted-p))

  ;; SELECT: a test (any S-SQL expression) and an ordering, answered as a list
  ;; of instances.
  (dolist (f (pomo:select-dao 'fruit (:< 'price 200) 'id))
    (format t "row: ~a ~a ~a~%" (fruit-id f) (fruit-name f) (fruit-price f)))

  ;; DELETE takes the instance too.
  (pomo:delete-dao (pomo:get-dao 'fruit 3))
  (format t "count: ~a~%"
          (pomo:query (:select (:count '*) :from 'fruit) :single)))

;; The table is left behind on purpose, so you can look at it with psql after
;; the run; the drop at the top is what makes the next run start clean.


---

# FILE: references/examples/deep-learning-from-scratch/LICENSE.md

The programs in this directory are a rontolisp port of the sample code of
the book "Deep Learning from Scratch" (ゼロから作るDeep Learning, O'Reilly
Japan) by Koki Saitoh, https://github.com/oreilly-japan/deep-learning-from-scratch,
distributed under the MIT License. The pretrained weights in
ch03/sample-weight.bin are converted from that repository's
ch03/sample_weight.pkl. The original license follows.

---

The MIT License (MIT)

Copyright (c) 2016 Koki Saitoh

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


---

# FILE: references/examples/deep-learning-from-scratch/README.md

# Deep Learning from Scratch, in rontolisp

A rontolisp port of [the sample code of the book **"Deep Learning from
Scratch"** (ゼロから作るDeep Learning, O'Reilly Japan) by Koki Saitoh](https://github.com/oreilly-japan/deep-learning-from-scratch) —
chapters 2-8: perceptrons, MNIST inference, numerical gradients,
backpropagation through class-based layers, the training techniques
chapter (optimizers, weight initialization, Batch Normalization, Dropout,
weight decay, hyperparameter search), and the CNN chapters (im2col
convolution/pooling layers, SimpleConvNet, the ch08 deep network). The
original code is MIT-licensed (see [LICENSE.md](LICENSE.md)).

The port maps numpy to the `linalg:` package (axis reductions, broadcasting,
`linalg:randn`/`choice` seeded RNG, `take-rows`/`gather`/`one-hot`
indexing), Python classes to the CLOS subset (`defclass` layers with
`forward`/`backward` generics), and `params`/`grads` dicts to string-keyed
hash tables. matplotlib plots become printed tables, trajectories and text
histograms. Arrays are double-float (`#d`) except where a script is about
reduced precision (ch08's half-float port casts to packed single-float
`#f`), and adding `--simd` speeds a script up **without changing a byte
of its output**.

## Setup

```bash
./download-mnist.sh    # fetches + decompresses the 4 MNIST idx files (~55 MB) into dataset/
```

The pretrained weights are already committed: `ch03/sample-weight.bin`
(the book's `sample_weight.pkl`), `ch07/params.bin` (`params.pkl`) and
`ch08/deep-convnet-params.bin` (`deep_convnet_params.pkl`), each
re-exported from the book repo's pickle by
`tools/export-sample-weight.py`.

## Running

Everything runs from this directory on all four backends. With the native
binary (or `java -jar $JAR`):

```bash
rontolisp ch05/train-neuralnet.lisp              # interpreter
rontolisp ch05/train-neuralnet.lisp --simd       # same output, much faster

rontolisp ch05/train-neuralnet.lisp -o Prog.class && java -cp .:$JAR_CLASSPATH Prog

rontolisp ch05/train-neuralnet.lisp -o prog.wasm --optimize && \
  wasmtime run -W gc --dir . prog.wasm

rontolisp ch05/train-neuralnet.lisp -o comp.wasm --component && \
  wasmtime run -W gc=y --dir . comp.wasm
```

The MNIST scripts read `dataset/*-ubyte` relative to this directory, so the
WASM backends need the `--dir .` preopen. Training scripts declare their
knobs (`*train-limit*`, `*batch-size*`, `*epochs*`, ...) as `defparameter`s
at the top — the defaults are scaled down from the book's (500 instead of
60000 train images, etc.) so the plain interpreter finishes in a few
minutes per script (the ch06 comparison/overfitting scripts take the
longest, roughly 4-8 minutes; the same runs are seconds on the JVM or
under `--simd`); raise the knobs accordingly. Weight
initialization and mini-batch sampling use `linalg:seed`, so they reproduce
bit-identically everywhere; only values that pass through `exp`/`log` (loss
prints) can differ in their last digits on WASM.

## The programs

| Book | Port | What it shows |
| --- | --- | --- |
| ch02 `and_gate.py` ... | `ch02/{and,nand,or,xor}-gate.lisp` | Perceptrons; XOR needs a second layer |
| ch03 activation plots | `ch03/activation-functions.lisp` | step/sigmoid/relu as a printed table |
| ch03 `mnist_show.py` | `ch03/mnist-show.lisp` | The first training digit as ASCII art |
| ch03 `neuralnet_mnist.py` | `ch03/neuralnet-mnist.lisp` | Inference with the book's pretrained weights, one image at a time |
| ch03 `neuralnet_mnist_batch.py` | `ch03/neuralnet-mnist-batch.lisp` | The same, 100 images per matrix product |
| ch04 `gradient_1d.py` / `gradient_2d.py` | `ch04/gradient-{1d,2d}.lisp` | Numerical differentiation |
| ch04 `gradient_method.py` | `ch04/gradient-method.lisp` | Gradient descent, incl. bad learning rates |
| ch04 `gradient_simplenet.py` | `ch04/gradient-simplenet.lisp` | The numerical gradient of a softmax loss |
| ch04 `two_layer_net.py` + `train_neuralnet.py` | `ch04/two-layer-net.lisp` + `ch04/train-neuralnet.lisp` | MNIST training with sigmoid-grad backprop |
| ch05 `layer_naive.py` + `buy_apple*.py` | `ch05/layer-naive.lisp` + `ch05/buy-apple{,-orange}.lisp` | Computational-graph layers (first CLOS classes) |
| ch05 `two_layer_net.py` | `ch05/two-layer-net.lisp` | Affine/ReLU/SoftmaxWithLoss layer objects |
| ch05 `gradient_check.py` | `ch05/gradient-check.lisp` | Backprop vs numerical gradients (< 1e-6) |
| ch05 `train_neuralnet.py` | `ch05/train-neuralnet.lisp` | Training through the layer stack |
| ch06 `optimizer_compare_naive.py` | `ch06/optimizer-compare-naive.lisp` | SGD/Momentum/AdaGrad/Adam trajectories |
| ch06 `optimizer_compare_mnist.py` | `ch06/optimizer-compare-mnist.lisp` | The same four racing on MNIST |
| ch06 `weight_init_activation_histogram.py` | `ch06/weight-init-activation-histogram.lisp` | Saturation/collapse/Xavier, as text histograms |
| ch06 `weight_init_compare.py` | `ch06/weight-init-compare.lisp` | std=0.01 stalls; Xavier/He learn |
| ch06 `batch_norm_gradient_check.py` | `ch06/batch-norm-gradient-check.lisp` | The BatchNorm backward pass, verified |
| ch06 `batch_norm_test.py` | `ch06/batch-norm-test.lisp` | BN learns even from starved init scales |
| ch06 `overfit_weight_decay.py` | `ch06/overfit-weight-decay.lisp` | 300-image overfitting; L2 decay caps it |
| ch06 `overfit_dropout.py` | `ch06/overfit-dropout.lisp` | The same, tamed by Dropout |
| ch06 `hyperparameter_optimization.py` | `ch06/hyperparameter-optimization.lisp` | Random search over lr / weight decay |
| ch07 `simple_convnet.py` | `ch07/simple-convnet.lisp` | Conv-Relu-Pool-Affine-Relu-Affine over im2col |
| ch07 `gradient_check.py` | `ch07/gradient-check.lisp` | Convolution/Pooling backprop, verified (< 1e-6) |
| ch07 `train_convnet.py` | `ch07/train-convnet.lisp` | Training the SimpleConvNet with Adam |
| ch07 `visualize_filter.py` | `ch07/visualize-filter.lisp` | W1's 5x5 filters before/after learning, as ASCII grids |
| ch08 `deep_convnet.py` | `ch08/deep-convnet.lisp` | The 16-16/32-32/64-64 pyramid, He init, Dropout |
| ch08 `train_deepnet.py` | `ch08/train-deepnet.lisp` | Training the deep CNN (smoke-scale defaults) |
| ch08 `misclassified_mnist.py` | `ch08/misclassified-mnist.lisp` | The pretrained net's mistakes, drawn as ASCII digits |
| ch08 `half_float_network.py` | `ch08/half-float-network.lisp` | Accuracy unchanged at reduced precision (float16 becomes packed `#f`) |

Shared library files mirror the book's `common/`: `functions.lisp`
(softmax, cross-entropy, ...), `gradient.lisp` (numerical gradients),
`layers.lisp` (CLOS layers incl. Convolution/Pooling, BatchNormalization
and Dropout), `util.lisp` (im2col/col2im), `optimizer.lisp`
(SGD/Momentum/Nesterov/AdaGrad/RMSprop/Adam), `multi-layer-net.lisp`,
`multi-layer-net-extend.lisp`, `trainer.lisp`, and `dataset/mnist.lisp`
(the idx / binary-weight loaders).

The CNN scripts are the heavy ones: a convolution forward is ~100x an
MLP's arithmetic. im2col turns it into `linalg:matmul`, and `--simd`
also intercepts the unfold itself plus the CNN-shaped call forms
(axes-permutation transpose, broadcasting, axis reductions), so
`ch07/train-convnet.lisp` at its scaled-down defaults is ~5 minutes
interpreted, ~20 seconds under `--simd`, and ~2 seconds compiled to the
JVM.


---

# FILE: references/examples/deep-learning-from-scratch/ch02/and-gate.lisp

;; ch02/and_gate.py -- the AND perceptron (Deep Learning from Scratch).
;;
;; A single perceptron with hand-picked weights w = (0.5 0.5) and bias
;; b = -0.7: fires when w.x + b > 0. Pure arithmetic, so the output is
;; byte-identical on every backend.
;;
;;   rontolisp ch02/and-gate.lisp

(defun and-gate (x1 x2)
  (let* ((x (linalg:from-list (list x1 x2)))
         (w (linalg:from-list '(0.5 0.5)))
         (b -0.7)
         (tmp (+ (linalg:sum (linalg:mul w x)) b)))
    (if (<= tmp 0) 0 1)))

(dolist (xs '((0 0) (1 0) (0 1) (1 1)))
  (format t "(~a, ~a) -> ~a~%" (car xs) (cadr xs)
          (and-gate (car xs) (cadr xs))))


---

# FILE: references/examples/deep-learning-from-scratch/ch02/nand-gate.lisp

;; ch02/nand_gate.py -- the NAND perceptron (Deep Learning from Scratch).
;;
;; The AND perceptron with its weights and bias negated: w = (-0.5 -0.5),
;; b = 0.7.
;;
;;   rontolisp ch02/nand-gate.lisp

(defun nand-gate (x1 x2)
  (let* ((x (linalg:from-list (list x1 x2)))
         (w (linalg:from-list '(-0.5 -0.5)))
         (b 0.7)
         (tmp (+ (linalg:sum (linalg:mul w x)) b)))
    (if (<= tmp 0) 0 1)))

(dolist (xs '((0 0) (1 0) (0 1) (1 1)))
  (format t "(~a, ~a) -> ~a~%" (car xs) (cadr xs)
          (nand-gate (car xs) (cadr xs))))


---

# FILE: references/examples/deep-learning-from-scratch/ch02/or-gate.lisp

;; ch02/or_gate.py -- the OR perceptron (Deep Learning from Scratch).
;;
;; w = (0.5 0.5) like AND, but the weaker bias b = -0.2 lets a single
;; active input fire the perceptron.
;;
;;   rontolisp ch02/or-gate.lisp

(defun or-gate (x1 x2)
  (let* ((x (linalg:from-list (list x1 x2)))
         (w (linalg:from-list '(0.5 0.5)))
         (b -0.2)
         (tmp (+ (linalg:sum (linalg:mul w x)) b)))
    (if (<= tmp 0) 0 1)))

(dolist (xs '((0 0) (1 0) (0 1) (1 1)))
  (format t "(~a, ~a) -> ~a~%" (car xs) (cadr xs) (or-gate (car xs) (cadr xs))))


---

# FILE: references/examples/deep-learning-from-scratch/ch02/xor-gate.lisp

;; ch02/xor_gate.py -- XOR from stacked perceptrons (Deep Learning from
;; Scratch).
;;
;; XOR is not linearly separable, so no single perceptron computes it; the
;; book's punchline is that one extra layer does: XOR(x1, x2) =
;; AND(NAND(x1, x2), OR(x1, x2)). The Python script imports the three gates
;; from their modules; a rontolisp load would also run their truth tables
;; (no `if __name__ == "__main__"` guard exists), so the gates are defined
;; here again.
;;
;;   rontolisp ch02/xor-gate.lisp

(defun %gate (x1 x2 w1 w2 b)
  ;; One perceptron: fires when w.x + b > 0.
  (let* ((x (linalg:from-list (list x1 x2)))
         (w (linalg:from-list (list w1 w2)))
         (tmp (+ (linalg:sum (linalg:mul w x)) b)))
    (if (<= tmp 0) 0 1)))

(defun and-gate (x1 x2) (%gate x1 x2 0.5 0.5 -0.7))
(defun nand-gate (x1 x2) (%gate x1 x2 -0.5 -0.5 0.7))
(defun or-gate (x1 x2) (%gate x1 x2 0.5 0.5 -0.2))

(defun xor-gate (x1 x2)
  (let ((s1 (nand-gate x1 x2)) (s2 (or-gate x1 x2))) (and-gate s1 s2)))

(dolist (xs '((0 0) (1 0) (0 1) (1 1)))
  (format t "(~a, ~a) -> ~a~%" (car xs) (cadr xs)
          (xor-gate (car xs) (cadr xs))))


---

# FILE: references/examples/deep-learning-from-scratch/ch03/activation-functions.lisp

;; ch03 step_function.py / sigmoid.py / relu.py / sig_step_compare.py merged
;; (Deep Learning from Scratch).
;;
;; The four plot scripts draw the activation curves with matplotlib; here
;; the same functions are tabulated at a few sample points instead. The
;; sigmoid column is rounded to 4 decimals; the sample points sit far from
;; rounding boundaries, so the table prints identically on every backend
;; even though WASM's exp is a polynomial approximation (~1e-6 relative).
;;
;;   rontolisp ch03/activation-functions.lisp

(load "../common/functions.lisp")

(defparameter *xs*
  (linalg:from-list '(-5.0 -2.0 -1.0 -0.5 0.0 0.5 1.0 2.0 5.0)))

(let ((step (step-function *xs*)) (sig (sigmoid *xs*)) (rel (relu *xs*)))
  (format t "     x   step  sigmoid    relu~%")
  (dotimes (i (linalg:size *xs*))
    (format t "~6,1f  ~5d  ~7,4f  ~6,1f~%" (aref *xs* i)
            (truncate (aref step i)) (aref sig i) (aref rel i))))


---

# FILE: references/examples/deep-learning-from-scratch/ch03/mnist-show.lisp

;; ch03/mnist_show.py -- display one MNIST digit (Deep Learning from Scratch).
;;
;; The book renders the first training image with PIL; here the 28x28 pixels
;; become an ASCII-art intensity ramp instead. Output is byte-identical on
;; every backend (no floats are printed).
;;
;; Data: run ../download-mnist.sh once, then from examples/deep-learning-from-scratch/:
;;   rontolisp ch03/mnist-show.lisp
;;   rontolisp ch03/mnist-show.lisp -o Prog.class && java -cp .: Prog
;;   rontolisp ch03/mnist-show.lisp -o prog.wasm --optimize && wasmtime run -W gc --dir . prog.wasm
;;   rontolisp ch03/mnist-show.lisp -o comp.wasm --component && \
;;     wasmtime run -W gc=y --dir . comp.wasm

(load "../dataset/mnist.lisp")

(defparameter *ramp* " .:-=+*#%@")

(defun render-image (img row)
  ;; One 28x28 image (row ROW of the (n x 784) matrix) as ASCII art: each
  ;; pixel in [0,1] indexes the 10-step intensity ramp.
  (dotimes (y 28)
    (let ((line ""))
      (dotimes (x 28)
        (let* ((v (aref img row (+ (* y 28) x)))
               (i (min 9 (truncate (* v 10)))))
          (setq line (concatenate 'string line (subseq *ramp* i (+ i 1))))))
      (write-line line))))

(let ((img (mnist-load-images "dataset/train-images-idx3-ubyte" 1))
      (lab (mnist-load-labels "dataset/train-labels-idx1-ubyte" 1)))
  (format t "label: ~a~%" (truncate (aref lab 0)))
  (format t "shape: ~a~%" (linalg:shape img))
  (render-image img 0))


---

# FILE: references/examples/deep-learning-from-scratch/ch03/neuralnet-mnist-batch.lisp

;; ch03/neuralnet_mnist_batch.py -- batch MNIST inference (Deep Learning
;; from Scratch).
;;
;; The same pretrained network as neuralnet-mnist.lisp, but 100 images per
;; forward pass: the (100 x 784) batch flows through mat.mat products and
;; the bias broadcasts over the rows, then argmax(axis=1) classifies the
;; whole batch at once -- the book's point about vectorized inference.
;;
;; Data: run ../download-mnist.sh once, then from examples/deep-learning-from-scratch/:
;;   rontolisp ch03/neuralnet-mnist-batch.lisp

(load "../common/functions.lisp")
(load "../dataset/mnist.lisp")

(defparameter *test-limit* 1000)
(defparameter *batch-size* 100)

(defun predict (network x)
  ;; x: an (N x 784) batch.
  (let* ((a1
          (linalg:add (linalg:matmul x (getf network :w1)) (getf network :b1)))
         (z1 (sigmoid a1))
         (a2
          (linalg:add (linalg:matmul z1 (getf network :w2)) (getf network :b2)))
         (z2 (sigmoid a2))
         (a3
          (linalg:add (linalg:matmul z2 (getf network :w3))
                      (getf network :b3))))
    (softmax a3)))

(let ((x (mnist-load-images "dataset/t10k-images-idx3-ubyte" *test-limit*))
      (target (mnist-load-labels "dataset/t10k-labels-idx1-ubyte" *test-limit*))
      (network (load-sample-weight "ch03/sample-weight.bin"))
      (accuracy-cnt 0))
  (do ((i 0 (+ i *batch-size*)))
      ((>= i *test-limit*))
    (let* ((idx (linalg:arange i (+ i *batch-size*)))
           (x-batch (linalg:take-rows x idx))
           (t-batch (linalg:take-rows target idx))
           (p (linalg:argmax (predict network x-batch) :axis 1)))
      (setq accuracy-cnt
            (+ accuracy-cnt (truncate (linalg:sum (linalg:equal p t-batch)))))))
  (format t "Accuracy: ~a/~a~%" accuracy-cnt *test-limit*))


---

# FILE: references/examples/deep-learning-from-scratch/ch03/neuralnet-mnist.lisp

;; ch03/neuralnet_mnist.py -- MNIST inference with the book's pretrained
;; 784-50-100-10 network (Deep Learning from Scratch).
;;
;; The weights come from sample-weight.bin (the book's sample_weight.pkl
;; re-exported by tools/export-sample-weight.py; already committed).
;; One image is classified at a time, exactly like the book; see
;; neuralnet-mnist-batch.lisp for the batch version. The book reaches
;; accuracy 0.9352 over the full 10000-image test set; the default here
;; evaluates the first *test-limit* images to keep the interpreter run
;; short (raise the knob for the full set).
;;
;; Data: run ../download-mnist.sh once, then from examples/deep-learning-from-scratch/:
;;   rontolisp ch03/neuralnet-mnist.lisp

(load "../common/functions.lisp")
(load "../dataset/mnist.lisp")

(defparameter *test-limit* 1000)

(defun predict (network x)
  ;; x: one flattened image (784-vector). vec.mat dot + bias broadcast,
  ;; sigmoid hidden layers, softmax output -- the book's forward pass.
  (let* ((a1 (linalg:add (linalg:dot x (getf network :w1)) (getf network :b1)))
         (z1 (sigmoid a1))
         (a2 (linalg:add (linalg:dot z1 (getf network :w2)) (getf network :b2)))
         (z2 (sigmoid a2))
         (a3
          (linalg:add (linalg:dot z2 (getf network :w3)) (getf network :b3))))
    (softmax a3)))

(let ((x (mnist-load-images "dataset/t10k-images-idx3-ubyte" *test-limit*))
      (target (mnist-load-labels "dataset/t10k-labels-idx1-ubyte" *test-limit*))
      (network (load-sample-weight "ch03/sample-weight.bin"))
      (accuracy-cnt 0))
  (dotimes (i (car (linalg:shape x)))
    (let* ((y (predict network (linalg:row x i))) ; numpy x[i]: the axis-0 slice, axis dropped
           (p (linalg:argmax y)))
      (when (= p (aref target i)) (setq accuracy-cnt (+ accuracy-cnt 1)))))
  (format t "Accuracy: ~a/~a~%" accuracy-cnt *test-limit*))


---

# FILE: references/examples/deep-learning-from-scratch/ch04/gradient-1d.lisp

;; ch04/gradient_1d.py -- numerical differentiation (Deep Learning from
;; Scratch).
;;
;; numerical_diff of f(x) = 0.01x^2 + 0.1x at x = 5 and x = 10 (the book
;; also draws the tangent lines; the printed derivatives are the point).
;; The true derivative is 0.02x + 0.1, i.e. 0.2 and 0.3.
;;
;;   rontolisp ch04/gradient-1d.lisp

(defun numerical-diff (f x)
  ;; (f(x+h) - f(x-h)) / 2h with h = 1e-4.
  (let ((h 1.0e-4)) (/ (- (funcall f (+ x h)) (funcall f (- x h))) (* 2 h))))

(defun function-1 (x) (+ (* 0.01 x x) (* 0.1 x)))

;; The central difference of a quadratic is exact up to float rounding;
;; round to 8 decimals so the output is identical on every backend (the
;; WASM ~f scaling must stay inside the i31 integer range).
(format t "df/dx at  5: ~,8f~%" (numerical-diff (function function-1) 5.0))
(format t "df/dx at 10: ~,8f~%" (numerical-diff (function function-1) 10.0))


---

# FILE: references/examples/deep-learning-from-scratch/ch04/gradient-2d.lisp

;; ch04/gradient_2d.py -- the gradient of f(x0, x1) = x0^2 + x1^2
;; (Deep Learning from Scratch).
;;
;; The book draws the gradient field with a quiver plot; here the gradient
;; is printed at the three sample points the book's text discusses. The
;; true gradient is (2*x0, 2*x1).
;;
;;   rontolisp ch04/gradient-2d.lisp

(load "../common/gradient.lisp")

(defun function-2 (x)
  ;; x[0]^2 + x[1]^2 over the whole vector.
  (linalg:sum (linalg:square x)))

(dolist (pt '((3.0 4.0) (0.0 2.0) (3.0 0.0)))
  (let ((g (numerical-gradient (function function-2) (linalg:from-list pt))))
    (format t "grad at (~a, ~a) = (~,4f, ~,4f)~%" (car pt) (cadr pt) (aref g 0)
            (aref g 1))))


---

# FILE: references/examples/deep-learning-from-scratch/ch04/gradient-method.lisp

;; ch04/gradient_method.py -- gradient descent on f(x0, x1) = x0^2 + x1^2
;; (Deep Learning from Scratch).
;;
;; 20 steps from (-3, 4) with learning rate 0.1, printing the trajectory
;; the book plots. The book's text also shows why the learning rate
;; matters; the too-large / too-small runs are reproduced after the
;; well-tuned one.
;;
;;   rontolisp ch04/gradient-method.lisp

(load "../common/gradient.lisp")

(defun function-2 (x) (linalg:sum (linalg:square x)))

(defun gradient-descent (f init-x lr step-num &optional print-steps)
  ;; Returns x after step-num updates x <- x - lr * grad; with print-steps,
  ;; prints every 5th position along the way.
  (let ((x init-x))
    (dotimes (i step-num)
      (when (and print-steps (= (mod i 5) 0))
        (format t "  step ~2d: (~,6f, ~,6f)~%" i (aref x 0) (aref x 1)))
      (let ((grad (numerical-gradient f x)))
        (setq x (linalg:sub x (linalg:mul lr grad)))))
    x))

(format t "lr = 0.1 (well-tuned):~%")
(let ((x
       (gradient-descent (function function-2) (linalg:from-list '(-3.0 4.0))
                         0.1 20 t)))
  (format t "  final:   (~,6f, ~,6f)~%" (aref x 0) (aref x 1)))

;; Too large a learning rate diverges; too small barely moves (the book's
;; lr=10.0 / lr=1e-10 comparison, 100 steps each).
(let ((x
       (gradient-descent (function function-2) (linalg:from-list '(-3.0 4.0))
                         10.0 100)))
  (format t "lr = 10.0:  |x| ~a 1e10 after 100 steps~%"
          (if (> (linalg:norm x) 1.0e10) "diverged beyond" "stayed under")))
(let ((x
       (gradient-descent (function function-2) (linalg:from-list '(-3.0 4.0))
                         1.0e-10 100)))
  (format t "lr = 1e-10: still at (~,4f, ~,4f) after 100 steps~%" (aref x 0)
          (aref x 1)))


---

# FILE: references/examples/deep-learning-from-scratch/ch04/gradient-simplenet.lisp

;; ch04/gradient_simplenet.py -- the numerical gradient of a one-matrix
;; network's loss (Deep Learning from Scratch).
;;
;; simpleNet holds one 2x3 weight matrix W; the loss is the softmax
;; cross-entropy of x.W against the one-hot target t. The numerical
;; gradient dW is what the book's ch04 climax computes. Weights come from
;; linalg:randn under a fixed seed, so the output is deterministic (and,
;; up to WASM's exp/log approximation, backend-identical).
;;
;;   rontolisp ch04/gradient-simplenet.lisp

(load "../common/functions.lisp")
(load "../common/gradient.lisp")

(linalg:seed 0)

(defparameter *w* (linalg:randn '(2 3)))

(defun net-predict (x) (linalg:dot x *w*))

(defun net-loss (x target)
  (cross-entropy-error (softmax (net-predict x)) target))

(let* ((x (linalg:from-list '(0.6 0.9)))
       (target (linalg:from-list '(0 0 1)))
       (f (lambda (w) (net-loss x target)))
       (dw (numerical-gradient f *w*)))
  (format t "W:~%")
  (dotimes (i 2)
    (format t "  ~,6f ~,6f ~,6f~%" (aref *w* i 0) (aref *w* i 1)
            (aref *w* i 2)))
  (format t "loss: ~,6f~%" (net-loss x target))
  (format t "dW:~%")
  (dotimes (i 2)
    (format t "  ~,6f ~,6f ~,6f~%" (aref dw i 0) (aref dw i 1) (aref dw i 2))))


---

# FILE: references/examples/deep-learning-from-scratch/ch04/train-neuralnet.lisp

;; ch04/train_neuralnet.py -- mini-batch training of the two-layer net
;; (Deep Learning from Scratch).
;;
;; The book trains 10000 iterations (batch 100, lr 0.1) over the full
;; 60000-image set; the defaults here are scaled to *train-limit* 500
;; images, batch 16 and three epochs so the plain interpreter finishes in
;; a couple of minutes -- raise the knobs (and add --simd, whose
;; all-double output is byte-identical) for a longer run. The learning
;; rate is raised to 1.0: with 1/120 of the book's data the 0.1 sigmoid
;; net would still be in its flat warm-up phase when the run ends. The
;; gradient is the analytic tln-gradient, the same choice as the book's
;; active line (the numerical one is available but ~1000x slower). Loss
;; values pass through exp/log, so the last digits can differ on WASM;
;; accuracies print as exact correct/total counts.
;;
;; Data: run ../download-mnist.sh once, then from examples/deep-learning-from-scratch/:
;;   rontolisp ch04/train-neuralnet.lisp

(load "two-layer-net.lisp")
(load "../dataset/mnist.lisp")

(defparameter *train-limit* 500)
(defparameter *test-limit* 200)
(defparameter *batch-size* 16)
(defparameter *learning-rate* 1.0)
(defparameter *epochs* 3)

(linalg:seed 42)

(let* ((x-train
        (mnist-load-images "dataset/train-images-idx3-ubyte" *train-limit*))
       (t-train
        (mnist-load-labels "dataset/train-labels-idx1-ubyte" *train-limit* 0 t))
       (x-test
        (mnist-load-images "dataset/t10k-images-idx3-ubyte" *test-limit*))
       (t-test
        (mnist-load-labels "dataset/t10k-labels-idx1-ubyte" *test-limit* 0 t))
       (params (make-two-layer-net 784 50 10))
       (iter-per-epoch (floor *train-limit* *batch-size*))
       (iters-num (* *epochs* iter-per-epoch)))
  (dotimes (i iters-num)
    (let* ((batch-mask (linalg:choice *train-limit* *batch-size*))
           (x-batch (linalg:take-rows x-train batch-mask))
           (t-batch (linalg:take-rows t-train batch-mask))
           (grads (tln-gradient params x-batch t-batch)))
      (tln-update! params grads *learning-rate*)
      (when (= (mod i 10) 0)
        (format t "iter ~3d  loss ~,4f~%" i (tln-loss params x-batch t-batch)))
      (when (= (mod (+ i 1) iter-per-epoch) 0)
        (format t "train acc, test acc | ~a/~a, ~a/~a~%"
                (tln-accuracy-count params x-train t-train) *train-limit*
                (tln-accuracy-count params x-test t-test) *test-limit*)))))


---

# FILE: references/examples/deep-learning-from-scratch/ch04/two-layer-net.lisp

;; ch04/two_layer_net.py -- the two-layer network, ch04 style (Deep
;; Learning from Scratch). A library file loaded by train-neuralnet.lisp.
;;
;; The network is its params dict -- a hash table keyed "W1" "b1" "W2" "b2"
;; like the book's -- and the operations are plain functions over it (the
;; class-based layer architecture arrives in ch05). Both gradient methods
;; are here: the numerical one the chapter builds, and the
;; sigmoid_grad-based analytic backprop the book ships alongside it for
;; speed.

(load "../common/functions.lisp")
(load "../common/gradient.lisp")

(defparameter *tln-keys* '("W1" "b1" "W2" "b2"))

(defun make-two-layer-net
    (input-size hidden-size output-size &optional (weight-init-std 0.01))
  (let ((params (make-hash-table :test 'equal)))
    (setf (gethash "W1" params)
     (linalg:mul weight-init-std (linalg:randn (list input-size hidden-size))))
    (setf (gethash "b1" params) (linalg:zeros hidden-size))
    (setf (gethash "W2" params)
     (linalg:mul weight-init-std (linalg:randn (list hidden-size output-size))))
    (setf (gethash "b2" params) (linalg:zeros output-size))
    params))

(defun tln-predict (params x)
  (let* ((a1
          (linalg:add (linalg:matmul x (gethash "W1" params))
                      (gethash "b1" params)))
         (z1 (sigmoid a1))
         (a2
          (linalg:add (linalg:matmul z1 (gethash "W2" params))
                      (gethash "b2" params))))
    (softmax a2)))

(defun tln-loss (params x target)
  (cross-entropy-error (tln-predict params x) target))

(defun tln-accuracy-count (params x target)
  ;; The number of correctly classified rows (one-hot target). An integer
  ;; count prints identically on every backend, where the book's float
  ;; accuracy would not; callers show it as count/total.
  (let ((y (linalg:argmax (tln-predict params x) :axis 1))
        (tl (linalg:argmax target :axis 1)))
    (truncate (linalg:sum (linalg:equal y tl)))))

(defun tln-numerical-gradient (params x target)
  ;; grads via central differences over every parameter element -- the
  ;; ch04 method (slow; used by ch05's gradient check).
  (let ((grads (make-hash-table :test 'equal))
        (loss-w (lambda (w) (tln-loss params x target))))
    (dolist (key *tln-keys*)
      (setf (gethash key grads)
            (numerical-gradient loss-w (gethash key params))))
    grads))

(defun tln-gradient (params x target)
  ;; The book's analytic shortcut: dy = (y - t)/batch, then the chain rule
  ;; through sigmoid_grad -- the un-abstracted form of ch05's backprop.
  (let* ((w2 (gethash "W2" params))
         (batch (car (linalg:shape x)))
         (a1
          (linalg:add (linalg:matmul x (gethash "W1" params))
                      (gethash "b1" params)))
         (z1 (sigmoid a1))
         (a2 (linalg:add (linalg:matmul z1 w2) (gethash "b2" params)))
         (y (softmax a2))
         (dy (linalg:div (linalg:sub y target) batch))
         (dz1 (linalg:matmul dy (linalg:transpose w2)))
         (da1 (linalg:mul (sigmoid-grad a1) dz1))
         (grads (make-hash-table :test 'equal)))
    (setf (gethash "W2" grads) (linalg:matmul (linalg:transpose z1) dy))
    (setf (gethash "b2" grads) (linalg:sum dy :axis 0))
    (setf (gethash "W1" grads) (linalg:matmul (linalg:transpose x) da1))
    (setf (gethash "b1" grads) (linalg:sum da1 :axis 0))
    grads))

(defun tln-update! (params grads lr)
  ;; params[key] -= lr * grads[key], element-wise IN PLACE like the book's
  ;; training loops (ch05's layers alias these same arrays).
  (dolist (key *tln-keys*)
    (let ((p (gethash key params)) (g (gethash key grads)))
      (dotimes (k (linalg:size p))
        (setf (row-major-aref p k)
              (- (row-major-aref p k) (* lr (row-major-aref g k))))))))


---

# FILE: references/examples/deep-learning-from-scratch/ch05/buy-apple-orange.lisp

;; ch05/buy_apple_orange.py -- backpropagation through the larger shopping
;; graph (Deep Learning from Scratch).
;;
;; price = (apple * apple_num + orange * orange_num) * tax; the add node
;; passes gradients through unchanged, the mul nodes swap their inputs.
;;
;;   rontolisp ch05/buy-apple-orange.lisp

(load "layer-naive.lisp")

(let ((apple 100)
      (apple-num 2)
      (orange 150)
      (orange-num 3)
      (tax 1.1)
      (mul-apple-layer (make-instance 'mul-layer))
      (mul-orange-layer (make-instance 'mul-layer))
      (add-apple-orange-layer (make-instance 'add-layer))
      (mul-tax-layer (make-instance 'mul-layer)))
  ;; forward
  (let* ((apple-price (forward2 mul-apple-layer apple apple-num))
         (orange-price (forward2 mul-orange-layer orange orange-num))
         (all-price (forward2 add-apple-orange-layer apple-price orange-price))
         (price (forward2 mul-tax-layer all-price tax)))
    ;; backward
    (let* ((dprice 1)
           (dtax-pair (backward2 mul-tax-layer dprice))
           (dall-price (car dtax-pair))
           (dtax (cadr dtax-pair))
           (dadd-pair (backward2 add-apple-orange-layer dall-price))
           (dapple-price (car dadd-pair))
           (dorange-price (cadr dadd-pair))
           (dorange-pair (backward2 mul-orange-layer dorange-price))
           (dorange (car dorange-pair))
           (dorange-num (cadr dorange-pair))
           (dapple-pair (backward2 mul-apple-layer dapple-price))
           (dapple (car dapple-pair))
           (dapple-num (cadr dapple-pair)))
      (format t "price: ~a~%" (truncate price))
      (format t "dApple: ~,1f~%" dapple)
      (format t "dApple_num: ~a~%" (truncate dapple-num))
      (format t "dOrange: ~,1f~%" dorange)
      (format t "dOrange_num: ~a~%" (truncate dorange-num))
      (format t "dTax: ~a~%" (truncate dtax)))))


---

# FILE: references/examples/deep-learning-from-scratch/ch05/buy-apple.lisp

;; ch05/buy_apple.py -- backpropagation through the apple-shopping graph
;; (Deep Learning from Scratch).
;;
;; price = apple * apple_num * tax, then the gradients flow backwards:
;; d(price)/d(apple) = 2.2, d/d(apple_num) = 110, d/d(tax) = 200.
;;
;;   rontolisp ch05/buy-apple.lisp

(load "layer-naive.lisp")

(let ((apple 100)
      (apple-num 2)
      (tax 1.1)
      (mul-apple-layer (make-instance 'mul-layer))
      (mul-tax-layer (make-instance 'mul-layer)))
  ;; forward
  (let* ((apple-price (forward2 mul-apple-layer apple apple-num))
         (price (forward2 mul-tax-layer apple-price tax)))
    ;; backward
    (let* ((dprice 1)
           (dtax-pair (backward2 mul-tax-layer dprice))
           (dapple-price (car dtax-pair))
           (dtax (cadr dtax-pair))
           (dapple-pair (backward2 mul-apple-layer dapple-price))
           (dapple (car dapple-pair))
           (dapple-num (cadr dapple-pair)))
      (format t "price: ~a~%" (truncate price))
      (format t "dApple: ~,1f~%" dapple)
      (format t "dApple_num: ~a~%" (truncate dapple-num))
      (format t "dTax: ~a~%" (truncate dtax)))))


---

# FILE: references/examples/deep-learning-from-scratch/ch05/gradient-check.lisp

;; ch05/gradient_check.py -- backpropagation vs numerical gradients (Deep
;; Learning from Scratch).
;;
;; The correctness gate of the whole layer library: the analytic gradients
;; from backprop must agree with the ch04 central differences to ~1e-9.
;; The book checks a 3-image MNIST batch through the full 784-50-10 net;
;; numerically differentiating its 39760 parameters is hours of
;; interpreter time, so this port checks a 20-10-10 net on a seeded random
;; batch instead -- the same mathematics, no data files, runs anywhere in
;; under a second. The raw mean differences are ~1e-10 and their last
;; digits differ per backend, so each key prints a PASS/FAIL verdict
;; against 1e-6 instead of the raw float.
;;
;;   rontolisp ch05/gradient-check.lisp

(load "two-layer-net.lisp")

(linalg:seed 42)

(let* ((net (make-two-layer-net 20 10 10))
       (x-batch (linalg:randn '(3 20)))
       (t-batch (linalg:one-hot (linalg:from-list '(1 7 3)) 10))
       (grad-numerical (net-numerical-gradient net x-batch t-batch))
       (grad-backprop (net-gradient net x-batch t-batch)))
  (dolist (key *tln-keys*)
    (let* ((gn (gethash key grad-numerical))
           (gb (gethash key grad-backprop))
           (diff
            (/ (linalg:sum (linalg:abs (linalg:sub gb gn))) (linalg:size gn))))
      (format t "~a: ~a~%" key
              (if (< diff 1.0e-6) "PASS (mean |diff| < 1e-6)" "FAIL")))))


---

# FILE: references/examples/deep-learning-from-scratch/ch05/layer-naive.lisp

;; ch05/layer_naive.py -- MulLayer and AddLayer (Deep Learning from
;; Scratch). A library file loaded by buy-apple.lisp and
;; buy-apple-orange.lisp.
;;
;; The book's first computational-graph layers, over plain numbers. Their
;; forward takes TWO inputs and backward returns TWO gradients, so they
;; get their own generic pair (forward2 / backward2, the latter returning
;; the list (dx dy)) -- generics have a fixed arity per name in
;; rontolisp's CLOS subset, and the array layers' forward/backward in
;; common/layers.lisp are one-input.

(defgeneric forward2 (layer x y))

(defgeneric backward2 (layer dout))

(defclass mul-layer ()
  ((x :initform nil :accessor mul-layer-x)
   (y :initform nil :accessor mul-layer-y)))

(defmethod forward2 ((layer mul-layer) x y)
  (setf (mul-layer-x layer) x)
  (setf (mul-layer-y layer) y)
  (* x y))

(defmethod backward2 ((layer mul-layer) dout)
  ;; dx = dout * y, dy = dout * x -- the swap is the point of the chapter.
  (list (* dout (mul-layer-y layer)) (* dout (mul-layer-x layer))))

(defclass add-layer ())

(defmethod forward2 ((layer add-layer) x y) (+ x y))

(defmethod backward2 ((layer add-layer) dout) (list dout dout))


---

# FILE: references/examples/deep-learning-from-scratch/ch05/train-neuralnet.lisp

;; ch05/train_neuralnet.py -- mini-batch training through the layer-based
;; network (Deep Learning from Scratch).
;;
;; The same training loop as ch04's, but the gradient now comes from
;; backpropagation through the Affine -> Relu -> Affine layer objects --
;; the payoff of the chapter. Scaled like ch04 (500 train images, batch
;; 16, three epochs; lr 0.3 instead of the book's 0.1, tuned for the small
;; set -- the ReLU net needs less than ch04's slow-starting sigmoid took);
;; raise the knobs for a longer run, or add --simd for the same output
;; faster.
;;
;; Data: run ../download-mnist.sh once, then from examples/deep-learning-from-scratch/:
;;   rontolisp ch05/train-neuralnet.lisp

(load "two-layer-net.lisp")
(load "../dataset/mnist.lisp")

(defparameter *train-limit* 500)
(defparameter *test-limit* 200)
(defparameter *batch-size* 16)
(defparameter *learning-rate* 0.3)
(defparameter *epochs* 3)

(linalg:seed 42)

(defun update-params! (params grads lr)
  ;; params[key] -= lr * grads[key], IN PLACE (the layers alias the same
  ;; arrays).
  (dolist (key *tln-keys*)
    (let ((p (gethash key params)) (g (gethash key grads)))
      (dotimes (k (linalg:size p))
        (setf (row-major-aref p k)
              (- (row-major-aref p k) (* lr (row-major-aref g k))))))))

(let* ((x-train
        (mnist-load-images "dataset/train-images-idx3-ubyte" *train-limit*))
       (t-train
        (mnist-load-labels "dataset/train-labels-idx1-ubyte" *train-limit* 0 t))
       (x-test
        (mnist-load-images "dataset/t10k-images-idx3-ubyte" *test-limit*))
       (t-test
        (mnist-load-labels "dataset/t10k-labels-idx1-ubyte" *test-limit* 0 t))
       (net (make-two-layer-net 784 50 10))
       (iter-per-epoch (floor *train-limit* *batch-size*))
       (iters-num (* *epochs* iter-per-epoch)))
  (dotimes (i iters-num)
    (let* ((batch-mask (linalg:choice *train-limit* *batch-size*))
           (x-batch (linalg:take-rows x-train batch-mask))
           (t-batch (linalg:take-rows t-train batch-mask))
           (grads (net-gradient net x-batch t-batch)))
      (update-params! (net-params net) grads *learning-rate*)
      (when (= (mod i 10) 0)
        (format t "iter ~3d  loss ~,4f~%" i (net-loss net x-batch t-batch)))
      (when (= (mod (+ i 1) iter-per-epoch) 0)
        (format t "train acc, test acc | ~a/~a, ~a/~a~%"
                (net-accuracy-count net x-train t-train) *train-limit*
                (net-accuracy-count net x-test t-test) *test-limit*)))))


---

# FILE: references/examples/deep-learning-from-scratch/ch05/two-layer-net.lisp

;; ch05/two_layer_net.py -- the layer-based two-layer network (Deep
;; Learning from Scratch). A library file loaded by gradient-check.lisp
;; and train-neuralnet.lisp.
;;
;; The ch04 net rewritten over the CLOS layers of common/layers.lisp: the
;; network is Affine -> Relu -> Affine chained through an ordered layer
;; LIST (the book's OrderedDict), plus SoftmaxWithLoss at the end. The
;; params hash holds the SAME array objects the affine layers do, so an
;; in-place optimizer update is immediately visible to the layers -- the
;; book's aliasing contract. Backprop is one fold over the reversed list.

(load "../common/layers.lisp")
(load "../common/gradient.lisp")

(defclass two-layer-net ()
  ((params :initarg :params :accessor net-params)
   (layers :initarg :layers :accessor net-layers)
   (affine1 :initarg :affine1 :accessor net-affine1)
   (affine2 :initarg :affine2 :accessor net-affine2)
   (last-layer :initarg :last-layer :accessor net-last-layer)))

(defparameter *tln-keys* '("W1" "b1" "W2" "b2"))

(defun make-two-layer-net
    (input-size hidden-size output-size &optional (weight-init-std 0.01))
  (let ((params (make-hash-table :test 'equal)))
    (setf (gethash "W1" params)
     (linalg:mul weight-init-std (linalg:randn (list input-size hidden-size))))
    (setf (gethash "b1" params) (linalg:zeros hidden-size))
    (setf (gethash "W2" params)
     (linalg:mul weight-init-std (linalg:randn (list hidden-size output-size))))
    (setf (gethash "b2" params) (linalg:zeros output-size))
    (let ((affine1
           (make-instance 'affine
                          :w (gethash "W1" params)
                          :b (gethash "b1" params)))
          (affine2
           (make-instance 'affine
                          :w (gethash "W2" params)
                          :b (gethash "b2" params))))
      (make-instance 'two-layer-net
                     :params params
                     :layers (list affine1 (make-instance 'relu-layer) affine2)
                     :affine1 affine1
                     :affine2 affine2
                     :last-layer (make-instance 'softmax-with-loss)))))

(defgeneric predict (net x))

(defgeneric net-loss (net x target))

(defgeneric net-gradient (net x target))

(defgeneric net-numerical-gradient (net x target))

(defgeneric net-accuracy-count (net x target))

(defmethod predict ((net two-layer-net) x)
  (let ((out x))
    (dolist (layer (net-layers net)) (setq out (forward layer out)))
    out))

(defmethod net-loss ((net two-layer-net) x target)
  (loss-forward (net-last-layer net) (predict net x) target))

(defmethod net-accuracy-count ((net two-layer-net) x target)
  ;; Correctly classified rows; target may be one-hot or a label vector.
  (let ((y (linalg:argmax (predict net x) :axis 1))
        (tl
         (if (= (linalg:ndim target) 1) target (linalg:argmax target :axis 1))))
    (truncate (linalg:sum (linalg:equal y tl)))))

(defmethod net-gradient ((net two-layer-net) x target)
  ;; forward for the caches, then fold backward over the reversed layers;
  ;; the affine layers leave dW/db behind.
  (net-loss net x target)
  (let ((dout (loss-backward (net-last-layer net))))
    (dolist (layer (reverse (net-layers net)))
      (setq dout (backward layer dout))))
  (let ((grads (make-hash-table :test 'equal)))
    (setf (gethash "W1" grads) (affine-dw (net-affine1 net)))
    (setf (gethash "b1" grads) (affine-db (net-affine1 net)))
    (setf (gethash "W2" grads) (affine-dw (net-affine2 net)))
    (setf (gethash "b2" grads) (affine-db (net-affine2 net)))
    grads))

(defmethod net-numerical-gradient ((net two-layer-net) x target)
  ;; Central differences over every parameter element (slow; the
  ;; gradient-check yardstick).
  (let ((grads (make-hash-table :test 'equal))
        (loss-w (lambda (w) (net-loss net x target))))
    (dolist (key *tln-keys*)
      (setf (gethash key grads)
            (numerical-gradient loss-w (gethash key (net-params net)))))
    grads))


---

# FILE: references/examples/deep-learning-from-scratch/ch06/batch-norm-gradient-check.lisp

;; ch06/batch_norm_gradient_check.py -- backprop through Batch
;; Normalization vs numerical gradients (Deep Learning from Scratch).
;;
;; The BatchNormalization backward pass is the most intricate derivation
;; in the book; this check pins it against central differences. Like
;; ch05/gradient-check.lisp, a small seeded 8-(5 4)-3 net over a random
;; batch replaces the book's 784-100-100-10 MNIST batch (numerically
;; differentiating that takes hours of interpreter time) -- same
;; mathematics, no data files. gamma/beta gradients are checked too.
;;
;;   rontolisp ch06/batch-norm-gradient-check.lisp

(load "../common/multi-layer-net-extend.lisp")

(linalg:seed 42)

(let* ((net (make-multi-layer-net-extend 8 '(5 4) 3 :use-batchnorm t))
       (x-batch (linalg:randn '(4 8)))
       (t-batch (linalg:one-hot (linalg:from-list '(1 0 2 1)) 3))
       (grad-numerical (net-numerical-gradient net x-batch t-batch))
       (grad-backprop (net-gradient net x-batch t-batch)))
  (dolist (key (mlne-keys net))
    (let* ((gn (gethash key grad-numerical))
           (gb (gethash key grad-backprop))
           (diff
            (/ (linalg:sum (linalg:abs (linalg:sub gb gn))) (linalg:size gn))))
      (format t "~a: ~a~%" key
              (if (< diff 1.0e-5) "PASS (mean |diff| < 1e-5)" "FAIL")))))


---

# FILE: references/examples/deep-learning-from-scratch/ch06/batch-norm-test.lisp

;; ch06/batch_norm_test.py -- does Batch Normalization help? (Deep
;; Learning from Scratch).
;;
;; The book trains a with-BN / without-BN pair for 16 weight-init scales
;; and plots the accuracy curves; here two representative scales are run
;; (a healthy 0.1 and a starved 0.01) and the per-epoch train accuracies
;; are printed side by side. With badly scaled initial weights the plain
;; net cannot start learning while the BN net can -- the book's punchline.
;;
;; Data: run ../download-mnist.sh once, then from examples/deep-learning-from-scratch/:
;;   rontolisp ch06/batch-norm-test.lisp   (add --simd to speed it up)

(load "../dataset/mnist.lisp")
(load "../common/multi-layer-net-extend.lisp")
(load "../common/optimizer.lisp")

(defparameter *train-limit* 300)
(defparameter *batch-size* 16)
(defparameter *epochs* 3)

(defun train-once (x-train t-train w-scale use-bn)
  ;; Returns the list of per-epoch train-accuracy counts.
  (linalg:seed 42)
  (let ((net
         (make-multi-layer-net-extend 784 '(20 20) 10
                                      :weight-init-std w-scale
                                      :use-batchnorm use-bn))
        (opt (make-instance 'sgd :lr 0.5))
        (iter-per-epoch (floor *train-limit* *batch-size*))
        (accs nil))
    (dotimes (i (* *epochs* iter-per-epoch))
      (let* ((batch-mask (linalg:choice *train-limit* *batch-size*))
             (x-batch (linalg:take-rows x-train batch-mask))
             (t-batch (linalg:take-rows t-train batch-mask))
             (grads (net-gradient net x-batch t-batch)))
        (update opt (mlne-params net) grads)
        (when (= (mod (+ i 1) iter-per-epoch) 0)
          (setq accs (cons (net-accuracy-count net x-train t-train) accs)))))
    (reverse accs)))

(let ((x-train
       (mnist-load-images "dataset/train-images-idx3-ubyte" *train-limit*))
      (t-train
       (mnist-load-labels "dataset/train-labels-idx1-ubyte" *train-limit* 0 t)))
  (dolist (w-scale '(0.1 0.01))
    (format t "w-scale = ~a (train acc per epoch, of ~a):~%" w-scale
            *train-limit*)
    (format t "  with BatchNorm:    ~a~%"
            (train-once x-train t-train w-scale t))
    (format t "  without BatchNorm: ~a~%"
            (train-once x-train t-train w-scale nil))))


---

# FILE: references/examples/deep-learning-from-scratch/ch06/hyperparameter-optimization.lisp

;; ch06/hyperparameter_optimization.py -- random search over the learning
;; rate and weight-decay strength (Deep Learning from Scratch).
;;
;; lr is drawn log-uniformly (the book searches 10^U(-6, -2) over 50
;; epochs; three epochs need livelier nets, so this port searches
;; 10^U(-3, 0)) and lambda from 10^U(-8, -4); each trial
;; trains briefly on a small train/validation split, and the trials are
;; ranked by validation accuracy -- random search over log-uniform ranges,
;; the book's recipe. The book runs 100 trials of 50 epochs; here 8 trials
;; of 3 epochs on a [10 10] net keep the interpreter run short, and the
;; ranking still finds the healthy-lr region.
;;
;; Data: run ../download-mnist.sh once, then from examples/deep-learning-from-scratch/:
;;   rontolisp ch06/hyperparameter-optimization.lisp   (add --simd to speed it up)

(load "../dataset/mnist.lisp")
(load "../common/multi-layer-net.lisp")
(load "../common/trainer.lisp")

(defparameter *train-limit* 500)
(defparameter *validation-rate* 0.2)
(defparameter *trials* 8)
(defparameter *epochs* 3)

(linalg:seed 42)

(let* ((x-all
        (mnist-load-images "dataset/train-images-idx3-ubyte" *train-limit*))
       (t-all
        (mnist-load-labels "dataset/train-labels-idx1-ubyte" *train-limit* 0 t))
       ;; shuffle, then split off the validation set (the book's
       ;; shuffle_dataset + slicing)
       (perm (linalg:permutation *train-limit*))
       (x-shuffled (linalg:take-rows x-all perm))
       (t-shuffled (linalg:take-rows t-all perm))
       (val-num (truncate (* *train-limit* *validation-rate*)))
       (x-val (linalg:take-rows x-shuffled (linalg:arange val-num)))
       (t-val (linalg:take-rows t-shuffled (linalg:arange val-num)))
       (x-train
        (linalg:take-rows x-shuffled (linalg:arange val-num *train-limit*)))
       (t-train
        (linalg:take-rows t-shuffled (linalg:arange val-num *train-limit*)))
       (results nil))
  (dotimes (trial *trials*)
    (let* ((lr (expt 10.0 (aref (linalg:uniform -3.0 0.0 1) 0)))
           (lam (expt 10.0 (aref (linalg:uniform -8.0 -4.0 1) 0)))
           (net (make-multi-layer-net 784 '(10 10) 10 :weight-decay-lambda lam))
           (accs
            (train net (mln-params net) x-train t-train x-val t-val
                   :epochs *epochs*
                   :mini-batch-size 16
                   :optimizer (make-instance 'sgd :lr lr)
                   :verbose nil))
           (final-val (cadr (car (last accs)))))
      (format t "trial ~a: val acc ~a/~a | lr ~,6f, weight decay ~,10f~%"
              (+ trial 1) final-val val-num lr lam)
      (setq results (cons (list final-val (+ trial 1) lr) results))))
  (format t "=========== ranking (by val acc) ===========~%")
  (dolist (r (sort results (lambda (a b) (> (car a) (car b)))))
    (format t "trial ~a: val acc ~a/~a (lr ~,6f)~%" (cadr r) (car r) val-num
            (caddr r))))


---

# FILE: references/examples/deep-learning-from-scratch/ch06/optimizer-compare-mnist.lisp

;; ch06/optimizer_compare_mnist.py -- the four optimizers on MNIST (Deep
;; Learning from Scratch).
;;
;; The book races SGD / Momentum / AdaGrad / Adam over a [100 100 100 100]
;; net for 2000 iterations and plots the smoothed losses; here each
;; optimizer trains a [30 30] net (fresh seeded weights per run) and the
;; batch loss is printed every 20 iterations. Adam and AdaGrad pull ahead
;; of plain SGD early, like the book's figure.
;;
;; Data: run ../download-mnist.sh once, then from examples/deep-learning-from-scratch/:
;;   rontolisp ch06/optimizer-compare-mnist.lisp   (add --simd to speed it up)

(load "../dataset/mnist.lisp")
(load "../common/multi-layer-net.lisp")
(load "../common/optimizer.lisp")

(defparameter *train-limit* 500)
(defparameter *batch-size* 16)
(defparameter *iters* 100)

(let ((x-train
       (mnist-load-images "dataset/train-images-idx3-ubyte" *train-limit*))
      (t-train
       (mnist-load-labels "dataset/train-labels-idx1-ubyte" *train-limit* 0 t)))
  ;; the book compares the optimizers at their class defaults
  ;; (SGD/Momentum/AdaGrad lr 0.01, Adam lr 0.001)
  (dolist (entry
           (list (list "SGD" (make-instance 'sgd))
                 (list "Momentum" (make-instance 'momentum))
                 (list "AdaGrad" (make-instance 'adagrad))
                 (list "Adam" (make-instance 'adam))))
    (let ((name (car entry)) (opt (cadr entry)))
      (linalg:seed 42)
      (let ((net (make-multi-layer-net 784 '(30 30) 10)))
        (format t "~a:~%" name)
        (dotimes (i *iters*)
          (let* ((batch-mask (linalg:choice *train-limit* *batch-size*))
                 (x-batch (linalg:take-rows x-train batch-mask))
                 (t-batch (linalg:take-rows t-train batch-mask))
                 (grads (net-gradient net x-batch t-batch)))
            (update opt (mln-params net) grads)
            (when (= (mod i 20) 0)
              (format t "  iter ~3d  loss ~,4f~%" i
                      (net-loss net x-batch t-batch)))))
        (let ((mask (linalg:arange *train-limit*)))
          (format t "  final train acc: ~a/~a~%"
                  (net-accuracy-count net (linalg:take-rows x-train mask)
                                      (linalg:take-rows t-train mask))
                  *train-limit*))))))


---

# FILE: references/examples/deep-learning-from-scratch/ch06/optimizer-compare-naive.lisp

;; ch06/optimizer_compare_naive.py -- SGD / Momentum / AdaGrad / Adam on
;; f(x, y) = x^2/20 + y^2 (Deep Learning from Scratch).
;;
;; The book draws each optimizer's trajectory over the contour plot; here
;; every 5th position is printed instead. The anisotropic bowl is the
;; point: SGD zigzags, Momentum overshoots and swings back, AdaGrad's
;; per-axis scaling goes straight, Adam curves gently. The gradient is
;; analytic (df = x/10, 2y) -- no exp/log -- so the output is
;; byte-identical on every backend.
;;
;;   rontolisp ch06/optimizer-compare-naive.lisp

(load "../common/optimizer.lisp")

;; params/grads hold 1-element linalg vectors so the shared optimizer
;; update (element loops over arrays) applies unchanged.
(defun run-optimizer (name opt)
  (let ((params (make-hash-table :test 'equal))
        (grads (make-hash-table :test 'equal)))
    (setf (gethash "x" params) (linalg:from-list '(-7.0)))
    (setf (gethash "y" params) (linalg:from-list '(2.0)))
    (setf (gethash "x" grads) (linalg:zeros 1))
    (setf (gethash "y" grads) (linalg:zeros 1))
    (format t "~a:~%" name)
    (let ((px (gethash "x" params))
          (py (gethash "y" params))
          (gx (gethash "x" grads))
          (gy (gethash "y" grads)))
      (dotimes (i 30)
        (when (= (mod i 5) 0)
          (format t "  step ~2d: (~,6f, ~,6f)~%" i (aref px 0) (aref py 0)))
        (setf (aref gx 0) (/ (aref px 0) 10.0))
        (setf (aref gy 0) (* 2.0 (aref py 0)))
        (update opt params grads))
      (format t "  final:   (~,6f, ~,6f)~%" (aref px 0) (aref py 0)))))

(run-optimizer "SGD" (make-instance 'sgd :lr 0.95))
(run-optimizer "Momentum" (make-instance 'momentum :lr 0.1))
(run-optimizer "AdaGrad" (make-instance 'adagrad :lr 1.5))
(run-optimizer "Adam" (make-instance 'adam :lr 0.3))


---

# FILE: references/examples/deep-learning-from-scratch/ch06/overfit-dropout.lisp

;; ch06/overfit_dropout.py -- taming overfitting with Dropout (Deep
;; Learning from Scratch).
;;
;; The same 300-image overfitting setup as overfit-weight-decay.lisp, but
;; the cure is Dropout: randomly silencing units during training keeps
;; train accuracy from saturating and narrows the train/test gap. Uses
;; the shared trainer (common/trainer.lisp), like the book's script uses
;; its Trainer class. The book runs dropout 0.2 with lr 0.01 for 301
;; epochs on a [100]*6 net; here the same ratio with lr 0.2 for 20 epochs
;; on [20 20] -- dropout keeps train accuracy off the 100% ceiling.
;;
;; Data: run ../download-mnist.sh once, then from examples/deep-learning-from-scratch/:
;;   rontolisp ch06/overfit-dropout.lisp   (add --simd to speed it up)

(load "../dataset/mnist.lisp")
(load "../common/multi-layer-net-extend.lisp")
(load "../common/trainer.lisp")

(defparameter *train-limit* 300)
(defparameter *test-limit* 300)

(let ((x-train
       (mnist-load-images "dataset/train-images-idx3-ubyte" *train-limit*))
      (t-train
       (mnist-load-labels "dataset/train-labels-idx1-ubyte" *train-limit* 0 t))
      (x-test (mnist-load-images "dataset/t10k-images-idx3-ubyte" *test-limit*))
      (t-test
       (mnist-load-labels "dataset/t10k-labels-idx1-ubyte" *test-limit* 0 t)))
  (format t "without dropout (overfits):~%")
  (linalg:seed 42)
  (let ((net (make-multi-layer-net-extend 784 '(20 20) 10)))
    (train net (mlne-params net) x-train t-train x-test t-test
           :epochs 20
           :mini-batch-size 32
           :optimizer (make-instance 'sgd :lr 0.2)))
  (format t "with dropout 0.2:~%")
  (linalg:seed 42)
  (let ((net
         (make-multi-layer-net-extend 784 '(20 20) 10
                                      :use-dropout t
                                      :dropout-ratio 0.2)))
    (train net (mlne-params net) x-train t-train x-test t-test
           :epochs 20
           :mini-batch-size 32
           :optimizer (make-instance 'sgd :lr 0.2))))


---

# FILE: references/examples/deep-learning-from-scratch/ch06/overfit-weight-decay.lisp

;; ch06/overfit_weight_decay.py -- provoking and taming overfitting with
;; weight decay (Deep Learning from Scratch).
;;
;; An oversized net is trained on only 300 images so it memorizes them:
;; train accuracy runs toward 100% while test accuracy stalls -- the gap
;; IS the overfitting. The same run with L2 weight decay (lambda = 0.1)
;; keeps train accuracy from saturating (the book's own figure shows the
;; same signature: decay mainly caps the memorization). The book trains
;; [100]*6 with lr 0.01 for 200 epochs; here a [30 30] net with lr 0.2 for
;; 30 epochs and lambda 0.05 shows the same shape.
;;
;; Data: run ../download-mnist.sh once, then from examples/deep-learning-from-scratch/:
;;   rontolisp ch06/overfit-weight-decay.lisp   (add --simd to speed it up)

(load "../dataset/mnist.lisp")
(load "../common/multi-layer-net.lisp")
(load "../common/optimizer.lisp")

(defparameter *train-limit* 300)
(defparameter *test-limit* 300)
(defparameter *batch-size* 32)
(defparameter *epochs* 20)

(defun train-once (x-train t-train x-test t-test lam)
  (linalg:seed 42)
  (let ((net (make-multi-layer-net 784 '(20 20) 10 :weight-decay-lambda lam))
        (opt (make-instance 'sgd :lr 0.2))
        (iter-per-epoch (floor *train-limit* *batch-size*))
        (epoch 0))
    (dotimes (i (* *epochs* iter-per-epoch))
      (let* ((batch-mask (linalg:choice *train-limit* *batch-size*))
             (x-batch (linalg:take-rows x-train batch-mask))
             (t-batch (linalg:take-rows t-train batch-mask))
             (grads (net-gradient net x-batch t-batch)))
        (update opt (mln-params net) grads)
        (when (= (mod (+ i 1) iter-per-epoch) 0)
          (setq epoch (+ epoch 1))
          (when (= (mod epoch 6) 0)
            (format t "  epoch ~2d: train ~a/~a, test ~a/~a~%" epoch
                    (net-accuracy-count net x-train t-train) *train-limit*
                    (net-accuracy-count net x-test t-test) *test-limit*)))))))

(let ((x-train
       (mnist-load-images "dataset/train-images-idx3-ubyte" *train-limit*))
      (t-train
       (mnist-load-labels "dataset/train-labels-idx1-ubyte" *train-limit* 0 t))
      (x-test (mnist-load-images "dataset/t10k-images-idx3-ubyte" *test-limit*))
      (t-test
       (mnist-load-labels "dataset/t10k-labels-idx1-ubyte" *test-limit* 0 t)))
  (format t "weight decay lambda = 0 (overfits):~%")
  (train-once x-train t-train x-test t-test 0)
  (format t "weight decay lambda = 0.05:~%")
  (train-once x-train t-train x-test t-test 0.05))


---

# FILE: references/examples/deep-learning-from-scratch/ch06/weight-init-activation-histogram.lisp

;; ch06/weight_init_activation_histogram.py -- activation distributions
;; under different weight initializations (Deep Learning from Scratch).
;;
;; 1000 samples flow through five 100-unit sigmoid layers; the book
;; histograms each layer's activations for std=1, std=0.01 and the Xavier
;; initialization. Text buckets replace the plots: std=1 saturates at 0/1
;; (vanishing gradients), std=0.01 collapses to 0.5 (no representation
;; power), Xavier stays spread out -- the book's three panels.
;;
;;   rontolisp ch06/weight-init-activation-histogram.lisp

(load "../common/functions.lisp")

(defparameter *hidden-layer-num* 5)
(defparameter *node-num* 100)

(defun activation-histogram (init-name w-scale)
  (linalg:seed 1)
  (let ((x (linalg:randn '(1000 100))) (activations nil))
    (dotimes (i *hidden-layer-num*)
      (let* ((w
              (linalg:mul w-scale (linalg:randn (list *node-num* *node-num*))))
             (a (linalg:matmul x w))
             (z (sigmoid a)))
        (setq activations (cons z activations))
        (setq x z)))
    (format t "~a:~%" init-name)
    (let ((idx 0))
      (dolist (z (reverse activations))
        (setq idx (+ idx 1))
        ;; ten buckets over [0, 1]
        (let ((counts (make-array 10 :initial-element 0)))
          (dotimes (k (linalg:size z))
            (let ((b (min 9 (truncate (* 10 (row-major-aref z k))))))
              (setf (aref counts b) (+ (aref counts b) 1))))
          (format t "  layer ~a:" idx)
          (dotimes (b 10) (format t " ~5d" (aref counts b)))
          (format t "~%"))))))

(activation-histogram "std = 1.0" 1.0)
(activation-histogram "std = 0.01" 0.01)
(activation-histogram "Xavier (sqrt(1/n))" (sqrt (/ 1.0 100)))


---

# FILE: references/examples/deep-learning-from-scratch/ch06/weight-init-compare.lisp

;; ch06/weight_init_compare.py -- std=0.01 vs Xavier vs He on MNIST (Deep
;; Learning from Scratch).
;;
;; Three identical [30 30] ReLU nets, differing only in weight
;; initialization, race under plain SGD: std=0.01 stalls (its activations
;; carry no signal), Xavier learns, He learns fastest with ReLU -- the
;; book's figure as printed losses.
;;
;; Data: run ../download-mnist.sh once, then from examples/deep-learning-from-scratch/:
;;   rontolisp ch06/weight-init-compare.lisp   (add --simd to speed it up)

(load "../dataset/mnist.lisp")
(load "../common/multi-layer-net.lisp")
(load "../common/optimizer.lisp")

(defparameter *train-limit* 500)
(defparameter *batch-size* 16)
(defparameter *iters* 100)

(let ((x-train
       (mnist-load-images "dataset/train-images-idx3-ubyte" *train-limit*))
      (t-train
       (mnist-load-labels "dataset/train-labels-idx1-ubyte" *train-limit* 0 t)))
  (dolist (entry
           (list (list "std=0.01" 0.01) (list "Xavier" 'xavier)
                 (list "He" 'he)))
    (let ((name (car entry))
          (init (cadr entry))
          (opt (make-instance 'sgd :lr 0.5)))
      (linalg:seed 42)
      (let ((net (make-multi-layer-net 784 '(30 30) 10 :weight-init-std init)))
        (format t "~a:~%" name)
        (dotimes (i *iters*)
          (let* ((batch-mask (linalg:choice *train-limit* *batch-size*))
                 (x-batch (linalg:take-rows x-train batch-mask))
                 (t-batch (linalg:take-rows t-train batch-mask))
                 (grads (net-gradient net x-batch t-batch)))
            (update opt (mln-params net) grads)
            (when (= (mod i 20) 0)
              (format t "  iter ~3d  loss ~,4f~%" i
                      (net-loss net x-batch t-batch)))))))))


---

# FILE: references/examples/deep-learning-from-scratch/ch07/gradient-check.lisp

;; ch07/gradient_check.py -- backprop vs numerical gradients through the
;; CNN (Deep Learning from Scratch).
;;
;; The correctness gate of the Convolution/Pooling layers: the analytic
;; gradients from backprop (im2col/col2im, the argmax scatter) must agree
;; with the ch04 central differences. The book checks a synthetic 10x10
;; single image through a filter_num=10 SimpleConvNet; numerically
;; sweeping its 3400 parameters is nearly a minute of interpreter time,
;; so this port shrinks filter-num to 3 (630 parameters) -- the same
;; mathematics, no data files, seconds anywhere. weight-init-std is 0.1
;; instead of the default 0.01: with 0.01-scale activations the h = 1e-4
;; central difference itself crosses ReLU kinks and flips pooling argmax
;; ties, polluting the NUMERICAL side to ~1e-5 (backprop is exact either
;; way). The raw mean differences are ~1e-8 and their last digits differ
;; per backend, so each key prints a PASS/FAIL verdict against 1e-6
;; instead of the raw float.
;;
;;   rontolisp ch07/gradient-check.lisp

(load "simple-convnet.lisp")

(linalg:seed 42)

(let* ((net
        (make-simple-convnet :input-dim '(1 10 10)
                             :filter-num 3
                             :filter-size 3
                             :filter-pad 0
                             :filter-stride 1
                             :hidden-size 10
                             :output-size 10
                             :weight-init-std 0.1))
       (x-batch (linalg:reshape (linalg:rand 100) '(1 1 10 10)))
       (t-batch (linalg:one-hot #(1) 10))
       (grad-numerical (net-numerical-gradient net x-batch t-batch))
       (grad-backprop (net-gradient net x-batch t-batch)))
  (dolist (key *scn-keys*)
    (let* ((gn (gethash key grad-numerical))
           (gb (gethash key grad-backprop))
           (diff
            (/ (linalg:sum (linalg:abs (linalg:sub gb gn))) (linalg:size gn))))
      (format t "~a: ~a~%" key
              (if (< diff 1.0e-6) "PASS (mean |diff| < 1e-6)" "FAIL")))))


---

# FILE: references/examples/deep-learning-from-scratch/ch07/simple-convnet.lisp

;; ch07/simple_convnet.py -- the first CNN (Deep Learning from Scratch).
;; A library file loaded by gradient-check.lisp and train-convnet.lisp.
;;
;; Conv -> Relu -> Pool(2x2) -> Affine -> Relu -> Affine, plus
;; SoftmaxWithLoss, over the CLOS layers of common/layers.lisp (the ch07
;; Convolution/Pooling included). As in ch05, the params hash holds the
;; SAME array objects the layers do, so an in-place optimizer update is
;; immediately visible -- the book's aliasing contract. Inputs are rank-4
;; NCHW batches (reshape the flat MNIST rows to (n 1 28 28) first).

(load "../common/layers.lisp")
(load "../common/gradient.lisp")
(require "rlw1" "../dataset/rlw1.lisp")

(defclass simple-convnet ()
  ((params :initarg :params :accessor scn-params)
   (layers :initarg :layers :accessor scn-layers)
   (conv1 :initarg :conv1 :accessor scn-conv1)
   (affine1 :initarg :affine1 :accessor scn-affine1)
   (affine2 :initarg :affine2 :accessor scn-affine2)
   (last-layer :initarg :last-layer :accessor scn-last-layer)))

(defparameter *scn-keys* '("W1" "b1" "W2" "b2" "W3" "b3"))

(defun make-simple-convnet (&key (input-dim '(1 28 28)) (filter-num 30)
                                 (filter-size 5) (filter-pad 0)
                                 (filter-stride 1) (hidden-size 100)
                                 (output-size 10) (weight-init-std 0.01))
  (let* ((channels (car input-dim))
         (input-size (car (cdr input-dim)))
         (conv-output-size
          (+ 1
             (floor (- (+ input-size (* 2 filter-pad)) filter-size)
                    filter-stride)))
         (pool-output-size
          (* filter-num (floor conv-output-size 2) (floor conv-output-size 2)))
         (params (make-hash-table :test 'equal)))
    (setf (gethash "W1" params)
          (linalg:mul weight-init-std
           (linalg:randn (list filter-num channels filter-size filter-size))))
    (setf (gethash "b1" params) (linalg:zeros filter-num))
    (setf (gethash "W2" params)
          (linalg:mul weight-init-std
                      (linalg:randn (list pool-output-size hidden-size))))
    (setf (gethash "b2" params) (linalg:zeros hidden-size))
    (setf (gethash "W3" params)
     (linalg:mul weight-init-std (linalg:randn (list hidden-size output-size))))
    (setf (gethash "b3" params) (linalg:zeros output-size))
    (let ((conv1
           (make-instance 'convolution
                          :w (gethash "W1" params)
                          :b (gethash "b1" params)
                          :stride filter-stride
                          :pad filter-pad))
          (affine1
           (make-instance 'affine
                          :w (gethash "W2" params)
                          :b (gethash "b2" params)))
          (affine2
           (make-instance 'affine
                          :w (gethash "W3" params)
                          :b (gethash "b3" params))))
      (make-instance 'simple-convnet
                     :params params
                     :layers (list conv1 (make-instance 'relu-layer)
                                   (make-instance 'pooling
                                                  :pool-h 2
                                                  :pool-w 2
                                                  :stride 2) affine1
                                   (make-instance 'relu-layer) affine2)
                     :conv1 conv1
                     :affine1 affine1
                     :affine2 affine2
                     :last-layer (make-instance 'softmax-with-loss)))))

(defgeneric predict (net x))

(defgeneric net-loss (net x target))

(defgeneric net-gradient (net x target))

(defgeneric net-numerical-gradient (net x target))

(defgeneric net-accuracy-count (net x target))

(defmethod predict ((net simple-convnet) x)
  (let ((out x))
    (dolist (layer (scn-layers net)) (setq out (forward layer out)))
    out))

(defmethod net-loss ((net simple-convnet) x target)
  (loss-forward (scn-last-layer net) (predict net x) target))

(defmethod net-accuracy-count ((net simple-convnet) x target)
  ;; Correctly classified batch rows; target may be one-hot or labels.
  (let ((y (linalg:argmax (predict net x) :axis 1))
        (tl
         (if (= (linalg:ndim target) 1) target (linalg:argmax target :axis 1))))
    (truncate (linalg:sum (linalg:equal y tl)))))

(defmethod net-gradient ((net simple-convnet) x target)
  ;; forward for the caches, then fold backward over the reversed layers;
  ;; conv1 leaves dW/db behind like the affine layers.
  (net-loss net x target)
  (let ((dout (loss-backward (scn-last-layer net))))
    (dolist (layer (reverse (scn-layers net)))
      (setq dout (backward layer dout))))
  (let ((grads (make-hash-table :test 'equal)))
    (setf (gethash "W1" grads) (conv-dw (scn-conv1 net)))
    (setf (gethash "b1" grads) (conv-db (scn-conv1 net)))
    (setf (gethash "W2" grads) (affine-dw (scn-affine1 net)))
    (setf (gethash "b2" grads) (affine-db (scn-affine1 net)))
    (setf (gethash "W3" grads) (affine-dw (scn-affine2 net)))
    (setf (gethash "b3" grads) (affine-db (scn-affine2 net)))
    grads))

(defmethod net-numerical-gradient ((net simple-convnet) x target)
  ;; Central differences over every parameter element (slow; the
  ;; gradient-check yardstick).
  (let ((grads (make-hash-table :test 'equal))
        (loss-w (lambda (w) (net-loss net x target))))
    (dolist (key *scn-keys*)
      (setf (gethash key grads)
            (numerical-gradient loss-w (gethash key (scn-params net)))))
    grads))

(defgeneric net-load-params (net path element-type))

(defmethod net-load-params ((net simple-convnet) path element-type)
  ;; The book's load_params over an RLW1 export of params.pkl (W1 b1 W2 b2
  ;; W3 b3): replace the hash entries, then re-point the parameter layers'
  ;; shared arrays. element-type nil = double, 'single-float = packed #f
  ;; (linalg is width-polymorphic, so the whole net then runs single).
  (let ((params (scn-params net)))
    (do ((keys *scn-keys* (cdr keys))
         (arrays (load-rlw1 path element-type) (cdr arrays)))
        ((null keys))
      (setf (gethash (car keys) params) (car arrays)))
    (layer-set-params (scn-conv1 net) (gethash "W1" params)
                      (gethash "b1" params))
    (layer-set-params (scn-affine1 net) (gethash "W2" params)
                      (gethash "b2" params))
    (layer-set-params (scn-affine2 net) (gethash "W3" params)
                      (gethash "b3" params))
    net))


---

# FILE: references/examples/deep-learning-from-scratch/ch07/train-convnet.lisp

;; ch07/train_convnet.py -- training the SimpleConvNet on MNIST (Deep
;; Learning from Scratch).
;;
;; Conv -> Relu -> Pool -> Affine -> Relu -> Affine with Adam, through the
;; common trainer. Scaled down hard from the book (100 train images
;; instead of 5000, batch 10, one epoch, accuracy evaluated on the full
;; small subsets): a CNN forward is ~100x an MLP's, so the plain
;; interpreter takes a few minutes here -- add --simd for the same output
;; much faster (im2col turns the convolution into linalg:matmul, which is
;; --simd-intercepted), or compile to the JVM. Raise the knobs toward the
;; book's 5000/batch 100/20 epochs accordingly.
;;
;; Data: run ../download-mnist.sh once, then from examples/deep-learning-from-scratch/:
;;   rontolisp ch07/train-convnet.lisp

(load "simple-convnet.lisp")
(load "../common/trainer.lisp")
(load "../dataset/mnist.lisp")

(defparameter *train-limit* 100)
(defparameter *test-limit* 50)
(defparameter *batch-size* 10)
(defparameter *epochs* 1)

(linalg:seed 42)

(let* ((x-train
        (linalg:reshape
         (mnist-load-images "dataset/train-images-idx3-ubyte" *train-limit*)
         (list *train-limit* 1 28 28)))
       (t-train
        (mnist-load-labels "dataset/train-labels-idx1-ubyte" *train-limit* 0 t))
       (x-test
        (linalg:reshape
         (mnist-load-images "dataset/t10k-images-idx3-ubyte" *test-limit*)
         (list *test-limit* 1 28 28)))
       (t-test
        (mnist-load-labels "dataset/t10k-labels-idx1-ubyte" *test-limit* 0 t))
       (net
        (make-simple-convnet :input-dim '(1 28 28)
                             :filter-num 30
                             :filter-size 5
                             :filter-pad 0
                             :filter-stride 1
                             :hidden-size 100
                             :output-size 10
                             :weight-init-std 0.01)))
  (train net (scn-params net) x-train t-train x-test t-test
         :epochs *epochs*
         :mini-batch-size *batch-size*
         :optimizer (make-instance 'adam :lr 0.001)))


---

# FILE: references/examples/deep-learning-from-scratch/ch07/visualize-filter.lisp

;; ch07/visualize_filter.py -- the SimpleConvNet's first-layer filters
;; before and after learning (Deep Learning from Scratch).
;;
;; The book imshows the 30 5x5 filters of W1 on a random-initialized net,
;; then again after load_params("params.pkl"). Here each filter renders as
;; ASCII art instead (the ch03 mnist-show ramp), min-max normalized per
;; filter like imshow, 8 filters per row like the book's grid; the trained
;; weights come from ch07/params.bin (params.pkl re-exported through
;; tools/export-sample-weight.py). The random init is seeded, so output is
;; byte-identical on every backend.
;;
;; Data: ch07/params.bin is committed. From examples/deep-learning-from-scratch/:
;;   rontolisp ch07/visualize-filter.lisp
;;   rontolisp ch07/visualize-filter.lisp -o Prog.class && java -cp .: Prog
;;   rontolisp ch07/visualize-filter.lisp -o prog.wasm --optimize && wasmtime run -W gc --dir . prog.wasm
;;   rontolisp ch07/visualize-filter.lisp -o comp.wasm --component && \
;;     wasmtime run -W gc=y --dir . comp.wasm

(load "simple-convnet.lisp")

(defparameter *ramp* " .:-=+*#%@")

(defun %filter-min (w i)
  ;; Smallest weight of filter i's first channel.
  (let* ((d (linalg:shape w)) (mn (aref w i 0 0 0)))
    (dotimes (y (nth 2 d))
      (dotimes (x (nth 3 d))
        (when (< (aref w i 0 y x) mn) (setq mn (aref w i 0 y x)))))
    mn))

(defun %filter-max (w i)
  ;; Largest weight of filter i's first channel.
  (let* ((d (linalg:shape w)) (mx (aref w i 0 0 0)))
    (dotimes (y (nth 2 d))
      (dotimes (x (nth 3 d))
        (when (> (aref w i 0 y x) mx) (setq mx (aref w i 0 y x)))))
    mx))

(defun render-filters (w)
  ;; W (FN C FH FW) as an ASCII grid, 8 filters per row: each filter's
  ;; first-channel plane min-max normalized to the 10-step intensity ramp
  ;; (the book's per-image imshow scaling; denser character = larger
  ;; weight, so the ramp is the book's gray_r inverted).
  (let* ((d (linalg:shape w)) (fn (car d)) (fh (nth 2 d)) (fw (nth 3 d)))
    (do ((base 0 (+ base 8)))
        ((>= base fn))
      (let ((n (min 8 (- fn base))))
        (dotimes (y fh)
          (let ((line ""))
            (dotimes (k n)
              (let* ((i (+ base k))
                     (mn (%filter-min w i))
                     (mx (%filter-max w i))
                     (span (- mx mn)))
                (dotimes (x fw)
                  (let* ((v (aref w i 0 y x))
                         (r
                          (if (= span 0.0)
                              0
                              (truncate (* (/ (- v mn) span) 10))))
                         (idx (min 9 r)))
                    (setq line
                     (concatenate 'string line (subseq *ramp* idx (+ idx 1))))))
                (setq line (concatenate 'string line " "))))
            (write-line line)))
        (terpri)))))

(linalg:seed 42)
(let ((net (make-simple-convnet)))
  (write-line "W1 before learning (seeded random init):")
  (terpri)
  (render-filters (gethash "W1" (scn-params net)))
  (net-load-params net "ch07/params.bin" nil)
  (write-line "W1 after learning (ch07/params.bin):")
  (terpri)
  (render-filters (gethash "W1" (scn-params net))))


---

# FILE: references/examples/deep-learning-from-scratch/ch08/deep-convnet.lisp

;; ch08/deep_convnet.py -- the deep CNN (Deep Learning from Scratch).
;; A library file loaded by train-deepnet.lisp.
;;
;; The 16-16 / 32-32 / 64-64 convolution pyramid: (Conv Relu Conv Relu
;; Pool) x 3 -> Affine Relu Dropout -> Affine Dropout -> SoftmaxWithLoss,
;; with He-scaled initialization over each layer's incoming-connection
;; count (the book's pre_node_nums). Geometry is fixed to 28x28 inputs
;; like the book: pad 1 keeps each conv shape-preserving except conv4's
;; pad 2 (14 -> 16), so the three 2x2 pools land on 64 channels of 4x4.
;; The eight parameter layers (six conv + two affine) share their arrays
;; with the params hash under "W1".."W8"/"b1".."b8"; the layer-dw/db
;; generics fold their cached gradients back per key.

(load "../common/layers.lisp")
(require "rlw1" "../dataset/rlw1.lisp")

(defclass deep-convnet ()
  ((params :initarg :params :accessor dcn-params)
   (layers :initarg :layers :accessor dcn-layers)
   (grad-layers :initarg :grad-layers :accessor dcn-grad-layers)
   (last-layer :initarg :last-layer :accessor dcn-last-layer)))

(defparameter *dcn-keys*
  '("W1" "b1" "W2" "b2" "W3" "b3" "W4" "b4" "W5" "b5" "W6" "b6" "W7" "b7" "W8"
    "b8"))

(defun make-deep-convnet (&key (hidden-size 50) (output-size 10))
  ;; conv-specs: (filter-num prev-channels pad) per conv layer; filter
  ;; size 3 and stride 1 throughout. pre-node-nums are the incoming
  ;; connections of W1..W8 (prev-channels * 3 * 3 for a conv, the input
  ;; width for an affine); He init scales each randn by sqrt(2/n).
  (let ((conv-specs
         '((16 1 1) (16 16 1) (32 16 1) (32 32 2) (64 32 1) (64 64 1)))
        (pre-node-nums '(9 144 144 288 288 576 1024 50))
        (pool-flat (* 64 4 4))
        (params (make-hash-table :test 'equal)))
    (do ((idx 1 (+ idx 1))
         (specs conv-specs (cdr specs))
         (fans pre-node-nums (cdr fans)))
        ((null specs))
      (let* ((spec (car specs)) (fn (car spec)) (prev (car (cdr spec))))
        (setf (gethash (format nil "W~a" idx) params)
              (linalg:mul (sqrt (/ 2.0 (car fans)))
                          (linalg:randn (list fn prev 3 3))))
        (setf (gethash (format nil "b~a" idx) params) (linalg:zeros fn))))
    (setf (gethash "W7" params)
          (linalg:mul (sqrt (/ 2.0 1024))
                      (linalg:randn (list pool-flat hidden-size))))
    (setf (gethash "b7" params) (linalg:zeros hidden-size))
    (setf (gethash "W8" params)
          (linalg:mul (sqrt (/ 2.0 50))
                      (linalg:randn (list hidden-size output-size))))
    (setf (gethash "b8" params) (linalg:zeros output-size))
    (let ((convs nil))
      (do ((idx 1 (+ idx 1)) (specs conv-specs (cdr specs)))
          ((null specs))
        (setq convs
              (cons (make-instance 'convolution
                                   :w (gethash (format nil "W~a" idx) params)
                                   :b (gethash (format nil "b~a" idx) params)
                                   :stride 1
                                   :pad (nth 2 (car specs))) convs)))
      (let* ((convs (reverse convs))
             (affine1
              (make-instance 'affine
                             :w (gethash "W7" params)
                             :b (gethash "b7" params)))
             (affine2
              (make-instance 'affine
                             :w (gethash "W8" params)
                             :b (gethash "b8" params))))
        (make-instance 'deep-convnet
                       :params params
                       :layers (list (nth 0 convs) (make-instance 'relu-layer)
                                     (nth 1 convs) (make-instance 'relu-layer)
                                     (make-instance 'pooling
                                                    :pool-h 2
                                                    :pool-w 2
                                                    :stride 2) (nth 2 convs)
                                     (make-instance 'relu-layer) (nth 3 convs)
                                     (make-instance 'relu-layer)
                                     (make-instance 'pooling
                                                    :pool-h 2
                                                    :pool-w 2
                                                    :stride 2) (nth 4 convs)
                                     (make-instance 'relu-layer) (nth 5 convs)
                                     (make-instance 'relu-layer)
                                     (make-instance 'pooling
                                                    :pool-h 2
                                                    :pool-w 2
                                                    :stride 2) affine1
                                     (make-instance 'relu-layer)
                                     (make-instance 'dropout :ratio 0.5) affine2
                                     (make-instance 'dropout :ratio 0.5))
                       :grad-layers (append convs (list affine1 affine2))
                       :last-layer (make-instance 'softmax-with-loss))))))

;; The per-parameter-layer gradient accessors: conv and affine cache their
;; dW/db in differently named slots, so one generic pair folds both.

(defgeneric layer-dw (layer))

(defgeneric layer-db (layer))

(defmethod layer-dw ((layer convolution)) (conv-dw layer))

(defmethod layer-dw ((layer affine)) (affine-dw layer))

(defmethod layer-db ((layer convolution)) (conv-db layer))

(defmethod layer-db ((layer affine)) (affine-db layer))

(defgeneric predict (net x))

(defgeneric net-loss (net x target))

(defgeneric net-gradient (net x target))

(defgeneric net-accuracy-count (net x target))

(defmethod predict ((net deep-convnet) x)
  (let ((out x))
    (dolist (layer (dcn-layers net)) (setq out (forward layer out)))
    out))

(defmethod net-loss ((net deep-convnet) x target)
  (loss-forward (dcn-last-layer net) (predict net x) target))

(defmethod net-accuracy-count ((net deep-convnet) x target)
  ;; Correctly classified batch rows, with dropout switched to its
  ;; evaluation behavior; target may be one-hot or labels.
  (let* ((y (let ((*train-p* nil)) (predict net x)))
         (yl (linalg:argmax y :axis 1))
         (tl
          (if (= (linalg:ndim target) 1)
              target
              (linalg:argmax target :axis 1))))
    (truncate (linalg:sum (linalg:equal yl tl)))))

(defmethod net-gradient ((net deep-convnet) x target)
  ;; forward for the caches, then fold backward over the reversed layers;
  ;; each parameter layer leaves dW/db behind, collected as W1..W8.
  (net-loss net x target)
  (let ((dout (loss-backward (dcn-last-layer net))))
    (dolist (layer (reverse (dcn-layers net)))
      (setq dout (backward layer dout))))
  (let ((grads (make-hash-table :test 'equal)))
    (do ((idx 1 (+ idx 1)) (layers (dcn-grad-layers net) (cdr layers)))
        ((null layers) grads)
      (setf (gethash (format nil "W~a" idx) grads) (layer-dw (car layers)))
      (setf (gethash (format nil "b~a" idx) grads) (layer-db (car layers))))))

(defgeneric net-load-params (net path element-type))

(defmethod net-load-params ((net deep-convnet) path element-type)
  ;; The book's load_params over an RLW1 export of deep_convnet_params.pkl
  ;; (W1 b1 .. W8 b8): replace the hash entries, then re-point the eight
  ;; parameter layers' shared arrays. element-type nil = double,
  ;; 'single-float = packed #f (linalg is width-polymorphic, so the whole
  ;; net then runs single -- the half-float chapter's cast).
  (let ((params (dcn-params net)))
    (do ((keys *dcn-keys* (cdr keys))
         (arrays (load-rlw1 path element-type) (cdr arrays)))
        ((null keys))
      (setf (gethash (car keys) params) (car arrays)))
    (do ((idx 1 (+ idx 1)) (layers (dcn-grad-layers net) (cdr layers)))
        ((null layers))
      (layer-set-params (car layers) (gethash (format nil "W~a" idx) params)
                        (gethash (format nil "b~a" idx) params)))
    net))


---

# FILE: references/examples/deep-learning-from-scratch/ch08/half-float-network.lisp

;; ch08/half_float_network.py -- the trained deep CNN at reduced precision
;; (Deep Learning from Scratch).
;;
;; The book casts the network parameters and the test images from float64
;; to float16 and shows the test accuracy is unchanged. rontolisp's packed
;; reduced width is single-float (#f) -- literally half a double's 64
;; bits -- so the "half" cast here reloads the pretrained weights as
;; 'single-float through the width-polymorphic RLW1 loader (an f32 value
;; is exact in both widths) and copies the images into a packed #f array;
;; linalg preserves the #f width through every layer, so the whole forward
;; pass then runs single. Scaled down from the book's 10000 test images to
;; 1000. The two deep-convnet passes want acceleration: ~40 s on WASM
;; --simd, ~3 minutes under interpreter --simd or compiled to the JVM;
;; the plain interpreter runs the linalg defuns as scalar loops and takes
;; hours -- shrink *sampled* first. Output is byte-identical on every
;; backend (inference is exact IEEE arithmetic).
;;
;; Data: run ../download-mnist.sh once, then from examples/deep-learning-from-scratch/:
;;   rontolisp ch08/half-float-network.lisp --simd
;;   rontolisp ch08/half-float-network.lisp -o Prog.class && java -cp .: Prog
;;   rontolisp ch08/half-float-network.lisp -o prog.wasm --simd --optimize && \
;;     wasmtime run -W gc --dir . prog.wasm
;;   rontolisp ch08/half-float-network.lisp -o comp.wasm --component --simd && \
;;     wasmtime run -W gc=y --dir . comp.wasm

(load "deep-convnet.lisp")
(load "../dataset/mnist.lisp")

(defparameter *sampled* 1000) ; the book samples 10000
(defparameter *batch-size* 100)

(defun astype-single (a)
  ;; numpy's x.astype(np.float16) analog: copy into a packed #f array (the
  ;; store narrows each double to single precision).
  (let ((out
         (make-array (linalg:shape a)
                     :element-type 'single-float
                     :initial-element 0.0)))
    (dotimes (i (array-total-size a))
      (setf (row-major-aref out i) (row-major-aref a i)))
    out))

(defun batched-accuracy (net x target)
  ;; net-accuracy-count over *batch-size* slices (one full-sample forward
  ;; would materialize ~900 MB of im2col unfolds).
  (let ((acc 0))
    (do ((start 0 (+ start *batch-size*)))
        ((>= start *sampled*) acc)
      (let ((idx (linalg:arange start (+ start *batch-size*))))
        (setq acc
              (+ acc
                 (net-accuracy-count net (linalg:take-rows x idx)
                                     (linalg:take-rows target idx))))))))

(let* ((x-test
        (linalg:reshape
         (mnist-load-images "dataset/t10k-images-idx3-ubyte" *sampled*)
         (list *sampled* 1 28 28)))
       (t-test (mnist-load-labels "dataset/t10k-labels-idx1-ubyte" *sampled*))
       (net (make-deep-convnet :hidden-size 50 :output-size 10)))
  (net-load-params net "ch08/deep-convnet-params.bin" nil)
  (write-line "calculate accuracy (double-float) ...")
  (format t "~a/~a~%" (batched-accuracy net x-test t-test) *sampled*)
  ;; the float16 cast: reload the params as packed #f, cast the images
  (net-load-params net "ch08/deep-convnet-params.bin" 'single-float)
  (let ((x-single (astype-single x-test)))
    (write-line "calculate accuracy (single-float) ...")
    (format t "~a/~a~%" (batched-accuracy net x-single t-test) *sampled*)))


---

# FILE: references/examples/deep-learning-from-scratch/ch08/misclassified-mnist.lisp

;; ch08/misclassified_mnist.py -- which test digits the trained deep CNN
;; still gets wrong (Deep Learning from Scratch).
;;
;; The pretrained weights (ch08/deep-convnet-params.bin, the book's
;; deep_convnet_params.pkl re-exported through tools/export-sample-weight.py)
;; load into the deep-convnet, test accuracy is evaluated in batches, and
;; each misclassified digit prints with its (label, inference) pair -- as
;; ASCII art (the ch03 mnist-show ramp) instead of the book's plot grid.
;; Scaled down from the book's full 10000-image test set to its own
;; commented-out suggestion of 1000 (the trained net misses exactly 20 of
;; those -- the numpy original misses the same 20). A deep-convnet forward
;; this size wants acceleration: ~19 s on WASM --simd, ~1.5 minutes under
;; interpreter --simd, ~2 minutes compiled to the JVM; the plain
;; interpreter runs the linalg defuns as scalar loops and takes hours --
;; shrink *sampled* first. Output is byte-identical on every backend
;; (inference is exact IEEE arithmetic; no exp/log).
;;
;; Data: run ../download-mnist.sh once, then from examples/deep-learning-from-scratch/:
;;   rontolisp ch08/misclassified-mnist.lisp --simd
;;   rontolisp ch08/misclassified-mnist.lisp -o Prog.class && java -cp .: Prog
;;   rontolisp ch08/misclassified-mnist.lisp -o prog.wasm --simd --optimize && \
;;     wasmtime run -W gc --dir . prog.wasm
;;   rontolisp ch08/misclassified-mnist.lisp -o comp.wasm --component --simd && \
;;     wasmtime run -W gc=y --dir . comp.wasm

(load "deep-convnet.lisp")
(load "../dataset/mnist.lisp")

(defparameter *sampled* 1000) ; the book evaluates all 10000
(defparameter *batch-size* 100)
(defparameter *max-view* 20)

(defparameter *ramp* " .:-=+*#%@")

(defun render-digit (x i)
  ;; Row i of the rank-4 (n 1 28 28) test batch as ASCII art: each pixel
  ;; in [0,1] indexes the 10-step intensity ramp (ch03 mnist-show).
  (dotimes (y 28)
    (let ((line ""))
      (dotimes (col 28)
        (let* ((v (aref x i 0 y col)) (idx (min 9 (truncate (* v 10)))))
          (setq line (concatenate 'string line (subseq *ramp* idx (+ idx 1))))))
      (write-line line))))

(let* ((x-test
        (linalg:reshape
         (mnist-load-images "dataset/t10k-images-idx3-ubyte" *sampled*)
         (list *sampled* 1 28 28)))
       (t-test (mnist-load-labels "dataset/t10k-labels-idx1-ubyte" *sampled*))
       (net (make-deep-convnet :hidden-size 50 :output-size 10)))
  (net-load-params net "ch08/deep-convnet-params.bin" nil)
  (write-line "calculating test accuracy ...")
  (let ((classified
         (make-array *sampled*
                     :element-type 'double-float
                     :initial-element 0.0))
        (acc 0))
    (do ((start 0 (+ start *batch-size*)))
        ((>= start *sampled*))
      (let* ((idx (linalg:arange start (+ start *batch-size*)))
             (tx (linalg:take-rows x-test idx))
             (y (let ((*train-p* nil)) (predict net tx)))
             (yl (linalg:argmax y :axis 1)))
        (dotimes (k *batch-size*)
          (setf (aref classified (+ start k)) (aref yl k))
          (when (= (aref yl k) (aref t-test (+ start k)))
            (setq acc (+ acc 1))))))
    (format t "test accuracy: ~a/~a~%" acc *sampled*)
    (write-line "======= misclassified result =======")
    (let ((view 1))
      (dotimes (i *sampled*)
        (when (and (<= view *max-view*)
                   (/= (aref classified i) (aref t-test i)))
          (format t "view ~a: label ~a, inference ~a~%" view
                  (truncate (aref t-test i)) (truncate (aref classified i)))
          (render-digit x-test i)
          (setq view (+ view 1)))))))


---

# FILE: references/examples/deep-learning-from-scratch/ch08/train-deepnet.lisp

;; ch08/train_deepnet.py -- training the deep CNN on MNIST (Deep Learning
;; from Scratch).
;;
;; The 8-parameter-layer pyramid of deep-convnet.lisp with Adam, through
;; the common trainer. The book reaches ~99.4% test accuracy over 20
;; epochs on the full 60000 images -- hours even in numpy; this port's
;; defaults are a smoke-scale run (30 train images, batch 5, one epoch)
;; that exercises every layer's forward/backward: ~12 minutes on the
;; plain interpreter, a minute and a half under --simd, seconds on the
;; JVM. Raise the knobs toward the book's settings if you have the
;; patience.
;;
;; Data: run ../download-mnist.sh once, then from examples/deep-learning-from-scratch/:
;;   rontolisp ch08/train-deepnet.lisp

(load "deep-convnet.lisp")
(load "../common/trainer.lisp")
(load "../dataset/mnist.lisp")

(defparameter *train-limit* 30)
(defparameter *test-limit* 10)
(defparameter *batch-size* 5)
(defparameter *epochs* 1)

(linalg:seed 42)

(let* ((x-train
        (linalg:reshape
         (mnist-load-images "dataset/train-images-idx3-ubyte" *train-limit*)
         (list *train-limit* 1 28 28)))
       (t-train
        (mnist-load-labels "dataset/train-labels-idx1-ubyte" *train-limit* 0 t))
       (x-test
        (linalg:reshape
         (mnist-load-images "dataset/t10k-images-idx3-ubyte" *test-limit*)
         (list *test-limit* 1 28 28)))
       (t-test
        (mnist-load-labels "dataset/t10k-labels-idx1-ubyte" *test-limit* 0 t))
       (net (make-deep-convnet :hidden-size 50 :output-size 10)))
  (train net (dcn-params net) x-train t-train x-test t-test
         :epochs *epochs*
         :mini-batch-size *batch-size*
         :optimizer (make-instance 'adam :lr 0.001)))


---

# FILE: references/examples/deep-learning-from-scratch/common/functions.lisp

;; common/functions.py -- activation and loss functions (Deep Learning from
;; Scratch). Everything is batch-oriented over linalg arrays: x is a vector
;; or an (N x D) matrix, and the axis reductions / broadcasting follow the
;; book's numpy code line for line.

(defun identity-function (x) x)

(defun step-function (x)
  ;; np.array(x > 0, dtype=int) -- a 0.0/1.0 mask here.
  (linalg:greater x 0))

(defun sigmoid (x)
  ;; 1 / (1 + exp(-x))
  (linalg:div 1 (linalg:add 1 (linalg:exp (linalg:negative x)))))

(defun sigmoid-grad (x)
  ;; (1 - sigmoid(x)) * sigmoid(x)
  (let ((s (sigmoid x))) (linalg:mul (linalg:sub 1 s) s)))

(defun relu (x)
  ;; np.maximum(0, x)
  (linalg:relu x))

(defun relu-grad (x)
  ;; grad[x >= 0] = 1 -- the boolean mask IS the gradient.
  (linalg:greater-equal x 0))

(defun softmax (x)
  ;; exp(x - rowmax) / rowsum, stabilized along the last axis exactly like
  ;; the book: x - np.max(x, axis=-1, keepdims=True), then the keepdims sum
  ;; broadcasts back over each row. Works for a vector and an (N x D) batch.
  (let* ((shifted (linalg:sub x (linalg:amax x :axis -1 :keepdims t)))
         (e (linalg:exp shifted)))
    (linalg:div e (linalg:sum e :axis -1 :keepdims t))))

(defun sum-squared-error (y target)
  ;; 0.5 * sum((y - t)^2)
  (* 0.5 (linalg:sum (linalg:square (linalg:sub y target)))))

(defun cross-entropy-error (y target)
  ;; The book's four-case loss: y and target may be a single sample (vector)
  ;; or a batch (matrix), and target may be one-hot or a plain label vector.
  ;; One-hot targets are collapsed to labels with argmax(axis=1), then the
  ;; per-row correct-class probabilities are picked out with gather --
  ;; y[np.arange(batch_size), t] in the book.
  (let* ((yb
          (if (= (linalg:ndim y) 1)
              (linalg:reshape y (list 1 (linalg:size y)))
              y))
         (tb
          (if (= (linalg:ndim target) 1)
              (if (= (linalg:size target) (linalg:size yb))
                  (linalg:reshape target (list 1 (linalg:size target)))
                  target)
              target))
         (tl
          (if (= (linalg:size tb) (linalg:size yb))
              (linalg:argmax tb :axis 1)
              (linalg:flatten tb)))
         (batch (car (linalg:shape yb))))
    (- 0
       (/ (linalg:sum (linalg:log (linalg:add (linalg:gather yb tl) 1.0e-7)))
          batch))))

(defun softmax-loss (x target) (cross-entropy-error (softmax x) target))


---

# FILE: references/examples/deep-learning-from-scratch/common/gradient.lisp

;; common/gradient.py -- numerical gradients (Deep Learning from Scratch).
;;
;; The book's numerical_gradient walks every element of x with np.nditer,
;; nudging it +/- h IN PLACE (f re-reads the same array) and restoring it;
;; the row-major-aref walk below is the same thing for any rank. Slow by
;; design -- the book uses it to verify backpropagation, not to train.

(defun numerical-gradient (f x)
  ;; grad[k] = (f(x + h*e_k) - f(x - h*e_k)) / 2h, central differences with
  ;; h = 1e-4. x is temporarily mutated and restored; f takes x itself.
  (let ((h 1.0e-4) (grad (linalg:zeros-like x)))
    (dotimes (k (linalg:size x))
      (let ((tmp (row-major-aref x k)))
        (setf (row-major-aref x k) (+ tmp h))
        (let ((fxh1 (funcall f x)))
          (setf (row-major-aref x k) (- tmp h))
          (let ((fxh2 (funcall f x)))
            (setf (row-major-aref grad k) (/ (- fxh1 fxh2) (* 2 h)))
            (setf (row-major-aref x k) tmp)))))
    grad))


---

# FILE: references/examples/deep-learning-from-scratch/common/layers.lisp

;; common/layers.py -- the class-based layers (Deep Learning from Scratch),
;; on rontolisp's CLOS subset: one class per layer, a forward/backward
;; generic-function pair, and per-instance state (cached inputs, masks,
;; dW/db) in slots. Two adaptations to the subset's fixed-arity generics:
;;
;; - the book's train_flg parameter becomes the special variable *train-p*
;;   (bind (let ((*train-p* nil)) ...) around evaluation);
;; - SoftmaxWithLoss's two-argument forward gets its own generic pair,
;;   loss-forward / loss-backward.
;;
;; Accessor names are class-prefixed because CLOS accessors are plain
;; defuns here (globally unique names required).

(load "functions.lisp")
(load "util.lisp")

(defvar *train-p* t)

(defgeneric forward (layer x))

(defgeneric backward (layer dout))

;; --- Relu ----------------------------------------------------------------------

(defclass relu-layer () ((mask :initform nil :accessor relu-mask)))

(defmethod forward ((layer relu-layer) x)
  ;; The book keeps mask = (x <= 0) and zeroes through it; the equivalent
  ;; kept-elements mask (x > 0) is stored instead and multiplied in.
  (let ((keep (linalg:greater x 0)))
    (setf (relu-mask layer) keep)
    (linalg:mul x keep)))

(defmethod backward ((layer relu-layer) dout)
  (linalg:mul dout (relu-mask layer)))

;; --- Sigmoid -------------------------------------------------------------------

(defclass sigmoid-layer () ((out :initform nil :accessor sigmoid-out)))

(defmethod forward ((layer sigmoid-layer) x)
  (let ((out (sigmoid x)))
    (setf (sigmoid-out layer) out)
    out))

(defmethod backward ((layer sigmoid-layer) dout)
  (let ((out (sigmoid-out layer)))
    (linalg:mul dout (linalg:mul (linalg:sub 1 out) out))))

;; --- Affine --------------------------------------------------------------------

(defclass affine ()
  ((w :initarg :w :accessor affine-w) ; SHARED with params["W"]
   (b :initarg :b :accessor affine-b) (x :initform nil :accessor affine-x)
   (original-shape :initform nil :accessor affine-original-shape)
   (dw :initform nil :accessor affine-dw)
   (db :initform nil :accessor affine-db)))

(defmethod forward ((layer affine) x)
  ;; Tensor support like the book: a rank > 2 input is flattened to
  ;; (N, -1) for the product and restored in backward.
  (let* ((shape (linalg:shape x))
         (x2 (if (cdr (cdr shape)) (linalg:reshape x (list (car shape) -1)) x)))
    (setf (affine-original-shape layer) shape)
    (setf (affine-x layer) x2)
    (linalg:add (linalg:matmul x2 (affine-w layer)) (affine-b layer))))

(defmethod backward ((layer affine) dout)
  (let ((dx (linalg:matmul dout (linalg:transpose (affine-w layer)))))
    (setf (affine-dw layer)
          (linalg:matmul (linalg:transpose (affine-x layer)) dout))
    (setf (affine-db layer) (linalg:sum dout :axis 0))
    (linalg:reshape dx (affine-original-shape layer))))

;; --- SoftmaxWithLoss -----------------------------------------------------------

(defclass softmax-with-loss ()
  ((loss :initform nil :accessor swl-loss) (y :initform nil :accessor swl-y)
   (target :initform nil :accessor swl-target)))

(defgeneric loss-forward (layer x target))

(defgeneric loss-backward (layer))

(defmethod loss-forward ((layer softmax-with-loss) x target)
  (setf (swl-target layer) target)
  (setf (swl-y layer) (softmax x))
  (setf (swl-loss layer) (cross-entropy-error (swl-y layer) target))
  (swl-loss layer))

(defmethod loss-backward ((layer softmax-with-loss))
  ;; dx = (y - one-hot(t)) / batch. The book scatters -1 into a copy of y
  ;; when t is a label vector (dx[np.arange(batch), t] -= 1); subtracting
  ;; the one-hot matrix is the same arithmetic without mutation.
  (let* ((y (swl-y layer))
         (target (swl-target layer))
         (batch (car (linalg:shape y)))
         (hot
          (if (= (linalg:size target) (linalg:size y))
              target
              (linalg:one-hot target (car (cdr (linalg:shape y)))))))
    (linalg:div (linalg:sub y hot) batch)))

;; --- Dropout -------------------------------------------------------------------

(defclass dropout ()
  ((ratio :initarg :ratio :initform 0.5 :accessor dropout-ratio)
   (mask :initform nil :accessor dropout-mask)))

(defmethod forward ((layer dropout) x)
  (if *train-p*
      (let ((mask
             (linalg:greater (linalg:rand (linalg:shape x))
                             (dropout-ratio layer))))
        (setf (dropout-mask layer) mask)
        (linalg:mul x mask))
      (linalg:mul x (- 1.0 (dropout-ratio layer)))))

(defmethod backward ((layer dropout) dout)
  (linalg:mul dout (dropout-mask layer)))

;; --- Convolution (ch07) ----------------------------------------------------------

(defclass convolution ()
  ((w :initarg :w :accessor conv-w) ; (FN C FH FW), SHARED with params
   (b :initarg :b :accessor conv-b) ; (FN)
   (stride :initarg :stride :initform 1 :accessor conv-stride)
   (pad :initarg :pad :initform 0 :accessor conv-pad)
   (x :initform nil :accessor conv-x) (col :initform nil :accessor conv-col)
   (col-w :initform nil :accessor conv-col-w)
   (dw :initform nil :accessor conv-dw) (db :initform nil :accessor conv-db)))

(defmethod forward ((layer convolution) x)
  ;; im2col turns the convolution into one (N*oh*ow, C*FH*FW) x
  ;; (C*FH*FW, FN) matrix product; the result folds back to NCHW through
  ;; reshape + transpose like the book.
  (let* ((wd (linalg:shape (conv-w layer)))
         (fn (car wd))
         (fh (nth 2 wd))
         (fw (nth 3 wd))
         (xd (linalg:shape x))
         (n (car xd))
         (h (nth 2 xd))
         (w (nth 3 xd))
         (stride (conv-stride layer))
         (pad (conv-pad layer))
         (out-h (+ 1 (floor (- (+ h (* 2 pad)) fh) stride)))
         (out-w (+ 1 (floor (- (+ w (* 2 pad)) fw) stride)))
         (col (im2col x fh fw stride pad))
         (col-w (linalg:transpose (linalg:reshape (conv-w layer) (list fn -1))))
         (out (linalg:add (linalg:matmul col col-w) (conv-b layer))))
    (setf (conv-x layer) x)
    (setf (conv-col layer) col)
    (setf (conv-col-w layer) col-w)
    (linalg:transpose (linalg:reshape out (list n out-h out-w -1)) '(0 3 1 2))))

(defmethod backward ((layer convolution) dout)
  ;; dout (N FN oh ow) -> (N*oh*ow, FN); dW/db fall out of the matrix
  ;; product's transposes, and dx scatters back through col2im.
  (let* ((wd (linalg:shape (conv-w layer)))
         (fn (car wd))
         (c (nth 1 wd))
         (fh (nth 2 wd))
         (fw (nth 3 wd))
         (dout2
          (linalg:reshape (linalg:transpose dout '(0 2 3 1)) (list -1 fn))))
    (setf (conv-db layer) (linalg:sum dout2 :axis 0))
    (setf (conv-dw layer)
          (linalg:reshape (linalg:transpose
                           (linalg:matmul (linalg:transpose (conv-col layer))
                                          dout2)) (list fn c fh fw)))
    (col2im (linalg:matmul dout2 (linalg:transpose (conv-col-w layer)))
            (linalg:shape (conv-x layer)) fh fw (conv-stride layer)
            (conv-pad layer))))

;; --- Pooling (ch07) --------------------------------------------------------------

(defclass pooling ()
  ((pool-h :initarg :pool-h :accessor pool-h)
   (pool-w :initarg :pool-w :accessor pool-w)
   (stride :initarg :stride :initform 1 :accessor pool-stride)
   (pad :initarg :pad :initform 0 :accessor pool-pad)
   (x :initform nil :accessor pool-x)
   (arg-max :initform nil :accessor pool-arg-max)))

(defmethod forward ((layer pooling) x)
  ;; Max pooling over the im2col unfold: reshaping the (N*oh*ow, C*ph*pw)
  ;; matrix to (-1, ph*pw) makes each row one window of one channel, so
  ;; the max and its argmax are per-row axis-1 reductions (the book).
  (let* ((xd (linalg:shape x))
         (n (car xd))
         (c (nth 1 xd))
         (h (nth 2 xd))
         (w (nth 3 xd))
         (ph (pool-h layer))
         (pw (pool-w layer))
         (stride (pool-stride layer))
         (out-h (+ 1 (floor (- h ph) stride)))
         (out-w (+ 1 (floor (- w pw) stride)))
         (col
          (linalg:reshape (im2col x ph pw stride (pool-pad layer))
                          (list -1 (* ph pw))))
         (am (linalg:argmax col :axis 1))
         (out (linalg:amax col :axis 1)))
    (setf (pool-x layer) x)
    (setf (pool-arg-max layer) am)
    (linalg:transpose (linalg:reshape out (list n out-h out-w c)) '(0 3 1 2))))

(defmethod backward ((layer pooling) dout)
  ;; The book scatters dout into a zero (size, pool-size) matrix at each
  ;; window's cached argmax (dmax[np.arange(size), argmax] = dout);
  ;; multiplying the argmax one-hot matrix by the dout column is the same
  ;; scatter without mutation (the loss-backward pattern).
  (let* ((dd (linalg:shape dout))
         (n (car dd))
         (oh (nth 2 dd))
         (ow (nth 3 dd))
         (pool-size (* (pool-h layer) (pool-w layer)))
         (flat (linalg:reshape (linalg:transpose dout '(0 2 3 1)) (list -1 1)))
         (dmax
          (linalg:mul (linalg:one-hot (pool-arg-max layer) pool-size) flat))
         (dcol (linalg:reshape dmax (list (* n oh ow) -1))))
    (col2im dcol (linalg:shape (pool-x layer)) (pool-h layer) (pool-w layer)
            (pool-stride layer) (pool-pad layer))))

;; --- BatchNormalization ---------------------------------------------------------

(defclass batch-normalization ()
  ((gamma :initarg :gamma :accessor bn-gamma) ; SHARED with params["gamma"]
   (beta :initarg :beta :accessor bn-beta)
   (momentum :initarg :momentum :initform 0.9 :accessor bn-momentum)
   (input-shape :initform nil :accessor bn-input-shape)
   (running-mean :initform nil :accessor bn-running-mean)
   (running-var :initform nil :accessor bn-running-var)
   (batch-size :initform nil :accessor bn-batch-size)
   (xc :initform nil :accessor bn-xc) (xn :initform nil :accessor bn-xn)
   (std :initform nil :accessor bn-std)
   (dgamma :initform nil :accessor bn-dgamma)
   (dbeta :initform nil :accessor bn-dbeta)))

(defmethod forward ((layer batch-normalization) x)
  (setf (bn-input-shape layer) (linalg:shape x))
  (let* ((x2
          (if (/= (linalg:ndim x) 2)
              (linalg:reshape x (list (car (linalg:shape x)) -1))
              x))
         (d (car (cdr (linalg:shape x2)))))
    (unless (bn-running-mean layer)
      (setf (bn-running-mean layer) (linalg:zeros d))
      (setf (bn-running-var layer) (linalg:zeros d)))
    (let ((out
           (if *train-p*
               (let* ((mu (linalg:mean x2 :axis 0))
                      (xc (linalg:sub x2 mu))
                      (var (linalg:mean (linalg:square xc) :axis 0))
                      (std (linalg:sqrt (linalg:add var 10.0e-7)))
                      (xn (linalg:div xc std))
                      (m (bn-momentum layer)))
                 (setf (bn-batch-size layer) (car (linalg:shape x2)))
                 (setf (bn-xc layer) xc)
                 (setf (bn-xn layer) xn)
                 (setf (bn-std layer) std)
                 (setf (bn-running-mean layer)
                       (linalg:add (linalg:mul m (bn-running-mean layer))
                                   (linalg:mul (- 1 m) mu)))
                 (setf (bn-running-var layer)
                       (linalg:add (linalg:mul m (bn-running-var layer))
                                   (linalg:mul (- 1 m) var)))
                 xn)
               (linalg:div (linalg:sub x2 (bn-running-mean layer))
                (linalg:sqrt (linalg:add (bn-running-var layer) 10.0e-7))))))
      (linalg:reshape
       (linalg:add (linalg:mul (bn-gamma layer) out) (bn-beta layer))
       (bn-input-shape layer)))))

(defmethod backward ((layer batch-normalization) dout)
  (let* ((dout2
          (if (/= (linalg:ndim dout) 2)
              (linalg:reshape dout (list (car (linalg:shape dout)) -1))
              dout))
         (xn (bn-xn layer))
         (xc (bn-xc layer))
         (std (bn-std layer))
         (batch (bn-batch-size layer))
         (dbeta (linalg:sum dout2 :axis 0))
         (dgamma (linalg:sum (linalg:mul xn dout2) :axis 0))
         (dxn (linalg:mul (bn-gamma layer) dout2))
         (dxc (linalg:div dxn std))
         (dstd
          (linalg:negative
           (linalg:sum (linalg:div (linalg:mul dxn xc) (linalg:mul std std))
                       :axis 0)))
         (dvar (linalg:div (linalg:mul 0.5 dstd) std))
         (dxc2 (linalg:add dxc (linalg:mul (/ 2.0 batch) (linalg:mul xc dvar))))
         (dmu (linalg:sum dxc2 :axis 0))
         (dx (linalg:sub dxc2 (linalg:div dmu batch))))
    (setf (bn-dgamma layer) dgamma)
    (setf (bn-dbeta layer) dbeta)
    (linalg:reshape dx (bn-input-shape layer))))

;; --- parameter re-pointing (ch07/ch08 net-load-params) --------------------------

;; The book's load_params reassigns layer.W/layer.b after replacing the
;; params dict; conv and affine name those slots differently, so one
;; generic folds both (the layer-dw/layer-db pattern of ch08).

(defgeneric layer-set-params (layer w b))

(defmethod layer-set-params ((layer convolution) w b)
  (setf (conv-w layer) w)
  (setf (conv-b layer) b))

(defmethod layer-set-params ((layer affine) w b)
  (setf (affine-w layer) w)
  (setf (affine-b layer) b))


---

# FILE: references/examples/deep-learning-from-scratch/common/multi-layer-net-extend.lisp

;; common/multi_layer_net_extend.py -- the multi-layer network with
;; Batch Normalization, Dropout and weight decay (Deep Learning from
;; Scratch). A library file for the ch06 scripts.
;;
;; The MultiLayerNet stack extended per hidden layer to
;; Affine -> [BatchNorm] -> activation -> [Dropout]; gamma/beta join the
;; params/grads hashes when BatchNorm is on. The book's train_flg
;; parameter is the special variable *train-p* (layers.lisp): net-gradient
;; binds it to t, net-accuracy-count to nil. Do not load this file and
;; multi-layer-net.lisp into the same program -- each pulls in layers.lisp
;; itself.

(load "layers.lisp")
(load "gradient.lisp")

(defclass multi-layer-net-extend ()
  ((params :initarg :params :accessor mlne-params)
   (layers :initarg :layers :accessor mlne-layers)
   (affines :initarg :affines :accessor mlne-affines)
   (batchnorms :initarg :batchnorms :accessor mlne-batchnorms) ; per hidden, or nil
   (keys :initarg :keys :accessor mlne-keys)
   (weight-decay-lambda :initarg :weight-decay-lambda
                        :accessor mlne-weight-decay-lambda)
   (use-batchnorm :initarg :use-batchnorm :accessor mlne-use-batchnorm)
   (last-layer :initarg :last-layer :accessor mlne-last-layer)))

(defun make-multi-layer-net-extend (input-size hidden-size-list output-size &key
                                               (activation 'relu)
                                               (weight-init-std 'relu)
                                               (weight-decay-lambda 0)
                                               use-dropout (dropout-ratio 0.5)
                                               use-batchnorm)
  (let* ((all-sizes
          (append (list input-size) hidden-size-list (list output-size)))
         (n-hidden (length hidden-size-list))
         (params (make-hash-table :test 'equal))
         (keys nil)
         (affines nil)
         (batchnorms nil)
         (layer-list nil))
    (do ((idx 1 (+ idx 1)) (sizes all-sizes (cdr sizes)))
        ((null (cdr sizes)))
      (let ((prev (car sizes))
            (next (cadr sizes))
            (wk (format nil "W~a" idx))
            (bk (format nil "b~a" idx)))
        (setf (gethash wk params)
              (linalg:mul (%init-scale weight-init-std prev)
                          (linalg:randn (list prev next))))
        (setf (gethash bk params) (linalg:zeros next))
        (setq keys (cons bk (cons wk keys)))))
    (do ((idx 1 (+ idx 1)))
        ((> idx (+ n-hidden 1)))
      (let ((aff
             (make-instance 'affine
                            :w (gethash (format nil "W~a" idx) params)
                            :b (gethash (format nil "b~a" idx) params))))
        (setq affines (cons aff affines))
        (setq layer-list (cons aff layer-list))
        (when (<= idx n-hidden)
          (when use-batchnorm
            (let ((gk (format nil "gamma~a" idx))
                  (bk2 (format nil "beta~a" idx))
                  (size (nth (- idx 1) hidden-size-list)))
              (setf (gethash gk params) (linalg:ones size))
              (setf (gethash bk2 params) (linalg:zeros size))
              (setq keys (append keys (list gk bk2)))
              (let ((bn
                     (make-instance 'batch-normalization
                                    :gamma (gethash gk params)
                                    :beta (gethash bk2 params))))
                (setq batchnorms (cons bn batchnorms))
                (setq layer-list (cons bn layer-list)))))
          (setq layer-list
                (cons (if (eq activation 'sigmoid)
                          (make-instance 'sigmoid-layer)
                          (make-instance 'relu-layer)) layer-list))
          (when use-dropout
            (setq layer-list
                  (cons (make-instance 'dropout :ratio dropout-ratio)
                        layer-list))))))
    (make-instance 'multi-layer-net-extend
                   :params params
                   :layers (reverse layer-list)
                   :affines (reverse affines)
                   :batchnorms (reverse batchnorms)
                   :keys keys
                   :weight-decay-lambda weight-decay-lambda
                   :use-batchnorm use-batchnorm
                   :last-layer (make-instance 'softmax-with-loss))))

;; %init-scale lives in multi-layer-net.lisp in spirit, but the two files
;; are never loaded together, so it is duplicated here verbatim.
(defun %init-scale (weight-init-std prev-size)
  (cond ((numberp weight-init-std) weight-init-std)
        ((member weight-init-std '(relu he)) (sqrt (/ 2.0 prev-size)))
        ((member weight-init-std '(sigmoid xavier)) (sqrt (/ 1.0 prev-size)))
        (t (error "unknown weight-init-std"))))

(defgeneric predict (net x))

(defgeneric net-loss (net x target))

(defgeneric net-gradient (net x target))

(defgeneric net-numerical-gradient (net x target))

(defgeneric net-accuracy-count (net x target))

(defmethod predict ((net multi-layer-net-extend) x)
  (let ((out x))
    (dolist (layer (mlne-layers net)) (setq out (forward layer out)))
    out))

(defmethod net-loss ((net multi-layer-net-extend) x target)
  (let ((decay 0) (lam (mlne-weight-decay-lambda net)))
    (dolist (aff (mlne-affines net))
      (setq decay
            (+ decay (* 0.5 lam (linalg:sum (linalg:square (affine-w aff)))))))
    (+ (loss-forward (mlne-last-layer net) (predict net x) target) decay)))

(defmethod net-accuracy-count ((net multi-layer-net-extend) x target)
  (let* ((y (let ((*train-p* nil)) (predict net x)))
         (yl (linalg:argmax y :axis 1))
         (tl
          (if (= (linalg:ndim target) 1)
              target
              (linalg:argmax target :axis 1))))
    (truncate (linalg:sum (linalg:equal yl tl)))))

(defmethod net-gradient ((net multi-layer-net-extend) x target)
  (let ((*train-p* t))
    (net-loss net x target)
    (let ((dout (loss-backward (mlne-last-layer net))))
      (dolist (layer (reverse (mlne-layers net)))
        (setq dout (backward layer dout)))))
  (let ((grads (make-hash-table :test 'equal))
        (lam (mlne-weight-decay-lambda net))
        (idx 0))
    (dolist (aff (mlne-affines net))
      (setq idx (+ idx 1))
      (setf (gethash (format nil "W~a" idx) grads)
            (linalg:add (affine-dw aff) (linalg:mul lam (affine-w aff))))
      (setf (gethash (format nil "b~a" idx) grads) (affine-db aff)))
    (let ((idx2 0))
      (dolist (bn (mlne-batchnorms net))
        (setq idx2 (+ idx2 1))
        (setf (gethash (format nil "gamma~a" idx2) grads) (bn-dgamma bn))
        (setf (gethash (format nil "beta~a" idx2) grads) (bn-dbeta bn))))
    grads))

(defmethod net-numerical-gradient ((net multi-layer-net-extend) x target)
  (let ((grads (make-hash-table :test 'equal))
        (loss-w (lambda (w) (let ((*train-p* t)) (net-loss net x target)))))
    (dolist (key (mlne-keys net))
      (setf (gethash key grads)
            (numerical-gradient loss-w (gethash key (mlne-params net)))))
    grads))


---

# FILE: references/examples/deep-learning-from-scratch/common/multi-layer-net.lisp

;; common/multi_layer_net.py -- the fully-connected multi-layer network
;; (Deep Learning from Scratch). A library file for the ch06 scripts.
;;
;; An arbitrary stack of Affine -> activation layers with He/Xavier weight
;; initialization and L2 weight decay, over the CLOS layers of
;; common/layers.lisp. params/grads are string-keyed hash tables
;; ("W1".."Wn+1", "b1".."bn+1"); mln-keys keeps the deterministic key
;; order (maphash order is unspecified). Do not load this file and
;; multi-layer-net-extend.lisp into the same program -- each pulls in
;; layers.lisp itself.

(load "layers.lisp")
(load "gradient.lisp")

(defclass multi-layer-net ()
  ((params :initarg :params :accessor mln-params)
   (layers :initarg :layers :accessor mln-layers)
   (affines :initarg :affines :accessor mln-affines) ; ordered, 1..n+1
   (keys :initarg :keys :accessor mln-keys)          ; ("W1" "b1" ...)
   (weight-decay-lambda :initarg :weight-decay-lambda
                        :accessor mln-weight-decay-lambda)
   (last-layer :initarg :last-layer :accessor mln-last-layer)))

(defun %init-scale (weight-init-std prev-size)
  ;; A number is used as-is; 'relu / 'he give the He scale sqrt(2/n),
  ;; 'sigmoid / 'xavier the Xavier scale sqrt(1/n).
  (cond ((numberp weight-init-std) weight-init-std)
        ((member weight-init-std '(relu he)) (sqrt (/ 2.0 prev-size)))
        ((member weight-init-std '(sigmoid xavier)) (sqrt (/ 1.0 prev-size)))
        (t (error "unknown weight-init-std"))))

(defun make-multi-layer-net (input-size hidden-size-list output-size &key
                                        (activation 'relu)
                                        (weight-init-std 'relu)
                                        (weight-decay-lambda 0))
  (let* ((all-sizes
          (append (list input-size) hidden-size-list (list output-size)))
         (n-hidden (length hidden-size-list))
         (params (make-hash-table :test 'equal))
         (keys nil)
         (affines nil)
         (layer-list nil))
    ;; weights: W_idx is (all-sizes[idx-1] x all-sizes[idx])
    (do ((idx 1 (+ idx 1)) (sizes all-sizes (cdr sizes)))
        ((null (cdr sizes)))
      (let ((prev (car sizes))
            (next (cadr sizes))
            (wk (format nil "W~a" idx))
            (bk (format nil "b~a" idx)))
        (setf (gethash wk params)
              (linalg:mul (%init-scale weight-init-std prev)
                          (linalg:randn (list prev next))))
        (setf (gethash bk params) (linalg:zeros next))
        (setq keys (cons bk (cons wk keys)))))
    (setq keys (reverse keys))
    ;; layers: Affine -> activation per hidden layer, final Affine
    (do ((idx 1 (+ idx 1)))
        ((> idx (+ n-hidden 1)))
      (let ((aff
             (make-instance 'affine
                            :w (gethash (format nil "W~a" idx) params)
                            :b (gethash (format nil "b~a" idx) params))))
        (setq affines (cons aff affines))
        (setq layer-list (cons aff layer-list))
        (when (<= idx n-hidden)
          (setq layer-list
                (cons (if (eq activation 'sigmoid)
                          (make-instance 'sigmoid-layer)
                          (make-instance 'relu-layer)) layer-list)))))
    (make-instance 'multi-layer-net
                   :params params
                   :layers (reverse layer-list)
                   :affines (reverse affines)
                   :keys keys
                   :weight-decay-lambda weight-decay-lambda
                   :last-layer (make-instance 'softmax-with-loss))))

(defgeneric predict (net x))

(defgeneric net-loss (net x target))

(defgeneric net-gradient (net x target))

(defgeneric net-numerical-gradient (net x target))

(defgeneric net-accuracy-count (net x target))

(defmethod predict ((net multi-layer-net) x)
  (let ((out x))
    (dolist (layer (mln-layers net)) (setq out (forward layer out)))
    out))

(defmethod net-loss ((net multi-layer-net) x target)
  ;; cross-entropy + the 0.5 * lambda * sum(W^2) L2 penalty per weight.
  (let ((decay 0) (lam (mln-weight-decay-lambda net)) (idx 0))
    (dolist (aff (mln-affines net))
      (setq idx (+ idx 1))
      (setq decay
            (+ decay (* 0.5 lam (linalg:sum (linalg:square (affine-w aff)))))))
    (+ (loss-forward (mln-last-layer net) (predict net x) target) decay)))

(defmethod net-accuracy-count ((net multi-layer-net) x target)
  (let* ((y (let ((*train-p* nil)) (predict net x)))
         (yl (linalg:argmax y :axis 1))
         (tl
          (if (= (linalg:ndim target) 1)
              target
              (linalg:argmax target :axis 1))))
    (truncate (linalg:sum (linalg:equal yl tl)))))

(defmethod net-gradient ((net multi-layer-net) x target)
  ;; Backprop; grads["Wi"] = dW_i + lambda * W_i (the weight-decay term).
  (net-loss net x target)
  (let ((dout (loss-backward (mln-last-layer net))))
    (dolist (layer (reverse (mln-layers net)))
      (setq dout (backward layer dout))))
  (let ((grads (make-hash-table :test 'equal))
        (lam (mln-weight-decay-lambda net))
        (idx 0))
    (dolist (aff (mln-affines net))
      (setq idx (+ idx 1))
      (setf (gethash (format nil "W~a" idx) grads)
            (linalg:add (affine-dw aff) (linalg:mul lam (affine-w aff))))
      (setf (gethash (format nil "b~a" idx) grads) (affine-db aff)))
    grads))

(defmethod net-numerical-gradient ((net multi-layer-net) x target)
  (let ((grads (make-hash-table :test 'equal))
        (loss-w (lambda (w) (net-loss net x target))))
    (dolist (key (mln-keys net))
      (setf (gethash key grads)
            (numerical-gradient loss-w (gethash key (mln-params net)))))
    grads))


---

# FILE: references/examples/deep-learning-from-scratch/common/optimizer.lisp

;; common/optimizer.py -- the parameter-update rules (Deep Learning from
;; Scratch): SGD, Momentum, Nesterov, AdaGrad, RMSprop and Adam as CLOS
;; classes sharing one (update optimizer params grads) generic.
;;
;; params and grads are hash tables keyed by strings ("W1", "b1", ...),
;; exactly like the book's dicts, and every rule updates the parameter
;; arrays element-wise IN PLACE -- the layers hold the same array objects
;; (the book's aliasing contract), so an update must never replace them.
;; Optimizer state (v/h/m) is created lazily on the first update, like the
;; book's `if self.v is None`.

(defun %opt-state (params)
  ;; A fresh state dict: one zero array per parameter, matching shapes.
  (let ((state (make-hash-table :test 'equal)))
    (maphash (lambda (key p) (setf (gethash key state) (linalg:zeros-like p)))
             params)
    state))

(defgeneric update (optimizer params grads))

;; --- SGD -----------------------------------------------------------------------

(defclass sgd () ((lr :initarg :lr :initform 0.01 :accessor sgd-lr)))

(defmethod update ((opt sgd) params grads)
  (let ((lr (sgd-lr opt)))
    (maphash (lambda (key p)
               (let ((g (gethash key grads)))
                 (dotimes (k (linalg:size p))
                   (setf (row-major-aref p k)
                    (- (row-major-aref p k) (* lr (row-major-aref g k)))))))
             params)))

;; --- Momentum ------------------------------------------------------------------

(defclass momentum ()
  ((lr :initarg :lr :initform 0.01 :accessor mom-lr)
   (momentum :initarg :momentum :initform 0.9 :accessor mom-momentum)
   (v :initform nil :accessor mom-v)))

(defmethod update ((opt momentum) params grads)
  (unless (mom-v opt) (setf (mom-v opt) (%opt-state params)))
  (let ((lr (mom-lr opt)) (m (mom-momentum opt)))
    (maphash (lambda (key p)
               (let ((g (gethash key grads)) (v (gethash key (mom-v opt))))
                 (dotimes (k (linalg:size p))
                   (let ((vk
                          (- (* m (row-major-aref v k))
                             (* lr (row-major-aref g k)))))
                     (setf (row-major-aref v k) vk)
                     (setf (row-major-aref p k) (+ (row-major-aref p k) vk))))))
             params)))

;; --- Nesterov ------------------------------------------------------------------

(defclass nesterov ()
  ((lr :initarg :lr :initform 0.01 :accessor nes-lr)
   (momentum :initarg :momentum :initform 0.9 :accessor nes-momentum)
   (v :initform nil :accessor nes-v)))

(defmethod update ((opt nesterov) params grads)
  (unless (nes-v opt) (setf (nes-v opt) (%opt-state params)))
  (let ((lr (nes-lr opt)) (m (nes-momentum opt)))
    (maphash (lambda (key p)
               (let ((g (gethash key grads)) (v (gethash key (nes-v opt))))
                 (dotimes (k (linalg:size p))
                   (let ((vk (row-major-aref v k)) (gk (row-major-aref g k)))
                     (setf (row-major-aref p k)
                      (- (+ (row-major-aref p k) (* m m vk)) (* (+ 1 m) lr gk)))
                     (setf (row-major-aref v k) (- (* m vk) (* lr gk)))))))
             params)))

;; --- AdaGrad -------------------------------------------------------------------

(defclass adagrad ()
  ((lr :initarg :lr :initform 0.01 :accessor ada-lr)
   (h :initform nil :accessor ada-h)))

(defmethod update ((opt adagrad) params grads)
  (unless (ada-h opt) (setf (ada-h opt) (%opt-state params)))
  (let ((lr (ada-lr opt)))
    (maphash (lambda (key p)
               (let ((g (gethash key grads)) (h (gethash key (ada-h opt))))
                 (dotimes (k (linalg:size p))
                   (let* ((gk (row-major-aref g k))
                          (hk (+ (row-major-aref h k) (* gk gk))))
                     (setf (row-major-aref h k) hk)
                     (setf (row-major-aref p k)
                           (- (row-major-aref p k)
                              (/ (* lr gk) (+ (sqrt hk) 1.0e-7)))))))) params)))

;; --- RMSprop -------------------------------------------------------------------

(defclass rmsprop ()
  ((lr :initarg :lr :initform 0.01 :accessor rms-lr)
   (decay-rate :initarg :decay-rate :initform 0.99 :accessor rms-decay-rate)
   (h :initform nil :accessor rms-h)))

(defmethod update ((opt rmsprop) params grads)
  (unless (rms-h opt) (setf (rms-h opt) (%opt-state params)))
  (let ((lr (rms-lr opt)) (d (rms-decay-rate opt)))
    (maphash (lambda (key p)
               (let ((g (gethash key grads)) (h (gethash key (rms-h opt))))
                 (dotimes (k (linalg:size p))
                   (let* ((gk (row-major-aref g k))
                          (hk (+ (* d (row-major-aref h k)) (* (- 1 d) gk gk))))
                     (setf (row-major-aref h k) hk)
                     (setf (row-major-aref p k)
                           (- (row-major-aref p k)
                              (/ (* lr gk) (+ (sqrt hk) 1.0e-7)))))))) params)))

;; --- Adam ----------------------------------------------------------------------

(defclass adam ()
  ((lr :initarg :lr :initform 0.001 :accessor adam-lr)
   (beta1 :initarg :beta1 :initform 0.9 :accessor adam-beta1)
   (beta2 :initarg :beta2 :initform 0.999 :accessor adam-beta2)
   (iter :initform 0 :accessor adam-iter) (m :initform nil :accessor adam-m)
   (v :initform nil :accessor adam-v)))

(defmethod update ((opt adam) params grads)
  (unless (adam-m opt)
    (setf (adam-m opt) (%opt-state params))
    (setf (adam-v opt) (%opt-state params)))
  (setf (adam-iter opt) (+ (adam-iter opt) 1))
  (let* ((b1 (adam-beta1 opt))
         (b2 (adam-beta2 opt))
         (iter (adam-iter opt))
         ;; the book's bias-corrected step size
         (lr-t
          (/ (* (adam-lr opt) (sqrt (- 1.0 (expt b2 iter))))
             (- 1.0 (expt b1 iter)))))
    (maphash (lambda (key p)
               (let ((g (gethash key grads))
                     (m (gethash key (adam-m opt)))
                     (v (gethash key (adam-v opt))))
                 (dotimes (k (linalg:size p))
                   (let* ((gk (row-major-aref g k))
                          (mk
                           (+ (row-major-aref m k)
                              (* (- 1 b1) (- gk (row-major-aref m k)))))
                          (vk
                           (+ (row-major-aref v k)
                              (* (- 1 b2) (- (* gk gk) (row-major-aref v k))))))
                     (setf (row-major-aref m k) mk)
                     (setf (row-major-aref v k) vk)
                     (setf (row-major-aref p k)
                           (- (row-major-aref p k)
                              (/ (* lr-t mk) (+ (sqrt vk) 1.0e-7))))))))
             params)))


---

# FILE: references/examples/deep-learning-from-scratch/common/trainer.lisp

;; common/trainer.py -- the training loop (Deep Learning from Scratch).
;;
;; The book's Trainer class is glue, so it becomes one function over the
;; net generics (net-gradient / net-loss / net-accuracy-count), usable
;; with any of the network classes. Batches come from linalg:choice, the
;; optimizer is any instance answering the (update optimizer params grads)
;; generic of common/optimizer.lisp, and accuracies are evaluated per
;; epoch (optionally on a subset, the book's
;; evaluate_sample_num_per_epoch). Returns the list of per-epoch
;; (train-count test-count) pairs.

(load "optimizer.lisp")

(defun train (net params x-train t-train x-test t-test &key (epochs 3)
                  (mini-batch-size 16) optimizer (verbose t) (eval-limit nil))
  (let* ((train-size (car (linalg:shape x-train)))
         (test-size (car (linalg:shape x-test)))
         (iter-per-epoch (max (floor train-size mini-batch-size) 1))
         (max-iter (* epochs iter-per-epoch))
         (opt (if optimizer optimizer (make-instance 'sgd :lr 0.01)))
         (n-train-eval (if eval-limit (min eval-limit train-size) train-size))
         (n-test-eval (if eval-limit (min eval-limit test-size) test-size))
         (x-train-eval
          (if eval-limit
              (linalg:take-rows x-train (linalg:arange n-train-eval))
              x-train))
         (t-train-eval
          (if eval-limit
              (linalg:take-rows t-train (linalg:arange n-train-eval))
              t-train))
         (x-test-eval
          (if eval-limit
              (linalg:take-rows x-test (linalg:arange n-test-eval))
              x-test))
         (t-test-eval
          (if eval-limit
              (linalg:take-rows t-test (linalg:arange n-test-eval))
              t-test))
         (acc-list nil)
         (epoch 0))
    (dotimes (i max-iter)
      (let* ((batch-mask (linalg:choice train-size mini-batch-size))
             (x-batch (linalg:take-rows x-train batch-mask))
             (t-batch (linalg:take-rows t-train batch-mask))
             (grads (net-gradient net x-batch t-batch)))
        (update opt params grads)
        (when (= (mod (+ i 1) iter-per-epoch) 0)
          (setq epoch (+ epoch 1))
          (let ((train-count (net-accuracy-count net x-train-eval t-train-eval))
                (test-count (net-accuracy-count net x-test-eval t-test-eval)))
            (setq acc-list (cons (list train-count test-count) acc-list))
            (when verbose
              (format t "=== epoch ~a: train acc ~a/~a, test acc ~a/~a ===~%"
                      epoch train-count n-train-eval test-count
                      n-test-eval))))))
    (reverse acc-list)))


---

# FILE: references/examples/deep-learning-from-scratch/common/util.lisp

;; common/util.py -- im2col / col2im (Deep Learning from Scratch).
;;
;; The two window transforms that turn a convolution into one matrix
;; product: im2col unfolds a rank-4 NCHW batch into the matrix whose row
;; (n, out-y, out-x) holds the filter-h x filter-w window of every channel
;; at that output position, and col2im is its adjoint -- a scatter-ADD fold
;; back into the image shape (overlapping windows accumulate), the shape
;; the convolution backward pass needs. The heavy index loops live in
;; linalg (linalg::%la-im2col / %la-col2im): direct index arithmetic
;; equivalent to the book's pad + strided-slice + 6-D transpose
;; composition, without materializing the scratch tensors.

(defun im2col (input-data filter-h filter-w &optional (stride 1) (pad 0))
  ;; (N C H W) -> (N*out-h*out-w, C*filter-h*filter-w); elements that fall
  ;; in the zero padding read 0.0.
  (linalg::%la-im2col input-data filter-h filter-w stride pad))

(defun col2im (col input-shape filter-h filter-w &optional (stride 1) (pad 0))
  ;; The im2col adjoint: scatter-adds col back into a fresh zero array of
  ;; input-shape (a dims list (N C H W)); padding contributions are dropped.
  (linalg::%la-col2im col input-shape filter-h filter-w stride pad))


---

# FILE: references/examples/deep-learning-from-scratch/dataset/mnist.lisp

;; MNIST idx-file loader (the port of the book's dataset/mnist.py).
;;
;; Reads the DECOMPRESSED idx files fetched by ../download-mnist.sh through
;; binary file streams (read-byte over :element-type '(unsigned-byte 8)),
;; which work on every backend -- the WASM targets need the preopen flag
;; --dir . and paths relative to the example root. Reading is byte-at-a-time,
;; so the loaders take a LIMIT (row count) and only consume what a script
;; needs; the full 60000-image train set is never required by the examples.
;;
;; The big-endian binary primitives and the RLW1 weight reader live in
;; rlw1.lisp (a provide/require module shared with the ch07/ch08
;; net-load-params); this file keeps the idx loaders and the plist-shaped
;; loader for the book's pretrained ch03 weights (sample-weight.bin).

(require "rlw1" "rlw1.lisp")

;; --- MNIST loaders ------------------------------------------------------------

(defun mnist-load-images (path &optional (limit 1000) (offset 0))
  ;; LIMIT images from an idx3 image file starting at row OFFSET, as a
  ;; (limit x 784) packed double matrix with pixels normalized to [0, 1]
  ;; (the book loader's normalize=True, flatten=True shape). Doubles keep
  ;; every linalg op in the #d default width, where --simd output is
  ;; bit-identical to the scalar path.
  (with-open-file (s path :element-type '(unsigned-byte 8))
    (let ((magic (%read-be32 s))
          (count (%read-be32 s))
          (rows (%read-be32 s))
          (cols (%read-be32 s)))
      (unless (= magic 2051)
        (error "not an idx3 image file (run ./download-mnist.sh first)"))
      (let* ((pixels (* rows cols))
             (n (min limit (- count offset)))
             (out
              (make-array (list n pixels)
                          :element-type 'double-float
                          :initial-element 0.0)))
        (%skip-bytes s (* offset pixels))
        (dotimes (i (* n pixels))
          (setf (row-major-aref out i) (/ (read-byte s) 255.0)))
        out))))

(defun mnist-load-labels (path &optional (limit 1000) (offset 0) one-hot)
  ;; LIMIT labels from an idx1 label file starting at row OFFSET: a packed
  ;; double vector of label values 0..9, or with ONE-HOT the (limit x 10)
  ;; one-hot matrix (the book loader's one_hot_label=True).
  (with-open-file (s path :element-type '(unsigned-byte 8))
    (let ((magic (%read-be32 s)) (count (%read-be32 s)))
      (unless (= magic 2049)
        (error "not an idx1 label file (run ./download-mnist.sh first)"))
      (let* ((n (min limit (- count offset)))
             (lab
              (make-array n :element-type 'double-float :initial-element 0.0)))
        (%skip-bytes s offset)
        (dotimes (i n) (setf (aref lab i) (read-byte s)))
        (if one-hot (linalg:one-hot lab 10) lab)))))

;; --- pretrained ch03 weights ---------------------------------------------------

(defun load-sample-weight (path)
  ;; The book's pretrained 784-50-100-10 network from sample-weight.bin
  ;; (see tools/export-sample-weight.py for the format), as the plist
  ;; (:w1 W1 :b1 b1 :w2 W2 :b2 b2 :w3 W3 :b3 b3) of packed double arrays.
  (let ((v (load-rlw1 path nil)))
    (list :w1 (nth 0 v)
          :b1 (nth 1 v)
          :w2 (nth 2 v)
          :b2 (nth 3 v)
          :w3 (nth 4 v)
          :b3 (nth 5 v))))


---

# FILE: references/examples/deep-learning-from-scratch/dataset/rlw1.lisp

;; RLW1 pretrained-weight reader + the shared big-endian binary primitives.
;;
;; RLW1 is the simple binary format tools/export-sample-weight.py re-exports
;; the book's pickled numpy params into ("RLW1", u8 count, then per array
;; u8 ndim / u32 dims / big-endian f32 data, row-major); rontolisp cannot
;; read pickle. Parsed byte-at-a-time with read-byte, which works on every
;; backend -- the WASM targets need the preopen flag --dir . and paths
;; relative to the example root.
;;
;; A provide/require module (not a plain load) because it sits on a diamond:
;; dataset/mnist.lisp needs the primitives for the idx headers, and the
;; ch07/ch08 net classes need load-rlw1 for their net-load-params, so a
;; script loading both must splice this file exactly once.

(provide "rlw1")

(defun %read-be32 (s)
  ;; A big-endian unsigned 32-bit integer. Every value read this way in the
  ;; idx/RLW1 headers is far below 2^30, so the arithmetic stays inside the
  ;; WASM i31 integer range.
  (let* ((b0 (read-byte s))
         (b1 (read-byte s))
         (b2 (read-byte s))
         (b3 (read-byte s)))
    (+ (* b0 16777216) (* b1 65536) (* b2 256) b3)))

(defun %read-f32 (s)
  ;; A big-endian IEEE-754 single float, widened exactly to a double. The
  ;; sign/exponent/mantissa are assembled without ever forming the full
  ;; 32-bit word (the 23-bit mantissa is the largest integer built, i31-safe).
  (let* ((b0 (read-byte s))
         (b1 (read-byte s))
         (b2 (read-byte s))
         (b3 (read-byte s))
         (sign (if (>= b0 128) -1.0 1.0))
         (e (+ (* (mod b0 128) 2) (floor (/ b1 128))))
         (m (+ (* (mod b1 128) 65536) (* b2 256) b3)))
    (if (= e 0)
        ;; Zero / subnormal (the book's weights contain none of the latter;
        ;; a subnormal degrades to signed zero here).
        (* sign 0.0)
        (* sign (+ 1.0 (/ m 8388608.0)) (expt 2.0 (- e 127))))))

(defun %skip-bytes (s n)
  ;; Consumes n bytes (there is no seek in the stream API).
  (dotimes (i n) (read-byte s)))

(defun %rlw1-make (shape element-type)
  ;; Both branches take a LITERAL :element-type, so every backend picks the
  ;; packed double[]/float[] representation statically (the linalg
  ;; %la-make pattern); nil / anything else defaults to double.
  (if (eq element-type 'single-float)
      (make-array shape :element-type 'single-float :initial-element 0.0)
      (make-array shape :element-type 'double-float :initial-element 0.0)))

(defun load-rlw1 (path element-type)
  ;; Every array of an RLW1 file, in file order, as a list of packed arrays
  ;; of the requested width (nil = double-float). An f32 value is exact in
  ;; both widths, so 'single-float loses nothing over the file content.
  (with-open-file (s path :element-type '(unsigned-byte 8))
    (unless (and (= (read-byte s) 82) (= (read-byte s) 76) (= (read-byte s) 87)
                 (= (read-byte s) 49))
      (error "not an RLW1 weight file (see tools/export-sample-weight.py)"))
    (let ((count (read-byte s)) (arrays nil))
      (dotimes (k count)
        (let* ((ndim (read-byte s)) (dims nil))
          (dotimes (d ndim) (setq dims (cons (%read-be32 s) dims)))
          (setq dims (reverse dims))
          (let* ((shape (if (cdr dims) dims (car dims)))
                 (a (%rlw1-make shape element-type)))
            (dotimes (i (array-total-size a))
              (setf (row-major-aref a i) (%read-f32 s)))
            (setq arrays (cons a arrays)))))
      (reverse arrays))))


---

# FILE: references/examples/deep-learning-from-scratch/download-mnist.sh

#!/bin/sh
# Fetches the four MNIST idx files into dataset/ and decompresses them
# (rontolisp has no gzip support, so the Lisp loaders read the raw idx
# files). The mirror is the same one the book's dataset/mnist.py uses.
# Run once before any MNIST example: ./download-mnist.sh
set -e
cd "$(dirname "$0")/dataset"
base=https://ossci-datasets.s3.amazonaws.com/mnist
for f in train-images-idx3-ubyte train-labels-idx1-ubyte \
         t10k-images-idx3-ubyte t10k-labels-idx1-ubyte; do
  if [ -f "$f" ]; then
    echo "$f: already present"
    continue
  fi
  echo "downloading $f.gz ..."
  curl -fsSL -o "$f.gz" "$base/$f.gz"
  gunzip -f "$f.gz"
done
echo "done."


---

# FILE: references/examples/deep-learning-from-scratch/tools/export-sample-weight.py

#!/usr/bin/env python3
"""One-time converter: a pickled params dict of numpy arrays -> an RLW1 binary.

rontolisp cannot read pickle, so pretrained network weights are re-exported as
a simple big-endian binary the Lisp loader (dataset/rlw1.lisp, load-rlw1)
parses with read-byte:

    "RLW1"                        4 magic bytes
    u8   array-count
    per array (in the given key order):
      u8   ndim
      u32  dims[ndim]             big-endian
      f32  data[prod(dims)]       big-endian IEEE-754, row-major

Usage:
    python3 export-sample-weight.py PKL [-o OUT.bin] [--keys K1 K2 ...]

Defaults reproduce the original ch03 export (sample_weight.pkl -> the
committed ch03/sample-weight.bin): keys W1 b1 W2 b2 W3 b3, output
../ch03/sample-weight.bin next to this script. The ch07/ch08 pretrained
params were exported from the book repo with:

    python3 export-sample-weight.py .../ch07/params.pkl \
        -o ../ch07/params.bin
    python3 export-sample-weight.py .../ch08/deep_convnet_params.pkl \
        -o ../ch08/deep-convnet-params.bin \
        --keys W1 b1 W2 b2 W3 b3 W4 b4 W5 b5 W6 b6 W7 b7 W8 b8

The .bin files are committed, so this script only matters when re-exporting
from the book repo.
"""

import argparse
import os
import pickle
import struct


def main() -> None:
    parser = argparse.ArgumentParser(
        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
    )
    parser.add_argument("pkl", help="pickled dict of numpy arrays (from the book repo)")
    parser.add_argument(
        "-o",
        "--out",
        default=os.path.join(os.path.dirname(__file__), "..", "ch03", "sample-weight.bin"),
        help="output .bin path (default: ../ch03/sample-weight.bin)",
    )
    parser.add_argument(
        "--keys",
        nargs="+",
        default=["W1", "b1", "W2", "b2", "W3", "b3"],
        help="array keys in the order the Lisp loader expects (default: W1 b1 W2 b2 W3 b3)",
    )
    args = parser.parse_args()

    with open(args.pkl, "rb") as f:
        net = pickle.load(f)
    missing = [k for k in args.keys if k not in net]
    if missing:
        parser.error(f"keys not in {args.pkl}: {missing} (has {sorted(net)})")
    with open(args.out, "wb") as f:
        f.write(b"RLW1")
        f.write(bytes([len(args.keys)]))
        for key in args.keys:
            a = net[key].astype(">f4")
            f.write(bytes([a.ndim]))
            for d in a.shape:
                f.write(struct.pack(">I", d))
            f.write(a.tobytes())
    print(f"wrote {os.path.normpath(args.out)} ({os.path.getsize(args.out)} bytes)")


if __name__ == "__main__":
    main()


---

# FILE: references/examples/examples.yaml

# Manifest of the non-GUI examples: the backends each can be verified on, plus
# the inputs it needs and the expected result to check.
#
# Consumed by src/test/java/am/ik/rontolisp/e2e/ExamplesE2eTest.java, which turns
# every (example x backend) pair into one JUnit dynamic test. See that class and
# examples/README.md for how to run it.
#
# backend tokens
# --------------
#   RUN (the program runs to completion; we assert exit 0 and check the output):
#     interpreter      FILE [args]
#     jvm              FILE -o Prog.class   then  java -cp .: Prog [args]
#                     (the jar is on the classpath because a compiled program that
#                     reaches a runtime support class -- an HTTP server, a socket,
#                     a Gray stream -- needs it; most are self-contained and do not)
#     wasm             FILE -o prog.wasm --optimize   then  wasmtime run -W gc --dir . prog.wasm [args]
#   COMPILE (blocking servers / host-invoked modules never return on their own,
#   so we only build them and assert the compile succeeds):
#     jvm-compile      FILE -o Prog.class
#     wasm-component   FILE -o prog.wasm --component --optimize
#     no-gc            FILE -o prog.wasm --no-gc --optimize
#     no-gc-simd       FILE -o prog.wasm --no-gc --simd --optimize
#
# Every WASM compile passes --optimize (the dead-code tree-shaker) explicitly,
# for the record -- it has been the CLI default since todo-448, so the flag no
# longer changes what gets built here; it is also a no-op under --component.
# The `wasm` (run) backend is skipped when wasmtime is not on PATH; the compile
# backends need no runtime.
#
# per-example fields (all optional except path/backends)
# ------------------------------------------------------
#   args         command-line arguments appended when the program is run
#   stdin        text fed to the program's standard input (inline)
#   stdinFile    ...or from a file, resolved under examples/
#   expect       how to check RUN output -- exactly one of:
#                  equals:    stdout must equal this text (hard-coded here)
#                  file:      stdout must equal this file's contents (under examples/)
#                  contains:  every listed substring must appear (partial match)
#                  skip: true do not check output (random / non-repeatable results)
#                omit expect entirely -> baseline "exit 0 and non-empty output".
#                equals/file are compared against ALL run backends, so a per-backend
#                divergence is a real failure.
#   systemPath   directory -- or a LIST of directories -- added as
#                --system-path (the ASDF source registry). Each element is
#                resolved against the repository root and they are joined with
#                the platform path separator, so a system whose dependencies
#                are vendored side by side (rove needs rove + dissect +
#                cl-ppcre) names them all. Passed to the interpreter leg too.
#   workDir      sub-dir under examples/ the process runs from (default: none,
#                i.e. the throwaway workdir itself). Set it when the script's
#                CWD-relative reads assume a book root (e.g. deep-learning-from-
#                scratch/); the leg's CWD becomes work//.
#   workFiles    files to stage beside the program before it runs (relative to
#                workDir when set, otherwise relative to examples/). Copied 1:1
#                into the workspace, so the mirrored slice looks like the
#                fragment of examples/ the script was written against. A missing
#                file skips the leg (assumption abort) rather than failing --
#                for gitignored dataset dumps you fetch via
#                deep-learning-from-scratch/download-mnist.sh.
#   note         free-form comment
#
# GUI examples (jvm/, minesweeper, rainbow/, the webgl-*/ and other browser
# demos) are intentionally absent: they open a window or run in a page and
# cannot be checked headless. Two exceptions, both for the same reason -- the
# part that is not GUI: browser/wit-component's Lisp half is a pure-compute
# module with no host imports, so its COMPILE leg pins its WIT world against the
# program even though its jco/browser half cannot run here; and
# browser/minesweeper's rules live in a rendering-free core, which
# minesweeper-core-test.lisp loads and checks with rove.

examples:
  # --- console: pure, cross-backend algorithms & I/O -------------------------
  - path: console/calc.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/calc.txt
  - path: console/contact-book.lisp
    backends: [interpreter, jvm, wasm]
    note: per-contact listing order follows hash-table iteration, which varies by backend
    expect:
      contains:
        - "Contact Book (3 entries):"
        - "Alice -> alice@newdomain.com"
        - "Alice (alice@newdomain.com)"
  - path: console/error-handling.lisp
    backends: [interpreter, jvm, wasm]
    note: catching compiles via the wasm exception-handling proposal; the driver passes -W exceptions=y
    expect:
      file: .expected/error-handling.txt
  - path: console/hanoi.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/hanoi.txt
  - path: console/l-system.lisp
    backends: [interpreter, jvm, wasm]
    note: the character-distribution lines follow hash-table iteration order
    expect:
      contains:
        - "L-system String Rewriting"
        - "Total length: 611"
        - "At iteration 12:"
        - "Length: 16382 characters"
  - path: console/life.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/life.txt
  - path: console/line-numbers.lisp
    backends: [interpreter, jvm, wasm]
    note: writes poem.txt / poem-numbered.txt into the working directory
    expect:
      file: .expected/line-numbers.txt
  - path: console/mandelbrot.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/mandelbrot.txt
  - path: console/nqueens.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/nqueens.txt
  - path: console/parse-numbers.lisp
    backends: [interpreter, jvm, wasm]
    note: writes numbers.txt into the working directory; small output hard-coded inline
    expect:
      equals: |
        count=5 sum=97 min=3 max=42
        letters=9 digits=6
  - path: console/roman.lisp
    backends: [interpreter, jvm, wasm]
    systemPath: [src/test/resources/rove, src/test/resources/dissect, src/test/resources/cl-ppcre]
    note: >-
      the example CHECKS ITSELF with rove: the 1..3999 round-trip and the
      known encodings/decodings are assertions, and the file ends with
      (uiop:quit (if (run-suite *package*) 0 1)) -- so a broken encoder exits
      non-zero and fails this leg instead of printing a line nobody reads. The
      needles are the demo tables and rove's summary only; the per-assertion
      lines print the forms, which spell symbols package-qualified until
      .todo/391 lands, and rove appends a "(Nms)" duration to any assertion
      slower than 37ms.
    expect:
      contains:
        - "Integer -> Roman:"
        - "     1 -> I"
        - "   400 -> CD"
        - "  3999 -> MMMCMXCIX"
        - "Roman -> Integer:"
        - '  "MCMXCIX" -> 1999'
        - "✓ 3 tests completed"
        - "All 3 tests passed."
  - path: console/sieve.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/sieve.txt
  - path: console/sorting.lisp
    backends: [interpreter, jvm, wasm]
    note: small, fully deterministic output hard-coded inline
    expect:
      equals: |
        input:               (5 -3 8 -1 2 -7 4 0 6)
        quicksort   (<):     (-7 -3 -1 0 2 4 5 6 8)
        merge-sort  (<):     (-7 -3 -1 0 2 4 5 6 8)
        built-in sort (<):   (-7 -3 -1 0 2 4 5 6 8)
        quicksort |x| desc:  (8 -7 6 5 4 -3 2 -1 0)
  - path: console/word-frequency.lisp
    backends: [interpreter, jvm, wasm]
    note: ranks with tied counts reorder by hash-table iteration order
    expect:
      contains:
        - "Total unique words: 30"
        - "Top 10 words:"
        - '"to" 4'
        - "Words appearing exactly once:"

  # --- ml: numerical computing & machine learning ----------------------------
  - path: ml/deep-digits.lisp
    backends: [interpreter, jvm, wasm]
    note: deterministic (fixed-seed LCG init, integer-scaled loss)
    expect:
      file: .expected/deep-digits.txt
  - path: ml/heat3d.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/heat3d.txt
  - path: ml/linear-regression.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/linear-regression.txt
  - path: ml/maze-rl.lisp
    backends: [interpreter, jvm, wasm]
    note: unseeded random -- path, value count and the drawn maze differ every run
    expect:
      skip: true
  - path: ml/mlp.lisp
    backends: [interpreter, jvm, wasm]
    note: random init/data -- only the fixed header lines are stable
    expect:
      contains:
        - "Circle classification, 2-8-1 MLP"
        - "train=150 examples, test=60 examples"
        - "Final train accuracy:"
  - path: ml/nn.lisp
    backends: [interpreter, jvm, wasm]
    note: random weight init -- losses/predictions vary, headings are stable
    expect:
      contains:
        - "Training XOR (2-4-1 network)..."
        - "Predictions after training:"
  - path: ml/nn-vec.lisp
    backends: [interpreter, jvm, wasm]
    note: nn.lisp rewritten with the vec (vec:matvec GEMV, Hadamard deltas) and linalg (transpose/outer/emap) packages instead of hand-written loops; single-float (#f) throughout -- linalg is width-polymorphic and preserves the #f width, so the JVM --simd pass stays f32; random init, so only the headings are stable
    expect:
      contains:
        - "Training XOR (2-4-1 network, vec + linalg)..."
        - "Predictions after training:"
  - path: ml/numerical-calculus.lisp
    backends: [interpreter, jvm, wasm]
    note: numpy-style diff/gradient (linalg calculus) -- every sample is a polynomial at integer coordinates, so the differences are exact and the output byte-identical on every backend
    expect:
      file: .expected/numerical-calculus.txt
  - path: ml/simd-dot.lisp
    backends: [interpreter, jvm, wasm]
    note: the minimal --simd showcase -- one vec:dot over 1024 doubles, 4000 times. The vector is 0.0..1023.0, so the dot is an exact integer and every partial sum is exactly representable; lane-order folding therefore cannot change it. The elapsed-time line is not checked. Run it a second time with --simd - interpreter 2.59 s -> 2.3 ms, wasm-GC 273 -> 2.4 ms
    expect:
      contains:
        - "(vec:dot v v) over 1024 doubles, 4000 times = 4096000 multiply-adds"
        - "sum of squares below 1024 = 357389824"
  - path: ml/simd-gemv.lisp
    backends: [interpreter, jvm, wasm]
    note: the --simd showcase -- 100 steps of vec:matvec (GEMV) + vec:dot on a 256x256 single-float matrix; deterministic (fixed-seed LCG weights, argmax indices printed instead of floats, so lane-order rounding cannot move them). The elapsed-time line is not checked. Run it a second time with --simd - wasm-GC 467 -> 3.9 ms, interpreter 4.6 s -> 15 ms
    expect:
      contains:
        - "simd-gemv: 100 steps of (vec:matvec w x) on a 256x256 single-float matrix"
        - "argmax after steps 1-10: (0 14 82 126 14 140 126 79 134 175)"
        - "argmax after step 100:   85"
  - path: ml/simd-gemv-nogc.lisp
    backends: [no-gc, no-gc-simd]
    note: simd-gemv's inner loop as a --no-gc reactor (the rank-2 packed matrix layout + vec:matvec GEMV) -- the host invokes the fingerprint export (wasmtime run --invoke fingerprint prog.wasm 100 -> 85, matching simd-gemv.lisp's step-100 argmax on both lowerings; verified manually, the harness only compiles). matvec-into/scale-into keep the never-freed bump heap at exactly three blocks
  - path: ml/tiny-llm.lisp
    backends: [interpreter, jvm, wasm]
    note: a 2-layer transformer decoder -- llama2's forward() without the tokenizer or the weight loader (RMSNorm, Q/K/V projections, causal self-attention over a KV cache, softmax, SwiGLU FFN, residuals, classifier head, greedy argmax decode). Deterministic (fixed-seed pseudo-random weights, token ids printed, not floats). 13 GEMVs per forward pass, so it is the example --simd earns its keep on. The elapsed-time line is not checked. Run it a second time with --simd - native interpreter 11.3 s -> 20 ms, wasm-GC 891 -> 7.8 ms. Uses vec:matvec and linalg:, so no --no-gc. ~13 s on the interpreter leg
    expect:
      contains:
        - "tiny-llm: 2-layer transformer decoder, dim=256 hidden=512 ctx=12 vocab=48, single-float"
        - "prompt:    (3 14 1 5)"
        - "generated: (39 27 23 18 42 7 5 39 27)"

  # --- llama2: llama2.c ported whole (see its README.md) ----------------------
  # Both entries run under --simd (the interpreter's scalar kernels take ~15 s per
  # stories15M token) with the knobs in the environment (a program has no argv).
  # Greedy decoding, so the story is byte-identical on every backend AND to the C
  # program's -- the whole point of an `equals`.
  - path: llama2/llama2.lisp
    backends: [interpreter, jvm, wasm, wasm-component]
    workDir: llama2
    workFiles: [stories260K.bin, tok512.bin]
    simd: true
    env:
      LLAMA2_CHECKPOINT: stories260K.bin
      LLAMA2_TOKENIZER: tok512.bin
      LLAMA2_PROMPT: Once upon a time
      LLAMA2_TEMPERATURE: "0"
      LLAMA2_STEPS: "40"
    note: run.c ported whole -- checkpoint loader (read-sequence over packed single-float arrays), tokenizer + BPE encoder, forward pass, sampler, generate -- over the checked-in 1 MB stories260K model. Greedy (temperature 0), 40 tokens, so the story is the same on every backend and in `run stories260K.bin -z tok512.bin -t 0 -n 40 -i "Once upon a time"`. The tok/s line goes to stderr and is not checked
    expect:
      equals: |
        Once upon a time, there was a little girl named Lily. She loved to play outside in the park. One day, she saw a big, r
  - path: llama2/llama2.lisp
    backends: [interpreter, jvm, wasm]
    workDir: llama2
    workFiles: [stories15M.bin, tokenizer.bin]
    simd: true
    env:
      LLAMA2_PROMPT: Once upon a time
      LLAMA2_TEMPERATURE: "0"
      LLAMA2_STEPS: "60"
    note: the llama2.c README demo itself, over stories15M.bin (60 MB, gitignored -- llama2/download-stories15M.sh); the RUN legs skip themselves when it is absent. 60 greedy tokens = the same text `run stories15M.bin -t 0 -n 60 -i "Once upon a time"` prints. Measured with --simd - JVM 87 tok/s, wasm-GC 46 tok/s, native interpreter 15-25 tok/s
    expect:
      equals: |
        Once upon a time, there was a little girl named Lily. She loved to play outside in the sunshine. One day, she saw a big, red ball in the sky. It was the sun! She thought it was so pretty.
        Lily wanted to play with the ball

  # --- deep-learning-from-scratch: the book, ported (see its README.md) -------
  # The data-free scripts print byte-identically on every backend (weights come
  # from the seeded linalg RNG; the exp-using ones round through truncate or a
  # bucketed histogram, which the WASM exp approximation cannot flip). Scripts
  # that read committed weight files (ch07/params.bin, ch03/sample-weight.bin)
  # get RUN legs, with the file staged via workFiles. Scripts that also read
  # dataset/*-ubyte (the gitignored idx dumps -- fetch via ./download-mnist.sh)
  # ALSO get RUN legs, but each leg skips itself when the dumps are absent
  # (assumption abort, not failure) -- CI without the dumps only exercises the
  # compile-only wasm-component leg, a developer with them run gets the full
  # cross-backend accuracy check. Only the ch08 training scripts stay
  # compile-only: their deep-convnet forward pass is minutes per interpreter
  # leg (revisit once it shrinks).
  - path: deep-learning-from-scratch/ch02/and-gate.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/dlfs-and-gate.txt
  - path: deep-learning-from-scratch/ch02/nand-gate.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/dlfs-nand-gate.txt
  - path: deep-learning-from-scratch/ch02/or-gate.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/dlfs-or-gate.txt
  - path: deep-learning-from-scratch/ch02/xor-gate.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/dlfs-xor-gate.txt
  - path: deep-learning-from-scratch/ch03/activation-functions.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/dlfs-activation-functions.txt
  - path: deep-learning-from-scratch/ch03/mnist-show.lisp
    backends: [interpreter, jvm, wasm, wasm-component]
    workDir: deep-learning-from-scratch
    workFiles: [dataset/train-images-idx3-ubyte, dataset/train-labels-idx1-ubyte]
    note: reads dataset/train-*-ubyte (gitignored -- ./download-mnist.sh); RUN legs skip themselves when the dumps are absent
    expect:
      file: .expected/dlfs-mnist-show.txt
  - path: deep-learning-from-scratch/ch03/neuralnet-mnist.lisp
    backends: [interpreter, jvm, wasm, wasm-component]
    workDir: deep-learning-from-scratch
    workFiles: [ch03/sample-weight.bin, dataset/t10k-images-idx3-ubyte, dataset/t10k-labels-idx1-ubyte]
    note: reads dataset/t10k-*-ubyte (gitignored -- ./download-mnist.sh) + ch03/sample-weight.bin (committed); RUN legs skip themselves when the idx dumps are absent; softmax + argmax over 1000 images -> the same class per image on every backend
    expect:
      equals: |
        Accuracy: 932/1000
  - path: deep-learning-from-scratch/ch03/neuralnet-mnist-batch.lisp
    backends: [interpreter, jvm, wasm, wasm-component]
    workDir: deep-learning-from-scratch
    workFiles: [ch03/sample-weight.bin, dataset/t10k-images-idx3-ubyte, dataset/t10k-labels-idx1-ubyte]
    note: reads dataset/t10k-*-ubyte (gitignored -- ./download-mnist.sh) + ch03/sample-weight.bin (committed); RUN legs skip themselves when the idx dumps are absent
    expect:
      equals: |
        Accuracy: 932/1000
  - path: deep-learning-from-scratch/ch04/gradient-1d.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/dlfs-gradient-1d.txt
  - path: deep-learning-from-scratch/ch04/gradient-2d.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/dlfs-gradient-2d.txt
  - path: deep-learning-from-scratch/ch04/gradient-method.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/dlfs-gradient-method.txt
  - path: deep-learning-from-scratch/ch04/gradient-simplenet.lisp
    backends: [interpreter, jvm]
    note: the numerical dW passes through exp/log (softmax loss), whose last printed digits differ on WASM
    expect:
      contains:
        - "loss:"
        - "dW:"
  - path: deep-learning-from-scratch/ch04/train-neuralnet.lisp
    backends: [jvm-compile, wasm-component]
    note: reads dataset/train-*-ubyte (./download-mnist.sh); ~2 min interpreted, seconds under --simd
  - path: deep-learning-from-scratch/ch05/buy-apple.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/dlfs-buy-apple.txt
  - path: deep-learning-from-scratch/ch05/buy-apple-orange.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/dlfs-buy-apple-orange.txt
  - path: deep-learning-from-scratch/ch05/gradient-check.lisp
    backends: [interpreter, jvm, wasm]
    note: the correctness gate of the CLOS layer library -- backprop must match central differences
    expect:
      file: .expected/dlfs-gradient-check.txt
  - path: deep-learning-from-scratch/ch05/train-neuralnet.lisp
    backends: [jvm-compile, wasm-component]
    note: reads dataset/train-*-ubyte (./download-mnist.sh)
  - path: deep-learning-from-scratch/ch06/optimizer-compare-naive.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/dlfs-optimizer-compare-naive.txt
  - path: deep-learning-from-scratch/ch06/optimizer-compare-mnist.lisp
    backends: [jvm-compile, wasm-component]
    note: reads dataset/train-*-ubyte (./download-mnist.sh)
  - path: deep-learning-from-scratch/ch06/weight-init-activation-histogram.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/dlfs-weight-init-activation-histogram.txt
  - path: deep-learning-from-scratch/ch06/weight-init-compare.lisp
    backends: [jvm-compile, wasm-component]
    note: reads dataset/train-*-ubyte (./download-mnist.sh)
  - path: deep-learning-from-scratch/ch06/batch-norm-gradient-check.lisp
    backends: [interpreter, jvm, wasm]
    expect:
      file: .expected/dlfs-batch-norm-gradient-check.txt
  - path: deep-learning-from-scratch/ch06/batch-norm-test.lisp
    backends: [jvm-compile, wasm-component]
    note: reads dataset/train-*-ubyte (./download-mnist.sh)
  - path: deep-learning-from-scratch/ch06/overfit-weight-decay.lisp
    backends: [jvm-compile, wasm-component]
    note: reads dataset/*-ubyte (./download-mnist.sh)
  - path: deep-learning-from-scratch/ch06/overfit-dropout.lisp
    backends: [jvm-compile, wasm-component]
    note: reads dataset/*-ubyte (./download-mnist.sh)
  - path: deep-learning-from-scratch/ch06/hyperparameter-optimization.lisp
    backends: [jvm-compile, wasm-component]
    note: reads dataset/train-*-ubyte (./download-mnist.sh)
  - path: deep-learning-from-scratch/ch07/gradient-check.lisp
    backends: [interpreter, jvm, wasm]
    note: the correctness gate of the Convolution/Pooling layers (im2col/col2im backprop)
    expect:
      file: .expected/dlfs-convnet-gradient-check.txt
  - path: deep-learning-from-scratch/ch07/train-convnet.lisp
    backends: [jvm-compile, wasm-component]
    note: reads dataset/*-ubyte (./download-mnist.sh)
  - path: deep-learning-from-scratch/ch07/visualize-filter.lisp
    backends: [interpreter, jvm, wasm, wasm-component]
    workDir: deep-learning-from-scratch
    workFiles: [ch07/params.bin]
    note: reads ch07/params.bin (committed) -- staged into the workspace via workFiles; output byte-identical across all four backends
    expect:
      file: .expected/dlfs-visualize-filter.txt
  - path: deep-learning-from-scratch/ch08/train-deepnet.lisp
    backends: [jvm-compile, wasm-component]
    note: reads dataset/*-ubyte (./download-mnist.sh)
  - path: deep-learning-from-scratch/ch08/misclassified-mnist.lisp
    backends: [jvm-compile, wasm-component]
    note: reads dataset/t10k-*-ubyte + ch08/deep-convnet-params.bin
  - path: deep-learning-from-scratch/ch08/half-float-network.lisp
    backends: [jvm-compile, wasm-component]
    note: reads dataset/t10k-*-ubyte + ch08/deep-convnet-params.bin

  # --- llm-from-scratch: the book's Transformer chapter, ported (README.md) ---
  # Deterministic on every backend: every weight, dropout mask and epoch shuffle
  # comes from the seeded linalg generator (integer arithmetic, bit-identical
  # everywhere), and every printed float is rounded to a few decimals so the
  # low-order digits of the WASM exp/log/sin approximations cannot show through.
  # section5 is the training program -- it is the slow one, which is why its
  # shapes are the shrunken ones its README documents next to the book's, and
  # why it is one of the two entries here that run under --simd.
  - path: llm-from-scratch/transformer/shapes.lisp
    backends: [interpreter, jvm, wasm, wasm-component]
    note: attention.py's __main__ plus the same shape check for the whole model -- shapes and counts only, so nothing float-sensitive is printed
    expect:
      file: .expected/llm-shapes.txt
  - path: llm-from-scratch/chapter02/section2.lisp
    backends: [interpreter, jvm, wasm, wasm-component]
    note: notebook 2.2 -- attention as pure linalg, no autograd and no modules
    expect:
      file: .expected/llm-section2.txt
  - path: llm-from-scratch/chapter02/section3.lisp
    backends: [interpreter, jvm, wasm]
    note: notebook 2.3 -- embedding shapes, the positional encoding and its dot products, the feed-forward block, the FFN-vs-SkipConnection identity training, LayerNorm
    expect:
      file: .expected/llm-section3.txt
  - path: llm-from-scratch/chapter02/section4.lisp
    backends: [interpreter, jvm, wasm, wasm-component]
    note: notebook 2.4 -- cross entropy between discretised Gaussians, against a uniform, the one-hot case and a probability target
    expect:
      file: .expected/llm-section4.txt
  - path: llm-from-scratch/chapter02/section5.lisp
    backends: [interpreter, jvm, wasm]
    simd: true
    note: notebook 2.5 -- the masks, the vocabulary, and a Transformer trained on the in-file 12-pair ja->en corpus then decoded greedily. Under --simd for the margin - the interpreter leg is 2:06 scalar against the 240 s cap and 1:19 with the flag, and every leg prints the same bytes either way (verified on all three)
    expect:
      file: .expected/llm-section5.txt

  # --- llm-from-scratch: the book's GPT chapter, ported (README.md) -----------
  # Same determinism rule as chapter 2 above, and it now also covers a SAMPLED
  # text: torch:multinomial draws from the same seeded generator, so the two
  # generated passages are byte-identical on every backend. train-gpt-soseki is
  # the slow one -- its corpus is the public domain opening of 『吾輩は猫である』,
  # inlined, since nothing here downloads. It and chapter02/section5 are the only
  # two llm-from-scratch entries under --simd; the six shape/notebook entries
  # around them stay scalar, so both paths keep their coverage.
  - path: llm-from-scratch/gpt/shapes.lisp
    backends: [interpreter, jvm, wasm, wasm-component]
    note: the gpt package's own shape check -- shapes, counts and two exact predicates (the causal mask really blinds a position to its future; :top-k 1 makes sampling deterministic), so nothing float-sensitive is printed
    expect:
      file: .expected/llm-gpt-shapes.txt
  - path: llm-from-scratch/chapter03/section2.lisp
    backends: [interpreter, jvm, wasm, wasm-component]
    note: section 3.2 -- word-level tokenization and its normalizer, the special tokens, the two embedding tables, and a BPE learner whose 100 merges come out in the book's exact order
    expect:
      file: .expected/llm-section3-2.txt
  - path: llm-from-scratch/chapter03/train-gpt-soseki.lisp
    backends: [interpreter, jvm, wasm, wasm-component]
    simd: true
    note: the train_gpt_soseki notebook -- a character-level GPT trained by AdamW over two parameter groups on the inlined 漱石 corpus, then sampled with temperature and top-k. Under --simd for the margin - the interpreter leg is 2:43 scalar against the 240 s cap and 1:39 with the flag, and every leg prints the same bytes either way (verified on all four). Most of what is left is the un-intercepted rank-3 matmul, not the flag's fault
    expect:
      file: .expected/llm-train-gpt-soseki.txt

  # --- console/--no-gc: compiled to a plain MVP module, invoked by a host -----
  - path: console/mandelbrot-nogc.lisp
    backends: [no-gc, wasm-component]
    note: implements mandelbrot_component.wit via rontolisp:wit-export -- the :string-RESULT counterpart of count-vowels' :string parameter, on the console rather than in a page. The no-gc leg pins that the world lowers to the byte-identical core module the memory-reading host has always called; wasm-component pins the wasm-GC lowering of the same world, at exactly the 7-parameter callable limit. Host-invoked (wasmtime run --invoke 'mandelbrot(-2.5, 1.0, -1.2, 1.2, 70, 30, 30)'), so the harness only compiles
  - path: count-vowels/count-vowels.lisp
    backends: [no-gc, wasm-component]
    note: implements count_vowels_component.wit via rontolisp:wit-export -- the world is the export list, so this also pins the contract check on both component paths (the README builds it --no-gc --component; the wasm-component backend here exercises the wasm-GC lowering of the same world). Host-invoked (wasmtime run --invoke 'count-vowels("Hello, World!")' -> 3), so the harness only compiles

  # --- browser/minesweeper: the rules of a GUI example, checked head-less -----
  - path: browser/minesweeper/minesweeper-core-test.lisp
    backends: [interpreter, jvm, wasm]
    systemPath: [src/test/resources/rove, src/test/resources/dissect, src/test/resources/cl-ppcre]
    note: >-
      the two minesweeper front-ends (browser/WASM and Swing) open a window and
      are excluded like every other GUI example, but the rules they share --
      minesweeper-core.lisp, a pure state machine that touches neither screen nor
      entropy -- are not GUI at all. This file loads it and asserts them with
      rove (geometry, the flood fill, the win and loss transitions, flagging, and
      the placement rule that keeps the first click safe), ending with
      (uiop:quit (if (run-suite *package*) 0 1)). Nothing verified that core
      before. The needles are rove's summary only; the per-assertion lines print
      forms, which spell symbols package-qualified until .todo/391 lands.
    expect:
      contains:
        - "✓ 7 tests completed"
        - "All 7 tests passed."

  # --- browser/wit-component: a component loaded by a page, with no glue -------
  - path: browser/wit-component/fractal.lisp
    backends: [no-gc, wasm-component]
    note: implements wit/fractal.wit via rontolisp:wit-export. The page loads the --no-gc --component build through jco (browsers cannot be driven headless here, so only the compile legs run); wasm-component additionally pins that the same world lowers on wasm-GC, which is why every export stays within the 7-parameter callable limit. Host-invoked (wasmtime run --invoke 'escape-time(0.0, 0.0, 50)' -> 50), so the harness only compiles

  # --- wit/world: a handed-over WIT world, scaffolded and implemented ---------
  - path: wit/world/analyzer.lisp
    backends: [jvm-compile, wasm-component]
    note: implements wit/analyzer.wit (package example:analyzer, world analyzer) via rontolisp:wit-export, its skeleton generated by --scaffold-wit. wasm-component pins the wasm-GC lowering of the whole world -- s32/string/bool results plus an async func that prints; jvm-compile pins that the same contract check runs on a backend exporting nothing. Host-invoked (wasmtime run --invoke 'word-count("...")'), so the harness only compiles

  # --- wit/keyvalue: a WIT interface CALLED, with a provider per backend -------
  - path: wit/keyvalue/page-hits.lisp
    backends: [interpreter, jvm, wasm-component]
    note: >-
      binds wit/keyvalue.wit (the REAL wasi:keyvalue/store@0.2.0-draft, vendored
      verbatim) with rontolisp:wit-import, and the provider behind it differs per
      backend with no change to the program. rontolisp ships NO provider for any
      interface, so the two JVM-family legs run against a store written in Lisp
      beside the program: the interpreter leg requires memory-store.lisp (a
      portable hash-table store), and the jvm leg runs the SAME source with
      java-store.lisp required on top under #+rontolisp-jvm, whose
      rontolisp:wit-provide REPLACES it with a java.util.LinkedHashMap and prints
      a ";; [java store]" trace -- hence contains, not equals (the program's own
      lines are identical on both). The wasm-component leg pins the canonical-ABI
      import lowering (canon lower: result / option / variant / record /
      list / list / bool / resource handles all marshalled); the
      harness only compiles it, because RUNNING it needs a host that provides
      wasi:keyvalue -- and wasmtime does: `wasmtime run -W gc=y -W exceptions=y
      -S keyvalue=y prog.wasm` prints
      exactly the interpreter's lines, from a store that has never heard of this
      program. No Preview 1 leg: a core import carries flat values only, and every
      function of this interface returns a result. No --no-gc leg: its MVP module
      imports nothing
    expect:
      contains:
        - "hits per page:"
        - "  /index = 3"
        - "  /pricing = 2"
        - "/docs exists?      yes"
        - "/docs exists now?  no"
        - 'keys:              ("/index" "/pricing")'
        - "bad store:         NO-SUCH-STORE"
        - 'seeded:            ("/a" "/b")'

  - path: wit/keyvalue/page-hits-server.lisp
    backends: [jvm-compile, wasm-component]
    note: >-
      the same counter behind rontolisp:http-handler -- the pairing a served
      component NEEDS, since a wasi:http host instantiates it afresh per request
      and a global hash table would read back empty every time. wasm-component is
      the leg that matters: it pins that a serve-mode component
      (wasi:http/incoming-handler, its HTTP glue the serve.lisp library over
      wit-imported wasi:http, its own index bookkeeping) ALSO carries an ADDITIONAL
      user WIT instance import alongside its fixed wasi:http surface; jvm-compile
      pins that the same source
      compiles where the store is a Lisp provider instead. Compile-only, like
      every blocking server here: running it means `wasmtime serve -W gc=y -W
      exceptions=y -S keyvalue=y server.wasm` (whose in-memory kv store is rebuilt
      per instance, so the tally restarts each request) or `wash dev` on wasmCloud,
      where an out-of-process provider holds the counts and they accumulate

  # --- wit/lisp-calls-rust & wit/rust-calls-lisp: cross-language wac compose ---
  - path: wit/lisp-calls-rust/app.lisp
    backends: [interpreter, jvm, wasm-component]
    note: >-
      the Lisp half of the "Lisp calls Rust" example (see
      wit/lisp-calls-rust/README.md and build.sh): a command that imports the
      interface example:textkit/casing with rontolisp:wit-import and calls shout.
      It also binds a two-line rontolisp:wit-provide, so the interpreter and jvm
      legs RUN standalone (the Lisp provider answers) and their output is checked.
      wasm-component pins that the import lowers (canon lower, a plain
      string->string func); the harness only compiles that leg, because RUNNING it
      needs the Rust component (scaffolded with cargo-component) plug-composed in
      with `wac`, off the test path -- and there the wit-provide is inert, so the
      Rust component answers with the same output.
    expect:
      equals: |
        hello world  ->  HELLO WORLD!
        component model  ->  COMPONENT MODEL!
        rust and lisp  ->  RUST AND LISP!
  - path: wit/rust-calls-lisp/counter.lisp
    backends: [wasm-component]
    note: >-
      the Lisp half of the "Rust calls Lisp" example: a reactor that exports the
      plain function vowel-count via rontolisp:wit-export (world counter), exactly
      the shape the Rust describer imports. wasm-component pins the wasm-GC lowering
      of that world; the Rust component calls this export through `wac` composition
      (`wasmtime run --invoke 'describe("hello world")' vowels.wasm` -> a sentence
      with the Lisp-computed vowel count), off the test path
  - path: wit/pipeline/app.lisp
    backends: [wasm-component]
    note: >-
      the Lisp command of the THREE-component pipeline (wit/pipeline/README.md,
      composition.wac): imports example:pipeline/shout with rontolisp:wit-import.
      wasm-component pins the import lowering; RUNNING it needs the Rust shouter and
      the Lisp counter wired in with a single `wac compose` (a chain plug cannot
      do in one step), off the test path. Assembled, `wasmtime run -W gc=y
      pipeline.wasm` prints "hello world  ->  HELLO WORLD!!!"
  - path: wit/pipeline/stats.lisp
    backends: [wasm-component]
    note: >-
      the Lisp counter of the same pipeline: exports the plain function vowel-count
      via rontolisp:wit-export (world statistician), which the Rust shouter imports.
      wasm-component pins the wasm-GC lowering; the Rust shouter calls this export,
      and the Lisp app calls the shouter, all wired by wit/pipeline/composition.wac
      through `wac compose`, off the test path

  # --- net: sockets / HTTP servers & clients (blocking -> compile-only) -------
  - path: net/echo-server.lisp
    backends: [jvm-compile, wasm-component]
  - path: net/echo-client.lisp
    backends: [jvm-compile, wasm-component]
  - path: net/http-hello.lisp
    backends: [jvm-compile, wasm-component]
  - path: net/http-handler.lisp
    backends: [jvm-compile, wasm-component]
  - path: net/http-handler-cl-who.lisp
    backends: [jvm-compile, wasm-component]
    systemPath: src/test/resources/cl-who
  - path: net/httpbin.lisp
    backends: [jvm-compile, wasm-component]
  - path: net/httpbin-clack.lisp
    backends: [jvm-compile, wasm-component]
    note: >-
      plain Clack, and the ONE source for every host:
      cloudflare-workers/httpbin-clack-one-source compiles THIS file (not a
      copy) as its deployed Worker, because :server :rontolisp takes the reactor
      shape under --no-wasi. What this pins is that the file the Worker ships is
      still a portable Clack application -- an application FUNCTION plus a
      middleware applied directly, which is also the spelling that keeps
      clackup's pathname branch provably dead -- on the two backends where
      clackup can actually bind (a blocking server, hence compile-only; Preview
      1 has no incoming TCP). The --no-wasi leg has no backend token here;
      RontoLispCliTest pins the synthesized handle-request export.
  - path: net/httpbin-tiny-routes.lisp
    backends: [jvm-compile, wasm-component]
    note: >-
      the same endpoints composed with tiny-routes instead: route macros, a
      /status/:code template, the decline protocol, and `pipe` threading the
      table through wrap-request-body / wrap-query-parameters /
      wrap-response-content-type -- so what this pins beyond the clack case is
      that the library's own middleware stack compiles and that the lite
      system's ql:quickload spelling resolves on both compile paths. Blocking
      server, hence compile-only.
  - path: net/httpbin-ningle.lisp
    backends: [jvm-compile, wasm-component]
    note: >-
      the same endpoints a third way, on a ningle application OBJECT: routes
      assigned in a loop with an :ANY fallback per path for the 405, a
      controller that returns a STRING and mutates ningle:*response*, a
      `:regexp t` /status rule whose :captures bind, and an overridden
      ningle:not-found METHOD for the 404. What it pins beyond the tiny-routes
      case is the other routing model on the two backends where clackup can bind
      -- a `defmethod` on a CLOS library generic,
      requirement closures compiled when the route is defined, and the whole
      lack-request chain (http-body / fast-http / smart-buffer) compiling, which
      ningle reads every request through.
  - path: net/httpbin-clos.lisp
    backends: [jvm-compile, wasm-component]
  - path: net/httpbin-jzon.lisp
    backends: [jvm-compile, wasm-component]
  - path: net/magic-8-ball.lisp
    backends: [jvm-compile, wasm-component]
  - path: net/dog-fetcher.lisp
    backends: [jvm-compile, wasm-component]
  - path: net/linalg-api.lisp
    backends: [jvm-compile, wasm-component]
  - path: net/kv-server.lisp
    backends: [jvm-compile, wasm-component]
  - path: net/https-hello.lisp
    backends: [jvm-compile]
    note: TLS is a compile error on WASM -- JVM/interpreter only
  - path: net/kv-server-tls.lisp
    backends: [jvm-compile]
    note: TLS is a compile error on WASM -- JVM/interpreter only

  # --- cloudflare-workers: a wasm-export handler driven by a JavaScript host ---
  - path: cloudflare-workers/hello/worker.lisp
    backends: [no-gc, wasm-component]
    note: >-
      the simplest Worker, and the one with no library in it: three
      wasm-export'ed functions a JavaScript host calls directly. Host-invoked,
      so the harness only compiles it. no-gc pins the plain MVP module the
      Worker imports; wasm-component pins that the same three exports lower
      through the canonical ABI, which is what
      cloudflare-workers/httpbin-component transpiles with jco.
  - path: cloudflare-workers/hello-clack/check.lisp
    backends: [interpreter, jvm, wasm]
    note: >-
      the smallest Clack application on a Worker, and the counterpart of hello/:
      worker.lisp is THREE forms -- quickload, the application defun, clackup
      with :server :reactor -- and contains no Worker-specific code at all,
      because the handler backend stores the app and the compiler synthesizes
      the exported entry point. The demo drives it through that backend's
      `dispatch`, the same function the synthesized wasm-export calls, so what
      this pins is the clackup route itself on every backend the compiler has.
      Checked with contains, not equals: the demo prints clackup's own start-up
      banner, and the JSON key ORDER follows hash-table iteration order, which
      differs per backend.
    expect:
      contains:
        - 'Hello from Clack on Cloudflare Workers'
        - 'GET /anything'
        - '"status":200'
  - path: cloudflare-workers/hello-tiny-routes/check.lisp
    backends: [interpreter, jvm, wasm]
    note: >-
      the same greeting COMPOSED instead of written: three routes through
      tiny-routes/lite (loaded by ql:quickload, so the lite system resolves on
      the compile paths too), threaded through wrap-response-content-type by
      `pipe` and handed to clackup with :server :reactor. What it pins beyond
      hello-clack's case is the routing and the middleware seam on every backend
      the compiler has -- the "/hello/:name" path template binding a parameter
      through the ppcre-free matcher, and a path no route claims declining into
      the catch-all 404. Checked with contains, not equals: the demo prints
      clackup's own start-up banner, and JSON key order differs per backend.
    expect:
      contains:
        - 'Hello from tiny-routes on Cloudflare Workers'
        - 'Hello, rontolisp!'
        - 'no route for /anything'
        - '"status":404'
  - path: cloudflare-workers/hello-ningle/check.lisp
    backends: [interpreter, jvm, wasm]
    note: >-
      the same greeting a third way: ningle, where the application is a CLOS
      OBJECT and each route is a `setf` on it. What it pins beyond
      hello-tiny-routes' case is ningle's own model on every backend the
      compiler has -- a bare STRING as a controller (ningle answers it as the
      body), the "/hello/:name" template binding a parameter through myway's
      ppcre scanner, and the 404 coming from an overridden ningle:not-found
      METHOD, which is a defmethod on a library generic from the application's
      package. The --no-wasi Worker build itself is NOT covered here -- it needs
      node, not a rontolisp backend -- but the reactor path this drives is the
      same one. Checked with contains, not equals: the demo prints clackup's own
      start-up banner, and JSON key order differs per backend.
    expect:
      contains:
        - 'Hello from ningle on Cloudflare Workers'
        - 'Hello, rontolisp!'
        - 'no route for /anything'
        - '"status":404'
  - path: cloudflare-workers/httpbin/check.lisp
    backends: [interpreter, jvm, wasm]
    systemPath: [src/test/resources/rove, src/test/resources/dissect, src/test/resources/cl-ppcre]
    note: >-
      loads worker.lisp -- the LIBRARY-FREE flavour: plain defuns, and thirty
      lines of reactor adapter written out under them where the other three call
      clack:clackup -- and drives it over the requests the README curls, then
      CHECKS ITSELF with rove: every needle this entry used to spell is now an
      assertion over the PARSED reply, and the file ends with
      (uiop:quit (if (run-suite *package*) 0 1)), so a drifted answer exits
      non-zero instead of quietly no longer matching. The adapter calls the same
      rontolisp::%http-make-env / %http-normalize-response entry points the
      handler backends do, so what it pins is that a hand-written adapter builds
      the same environment on every backend: the "?" split and the
      percent-decoding of the raw target (/%67et answers "path":"/get"), the
      content-length the buffered :raw-body needs, and the response header ALIST
      crossing as a JSON array of pairs rather than an object. The Worker itself
      calls the wasm-export'ed handle-request from JavaScript and is built by the
      directory's build.sh (--no-wasi --optimize, a shape no backend token here
      covers). The needles left here are rove's summary only -- the exchange is
      still printed, but its key ORDER inside a JSON object follows hash-table
      iteration order and differs per backend, and the per-assertion lines print
      forms, which spell symbols package-qualified until .todo/391 lands.
    expect:
      contains:
        - "✓ 6 tests completed"
        - "All 6 tests passed."
  - path: cloudflare-workers/httpbin-clack/check.lisp
    backends: [interpreter, jvm, wasm]
    note: >-
      loads worker.lisp -- the same endpoints as plain Clack, over the same
      probes as the httpbin case above, so the pair runs one set of requests
      through the built-in handler backend and through a hand-written copy of
      what it does. A divergence shows up as the two cases disagreeing, which is
      what pins the backend's `dispatch` (the very function the synthesized
      handle-request export calls) on every backend the compiler has, rather
      than only on the WASM one that has an export. It also pins Clack's other
      composition unit -- a MIDDLEWARE, a function from application to
      application, is what sets the content-type header here -- and, on the
      unparseable-body probe, that lack's default backtrace middleware only
      PRINTS (to *error-output*, a sink on the Worker) while the 200 with
      "json":null still comes back. Checked with contains, not equals: the key
      ORDER inside a JSON object follows hash-table iteration order, which
      differs per backend.
    expect:
      contains:
        - '\"path\":\"/get\"'
        - '\"json\":{\"name\":\"rontolisp\"}'
        - '\"json\":null'
        - '\"allowed\":\"POST\"'
        - '"status":404'
        - '[["content-type","application/json"]]'
  - path: cloudflare-workers/httpbin-tiny-routes/check.lisp
    backends: [interpreter, jvm, wasm]
    note: >-
      loads worker.lisp -- the same endpoints written on the REAL tiny-routes
      API, loaded as "tiny-routes/lite": the opt-in system whose ppcre-free
      path-template matcher keeps cl-ppcre out of the module (.kb/asdf.md; the
      corpus pinning lite == full is TinyRoutesLiteE2eTest). What this pins
      beyond httpbin-clack's case is the routed Worker end to end on every
      backend the compiler has: a method-specific route (define-get,
      define-post, ... and wrap-request-matches-method for the PATCH one)
      DECLINING on the wrong method so the single catch-all answers httpbin's
      own 405, the /status/:code path template binding a parameter through the
      lite matcher, a non-numeric :code declining into the catch-all's 404
      instead, the library's own middleware stack under `pipe` supplying the
      request body / query parameters / content-type header the handlers never
      touch, and the ql:quickload spelling of the lite system resolving on the
      compile paths. Checked with contains, not equals: JSON key order follows
      hash-table iteration order, which differs per backend.
    expect:
      contains:
        - '\"path\":\"/get\"'
        - '\"json\":{\"name\":\"rontolisp\"}'
        - '\"allowed\":\"POST\"'
        - '"status":418'
        - '\"path\":\"/status/teapot\"'
        - '"status":404'
        - '[["content-type","text/plain; charset=utf-8"]]'
  - path: cloudflare-workers/httpbin-ningle/check.lisp
    backends: [interpreter, jvm, wasm]
    note: >-
      loads worker.lisp -- the same endpoints in ningle's model: what this pins
      is the four ningle mechanisms the file is written on, each of which the
      tiny-routes case cannot reach.
      Routes assigned in a LOOP (`setf` on the application object, so the five
      echo endpoints and their five :ANY fallbacks are one dolist), a controller
      that returns a STRING and says the rest by mutating ningle:*response*
      (status and headers, never the Clack triple), a request that arrives
      DECODED so one controller serves every method -- the query string as
      `args` and the parsed body as `form`, both the lack-request chain's work,
      for a JSON body and a form-encoded one alike -- and DECLINING as the only
      thing a ningle route can do, which is why the :ANY rule assigned after
      each method rule answers the 405, why /status is a `:regexp t` rule whose
      :captures bind (a non-numeric code matches nothing), and why the 404 comes
      from an overridden ningle:not-found METHOD. Checked with contains, not
      equals: JSON key order follows hash-table iteration order, which differs
      per backend.
    expect:
      contains:
        - '\"path\":\"/get\"'
        # One key per needle: a needle spanning two keys of one object would
        # depend on their order.
        - '\"a\":\"1\"'
        - '\"b\":\"two\"'
        - '\"form\":{\"name\":\"rontolisp\"}'
        - '\"form\":{\"name\":\"lisp\"}'
        - '\"allowed\":\"POST\"'
        - '\"path\":\"/anything\"'
        - '"status":418'
        - '\"path\":\"/status/teapot\"'
        - '"status":404'
        - '[["content-type","text/plain; charset=utf-8"]]'

  # --- size-report/: the two programs the cross-language size table measures.
  # They live outside examples/ because they exist to be measured rather than
  # read; size-report/measure.sh reports them and this manifest pins that they
  # still compute the right thing on every backend.
  - path: ../size-report/programs/hello_world/hello_world.lisp
    backends: [interpreter, jvm, wasm, wasm-component]
    note: >-
      the size-comparison floor. What matters about it is the size of the compiled
      module, but the number only means something if the module still prints, so
      the run legs pin the output and the wasm-component leg pins that the WASI 0.3
      build of it compiles too.
    expect:
      equals: |
        Hello, World!
  - path: ../size-report/programs/pi_approx/pi_approx.lisp
    backends: [interpreter, jvm, wasm, wasm-component]
    note: >-
      a million Leibniz terms in f64 and one ~,15F. The equals check is the point:
      the 15 decimal places must be identical on every backend, or the size table
      would be comparing modules that do not agree on what they compute.
    expect:
      equals: |
        pi = 3.141591653589774
  - path: ../size-report/programs/hello_world/hello_world-nogc.lisp
    backends: [no-gc]
    note: >-
      the --no-gc floor: a plain MVP core module that prints through fd_write with
      no wasm-GC at all. Host-invoked (wasmtime run --invoke say-hello), so the
      harness only compiles it.
  - path: ../size-report/programs/pi_approx/pi_approx-nogc.lisp
    backends: [no-gc]
    note: >-
      the same Leibniz loop as an MVP core module -- the smallest artifact
      measured. It prints with princ rather than ~,15F, so the gap to the wasm-GC
      build is what the GC value model and the fixed-decimal rendering cost, not
      the loop. Host-invoked (wasmtime run --invoke
      approx-pi), so the harness only compiles it.


---

# FILE: references/examples/jvm/java-interop.lisp

;; Building a Swing UI directly through the java interop package -- no bespoke
;; wrapper per widget. The full Swing API is reachable by reflection.

(defvar *frame* (java:new "javax.swing.JFrame" "java interop PoC"))
(defvar *label* (java:new "javax.swing.JLabel" "click count: 0"))
(defvar *button* (java:new "javax.swing.JButton" "Increment"))
(defvar *panel*
  (java:new "javax.swing.JPanel" (java:new "java.awt.BorderLayout" 12 12)))

(defvar *count* 0)

;; The ActionListener is a rontolisp lambda turned into a java interface via a
;; dynamic proxy. It is invoked as (lambda method-name event...) on every click.
(java:call *button* "addActionListener"
           (java:proxy "java.awt.event.ActionListener"
                       (lambda (method event)
                         (setq *count* (+ *count* 1))
                         (java:call *label* "setText"
                                    (concatenate 'string "click count: "
                                                 (princ-to-string *count*))))))

;; Center constants etc. are just static fields.
(java:call *panel* "add" *label* (java:field "java.awt.BorderLayout" "CENTER"))
(java:call *panel* "add" *button* (java:field "java.awt.BorderLayout" "SOUTH"))

(java:call *frame* "setContentPane" *panel*)
(java:call *frame* "setDefaultCloseOperation"
           (java:field "javax.swing.WindowConstants" "DISPOSE_ON_CLOSE"))
(java:call *frame* "setSize" 360 180)
(java:call *frame* "setLocationRelativeTo" nil)
(java:call *frame* "setVisible" t)

(print "java-interop window is open; click Increment")


---

# FILE: references/examples/jvm/life-gui.lisp

;;;; Conway's Game of Life -- Swing front-end.
;;;;
;;;; Loads the rendering-free core (life-core.lisp) and the reusable grid view
;;;; (the `swing` package, swing.lisp), then animates successive generations:
;;;; each timer tick advances
;;;; the world one step, repaints every cell, and updates the status line. A small
;;;; toroidal Life world decays to a stable "ash" of still lifes and blinkers
;;;; after a couple hundred generations, so once it has run long enough this demo
;;;; reseeds with the classic patterns plus a fresh random soup to stay lively.
;;;; Close the window to stop. JVM only (Swing), and needs a display.
;;;;
;;;; Run from anywhere (the load and the require resolve relative to this file;
;;;; the compile path inlines them):
;;;;   java -jar target/rontolisp-0.1.0-SNAPSHOT-exec.jar examples/jvm/life-gui.lisp
;;;;   java -jar ...-exec.jar examples/jvm/life-gui.lisp -o Life.class && java Life

(load "../console/life-core.lisp")
(require :swing "swing.lisp")

(defparameter *color-alive* (swing:rgb 90 200 250))
(defparameter *color-dead* (swing:rgb 28 30 36))

(defparameter *win* (swing:grid-window "rontolisp life" *rows* *cols* 18))

;; The current world and generation counter, advanced by the animation tick.
(defparameter *g* (life-seed))
(defparameter *gen* 0)

;; Repaint every cell from a grid: alive cells cyan, dead cells dark.
(defun render-life (grid)
  (let ((r 0))
    (while (< r *rows*)
      (let ((c 0))
        (while (< c *cols*)
          (swing:paint *win* r c
                       (if (= (aref grid r c) 1) *color-alive* *color-dead*))
          (setq c (+ c 1))))
      (setq r (+ r 1)))))

;; After this many generations the world has usually settled into ash; reseed.
(defparameter *reseed-at* 160)

;; Sprinkle n random live cells into the grid.
(defun sprinkle (grid n)
  (let ((i 0))
    (while (< i n)
      (setf (aref grid (random *rows*) (random *cols*)) 1)
      (setq i (+ i 1)))))

(render-life *g*)

(swing:animate 120
               (lambda ()
                 (when (>= *gen* *reseed-at*)
                   (setq *g* (life-seed))
                   (sprinkle *g* 140)
                   (setq *gen* 0))
                 (swing:status *win*
                               (concatenate 'string "  generation: "
                                            (princ-to-string *gen*)
                                            "   population: "
                                            (princ-to-string
                                             (population *g* *rows* *cols*))))
                 (render-life *g*)
                 (setq *g* (next-gen *g* *rows* *cols*))
                 (setq *gen* (+ *gen* 1))
                 t))

(print "life window is open; close it to stop the simulation")


---

# FILE: references/examples/jvm/swing.lisp

;;;; swing.lisp -- a tiny reusable Swing helper library for rontolisp examples.
;;;;
;;;; It is written entirely on top of the generic `java:` interop package (no
;;;; bespoke Java class): a grid window is a frame holding a status label on top
;;;; and a GridLayout of one JPanel per cell in the center. Examples reuse it as
;;;; the rendering layer so their core logic stays free of any UI code.
;;;;
;;;; The helpers live in a `swing` package of their own; a program splices the
;;;; library in at compile time (idempotently, thanks to the provide below) with
;;;;
;;;;   (require :swing "swing.lisp")
;;;;
;;;; and calls the qualified names (swing:rgb ...), (swing:grid-window ...), ...
;;;;
;;;; Swing is reachable on the JVM -- interpret the program, or compile it to a
;;;; .class (the WASM backend cannot lower a java object) -- and it needs a
;;;; machine with a display. The
;;;; requiring example resolves "swing.lisp" relative to its own directory, so
;;;; run it from anywhere, e.g.:
;;;;   java -jar target/rontolisp-0.1.0-SNAPSHOT-exec.jar examples/jvm/life-gui.lisp
;;;;
;;;; API:
;;;;   (swing:rgb r g b)                    -> a 0-255 RGB java.awt.Color
;;;;   (swing:grid-window title rows cols size) -> a window handle (shown on screen)
;;;;   (swing:cell win r c)                 -> the JPanel at (row col)
;;;;   (swing:paint win r c color)          -> set that cell's background color
;;;;   (swing:fill win color)               -> paint every cell one color
;;;;   (swing:status win text)              -> set the top status-label text
;;;;   (swing:animate delay step-fn)        -> start a repeating timer; step-fn is
;;;;                                           called with no args every `delay` ms
;;;;                                           and the timer stops when it returns nil
;;;;
;;;; For grids that need text and clicks (e.g. Minesweeper) use the label variant:
;;;;   (swing:label-grid-window title rows cols size) -> a clickable text grid
;;;;   (swing:cell-text win r c text)       -> set a cell's centred text
;;;;   (swing:cell-fg win r c color)        -> set a cell's text colour
;;;;   (swing:on-cell-click win handler)    -> handler is called (row col button)
;;;;                                           on click (button: 1 left, 3 right)

(provide :swing)

(defpackage swing
  (:use cl)
  (:export rgb grid-window label-grid-window cell paint fill status cell-text
           cell-fg on-cell-click animate))

(in-package swing)

;; A window handle is a small symbol-keyed hash table.
(defun rgb (r g b) (java:new "java.awt.Color" r g b))

(defun cell (win r c) (gethash (list r c) (gethash 'cells win)))

(defun paint (win r c color) (java:call (cell win r c) "setBackground" color))

(defun status (win text) (java:call (gethash 'status win) "setText" text))

(defun fill (win color)
  (let ((rows (gethash 'rows win)) (cols (gethash 'cols win)) (r 0))
    (while (< r rows)
      (let ((c 0))
        (while (< c cols)
          (paint win r c color)
          (setq c (+ c 1))))
      (setq r (+ r 1)))))

(defun grid-window (title rows cols size)
  (let ((frame (java:new "javax.swing.JFrame" title))
        (grid
         (java:new "javax.swing.JPanel"
                   (java:new "java.awt.GridLayout" rows cols 1 1)))
        (status (java:new "javax.swing.JLabel" " "))
        (cells (make-hash-table :test 'equal))
        (win (make-hash-table :test 'equal)))
    (java:call grid "setBackground" (rgb 120 120 120))
    ;; One JPanel per cell, in row-major order so GridLayout lays them out
    ;; left-to-right, top-to-bottom; remember each by (row col).
    (let ((r 0))
      (while (< r rows)
        (let ((c 0))
          (while (< c cols)
            (let ((cell (java:new "javax.swing.JPanel")))
              (java:call cell "setPreferredSize"
                         (java:new "java.awt.Dimension" size size))
              (java:call cell "setBackground" (rgb 255 255 255))
              (java:call grid "add" cell)
              (setf (gethash (list r c) cells) cell))
            (setq c (+ c 1))))
        (setq r (+ r 1))))
    (setf (gethash 'frame win) frame)
    (setf (gethash 'grid win) grid)
    (setf (gethash 'status win) status)
    (setf (gethash 'cells win) cells)
    (setf (gethash 'rows win) rows)
    (setf (gethash 'cols win) cols)
    (java:call frame "add" status (java:field "java.awt.BorderLayout" "NORTH"))
    (java:call frame "add" grid (java:field "java.awt.BorderLayout" "CENTER"))
    (java:call frame "setDefaultCloseOperation"
               (java:field "javax.swing.WindowConstants" "EXIT_ON_CLOSE"))
    (java:call frame "pack")
    (java:call frame "setLocationRelativeTo" nil)
    (java:call frame "setVisible" t)
    win))

;; --- clickable, text-capable grid -------------------------------------------
;;
;; grid-window's cells are blank JPanels -- perfect for a painter like Life
;; but they cannot show text or, on their own, tell a left-click from a right one.
;; The three helpers below build a grid whose cells are opaque JLabels instead
;; (JButton backgrounds are ignored by the macOS Aqua look-and-feel, so a label
;; is the portable way to get both a background colour AND centred text). The
;; window handle has the same shape, so cell / paint / status /
;; fill all work on it unchanged.

;; Like grid-window, but every cell is a centred, bold JLabel that can hold
;; text (cell-text) and receive per-cell clicks (on-cell-click).
(defun label-grid-window (title rows cols size)
  (let ((frame (java:new "javax.swing.JFrame" title))
        (grid
         (java:new "javax.swing.JPanel"
                   (java:new "java.awt.GridLayout" rows cols 1 1)))
        (status (java:new "javax.swing.JLabel" " "))
        (cells (make-hash-table :test 'equal))
        (win (make-hash-table :test 'equal))
        (font
         (java:new "java.awt.Font" "SansSerif" 1 (floor (/ (* size 6) 10)))))
    (java:call grid "setBackground" (rgb 120 120 120))
    (let ((r 0))
      (while (< r rows)
        (let ((c 0))
          (while (< c cols)
            (let ((cell (java:new "javax.swing.JLabel" "")))
              (java:call cell "setOpaque" t)
              (java:call cell "setHorizontalAlignment"
                         (java:field "javax.swing.SwingConstants" "CENTER"))
              (java:call cell "setPreferredSize"
                         (java:new "java.awt.Dimension" size size))
              (java:call cell "setFont" font)
              (java:call cell "setBackground" (rgb 255 255 255))
              (java:call grid "add" cell)
              (setf (gethash (list r c) cells) cell))
            (setq c (+ c 1))))
        (setq r (+ r 1))))
    (setf (gethash 'frame win) frame)
    (setf (gethash 'grid win) grid)
    (setf (gethash 'status win) status)
    (setf (gethash 'cells win) cells)
    (setf (gethash 'rows win) rows)
    (setf (gethash 'cols win) cols)
    (java:call frame "add" status (java:field "java.awt.BorderLayout" "NORTH"))
    (java:call frame "add" grid (java:field "java.awt.BorderLayout" "CENTER"))
    (java:call frame "setDefaultCloseOperation"
               (java:field "javax.swing.WindowConstants" "EXIT_ON_CLOSE"))
    (java:call frame "pack")
    (java:call frame "setLocationRelativeTo" nil)
    (java:call frame "setVisible" t)
    win))

;; Set the centred text of a label cell (e.g. a mine count).
(defun cell-text (win r c text) (java:call (cell win r c) "setText" text))

;; Set a label cell's text colour.
(defun cell-fg (win r c color) (java:call (cell win r c) "setForeground" color))

;; Call HANDLER with (row col button) on every click of any cell, where button
;; is the java.awt.event.MouseEvent button code (1 = left, 2 = middle, 3 = right).
;; Each cell gets its own MouseListener, a java:proxy over a lambda that closes
;; over that cell's fixed row/col.
(defun on-cell-click (win handler)
  (let ((rows (gethash 'rows win)) (cols (gethash 'cols win)) (r 0))
    (while (< r rows)
      (let ((c 0))
        (while (< c cols)
          (let ((cell (cell win r c)) (cr r) (cc c))
            (java:call cell "addMouseListener"
                       (java:proxy "java.awt.event.MouseListener"
                                   (lambda (method event)
                                     (when (equal method "mouseClicked")
                                       (funcall handler cr cc
                                        (java:call event "getButton")))))))
          (setq c (+ c 1))))
      (setq r (+ r 1)))))

;; Run step-fn every `delay` ms on the Swing event thread until it returns nil.
;; step-fn is a zero-argument rontolisp function; the javax.swing.Timer's
;; ActionListener is a java:proxy over a lambda that closes over the timer so it
;; can stop itself.
(defun animate (delay step-fn)
  (let ((timer nil))
    (setq timer
          (java:new "javax.swing.Timer" delay
                    (java:proxy "java.awt.event.ActionListener"
                                (lambda (method event)
                                  (unless (funcall step-fn)
                                    (java:call timer "stop"))))))
    (java:call timer "start")
    timer))

(in-package cl-user)


---

# FILE: references/examples/llama2/README.md

# llama2.c in rontolisp

[Andrej Karpathy's llama2.c](https://github.com/karpathy/llama2.c) `run.c`,
ported whole to one Lisp file: the checkpoint loader, the SentencePiece-style
tokenizer with its BPE encoder, the Llama 2 forward pass (RMSNorm, RoPE,
multi-head causal attention over a KV cache, SwiGLU, the classifier head), the
temperature / top-p sampler with run.c's own xorshift generator, and the
generate loop. Given a checkpoint the C program reads, it tells the same
stories -- token for token, at temperature 0 and at any seed.

The 1 MB `stories260K.bin` + `tok512.bin` pair is checked in (from
[karpathy/tinyllamas](https://huggingface.co/karpathy/tinyllamas), MIT). The
model the llama2.c README demos, `stories15M.bin` (60 MB), is one script away:

```bash
./download-stories15M.sh          # stories15M.bin + tokenizer.bin, into this directory
```

## Running

The knobs are run.c's flags, read from the environment (a rontolisp program has
no argv yet):

| variable | run.c flag | default |
| --- | --- | --- |
| `LLAMA2_CHECKPOINT` | the positional checkpoint | `stories15M.bin` |
| `LLAMA2_TOKENIZER` | `-z` | `tokenizer.bin` |
| `LLAMA2_PROMPT` | `-i` | empty |
| `LLAMA2_STEPS` | `-n` | 256 |
| `LLAMA2_TEMPERATURE` | `-t` | 1.0 (0 = greedy) |
| `LLAMA2_TOPP` | `-p` | 0.9 |
| `LLAMA2_SEED` | `-s` | the clock |

From this directory, on all four backends:

```bash
export LLAMA2_PROMPT="Once upon a time" LLAMA2_TEMPERATURE=0

rontolisp llama2.lisp --simd                                    # interpreter
rontolisp llama2.lisp -o Prog.class --simd && java --add-modules jdk.incubator.vector Prog
rontolisp llama2.lisp -o llama2.wasm --simd && \
  wasmtime run -W gc --dir . --env LLAMA2_PROMPT --env LLAMA2_TEMPERATURE llama2.wasm
rontolisp llama2.lisp -o llama2.wasm --simd --component && \
  wasmtime run -W gc --dir . --env LLAMA2_PROMPT --env LLAMA2_TEMPERATURE llama2.wasm
```

Every one of them prints

```
Once upon a time, there was a little girl named Lily. She loved to play outside in the sunshine. One day, she saw a big, red ball in the sky. It was the sun! She thought it was so pretty.
Lily wanted to play with the ball...
```

which is what `./run stories15M.bin -t 0 -i "Once upon a time"` prints -- the
whole 256-token story is byte-identical. The small model runs the same way with
`LLAMA2_CHECKPOINT=stories260K.bin LLAMA2_TOKENIZER=tok512.bin`.

## Why `--simd`

Decoding is one token at a time, so every matrix in the model multiplies a
vector: the whole forward pass is GEMV (`vec:matvec`), 15 million multiply-adds
per token for stories15M, and `--simd` lowers it to CPU vector instructions.
Measured on one core, 256 tokens of stories15M:

| backend | scalar | `--simd` |
| --- | --- | --- |
| JVM | 23 tok/s | 87 tok/s |
| wasm-GC (`wasmtime`) | 0.4 tok/s | 46 tok/s |
| interpreter | ~15 s per token | 15-25 tok/s |
| `run.c -O2` (one thread) | 65 tok/s | |
| Java Vector API port of run.c ([kishida's gist](https://gist.github.com/kishida/05656bfcbe840f269784f7dbbee5928e)) | 100 tok/s | 187 tok/s |

stories15M is 60 MB of weights streamed once per token, so the ceiling is memory
bandwidth (~11 GB/s here = ~190 tok/s), which the Java port reaches. The rontolisp
JVM run spends ~80% of its time in the same kind of GEMV kernel and the rest in the
boxed attention / RoPE loops; the gap to the Java port is that glue plus the
`--simd` kernel's deliberately pinned 128-bit accumulation (one chain per row, so
results agree bit for bit with the WASM `f32x4` kernels on every host).

The interpreter's `--simd` needs the native binary or
`java --add-modules jdk.incubator.vector -jar ...`; without the Vector API it
runs the scalar `vec.lisp` kernels, one interpreted form per multiply-add,
which is fine for stories260K and not for stories15M.

The checkpoint's 15 million little-endian `float32`s load in about 0.2 s on
every backend: `read-sequence` over a packed single-float array reads raw
IEEE-754 elements in bulk, one transfer per weight matrix, so the loader is a
`make-array` and a `read-sequence` per tensor.

[`../ml/tiny-llm.lisp`](../ml/tiny-llm.lisp) is the arithmetic core of this
file with the I/O taken away, and explains the KV-cache layout (keys row-major,
values transposed) that makes both halves of attention a GEMV.


---

# FILE: references/examples/llama2/download-stories15M.sh

#!/bin/sh
# Fetches the checkpoint the llama2.c README demos -- stories15M.bin (60 MB, from
# Andrej Karpathy's tinyllamas on Hugging Face) -- and its tokenizer.bin (from
# the llama2.c repository) into this directory. Both are gitignored; the 1 MB
# stories260K.bin + tok512.bin pair is checked in beside this script.
# Run once: ./download-stories15M.sh
set -e
cd "$(dirname "$0")"
fetch() {
  if [ -f "$1" ]; then
    echo "$1: already present"
  else
    echo "downloading $1 ..."
    curl -fsSL -o "$1" "$2"
  fi
}
fetch stories15M.bin https://huggingface.co/karpathy/tinyllamas/resolve/main/stories15M.bin
fetch tokenizer.bin https://github.com/karpathy/llama2.c/raw/master/tokenizer.bin
echo "done."


---

# FILE: references/examples/llama2/llama2.lisp

;;;; llama2.c in rontolisp: run a Llama 2 model from a llama2.c checkpoint.
;;;;
;;;; This is Andrej Karpathy's run.c (https://github.com/karpathy/llama2.c)
;;;; ported whole: the checkpoint loader, the SentencePiece-style tokenizer with
;;;; its BPE encoder, the transformer forward pass, the temperature / top-p
;;;; sampler and the generate loop. Feed it a checkpoint the C program reads
;;;; and it tells the same stories -- token for token, at temperature 0 and at
;;;; any seed (the sampler is run.c's xorshift, bit for bit). ml/tiny-llm.lisp is
;;;; the arithmetic core of this file with the I/O taken away; this is the whole
;;;; engine.
;;;;
;;;; SETUP
;;;; -----
;;;; stories260K.bin + tok512.bin (the smallest TinyStories model, 1 MB) are
;;;; checked in beside this file. The model the llama2.c README demos,
;;;; stories15M.bin (60 MB) + tokenizer.bin, is one script away:
;;;;
;;;;   ./download-stories15M.sh          # from this directory
;;;;
;;;; RUN IT
;;;; ------
;;;; The knobs are run.c's command-line flags, read from the environment (a
;;;; rontolisp program has no argv yet): LLAMA2_CHECKPOINT (the positional
;;;; checkpoint, default stories15M.bin), LLAMA2_TOKENIZER (-z, default
;;;; tokenizer.bin), LLAMA2_PROMPT (-i), LLAMA2_STEPS (-n, default 256),
;;;; LLAMA2_TEMPERATURE (-t, default 1.0), LLAMA2_TOPP (-p, default 0.9),
;;;; LLAMA2_SEED (-s, default: the clock). From this directory:
;;;;
;;;;   export LLAMA2_PROMPT="Once upon a time" LLAMA2_TEMPERATURE=0
;;;;   rontolisp llama2.lisp --simd                                  # interpreter
;;;;   rontolisp llama2.lisp -o Prog.class --simd && java --add-modules jdk.incubator.vector Prog
;;;;   rontolisp llama2.lisp -o llama2.wasm --simd && \
;;;;     wasmtime run -W gc --dir . --env LLAMA2_PROMPT --env LLAMA2_TEMPERATURE llama2.wasm
;;;;   rontolisp llama2.lisp -o llama2.wasm --simd --component && \
;;;;     wasmtime run -W gc --dir . --env LLAMA2_PROMPT --env LLAMA2_TEMPERATURE llama2.wasm
;;;;
;;;; Temperature 0 is greedy decoding: the story is the same on every run, every
;;;; backend and in the C program (the whole 256-token story of the prompt above
;;;; is byte-identical on all of them). At a temperature above 0 the same
;;;; LLAMA2_SEED picks the same story as `run stories15M.bin -s SEED`.
;;;;
;;;; WHERE THE TIME GOES, AND WHY --simd
;;;; -----------------------------------
;;;; Every matrix multiplies a vector -- decoding is one token at a time -- so
;;;; the whole model is GEMV (`vec:matvec`), 15 million multiply-adds per token
;;;; for stories15M, which `--simd` lowers to CPU vector instructions. What is
;;;; left -- the boxed attention and RoPE loops -- is why the JVM lands at about
;;;; half of a Java Vector API port of run.c (which is purely memory-bound on the
;;;; 60 MB of weights). The KV
;;;; cache is laid out per head as in tiny-llm.lisp: keys row-major (seq-len x
;;;; head-size), values TRANSPOSED (head-size x seq-len), so both halves of
;;;; attention are a GEMV as well. Measured (stories15M, 60 tokens, one core):
;;;;
;;;;   JVM         23 tok/s ->  87 tok/s with --simd
;;;;   wasm-GC    0.4 tok/s ->  46 tok/s with --simd     (run.c -O2: 65 tok/s)
;;;;   interpreter  ~ 15 s/token -> 15-25 tok/s with --simd (native binary, or
;;;;                java --add-modules jdk.incubator.vector -jar ...)
;;;;
;;;; Without --simd the interpreter runs the scalar vec.lisp definitions, one
;;;; interpreted form per multiply-add: fine for stories260K, not for stories15M.
;;;;
;;;; The checkpoint is 15 million little-endian float32s. They are read with
;;;; `read-sequence` into packed single-float arrays -- one bulk transfer per
;;;; weight matrix, ~0.2 s on every backend.

;;; --- knobs (run.c's flags, from the environment) -----------------------------
(defun env-or (name default)
  (let ((v (uiop:getenv name))) (if (and v (> (length v) 0)) v default)))

(defun env-number (name default)
  (let ((v (uiop:getenv name)))
    (if (and v (> (length v) 0)) (read-from-string v) default)))

(defparameter *checkpoint* (env-or "LLAMA2_CHECKPOINT" "stories15M.bin"))
(defparameter *tokenizer* (env-or "LLAMA2_TOKENIZER" "tokenizer.bin"))
(defparameter *prompt* (env-or "LLAMA2_PROMPT" ""))
(defparameter *steps* (env-number "LLAMA2_STEPS" 256))
(defparameter *temperature* (env-number "LLAMA2_TEMPERATURE" 1.0))
(defparameter *topp* (env-number "LLAMA2_TOPP" 0.9))
(defparameter *seed* (env-number "LLAMA2_SEED" (get-universal-time)))

;;; --- little-endian binary reading -------------------------------------------
;;; The checkpoint is raw little-endian int32 / float32, exactly what run.c
;;; mmaps. Floats go straight into packed single-float arrays: `read-sequence`
;;; over a packed float array reads raw IEEE-754 elements in bulk.

(defun read-i32 (s)
  ;; A little-endian signed 32-bit integer.
  (let* ((b0 (read-byte s))
         (b1 (read-byte s))
         (b2 (read-byte s))
         (b3 (read-byte s))
         (u (+ b0 (* b1 256) (* b2 65536) (* (mod b3 128) 16777216))))
    (if (>= b3 128) (- u 2147483648) u)))

(defun read-f32-vector (s n)
  (let ((v (make-array n :element-type 'single-float :initial-element 0.0)))
    (read-sequence v s)
    v))

(defun read-f32-matrix (s rows cols)
  (let ((m
         (make-array (list rows cols)
                     :element-type 'single-float
                     :initial-element 0.0)))
    (read-sequence m s)
    m))

(defun skip-f32 (s n)
  (read-f32-vector s n)
  nil)

;;; --- the checkpoint: config + weights ---------------------------------------
;;; A model is a plist. Per-layer weights are simple vectors indexed by layer.

(defun load-checkpoint (path)
  (with-open-file (s path :element-type '(unsigned-byte 8))
    (let* ((dim (read-i32 s))
           (hidden (read-i32 s))
           (n-layers (read-i32 s))
           (n-heads (read-i32 s))
           (n-kv-heads (read-i32 s))
           (vocab-signed (read-i32 s))
           (seq-len (read-i32 s))
           ;; a negative vocab size means the classifier is NOT shared with the
           ;; token embedding table
           (shared (> vocab-signed 0))
           (vocab (abs vocab-signed))
           (head-size (floor dim n-heads))
           (kv-dim (* head-size n-kv-heads))
           (per-layer
            (lambda (f)
              (let ((v (make-array n-layers)))
                (dotimes (l n-layers v) (setf (aref v l) (funcall f)))))))
      (let* ((emb (read-f32-matrix s vocab dim))
             (rms-att (funcall per-layer (lambda () (read-f32-vector s dim))))
             (wq (funcall per-layer (lambda () (read-f32-matrix s dim dim))))
             (wk (funcall per-layer (lambda () (read-f32-matrix s kv-dim dim))))
             (wv (funcall per-layer (lambda () (read-f32-matrix s kv-dim dim))))
             (wo (funcall per-layer (lambda () (read-f32-matrix s dim dim))))
             (rms-ffn (funcall per-layer (lambda () (read-f32-vector s dim))))
             (w1 (funcall per-layer (lambda () (read-f32-matrix s hidden dim))))
             (w2 (funcall per-layer (lambda () (read-f32-matrix s dim hidden))))
             (w3 (funcall per-layer (lambda () (read-f32-matrix s hidden dim))))
             (rms-final (read-f32-vector s dim)))
        ;; skip what used to be freq_cis_real / freq_cis_imag (RoPE is computed)
        (skip-f32 s (* seq-len head-size))
        (let ((wcls (if shared emb (read-f32-matrix s vocab dim))))
          (list :dim dim
                :hidden hidden
                :n-layers n-layers
                :n-heads n-heads
                :n-kv-heads n-kv-heads
                :vocab vocab
                :seq-len seq-len
                :head-size head-size
                :kv-dim kv-dim
                :emb emb
                :rms-att rms-att
                :wq wq
                :wk wk
                :wv wv
                :wo wo
                :rms-ffn rms-ffn
                :w1 w1
                :w2 w2
                :w3 w3
                :rms-final rms-final
                :wcls wcls))))))

;;; --- the tokenizer ------------------------------------------------------------
;;; tokenizer.bin: int32 max-token-length, then per token float32 score,
;;; int32 length, UTF-8 bytes. Pieces are decoded to strings (a character is a
;;; code point on every backend); byte-fallback tokens are the strings "<0x00>"
;;; .. "<0xFF>" at ids 3..258.

(defun utf8-decode (bytes n)
  ;; The first n bytes of a byte vector as a string (a character is a code point).
  (let ((chars '()) (i 0))
    (loop while (< i n)
          do
            (let* ((b0 (aref bytes i))
                   (len
                    (cond ((< b0 128) 1) ((< b0 224) 2) ((< b0 240) 3) (t 4)))
                   (cp
                    (cond ((= len 1) b0)
                          ((= len 2) (logand b0 31))
                          ((= len 3) (logand b0 15))
                          (t (logand b0 7)))))
              (dotimes (k (- len 1))
                (setq cp (+ (* cp 64) (logand (aref bytes (+ i 1 k)) 63))))
              (push (code-char cp) chars)
              (setq i (+ i len))))
    (coerce (nreverse chars) 'string)))

(defun load-tokenizer (path vocab)
  ;; -> (list pieces scores index) : piece strings, their scores, and a hash
  ;; table string -> id for the encoder. The scores and lengths are read through
  ;; one-element packed vectors: `read-sequence` over a packed buffer reads raw
  ;; little-endian elements, so this is the checkpoint loader's idiom in small.
  (with-open-file (s path :element-type '(unsigned-byte 8))
    (let* ((u32 (make-array 1 :element-type '(unsigned-byte 32)))
           (f32 (make-array 1 :element-type 'single-float :initial-element 0.0))
           (max-len
            (progn
              (read-sequence u32 s)
              (aref u32 0)))
           (buf (make-array max-len :element-type '(unsigned-byte 8)))
           (pieces (make-array vocab))
           (scores
            (make-array vocab :element-type 'single-float :initial-element 0.0))
           (index (make-hash-table :test 'equal)))
      (dotimes (i vocab)
        (read-sequence f32 s)
        (setf (aref scores i) (aref f32 0))
        (read-sequence u32 s)
        (let ((len (aref u32 0)))
          (read-sequence buf s :end len)
          (let ((piece (utf8-decode buf len)))
            (setf (aref pieces i) piece)
            (unless (gethash piece index) (setf (gethash piece index) i)))))
      (list pieces scores index))))

(defun byte-token-p (piece)
  ;; "<0xNN>" -> the byte NN, else nil.
  (and (= (length piece) 6) (char= (char piece 0) #\<)
       (char= (char piece 1) #\0) (char= (char piece 2) #\x)
       (char= (char piece 5) #\>)
       (parse-integer piece :start 3 :end 5 :radix 16)))

(defun decode-piece (tok prev-token pieces)
  ;; run.c's decode(): after BOS the leading space of a piece is dropped, and a
  ;; byte token stands for its raw byte -- printed only when it is printable
  ;; ASCII or whitespace (safe_printf), since a lone byte of a multi-byte
  ;; sequence has no character to show.
  (let* ((piece (aref pieces tok))
         (piece
          (if (and (= prev-token 1) (> (length piece) 0)
                   (char= (char piece 0) #\Space))
              (subseq piece 1)
              piece))
         (b (byte-token-p piece)))
    (cond ((null b) piece)
          ((or (and (>= b 32) (< b 127)) (= b 10) (= b 9) (= b 13))
           (string (code-char b)))
          (t ""))))

(defun encode (text pieces scores index bos eos)
  ;; run.c's encode(): the dummy-prefix space, one token per character (or its
  ;; byte tokens when the character is not in the vocabulary), then repeatedly
  ;; merge the adjacent pair whose concatenation has the highest score.
  (let ((tokens (make-array (+ (* 2 (length text)) 3) :fill-pointer 0)))
    (when bos (vector-push 1 tokens))
    (when (> (length text) 0) (vector-push (gethash " " index) tokens))
    (dotimes (i (length text))
      (let* ((c (string (char text i))) (id (gethash c index)))
        (if id
            (vector-push id tokens)
            ;; byte fallback: the UTF-8 bytes of the character, ids 3..258
            (let ((cp (char-code (char text i))))
              (dolist (b
                       (cond ((< cp 128) (list cp))
                             ((< cp 2048)
                              (list (+ 192 (ash cp -6)) (+ 128 (logand cp 63))))
                             ((< cp 65536)
                              (list (+ 224 (ash cp -12))
                                    (+ 128 (logand (ash cp -6) 63))
                                    (+ 128 (logand cp 63))))
                             (t (list (+ 240 (ash cp -18))
                                      (+ 128 (logand (ash cp -12) 63))
                                      (+ 128 (logand (ash cp -6) 63))
                                      (+ 128 (logand cp 63))))))
                (vector-push (+ b 3) tokens))))))
    ;; merge loop
    (loop
      (let ((best-score -1e10) (best-id nil) (best-idx nil))
        (dotimes (i (- (fill-pointer tokens) 1))
          (let* ((merged
                  (concatenate 'string (aref pieces (aref tokens i))
                               (aref pieces (aref tokens (+ i 1)))))
                 (id (gethash merged index)))
            (when (and id (> (aref scores id) best-score))
              (setq best-score (aref scores id) best-id id best-idx i))))
        (unless best-idx (return))
        (setf (aref tokens best-idx) best-id)
        (let ((n (fill-pointer tokens)))
          (do ((i (+ best-idx 1) (+ i 1)))
              ((>= i (- n 1)))
            (setf (aref tokens i) (aref tokens (+ i 1))))
          (setf (fill-pointer tokens) (- n 1)))))
    (when eos (vector-push 2 tokens))
    (coerce tokens 'list)))

;;; --- the forward pass ---------------------------------------------------------

(defparameter *eps* 0.00001)

(defun make-state (model)
  ;; The KV cache: per layer, per kv-head, keys (seq-len x hs) row-major and
  ;; values (hs x seq-len) transposed -- see the header. Plus the RoPE tables.
  (let* ((n-layers (getf model :n-layers))
         (n-kv (getf model :n-kv-heads))
         (seq-len (getf model :seq-len))
         (hs (getf model :head-size))
         (kc (make-array (list n-layers n-kv)))
         (vt (make-array (list n-layers n-kv)))
         (half (floor hs 2))
         (rope-cos
          (make-array (list seq-len half)
                      :element-type 'single-float
                      :initial-element 0.0))
         (rope-sin
          (make-array (list seq-len half)
                      :element-type 'single-float
                      :initial-element 0.0)))
    (dotimes (l n-layers)
      (dotimes (h n-kv)
        (setf (aref kc l h)
              (linalg:zeros (list seq-len hs) :element-type 'single-float))
        (setf (aref vt l h)
              (linalg:zeros (list hs seq-len) :element-type 'single-float))))
    ;; RoPE: freq_i = 1 / 10000^(2i/hs), angle = pos * freq_i
    (dotimes (pos seq-len)
      (dotimes (i half)
        (let ((angle (* pos (/ 1.0 (expt 10000.0 (/ (* 2.0 i) hs))))))
          (setf (aref rope-cos pos i) (cos angle))
          (setf (aref rope-sin pos i) (sin angle)))))
    (list :kc kc
          :vt vt
          :rope-cos rope-cos
          :rope-sin rope-sin
          :att (vec:zeros seq-len :element-type 'single-float))))

(defun rmsnorm (x g)
  ;; x / rms(x) * g, the sum of squares being one vec:dot
  (vec:mul (vec:scale x (/ 1.0 (sqrt (+ (/ (vec:dot x x) (length x)) *eps*))))
           g))

(defun rope (v n-heads hs pos rope-cos rope-sin)
  ;; Rotate every head's (even, odd) pairs in place.
  (let ((half (floor hs 2)))
    (dotimes (h n-heads)
      (dotimes (i half)
        (let* ((j (+ (* h hs) (* 2 i)))
               (fcr (aref rope-cos pos i))
               (fci (aref rope-sin pos i))
               (v0 (aref v j))
               (v1 (aref v (+ j 1))))
          (setf (aref v j) (- (* v0 fcr) (* v1 fci)))
          (setf (aref v (+ j 1)) (+ (* v0 fci) (* v1 fcr))))))))

(defun attention (model state l q k v pos)
  ;; Multi-head causal attention over the KV cache; returns the concatenated
  ;; head outputs (dim), before the wo projection.
  (let* ((n-heads (getf model :n-heads))
         (n-kv (getf model :n-kv-heads))
         (kv-mul (floor n-heads n-kv))
         (hs (getf model :head-size))
         (dim (getf model :dim))
         (kc (getf state :kc))
         (vt (getf state :vt))
         (att (getf state :att))
         (out (vec:zeros dim :element-type 'single-float))
         (qh (vec:zeros hs :element-type 'single-float))
         (inv-sqrt-hs (/ 1.0 (sqrt hs))))
    ;; append this position's keys and values to the cache
    (dotimes (h n-kv)
      (let ((kch (aref kc l h)) (vth (aref vt l h)) (base (* h hs)))
        (dotimes (i hs)
          (setf (aref kch pos i) (aref k (+ base i)))
          (setf (aref vth i pos) (aref v (+ base i))))))
    (dotimes (h n-heads)
      (let ((kch (aref kc l (floor h kv-mul)))
            (vth (aref vt l (floor h kv-mul)))
            (base (* h hs)))
        (dotimes (i hs) (setf (aref qh i) (aref q (+ base i))))
        ;; every score at once: (K q) / sqrt(hs); positions past pos stay 0
        (let ((scores (vec:matvec kch qh)) (top -1e30) (z 0.0))
          ;; softmax over 0..pos into att (the rest of att is 0 = the causal mask)
          (dotimes (u (+ pos 1))
            (let ((sc (* (aref scores u) inv-sqrt-hs)))
              (setf (aref att u) sc)
              (when (> sc top) (setq top sc))))
          (dotimes (u (+ pos 1))
            (let ((e (exp (- (aref att u) top))))
              (setf (aref att u) e)
              (setq z (+ z e))))
          (dotimes (u (+ pos 1)) (setf (aref att u) (/ (aref att u) z)))
          ;; the weighted sum of the values: one GEMV over the transposed cache
          (let ((oh (vec:matvec vth att)))
            (dotimes (i hs) (setf (aref out (+ base i)) (aref oh i)))))))
    out))

(defun silu (h)
  ;; x * sigmoid(x) over the whole vector: four vec ufuncs instead of one boxed
  ;; funcall per element
  (vec:mul h
           (vec:reciprocal
            (vec:add (vec:ones (length h) :element-type 'single-float)
                     (vec:exp (vec:negative h))))))

(defun forward (model state token pos)
  ;; -> the logits over the vocabulary
  (let* ((n-heads (getf model :n-heads))
         (n-kv (getf model :n-kv-heads))
         (hs (getf model :head-size))
         (rope-cos (getf state :rope-cos))
         (rope-sin (getf state :rope-sin))
         (x (linalg:row (getf model :emb) token)))
    (dotimes (l (getf model :n-layers))
      ;; attention block
      (let* ((xb (rmsnorm x (aref (getf model :rms-att) l)))
             (q (vec:matvec (aref (getf model :wq) l) xb))
             (k (vec:matvec (aref (getf model :wk) l) xb))
             (v (vec:matvec (aref (getf model :wv) l) xb)))
        (rope q n-heads hs pos rope-cos rope-sin)
        (rope k n-kv hs pos rope-cos rope-sin)
        (setq x
              (vec:add x
                       (vec:matvec (aref (getf model :wo) l)
                                   (attention model state l q k v pos)))))
      ;; feed-forward block: w2 (silu(w1 x) * w3 x)
      (let* ((xb (rmsnorm x (aref (getf model :rms-ffn) l)))
             (h1 (vec:matvec (aref (getf model :w1) l) xb))
             (h3 (vec:matvec (aref (getf model :w3) l) xb)))
        (setq x
              (vec:add x
               (vec:matvec (aref (getf model :w2) l) (vec:mul (silu h1) h3))))))
    (vec:matvec (getf model :wcls) (rmsnorm x (getf model :rms-final)))))

;;; --- the sampler ---------------------------------------------------------------
;;; run.c's xorshift64* generator, bit for bit (64-bit integers are exact on
;;; every backend), so a seed picks the same random stream as the C program.

(defvar *rng-state* 0)
(defparameter +mask64+ 18446744073709551615)

(defun random-u32 ()
  (setq *rng-state* (logxor *rng-state* (ash *rng-state* -12)))
  (setq *rng-state* (logand (logxor *rng-state* (ash *rng-state* 25)) +mask64+))
  (setq *rng-state* (logxor *rng-state* (ash *rng-state* -27)))
  (ash (logand (* *rng-state* 2685821657736338717) +mask64+) -32))

(defun random-f32 ()
  ;; [0, 1)
  (/ (ash (random-u32) -8) 16777216.0))

(defun sample-argmax (logits) (linalg:argmax logits))

(defun softmax-into-list (logits temperature)
  ;; -> a list of (probability . id) over the vocabulary
  (let* ((n (length logits))
         (top -1e30)
         (z 0.0)
         (probs
          (make-array n :element-type 'single-float :initial-element 0.0)))
    (dotimes (i n)
      (let ((v (/ (aref logits i) temperature)))
        (setf (aref probs i) v)
        (when (> v top) (setq top v))))
    (dotimes (i n)
      (let ((e (exp (- (aref probs i) top))))
        (setf (aref probs i) e)
        (setq z (+ z e))))
    (let ((out '()))
      (dotimes (i n (nreverse out)) (push (cons (/ (aref probs i) z) i) out)))))

(defun sample-mult (probs coin)
  ;; sample index from the probabilities (they must sum to 1)
  (let ((cdf 0.0))
    (dolist (p probs (car (last probs)))
      (setq cdf (+ cdf (car p)))
      (when (< coin cdf) (return (cdr p))))))

(defun sample-topp (probs topp coin)
  ;; nucleus sampling: the smallest set of tokens whose cumulative probability
  ;; exceeds topp, sampled from. Tokens below (1 - topp) / (n - 1) cannot be
  ;; part of that set, so they are dropped before the sort (run.c's cutoff).
  (let* ((n (length probs))
         (cutoff (/ (- 1.0 topp) (- n 1)))
         (candidates
          (sort (remove-if (lambda (p) (< (car p) cutoff)) probs)
                (lambda (a b) (> (car a) (car b)))))
         (cum 0.0)
         (kept '()))
    (dolist (p candidates)
      (push p kept)
      (setq cum (+ cum (car p)))
      (when (> cum topp) (return)))
    (setq kept (nreverse kept))
    ;; sample from the truncated list
    (let ((r (* coin cum)) (cdf 0.0))
      (dolist (p kept (cdr (car (last kept))))
        (setq cdf (+ cdf (car p)))
        (when (< r cdf) (return (cdr p)))))))

(defun sample (logits)
  (if (= *temperature* 0)
      (sample-argmax logits)
      (let ((probs (softmax-into-list logits *temperature*))
            (coin (random-f32)))
        (if (or (<= *topp* 0) (>= *topp* 1))
            (sample-mult probs coin)
            (sample-topp probs *topp* coin)))))

;;; --- generate ------------------------------------------------------------------

(defun generate (model state tokenizer prompt steps)
  (let* ((pieces (first tokenizer))
         (prompt-tokens
          (encode prompt pieces (second tokenizer) (third tokenizer) t nil))
         (token (first prompt-tokens))
         (rest (rest prompt-tokens))
         (start nil)
         (pos 0))
    (loop while (< pos steps)
          do
            (let* ((logits (forward model state token pos))
                   (next (if rest (pop rest) (sample logits))))
              (setq pos (+ pos 1))
              ;; the BOS token delimits sequences: stop on it
              (when (= next 1) (return))
              (write-string (decode-piece next token pieces))
              (finish-output)
              (setq token next)
              (unless start (setq start (get-internal-real-time)))))
    (terpri)
    (when (and start (> pos 1))
      (format *error-output* "achieved tok/s: ~,2f~%"
       (/ (* (- pos 1) 1000.0) (max 1 (- (get-internal-real-time) start)))))))

;;; --- main ------------------------------------------------------------------------

(let* ((t0 (get-internal-real-time))
       (model (load-checkpoint *checkpoint*))
       (t1 (get-internal-real-time)))
  (when (or (<= *steps* 0) (> *steps* (getf model :seq-len)))
    (setq *steps* (getf model :seq-len)))
  (let ((tokenizer (load-tokenizer *tokenizer* (getf model :vocab)))
        (state (make-state model)))
    (format *error-output*
            "loaded ~a: dim=~a hidden=~a layers=~a heads=~a kv-heads=~a vocab=~a seq-len=~a in ~a ms (tokenizer + kv cache ~a ms)~%"
            *checkpoint* (getf model :dim) (getf model :hidden)
            (getf model :n-layers) (getf model :n-heads)
            (getf model :n-kv-heads) (getf model :vocab) (getf model :seq-len)
            (- t1 t0) (- (get-internal-real-time) t1))
    (setq *rng-state* (logand *seed* +mask64+))
    (generate model state tokenizer *prompt* *steps*)))


---

# FILE: references/examples/llm-from-scratch/README.md

# LLM from Scratch — the Transformer and GPT chapters, in rontolisp

A rontolisp port of chapters 2 and 3 of [**『作ってわかる大規模言語モデルの仕組み』**
(Elith Inc., Nikkei BP, 2026) — its sample
repository](https://github.com/elith-co-jp/book-llm-from-scratch): the reusable
`llm_from_scratch/transformer/` and `llm_from_scratch/gpt/` packages, the
chapter 2 notebooks (sections 2.2 - 2.5) and the chapter 3 ones (section 3.2 and
the 漱石 training notebook). Nothing from that repository is vendored here; this
is a rewrite of its PyTorch code, which maps onto the [`torch`
package](https://github.com/making/rontolisp/blob/develop/examples/doc/en/guides/neural-networks.md) almost line for line, and onto
[`linalg`](https://github.com/making/rontolisp/blob/develop/examples/doc/en/guides/linear-algebra.md) for the array math underneath
it.

Everything here runs on all four backends; nothing is downloaded and nothing is
vendored. The plots are the one thing that does not port — each notebook figure
becomes the numbers it was drawn from, which is also what makes the example
testable.

## What maps to what

| book | here |
| --- | --- |
| `transformer/attention.py` | [`transformer/attention.lisp`](transformer/attention.lisp) |
| `transformer/utils.py` | [`transformer/utils.lisp`](transformer/utils.lisp) |
| `transformer/transformer.py` | [`transformer/transformer.lisp`](transformer/transformer.lisp) |
| its `__main__` shape check | [`transformer/shapes.lisp`](transformer/shapes.lisp) |
| `notebooks/chapter02/section2.ipynb` | [`chapter02/section2.lisp`](chapter02/section2.lisp) |
| `notebooks/chapter02/section3.ipynb` | [`chapter02/section3.lisp`](chapter02/section3.lisp) |
| `notebooks/chapter02/section4.ipynb` | [`chapter02/section4.lisp`](chapter02/section4.lisp) |
| `notebooks/chapter02/section5.ipynb` | [`chapter02/section5.lisp`](chapter02/section5.lisp) |
| `gpt/tokenizer.py` | [`gpt/tokenizer.lisp`](gpt/tokenizer.lisp) |
| `gpt/dataset.py` | [`gpt/dataset.lisp`](gpt/dataset.lisp) |
| `gpt/model.py` | [`gpt/model.lisp`](gpt/model.lisp) |
| `gpt/trainer.py` | [`gpt/trainer.lisp`](gpt/trainer.lisp) |
| — (the same idea, for `gpt/`) | [`gpt/shapes.lisp`](gpt/shapes.lisp) |
| `notebooks/chapter03/section03_tokenizer.py` | [`chapter03/section2.lisp`](chapter03/section2.lisp) |
| `notebooks/chapter03/train_gpt_soseki.ipynb` | [`chapter03/train-gpt-soseki.lisp`](chapter03/train-gpt-soseki.lisp) |

And, inside the code:

| PyTorch | rontolisp |
| --- | --- |
| `nn.Module` subclass | `torch:module` + a `forward` defun (no CLOS — see [`.kb/torch.md`](https://github.com/making/rontolisp/blob/develop/examples/.kb/torch.md)) |
| `self.linear = nn.Linear(...)` | a `:linear` entry in the module's **fields plist**, read back with `torch:field` |
| `nn.ModuleList([...])` | a plain LIST in a field; `torch:parameters` recurses into it |
| `nn.ReLU()` inside `nn.Sequential` | `(function torch:relu)` — `torch:forward` applies a bare function too |
| `register_buffer("pe", pe)` | a raw `linalg` array in a field: it is not a parameter, so nothing collects or trains it |
| `torch.bmm(q, k.transpose(1, 2))` | `(torch:matmul q (torch:transpose k '(0 2 1)))` |
| `score.masked_fill(mask, -inf)` | `(torch:masked-fill score mask *neg-infinity*)` |
| `torch.optim.Adam(model.parameters())` | `(torch:adam model)` — an optimizer takes a module directly |
| `torch.nn.utils.rnn.pad_sequence` | `torch:pad-sequence` (always batch-first) |
| `DataLoader(..., shuffle=True)` | `torch:shuffled-batches` — a batch is an ordinary list |
| `@torch.inference_mode` | `torch:no-grad` |
| `nn.GELU()` | `(function torch:gelu)` — exact by default, `:approximate :tanh` for the GPT form |
| `torch.triu(ones(T, T), diagonal=1).bool()` | `(torch:subsequent-mask T)` — already `(1 T T)`, so it broadcasts over the batch |
| `self.apply(self._init_weights)` | a walk over `torch:fields`, dispatching on `torch:module-kind` |
| `AdamW(groups, betas=(0.9, 0.95))` | two `torch:adamw` optimizers over disjoint parameter lists |
| `clip_grad_norm_(params, 1.0)` | `torch:clip-grad-norm` — returns the norm it measured |
| `torch.topk(logits, k)` | `torch:topk` (values, or `:indices t` — one of the pair) |
| `torch.multinomial(probs, 1)` | `torch:multinomial` — the seeded generator, so a SAMPLE reproduces |

## Running

`load` resolves relative to the file doing the loading, so a program runs from
any directory. On all four backends:

```bash
cd chapter02
rontolisp section2.lisp                                     # interpreter
rontolisp section2.lisp -o Prog.class && java Prog          # JVM
rontolisp section2.lisp -o prog.wasm && wasmtime run -W gc prog.wasm
rontolisp section2.lisp -o comp.wasm --component && wasmtime run -W gc=y comp.wasm
```

Chapter 3 is the same, from `chapter03/` (or `gpt/` for `shapes.lisp`).

The output is identical on every backend. Weight initialization, dropout masks,
the epoch shuffle **and the top-k sampling of chapter 3** all draw from the
seeded `linalg` generator, whose arithmetic is integer and therefore
bit-identical everywhere; the printed floats are rounded to a few decimals so
the low-order digits of the WASM `exp`/`log` approximations cannot show
through. That is why the two generated 漱石 passages are the same text on the
interpreter, the JVM and wasm-GC rather than merely the same kind of text.

## `--simd`

Every operation here computes through `linalg`, so
[`--simd`](https://github.com/making/rontolisp/blob/develop/examples/doc/en/guides/simd-acceleration.md) applies with nothing to
change in the source, and it leaves the output byte-identical. The two training
programs are tested with the flag for that reason — it buys back the margin
their interpreter leg needs:

| | scalar | `--simd` |
| --- | --- | --- |
| `chapter02/section5.lisp`, interpreter | 2m06 | 1m19 |
| `chapter03/train-gpt-soseki.lisp`, interpreter | 2m43 | 1m39 |

That is a far smaller win than the flag usually gives — `examples/llama2/` goes
from 11.3 s to 20 ms on the same backend — and the reason is a single gap:
batched (rank >= 3) matrix multiplication, which is what an attention layer
almost entirely *is*, is not in the accelerated set yet. Until it is, a
transformer gains less from `--simd` than a plain MLP does. See
[Accelerating linalg](https://github.com/making/rontolisp/blob/develop/examples/doc/en/guides/simd-acceleration.md#accelerating-linalg).

## The shapes: the book's, and the ones that are tested

The notebook trains a `d_model` = 512, 6-block, 8-head Transformer for 20
epochs over `small_parallel_enja` on a GPU. That is the **documented**
configuration, and this port would run it unchanged. What is actually tested is
a shrunken one, because the point is that the pipeline is right, not that a
laptop can train a translator:

| | book | tested here |
| --- | --- | --- |
| corpus | `small_parallel_enja`, cloned at run time | 8 sentence pairs, in `section5.lisp` itself |
| `d_model` | 512 | 8 |
| blocks / heads | 6 / 8 | 1 / 2 |
| feed-forward width | 512 | 16 |
| section 2.3.3 feed-forward | 512 → 2048 → 512 | 64 → 256 → 64 |
| section 2.3.4 identity training | 10000 × 10, 100 epochs | 64 × 10, 40 epochs |

Every one of those is a `defparameter` at the top of its file: raise them (and
add data) to walk back toward the book's run. The trained model **memorises**
its eight pairs — it reproduces all eight target sentences exactly, and it does
not generalise beyond them. That is what a corpus this small can do, and saying
so is more useful than pretending otherwise.

Chapter 3's notebook trains a `n_embd` = 384, 6-layer, 6-head GPT for 5000
steps over the whole of 『吾輩は猫である』 on a T4, and the same applies:

| | book | tested here |
| --- | --- | --- |
| corpus | the novel, downloaded from 青空文庫 | its opening (448 characters), in `train-gpt-soseki.lisp` itself |
| `block_size` | 256 | 8 |
| `n_embd` | 384 | 8 |
| layers / heads | 6 / 6 | 1 / 2 |
| batch size | 64 | 4 |
| steps | 5000 | 100 |
| generated tokens | 200 | 30 |

100 steps over 448 characters is enough to make the point and no more: the
training loss falls from `4.93` — which is `log(138)`, the uniform guess over
the 138 distinct characters — to about `2.99`, and the samples come out as
recognisable 漱石 fragments (`である。`, `というもの`, `の顔`) strung together
without sentences. Raise the numbers above and the same program keeps going.

## The two places this port deliberately differs from the book

Both are in [`gpt/trainer.lisp`](gpt/trainer.lisp), and both would otherwise
carry a defect across rather than a design:

- **The warmup is applied, not merely printed.** The book's `get_lr` returns
  `base * step / warmup_steps` for the log line, but nothing writes it back —
  its `CosineAnnealingLR` only starts stepping after the warmup, so the
  optimizer runs the whole warmup at the base rate and the schedule it prints is
  not the schedule it trains with. Here `gpt-trainer-lr` is the single answer
  and the loop writes it into both optimizers.
- **`forward(idx, targets=None)` splits in two.** It returns the
  `(logits, loss)` tuple in Python; here `gpt-forward` answers the logits and
  `gpt-loss` the loss, because a forward whose result shape depends on whether
  an optional argument was passed is a tuple only Python's caller destructures
  cheaply.

Two more differences are the port's, not the book's, and are noted where they
happen: `nn.Module.apply` becomes a walk over `torch:fields` dispatching on
`torch:module-kind` (what a layer *is*, rather than a substring of its dotted
parameter name), and the elapsed-time column of the training log is dropped —
it is the one number that cannot come out the same on four backends.

## Sizes

Compiled artifact sizes are measured, not quoted here: see
[`size-report/results/`](https://github.com/making/rontolisp/blob/develop/examples/size-report/results).


---

# FILE: references/examples/llm-from-scratch/chapter02/section2.lisp

;; chapter02/section2.lisp -- notebook sections 2.2.2 and 2.2.3, ported.
;;
;; Attention as pure array math: the book's two numpy functions (softmax and
;; attention) over n unit vectors arranged around a circle, the experiment where
;; one key is made five times longer, and the comparison that motivates the
;; 1/sqrt(d_k) scale. No autograd and no modules -- this section is `linalg'
;; alone, which is also why it is the cheapest program of the port.
;;
;; The notebook's four figures become the numbers they were plotted from.
;;
;;   rontolisp chapter02/section2.lisp

(defun np-softmax (x)
  ;; The book's softmax: exp of the max-subtracted input, normalized along the
  ;; last axis. Subtracting the GLOBAL maximum (np.max) rather than the per-row
  ;; one leaves the result unchanged -- softmax is shift invariant per row.
  (let ((e (linalg:exp (linalg:sub x (linalg:amax x)))))
    (linalg:div e (linalg:sum e :axis -1 :keepdims t))))

(defun np-attention (query key value)
  ;; (output attention-weights), the book's two-value return.
  (let ((weights (np-softmax (linalg:dot query (linalg:transpose key)))))
    (list (linalg:dot weights value) weights)))

(defun np-scaled-attention (query key value)
  ;; The same with the 1/sqrt(d) scale of section 2.2.3.
  (let* ((d (car (last (linalg:shape query))))
         (weights
          (linalg:div (linalg:dot query (linalg:transpose key))
                      (sqrt (* 1.0 d))))
         (normalized (np-softmax weights)))
    (list (linalg:dot normalized value) normalized)))

(defun unit-circle-vectors (n)
  ;; n unit vectors spaced evenly around the circle -- the book's `vectors'.
  (let ((v (linalg:zeros (list n 2))) (theta (/ (* 2.0 pi) n)))
    (dotimes (i n)
      (setf (aref v i 0) (cos (* theta i)))
      (setf (aref v i 1) (sin (* theta i))))
    v))

(defun print-weights (label weights)
  (format t "~a:" label)
  (dotimes (i (linalg:size weights)) (format t " ~,3f" (aref weights i)))
  (format t "~%"))

;; --- 2.2.2: attention over ten unit vectors ---------------------------------

(defparameter *n* 10)

(defparameter *vectors* (unit-circle-vectors *n*))

(defparameter *query*
  (linalg:from-list (list (/ 1.0 (sqrt 2.0)) (/ 1.0 (sqrt 2.0)))))

(defparameter *result* (np-attention *query* *vectors* *vectors*))

(format t "output vector: (~,3f ~,3f)~%" (aref (car *result*) 0)
        (aref (car *result*) 1))
(format t "weights shape: ~a~%" (linalg:shape (cadr *result*)))
(format t "weights sum:   ~,3f~%" (linalg:sum (cadr *result*)))
(print-weights "weights" (cadr *result*))

;; The vector nearest the query (45 degrees, between v2 and v3 of the book's
;; 1-based names) must attract the most attention.
(format t "argmax weight: v~a~%" (+ 1 (linalg:argmax (cadr *result*))))

;; --- one key made five times longer -----------------------------------------

(defparameter *long-vectors* (linalg:add *vectors* 0.0))

(dotimes (j 2)
  (setf (aref *long-vectors* 3 j) (* 5.0 (aref *long-vectors* 3 j))))

(defparameter *long-result*
  (np-attention *query* *long-vectors* *long-vectors*))

(print-weights "weights (v4 x5)" (cadr *long-result*))
(format t "argmax weight: v~a~%" (+ 1 (linalg:argmax (cadr *long-result*))))
(format t "largest weight: ~,3f -> ~,3f~%" (linalg:amax (cadr *result*))
        (linalg:amax (cadr *long-result*)))

;; --- three queries at once ---------------------------------------------------

(defparameter *queries*
  (linalg:from-list
   (list (list (/ 1.0 (sqrt 2.0)) (/ 1.0 (sqrt 2.0))) (list 1.0 0.0)
         (list 0.0 1.0))))

(defparameter *batched* (np-attention *queries* *vectors* *vectors*))

(format t "output shape:  ~a~%" (linalg:shape (car *batched*)))
(format t "weights shape: ~a~%" (linalg:shape (cadr *batched*)))
(print-weights "row sums     " (linalg:sum (cadr *batched*) :axis 1))

;; --- 2.2.3: why the scores are scaled ---------------------------------------
;; Twenty random keys against a random query, in 1 and in 100 dimensions, with
;; and without the 1/sqrt(d) scale. Without it the 100-dimensional dot products
;; spread out so far that the softmax collapses onto a single key; the scale
;; brings the largest weight back to the same order as the 1-dimensional case.

(linalg:seed 42)

(defparameter *n-keys* 20)

(dolist (dim '(1 100))
  (let* ((q (linalg:randn (list dim)))
         (k (linalg:randn (list *n-keys* dim)))
         (scores (linalg:dot k q))
         (unscaled (np-softmax scores))
         (scaled (np-softmax (linalg:div scores (sqrt (* 1.0 dim))))))
    (format t "~a dim: score spread ~,2f, largest weight ~,3f -> ~,3f~%" dim
            (- (linalg:amax scores) (linalg:amin scores)) (linalg:amax unscaled)
            (linalg:amax scaled))))

;; The scaled attention of section 2.2.3, over the circle again: the same
;; weights the unscaled call gives, only flatter.
(defparameter *scaled-result* (np-scaled-attention *query* *vectors* *vectors*))

(print-weights "scaled weights" (cadr *scaled-result*))


---

# FILE: references/examples/llm-from-scratch/chapter02/section3.lisp

;; chapter02/section3.lisp -- notebook section 2.3, ported.
;;
;; The four pieces a Transformer block is made of, each checked the way the
;; notebook checks it: the embedding table (2.3.1), the sinusoidal positional
;; encoding and the dot products between its rows (2.3.2), the position-wise
;; feed-forward network (2.3.3), the residual connection that makes the identity
;; learnable (2.3.4), and layer normalization (2.3.5).
;;
;; The notebook's heat map and its two line plots become the numbers behind
;; them: the encoding's first rows, and the theoretical dot product against the
;; measured one.
;;
;;   rontolisp chapter02/section3.lisp

(load "../transformer/utils.lisp")

(linalg:seed 42)

;; --- 2.3.1: the embedding table ---------------------------------------------

(defparameter *vocabulary-size* 100)

(defparameter *embedding* (torch:embedding *vocabulary-size* 512))

(format t "embedding table:  ~a~%"
        (torch:shape (torch:field *embedding* :weight)))
(format t "embedded tokens:  ~a~%"
        (torch:shape (torch:forward *embedding* '(1 2 3))))
(format t "embedded batch:   ~a~%"
        (torch:shape
         (torch:forward *embedding* (torch:pad-sequence '((1 2 3) (4 5))))))

;; --- 2.3.2: the sinusoidal positional encoding ------------------------------

(defparameter *pe-d-model* 128)

(defparameter *pe-length* 100)

(defparameter *pe*
  (linalg:squeeze (sinusoidal-position-encoding *pe-d-model* *pe-length*)
                  :axis 0))

(format t "encoding shape:   ~a~%" (linalg:shape *pe*))
(format t "first four dimensions of the first four positions:~%")
(dotimes (pos 4)
  (format t "  pos ~a:" pos)
  (dotimes (j 4) (format t " ~,4f" (aref *pe* pos j)))
  (format t "~%"))

;; Every row has the same length, which is what makes the dot product below a
;; function of the position DIFFERENCE alone.
(format t "row norms equal:  ~a~%"
        (if (< (- (linalg:amax (linalg:sum (linalg:square *pe*) :axis 1))
                  (linalg:amin (linalg:sum (linalg:square *pe*) :axis 1)))
               1.0e-6)
            "yes"
            "no"))

;; The notebook plots pe . pe^T and compares it against the closed form
;;   dot(pos) = sum over d of cos(pos / 10000^(2d / d_model)).
;; Here the two are compared numerically instead, over every pair of rows that
;; many positions apart.
(defun theoretical-dot (offset d-model)
  (let ((total 0.0))
    (do ((d 0 (+ d 1)))
        ((>= d (/ d-model 2)) total)
      (setq total
            (+ total (cos (/ offset (expt 10000.0 (/ (* 2.0 d) d-model)))))))))

(defparameter *dot-products* (linalg:matmul *pe* (linalg:transpose *pe*)))

(format t "dot product by position difference (theory vs measured):~%")
(dolist (offset '(0 1 2 5 20 50))
  (let ((worst 0.0) (theory (theoretical-dot offset *pe-d-model*)))
    (dotimes (i (- *pe-length* offset))
      (let ((diff (abs (- (aref *dot-products* i (+ i offset)) theory))))
        (when (> diff worst) (setq worst diff))))
    (format t "  offset ~2,'0d: ~7,2f  max deviation < 1e-6: ~a~%" offset theory
            (if (< worst 1.0e-6) "yes" "no"))))

;; --- 2.3.3: the position-wise feed-forward network --------------------------

(defparameter *ff-d-model* 64)

(defparameter *feed-forward*
  (torch:sequential (torch:linear *ff-d-model* (* 4 *ff-d-model*))
                    (function torch:relu)
                    (torch:linear (* 4 *ff-d-model*) *ff-d-model*)))

(defparameter *ff-input* (torch:tensor (linalg:randn (list 1 10 *ff-d-model*))))

(format t "feed-forward in:  ~a~%" (torch:shape *ff-input*))
(format t "feed-forward out: ~a~%"
        (torch:shape (torch:forward *feed-forward* *ff-input*)))

;; --- 2.3.4: learning the identity, with and without a skip connection --------
;; The notebook trains a two-layer FFN and the same block wrapped in x + f(x) to
;; reproduce their own input. The residual one starts near zero error because
;; the identity is already its default behaviour. Shapes are scaled down from
;; the notebook's 10000 x 10 / 100 epochs so the plain interpreter finishes in
;; seconds; the shape of the two curves is the point, not their length.

(defun ffn (d-model d-ff)
  (torch:sequential (torch:linear d-model d-ff) (function torch:relu)
                    (torch:linear d-ff d-model)))

(defun skip-connection-forward (self x)
  (torch:add x (torch:forward (torch:field self :sublayer) x)))

(defun skip-connection (d-model d-ff)
  (torch:module :skip-connection (list :sublayer (ffn d-model d-ff))
                (function skip-connection-forward)))

(defparameter *epochs* 40)

(defparameter *data* (torch:tensor (linalg:randn '(64 10))))

(defparameter *plain* (ffn 10 32))

(defparameter *residual* (skip-connection 10 32))

(defparameter *plain-optimizer* (torch:adam *plain* :lr 0.01))

(defparameter *residual-optimizer* (torch:adam *residual* :lr 0.01))

(defun identity-loss (model)
  (torch:mse-loss (torch:forward model *data*) *data*))

(format t "identity training (mean squared error):~%")
(dotimes (epoch *epochs*)
  (let ((plain-loss (identity-loss *plain*))
        (residual-loss (identity-loss *residual*)))
    (when (= 0 (mod epoch 10))
      (format t "  epoch ~2,'0d: FFN ~,4f  SkipConnection ~,4f~%" epoch
              (torch:item plain-loss) (torch:item residual-loss)))
    (torch:zero-grad *plain-optimizer*)
    (torch:zero-grad *residual-optimizer*)
    (torch:backward plain-loss)
    (torch:backward residual-loss)
    (torch:step *plain-optimizer*)
    (torch:step *residual-optimizer*)))

(format t "  final:   FFN ~,4f  SkipConnection ~,4f~%"
        (torch:item (identity-loss *plain*))
        (torch:item (identity-loss *residual*)))
(format t "  the residual block starts and finishes lower: ~a~%"
        (if (< (torch:item (identity-loss *residual*))
               (torch:item (identity-loss *plain*)))
            "yes"
            "no"))

;; --- 2.3.5: layer normalization ---------------------------------------------

(defparameter *activations* (torch:tensor (linalg:randn '(20 5 10))))

(defparameter *normalized* (layer-norm *activations* :eps 1.0e-8))

(defparameter *feature* (torch:slice *normalized* '((3 4) (2 3))))

(format t "normalized shape: ~a~%" (torch:shape *normalized*))
(format t "feature mean:     ~,6f~%" (abs (torch:item (torch:mean *feature*))))
(format t "feature std:      ~,6f~%" (torch:item (torch:std *feature* :ddof 0)))


---

# FILE: references/examples/llm-from-scratch/chapter02/section4.lisp

;; chapter02/section4.lisp -- notebook section 2.4, ported.
;;
;; Cross entropy: what it measures between two distributions, why it is the
;; loss a language model is trained with, and how the general definition
;; collapses onto `torch:cross-entropy-loss' once the target is a one-hot
;; vector -- which is what a next-token target always is.
;;
;; The notebook's bar charts become the discretised distributions themselves.
;;
;;   rontolisp chapter02/section4.lisp

(defun normal-pdf (x mean sigma)
  (let ((z (/ (- x mean) sigma)))
    (/ (exp (* -0.5 z z)) (* sigma (sqrt (* 2.0 pi))))))

(defun discretised-normal (xs mean sigma)
  (let ((out (linalg:zeros (linalg:shape xs))))
    (dotimes (i (linalg:size xs))
      (setf (aref out i) (normal-pdf (aref xs i) mean sigma)))
    out))

(defun cross-entropy (p q)
  ;; The book's numpy definition: -sum(p log q). Note it is NOT symmetric.
  (- (linalg:sum (linalg:mul p (linalg:log q)))))

(defun print-distribution (label p)
  (format t "~a:" label)
  (dotimes (i (linalg:size p)) (format t " ~,3f" (aref p i)))
  (format t "~%"))

;; --- the three discretised distributions ------------------------------------

(defparameter *xs* (linalg:linspace -3.0 3.0 7))

(defparameter *gaussian-1* (discretised-normal *xs* 0.0 1.0))

(defparameter *gaussian-2* (discretised-normal *xs* 0.0 1.5))

(defparameter *uniform* (linalg:full '(7) (/ 1.0 7)))

(print-distribution "N(0, 1.0)  " *gaussian-1*)
(print-distribution "N(0, 1.5)  " *gaussian-2*)
(print-distribution "uniform    " *uniform*)

(format t "H(N(0,1), N(0,1.0)) = ~,4f~%"
        (cross-entropy *gaussian-1* *gaussian-1*))
(format t "H(N(0,1), N(0,1.5)) = ~,4f~%"
        (cross-entropy *gaussian-1* *gaussian-2*))
(format t "H(N(0,1), uniform)  = ~,4f~%" (cross-entropy *gaussian-1* *uniform*))
(format t "closest to itself:    ~a~%"
        (if (and (< (cross-entropy *gaussian-1* *gaussian-1*)
                    (cross-entropy *gaussian-1* *gaussian-2*))
                 (< (cross-entropy *gaussian-1* *gaussian-2*)
                    (cross-entropy *gaussian-1* *uniform*)))
            "yes"
            "no"))

;; --- the one-hot target ------------------------------------------------------
;; A next-token target picks ONE word out of the vocabulary, so p is one-hot and
;; -sum(p log q) is just -log q[k]. That is what torch:cross-entropy-loss
;; computes, straight from the logits, with k as an integer class index.

(defparameter *vocabulary* '("吾輩" "僕" "猫" "犬" "ます" "です" "。"))

(defparameter *one-hot* (linalg:one-hot (linalg:from-list '(2)) 7))

(print-distribution "one-hot    " (linalg:row *one-hot* 0))
(format t "target word:          ~a~%" (nth 2 *vocabulary*))

(defparameter *logits* (torch:tensor '((0.5 1.0 2.5 0.2 0.1 0.4 0.3))))

(defparameter *probabilities* (torch:softmax *logits* :axis -1))

(print-distribution "model q    " (linalg:row (torch:data *probabilities*) 0))
(format t "-log q[2]           = ~,4f~%"
        (- (log (aref (torch:data *probabilities*) 0 2))))
(format t "cross-entropy-loss  = ~,4f~%"
        (torch:item (torch:cross-entropy-loss *logits* '(2))))

;; --- probability targets -----------------------------------------------------
;; nn.CrossEntropyLoss also accepts a full probability vector as the target --
;; the notebook's last cell. torch:cross-entropy-loss takes that spelling too:
;; a target whose SHAPE matches the logits is read as class probabilities, and
;; the loss is -sum(target * log-softmax(logits)) per position.

(defparameter *soft-logits* (torch:tensor '(0.1 0.2 0.7)))

(defparameter *mismatched-target* (torch:tensor '(0.7 0.2 0.1)))

(defparameter *aligned-target* (torch:tensor '(0.1 0.2 0.7)))

(format t "mismatched target   = ~,4f~%"
 (torch:item (torch:cross-entropy-loss *soft-logits* *mismatched-target*)))
(format t "aligned target      = ~,4f~%"
        (torch:item (torch:cross-entropy-loss *soft-logits* *aligned-target*)))


---

# FILE: references/examples/llm-from-scratch/chapter02/section5.lisp

;; chapter02/section5.lisp -- notebook section 2.5, ported.
;;
;; The whole chapter, end to end: the cross entropy a language model is trained
;; with (2.5.1), the padding and subsequent masks (2.5.2), and a Transformer
;; trained on a Japanese-English corpus and then decoded greedily (2.5.3).
;;
;; The notebook clones odashi/small_parallel_enja and trains a d_model=512,
;; 6-block, 8-head model for 20 epochs on a GPU. This port ships its own corpus
;; -- eight pairs, right here in the file, so the example is hermetic -- and a
;; model small enough to finish on the plain interpreter. It MEMORISES that
;; corpus; it does not generalise, and it is not meant to. The parameters at the
;; top are the knobs: raise them (and add data) to walk toward the book's run.
;;
;;   rontolisp chapter02/section5.lisp

(load "../transformer/transformer.lisp")

;; --- 2.5.1: cross entropy over a probability target -------------------------
;; The notebook feeds nn.CrossEntropyLoss a one-hot "logit" vector and two
;; probability targets. The loss is lower for the target that agrees with where
;; the model puts its mass.

(defparameter *logits* (torch:tensor '(1.0 0.0 0.0)))

(format t "cross entropy against (0.7 0.2 0.1): ~,4f~%"
 (torch:item (torch:cross-entropy-loss *logits* (torch:tensor '(0.7 0.2 0.1)))))
(format t "cross entropy against (0.1 0.2 0.7): ~,4f~%"
 (torch:item (torch:cross-entropy-loss *logits* (torch:tensor '(0.1 0.2 0.7)))))

;; --- 2.5.2: the two masks ----------------------------------------------------

(defun print-mask (label mask rows columns)
  (format t "~a ~a~%" label (linalg:shape mask))
  (dotimes (i rows)
    (format t " ")
    (dotimes (j columns) (format t " ~a" (truncate (aref mask 0 i j))))
    (format t "~%")))

(defparameter *sample-tokens*
  (torch:pad-sequence '((5 3 3) (1 9 4 3 1) (5 3 5 1))))

(defparameter *sample-padding-mask*
  (torch:padding-mask *sample-tokens* :pad-id 0))

(format t "tokens:  ~a~%"
        (mapcar (function truncate)
                (linalg:to-list (linalg:row (torch:data *sample-tokens*) 0))))
(print-mask "padding mask" *sample-padding-mask* 1 5)
(print-mask "subsequent mask" (torch:subsequent-mask 5) 5 5)

;; A masked score is filled with -infinity, so its softmax weight is exactly 0.
(defparameter *scores* (torch:tensor (linalg:zeros '(1 5 5))))

(defparameter *masked-weights*
  (torch:softmax
   (torch:masked-fill *scores* (torch:subsequent-mask 5) *neg-infinity*)
   :axis -1))

(format t "causal attention weights:~%")
(dotimes (i 5)
  (format t " ")
  (dotimes (j 5) (format t " ~,2f" (aref (torch:data *masked-weights*) 0 i j)))
  (format t "~%"))

;; --- 2.5.3: the corpus -------------------------------------------------------

(defparameter *corpus*
  '(("今日 の 天気 は 晴れ です 。" "the weather is fine today .")
    ("明日 の 天気 は 雨 です 。" "the weather is rainy tomorrow .")
    ("今日 は 寒い です 。" "it is cold today .") ("私 は 猫 が 好き です 。" "i like cats .")
    ("私 は 犬 が 好き です 。" "i like dogs .") ("彼 は 本 を 読み ます 。" "he reads a book .")
    ("私 は 学生 です 。" "i am a student .") ("猫 が 走り ます 。" "the cat runs .")))

(defparameter *specials* '("" "" "" ""))

(defparameter *unk-id* 0)

(defparameter *pad-id* 1)

(defparameter *bos-id* 2)

(defparameter *eos-id* 3)

(defun split-tokens (line)
  ;; The book's line.split(): whitespace-separated tokens, as a list of strings.
  (let ((tokens nil) (current nil))
    (dotimes (i (length line))
      (let ((ch (char line i)))
        (if (char= ch #\Space)
            (when current
              (setq tokens (cons (coerce (reverse current) 'string) tokens))
              (setq current nil))
            (setq current (cons ch current)))))
    (when current
      (setq tokens (cons (coerce (reverse current) 'string) tokens)))
    (reverse tokens)))

(defun wrap-sentence (line)
  ;; iter_corpus(): the token list with  in front and  behind.
  (append (list "") (split-tokens line) (list "")))

(defun build-vocabulary (sentences)
  ;; build_vocab_from_iterator(): the specials first, then every other token by
  ;; DESCENDING frequency, ties keeping first appearance -- Counter.most_common.
  ;; Returns (token-to-id id-to-token).
  (let ((counts (make-hash-table :test (function equal)))
        (seen nil)
        (highest 0))
    (dolist (tokens sentences)
      (dolist (token tokens)
        (if (gethash token counts)
            (setf (gethash token counts) (+ 1 (gethash token counts)))
            (progn
              (setf (gethash token counts) 1)
              (setq seen (cons token seen))))))
    (setq seen (reverse seen))
    (dolist (token seen)
      (when (> (gethash token counts) highest)
        (setq highest (gethash token counts))))
    (let ((ordered nil))
      (do ((c highest (- c 1)))
          ((< c 1))
        (dolist (token seen)
          (when (= (gethash token counts) c)
            (setq ordered (cons token ordered)))))
      (let ((token-to-id (make-hash-table :test (function equal)))
            (id-to-token nil)
            (next 0))
        (dolist (token (append *specials* (reverse ordered)))
          (unless (gethash token token-to-id)
            (setf (gethash token token-to-id) next)
            (setq id-to-token (cons token id-to-token))
            (setq next (+ next 1))))
        (list token-to-id (reverse id-to-token))))))

(defun tokens-to-ids (tokens token-to-id)
  ;; Vocab.__getitem__ with default_index = .
  (mapcar (lambda (token)
            (let ((id (gethash token token-to-id))) (if (null id) *unk-id* id)))
          tokens))

(defun ids-to-text (ids id-to-token)
  (let ((out ""))
    (dolist (id ids out)
      (setq out
            (if (string= out "")
                (nth id id-to-token)
                (concatenate 'string out " " (nth id id-to-token)))))))

(defparameter *source-sentences*
  (mapcar (lambda (pair) (wrap-sentence (car pair))) *corpus*))

(defparameter *target-sentences*
  (mapcar (lambda (pair) (wrap-sentence (cadr pair))) *corpus*))

(defparameter *source-vocabulary* (build-vocabulary *source-sentences*))

(defparameter *target-vocabulary* (build-vocabulary *target-sentences*))

(defparameter *source-ids*
  (mapcar (lambda (tokens) (tokens-to-ids tokens (car *source-vocabulary*)))
          *source-sentences*))

(defparameter *target-ids*
  (mapcar (lambda (tokens) (tokens-to-ids tokens (car *target-vocabulary*)))
          *target-sentences*))

(format t "source vocabulary: ~a tokens~%" (length (cadr *source-vocabulary*)))
(format t "target vocabulary: ~a tokens~%" (length (cadr *target-vocabulary*)))
(format t " id: ~a,  id: ~a,  id: ~a,  id: ~a~%" *unk-id*
        *pad-id* *bos-id* *eos-id*)
(format t "first source sentence: ~a~%" (car *source-ids*))
(format t "first target sentence: ~a~%" (car *target-ids*))

;; --- 2.5.3: training ---------------------------------------------------------

(defparameter *d-model* 8)

(defparameter *n-blocks* 1)

(defparameter *n-heads* 2)

(defparameter *d-k* 4)

(defparameter *d-v* 4)

(defparameter *d-ff* 16)

(defparameter *batch-size* 4)

(defparameter *epochs* 40)

(defparameter *learning-rate* 0.02)

(defparameter *max-length* 12)

(linalg:seed 7)

(defparameter *model*
  (transformer (length (cadr *source-vocabulary*))
               (length (cadr *target-vocabulary*)) *max-length* *d-model*
               *n-blocks* *n-heads* *d-k* *d-v* *d-ff*))

(defparameter *optimizer* (torch:adam *model* :lr *learning-rate*))

(format t "model parameters:  ~a tensors~%" (length (torch:parameters *model*)))

(defun batch-of (all indices) (mapcar (lambda (i) (nth i all)) indices))

(defun train-step (indices)
  ;; One optimizer step over a mini-batch, exactly the notebook's train() body:
  ;; the decoder reads the target without its last token and is scored against
  ;; the target without its first, under a padding mask on the source and a
  ;; padding + subsequent mask on the target.
  (let* ((source
          (torch:pad-sequence (batch-of *source-ids* indices)
                              :padding-value *pad-id*))
         (target
          (torch:pad-sequence (batch-of *target-ids* indices)
                              :padding-value *pad-id*))
         (length (cadr (torch:shape target)))
         (target-input (torch:slice target (list nil (list 0 (- length 1)))))
         (target-output (torch:slice target (list nil (list 1 length))))
         (source-mask (torch:padding-mask source :pad-id *pad-id*))
         (target-mask
          (linalg:add (torch:padding-mask target-input :pad-id *pad-id*)
                      (torch:subsequent-mask (- length 1))))
         (logits
          (torch:forward *model* source target-input source-mask target-mask
                         source-mask))
         (loss
          (torch:cross-entropy-loss logits target-output
                                    :ignore-index *pad-id*)))
    (torch:zero-grad *optimizer*)
    (torch:backward loss)
    (torch:step *optimizer*)
    (torch:item loss)))

(format t "training (cross entropy, ignore-index = ):~%")
(dotimes (epoch *epochs*)
  (let ((total 0.0) (steps 0))
    (dolist (batch (torch:shuffled-batches (length *corpus*) *batch-size*))
      (setq total (+ total (train-step batch)))
      (setq steps (+ steps 1)))
    (when (= 0 (mod epoch 5))
      (format t "  epoch ~2,'0d: ~,3f~%" epoch (/ total steps)))))

;; --- 2.5.3: greedy decoding --------------------------------------------------

(torch:eval *model*)

(format t "greedy decode:~%")
(defparameter *correct* 0)

(dotimes (i (length *corpus*))
  (let* ((source
          (torch:pad-sequence (list (nth i *source-ids*))
                              :padding-value *pad-id*))
         (decoded
          (transformer-inference *model* source *bos-id* *eos-id*
                                 :max-length *max-length*))
         (expected (nth i *target-ids*))
         (text (ids-to-text decoded (cadr *target-vocabulary*))))
    (when (equal decoded expected) (setq *correct* (+ 1 *correct*)))
    (format t "  ~a -> ~a~%" (car (nth i *corpus*)) text)))

(format t "sentences reproduced exactly: ~a / ~a~%" *correct* (length *corpus*))


---

# FILE: references/examples/llm-from-scratch/chapter03/section2.lisp

;; chapter03/section2.lisp -- notebooks/chapter03/section03_tokenizer.py,
;; ported: section 3.2, "Tokenizer".
;;
;; Word-level tokenization and its normalizer (3.2.1), the special tokens and
;; the truncation an input length forces (3.2.2), the two embedding tables
;; (3.2.3), and byte-pair encoding trained on a small corpus (3.2.4).
;;
;; Two things about the port:
;;
;;   * The book's re.findall(r"\w+|[^\w\s]", text) is a SCAN here, not a regexp
;;     call. The pattern is two character classes, which is what a scan is; and
;;     \w is taken as ASCII letter / digit / underscore, which is what this
;;     section's English demo text uses.
;;   * Python's dict preserves INSERTION order, and the BPE loop depends on it:
;;     max(pair_freqs, key=pair_freqs.get) breaks a tie by taking the pair seen
;;     FIRST. Every table below is therefore an ordered association list, and
;;     the tie rule is the strict > in bpe-best-pair -- which is what makes this
;;     port's merges come out in the book's exact order.
;;
;;   rontolisp chapter03/section2.lisp

;; --- 3.2.1 what a tokenizer is ----------------------------------------------

(defun word-char-p (ch)
  ;; Python's \w over ASCII text: a letter, a digit or an underscore.
  (if (or (alphanumericp ch) (char= ch #\_)) t nil))

(defun space-char-p (ch)
  ;; Python's \s, over the whitespace this section's text can hold.
  (if (or (char= ch #\Space) (char= ch #\Tab) (char= ch #\Newline)) t nil))

(defun tokenize-text (text)
  ;; re.findall(r"\w+|[^\w\s]", text): every RUN of word characters is one
  ;; token, every other non-space character is a token on its own, and
  ;; whitespace separates without producing anything.
  (let ((tokens nil) (start nil) (n (length text)))
    (dotimes (i n)
      (let ((ch (char text i)))
        (if (word-char-p ch)
            (when (null start) (setq start i))
            (progn
              (unless (null start)
                (setq tokens (cons (subseq text start i) tokens))
                (setq start nil))
              (unless (space-char-p ch)
                (setq tokens (cons (string ch) tokens)))))))
    (unless (null start) (setq tokens (cons (subseq text start n) tokens)))
    (reverse tokens)))

(defun preprocess-text (text)
  ;; The book's normalizer: lowercase, then a space either side of every
  ;; punctuation character (re.sub(r"([^\w\s])", r" \1 ")).
  (let ((lower (string-downcase text)) (out ""))
    (dotimes (i (length lower) out)
      (let ((ch (char lower i)))
        (setq out
              (if (or (word-char-p ch) (space-char-p ch))
                  (concatenate 'string out (string ch))
                  (concatenate 'string out " " (string ch) " ")))))))

(defun vocabulary-id (vocabulary token)
  ;; The token's id, or nil when the vocabulary does not hold it. The
  ;; vocabulary is an ordered alist of (token . id), which is a dict here.
  (dolist (cell vocabulary nil)
    (when (string= (car cell) token) (return (cdr cell)))))

(defun assign-token-ids (tokens vocabulary)
  ;; Every token's id, with the  id standing in for anything the
  ;; vocabulary does not hold.
  (let ((unknown (vocabulary-id vocabulary "")))
    (mapcar (lambda (token)
              (let ((id (vocabulary-id vocabulary token)))
                (if (null id) unknown id))) tokens)))

(defun token-counts (tokens)
  ;; collections.Counter: (token . count) in FIRST-SEEN order.
  (let ((acc nil))
    (dolist (token tokens (reverse acc))
      (let ((cell
             (dolist (c acc nil) (when (string= (car c) token) (return c)))))
        (if (null cell)
            (setq acc (cons (cons token 1) acc))
            (rplacd cell (+ (cdr cell) 1)))))))

(defun build-vocabulary
    (tokens &key max-size (min-freq 1) (special-tokens '("" "")))
  ;; The book's build_vocabulary: count, order by DESCENDING frequency, drop
  ;; anything rarer than min-freq, cut to max-size, and number the special
  ;; tokens first. The sort must be STABLE -- Python's sorted() is, so two
  ;; tokens of equal frequency keep the order they were first seen in, and the
  ;; resulting ids are reproducible rather than merely plausible.
  (let* ((counts (token-counts tokens))
         (ordered (stable-sort counts (lambda (x y) (> (cdr x) (cdr y)))))
         (kept nil)
         (n 0))
    (dolist (cell ordered)
      (when (and (>= (cdr cell) min-freq) (or (null max-size) (< n max-size)))
        (setq kept (cons (car cell) kept))
        (setq n (+ n 1))))
    (let ((vocabulary nil) (id 0))
      (dolist (token (append special-tokens (reverse kept)))
        (setq vocabulary (cons (cons token id) vocabulary))
        (setq id (+ id 1)))
      (reverse vocabulary))))

;; --- 3.2.2 what the input length forces -------------------------------------

(defun add-special-tokens (tokens max-length)
  ;; [CLS] before, [SEP] after, then [PAD] up to max-length.
  (let ((out (append (list "[CLS]") tokens (list "[SEP]"))))
    (dotimes (i (- max-length (length out)) out)
      (setq out (append out (list "[PAD]"))))))

(defun truncate-tokens (tokens max-length)
  ;; Longer than max-length: keep the first max-length - 1 and end with [SEP],
  ;; so a truncated sequence still carries its terminator.
  (if (> (length tokens) max-length)
      (append (subseq tokens 0 (- max-length 1)) (list "[SEP]"))
      tokens))

;; --- 3.2.4 byte-pair encoding ------------------------------------------------

(defun split-on-whitespace (text)
  ;; The book's simple_tokenizer: text.split(), i.e. runs of non-space.
  (let ((words nil) (start nil) (n (length text)))
    (dotimes (i n)
      (if (space-char-p (char text i))
          (unless (null start)
            (setq words (cons (subseq text start i) words))
            (setq start nil))
          (when (null start) (setq start i))))
    (unless (null start) (setq words (cons (subseq text start n) words)))
    (reverse words)))

(defun bpe-cell (table key)
  ;; The (key . value) cell of an ordered alist keyed by a STRING.
  (dolist (cell table nil) (when (string= (car cell) key) (return cell))))

(defun bpe-pair-cell (table pair)
  ;; The same, keyed by a PAIR of strings.
  (dolist (cell table nil)
    (when (and (string= (car (car cell)) (car pair))
               (string= (cadr (car cell)) (cadr pair)))
      (return cell))))

(defun bpe-word-freqs (corpus)
  ;; Every whitespace word of the corpus with its count, in first-seen order.
  (let ((acc nil))
    (dolist (text corpus (reverse acc))
      (dolist (word (split-on-whitespace text))
        (let ((cell (bpe-cell acc word)))
          (if (null cell)
              (setq acc (cons (cons word 1) acc))
              (rplacd cell (+ (cdr cell) 1))))))))

(defun bpe-alphabet (word-freqs)
  ;; The base vocabulary: every character occurring in a word, sorted.
  (let ((chars nil))
    (dolist (cell word-freqs)
      (let ((word (car cell)))
        (dotimes (i (length word)) (setq chars (cons (char word i) chars)))))
    (mapcar (function string)
            (sort (remove-duplicates chars) (function char<)))))

(defun bpe-splits (word-freqs)
  ;; Each word as its list of single-character tokens -- the state the merge
  ;; loop rewrites.
  (mapcar (lambda (cell)
            (let ((word (car cell)) (out nil))
              (dotimes (i (length word))
                (setq out (cons (string (char word i)) out)))
              (cons word (reverse out)))) word-freqs))

(defun bpe-pair-freqs (splits word-freqs)
  ;; How often each ADJACENT pair occurs, weighted by its word's count, in
  ;; first-seen order (Python's defaultdict insertion order).
  (let ((acc nil))
    (dolist (wc word-freqs)
      (let ((split (cdr (bpe-cell splits (car wc)))) (freq (cdr wc)))
        (unless (null (cdr split))
          (do ((p split (cdr p)))
              ((null (cdr p)))
            (let* ((pair (list (car p) (cadr p)))
                   (cell (bpe-pair-cell acc pair)))
              (if (null cell)
                  (setq acc (cons (cons pair freq) acc))
                  (rplacd cell (+ (cdr cell) freq))))))))
    (reverse acc)))

(defun bpe-best-pair (pair-freqs)
  ;; max(pair_freqs, key=pair_freqs.get): the most frequent pair, ties going to
  ;; the one seen FIRST -- which is what the strict > gives over an ordered
  ;; list, and what makes the merge order match the book's.
  (let ((best nil) (count 0))
    (dolist (cell pair-freqs best)
      (when (or (null best) (> (cdr cell) count))
        (setq best (car cell))
        (setq count (cdr cell))))))

(defun bpe-merge-split (split a b)
  ;; One split with every adjacent (a b) replaced by their concatenation.
  (let ((out nil) (p split))
    (do ()
        ((null p) (reverse out))
      (if (and (cdr p) (string= (car p) a) (string= (cadr p) b))
          (progn
            (setq out (cons (concatenate 'string a b) out))
            (setq p (cddr p)))
          (progn
            (setq out (cons (car p) out))
            (setq p (cdr p)))))))

(defun bpe-merge-pair (a b splits)
  ;; The merge applied to every word (the book's merge_pair), in place.
  (dolist (cell splits splits)
    (unless (null (cdr (cdr cell)))
      (rplacd cell (bpe-merge-split (cdr cell) a b)))))

(defun bpe-train (corpus &key (num-merges 10))
  ;; The BPE learner: start from the characters, then repeatedly merge the most
  ;; frequent adjacent pair into one token. Returns (vocabulary merges), the
  ;; merges being (a b merged) in the order they were learned -- that ORDER is
  ;; the model, since tokenizing replays it.
  (let* ((word-freqs (bpe-word-freqs corpus))
         (alphabet (bpe-alphabet word-freqs))
         (splits (bpe-splits word-freqs))
         (vocabulary (cons "" alphabet))
         (merges nil))
    (dotimes (i num-merges)
      (let* ((pair-freqs (bpe-pair-freqs splits word-freqs))
             (best (bpe-best-pair pair-freqs)))
        (when (null best) (return))
        (let ((merged (concatenate 'string (car best) (cadr best))))
          (bpe-merge-pair (car best) (cadr best) splits)
          (setq merges (cons (list (car best) (cadr best) merged) merges))
          (setq vocabulary (append vocabulary (list merged))))))
    (list vocabulary (reverse merges))))

(defun bpe-tokenize (merges text)
  ;; New text through the learned merges: split into characters, then replay
  ;; every merge in the order it was learned.
  (let ((splits
         (mapcar (lambda (word)
                   (let ((out nil))
                     (dotimes (i (length word))
                       (setq out (cons (string (char word i)) out)))
                     (reverse out))) (split-on-whitespace text)))
        (out nil))
    (dolist (merge merges)
      (setq splits
            (mapcar
             (lambda (split) (bpe-merge-split split (car merge) (cadr merge)))
             splits)))
    (dolist (split splits (reverse out))
      (dolist (token split) (setq out (cons token out))))))

;; --- the demonstrations ------------------------------------------------------

(format t "=== 3.2.1 word-level tokenization ===~%")

(defparameter *text* "I like apples. You like oranges, not apples.")

(format t "original text:      ~a~%" *text*)
(format t "tokens:             ~s~%" (tokenize-text *text*))

(defparameter *preprocessed* (tokenize-text (preprocess-text *text*)))

(format t "preprocessed:       ~s~%" *preprocessed*)

(defparameter *vocabulary*
  '(("" . 0) ("" . 1) ("i" . 2) ("like" . 3) ("apples" . 4) ("." . 5)
    ("you" . 6) ("oranges" . 7) ("," . 8) ("not" . 9)))

(format t "token ids:          ~s~%"
        (assign-token-ids *preprocessed* *vocabulary*))
(format t "built vocabulary:   ~s~%" (build-vocabulary *preprocessed*))

(format t "~%=== 3.2.2 special tokens and truncation ===~%")

(format t "with special:       ~s~%"
        (add-special-tokens '("i" "like" "apples" ".") 10))
(format t "truncated:          ~s~%"
        (truncate-tokens '("[CLS]" "i" "like" "apples" "." "you" "like"
                           "oranges" "," "not" "apples" "." "[SEP]") 10))

(format t "~%=== 3.2.3 the embedding tables ===~%")

(linalg:seed 42)

(defparameter *token-embedding* (torch:embedding 10 128))

(defparameter *token-ids* (torch:pad-sequence '((0 1 2 3 4) (5 6 7 8 9))))

(format t "input embeddings:   ~a~%"
        (torch:shape (torch:forward *token-embedding* *token-ids*)))

(defparameter *position-embedding* (torch:embedding 5 128))

(format t "position embeddings:~a~%"
        (torch:shape
         (torch:forward *position-embedding*
                        (torch:pad-sequence (list (list 0 1 2 3 4))))))

(format t "~%=== 3.2.4 byte-pair encoding ===~%")

(defparameter *corpus*
  '("Large language models are transforming the landscape of natural language processing."
    "Understanding tokenization is crucial for building efficient models."
    "This chapter explores the intricacies of BPE and its impact on LLM performance."
    "By mastering these techniques, you can enhance the capabilities of your models."
    "Each token in a sequence carries semantic meaning and contextual information."
    "Modern tokenizers split text into meaningful subword units called tokens."
    "A well-designed token vocabulary improves model efficiency and accuracy."
    "Token-level representations enable better handling of rare and unseen words."
    "Tokenization strategies directly affect how models process and understand text."
    "The choice of tokenization method influences both training speed and final performance."
    "Effective token boundaries help preserve linguistic structures in the data."
    "Language models and LLMs benefit from sophisticated tokenization approaches."))

(defparameter *bpe* (bpe-train *corpus* :num-merges 100))

(format t "vocabulary:         ~s~%" (car *bpe*))
(format t "merges:             ~s~%" (cadr *bpe*))

(defparameter *test-text* "Tokenization improves LLMs.")

(format t "original text:      ~a~%" *test-text*)
(format t "bpe tokens:         ~s~%" (bpe-tokenize (cadr *bpe*) *test-text*))


---

# FILE: references/examples/llm-from-scratch/chapter03/train-gpt-soseki.lisp

;; chapter03/train-gpt-soseki.lisp -- notebooks/chapter03/train_gpt_soseki.ipynb,
;; ported: a character-level GPT trained on 夏目漱石, then sampled from.
;;
;; The notebook downloads『吾輩は猫である』from 青空文庫 with requests +
;; BeautifulSoup and strips its 注記 and ルビ. Nothing is downloaded here and
;; nothing is vendored: the corpus below is the novel's OPENING, which is public
;; domain, inlined so the program is self-contained and runs on all four
;; backends -- the same choice chapter02/section5.lisp made for its parallel
;; corpus. Everything the notebook does with the full novel it does with this;
;; only the shapes shrink, and every one of them is a defparameter here.
;;
;; The notebook's loss PLOT does not port; the losses it was drawn from are
;; printed instead, which is also what makes this example testable.
;;
;;   rontolisp chapter03/train-gpt-soseki.lisp

(load "../gpt/trainer.lisp")

(linalg:seed 42)

;; --- the corpus --------------------------------------------------------------
;; 夏目漱石『吾輩は猫である』(1905) の冒頭 -- public domain.

(defparameter *soseki-lines*
  '("吾輩は猫である。名前はまだ無い。どこで生れたかとんと見当がつかぬ。"
    "何でも薄暗いじめじめした所でニャーニャー泣いていた事だけは記憶している。吾輩はここで始めて人間というものを見た。"
    "しかもあとで聞くとそれは書生という人間中で一番獰悪な種族であったそうだ。" "この書生というのは時々我々を捕えて煮て食うという話である。"
    "しかしその当時は何という考もなかったから別段恐しいとも思わなかった。"
    "ただ彼の掌に載せられてスーと持ち上げられた時何だかフワフワした感じがあったばかりである。"
    "掌の上で少し落ちついて書生の顔を見たのがいわゆる人間というものの見始であろう。"
    "この時妙なものだと思った感じが今でも残っている。第一毛をもって装飾されべきはずの顔がつるつるしてまるで薬缶だ。"
    "その後猫にもだいぶ逢ったがこんな片輪には一度も出会わした事がない。のみならず顔の真中があまりに突起している。"
    "そうしてその穴の中から時々ぷうぷうと煙を吹く。どうも咽せぽくて実に弱った。" "これが人間の飲む煙草というものである事はようやくこの頃知った。"))

(defparameter *text*
  (let ((out ""))
    (dolist (line *soseki-lines* out)
      (setq out (concatenate 'string out line)))))

;; --- the shapes --------------------------------------------------------------
;; The notebook's own: block-size 256, n-embd 384, 6 layers, 6 heads, 5000 steps
;; on a T4. Raise these back toward it and the program is unchanged.

(defparameter *block-size* 8)

(defparameter *n-embd* 8)

(defparameter *n-layer* 1)

(defparameter *n-head* 2)

(defparameter *dropout* 0.1)

(defparameter *batch-size* 4)

(defparameter *max-steps* 100)

(defparameter *warmup-steps* 8)

(defparameter *learning-rate* 0.04)

(defparameter *new-tokens* 30)

(format t "corpus size: ~a characters~%" (length *text*))

;; --- the tokenizer -----------------------------------------------------------

(defparameter *tokenizer* (simple-tokenizer *text*))

(format t "vocabulary: ~a distinct characters~%"
        (tokenizer-vocab-size *tokenizer*))
(format t "first ten: ~s~%" (subseq (torch:field *tokenizer* :chars) 0 10))

;; --- the data ----------------------------------------------------------------

(defparameter *loaders*
  (create-dataloaders *text* *tokenizer*
                      :block-size *block-size*
                      :batch-size *batch-size*
                      :train-split 0.9))

(defparameter *train-loader* (car *loaders*))

(defparameter *val-loader* (cadr *loaders*))

(format t "train batches: ~a | validation batches: ~a~%"
        (length (data-loader-batches *train-loader*))
        (length (data-loader-batches *val-loader*)))

;; --- the model ---------------------------------------------------------------

(defparameter *config*
  (gpt-config :vocab-size (tokenizer-vocab-size *tokenizer*)
              :n-embd *n-embd*
              :n-layer *n-layer*
              :n-head *n-head*
              :block-size *block-size*
              :dropout *dropout*))

(format t "configured size: ~,4f M parameters~%" (torch:forward *config*))

(defparameter *model* (gpt-from-config *config*))

;; --- training ----------------------------------------------------------------

(defparameter *trainer*
  (gpt-trainer *model* *train-loader* *val-loader*
               :learning-rate *learning-rate*
               :weight-decay 0.1
               :warmup-steps *warmup-steps*
               :max-steps *max-steps*
               :grad-clip 1.0))

(defparameter *losses*
  (gpt-trainer-train *trainer* :log-interval 25 :eval-interval 50))

(defparameter *train-losses* (car *losses*))

(format t "first loss ~,4f -> last loss ~,4f~%" (car *train-losses*)
        (car (last *train-losses*)))
(format t "loss fell: ~a~%"
        (< (car (last *train-losses*)) (car *train-losses*)))

;; The untrained model's loss on a vocabulary of this size is log(vocab-size) --
;; a uniform guess -- so beating it is the first thing training has to do.
(format t "beats a uniform guess: ~a~%"
        (< (car (last *train-losses*))
           (log (* 1.0 (tokenizer-vocab-size *tokenizer*)))))

;; --- generation --------------------------------------------------------------

(torch:eval *model*)

(defun show-sample (prompt)
  ;; The book's generation cell: encode the prompt, sample new-tokens more, and
  ;; decode the whole thing. Every draw comes from the seeded linalg generator,
  ;; so the sample is the same text on every backend.
  (let ((ids
         (gpt-generate *model* (tokenizer-encode *tokenizer* prompt)
                       *new-tokens*
                       :temperature 0.8
                       :top-k 5)))
    (format t "prompt ~a -> ~a~%" prompt (tokenizer-decode *tokenizer* ids))))

(show-sample "吾輩は")

(show-sample "この書生")


---

# FILE: references/examples/llm-from-scratch/gpt/dataset.lisp

;; gpt/dataset.lisp -- llm_from_scratch/gpt/dataset.py, ported.
;;
;; TextDataset and create_dataloaders: the corpus as one long token vector, and
;; the autoregressive (x y) pair at position i -- block-size tokens and the same
;; window shifted one to the right, so predicting y from x IS next-token
;; prediction.
;;
;; torch.utils.data has no counterpart here and needs none: a Dataset is a
;; module holding the token vector, and a DataLoader is a module holding an
;; index list -- the epoch's mini-batches are torch:shuffled-batches over it and
;; a batch is COLLATED on demand, which is the only part of a DataLoader that
;; does real work.

(load "tokenizer.lisp")

(defun text-dataset (text tokenizer &key (block-size 128))
  ;; The dataset over a corpus: fields :tokens (the ids as a simple vector,
  ;; indexable in constant time -- a list would make every item a walk),
  ;; :block-size and :length. Prints its size, like the book's __init__.
  (let* ((ids (tokenizer-encode tokenizer text))
         (n (length ids))
         (v (make-array n :initial-element 0))
         (i 0))
    (dolist (id ids)
      (setf (aref v i) id)
      (setq i (+ i 1)))
    (format t "dataset size: ~a tokens~%" n)
    (torch:module :text-dataset (list :tokens v
                                      :block-size block-size
                                      :length (- n block-size))
                  (function text-dataset-item))))

(defun text-dataset-length (dataset)
  ;; How many windows the corpus holds: tokens - block-size, so the last one
  ;; still has a target for its final position (the book's __len__).
  (torch:field dataset :length))

(defun text-dataset-item (dataset idx)
  ;; The (x y) pair at idx as two LISTS of ids: x is tokens[idx : idx+block],
  ;; y the same window shifted by one (the book's __getitem__).
  (let ((v (torch:field dataset :tokens))
        (b (torch:field dataset :block-size))
        (x nil)
        (y nil))
    (do ((k (- b 1) (- k 1)))
        ((< k 0) (list x y))
      (setq x (cons (aref v (+ idx k)) x))
      (setq y (cons (aref v (+ idx k 1)) y)))))

(defun text-dataset-batch (dataset indices)
  ;; The COLLATE step: the windows at indices as the two (batch block-size)
  ;; index tensors torch:embedding and torch:cross-entropy-loss take. Every
  ;; window is block-size long, so torch:pad-sequence pads nothing here and is
  ;; simply the batch-first stack.
  (let ((xs nil) (ys nil))
    (dolist (i indices)
      (let ((pair (text-dataset-item dataset i)))
        (setq xs (cons (car pair) xs))
        (setq ys (cons (cadr pair) ys))))
    (list (torch:pad-sequence (reverse xs)) (torch:pad-sequence (reverse ys)))))

(defun data-loader (dataset indices batch-size shuffle)
  ;; A loader over a SUBSET of the dataset (a DataLoader over a random_split
  ;; part): fields :dataset, :indices, :batch-size and :shuffle.
  (torch:module :data-loader (list :dataset dataset
                                   :indices indices
                                   :batch-size batch-size
                                   :shuffle shuffle)
                (function data-loader-batches)))

(defun data-loader-batches (loader)
  ;; One epoch as a list of INDEX batches -- reshuffled under :shuffle t, in
  ;; order otherwise. The tensors are not built here: data-loader-collate makes
  ;; each batch as the training loop reaches it, so an epoch costs one batch of
  ;; memory rather than all of them.
  (torch:shuffled-batches (torch:field loader :indices)
                          (torch:field loader :batch-size)
                          :shuffle (torch:field loader :shuffle)))

(defun data-loader-collate (loader indices)
  ;; One index batch as its (x y) tensor pair.
  (text-dataset-batch (torch:field loader :dataset) indices))

(defun create-dataloaders
    (text tokenizer &key (block-size 128) (batch-size 64) (train-split 0.9))
  ;; The book's create_dataloaders: one dataset, a RANDOM split of its windows
  ;; into a training and a validation part (torch.utils.data.random_split), and
  ;; a loader over each -- the training one reshuffled every epoch, the
  ;; validation one always in the same order. The split comes from the seeded
  ;; linalg generator, so linalg:seed reproduces it on every backend. Returns
  ;; (train-loader val-loader).
  (let* ((dataset (text-dataset text tokenizer :block-size block-size))
         (n (text-dataset-length dataset))
         (n-train (truncate (* train-split n)))
         (perm (linalg:permutation n))
         (train nil)
         (val nil))
    (do ((k (- n 1) (- k 1)))
        ((< k 0))
      (let ((i (truncate (aref perm k))))
        (if (< k n-train) (setq train (cons i train)) (setq val (cons i val)))))
    (list (data-loader dataset train batch-size t)
          (data-loader dataset val batch-size nil))))


---

# FILE: references/examples/llm-from-scratch/gpt/model.lisp

;; gpt/model.lisp -- llm_from_scratch/gpt/model.py, ported.
;;
;; The GPT architecture: chapter 2's multi-head attention under a CAUSAL mask, a
;; pre-LayerNorm transformer block, the decoder-only stack with LEARNED position
;; embeddings, and autoregressive sampling. Three things differ from chapter 2's
;; encoder/decoder model and this file is where each of them lives -- the mask,
;; the learned positions, and the norm moving to the FRONT of each sublayer.
;;
;; Two shapes of the book's code do not survive the port, and both are named
;; here rather than hidden:
;;
;;   * `forward(idx, targets=None)` returns the (logits, loss) TUPLE. Here
;;     gpt-forward answers the logits and gpt-loss answers the loss, because a
;;     forward whose result depends on whether an optional argument was passed
;;     is a tuple only Python's caller can destructure cheaply.
;;   * `self.apply(self._init_weights)` walks the module tree by isinstance.
;;     gpt-apply walks torch:fields and dispatches on torch:module-kind, which
;;     is the same walk over the plist that IS a module's registration.

(load "../transformer/attention.lisp")

(defparameter *init-std* 0.02)

;; --- causal (masked) self attention -----------------------------------------

(defun gpt-attention-forward (self x)
  ;; Self attention with query = key = value = x under the look-ahead mask, then
  ;; the residual dropout. torch:subsequent-mask is (1 seq seq) and broadcasts
  ;; over the batch, so the book's unsqueeze(0).expand(B, -1, -1) is nothing the
  ;; port has to spell.
  (torch:forward (torch:field self :resid-dropout)
                 (torch:forward (torch:field self :attention) x x x
                  (torch:subsequent-mask (cadr (torch:shape x))))))

(defun gpt-attention (n-embd n-head &key (dropout 0.1))
  ;; GPTMultiHeadAttention: chapter 2's multi-head-attention, with the head
  ;; width n-embd / n-head, plus a dropout on its output.
  (unless (= 0 (mod n-embd n-head))
    (error "gpt: n-embd must be a multiple of n-head"))
  (let ((d-k (/ n-embd n-head)))
    (torch:module :gpt-attention (list :n-head n-head
                                       :n-embd n-embd
                                       :attention
                                       (multi-head-attention n-head d-k d-k
                                                             n-embd)
                                       :resid-dropout (torch:dropout dropout))
                  (function gpt-attention-forward))))

;; --- the transformer block (pre-LN) -----------------------------------------

(defun gpt-block-forward (self x)
  ;; x + attn(ln1(x)), then + mlp(ln2(x)): the norm sits BEFORE each sublayer
  ;; and the residual path stays unnormalized all the way through, which is
  ;; what makes a deep stack trainable without a warmup on the residual scale.
  ;; Chapter 2's encoder block normalizes AFTER the addition instead.
  (let ((attended
         (torch:add x
                    (torch:forward (torch:field self :attn)
                     (torch:forward (torch:field self :ln-1) x)))))
    (torch:add attended
               (torch:forward (torch:field self :mlp)
                (torch:forward (torch:field self :ln-2) attended)))))

(defun gpt-block (n-embd n-head &key (dropout 0.1))
  ;; TransformerBlock: attention and a 4x-wide GELU feed-forward, each behind
  ;; its own LayerNorm and inside its own residual.
  (torch:module :gpt-block (list :ln-1 (torch:layer-norm n-embd)
                            :attn (gpt-attention n-embd n-head :dropout dropout)
                            :ln-2 (torch:layer-norm n-embd)
                            :mlp (torch:sequential
                                  (torch:linear n-embd (* 4 n-embd))
                                  (function torch:gelu)
                                  (torch:linear (* 4 n-embd) n-embd)
                                  (torch:dropout dropout)))
                (function gpt-block-forward)))

;; --- the model ---------------------------------------------------------------

(defun gpt-forward (self idx)
  ;; (batch seq) token ids -> (batch seq vocab) logits. The position embedding
  ;; is looked up for 0..seq-1 and broadcasts over the batch, which is the
  ;; book's arange(T).unsqueeze(0).
  (let* ((seq-len (cadr (torch:shape idx)))
         (x
          (torch:forward (torch:field self :drop)
                         (torch:add (torch:forward
                                     (torch:field self :token-embedding) idx)
                                    (torch:forward
                                     (torch:field self :position-embedding)
                                     (linalg:arange seq-len))))))
    (dolist (block (torch:field self :blocks)) (setq x (torch:forward block x)))
    (torch:forward (torch:field self :head)
                   (torch:forward (torch:field self :ln-f) x))))

(defun gpt-loss (model idx targets)
  ;; The training objective: cross entropy of the (batch seq vocab) logits
  ;; against the (batch seq) next tokens. torch:cross-entropy-loss flattens the
  ;; leading axes itself, so the book's two view() calls have nothing to do.
  (torch:cross-entropy-loss (gpt-forward model idx) targets))

(defun gpt (vocab-size &key (n-embd 768) (n-layer 12) (n-head 12)
                       (block-size 1024) (dropout 0.1))
  ;; The GPT language model: a token table, a LEARNED position table (chapter 2
  ;; used a fixed sinusoid), n-layer blocks, a final norm and a bias-free
  ;; projection onto the vocabulary. Weights are re-initialized the book's way
  ;; before it is returned.
  (let ((blocks nil))
    (dotimes (i n-layer)
      (setq blocks (cons (gpt-block n-embd n-head :dropout dropout) blocks)))
    (let ((model
           (torch:module :gpt (list :block-size block-size
                               :n-embd n-embd
                               :token-embedding
                               (torch:embedding vocab-size n-embd)
                               :position-embedding
                               (torch:embedding block-size n-embd)
                               :drop (torch:dropout dropout)
                               :blocks (reverse blocks)
                               :ln-f (torch:layer-norm n-embd)
                               :head (torch:linear n-embd vocab-size :bias nil))
                         (function gpt-forward))))
      (gpt-apply model (function gpt-init-module))
      model)))

;; --- weight initialization (nn.Module.apply) --------------------------------

(defun gpt-apply (v fn)
  ;; nn.Module.apply: fn over every module reachable from v -- through the
  ;; field VALUES (torch:fields) and through any list of submodules in one --
  ;; children before their parent, like PyTorch. A field holding something that
  ;; is neither a module nor a list is simply not a module and is skipped.
  (cond ((torch:modulep v)
         (do ((p (torch:fields v) (cddr p)))
             ((null p))
           (gpt-apply (cadr p) fn))
         (funcall fn v))
        ((consp v) (dolist (e v) (gpt-apply e fn)))))

(defun gpt-normal-init (parameter)
  ;; torch.nn.init.normal_(w, mean=0.0, std=0.02) over a parameter's own shape.
  (torch:set-data parameter
   (linalg:mul *init-std* (linalg:randn (torch:shape parameter)))))

(defun gpt-init-module (m)
  ;; The book's _init_weights, dispatched on torch:module-kind rather than on
  ;; isinstance: a linear layer gets N(0, 0.02) weights and a zero bias, an
  ;; embedding table N(0, 0.02), a LayerNorm a unit gain and a zero bias.
  ;; Every other kind -- dropout, sequential, the blocks, the model itself --
  ;; owns no parameter of its own and is left alone.
  (let ((kind (torch:module-kind m)))
    (cond ((eq kind :linear)
           (gpt-normal-init (torch:field m :weight))
           (let ((b (torch:field m :bias)))
             (unless (null b)
               (torch:set-data b (linalg:zeros (torch:shape b))))))
          ((eq kind :embedding) (gpt-normal-init (torch:field m :weight)))
          ((eq kind :layer-norm)
           (torch:set-data (torch:field m :weight)
                           (linalg:ones (torch:shape (torch:field m :weight))))
           (torch:set-data (torch:field m :bias)
            (linalg:zeros (torch:shape (torch:field m :bias))))))))

;; --- generation --------------------------------------------------------------

(defun gpt-top-k-filter (logits k)
  ;; The book's top-k step: every logit below the row's k-th largest becomes
  ;; -infinity, so the softmax gives it weight exactly 0 and only the k best
  ;; tokens can be drawn.
  (let* ((n (car (last (torch:shape logits))))
         (kk (min k n))
         (top (torch:topk logits kk))
         (threshold (linalg:slice top (list nil (list (- kk 1) kk)))))
    (torch:masked-fill logits (linalg:less (torch:data logits) threshold)
                       *neg-infinity*)))

(defun gpt-generate (model tokens max-new-tokens &key (temperature 1.0) top-k)
  ;; Autoregressive sampling, as a LIST of token ids beginning with the prompt:
  ;; predict, keep the LAST position's logits, divide by the temperature,
  ;; optionally keep only the top k, then draw from the softmax. The context is
  ;; cropped to the model's block-size, so the loop never outruns the position
  ;; table. Wrapped in torch:no-grad, like the book's @torch.no_grad().
  (torch:no-grad
    (let ((block-size (torch:field model :block-size)) (out tokens))
      (dotimes (step max-new-tokens out)
        (let* ((context
                (if (<= (length out) block-size) out (last out block-size)))
               (logits (gpt-forward model (torch:pad-sequence (list context))))
               (shape (torch:shape logits))
               (row
                (torch:div (torch:reshape (torch:slice logits
                                                       (list nil
                                                             (list
                                                              (- (cadr shape) 1)
                                                              (cadr shape))
                                                             nil))
                                          (list 1 (caddr shape))) temperature))
               (filtered (if (null top-k) row (gpt-top-k-filter row top-k)))
               (probs (torch:softmax filtered :axis -1)))
          (setq out
                (append out
                 (list (truncate (aref (torch:multinomial probs) 0 0))))))))))

;; --- the configuration record -----------------------------------------------

(defun gpt-config-model-size (self)
  ;; GPTConfig.get_model_size: the parameter count in MILLIONS, computed from
  ;; the shapes alone -- so a configuration can be sized before anything is
  ;; allocated. Both embedding tables, then per block the four attention
  ;; projections, the two feed-forward matrices and the two LayerNorms, then
  ;; the final norm and the output projection.
  (let* ((vocab (torch:field self :vocab-size))
         (n-embd (torch:field self :n-embd))
         (n-layer (torch:field self :n-layer))
         (block-size (torch:field self :block-size))
         (per-block
          (+ (* n-embd n-embd 4) (* n-embd n-embd 4) (* 4 n-embd n-embd)
             (* n-embd 4)))
         (params
          (+ (* vocab n-embd) (* block-size n-embd) (* per-block n-layer)
             (* n-embd 2) (* n-embd vocab))))
    (/ params 1000000.0)))

(defun gpt-config (&key (vocab-size 50257) (n-embd 768) (n-layer 12) (n-head 12)
                        (block-size 1024) (dropout 0.1))
  ;; The model's hyper-parameters as a module of pure fields -- no parameter, so
  ;; torch:parameters walks it and collects nothing. Its forward is
  ;; gpt-config-model-size, so (torch:forward config) sizes it.
  (torch:module :gpt-config (list :vocab-size vocab-size
                                  :n-embd n-embd
                                  :n-layer n-layer
                                  :n-head n-head
                                  :block-size block-size
                                  :dropout dropout)
                (function gpt-config-model-size)))

(defun gpt-from-config (config)
  ;; The model a configuration describes.
  (gpt (torch:field config :vocab-size)
       :n-embd (torch:field config :n-embd)
       :n-layer (torch:field config :n-layer)
       :n-head (torch:field config :n-head)
       :block-size (torch:field config :block-size)
       :dropout (torch:field config :dropout)))


---

# FILE: references/examples/llm-from-scratch/gpt/shapes.lisp

;; gpt/shapes.lisp -- the shape check of the gpt package, in the spirit of
;; attention.py's own `if __name__ == "__main__"` block (transformer/shapes.lisp).
;;
;; Everything printed here is a SHAPE, a count or an exact token id, so the
;; output is one text on every backend regardless of the low-order float digits.
;;
;;   rontolisp gpt/shapes.lisp

(load "trainer.lisp")

(linalg:seed 42)

(defparameter *corpus* "the quick brown fox jumps over the lazy dog. ")

(defparameter *tokenizer* (simple-tokenizer *corpus*))

(format t "vocabulary:                   ~a characters~%"
        (tokenizer-vocab-size *tokenizer*))
(format t "encode/decode round trip:     ~a~%"
        (string=
         (tokenizer-decode *tokenizer* (tokenizer-encode *tokenizer* *corpus*))
         *corpus*))

(defparameter *config*
  (gpt-config :vocab-size (tokenizer-vocab-size *tokenizer*)
              :n-embd 16
              :n-layer 2
              :n-head 2
              :block-size 8
              :dropout 0.1))

(format t "configured model size:        ~,4f M parameters~%"
        (torch:forward *config*))

(defparameter *model* (gpt-from-config *config*))

(format t "model parameters:             ~a tensors~%"
        (length (torch:parameters *model*)))

(defparameter *groups* (gpt-parameter-groups *model*))

(format t "decay / no-decay split:       ~a / ~a tensors~%"
        (length (car *groups*)) (length (cadr *groups*)))

(defparameter *loaders*
  (create-dataloaders *corpus* *tokenizer*
                      :block-size 8
                      :batch-size 4
                      :train-split 0.75))

(defparameter *train-loader* (car *loaders*))

(defparameter *batch*
  (data-loader-collate *train-loader*
                       (car (data-loader-batches *train-loader*))))

(format t "batch inputs / targets:       ~a / ~a~%" (torch:shape (car *batch*))
        (torch:shape (cadr *batch*)))
(format t "gpt logits:                   ~a~%"
        (torch:shape (gpt-forward *model* (car *batch*))))
(format t "loss is a scalar:             ~a~%"
        (null (torch:shape (gpt-loss *model* (car *batch*) (cadr *batch*)))))

;; The causal mask is what makes this a decoder: position i may not see j > i,
;; so the logits at position 0 cannot change when a LATER token is replaced.
(torch:eval *model*)

(defun first-position-logits (tokens)
  (linalg:to-list
   (linalg:flatten
    (linalg:slice
     (torch:data (gpt-forward *model* (torch:pad-sequence (list tokens))))
     '((0 1) (0 1))))))

(format t "causal mask holds:            ~a~%"
 (equal (first-position-logits '(1 2 3 4)) (first-position-logits '(1 2 3 9))))

(format t "generated length:             ~a tokens~%"
        (length
         (gpt-generate *model* (tokenizer-encode *tokenizer* "the") 6
                       :temperature 0.8
                       :top-k 4)))

;; With :top-k 1 only the single best token can be drawn, so sampling becomes
;; greedy decoding and two runs from the same prompt agree exactly.
(format t "top-k 1 is deterministic:     ~a~%"
        (equal
         (gpt-generate *model* (tokenizer-encode *tokenizer* "the") 8 :top-k 1)
         (gpt-generate *model* (tokenizer-encode *tokenizer* "the") 8
                       :top-k 1)))


---

# FILE: references/examples/llm-from-scratch/gpt/tokenizer.lisp

;; gpt/tokenizer.lisp -- llm_from_scratch/gpt/tokenizer.py, ported.
;;
;; SimpleTokenizer, the character-level tokenizer: the vocabulary is the SORTED
;; set of characters occurring in the corpus, an id is that character's position
;; in it. Python's `dict` becomes a hash table in each direction; `sorted(set(t))`
;; becomes remove-duplicates + sort, which is the same thing said in Lisp.
;;
;; A tokenizer is a torch:module with no parameters -- the tables are ordinary
;; fields, so torch:parameters walks it and collects nothing, exactly like the
;; positional encoding's buffer in transformer/utils.lisp.

(defun simple-tokenizer-chars (text)
  ;; The corpus' distinct characters, in ascending char-code order. This list IS
  ;; the vocabulary: index i holds the character encoded as i.
  (sort (remove-duplicates (coerce text 'list)) (function char<)))

(defun simple-tokenizer (text)
  ;; The tokenizer over a corpus: fields :chars (the vocabulary as a list),
  ;; :vocab-size, and the two lookup tables. The forward is `encode', so
  ;; (torch:forward tokenizer "text") works, but the two named functions below
  ;; are what the rest of the port calls.
  (let* ((chars (simple-tokenizer-chars text))
         (to-idx (make-hash-table))
         (from-idx (make-hash-table))
         (i 0))
    (dolist (ch chars)
      (setf (gethash ch to-idx) i)
      (setf (gethash i from-idx) ch)
      (setq i (+ i 1)))
    (torch:module :simple-tokenizer (list :chars chars
                                          :vocab-size (length chars)
                                          :char-to-idx to-idx
                                          :idx-to-char from-idx)
                  (function tokenizer-encode))))

(defun tokenizer-vocab-size (tokenizer) (torch:field tokenizer :vocab-size))

(defun tokenizer-encode (tokenizer text)
  ;; A string as a LIST of token ids. An unknown character encodes as 0, like
  ;; the book's `char_to_idx.get(ch, 0)`.
  (let ((table (torch:field tokenizer :char-to-idx)) (acc nil))
    (dotimes (i (length text) (reverse acc))
      (let ((id (gethash (char text i) table)))
        (setq acc (cons (if (null id) 0 id) acc))))))

(defun tokenizer-decode (tokenizer tokens)
  ;; Token ids -- a list, a linalg index array or a tensor -- back to a string.
  ;; An id outside the vocabulary contributes nothing, like the book's
  ;; `idx_to_char.get(int(idx), '')`.
  (let ((table (torch:field tokenizer :idx-to-char))
        (ids
         (cond ((consp tokens) tokens)
               ((torch:tensorp tokens)
                (linalg:to-list (linalg:flatten (torch:data tokens))))
               ((arrayp tokens) (linalg:to-list (linalg:flatten tokens)))
               (t (list tokens))))
        (out ""))
    (dolist (id ids out)
      (let ((ch (gethash (truncate id) table)))
        (unless (null ch) (setq out (concatenate 'string out (string ch))))))))


---

# FILE: references/examples/llm-from-scratch/gpt/trainer.lisp

;; gpt/trainer.lisp -- llm_from_scratch/gpt/trainer.py, ported.
;;
;; GPTTrainer: AdamW over two parameter GROUPS, a warmup-then-cosine learning
;; rate, gradient-norm clipping, and a step loop that evaluates on the held-out
;; split as it goes.
;;
;; Two deliberate differences from the book, both because the port would
;; otherwise carry a defect across:
;;
;;   * torch.optim's parameter GROUPS -- one weight-decayed, one not -- become
;;     TWO optimizers over the same model. The split itself is the book's, but
;;     decided by torch:module-kind rather than by `'bias' in name`: a
;;     substring test over dotted parameter names calls a layer named `blinear`
;;     a bias, while the kind is what the layer IS.
;;   * The book's get_lr multiplies by step/warmup_steps for the LOG LINE only;
;;     nothing writes that rate back, so its optimizer runs the whole warmup at
;;     the base rate and the printed schedule is not the one being trained
;;     with. Here gpt-trainer-lr is the single answer and the loop writes it
;;     into both optimizers, so the logged rate IS the rate.
;;
;; The elapsed-time column of the book's log line is dropped: it is the one
;; number that cannot come out the same on four backends, and this example is
;; checked by its output.

(load "model.lisp")
(load "dataset.lisp")

(defparameter *trainer-betas* '(0.9 0.95))

(defun gpt-parameter-groups (model)
  ;; The book's decay / no-decay split: a linear layer's WEIGHT decays;
  ;; its bias, every LayerNorm gain and bias, and every embedding table do not
  ;; -- which is exactly what `'bias' in name or 'ln' in name or 'embedding' in
  ;; name` selects, said in terms of the layer rather than of its name.
  ;; Returns (decay no-decay).
  (let ((decay nil) (no-decay nil))
    (gpt-apply model
               (lambda (sub)
                 (let ((kind (torch:module-kind sub)))
                   (do ((p (torch:fields sub) (cddr p)))
                       ((null p))
                     (let ((name (car p)) (v (cadr p)))
                       (when (and (torch:tensorp v) (torch:requires-grad-p v))
                         (if (and (eq kind :linear) (eq name :weight))
                             (setq decay (cons v decay))
                             (setq no-decay (cons v no-decay)))))))))
    (list (reverse decay) (reverse no-decay))))

(defun gpt-trainer (model train-loader val-loader &key (learning-rate 3.0e-4)
                          (weight-decay 0.1) (warmup-steps 1000)
                          (max-steps 10000) (grad-clip 1.0))
  ;; The trainer: the model, the two loaders, the schedule's knobs, and the two
  ;; AdamW optimizers the parameter split calls for. Both run at the same
  ;; learning rate -- only the decay differs, which is what a parameter group
  ;; is for.
  (let* ((groups (gpt-parameter-groups model))
         (decay (car groups))
         (no-decay (cadr groups)))
    (torch:module :gpt-trainer (list :model model
                                     :train-loader train-loader
                                     :val-loader val-loader
                                     :learning-rate learning-rate
                                     :warmup-steps warmup-steps
                                     :max-steps max-steps
                                     :grad-clip grad-clip
                                     :optimizers
                                     (list (torch:adamw decay
                                            :lr learning-rate
                                            :betas *trainer-betas*
                                            :weight-decay weight-decay)
                                           (torch:adamw no-decay
                                                        :lr learning-rate
                                                        :betas *trainer-betas*
                                                        :weight-decay 0.0)))
                  (function gpt-trainer-train))))

(defun gpt-trainer-lr (self step)
  ;; The learning rate AT step: a linear warmup from one warmup-step's worth of
  ;; the base rate up to it, then CosineAnnealingLR's
  ;; base * (1 + cos(pi * t / t-max)) / 2 over the remaining
  ;; t-max = max-steps - warmup-steps, decaying to 0 at the last step.
  (let* ((base (torch:field self :learning-rate))
         (warmup (torch:field self :warmup-steps))
         (total (torch:field self :max-steps)))
    (if (< step warmup)
        (/ (* base (+ step 1)) warmup)
        (let* ((t-max (max 1 (- total warmup)))
               (progress (min 1.0 (/ (* 1.0 (- step warmup)) t-max))))
          (* base 0.5 (+ 1.0 (cos (* pi progress))))))))

(defun gpt-trainer-set-lr (self lr)
  ;; The schedule's write: a learning rate is an ordinary optimizer FIELD, so
  ;; turning it needs no scheduler object.
  (dolist (o (torch:field self :optimizers)) (torch:set-field o :lr lr)))

(defun gpt-trainer-step (self x y)
  ;; One training step: forward, backward, clip, update -- and the loss as a
  ;; plain number, so nothing of the tape is retained past the step.
  (let ((loss (gpt-loss (torch:field self :model) x y))
        (clip (torch:field self :grad-clip)))
    (dolist (o (torch:field self :optimizers)) (torch:zero-grad o))
    (torch:backward loss)
    (when (> clip 0) (torch:clip-grad-norm (torch:field self :model) clip))
    (dolist (o (torch:field self :optimizers)) (torch:step o))
    (torch:item loss)))

(defun gpt-trainer-evaluate (self &key (max-batches 10))
  ;; The mean loss over up to max-batches validation batches, with the model in
  ;; EVALUATION mode (dropout off) and no tape built. The mode is restored
  ;; afterwards, like the book's model.eval() / model.train() pair.
  (let ((model (torch:field self :model))
        (loader (torch:field self :val-loader))
        (total 0.0)
        (count 0))
    (torch:eval model)
    (torch:no-grad
      (dolist (batch (data-loader-batches loader))
        (when (< count max-batches)
          (let ((pair (data-loader-collate loader batch)))
            (setq total
             (+ total (torch:item (gpt-loss model (car pair) (cadr pair)))))
            (setq count (+ count 1))))))
    (torch:train model)
    (if (= count 0) 0.0 (/ total count))))

(defun gpt-trainer-train (self &key (log-interval 100) (eval-interval 500))
  ;; The main loop: epochs over the training loader until max-steps is reached,
  ;; logging the running loss and the scheduled rate every log-interval steps
  ;; and evaluating every eval-interval. Returns (train-losses val-losses),
  ;; both newest last.
  (let ((model (torch:field self :model))
        (loader (torch:field self :train-loader))
        (max-steps (torch:field self :max-steps))
        (train-losses nil)
        (val-losses nil)
        (window nil)
        (step 0))
    (torch:train model)
    (format t "model parameters: ~a tensors~%"
            (length (torch:parameters model)))
    (do ()
        ((>= step max-steps))
      (dolist (batch (data-loader-batches loader))
        (when (< step max-steps)
          (gpt-trainer-set-lr self (gpt-trainer-lr self step))
          (let* ((pair (data-loader-collate loader batch))
                 (loss (gpt-trainer-step self (car pair) (cadr pair))))
            (setq train-losses (cons loss train-losses))
            (setq window (cons loss window))
            (when (= 0 (mod step log-interval))
              (format t "step ~a/~a | loss ~,4f | lr ~,6f~%" step max-steps
                      (gpt-mean window) (gpt-trainer-lr self step))
              (setq window nil))
            (when (and (> step 0) (= 0 (mod step eval-interval)))
              (let ((v (gpt-trainer-evaluate self)))
                (setq val-losses (cons v val-losses))
                (format t "validation loss ~,4f~%" v)))
            (setq step (+ step 1))))))
    (list (reverse train-losses) (reverse val-losses))))

(defun gpt-mean (values)
  ;; The mean of a list of numbers; 0.0 for the empty list.
  (let ((total 0.0) (n 0))
    (dolist (v values)
      (setq total (+ total v))
      (setq n (+ n 1)))
    (if (= n 0) 0.0 (/ total n))))


---

# FILE: references/examples/llm-from-scratch/transformer/attention.lisp

;; transformer/attention.lisp -- llm_from_scratch/transformer/attention.py,
;; ported.
;;
;; Dot-product attention, its scaled and masked form, one attention head and
;; multi-head attention. torch.bmm is `torch:matmul' (rank >= 3 is the stacked
;; product), nn.Module is `torch:module' plus a forward defun, and nn.ModuleList
;; is a plain LIST in a field -- torch:parameters recurses into it, so every
;; head's weights reach the optimizer.
;;
;; Loaded by transformer.lisp and by the chapter02 sections.

(defparameter *neg-infinity* (/ -1.0 0.0))

(defun dot-product-attention (query key value)
  ;; softmax(Q K^T) V over a (batch length d-model) batch: the scores are
  ;; (batch query-length key-length) and each query row's weights sum to 1.
  (torch:matmul
   (torch:softmax (torch:matmul query (torch:transpose key '(0 2 1))) :axis -1)
   value))

(defun scaled-dot-product-attention (query key value &optional mask)
  ;; The same with the 1/sqrt(d_k) scale, and an optional mask: every NON-ZERO
  ;; position is filled with -infinity before the softmax, so its weight comes
  ;; out exactly 0. The mask is a raw linalg array broadcasting over
  ;; (batch query-length key-length).
  (let* ((d-k (car (last (torch:shape query))))
         (score
          (torch:div (torch:matmul query (torch:transpose key '(0 2 1)))
                     (sqrt (* 1.0 d-k))))
         (masked
          (if (null mask) score (torch:masked-fill score mask *neg-infinity*))))
    (torch:matmul (torch:softmax masked :axis -1) value)))

(defun attention-head-forward (self query key value &optional mask)
  (scaled-dot-product-attention
   (torch:forward (torch:field self :linear-q) query)
   (torch:forward (torch:field self :linear-k) key)
   (torch:forward (torch:field self :linear-v) value) mask))

(defun attention-head (d-k d-v d-model)
  ;; One head: three projections of the (batch length d-model) input into the
  ;; head's own subspace, then scaled dot-product attention over them.
  (torch:module :attention-head (list :linear-q (torch:linear d-model d-k)
                                      :linear-k (torch:linear d-model d-k)
                                      :linear-v (torch:linear d-model d-v))
                (function attention-head-forward)))

(defun multi-head-attention-forward (self query key value &optional mask)
  (torch:forward (torch:field self :linear-o)
                 (torch:cat (mapcar (lambda (head)
                                      (torch:forward head query key value mask))
                                    (torch:field self :heads))
                            :axis -1)))

(defun multi-head-attention (n-heads d-k d-v d-model)
  ;; n-heads independent heads concatenated along the feature axis and mixed by
  ;; one more projection back to d-model.
  (let ((heads nil))
    (dotimes (i n-heads)
      (setq heads (cons (attention-head d-k d-v d-model) heads)))
    (torch:module :multi-head-attention (list :heads (reverse heads)
                                              :linear-o
                                              (torch:linear (* n-heads d-v)
                                                            d-model))
                  (function multi-head-attention-forward))))


---

# FILE: references/examples/llm-from-scratch/transformer/shapes.lisp

;; transformer/shapes.lisp -- the `if __name__ == "__main__"` block of
;; attention.py, plus the same check for the whole model.
;;
;; Everything printed here is a SHAPE or a token count, so the output is one
;; text on every backend regardless of the low-order float digits.
;;
;;   rontolisp transformer/shapes.lisp

(load "transformer.lisp")

(linalg:seed 42)

(defparameter *d-model* 16)

(defparameter *n-heads* 4)

(defparameter *d-k* (/ *d-model* *n-heads*))

(defparameter *d-v* *d-k*)

(defparameter *batch-size* 2)

(defparameter *query-len* 3)

(defparameter *key-len* 4)

(defparameter *query*
  (torch:tensor (linalg:randn (list *batch-size* *query-len* *d-model*))))

(defparameter *key*
  (torch:tensor (linalg:randn (list *batch-size* *key-len* *d-model*))))

(defparameter *value*
  (torch:tensor (linalg:randn (list *batch-size* *key-len* *d-model*))))

(format t "dot-product-attention:        ~a~%"
        (torch:shape (dot-product-attention *query* *key* *value*)))
(format t "scaled-dot-product-attention: ~a~%"
        (torch:shape (scaled-dot-product-attention *query* *key* *value*)))

(defparameter *multi-head*
  (multi-head-attention *n-heads* *d-k* *d-v* *d-model*))

(format t "multi-head-attention:         ~a~%"
        (torch:shape (torch:forward *multi-head* *query* *key* *value*)))
(format t "multi-head parameters:        ~a tensors~%"
        (length (torch:parameters *multi-head*)))

;; The whole model, at the shapes chapter 2 uses for its own smoke test.
(defparameter *model*
  (transformer 11 13 12 *d-model* 2 *n-heads* *d-k* *d-v* (* 2 *d-model*)))

(defparameter *src* (torch:pad-sequence '((2 5 6 7 3) (2 8 9 3))))

(defparameter *tgt* (torch:pad-sequence '((2 4 5 6 3) (2 7 8 3))))

(format t "transformer logits:           ~a~%"
        (torch:shape (torch:forward *model* *src* *tgt*)))
(format t "transformer parameters:       ~a tensors~%"
        (length (torch:parameters *model*)))
(format t "greedy decode length:         ~a tokens~%"
        (length
         (transformer-inference *model* (torch:pad-sequence '((2 5 6 7 3))) 2 3
                                :max-length 6)))


---

# FILE: references/examples/llm-from-scratch/transformer/transformer.lisp

;; transformer/transformer.lisp -- llm_from_scratch/transformer/transformer.py,
;; ported.
;;
;; The encoder/decoder Transformer of chapter 2: an encoder block (self
;; attention + feed forward, each wrapped in a residual and a LayerNorm), a
;; decoder block (masked self attention, source-target attention, feed forward),
;; the two stacks, the whole model, and the greedy `transformer-inference' loop.
;;
;; nn.Sequential is `torch:sequential' and nn.ReLU is `(function torch:relu)' --
;; torch:forward applies a plain function as readily as a module, so no
;; activation-module type exists in this package.

(load "attention.lisp")
(load "utils.lisp")

(defun encoder-block-forward (self x &optional src-padding-mask)
  (let* ((attended
          (torch:forward (torch:field self :layer-norm1)
                         (torch:add x
                                    (torch:forward (torch:field self :attention)
                                                   x x x src-padding-mask))))
         (fed
          (torch:forward (torch:field self :layer-norm2)
                         (torch:add attended
                                    (torch:forward
                                     (torch:field self :feed-forward)
                                     attended)))))
    fed))

(defun feed-forward-block (d-model d-ff)
  (torch:sequential (torch:linear d-model d-ff) (function torch:relu)
                    (torch:linear d-ff d-model)))

(defun encoder-block (d-model n-heads d-k d-v d-ff)
  (torch:module :encoder-block (list
                                :attention
                                (multi-head-attention n-heads d-k d-v d-model)
                                :layer-norm1 (torch:layer-norm d-model)
                                :feed-forward (feed-forward-block d-model d-ff)
                                :layer-norm2 (torch:layer-norm d-model))
                (function encoder-block-forward)))

(defun encoder-forward (self x &optional src-padding-mask)
  (let ((out
         (torch:forward (torch:field self :pe)
                        (torch:forward (torch:field self :embedding) x))))
    (dolist (block (torch:field self :blocks) out)
      (setq out (torch:forward block out src-padding-mask)))))

(defun encoder
    (vocabulary-size max-sequence-len d-model n-blocks n-heads d-k d-v d-ff)
  (let ((blocks nil))
    (dotimes (i n-blocks)
      (setq blocks (cons (encoder-block d-model n-heads d-k d-v d-ff) blocks)))
    (torch:module :encoder (list
                            :embedding (torch:embedding vocabulary-size d-model)
                            :pe (positional-encoding d-model max-sequence-len)
                            :blocks (reverse blocks))
                  (function encoder-forward))))

(defun decoder-block-forward
    (self x encoder-output &optional tgt-mask src-tgt-padding-mask)
  (let* ((self-attended
          (torch:forward (torch:field self :layer-norm1)
                         (torch:add x
                                    (torch:forward (torch:field self :attention)
                                                   x x x tgt-mask))))
         (cross-attended
          (torch:forward (torch:field self :layer-norm2)
                         (torch:add self-attended
                                    (torch:forward
                                     (torch:field self :attention-source-target)
                                     self-attended encoder-output encoder-output
                                     src-tgt-padding-mask))))
         (fed
          (torch:forward (torch:field self :layer-norm3)
                         (torch:add cross-attended
                                    (torch:forward
                                     (torch:field self :feed-forward)
                                     cross-attended)))))
    fed))

(defun decoder-block (d-model n-heads d-k d-v d-ff)
  (torch:module :decoder-block (list
                                :attention
                                (multi-head-attention n-heads d-k d-v d-model)
                                :layer-norm1 (torch:layer-norm d-model)
                                :attention-source-target
                                (multi-head-attention n-heads d-k d-v d-model)
                                :layer-norm2 (torch:layer-norm d-model)
                                :feed-forward (feed-forward-block d-model d-ff)
                                :layer-norm3 (torch:layer-norm d-model))
                (function decoder-block-forward)))

(defun decoder-forward
    (self x encoder-output &optional tgt-mask src-tgt-padding-mask)
  (let ((out
         (torch:forward (torch:field self :pe)
                        (torch:forward (torch:field self :embedding) x))))
    (dolist (block (torch:field self :blocks) out)
      (setq out
            (torch:forward block out encoder-output tgt-mask
                           src-tgt-padding-mask)))))

(defun decoder
    (vocabulary-size max-sequence-len d-model n-blocks n-heads d-k d-v d-ff)
  (let ((blocks nil))
    (dotimes (i n-blocks)
      (setq blocks (cons (decoder-block d-model n-heads d-k d-v d-ff) blocks)))
    (torch:module :decoder (list
                            :embedding (torch:embedding vocabulary-size d-model)
                            :pe (positional-encoding d-model max-sequence-len)
                            :blocks (reverse blocks))
                  (function decoder-forward))))

(defun transformer-forward
    (self src tgt &optional src-mask tgt-mask src-tgt-mask)
  (torch:forward (torch:field self :linear)
                 (torch:forward (torch:field self :decoder) tgt
                  (torch:forward (torch:field self :encoder) src src-mask)
                  tgt-mask src-tgt-mask)))

(defun transformer (src-vocab-size tgt-vocab-size max-sequence-len d-model
                                   n-blocks n-heads d-k d-v d-ff)
  ;; The whole model: an encoder over the source, a decoder over the target so
  ;; far, and one bias-free projection onto the target vocabulary.
  (torch:module :transformer (list :encoder
                                   (encoder src-vocab-size max-sequence-len
                                            d-model n-blocks n-heads d-k d-v
                                            d-ff)
                                   :decoder
                                   (decoder tgt-vocab-size max-sequence-len
                                            d-model n-blocks n-heads d-k d-v
                                            d-ff)
                                   :linear
                                   (torch:linear d-model tgt-vocab-size
                                                 :bias nil))
                (function transformer-forward)))

(defun transformer-inference
    (model src bos-token eos-token &key (max-length 20))
  ;; Greedy decoding, as a LIST of token ids beginning with bos-token: encode
  ;; the source once, then extend the target one argmax at a time until
  ;; eos-token or max-length. Wrapped in torch:no-grad -- nothing here needs a
  ;; tape, and building one would retain every intermediate.
  (torch:no-grad
    (let ((tokens (list bos-token))
          (encoder-output (torch:forward (torch:field model :encoder) src)))
      (dotimes (step max-length)
        (let* ((logits
                (torch:forward (torch:field model :linear)
                               (torch:forward (torch:field model :decoder)
                                              (torch:pad-sequence (list tokens))
                                              encoder-output)))
               (shape (torch:shape logits))
               (next
                (truncate
                 (torch:argmax
                  (torch:reshape (torch:slice logits
                                              (list nil
                                                    (list (- (cadr shape) 1)
                                                          (cadr shape)) nil))
                                 (list (caddr shape)))))))
          (setq tokens (append tokens (list next)))
          (when (= next eos-token) (return))))
      tokens)))


---

# FILE: references/examples/llm-from-scratch/transformer/utils.lisp

;; transformer/utils.lisp -- llm_from_scratch/transformer/utils.py, ported.
;;
;; Layer normalization and the sinusoidal positional encoding. The book's
;; free function layer_norm() is `layer-norm' below; its LayerNorm *class* is
;; the library's `torch:layer-norm' module, which is nn.LayerNorm's
;; (x - mean) / sqrt(var + eps) rather than the book's (x - mean) / (std + eps)
;; -- the two agree to eps, and the library follows PyTorch.
;;
;; Loaded by transformer.lisp and by the chapter02 sections; it defines no
;; top-level effect of its own.

(defun layer-norm (x &key (eps 1.0e-6))
  ;; The book's functional layer_norm: the last axis centred and divided by its
  ;; BIASED standard deviation (unbiased=False, i.e. :ddof 0). Composed from
  ;; torch ops, so it is differentiable end to end.
  (torch:div (torch:sub x (torch:mean x :axis -1 :keepdims t))
             (torch:add (torch:std x :axis -1 :keepdims t :ddof 0) eps)))

(defun sinusoidal-position-encoding (d-model sequence-length)
  ;; The "Attention Is All You Need" positional encoding as a RAW linalg array
  ;; of shape (1 sequence-length d-model) -- a constant buffer, so no tensor
  ;; and no gradient:
  ;;
  ;;   pe[pos, 2i]     = sin(pos / 10000^(2i / d_model))
  ;;   pe[pos, 2i + 1] = cos(pos / 10000^(2i / d_model))
  (let ((pe (linalg:zeros (list sequence-length d-model))))
    (dotimes (pos sequence-length)
      (do ((i 0 (+ i 2)))
          ((>= i d-model))
        (let ((angle (/ pos (expt 10000.0 (/ (* 1.0 i) d-model)))))
          (setf (aref pe pos i) (sin angle))
          (when (< (+ i 1) d-model) (setf (aref pe pos (+ i 1)) (cos angle))))))
    (linalg:expand-dims pe 0)))

(defun positional-encoding-forward (self x)
  ;; x * sqrt(d_model) + pe[:, :sequence_length]. The encoding is sliced to the
  ;; batch's own length and broadcasts over the batch axis.
  (let ((sequence-length (cadr (torch:shape x))))
    (torch:add (torch:mul x (torch:field self :scale))
               (linalg:slice (torch:field self :pe)
                             (list nil (list 0 sequence-length))))))

(defun positional-encoding (d-model max-sequence-length)
  ;; The PositionalEncoding module: the encoding table lives in the :pe field as
  ;; a raw array, which is exactly PyTorch's register_buffer -- torch:parameters
  ;; walks field VALUES and collects only parameter tensors, so a buffer is
  ;; reached by nothing and trained by nothing.
  (torch:module :positional-encoding (list :pe
                                           (sinusoidal-position-encoding d-model
                                            max-sequence-length)
                                           :scale (sqrt (* 1.0 d-model)))
                (function positional-encoding-forward)))


---

# FILE: references/examples/ml/deep-digits.lisp

;; A small deep neural network, numpy-style, with the linalg package.
;;
;; Learns to classify the ten digits 0-9 from 5x3 pixel bitmaps using a
;; 15 -> 16 -> 16 -> 10 multi-layer perceptron (two hidden leaky-ReLU
;; layers), trained by full-batch gradient descent with matrix
;; backpropagation and a 1/t learning-rate decay: every forward and
;; backward step is a linalg:matmul / transpose / elementwise operation
;; over the whole 10-sample batch at once, exactly like a numpy
;; implementation. Biases use the classic augmentation trick (a constant-1
;; column appended to each layer input).
;;
;; The example is fully deterministic and prints identical output on every
;; backend: weights are initialized from a hand-written linear congruential
;; generator with a fixed seed, leaky ReLU needs no transcendental
;; functions (float +/-/* are IEEE-identical everywhere), and the loss is
;; reported as a scaled integer to sidestep per-backend float printing
;; differences.
;;
;; Demonstrates: linalg:matmul, transpose, sub, mul (scalar broadcast and
;; Hadamard), emap, sum, argmax, from-list, shape.

;; --- deterministic pseudo-random weights ------------------------------------

(defvar *lcg-state* 42)

(defun lcg-next ()
  ;; A small Lehmer-style linear congruential generator (the ZX Spectrum
  ;; constants). Every intermediate value stays below 2^23, so the integer
  ;; arithmetic fits the WASM backend's i31 range and is deterministic on
  ;; every backend.
  (setq *lcg-state* (mod (+ (* *lcg-state* 75) 74) 65537))
  *lcg-state*)

(defun rand-matrix (rows cols)
  ;; A rows x cols matrix of floats uniformly spread in [-0.25, 0.25).
  (let ((m (make-array (list rows cols) :initial-element 0)))
    (do ((i 0 (+ i 1)))
        ((>= i rows) m)
      (do ((j 0 (+ j 1)))
          ((>= j cols))
        (setf (aref m i j) (- (/ (mod (lcg-next) 1000) 2000.0) 0.25))))))

;; --- matrix helpers on top of linalg -----------------------------------------

(defun with-bias (m)
  ;; Appends a constant-1 column: [m | 1].
  (let* ((shape (linalg:shape m))
         (rows (car shape))
         (cols (car (cdr shape)))
         (out (make-array (list rows (+ cols 1)) :initial-element 1)))
    (do ((i 0 (+ i 1)))
        ((>= i rows) out)
      (do ((j 0 (+ j 1)))
          ((>= j cols))
        (setf (aref out i j) (aref m i j))))))

(defun drop-last-col (m)
  ;; The inverse of with-bias: removes the last column.
  (let* ((shape (linalg:shape m))
         (rows (car shape))
         (cols (- (car (cdr shape)) 1))
         (out (make-array (list rows cols) :initial-element 0)))
    (do ((i 0 (+ i 1)))
        ((>= i rows) out)
      (do ((j 0 (+ j 1)))
          ((>= j cols))
        (setf (aref out i j) (aref m i j))))))

(defun mat-row (m i)
  ;; Row i of a matrix as a fresh vector.
  (let* ((cols (car (cdr (linalg:shape m))))
         (v (make-array cols :initial-element 0)))
    (do ((j 0 (+ j 1)))
        ((>= j cols) v)
      (setf (aref v j) (aref m i j)))))

(defun leaky-relu (m)
  ;; max(x, 0.1x): the small negative slope keeps every unit trainable
  ;; (a plain ReLU here dies with an unlucky init and a whole layer stuck
  ;; at zero gradient).
  (linalg:emap (lambda (x) (if (> x 0) x (* 0.1 x))) m))

(defun leaky-relu-mask (m)
  ;; The derivative of leaky-relu: 1 for positive pre-activations, else 0.1.
  (linalg:emap (lambda (x) (if (> x 0) 1 0.1)) m))

(defun sq-sum (m) (linalg:sum (linalg:emap (lambda (x) (* x x)) m)))

;; --- the dataset: 5x3 pixel bitmaps of the digits 0-9 ------------------------

(defun digit-bitmaps ()
  ;; One row per digit, 15 pixels each (row-major 5x3).
  (linalg:from-list
   '((1 1 1 1 0 1 1 0 1 1 0 1 1 1 1)    ; 0
     (0 1 0 1 1 0 0 1 0 0 1 0 1 1 1)    ; 1
     (1 1 1 0 0 1 1 1 1 1 0 0 1 1 1)    ; 2
     (1 1 1 0 0 1 1 1 1 0 0 1 1 1 1)    ; 3
     (1 0 1 1 0 1 1 1 1 0 0 1 0 0 1)    ; 4
     (1 1 1 1 0 0 1 1 1 0 0 1 1 1 1)    ; 5
     (1 1 1 1 0 0 1 1 1 1 0 1 1 1 1)    ; 6
     (1 1 1 0 0 1 0 0 1 0 0 1 0 0 1)    ; 7
     (1 1 1 1 0 1 1 1 1 1 0 1 1 1 1)    ; 8
     (1 1 1 1 0 1 1 1 1 0 0 1 1 1 1)))) ; 9

(defun one-hot-targets ()
  (let ((y (make-array '(10 10) :initial-element 0)))
    (do ((i 0 (+ i 1)))
        ((>= i 10) y)
      (setf (aref y i i) 1))))

;; --- the network --------------------------------------------------------------

(defun forward (a0 w1 w2 w3)
  ;; Returns (z1 a1b z2 a2b out): pre-activations, biased activations, output.
  (let* ((z1 (linalg:matmul a0 w1))
         (a1b (with-bias (leaky-relu z1)))
         (z2 (linalg:matmul a1b w2))
         (a2b (with-bias (leaky-relu z2)))
         (out (linalg:matmul a2b w3)))
    (list z1 a1b z2 a2b out)))

(defun predict (x w1 w2 w3)
  ;; Class = argmax of the output row for each sample.
  (let* ((state (forward (with-bias x) w1 w2 w3))
         (out (nth 4 state))
         (n (car (linalg:shape out)))
         (preds nil))
    (do ((i (- n 1) (- i 1)))
        ((< i 0) preds)
      (setq preds (cons (linalg:argmax (mat-row out i)) preds)))))

(defun main ()
  (let* ((x (digit-bitmaps))
         (y (one-hot-targets))
         (a0 (with-bias x)) ; 10x16 input batch
         (n (car (linalg:shape x)))
         (lr0 0.25)
         (w1 (rand-matrix 16 16))  ; 15 pixels + bias -> 16
         (w2 (rand-matrix 17 16))  ; 16 hidden + bias -> 16
         (w3 (rand-matrix 17 10))) ; 16 hidden + bias -> 10 classes
    (format t
     "network: 15 -> 16 (leaky relu) -> 16 (leaky relu) -> 10, ~a samples~%" n)
    (format t "input batch shape (with bias): ~a~%~%" (linalg:shape a0))
    (do ((epoch 1 (+ epoch 1)))
        ((> epoch 500))
      (let* ((lr (/ lr0 (+ 1 (/ epoch 100.0)))) ; 1/t learning-rate decay
             (state (forward a0 w1 w2 w3))
             (z1 (nth 0 state))
             (a1b (nth 1 state))
             (z2 (nth 2 state))
             (a2b (nth 3 state))
             (out (nth 4 state))
             (diff (linalg:sub out y))
             ;; Backpropagation, all as matrix products:
             ;; dOut = 2/n * (out - y)
             (dout (linalg:mul (/ 2.0 n) diff))
             (dw3 (linalg:matmul (linalg:transpose a2b) dout))
             (dz2
              (linalg:mul
               (drop-last-col (linalg:matmul dout (linalg:transpose w3)))
               (leaky-relu-mask z2)))
             (dw2 (linalg:matmul (linalg:transpose a1b) dz2))
             (dz1
              (linalg:mul
               (drop-last-col (linalg:matmul dz2 (linalg:transpose w2)))
               (leaky-relu-mask z1)))
             (dw1 (linalg:matmul (linalg:transpose a0) dz1)))
        (setq w3 (linalg:sub w3 (linalg:mul lr dw3)))
        (setq w2 (linalg:sub w2 (linalg:mul lr dw2)))
        (setq w1 (linalg:sub w1 (linalg:mul lr dw1)))
        (when (= (mod epoch 100) 0)
          ;; MSE loss, scaled to an integer so the output is identical on
          ;; every backend (float printing differs, float arithmetic does not).
          (format t "epoch ~a  loss (x1e6): ~a~%" epoch
                  (round (* 1000000 (/ (sq-sum diff) n)))))))
    (terpri)
    (let ((preds (predict x w1 w2 w3)) (correct 0))
      (do ((i 0 (+ i 1)) (rest preds (cdr rest)))
          ((>= i 10))
        (when (= (car rest) i) (setq correct (+ correct 1))))
      (format t "predictions on the training digits: ~a~%" preds)
      (format t "accuracy: ~a/10~%~%" correct))
    ;; Generalization: flip one deterministic pixel per bitmap and classify
    ;; again. Two of the corrupted bitmaps are genuinely ambiguous, so the
    ;; two "misses" are reasonable answers: 0 with its centre pixel filled
    ;; in is exactly the bitmap of 8, and 8 with one left-edge pixel cleared
    ;; is one pixel away from both 8 and 2.
    (let ((noisy (linalg:emap (lambda (p) p) (digit-bitmaps))))
      (do ((i 0 (+ i 1)))
          ((>= i 10))
        (let ((j (mod (* 7 (+ i 1)) 15)))
          (setf (aref noisy i j) (- 1 (aref noisy i j)))))
      (format t "predictions with one flipped pixel:  ~a~%"
              (predict noisy w1 w2 w3)))))

(main)


---

# FILE: references/examples/ml/heat3d.lisp

;; Heat diffusion in a 3-D voxel grid (rank-3 arrays).
;;
;; A 5x5x5 lattice starts with all its heat (1000 units) in the centre voxel
;; and diffuses it step by step to the six axis neighbours, with insulated
;; (no-flux) walls. Because rontolisp arithmetic is exact for integer and
;; rational inputs, every voxel value is an exact ratio and the total heat
;; stays *exactly* 1000 after every step -- identical on every backend.
;; The run stops after 4 steps: with alpha = 1/8 the denominators grow as
;; 8^step, and 4 steps keep every intermediate ratio inside the WASM
;; backend's i31 fixnum range (see doc/en/guides/math-backends.md).
;;
;; Demonstrates: rank-3 make-array / aref / (setf (aref ...)), the #nA
;; printed syntax, array-rank / array-dimensions / array-total-size,
;; row-major-aref for flat scans (and decoding a flat index back into
;; subscripts), and the rank-generic linalg operations (reshape, add, sum,
;; amax, array-equal) over rank-3 tensors.

(defun diffuse (grid alpha)
  ;; One explicit Euler step with insulated boundaries:
  ;; new[c] = old[c] + alpha * sum over the axis neighbours n of (old[n] - old[c]).
  ;; alpha <= 1/6 keeps the step stable (a voxel has at most 6 neighbours).
  (let* ((d (array-dimensions grid))
         (nx (car d))
         (ny (car (cdr d)))
         (nz (car (cdr (cdr d))))
         (out (make-array d :initial-element 0)))
    (dotimes (i nx)
      (dotimes (j ny)
        (dotimes (k nz)
          (let ((c (aref grid i j k)) (acc 0))
            (when (> i 0) (setq acc (+ acc (- (aref grid (- i 1) j k) c))))
            (when (< i (- nx 1))
              (setq acc (+ acc (- (aref grid (+ i 1) j k) c))))
            (when (> j 0) (setq acc (+ acc (- (aref grid i (- j 1) k) c))))
            (when (< j (- ny 1))
              (setq acc (+ acc (- (aref grid i (+ j 1) k) c))))
            (when (> k 0) (setq acc (+ acc (- (aref grid i j (- k 1)) c))))
            (when (< k (- nz 1))
              (setq acc (+ acc (- (aref grid i j (+ k 1)) c))))
            (setf (aref out i j k) (+ c (* alpha acc)))))))
    out))

(defun hottest (grid)
  ;; The (i j k) of the hottest voxel: a flat row-major scan with
  ;; row-major-aref (no nested subscript loops), then the flat index decoded
  ;; back into subscripts. array-row-major-index is the inverse direction.
  (let ((n (array-total-size grid)) (best 0))
    (do ((f 1 (+ f 1)))
        ((>= f n))
      (when (> (row-major-aref grid f) (row-major-aref grid best))
        (setq best f)))
    (let* ((d (array-dimensions grid))
           (ny (car (cdr d)))
           (nz (car (cdr (cdr d))))
           (k (mod best nz))
           (jj (/ (- best k) nz))
           (j (mod jj ny))
           (i (/ (- jj j) ny)))
      (list i j k))))

(defun heat-char (v top)
  ;; A one-character heat map bucket for v relative to the hottest value.
  (cond ((= v 0) " ") ((>= v (/ top 2)) "#") ((>= v (/ top 8)) "+") (t ".")))

(defun print-middle-slice (grid)
  ;; ASCII rendering of the z = middle slice.
  (let* ((d (array-dimensions grid))
         (nx (car d))
         (ny (car (cdr d)))
         (mid (/ (- (car (cdr (cdr d))) 1) 2))
         (top (linalg:amax grid)))
    (dotimes (i nx)
      (dotimes (j ny) (princ (heat-char (aref grid i j mid) top)))
      (terpri))))

(defun main ()
  ;; A small rank-3 tensor tour first: #3A printing, introspection, and the
  ;; rank-generic linalg elementwise operations.
  (let ((tensor (linalg:reshape (linalg:arange 8) '(2 2 2))))
    (format t "tensor:      ~a~%" tensor)
    (format t "rank/dims:   ~a ~a (~a elements)~%" (array-rank tensor)
            (array-dimensions tensor) (array-total-size tensor))
    (format t "add 10:      ~a~%" (linalg:add tensor 10))
    (format t "flat [1 0 1]: index ~a value ~a~%"
            (array-row-major-index tensor 1 0 1)
            (row-major-aref tensor (array-row-major-index tensor 1 0 1)))
    (format t "round trip:  ~a~%~%"
            (linalg:array-equal
             (linalg:reshape (linalg:flatten tensor) '(2 2 2)) tensor)))
  ;; The simulation: all heat starts in the centre voxel.
  (let ((grid (make-array '(5 5 5) :initial-element 0)) (alpha (/ 1 8)))
    (setf (aref grid 2 2 2) 1000)
    (dotimes (step 4)
      (format t "step ~a: total ~a, centre ~a, hottest ~a~%" step
              (linalg:sum grid) (aref grid 2 2 2) (hottest grid))
      (print-middle-slice grid)
      (setq grid (diffuse grid alpha)))
    (format t "step 4: total ~a, centre ~a, hottest ~a~%" (linalg:sum grid)
            (aref grid 2 2 2) (hottest grid))
    (print-middle-slice grid)
    ;; Insulated walls conserve heat exactly: the sum is still the integer
    ;; 1000 after every step, never a float epsilon away.
    (format t "conserved:   ~a~%" (= (linalg:sum grid) 1000))))

(main)


---

# FILE: references/examples/ml/linear-regression.lisp

;; Least-squares polynomial fitting with the linalg package.
;;
;; Fits a quadratic y = c0 + c1*x + c2*x^2 to five sample points by solving
;; the normal equations (A^T A) c = A^T y, where A is the Vandermonde matrix
;; of the sample xs. linalg computes in packed double-float (see the linear
;; algebra guide), so the fitted coefficients and residuals are floats. A
;; non-terminating float prints at fewer significant digits on WASM than on
;; the interpreter and JVM, so every float result below is reported as a
;; cross-backend-stable integer scaled by 1000 (round(x * 1000)); the integer
;; Vandermonde matrix and the integer-valued normal matrix print directly.
;;
;; Demonstrates: linalg:from-list, shape, transpose, matmul, dot, solve,
;; det, norm, sub, and element access with aref.

(defun vandermonde (xs degree)
  ;; One row per sample: (1 x x^2 ... x^degree).
  (let* ((n (length xs)) (m (make-array (list n (+ degree 1)))))
    (do ((row 0 (+ row 1)) (rest xs (cdr rest)))
        ((>= row n) m)
      (do ((col 0 (+ col 1)))
          ((> col degree))
        (setf (aref m row col) (expt (car rest) col))))))

(defun fit-polynomial (xs ys degree)
  ;; Solves (A^T A) c = A^T y for the coefficient vector c.
  (let* ((a (vandermonde xs degree))
         (at (linalg:transpose a))
         (ata (linalg:matmul at a))
         (aty (linalg:dot at (linalg:from-list ys))))
    (linalg:solve ata aty)))

(defun poly-eval (coeffs x)
  ;; Evaluates c0 + c1*x + ... at x (Horner's method).
  (let ((acc 0))
    (do ((i (- (length coeffs) 1) (- i 1)))
        ((< i 0) acc)
      (setq acc (+ (aref coeffs i) (* acc x))))))

(defun scaled (x)
  ;; Rounds a float to a cross-backend-stable integer (x * 1000). A
  ;; non-terminating float prints differently on WASM; the scaled integer
  ;; does not.
  (round (* x 1000)))

(defun main ()
  (let* ((xs '(-2 -1 0 1 2))
         (ys '(12 4 2 0 3))
         (degree 2)
         (a (vandermonde xs degree))
         (at (linalg:transpose a))
         (ata (linalg:matmul at a))
         (aty (linalg:dot at (linalg:from-list ys)))
         (coeffs (fit-polynomial xs ys degree)))
    (format t "samples:      xs=~a ys=~a~%" xs ys)
    (format t "vandermonde:  ~a  shape ~a~%" a (linalg:shape a))
    (format t "normal matrix A^T A: ~a  (det ~a)~%" ata (linalg:det ata))
    (format t "coefficients x1000 (c0 c1 c2): ~a~%"
            (mapcar #'scaled (coerce coeffs 'list)))
    ;; A floating-point least-squares fit: A^T A . c reproduces A^T y to
    ;; within rounding, so the residual of the normal equations is ~0.
    (format t "solution residual < 1e-6: ~a~%"
            (< (linalg:norm (linalg:sub (linalg:dot ata coeffs) aty)) 0.000001))
    (terpri)
    (format t "    x   y   fitted x1000   residual x1000~%")
    (do ((px xs (cdr px)) (py ys (cdr py)))
        ((null px))
      (let ((fitted (poly-eval coeffs (car px))))
        (format t "  ~a  ~a  ~a  ~a~%" (car px) (car py) (scaled fitted)
                (scaled (- (car py) fitted)))))
    (terpri)
    (let ((residual (linalg:sub (linalg:from-list ys) (linalg:dot a coeffs))))
      (format t "squared residual norm x1000: ~a~%"
              (scaled (linalg:dot residual residual))))))

(main)


---

# FILE: references/examples/ml/maze-rl.lisp

;;;; Tabular Q-learning that solves a grid maze, in rontolisp.
;;;;
;;;; An agent learns to walk from S to G through a maze of walls (#) by trial
;;;; and error.  The action-value function Q(state, action) is stored in a
;;;; hash table keyed by (row col action) with the standard `equal` test, which
;;;; is the idiomatic Common Lisp representation for a sparse Q-table.
;;;;
;;;; Exploration uses the built-in `random`, the idiomatic Common Lisp choice for
;;;; reinforcement learning.  Because `random` is not seedable and draws from a
;;;; different source per backend (`Math.random` on the interpreter/JVM, the WASI
;;;; `random_get` host function on WASM, real entropy in every mode), this
;;;; program prints a different learned path on each run and each backend.  The
;;;; algorithm still converges to a valid shortest-ish route every time; only the
;;;; exact path and value count vary.  (For a fully deterministic, bit-identical
;;;; cross-backend run, swap `random` for a self-contained LCG threaded through
;;;; as state -- earlier revisions of this file did exactly that.)

;;; ---------------------------------------------------------------------------
;;; Maze: a list of equal-length strings.  # is a wall, S the start, G the goal,
;;; . an open cell. A state is the list (row col).
;;; ---------------------------------------------------------------------------

(defun maze-rows (maze) (length maze))
(defun maze-cols (maze) (length (first maze)))
(defun maze-ref (maze r c) (char (nth r maze) c))

(defun wall-p (maze r c)
  (or (< r 0) (< c 0) (>= r (maze-rows maze)) (>= c (maze-cols maze))
      (char= (maze-ref maze r c) #\#)))

;; Scan the grid for the first cell holding CH, returning (row col) or nil.
(defun find-cell (maze ch)
  (let ((r 0) (rows (maze-rows maze)) (found nil))
    (while (and (< r rows) (null found))
      (let ((c 0) (cols (maze-cols maze)))
        (while (and (< c cols) (null found))
          (when (char= (maze-ref maze r c) ch) (setq found (list r c)))
          (setq c (+ c 1))))
      (setq r (+ r 1)))
    found))

;;; ---------------------------------------------------------------------------
;;; Actions: 0 up, 1 down, 2 left, 3 right.
;;; ---------------------------------------------------------------------------

(defun move (r c a)
  (cond ((= a 0) (list (- r 1) c))
        ((= a 1) (list (+ r 1) c))
        ((= a 2) (list r (- c 1)))
        (t (list r (+ c 1)))))

;;; ---------------------------------------------------------------------------
;;; Q-table: a hash table keyed by (row col action), default value 0.0.
;;; ---------------------------------------------------------------------------

(defun q-get (q r c a) (gethash (list r c a) q 0.0))
(defun q-set (q r c a v) (setf (gethash (list r c a) q) v))

(defun max-q (q r c) ; best action-value at (r c)
  (let ((best (q-get q r c 0)) (a 1))
    (while (< a 4)
      (let ((v (q-get q r c a))) (when (> v best) (setq best v)))
      (setq a (+ a 1)))
    best))

(defun best-action (q r c) ; argmax action (ties -> lowest index)
  (let ((ba 0) (bv (q-get q r c 0)) (a 1))
    (while (< a 4)
      (let ((v (q-get q r c a)))
        (when (> v bv)
          (setq bv v)
          (setq ba a)))
      (setq a (+ a 1)))
    ba))

(defun choose-action (q r c epsilon) ; epsilon-greedy
  (if (< (random 1.0) epsilon) (random 4) (best-action q r c)))

;;; ---------------------------------------------------------------------------
;;; Hyper-parameters travel together as a list (alpha gamma epsilon max-steps),
;;; which keeps each function's arity small (rontolisp's WASM backend allows up
;;; to seven parameters) and reads more clearly than a long positional list.
;;; ---------------------------------------------------------------------------

(defun hp-alpha (hp) (first hp))
(defun hp-gamma (hp) (second hp))
(defun hp-epsilon (hp) (third hp))
(defun hp-max-steps (hp) (fourth hp))

;;; ---------------------------------------------------------------------------
;;; One episode of Q-learning from the start cell, returning the step count.
;;; Q(s,a) <- Q(s,a) + alpha * (reward + gamma * max_a' Q(s',a') - Q(s,a))
;;; ---------------------------------------------------------------------------

(defun run-episode (maze q start goal hp)
  (let ((r (first start))
        (c (second start))
        (steps 0)
        (done nil)
        (alpha (hp-alpha hp))
        (gamma (hp-gamma hp))
        (epsilon (hp-epsilon hp))
        (max-steps (hp-max-steps hp)))
    (while (and (< steps max-steps) (not done))
      (let* ((a (choose-action q r c epsilon))
             (nxt (move r c a))
             (nr (first nxt))
             (nc (second nxt)))
        (when (wall-p maze nr nc) ; blocked: stay put
          (setq nr r)
          (setq nc c))
        (let* ((at-goal (and (= nr (first goal)) (= nc (second goal))))
               (reward (if at-goal 10.0 -0.1))
               (old (q-get q r c a))
               (future (if at-goal 0.0 (max-q q nr nc)))
               (target (+ reward (* gamma future)))
               (updated (+ old (* alpha (- target old)))))
          (q-set q r c a updated)
          (setq r nr)
          (setq c nc)
          (setq steps (+ steps 1))
          (when at-goal (setq done t)))))
    steps))

(defun train (maze q start goal hp episodes)
  (let ((e 0))
    (while (< e episodes)
      (run-episode maze q start goal hp)
      (setq e (+ e 1)))))

;;; ---------------------------------------------------------------------------
;;; Follow the greedy policy from start to goal, collecting the visited cells.
;;; ---------------------------------------------------------------------------

(defun greedy-path (maze q start goal max-steps)
  (let ((r (first start))
        (c (second start))
        (steps 0)
        (done nil)
        (cells (list start)))
    (while (and (< steps max-steps) (not done))
      (if (and (= r (first goal)) (= c (second goal)))
          (setq done t)
          (let* ((a (best-action q r c))
                 (nxt (move r c a))
                 (nr (first nxt))
                 (nc (second nxt)))
            (when (wall-p maze nr nc)
              (setq nr r)
              (setq nc c))
            (setq r nr)
            (setq c nc)
            (setq cells (cons (list r c) cells))
            (setq steps (+ steps 1)))))
    (reverse cells)))

;;; ---------------------------------------------------------------------------
;;; Render the maze with the learned path drawn as o.
;;; ---------------------------------------------------------------------------

(defun print-solution (maze path)
  (let ((r 0) (rows (maze-rows maze)))
    (while (< r rows)
      (let ((c 0) (cols (maze-cols maze)))
        (while (< c cols)
          (let ((ch (maze-ref maze r c)))
            (cond ((char= ch #\#) (princ "#"))
                  ((char= ch #\S) (princ "S"))
                  ((char= ch #\G) (princ "G"))
                  ((member (list r c) path :test #'equal) (princ "o"))
                  (t (princ "."))))
          (setq c (+ c 1))))
      (terpri)
      (setq r (+ r 1)))))

;;; ---------------------------------------------------------------------------
;;; Run
;;; ---------------------------------------------------------------------------

;; A 21x21 perfect maze (recursive backtracking, seed 20260626) with the goal at
;; the farthest reachable cell from S, so the unique solution is 132 steps long.
(defparameter *maze*
  (list "#####################" "#S#...#.....#.......#" "#.#.#.###.###.###.#.#"
        "#...#...#.....#G#.#.#" "#######.###.###.#.###" "#.....#...#.#...#...#"
        "#.#######.#.#.#.###.#" "#.#...#...#...#.#...#" "#.#.#.#.#########.#.#"
        "#...#...#...#.....#.#" "#.#######.#.#.#######" "#...#.....#.#.......#"
        "###.#####.#.#.#####.#" "#...#.....#.#...#.#.#" "#.###.#####.###.#.#.#"
        "#.....#.....#.....#.#" "#######.###########.#" "#.#...#.............#"
        "#.#.#.#############.#" "#...#...............#"
        "#####################"))

(defparameter *q* (make-hash-table :test 'equal))
(defparameter *start* (find-cell *maze* #\S))
(defparameter *goal* (find-cell *maze* #\G))
(defparameter *hp* (list 0.5 0.97 0.2 2000)) ; alpha gamma epsilon max-steps

(format t "Maze ~a x ~a, training tabular Q-learning...~%" (maze-rows *maze*)
        (maze-cols *maze*))

(train *maze* *q* *start* *goal* *hp* 8000)

(defparameter *path* (greedy-path *maze* *q* *start* *goal* 400))

(format t "Learned ~a state-action values~%" (hash-table-count *q*))
(format t "Greedy path length: ~a steps~%" (- (length *path*) 1))
(print-solution *maze* *path*)


---

# FILE: references/examples/ml/mlp.lisp

;;;; Generalized multi-layer perceptron in rontolisp
;;;; A network is a list of layers; each layer is (W b):
;;;;   W = out x in weight matrix (rank-2 array), b = out-length bias vector.
;;;; Task: binary classification of 2-D points (inside vs. outside a circle),
;;;; a non-linearly-separable problem. We train with SGD + backprop and
;;;; report accuracy on a held-out test set.
;;;;
;;;; Vectors are rank-1 arrays and matrices are rank-2 arrays, so the forward
;;;; and backward passes are plain indexed loops with O(1) aref access and
;;;; in-place weight updates. A layer's shape is read off its bias and input
;;;; vectors via (length v), so no dimensions are threaded through the code.

;;; ---------------------------------------------------------------------------
;;; Randomness via the built-in random.  random returns a value in [0, limit)
;;; of the limit's type; on the interpreter/JVM it draws from Math.random(),
;;; on WASM from the WASI random_get host function (so every run differs).
;;; ---------------------------------------------------------------------------

(defun random-weight () (- (random 1.0) 0.5)) ; -> (-0.5, 0.5)

;;; ---------------------------------------------------------------------------
;;; Array construction
;;; ---------------------------------------------------------------------------

(defun random-vector (n)
  (let ((v (make-array n :initial-element 0.0)))
    (dotimes (i n) (setf (aref v i) (random-weight)))
    v))

(defun random-matrix (rows cols)
  (let ((m (make-array (list rows cols) :initial-element 0.0)))
    (dotimes (i rows) (dotimes (j cols) (setf (aref m i j) (random-weight))))
    m))

(defun list->vector (lst) ; pack a list into a rank-1 array
  (let ((v (make-array (length lst) :initial-element 0.0)) (i 0))
    (dolist (e lst)
      (setf (aref v i) e)
      (setq i (+ i 1)))
    v))

;;; ---------------------------------------------------------------------------
;;; Activation and one layer:  a = sigmoid(W x + b)
;;; ---------------------------------------------------------------------------

(defun sigmoid (x) (/ 1.0 (+ 1.0 (exp (- 0.0 x)))))

(defun layer-forward (w b x)
  (let* ((rows (length b))
         (cols (length x))
         (a (make-array rows :initial-element 0.0)))
    (dotimes (i rows)
      (let ((s (aref b i)))
        (dotimes (j cols) (incf s (* (aref w i j) (aref x j))))
        (setf (aref a i) (sigmoid s))))
    a))

;;; ---------------------------------------------------------------------------
;;; Network construction
;;; ---------------------------------------------------------------------------

(defun layer-w (layer) (first layer))
(defun layer-b (layer) (second layer))

;; sizes = (n0 n1 ... nL): build one layer per consecutive pair.
(defun init-layers (sizes)
  (if (null (rest sizes))
      nil
      (cons (list (random-matrix (second sizes) (first sizes))
                  (random-vector (second sizes))) (init-layers (rest sizes)))))

;;; ---------------------------------------------------------------------------
;;; Forward pass.  Collect every activation, input first: (a0=x a1 ... aL)
;;; ---------------------------------------------------------------------------

(defun forward-all (layers x)
  (let ((acts (list x)) (a x))
    (dolist (layer layers)
      (setq a (layer-forward (layer-w layer) (layer-b layer) a))
      (setq acts (cons a acts)))
    (reverse acts)))

(defun predict (layers x) ; output activation vector
  (car (last (forward-all layers x))))

;;; ---------------------------------------------------------------------------
;;; Backpropagation over a single example (updates the layers in place).
;;; Walk layers from the output backward, carrying the running delta vector.
;;; The new delta must be computed from W before W is updated.
;;; ---------------------------------------------------------------------------

(defun output-delta (aL y) ; (aL - y) * aL * (1 - aL)
  (let* ((n (length aL)) (delta (make-array n :initial-element 0.0)))
    (dotimes (i n)
      (setf (aref delta i)
            (* (- (aref aL i) (aref y i)) (aref aL i) (- 1.0 (aref aL i)))))
    delta))

(defun train-example (layers x y lr)
  (let* ((acts (forward-all layers x)) ; (a0 ... aL)
         (rev-layers (reverse layers)) ; (layerL ... layer1)
         (rev-acts (reverse acts))     ; (aL ... a0)
         (delta (output-delta (first rev-acts) y))
         (cur (rest rev-acts))) ; (a_{prev} ... a0)
    (dolist (layer rev-layers)
      (let* ((w (layer-w layer))
             (b (layer-b layer))
             (a-prev (first cur))
             (rows (length b))
             (cols (length a-prev))
             (new-delta (make-array cols :initial-element 0.0)))
        ;; new-delta = (W^T delta) * a-prev * (1 - a-prev), read W before updating
        (dotimes (j cols)
          (let ((s 0.0))
            (dotimes (i rows) (incf s (* (aref w i j) (aref delta i))))
            (setf (aref new-delta j)
                  (* s (aref a-prev j) (- 1.0 (aref a-prev j))))))
        ;; descend: W -= lr * delta (outer) a-prev, b -= lr * delta
        (dotimes (i rows)
          (dotimes (j cols)
            (decf (aref w i j) (* lr (aref delta i) (aref a-prev j))))
          (decf (aref b i) (* lr (aref delta i))))
        (setq delta new-delta)
        (setq cur (rest cur))))
    layers))

;;; ---------------------------------------------------------------------------
;;; Loss, accuracy, training loop
;;; ---------------------------------------------------------------------------

(defun example-loss (layers ex)
  (let* ((a (predict layers (first ex))) (y (second ex)) (s 0.0))
    (dotimes (i (length a))
      (let ((d (- (aref a i) (aref y i)))) (incf s (* d d))))
    (* 0.5 s)))

(defun total-loss (layers data)
  (let ((s 0.0))
    (dolist (ex data) (incf s (example-loss layers ex)))
    s))

(defun classify (layers x) ; threshold the single output
  (if (> (aref (predict layers x) 0) 0.5) 1.0 0.0))

(defun accuracy (layers data)
  (let ((correct 0))
    (dolist (ex data)
      (when (= (classify layers (first ex)) (aref (second ex) 0))
        (setq correct (+ correct 1))))
    (/ (float correct) (length data))))

(defun train (layers data epochs lr)
  (let ((e 0))
    (while (< e epochs)
      (dolist (ex data) (train-example layers (first ex) (second ex) lr))
      (when (zerop (mod e 200))
        (format t "epoch ~a  loss ~a  train-acc ~a~%" e (total-loss layers data)
                (accuracy layers data)))
      (setq e (+ e 1))))
  layers)

;;; ---------------------------------------------------------------------------
;;; Synthetic data: point is class 1 if inside circle radius^2 < 0.5,
;;; coordinates drawn uniformly from [-1, 1)^2 via (- (random 2.0) 1.0).
;;; Each example is (input-vector target-vector), both rank-1 arrays.
;;; ---------------------------------------------------------------------------

(defun make-point ()
  (let* ((x (- (random 2.0) 1.0))
         (y (- (random 2.0) 1.0))
         (label (if (< (+ (* x x) (* y y)) 0.5) 1.0 0.0)))
    (list (list->vector (list x y)) (list->vector (list label)))))

(defun make-dataset (n)
  (let ((acc nil) (i 0))
    (while (< i n)
      (setq acc (cons (make-point) acc))
      (setq i (+ i 1)))
    acc))

;;; ---------------------------------------------------------------------------
;;; Run
;;; ---------------------------------------------------------------------------

(defparameter *train* (make-dataset 150))
(defparameter *test* (make-dataset 60))

(format t "Circle classification, 2-8-1 MLP~%")
(format t "train=~a examples, test=~a examples~%~%" (length *train*)
        (length *test*))

(defparameter *net* (init-layers (list 2 8 1)))
(setq *net* (train *net* *train* 2000 0.5))

(format t "~%Final train accuracy: ~a~%" (accuracy *net* *train*))
(format t "Final test  accuracy: ~a~%" (accuracy *net* *test*))


---

# FILE: references/examples/ml/nn-vec.lisp

;;;; Feed-forward neural network in rontolisp -- the vec / linalg version of
;;;; nn.lisp. Learns XOR with backprop + gradient descent, 2 -> 4 -> 1.
;;;;
;;;; nn.lisp writes every layer as explicit indexed loops over rank-1/rank-2
;;;; arrays. This version writes NONE of that arithmetic by hand: the whole
;;;; forward and backward pass is expressed with the vec and linalg packages,
;;;; numpy-style, so each line reads as the math it implements.
;;;;
;;;;   - vec: the vector operations. vec:matvec is the forward GEMV (W x, one
;;;;     vectorized dot per row of W -- SIMD-accelerated under the JVM --simd
;;;;     flag); vec:add / vec:sub / vec:mul (Hadamard) / vec:scale build the
;;;;     activations, deltas and bias updates; vec:dot forms the squared loss.
;;;;   - linalg: the matrix operations. linalg:transpose + linalg:dot give the
;;;;     backward W2^T d2; linalg:outer builds each weight gradient (delta (x)
;;;;     input); linalg:mul / linalg:sub apply the update; linalg:emap maps the
;;;;     sigmoid over a whole activation vector; a scalar broadcast gives 1 - a.
;;;;
;;;; The network is a plist (:w1 W1 :b1 b1 :w2 W2 :b2 b2) read with getf, and it
;;;; is immutable: linalg and vec return fresh packed arrays, so train-example
;;;; produces a NEW plist each step rather than mutating in place -- a functional
;;;; SGD loop. Everything is single-float (#f): linalg is width-polymorphic and
;;;; PRESERVES the packed #f width through every op (a constructor opts in with a
;;;; :element-type 'single-float; the transforms follow their input), so a weight update
;;;; (linalg:sub W ...) never widens back to double -- on the JVM --simd path the
;;;; forward and backward pass stay f32 end to end. The same source runs on every
;;;; backend (on wasm-GC the vec: kernels still compute in f64, but the values match).

;;; --- random weights via the built-in random ---
;;; random returns a value in [0, limit) of the limit's type. On the interpreter
;;; and JVM backends it draws from Math.random(); on WASM it draws real entropy
;;; from the WASI random_get host function (so every run differs).
(defun random-weight () (- (random 1.0) 0.5)) ; -> (-0.5, 0.5)

;;; --- array construction (packed single-float #f, shared by vec and linalg) ---
;;; A weight matrix is a rank-2 linalg array; a bias vector is a rank-1 array. Both
;;; opt into single-float: linalg:zeros takes :element-type 'single-float, and a bias
;;; is a bare single-float make-array (the packed type vec: and linalg: both ride on).
(defun random-matrix (rows cols)
  (let ((m (linalg:zeros (list rows cols) :element-type 'single-float)))
    (dotimes (i rows m)
      (dotimes (j cols) (setf (aref m i j) (random-weight))))))
(defun random-vector (n)
  (let ((v (make-array n :element-type 'single-float :initial-element 0.0)))
    (dotimes (i n v) (setf (aref v i) (random-weight)))))

;;; --- activation ---
(defun sigmoid (x) (/ 1.0 (+ 1.0 (exp (- 0.0 x)))))

;;; One layer: a = sigmoid(W x + b). vec:matvec is the GEMV W x, vec:add adds the
;;; bias, and linalg:emap maps sigmoid over the resulting activation vector -- no
;;; indices, no loops.
(defun layer-forward (w b x)
  (linalg:emap #'sigmoid (vec:add (vec:matvec w x) b)))

;;; --- network = the plist (:w1 W1 :b1 b1 :w2 W2 :b2 b2), read with getf ---
(defun init-net (n-in n-hid n-out)
  (list :w1 (random-matrix n-hid n-in)
        :b1 (random-vector n-hid)
        :w2 (random-matrix n-out n-hid)
        :b2 (random-vector n-out)))

(defun forward-output (net x)
  (layer-forward (getf net :w2) (getf net :b2)
                 (layer-forward (getf net :w1) (getf net :b1) x)))

;;; --- one backprop / SGD step over a single example -> a fresh network ---
;;; d2 = (a2 - y) (*) a2 (*) (1 - a2)              [(*) = Hadamard, vec:mul]
;;; d1 = (W2^T d2) (*) a1 (*) (1 - a1)             [W2^T d2 via linalg]
;;; W -= lr * (delta (x) input)                    [outer product, linalg]
;;; b -= lr * delta                                [vec]
(defun train-example (net x y lr)
  (let* ((w1 (getf net :w1))
         (b1 (getf net :b1))
         (w2 (getf net :w2))
         (b2 (getf net :b2))
         (a1 (layer-forward w1 b1 x))
         (a2 (layer-forward w2 b2 a1))
         (d2 (vec:mul (vec:mul (vec:sub a2 y) a2) (linalg:sub 1.0 a2)))
         (d1
          (vec:mul (vec:mul (linalg:dot (linalg:transpose w2) d2) a1)
                   (linalg:sub 1.0 a1))))
    (list :w1 (linalg:sub w1 (linalg:mul (linalg:outer d1 x) lr))
          :b1 (vec:sub b1 (vec:scale d1 lr))
          :w2 (linalg:sub w2 (linalg:mul (linalg:outer d2 a1) lr))
          :b2 (vec:sub b2 (vec:scale d2 lr)))))

;;; --- loss + training loop ---
;;; example loss = 1/2 ||a - y||^2, the squared error via a single vec:dot.
(defun example-loss (net ex)
  (let ((diff (vec:sub (forward-output net (first ex)) (second ex))))
    (* 0.5 (vec:dot diff diff))))
(defun total-loss (net data)
  (let ((s 0.0)) (dolist (ex data s) (incf s (example-loss net ex)))))
(defun train (net data epochs lr)
  (let ((e 0))
    (while (< e epochs)
      (dolist (ex data)
        (setq net (train-example net (first ex) (second ex) lr)))
      (when (zerop (mod e 1000))
        (format t "epoch ~a  loss ~a~%" e (total-loss net data)))
      (setq e (+ e 1)))
    net))

;;; --- run: learn XOR ---
;;; Inputs and targets are packed single-float vector literals (#f(...)) read
;;; directly as rank-1 arrays. They are never mutated -- only the weights change.
(defparameter *xor-data*
  (list (list #f(0.0 0.0) #f(0.0)) (list #f(0.0 1.0) #f(1.0))
        (list #f(1.0 0.0) #f(1.0)) (list #f(1.0 1.0) #f(0.0))))

(defparameter *net* (init-net 2 4 1))
(format t "Training XOR (2-4-1 network, vec + linalg)...~%")
(setq *net* (train *net* *xor-data* 10000 0.5))

(format t "~%Predictions after training:~%")
(dolist (ex *xor-data*)
  (let ((x (first ex)))
    (format t "  ~a ~a -> ~a  (target ~a)~%" (aref x 0) (aref x 1)
            (aref (forward-output *net* x) 0) (aref (second ex) 0))))


---

# FILE: references/examples/ml/nn.lisp

;;;; Feed-forward neural network in rontolisp
;;;; Learns the XOR function via backpropagation + gradient descent.
;;;; Topology: 2 inputs -> 4 hidden (sigmoid) -> 1 output (sigmoid).
;;;;
;;;; Vectors are rank-1 arrays and weight matrices are rank-2 arrays, so the
;;;; math is plain indexed arithmetic with O(1) aref access and in-place weight
;;;; updates -- close to how a real network is written. (length v) gives a
;;;; vector's size, so layer shapes are read off the bias/input vectors rather
;;;; than tracked separately.

;;; --- random weights via the built-in random ---
;;; random returns a value in [0, limit) of the limit's type. On the interpreter
;;; and JVM backends it draws from Math.random(); on WASM it draws real entropy
;;; from the WASI random_get host function (so every run differs).
(defun random-weight () (- (random 1.0) 0.5)) ; -> (-0.5, 0.5)

;;; --- array helpers ---
(defun random-vector (n)
  (let ((v (make-array n :initial-element 0.0)))
    (dotimes (i n) (setf (aref v i) (random-weight)))
    v))
(defun random-matrix (rows cols)
  (let ((m (make-array (list rows cols) :initial-element 0.0)))
    (dotimes (i rows) (dotimes (j cols) (setf (aref m i j) (random-weight))))
    m))

;;; --- activation ---
(defun sigmoid (x) (/ 1.0 (+ 1.0 (exp (- 0.0 x)))))

;;; One layer: a = sigmoid(W x + b). The output length is (length b) and the
;;; input length is (length x), so no dimensions need to be passed around.
(defun layer-forward (w b x)
  (let* ((rows (length b))
         (cols (length x))
         (a (make-array rows :initial-element 0.0)))
    (dotimes (i rows)
      (let ((s (aref b i)))
        (dotimes (j cols) (incf s (* (aref w i j) (aref x j))))
        (setf (aref a i) (sigmoid s))))
    a))

;;; --- network = (W1 b1 W2 b2) ---
(defun init-net (n-in n-hid n-out)
  (list (random-matrix n-hid n-in) (random-vector n-hid)
        (random-matrix n-out n-hid) (random-vector n-out)))
(defun net-w1 (net) (first net))
(defun net-b1 (net) (second net))
(defun net-w2 (net) (third net))
(defun net-b2 (net) (fourth net))

(defun forward-output (net x)
  (layer-forward (net-w2 net) (net-b2 net)
                 (layer-forward (net-w1 net) (net-b1 net) x)))

;;; --- one backprop / SGD step over a single example (updates net in place) ---
(defun train-example (net x y lr)
  (let* ((w1 (net-w1 net))
         (b1 (net-b1 net))
         (w2 (net-w2 net))
         (b2 (net-b2 net))
         (n-in (length x))
         (n-hid (length b1))
         (n-out (length b2))
         (a1 (layer-forward w1 b1 x))
         (a2 (layer-forward w2 b2 a1))
         (d2 (make-array n-out :initial-element 0.0))
         (d1 (make-array n-hid :initial-element 0.0)))
    ;; output delta: (a2 - y) * a2 * (1 - a2)
    (dotimes (i n-out)
      (setf (aref d2 i)
            (* (- (aref a2 i) (aref y i)) (aref a2 i) (- 1.0 (aref a2 i)))))
    ;; hidden delta: (W2^T d2) * a1 * (1 - a1)  -- computed before W2 changes
    (dotimes (j n-hid)
      (let ((s 0.0))
        (dotimes (i n-out) (incf s (* (aref w2 i j) (aref d2 i))))
        (setf (aref d1 j) (* s (aref a1 j) (- 1.0 (aref a1 j))))))
    ;; descend: W -= lr * delta (outer) input, b -= lr * delta
    (dotimes (i n-out)
      (dotimes (j n-hid) (decf (aref w2 i j) (* lr (aref d2 i) (aref a1 j))))
      (decf (aref b2 i) (* lr (aref d2 i))))
    (dotimes (j n-hid)
      (dotimes (k n-in) (decf (aref w1 j k) (* lr (aref d1 j) (aref x k))))
      (decf (aref b1 j) (* lr (aref d1 j))))
    net))

;;; --- loss + training loop ---
(defun example-loss (net ex)
  (let* ((a (forward-output net (first ex))) (y (second ex)) (s 0.0))
    (dotimes (i (length a))
      (let ((d (- (aref a i) (aref y i)))) (incf s (* d d))))
    (* 0.5 s)))
(defun total-loss (net data)
  (let ((s 0.0))
    (dolist (ex data) (incf s (example-loss net ex)))
    s))
(defun train (net data epochs lr)
  (let ((e 0))
    (while (< e epochs)
      (dolist (ex data) (train-example net (first ex) (second ex) lr))
      (when (zerop (mod e 1000))
        (format t "epoch ~a  loss ~a~%" e (total-loss net data)))
      (setq e (+ e 1))))
  net)

;;; --- run: learn XOR ---
;;; Inputs and targets are vector literals (#(...)) read directly as rank-1
;;; arrays. They are never mutated -- only the network's weights change.
(defparameter *xor-data*
  (list (list #(0.0 0.0) #(0.0)) (list #(0.0 1.0) #(1.0))
        (list #(1.0 0.0) #(1.0)) (list #(1.0 1.0) #(0.0))))

(defparameter *net* (init-net 2 4 1))
(format t "Training XOR (2-4-1 network)...~%")
(setq *net* (train *net* *xor-data* 10000 0.5))

(format t "~%Predictions after training:~%")
(dolist (ex *xor-data*)
  (let ((x (first ex)))
    (format t "  ~a ~a -> ~a  (target ~a)~%" (aref x 0) (aref x 1)
            (aref (forward-output *net* x) 0) (aref (second ex) 0))))


---

# FILE: references/examples/ml/numerical-calculus.lisp

;; Discrete calculus with the linalg package: numpy-style diff and gradient.
;;
;; linalg:diff is the n-th discrete difference along the last axis (each
;; step shortens the axis by one, and a matrix differences within each
;; row); linalg:gradient estimates the derivative of a vector of samples
;; with second-order central differences (first-order one-sided at the two
;; ends), so it keeps the input length. Every sample below is a polynomial
;; read at integer coordinates, where the difference formulas are exact --
;; the printed doubles are identical on every backend.
;;
;; Demonstrates: linalg:diff (orders 1 and 2, matrix rows), linalg:gradient
;; (unit, scalar and non-uniform coordinate spacing), square, arange, argmax.

(defun main ()
  ;; --- diff: discrete differences ------------------------------------------
  ;; Differencing the Fibonacci numbers reproduces them, shifted by two.
  (let ((fib #(1 1 2 3 5 8 13 21 34)))
    (format t "fib:            ~a~%" fib)
    (format t "diff fib:       ~a~%" (linalg:diff fib)))
  ;; The second difference of the squares is the constant 2 -- the discrete
  ;; analogue of d^2/dx^2 x^2 = 2.
  (let ((squares (linalg:square (linalg:arange 8))))
    (format t "squares:        ~a~%" squares)
    (format t "2nd difference: ~a~%" (linalg:diff squares :n 2)))
  ;; A matrix differences within each row (the last axis, like numpy).
  (format t "row diffs:      ~a~%" (linalg:diff #2A((1 3 6 10) (0 2 6 12))))
  (terpri)
  ;; --- gradient: projectile motion ------------------------------------------
  ;; Height of a ball thrown straight up at 30 m/s, h(t) = 30t - 5t^2,
  ;; sampled once per second. gradient recovers the velocity: the interior
  ;; points are exact (central differences are exact for a quadratic), the
  ;; two ends are first-order estimates (25 for a true 30).
  (let ((h #(0 25 40 45 40 25 0)))
    (format t "height h(t):    ~a~%" h)
    (format t "velocity:       ~a~%" (linalg:gradient h))
    (format t "apex at t=~a, where the velocity crosses zero~%"
            (linalg:argmax h)))
  ;; The true velocity 30 - 10t is linear, and gradient is exact for a
  ;; linear ramp even at the ends: constant acceleration, -10 m/s^2.
  (format t "acceleration:   ~a~%" (linalg:gradient #(30 20 10 0 -10 -20 -30)))
  (terpri)
  ;; --- gradient: sample spacing ---------------------------------------------
  ;; The same parabola y = x^2 read three ways: at unit spacing, every 2
  ;; units (pass the scalar spacing), and unevenly at x = (0 1 3 7) (pass
  ;; the coordinate vector -- the non-uniform interior formula is still
  ;; exact for a quadratic: 2x at x = 1 and 3).
  (format t "unit spacing:   ~a~%" (linalg:gradient #(0 1 4 9 16)))
  (format t "spacing 2:      ~a~%" (linalg:gradient #(0 4 16 36 64) 2))
  (format t "at x=(0 1 3 7): ~a~%" (linalg:gradient #(0 1 9 49) #(0 1 3 7))))

(main)


---

# FILE: references/examples/ml/simd-dot-jvm.lisp

;;;; The `--simd` demo sized for the JVM backend -- the one backend where
;;;; `examples/ml/simd-dot.lisp` is too short to show anything.
;;;;
;;;; On the JVM, `--simd` compiles `vec:dot` to a jdk.incubator.vector bridge.
;;;; That bridge is ordinary Java until the JIT compiles it down to real CPU
;;;; vector instructions; before that happens, every lane operation is a genuine
;;;; method call, far slower than the plain scalar loop it replaced. A run of a
;;;; few thousand dots is over before the JIT gets there, so on such a run
;;;; `--simd` LOSES. This program shows both sides of that cliff by timing the
;;;; same dot product twice:
;;;;
;;;;   cold: 4000 reps  -- simd-dot.lisp's exact workload, JIT warmup included
;;;;   warm: 100000 reps -- the steady state, which is all that matters to a
;;;;                        long-running process
;;;;
;;;; Compile it both ways and compare the two `warm` lines (the class needs the
;;;; incubator module at runtime):
;;;;
;;;;   rontolisp examples/ml/simd-dot-jvm.lisp -o Dot.class
;;;;   java Dot
;;;;
;;;;   rontolisp examples/ml/simd-dot-jvm.lisp -o Dot.class --simd
;;;;   java --add-modules jdk.incubator.vector Dot
;;;;
;;;; Expected shape: the cold line may well be SLOWER with `--simd` (warmup),
;;;; the warm line clearly faster. How much faster depends on the JVM -- whether
;;;; it turns the Vector API into vector instructions is its decision, not ours
;;;; -- so measure on the JVM you deploy on. See
;;;; doc/en/guides/simd-acceleration.md.
;;;;
;;;; The sum is the same exact integer in every loop and under every flag, for
;;;; the reason simd-dot.lisp explains: every partial sum of squares below 1024
;;;; is exactly representable as a double, in any summation order.
;;;;
;;;; This file is deliberately NOT in examples/examples.yaml: its point is
;;;; elapsed time, it is JVM-specific, and the 104000 interpreter-mode dots take
;;;; about a minute without `--simd`. Run it by hand, as above. For the
;;;; interpreter and WASM backends use simd-dot.lisp, which needs no warmup
;;;; phase.

(defparameter *v* (vec:arange 1024))
(defparameter *cold-reps* 4000)
(defparameter *warm-reps* 100000)

(format t "(vec:dot v v) over ~a doubles~%" (length *v*))

(let ((start (get-internal-real-time)) (sum 0.0))
  (dotimes (i *cold-reps*) (setq sum (vec:dot *v* *v*)))
  (format t "cold: ~a reps in ~a ms (JIT warmup included), sum = ~a~%"
          *cold-reps* (- (get-internal-real-time) start) (round sum)))

(let ((start (get-internal-real-time)) (sum 0.0))
  (dotimes (i *warm-reps*) (setq sum (vec:dot *v* *v*)))
  (format t "warm: ~a reps in ~a ms (steady state), sum = ~a~%" *warm-reps*
          (- (get-internal-real-time) start) (round sum)))


---

# FILE: references/examples/ml/simd-dot.lisp

;;;; The smallest program that shows what `--simd` does.
;;;;
;;;; One kernel, `vec:dot`, over one packed vector of 1024 doubles, four thousand
;;;; times. Nothing else. Run it twice:
;;;;
;;;;   rontolisp examples/ml/simd-dot.lisp
;;;;   rontolisp examples/ml/simd-dot.lisp --simd
;;;;
;;;;   rontolisp examples/ml/simd-dot.lisp -o dot.wasm        && wasmtime run -W gc dot.wasm
;;;;   rontolisp examples/ml/simd-dot.lisp -o dot.wasm --simd && wasmtime run -W gc dot.wasm
;;;;
;;;; The sum must not change. The elapsed time should. Measured on an Apple M4:
;;;;
;;;;   interpreter  2.59 s  -> 2.3 ms   with --simd   (1100x)
;;;;   wasm-GC       273 ms -> 2.4 ms   with --simd   (115x)
;;;;
;;;; (`-o Prog.class` finishes this in a few tens of milliseconds either way: too
;;;; short for a JIT to warm up. The JVM backend also has a wrinkle worth reading
;;;; about before you rely on it -- see doc/en/guides/simd-acceleration.md.)
;;;;
;;;; WHY THE ANSWER CANNOT CHANGE
;;;; ---------------------------
;;;; The vector holds 0.0, 1.0, ... 1023.0, so its dot product with itself is the
;;;; sum of the squares below 1024 -- an exact integer, and every partial sum on
;;;; the way there is exactly representable as a double. Folding that sum two
;;;; lanes at a time, which is what `--simd` does, therefore lands on the very
;;;; same value, bit for bit. That is the contract: acceleration never changes an
;;;; answer. (Over inexact inputs a reduction may differ in the last bit, because
;;;; the lanes add in a different order.)
;;;;
;;;; For matrices -- and for where a real LLM inference engine spends its time --
;;;; see simd-gemv.lisp.

(defparameter *v* (vec:arange 1024))
(defparameter *reps* 4000)

(format t "(vec:dot v v) over ~a doubles, ~a times = ~a multiply-adds~%"
        (length *v*) *reps* (* (length *v*) *reps*))

(let ((start (get-internal-real-time)) (sum 0.0))
  (dotimes (i *reps*) (setq sum (vec:dot *v* *v*)))
  (format t "sum of squares below 1024 = ~a~%" (round sum))
  (format t "elapsed: ~a ms~%" (- (get-internal-real-time) start)))


---

# FILE: references/examples/ml/simd-gemv-nogc.lisp

;;;; simd-gemv.lisp's inner loop, compiled to a plain linear-memory WASM module
;;;; with `--no-gc` -- the fastest way rontolisp can run a GEMV, and the only
;;;; backend where the SIMD speedup comes with no garbage collector at all.
;;;;
;;;; A `--no-gc` module is a pure-compute reactor: no `_start`, no printing --
;;;; the host calls an exported function and reads the returned integer. So
;;;; where simd-gemv.lisp prints its fingerprint, this one RETURNS it:
;;;;
;;;;   (fingerprint n)  builds the same fixed-seed 256x256 single-float matrix,
;;;;                    runs n steps of  x <- rms-normalize(W x),  and returns
;;;;                    argmax(x) -- which component of the vector is largest.
;;;;
;;;; Each step is one vec:matvec-into (the GEMV) and one vec:dot (the RMS), the
;;;; two kernels an LLM inference engine spends nearly all of its time in. The
;;;; -into kernels matter here more than anywhere: `--no-gc` bump-allocates and
;;;; never frees, so the loop writes into two pre-allocated vectors and the
;;;; whole run allocates exactly three blocks (W, x, y) no matter how many
;;;; steps it takes.
;;;;
;;;; RUN IT BOTH WAYS
;;;; ----------------
;;;;   rontolisp examples/ml/simd-gemv-nogc.lisp -o gemv.wasm --no-gc --optimize
;;;;   wasmtime run --invoke fingerprint gemv.wasm 100
;;;;
;;;;   rontolisp examples/ml/simd-gemv-nogc.lisp -o gemv.wasm --no-gc --simd --optimize
;;;;   wasmtime run --invoke fingerprint gemv.wasm 100
;;;;
;;;; Both print 85 -- the same dominant direction simd-gemv.lisp settles into on
;;;; every other backend -- and the argmax after each of steps 1-10 matches its
;;;; printed (0 14 82 126 14 140 126 79 134 175) too. The scalar build carries
;;;; no SIMD instruction at all (it runs even under `wasmtime -W simd=n -W
;;;; relaxed-simd=n`); the `--simd` build runs the same loop as f32x4 lanes.
;;;; The time difference is the point: measured on an Apple M4 at 20000 steps,
;;;; the scalar module takes ~600 ms and the `--simd` one ~120 ms (5x).
;;;;
;;;; DETERMINISM
;;;; -----------
;;;; The same Lehmer generator as simd-gemv.lisp, threaded through a local
;;;; instead of a global (`--no-gc` has no globals), so the matrix is
;;;; bit-identical to the one every other backend builds. argmax is an integer
;;;; fingerprint of every multiply-add, yet unmoved by the last-bit differences
;;;; lane-order (or f32-throughout) summation introduces.

;;; Which component of the vector is the largest.
(defun argmax (v)
  (let ((best 0))
    (dotimes (i (length v)) (when (> (aref v i) (aref v best)) (setq best i)))
    best))

;;; n steps of x <- rms-normalize(W x) over the fixed-seed matrix; returns
;;; argmax(x). W is a rank-2 packed single-float matrix built with make-array;
;;; the LCG state is a plain local: s <- (75 s + 74) mod 65537, and each draw
;;; maps to a single-float in [-1, 1) exactly as simd-gemv.lisp's lcg-uniform.
(defun fingerprint (n)
  (let ((dim 256) (eps 0.00001) (s 7))
    (let ((w (make-array (list dim dim) :element-type 'single-float))
          (x (vec:zeros dim :element-type 'single-float))
          (y (vec:zeros dim :element-type 'single-float)))
      (dotimes (i dim)
        (dotimes (j dim)
          (setq s (mod (+ (* s 75) 74) 65537))
          (setf (aref w i j) (- (/ (mod s 2048) 1024.0) 1.0))))
      (dotimes (i dim)
        (setq s (mod (+ (* s 75) 74) 65537))
        (setf (aref x i) (- (/ (mod s 2048) 1024.0) 1.0)))
      (dotimes (k n)
        (vec:matvec-into y w x)
        (vec:scale-into x y (/ 1.0 (sqrt (+ (/ (vec:dot y y) dim) eps)))))
      (argmax x))))

(rontolisp:wasm-export 'fingerprint :params '(:int) :returns :int)


---

# FILE: references/examples/ml/simd-gemv.lisp

;;;; The two kernels `--simd` accelerates, and the two an LLM inference engine
;;;; spends nearly all of its time in:
;;;;
;;;;   (vec:matvec w x)   y = W x, a matrix times a vector  -- one dot per row
;;;;   (vec:dot a b)      the sum of the products           -- one reduction
;;;;
;;;; Autoregressive decoding runs one token at a time, so every weight matrix in
;;;; a transformer is multiplied by a *vector*, never by another matrix: it is
;;;; all GEMV. And a GEMV is just a dot product per row, which is exactly the
;;;; shape a CPU's vector unit multiplies four (or two) elements at a time.
;;;;
;;;; So this program does nothing but that, a hundred times over: project a
;;;; vector through a random matrix, rescale it to unit root-mean-square, repeat.
;;;; The rescaling is RMSNorm with its gain vector dropped -- `vec:dot` of a
;;;; vector with itself is the sum of its squares -- and it is what keeps the
;;;; numbers from growing without bound, so the loop can run as long as we like.
;;;;
;;;; RUN IT BOTH WAYS
;;;; ----------------
;;;;   rontolisp examples/ml/simd-gemv.lisp                                    # scalar
;;;;   rontolisp examples/ml/simd-gemv.lisp --simd                             # Vector API
;;;;
;;;;   rontolisp examples/ml/simd-gemv.lisp -o gemv.wasm        && wasmtime run -W gc gemv.wasm
;;;;   rontolisp examples/ml/simd-gemv.lisp -o gemv.wasm --simd && wasmtime run -W gc gemv.wasm
;;;;
;;;; The printed indices must not change. The elapsed time should. Measured on an
;;;; Apple M4:
;;;;
;;;;   wasm-GC      467 ms  ->  3.9 ms   with --simd   (120x -- native f32x4)
;;;;   interpreter  4.67 s  -> 0.68 s    with --simd   (6.9x, the native binary)
;;;;
;;;; The JVM backend is the one to be careful with, and this example is the worst
;;;; case for it. `--simd` compiles to a jdk.incubator.vector bridge, and whether
;;;; that bridge becomes real CPU instructions is decided by the JVM that runs the
;;;; class, not by us; where it does not, each lane is emulated and `--simd` ends
;;;; up slower than not passing it. A single-float GEMV is the shape most exposed
;;;; to this, because it widens every f32 lane to f64 before accumulating. On the
;;;; JVMs measured here the result ranged from a 4x speedup to a 20x slowdown --
;;;; so measure on the JVM you deploy on. See doc/en/guides/simd-acceleration.md.
;;;;
;;;; A hundred steps is also too short for a JIT to warm up; raise *steps* first.
;;;;
;;;; DETERMINISM
;;;; -----------
;;;; The matrix comes from a fixed-seed generator, so it is the same on every
;;;; backend and every run. Only INTEGERS are printed: the WASM backend rounds
;;;; floats to about seven significant digits when printing, so a float would not
;;;; compare across backends. What is printed instead is `argmax` -- which
;;;; component of the vector is the largest -- an integer that depends on every
;;;; multiply-add that produced the vector, yet is unmoved by the last-bit
;;;; differences that reordering a sum into vector lanes introduces.

;;; --- size ------------------------------------------------------------------
;;; A row must hold at least 128 elements. Below that the interpreter and JVM
;;; vector kernels fall back to a scalar loop, because setting up the vector
;;; registers would cost more than it saves. (wasm-GC has no such threshold.)
(defparameter *dim* 256)
(defparameter *steps* 100)
(defparameter *eps* 0.00001)

;;; --- deterministic pseudo-random numbers -------------------------------------
;;; A Lehmer generator: every intermediate stays below 2^23, which fits the WASM
;;; backend's integer range, so the stream is identical on all backends.
(defvar *lcg-state* 7)

(defun lcg-next ()
  (setq *lcg-state* (mod (+ (* *lcg-state* 75) 74) 65537))
  *lcg-state*)

;;; a single-float in [-1, 1)
(defun lcg-uniform () (- (/ (mod (lcg-next) 2048) 1024.0) 1.0))

;;; --- the packed float arrays -------------------------------------------------
;;; `:element-type 'single-float` is what makes these packed (unboxed) arrays --
;;; the representation `--simd` needs, and the one `vec:` operates on. A rank-2
;;; array is the matrix; a rank-1 array is the vector.
(defun random-matrix (rows cols)
  (let ((m (make-array (list rows cols) :element-type 'single-float)))
    (dotimes (i rows m) (dotimes (j cols) (setf (aref m i j) (lcg-uniform))))))

(defun random-vector (n)
  (let ((v (vec:zeros n :element-type 'single-float)))
    (dotimes (i n v) (setf (aref v i) (lcg-uniform)))))

;;; --- the two kernels ---------------------------------------------------------
;;; RMSNorm without its gain vector: divide by the root mean square. `(vec:dot v v)`
;;; is the sum of the squares -- one accelerated reduction over the whole vector.
(defun rms-normalize (v)
  (vec:scale v (/ 1.0 (sqrt (+ (/ (vec:dot v v) (length v)) *eps*)))))

;;; One step: a GEMV, then the normalization. In a transformer this pair is a
;;; projection followed by a layer norm; here it is the whole program.
(defun step-once (w x) (rms-normalize (vec:matvec w x)))

;;; --- the fingerprint ---------------------------------------------------------
;;; Which component is the largest. Repeating the same matrix drives the vector
;;; toward that matrix's dominant direction, so the index moves for a few steps
;;; and then stops -- and where it stops is a fingerprint of every multiply-add
;;; along the way.
(defun argmax (v)
  (let ((best 0))
    (dotimes (i (length v) best)
      (when (> (aref v i) (aref v best)) (setq best i)))))

(defun iterate (w x n)
  (let ((indices '()))
    (dotimes (s n (reverse indices))
      (setq x (step-once w x))
      (setq indices (cons (argmax x) indices)))))

;;; --- run ---------------------------------------------------------------------
(defparameter *w* (random-matrix *dim* *dim*))
(defparameter *x* (random-vector *dim*))

(format t
 "simd-gemv: ~a steps of (vec:matvec w x) on a ~ax~a single-float matrix~%"
 *steps* *dim* *dim*)
(format t "~a multiply-adds, every one of them inside vec:matvec or vec:dot~%"
        (* *steps* (+ (* *dim* *dim*) *dim*)))

(let* ((start (get-internal-real-time))
       (indices (iterate *w* *x* *steps*))
       (elapsed (- (get-internal-real-time) start)))
  (format t "argmax after steps 1-10: ~a~%" (subseq indices 0 10))
  (format t "argmax after step ~a:   ~a  (the dominant direction)~%" *steps*
          (nth (- *steps* 1) indices))
  (format t "elapsed: ~a ms~%" elapsed)
  (format t
   "(re-run with --simd; the indices must not change, the time should)~%"))


---

# FILE: references/examples/ml/tiny-llm.lisp

;;;; A transformer decoder -- the arithmetic core of an LLM inference engine --
;;;; and the example where `--simd` earns its keep.
;;;;
;;;; This is llama2's `forward()` with the tokenizer and the weight loader taken
;;;; away: RMSNorm, Q/K/V projections, causal self-attention over a KV cache,
;;;; softmax, the output projection, a SwiGLU feed-forward network, residual
;;;; connections, a classifier head, and greedy (argmax) sampling. Stack more
;;;; layers, load real weights instead of the pseudo-random ones below, and you
;;;; have an inference engine.
;;;;
;;;; WHY IT IS FAST -- the KV cache layout
;;;; -------------------------------------
;;;; Autoregressive decoding is one token at a time, so every matrix here is
;;;; multiplied by a *vector*: it is all GEMV (`vec:matvec`), never GEMM. Nine
;;;; of the ten hot operations per layer are therefore a single `vec:matvec`,
;;;; which `--simd` lowers to CPU vector instructions.
;;;;
;;;; The tenth -- attention -- is only a GEMV if you store the cache correctly,
;;;; and that is the one design decision in this file worth stealing:
;;;;
;;;;   K cache:  row-major, (n-ctx x dim)   -- row t is the key at position t,
;;;;                                           so `(vec:matvec kc q)` computes
;;;;                                           ALL attention scores in one GEMV.
;;;;   V cache:  TRANSPOSED, (dim x n-ctx)  -- row j is the j-th component over
;;;;                                           time, so `(vec:matvec vt a)` is
;;;;                                           the attention-weighted sum of the
;;;;                                           value vectors, again one GEMV.
;;;;
;;;; Store V row-major and that second step becomes a scalar loop over the cache.
;;;; llama2.c makes exactly this choice; so does every fast engine.
;;;;
;;;; DETERMINISM
;;;; -----------
;;;; Weights come from a fixed-seed linear congruential generator, so the model
;;;; is identical on every backend and every run. Only INTEGERS are printed: the
;;;; WASM backend rounds floats to about seven significant digits when printing,
;;;; and its `exp` differs from the JVM's in the low bits, so a float would not
;;;; compare across backends. The generated token ids are a fingerprint of the
;;;; whole computation -- if any kernel were wrong, they would change.
;;;;
;;;; The model is untrained, so the tokens mean nothing. The arithmetic is real.
;;;;
;;;; RUN IT BOTH WAYS
;;;; ----------------
;;;;   rontolisp examples/ml/tiny-llm.lisp                                   # scalar
;;;;   rontolisp examples/ml/tiny-llm.lisp --simd                            # Vector API
;;;;
;;;;   rontolisp examples/ml/tiny-llm.lisp -o llm.wasm         && wasmtime run -W gc llm.wasm
;;;;   rontolisp examples/ml/tiny-llm.lisp -o llm.wasm --simd  && wasmtime run -W gc llm.wasm
;;;;
;;;; The token ids must not change. The elapsed time should. Measured on an M4
;;;; (decode only, weight init excluded):
;;;;
;;;;   wasm-GC    891 ms  ->    8 ms   with --simd   (114x -- native f32x4)
;;;;   interpreter 11.5 s ->  1.7 s    with --simd   (7x, the native binary)
;;;;
;;;; The JVM backend is the one to be careful with: `--simd` there depends on the
;;;; JVM compiling jdk.incubator.vector down to vector instructions, and a
;;;; single-float GEMV -- which widens every f32 lane to f64 before accumulating --
;;;; is the shape most likely to find an operation a given JVM does not. Across
;;;; the JVMs measured here the same class ranged from a 2.3x speedup to a 15x
;;;; slowdown. Measure before you trust it. See doc/en/guides/simd-acceleration.md.

;;; --- model size -------------------------------------------------------------
;;; dim must be >= 128: below that the JVM and interpreter Vector-API kernels
;;; fall back to a scalar loop (the vector setup costs more than it saves).
(defparameter *vocab* 48)
(defparameter *dim* 256)
(defparameter *hidden* 512)
(defparameter *layers* 2)
(defparameter *n-ctx* 12)
(defparameter *eps* 0.00001)

;;; --- deterministic pseudo-random weights ------------------------------------
;;; The same Lehmer generator deep-digits.lisp uses: every intermediate stays
;;; below 2^23, which fits the WASM backend's i31 integer range, so the weight
;;; stream is identical on all four backends.
(defvar *lcg-state* 7)

(defun lcg-next ()
  (setq *lcg-state* (mod (+ (* *lcg-state* 75) 74) 65537))
  *lcg-state*)

;;; A single-float in [-scale, scale).
(defun lcg-uniform (scale) (* scale (- (/ (mod (lcg-next) 2048) 1024.0) 1.0)))

(defun random-matrix (rows cols scale)
  (let ((m (linalg:zeros (list rows cols) :element-type 'single-float)))
    (dotimes (i rows m)
      (dotimes (j cols) (setf (aref m i j) (lcg-uniform scale))))))

;;; --- one decoder layer ------------------------------------------------------
;;; A layer is a plist of its seven weight matrices, its two RMSNorm gains, and
;;; its own KV cache -- per-layer, exactly as in a real engine.
(defun make-layer ()
  (list :wq (random-matrix *dim* *dim* 0.08)
        :wk (random-matrix *dim* *dim* 0.08)
        :wv (random-matrix *dim* *dim* 0.08)
        :wo (random-matrix *dim* *dim* 0.08)
        :w1 (random-matrix *hidden* *dim* 0.06)
        :w3 (random-matrix *hidden* *dim* 0.06)
        :w2 (random-matrix *dim* *hidden* 0.06)
        :ng1 (vec:ones *dim* :element-type 'single-float)
        :ng2 (vec:ones *dim* :element-type 'single-float)
        :kc (linalg:zeros (list *n-ctx* *dim*) :element-type 'single-float)   ; keys, row-major
        :vt (linalg:zeros (list *dim* *n-ctx*) :element-type 'single-float))) ; values, TRANSPOSED

;;; --- RMSNorm: x / rms(x) * g ------------------------------------------------
;;; vec:dot is the sum of squares -- one accelerated reduction, no loop.
(defun rmsnorm (x g)
  (vec:mul (vec:scale x (/ 1.0 (sqrt (+ (/ (vec:dot x x) *dim*) *eps*)))) g))

;;; --- causal self-attention over the KV cache --------------------------------
;;; Both halves are a GEMV, thanks to the cache layout described at the top.
;;; Positions after `pos` keep a zero attention weight, which is the causal mask:
;;; a zero weight contributes nothing to the `vt` GEMV, so no masking arithmetic
;;; is needed at all.
(defun attention (l x pos)
  (let ((q (vec:matvec (getf l :wq) x))
        (k (vec:matvec (getf l :wk) x))
        (v (vec:matvec (getf l :wv) x))
        (kc (getf l :kc))
        (vt (getf l :vt)))
    ;; append this position's key and value to the cache
    (dotimes (j *dim*)
      (setf (aref kc pos j) (aref k j))
      (setf (aref vt j pos) (aref v j)))
    ;; every attention score at once: scores = (K q) / sqrt(dim)
    (let ((scores (vec:scale (vec:matvec kc q) (/ 1.0 (sqrt *dim*))))
          (w (vec:zeros *n-ctx* :element-type 'single-float))
          (top -1000000.0)
          (z 0.0))
      ;; softmax over positions 0..pos, shifted by the max for stability
      (dotimes (u (+ pos 1))
        (when (> (aref scores u) top) (setq top (aref scores u))))
      (dotimes (u (+ pos 1))
        (let ((e (exp (- (aref scores u) top))))
          (setf (aref w u) e)
          (setq z (+ z e))))
      ;; the weighted sum of the value vectors, then the output projection
      (vec:matvec (getf l :wo) (vec:matvec vt (vec:scale w (/ 1.0 z)))))))

;;; --- SwiGLU feed-forward: w2 (silu(w1 h) * w3 h) ----------------------------
(defun silu (x) (/ x (+ 1.0 (exp (- 0.0 x)))))

(defun feed-forward (l h)
  (vec:matvec (getf l :w2)
              (vec:mul (linalg:emap #'silu (vec:matvec (getf l :w1) h))
                       (vec:matvec (getf l :w3) h))))

;;; --- the layer, with its two residual connections ---------------------------
(defun layer-forward (l x pos)
  (let ((h (vec:add x (attention l (rmsnorm x (getf l :ng1)) pos))))
    (vec:add h (feed-forward l (rmsnorm h (getf l :ng2))))))

;;; --- the model --------------------------------------------------------------
(defparameter *net*
  (let ((ls '()))
    (dotimes (i *layers* (reverse ls)) (setq ls (cons (make-layer) ls)))))
(defparameter *emb* (random-matrix *vocab* *dim* 0.5))
(defparameter *pos-emb* (random-matrix *n-ctx* *dim* 0.1))
(defparameter *ng-final* (vec:ones *dim* :element-type 'single-float))
(defparameter *w-cls* (random-matrix *vocab* *dim* 0.08))

;;; token embedding + learned position embedding
(defun embed (tok pos)
  (let ((x (vec:zeros *dim* :element-type 'single-float)))
    (dotimes (j *dim* x)
      (setf (aref x j) (+ (aref *emb* tok j) (aref *pos-emb* pos j))))))

;;; one full forward pass -> the logits over the vocabulary
(defun forward (tok pos)
  (let ((x (embed tok pos)))
    (dolist (l *net*) (setq x (layer-forward l x pos)))
    (vec:matvec *w-cls* (rmsnorm x *ng-final*))))

;;; --- greedy decode ----------------------------------------------------------
;;; Positions 0..n-1 of the prompt are the prefill; every later position feeds
;;; back the argmax of the previous step. The KV cache is what makes each step
;;; O(dim^2) instead of O(pos * dim^2).
(defparameter *prompt* '(3 14 1 5))

(defun generate ()
  (let ((tok (first *prompt*)) (out '()))
    (dotimes (pos *n-ctx* (reverse out))
      (let ((next (linalg:argmax (forward tok pos))))
        (if (< (+ pos 1) (length *prompt*))
            (setq tok (nth (+ pos 1) *prompt*))
            (progn
              (setq out (cons next out))
              (setq tok next)))))))

;;; --- run --------------------------------------------------------------------
;;; Every count below is an exact integer, so it prints identically everywhere.
(defun gemvs-per-token () (+ (* *layers* 6) 1))

(defun macs-per-token ()
  (+ (* *layers*
        (+ (* 4 *dim* *dim*)     ; wq wk wv wo
           (* 3 *dim* *hidden*)  ; w1 w3 w2
           (* 2 *n-ctx* *dim*))) ; the two attention GEMVs
     (* *vocab* *dim*)))         ; the classifier

(format t
        "tiny-llm: ~a-layer transformer decoder, dim=~a hidden=~a ctx=~a vocab=~a, single-float~%"
        *layers* *dim* *hidden* *n-ctx* *vocab*)
(format t
 "~a GEMVs and ~a multiply-adds per forward pass, nearly all of it vec:matvec~%"
 (gemvs-per-token) (macs-per-token))
(format t "prompt:    ~a~%" *prompt*)

(let* ((start (get-internal-real-time))
       (tokens (generate))
       (elapsed (- (get-internal-real-time) start)))
  (format t "generated: ~a~%" tokens)
  (format t "~a forward passes (~a prompt + ~a generated) in ~a ms~%" *n-ctx*
          (length *prompt*) (length tokens) elapsed)
  (format t
   "(re-run with --simd; the tokens must not change, the time should)~%"))


---

# FILE: references/examples/net/dog-fetcher.lisp

;; The dog fetcher -- a rontolisp reproduction of wasmCloud's "dog-fetcher"
;; example (https://wasmcloud.com/docs/v1/examples/rust/component/dog-fetcher/;
;; source: examples/rust/components/dog-fetcher in the wasmCloud repo): a
;; served handler that itself makes an outgoing HTTP request, i.e.
;; rontolisp:fetch inside rontolisp:http-handler -- the classic
;; proxy/aggregator shape. Every GET asks the dog.ceo API for a random dog
;; picture and answers with JSON:
;;
;;   GET /            -> {"dog": "https://images.dog.ceo/breeds/.../xxx.jpg"}
;;   (other paths 404; an upstream failure 502)
;;
;; Run (interpreter, blocking server on :8080):
;;   java -jar $JAR examples/net/dog-fetcher.lisp
;; Run (JVM class; needs the rontolisp jar on the classpath):
;;   java -jar $JAR examples/net/dog-fetcher.lisp -o DogFetcher.class && java -cp $JAR:. DogFetcher
;; Run (WASI component under wasmtime serve; the wasi:http/client import that
;; carries the outbound fetch is host-provided by default):
;;   java -jar $JAR examples/net/dog-fetcher.lisp -o dog-fetcher.wasm --component && \
;;     wasmtime serve -W gc=y -W exceptions=y dog-fetcher.wasm
;; Compile (--no-wasi reactor, THIS SAME SOURCE: http-handler becomes the
;; host-driven handle-request export, and --host-fetch lowers fetch onto the
;; host's own client, imported as env.fetch -- the Cloudflare Workers shape,
;; see examples/cloudflare-workers/dog-fetcher):
;;   java -jar $JAR examples/net/dog-fetcher.lisp -o dog-fetcher.wasm --no-wasi --host-fetch
;; Talk to it with:
;;   curl http://127.0.0.1:8080/

(defun json-response (status obj)
  (list status '(:content-type "application/json")
        (list (format nil "~a~%" (rontolisp:json-stringify obj)))))

;; One upstream round trip: dog.ceo answers {"message": "",
;; "status": "success"}. A failed fetch surfaces as a nil/non-200 response
;; plist, mapped to 502.
(rontolisp:async-defun fetch-dog ()
  ;; awaiting needs an async-defun; the response :body is an asynchronous
  ;; stream on every backend, drained with read-all.
  (let* ((res
          (rontolisp:await
           (rontolisp:fetch "https://dog.ceo/api/breeds/image/random")))
         (status (getf res :status))
         (body (rontolisp:await (rontolisp:read-all (getf res :body)))))
    (if (and (integerp status) (= status 200))
        (gethash "message" (rontolisp:json-parse body))
        nil)))

(rontolisp:async-defun handle (env)
  (if (string= (getf env :path-info) "/")
      (let ((dog (rontolisp:await (fetch-dog))))
        (if dog
            (json-response 200 (rontolisp:plist-hash-table (list :dog dog)))
            (json-response 502
                           (rontolisp:plist-hash-table
                            (list :error "the dog API did not answer")))))
      (json-response 404
                     (rontolisp:plist-hash-table (list :error "not found")))))

;; On the interpreter / JVM this blocks and serves on port 8080; under
;; --component the port argument is ignored (the host provides the socket).
(rontolisp:http-handler 'handle 8080)


---

# FILE: references/examples/net/echo-client.lisp

;; TCP echo client: connects to the echo server on 127.0.0.1:7777, sends every
;; line read from standard input and prints the server's reply, until stdin ends.
;;
;; Networking goes through the portable usocket API: with-client-socket
;; connects, binds a stream, and closes it on exit; the stream works with the
;; standard functions read-line / write-line. usocket:socket-connect signals
;; usocket:socket-error on failure (no server listening), caught below. The
;; API is a shim over the built-in rontolisp:tcp-* functions, so the same
;; source runs on the interpreter, the JVM, and a WASM component.
;;
;; Start examples/net/echo-server.lisp first (any backend), then:
;; Run (interpreter):        echo hello | java -jar $JAR examples/net/echo-client.lisp
;; Run (JVM):                java -jar $JAR examples/net/echo-client.lisp -o EchoClient.class && \
;;                           echo hello | java EchoClient
;; Run (WASM component):     java -jar $JAR examples/net/echo-client.lisp -o echo-client.wasm --component && \
;;                           echo hello | wasmtime run -W gc=y -W exceptions=y \
;;                             -S tcp=y -S inherit-network=y echo-client.wasm
(handler-case (usocket:with-client-socket (sock stream "127.0.0.1" 7777)
                (do ((line (read-line) (read-line)))
                    ((null line))
                  (write-line line stream)
                  (write-line (read-line stream))))
  (usocket:socket-error (e)
    (declare (ignore e))
    (write-line
     "cannot connect to 127.0.0.1:7777 (is echo-server.lisp running?)")))


---

# FILE: references/examples/net/echo-server.lisp

;; TCP echo server: listens on port 7777 and echoes every received line back to
;; the client, one connection at a time, until the client closes (read-line
;; returns nil at peer close). Serves connections forever -- stop it with Ctrl-C.
;;
;; Networking goes through the portable usocket API (socket-listen /
;; socket-accept / socket-stream and the with-server-socket macro), a shim over
;; the built-in rontolisp:tcp-* functions, so the same source runs on the
;; interpreter, the JVM, and a WASM component. A socket's stream works with the
;; standard functions: read-line / write-line / close. usocket:socket-listen
;; signals usocket:socket-error on failure (a busy port), caught below.
;;
;; Run (interpreter):        java -jar $JAR examples/net/echo-server.lisp
;; Run (JVM):                java -jar $JAR examples/net/echo-server.lisp -o EchoServer.class && java EchoServer
;; Run (WASM component):     java -jar $JAR examples/net/echo-server.lisp -o echo-server.wasm --component && \
;;                           wasmtime run -W gc=y -W exceptions=y \
;;                             -S tcp=y -S inherit-network=y echo-server.wasm
;; Talk to it with:          nc 127.0.0.1 7777   (or examples/net/echo-client.lisp)
(handler-case (let ((listener
                     (usocket:socket-listen "127.0.0.1" 7777 :reuse-address t)))
                (write-line "echo server listening on 127.0.0.1:7777")
                (do ((n 1 (+ n 1)))
                    (nil)
                  ;; with-server-socket closes the accepted socket on every exit.
                  (usocket:with-server-socket (sock
                                               (usocket:socket-accept listener))
                    (let ((stream (usocket:socket-stream sock)))
                      (write-line (format nil "client ~a connected" n))
                      (do ((line (read-line stream) (read-line stream)))
                          ((null line) (write-line "client disconnected"))
                        (write-line line stream))))))
  (usocket:socket-error (e)
    (declare (ignore e))
    (write-line "socket-listen failed (is port 7777 already in use?)")))


---

# FILE: references/examples/net/http-handler-cl-who.lisp

;; An HTTP handler that renders its HTML response with cl-who, the real
;; upstream (X)HTML markup library loaded through asdf:load-system. It is the
;; cl-who counterpart of http-handler.lisp (a plain text/plain handler):
;; with-html-output-to-string expands cl-who's markup DSL at macro-expansion
;; time, so the template below compiles to ordinary string building; str / esc
;; splice the (escaped) request path in at run time.
;;
;; The library is loaded with asdf, so pass its directory with --system-path
;; (the sources are vendored under src/test/resources/cl-who); the compile
;; paths splice the system in at compile time, so the produced class /
;; component is self-contained. asdf:load-system scopes the loaded sources'
;; in-package to the load (like Common Lisp binding *package* around load), so
;; the handler defined below stays in cl-user and http-handler resolves it by
;; its (quoted) symbol.
;;
;; Supported on the interpreter and JVM backends (a blocking server on :8080,
;; one virtual thread per request) and the WASI component backend (--component),
;; which compiles the handler into an async wasi:http/handler@0.3.0 component
;; served by wasmtime 46+.
;;
;; Run (interpreter, blocking server on :8080):
;;   rontolisp examples/net/http-handler-cl-who.lisp --system-path src/test/resources/cl-who
;; Run (JVM class; keep the rontolisp jar on the classpath):
;;   rontolisp examples/net/http-handler-cl-who.lisp -o App.class --system-path src/test/resources/cl-who && \
;;     java -cp target/rontolisp-0.1.0-SNAPSHOT-exec.jar:. App
;; Run (WASI component under wasmtime serve):
;;   rontolisp examples/net/http-handler-cl-who.lisp -o app.wasm --component --system-path src/test/resources/cl-who && \
;;     wasmtime serve -W gc=y -W exceptions=y app.wasm
;; Talk to it with:  curl http://127.0.0.1:8080/world

(asdf:load-system :cl-who)

;; The env plist's :path-info carries the (percent-decoded) path only; the
;; rendered page is the single string of the response body list.
(defun handle (env)
  (let ((path (getf env :path-info)))
    (list 200 '(:content-type "text/html; charset=utf-8")
          (list
           (cl-who:with-html-output-to-string (s)
             (:html (:head (:title "rontolisp + cl-who"))
                    (:body (:h1 "Hello, World!")
                     (:p "You requested " (:code (cl-who:esc path))))))))))

;; On the interpreter / JVM this blocks and serves on port 8080; under
;; --component the port argument is ignored (the host provides the socket).
(rontolisp:http-handler 'handle 8080)


---

# FILE: references/examples/net/http-handler.lisp

;; An HTTP handler function. The handler receives the Clack environment
;; property list (:request-method / :path-info / :query-string / :headers /
;; :raw-body / ...) and returns a Clack response list (status headers body),
;; where body is a list of strings. Unlike http-hello.lisp (a hand-rolled
;; raw-TCP server), rontolisp:http-handler adapts the request/response for you.
;;
;; Supported on the interpreter and JVM backends (a blocking server on :8080,
;; one virtual thread per request) and the WASI component backend (--component),
;; which compiles the handler into an async wasi:http/handler@0.3.0 component
;; served by wasmtime 46+ and hosted by wasmCloud (`wash dev`, wash 2.5.2+ --
;; see examples/wasmcloud/) and by Spin (the canary build,
;; https://github.com/spinframework/spin/releases/tag/canary -- see
;; http-handler/spin.toml). jco cannot run it yet.
;;
;; Run (interpreter, blocking server on :8080):
;;   java -jar $JAR examples/net/http-handler.lisp
;; Run (JVM class; it implements the embedded server's handler interface, so
;; keep the rontolisp jar on the classpath):
;;   java -jar $JAR examples/net/http-handler.lisp -o App.class && java -cp $JAR:. App
;; Run (WASI component under wasmtime serve):
;;   java -jar $JAR examples/net/http-handler.lisp -o app.wasm --component && \
;;     wasmtime serve -W gc=y -W exceptions=y app.wasm
;; Run (the same component under Spin, which owns the socket on :3000):
;;   cd examples/net/http-handler && spin build && spin up
;; Talk to it with:  curl http://127.0.0.1:8080/hello

;; :request-method is a keyword (:GET, :POST, ...); symbol-name turns it back
;; into the bare method name for the text body.
(defun handle (env)
  (list 200 '(:content-type "text/plain")
        (list
         (format nil "Hello from rontolisp!~%~a ~a~%"
                 (symbol-name (getf env :request-method))
                 (getf env :path-info)))))

;; On the interpreter / JVM this blocks and serves on port 8080; under
;; --component the port argument is ignored (the host provides the socket).
(rontolisp:http-handler 'handle 8080)


---

# FILE: references/examples/net/http-handler/spin.toml

# Spin manifest for the rontolisp:http-handler component.
#
#   spin build && spin up
#
# `spin build` runs the `rontolisp` binary (expected on the PATH) to compile
# ../http-handler.lisp into app.wasm, and `spin up` serves it on :3000.
#
# Needs a Spin whose embedded wasmtime is 46 or newer -- currently the canary
# build (4.1.0-pre0; https://github.com/spinframework/spin/releases/tag/canary),
# which enables the WebAssembly GC and exception-handling proposals by default.
# Released 4.0.2 cannot run it: its wasmtime 44 speaks the
# wasi:http@0.3.0-rc-2026-03-15 snapshot, so it rejects the component's
# wasi:http@0.3.0 imports whether or not GC is turned on.
spin_manifest_version = 2

[application]
name = "rontolisp-http-handler"
version = "0.1.0"
description = "A rontolisp:http-handler served as a wasi:http/handler@0.3.0 component."

[[trigger.http]]
route = "/..."
component = "hello"

[component.hello]
source = "app.wasm"

[component.hello.build]
command = "rontolisp ../http-handler.lisp -o app.wasm --component"
watch = ["../http-handler.lisp"]


---

# FILE: references/examples/net/http-hello.lisp

;; Minimal HTTP/1.1 server on port 8080: answers every request with a small
;; HTML page showing the request line and a running request counter, one
;; connection per request (Connection: close). Serves forever -- stop with
;; Ctrl-C.
;;
;; Networking goes through the portable usocket API (socket-listen /
;; socket-accept / socket-stream and the with-server-socket macro), a shim over
;; the built-in rontolisp:tcp-* functions, so the same source runs on the
;; interpreter, the JVM, and a WASM component. The socket's stream is a line
;; stream, and read-line strips one trailing carriage return, so HTTP's
;; CRLF-terminated request line and headers read as plain lines (the blank line
;; ending the headers reads as ""). Response header lines get their carriage
;; return back via code-char 13 before write-line appends the newline.
;;
;; Run (interpreter):        java -jar $JAR examples/net/http-hello.lisp
;; Run (JVM):                java -jar $JAR examples/net/http-hello.lisp -o HttpHello.class && java HttpHello
;; Run (WASM component):     java -jar $JAR examples/net/http-hello.lisp -o http-hello.wasm --component && \
;;                           wasmtime run -W gc=y -W exceptions=y \
;;                             -S tcp=y -S inherit-network=y http-hello.wasm
;; Talk to it with:          curl http://127.0.0.1:8080/   (or a browser)

;; Appends the carriage return of an HTTP CRLF line ending (write-line then
;; appends the newline).
(defun crlf (s) (concatenate 'string s (format nil "~a" (code-char 13))))

;; Consumes the request headers up to the blank line that ends them.
(defun drain-headers (sock)
  (do ((line (read-line sock) (read-line sock)))
      ((or (null line) (string= line "")))))

(handler-case (let ((listener
                     (usocket:socket-listen "127.0.0.1" 8080 :reuse-address t)))
                (write-line "http server listening on http://127.0.0.1:8080/")
                (do ((n 1 (+ n 1)))
                    (nil)
                  ;; with-server-socket closes the accepted socket on every exit.
                  (usocket:with-server-socket (sock
                                               (usocket:socket-accept listener))
                    (let* ((stream (usocket:socket-stream sock))
                           (request (read-line stream)))
                      (if request
                          (let ((body
                                 (format nil
                                         "
hello from rontolisp
request ~a: ~a
"
                                         n request)))
                            (drain-headers stream)
                            (write-line (crlf "HTTP/1.1 200 OK") stream)
                            (write-line (crlf "Content-Type: text/html") stream)
                            ;; + 1: write-line terminates the body with a newline
                            (write-line (crlf
                                         (format nil "Content-Length: ~a"
                                                 (+ (length body) 1))) stream)
                            (write-line (crlf "Connection: close") stream)
                            (write-line (crlf "") stream)
                            (write-line body stream)
                            (write-line
                             (format nil "served request ~a: ~a" n
                                     request))))))))
  (usocket:socket-error (e)
    (declare (ignore e))
    (write-line "socket-listen failed (is port 8080 already in use?)")))


---

# FILE: references/examples/net/httpbin-clack.lisp

;; The Clack flavour of httpbin.lisp, plain: an application is a FUNCTION of the
;; environment plist that returns the (status headers body) list, and a
;; middleware is a function from application to application. Clack has no
;; router, so dispatch is a `cond` over :path-info. rontolisp's server protocol
;; IS Clack's, so this is an ordinary Clack program on the real clack.
;;
;; :server :rontolisp serves on the TARGET's native inbound transport, chosen at
;; compile time, so this ONE file is every host's program -- including, unedited,
;; the Worker examples/cloudflare-workers/httpbin-clack-one-source deploys.
;; :port applies where the program owns the socket and is ignored where the host
;; does. Preview 1 has no incoming TCP: the program compiles and clackup fails at
;; run time.
;;
;;   rontolisp examples/net/httpbin-clack.lisp   # first run downloads clack/lack
;;   curl 'http://127.0.0.1:8080/get?a=1&b=two'
;;   curl -X POST -d '{"name":"rontolisp"}' http://127.0.0.1:8080/post

(ql:quickload "clack")

;;; --- the endpoints -----------------------------------------------------------

;; clack's :raw-body is a synchronous stream, and nil when there is no body.
(defun read-body (stream)
  (if (null stream)
      ""
      (with-output-to-string (out)
        (do ((ch (read-char stream nil nil) (read-char stream nil nil)))
            ((null ch))
          (write-char ch out)))))

;; Parse the body as JSON when it looks like one, and fall back to null when it
;; does not parse -- which is what the real httpbin does.
(defun body-json (body)
  (if (and (stringp body) (> (length body) 0)
           (or (eql (char body 0) #\{) (eql (char body 0) #\[)))
      (handler-case (rontolisp:json-parse body) (error () 'null))
      'null))

(defun json-body (object)
  (list (format nil "~a~%" (rontolisp:json-stringify object))))

;; plist-hash-table and alist-hash-table give json-stringify the string-keyed
;; hash tables it serializes as objects (:method becomes "method"; an empty
;; query still renders {}), and the env :headers already is one.
(defun echo (env with-body)
  (let ((info
         (rontolisp:plist-hash-table
          (list :args (rontolisp:alist-hash-table
                       (rontolisp:query-params (getf env :query-string)))
                :headers (getf env :headers)
                :method (symbol-name (getf env :request-method))
                :path (getf env :path-info)))))
    (when with-body
      (let ((body (read-body (getf env :raw-body))))
        (setf (gethash "data" info) body)
        (setf (gethash "json" info) (body-json body))))
    (list 200 nil (json-body info))))

;; :request-method is an interned keyword, so the check is eq.
(defun endpoint (env method with-body)
  (if (eq (getf env :request-method) method)
      (echo env with-body)
      (list 405 nil
            (json-body
             (rontolisp:plist-hash-table
              (list :error "method not allowed"
                    :allowed (symbol-name method)))))))

;;; --- the application ---------------------------------------------------------

;; :path-info carries the decoded path only -- the query string arrives
;; separately -- so the comparisons are exact.
(defun app (env)
  (let ((path (getf env :path-info)))
    (cond ((string= path "/get") (endpoint env :GET nil))
          ((string= path "/post") (endpoint env :POST t))
          ((string= path "/put") (endpoint env :PUT t))
          ((string= path "/patch") (endpoint env :PATCH t))
          ((string= path "/delete") (endpoint env :DELETE t))
          (t (list 404 nil
                   (json-body
                    (rontolisp:plist-hash-table
                     (list :error "not found" :path path))))))))

;; A middleware takes an application and returns one, which is why no endpoint
;; above sets a header. Several of them compose with lack:builder.
(defun wrap-json (app)
  (lambda (env)
    (let ((response (funcall app env)))
      (list* (first response)
             (list* :content-type "application/json" (second response))
             (cddr response)))))

(clack:clackup (wrap-json #'app) :server :rontolisp :port 8080 :use-thread nil)


---

# FILE: references/examples/net/httpbin-clos.lisp

;; The CLOS flavour of httpbin.lisp: the same miniature httpbin, but the echo
;; responses are CLOS instances instead of hash tables. rontolisp:json-stringify
;; serializes a standard-object as a JSON object -- each slot in definition
;; order -- and a slot may itself hold a hash table (a nested object). So a
;; class gives the fixed-shape envelope while "args" and "headers" stay hash
;; tables (their keys are dynamic). This is exactly how com.inuoe.jzon serializes
;; a standard-object, so switching rontolisp:json-stringify to
;; com.inuoe.jzon:stringify keeps the same output (see httpbin-jzon.lisp).
;;
;;   GET    /         -> an HTML index page (rendered with cl-who) listing the routes below
;;   GET    /get      -> {"args": {...}, "headers": {...}, "method": "GET",  "path": "/get"}
;;   POST   /post     -> the same plus {"data": "", "json": }
;;   PUT/PATCH/DELETE  -> ditto ; a wrong method 405, an unknown path 404.
;;
;; The index page is the real upstream cl-who ((X)HTML markup library), pulled
;; in with ql:quickload like postgres-web.lisp -- with-html-output-to-string
;; expands the template below at macro-expansion time, so it compiles to
;; ordinary string building on every backend, JVM and WASM included.
;;
;; Run (interpreter, blocking server on :8080):
;;   java -jar $JAR examples/net/httpbin-clos.lisp
;; Run (JVM class; needs the rontolisp jar on the classpath):
;;   java -jar $JAR examples/net/httpbin-clos.lisp -o HttpbinClos.class && java -cp $JAR:. HttpbinClos
;; Run (WASI component under wasmtime serve):
;;   java -jar $JAR examples/net/httpbin-clos.lisp -o httpbin-clos.wasm --component && \
;;     wasmtime serve -W gc=y -W exceptions=y httpbin-clos.wasm
;; Talk to it with:
;;   curl http://127.0.0.1:8080/
;;   curl 'http://127.0.0.1:8080/get?a=1&b=two'
;;   curl -X POST -d '{"name":"rontolisp"}' http://127.0.0.1:8080/post

(ql:quickload "cl-who")

;; --- request helpers ------------------------------------------------------

;; args and headers have dynamic keys, so they stay hash tables (which nest as
;; JSON objects inside the response instance). query-params gives an alist, so
;; rontolisp:alist-hash-table (a subset of alexandria:alist-hash-table) turns
;; it into a hash table; the env :headers already is one.

;; Parse the body as JSON when it looks like a JSON object or array, else the
;; symbol null (which stringifies to JSON null).
(defun body-json (body)
  (if (and (stringp body) (> (length body) 0)
           (or (eql (char body 0) #\{) (eql (char body 0) #\[)))
      (rontolisp:json-parse body)
      'null))

;; --- responses ------------------------------------------------------------

;; The echo envelope as a class: slots serialize as object keys in this order.
;; POST-family responses add the body fields, so they extend the base class --
;; inherited slots come first, giving args/headers/method/path/data/json.
(defclass echo-response ()
  ((args :initarg :args) (headers :initarg :headers) (method :initarg :method)
   (path :initarg :path)))

(defclass echo-with-body-response (echo-response)
  ((data :initarg :data) (json :initarg :json)))

(defun json-response (status obj)
  (list status '(:content-type "application/json")
        (list (format nil "~a~%" (rontolisp:json-stringify obj)))))

;; The index page: a plain cl-who template, no request data to escape.
(defun index-page ()
  (list 200 '(:content-type "text/html; charset=utf-8")
        (list
         (cl-who:with-html-output-to-string (s)
           (:html (:head (:title "rontolisp httpbin"))
                  (:body (:h1 "rontolisp httpbin")
                   (:p "A miniature httpbin: every route below echoes the "
                       "request back as JSON.")
                   (:ul
                    (:li (:code "GET /get") " -- args, headers, method, path")
                    (:li (:code "POST /post")
                     " -- ditto, plus the request body (raw and parsed JSON)")
                    (:li (:code "PUT /put") " -- ditto")
                    (:li (:code "PATCH /patch") " -- ditto")
                    (:li (:code "DELETE /delete") " -- ditto")) (:p "Try it:")
                   (:pre
                    (:code
                     (cl-who:esc "curl 'http://127.0.0.1:8080/get?a=1&b=two'")))
                   (:pre
                    (:code
                     (cl-who:esc
                      "curl -X POST -d '{\"name\":\"rontolisp\"}' http://127.0.0.1:8080/post")))))))))

(defun echo (env)
  (json-response 200
                 (make-instance 'echo-response
                                :args (rontolisp:alist-hash-table
                                       (rontolisp:query-params
                                        (getf env :query-string)))
                                :headers (getf env :headers)
                                :method (symbol-name (getf env :request-method))
                                :path (getf env :path-info))))

(defun echo-with-body (env)
  (json-response 200
                 (make-instance 'echo-with-body-response
                                :args (rontolisp:alist-hash-table
                                       (rontolisp:query-params
                                        (getf env :query-string)))
                                :headers (getf env :headers)
                                :method (symbol-name (getf env :request-method))
                                :path (getf env :path-info)
                                :data (getf env :body)
                                :json (body-json (getf env :body)))))

;; Echo the request (with the body fields when with-body is non-nil) only
;; when the request used the expected method; otherwise 405 (:request-method
;; is an interned keyword, so the comparison is eq). The ad-hoc error
;; objects stay hash tables (rontolisp:plist-hash-table), the flexible tool for
;; a shape that is not worth a class.
(defun echo-when (env expected with-body)
  (cond ((not (eq (getf env :request-method) expected))
         (json-response 405
                        (rontolisp:plist-hash-table
                         (list :error "method not allowed"
                               :allowed (symbol-name expected)))))
        (with-body (echo-with-body env))
        (t (echo env))))

;; --- routing --------------------------------------------------------------

(defun route (env)
  (let ((path (getf env :path-info)))
    (cond ((string= path "/") (index-page))
          ((string= path "/get") (echo-when env :GET nil))
          ((string= path "/post") (echo-when env :POST t))
          ((string= path "/put") (echo-when env :PUT t))
          ((string= path "/patch") (echo-when env :PATCH t))
          ((string= path "/delete") (echo-when env :DELETE t))
          (t (json-response 404
                            (rontolisp:plist-hash-table
                             (list :error "not found" :path path)))))))

;; The env :raw-body is an asynchronous stream on every backend; drain it once
;; here and hand the helpers an env whose :body is the whole string.
(rontolisp:async-defun handle (env)
  (let ((body (rontolisp:await (rontolisp:read-all (getf env :raw-body)))))
    (route (append (list :body body) env))))

;; On the interpreter / JVM this blocks and serves on port 8080; under
;; --component the port argument is ignored (the host provides the socket).
(rontolisp:http-handler 'handle 8080)


---

# FILE: references/examples/net/httpbin-clos/spin.toml

spin_manifest_version = 2

[application]
name = "httpbin"
version = "0.1.0"
description = "A rontolisp:http-handler served as a wasi:http/handler@0.3.0 component."

[[trigger.http]]
route = "/..."
component = "httpbin"

[component.httpbin]
source = "app.wasm"

[component.httpbin.build]
command = "rontolisp ../httpbin-clos.lisp -o app.wasm --component"
watch = ["../httpbin-clos.lisp"]


---

# FILE: references/examples/net/httpbin-jzon.lisp

;; The jzon flavour of httpbin.lisp: the very same program, but the JSON is
;; parsed and rendered by the real com.inuoe.jzon library instead of
;; rontolisp:json-parse / rontolisp:json-stringify. rontolisp:json-* is a
;; lightweight subset of jzon with the same value mapping, so the switch is
;; mechanical -- only the two call sites change (json-parse -> jzon:parse,
;; json-stringify -> jzon:stringify), and everything else (rontolisp:plist-hash-table
;; for the objects, the symbol null for JSON null) works unchanged. Reach for
;; jzon when you outgrow the subset (pretty printing, a streaming writer,
;; :replacer, custom serialization).
;;
;;   GET    /get      -> {"args": {...}, "headers": {...}, "method": "GET",  "path": "/get"}
;;   POST   /post     -> the same plus {"data": "", "json": }
;;   PUT/PATCH/DELETE  -> ditto ; a wrong method 405, an unknown path 404.
;;
;; ql:quickload downloads com.inuoe.jzon (and caches it) the first time -- at
;; compile time for the compiled backends, so the library is baked in.
;;
;; Run (interpreter, blocking server on :8080):
;;   java -jar $JAR examples/net/httpbin-jzon.lisp
;; Run (JVM class; needs the rontolisp jar on the classpath):
;;   java -jar $JAR examples/net/httpbin-jzon.lisp -o HttpbinJzon.class && java -cp $JAR:. HttpbinJzon
;; Run (WASI component under wasmtime serve):
;;   java -jar $JAR examples/net/httpbin-jzon.lisp -o httpbin-jzon.wasm --component && \
;;     wasmtime serve -W gc=y -W exceptions=y httpbin-jzon.wasm

(ql:quickload '#:com.inuoe.jzon)

;; --- request helpers ------------------------------------------------------

;; Parse the body with jzon when it looks like a JSON object or array, else the
;; symbol null (jzon's JSON-null sentinel, which jzon:stringify renders as null).
(defun body-json (body)
  (if (and (stringp body) (> (length body) 0)
           (or (eql (char body 0) #\{) (eql (char body 0) #\[)))
      (com.inuoe.jzon:parse body)
      'null))

;; --- responses ------------------------------------------------------------

(defun json-response (status obj)
  (list status '(:content-type "application/json")
        (list (format nil "~a~%" (com.inuoe.jzon:stringify obj)))))

;; The common echo fields, as a JSON object. rontolisp:plist-hash-table and
;; rontolisp:alist-hash-table (subsets of the alexandria utilities) build the
;; hash tables, which jzon:stringify serializes as objects (their keyword keys
;; down-cased); the standalone utilities need no change when the JSON library
;; does. The env :headers is already a string-keyed hash table, so it nests
;; as an object with no conversion.
(defun request-info (env)
  (rontolisp:plist-hash-table
   (list :args (rontolisp:alist-hash-table
                (rontolisp:query-params (getf env :query-string)))
         :headers (getf env :headers)
         :method (symbol-name (getf env :request-method))
         :path (getf env :path-info))))

(defun echo (env) (json-response 200 (request-info env)))

(defun echo-with-body (env)
  (let ((info (request-info env)))
    (setf (gethash "data" info) (getf env :body))
    (setf (gethash "json" info) (body-json (getf env :body)))
    (json-response 200 info)))

;; :request-method is an interned keyword, so the comparison is eq.
(defun echo-when (env expected with-body)
  (cond ((not (eq (getf env :request-method) expected))
         (json-response 405
                        (rontolisp:plist-hash-table
                         (list :error "method not allowed"
                               :allowed (symbol-name expected)))))
        (with-body (echo-with-body env))
        (t (echo env))))

;; --- routing --------------------------------------------------------------

(defun route (env)
  (let ((path (getf env :path-info)))
    (cond ((string= path "/get") (echo-when env :GET nil))
          ((string= path "/post") (echo-when env :POST t))
          ((string= path "/put") (echo-when env :PUT t))
          ((string= path "/patch") (echo-when env :PATCH t))
          ((string= path "/delete") (echo-when env :DELETE t))
          (t (json-response 404
                            (rontolisp:plist-hash-table
                             (list :error "not found" :path path)))))))

;; The env :raw-body is an asynchronous stream on every backend; drain it once
;; here and hand the helpers an env whose :body is the whole string.
(rontolisp:async-defun handle (env)
  (let ((body (rontolisp:await (rontolisp:read-all (getf env :raw-body)))))
    (route (append (list :body body) env))))

;; On the interpreter / JVM this blocks and serves on port 8080; under
;; --component the port argument is ignored (the host provides the socket).
(rontolisp:http-handler 'handle 8080)


---

# FILE: references/examples/net/httpbin-ningle.lisp

;; The ningle flavour of httpbin-clack.lisp: the application is a CLOS OBJECT.
;; Four things make it ningle rather than a second route table:
;;
;;   * a route is an ASSIGNMENT, so the five echo endpoints are a loop;
;;   * a controller returns the BODY and says the rest by mutating
;;     ningle:*response* -- the (status headers body) triple never appears;
;;   * a controller receives the PARAMETERS, and by then lack/request has decoded
;;     the query string and PARSED the body, so nothing here reads a stream or
;;     parses JSON;
;;   * declining means NOT MATCHING (returning nil answers an empty body), so
;;     every miss lands on ningle:not-found, a METHOD on the application class.
;;
;; Run (the first run downloads clack/lack/ningle into ~/.rontolisp/quicklisp):
;;   rontolisp examples/net/httpbin-ningle.lisp
;;   rontolisp examples/net/httpbin-ningle.lisp -o HttpbinNingle.class && \
;;     java -cp rontolisp-exec.jar:. HttpbinNingle
;;   rontolisp examples/net/httpbin-ningle.lisp -o httpbin-ningle.wasm --component && \
;;     wasmtime serve -W gc=y -W exceptions=y -S cli=y -S tcp=y -S inherit-network=y \
;;       httpbin-ningle.wasm
;; Preview 1 has no incoming TCP: the program compiles, clackup fails at run
;; time. Under --component the host owns the socket, so :port is ignored.
;;
;;   curl 'http://127.0.0.1:8080/get?a=1&b=two'
;;   curl -H 'content-type: application/json' -d '{"name":"rontolisp"}' \
;;     http://127.0.0.1:8080/post          # the parsed body comes back as "form"
;;   curl http://127.0.0.1:8080/status/418

(ql:quickload '("clack" "ningle"))

(defvar *app* (make-instance 'ningle:app))

;;; --- answering ---------------------------------------------------------------

(defun respond-json (object)
  (setf (lack.response:response-headers ningle:*response*)
        (list :content-type "application/json"))
  (format nil "~a~%" (rontolisp:json-stringify object)))

(defun respond-text (text)
  (setf (lack.response:response-headers ningle:*response*)
        (list :content-type "text/plain; charset=utf-8"))
  (format nil "~a~%" text))

(defun set-status (code)
  (setf (lack.response:response-status ningle:*response*) code))

;;; --- the echo endpoint -------------------------------------------------------

;; ONE controller for every echo endpoint: per method there is nothing left to
;; do. `args` is the query string and `form` the parsed body -- for a JSON post
;; that is the JSON object itself -- and the alist ningle hands the controller
;; is those two appended, which is why this one ignores it.
(defun echo (params)
  (declare (ignore params))
  (let ((request ningle:*request*))
    (respond-json
     (rontolisp:plist-hash-table
      (list :method (symbol-name (lack.request:request-method request))
            :path (lack.request:request-path-info request)
            :args (rontolisp:alist-hash-table
                   (lack.request:request-query-parameters request))
            :form (rontolisp:alist-hash-table
                   (lack.request:request-body-parameters request))
            :headers (lack.request:request-headers request))))))

;;; --- the routes --------------------------------------------------------------

;; Rules are tried in the order they were assigned, so each path gets two: the
;; ONE method it answers, then :ANY for the 405. That leaves not-found with only
;; the answer it is really for.
(dolist (endpoint
         '(("/get" . :GET) ("/post" . :POST) ("/put" . :PUT) ("/patch" . :PATCH)
           ("/delete" . :DELETE)))
  (let ((path (car endpoint)) (allowed (cdr endpoint)))
    (setf (ningle:route *app* path :method allowed) #'echo)
    (setf (ningle:route *app* path :method :ANY)
          (lambda (params)
            (declare (ignore params))
            (set-status 405)
            (respond-json
             (rontolisp:plist-hash-table
              (list :error "method not allowed"
                    :allowed (symbol-name allowed))))))))

;; :ANY used as itself rather than as a fallback.
(setf (ningle:route *app* "/anything" :method :ANY) #'echo)

;; myway's other rule spelling: a REGEX, whose capture groups arrive as
;; :captures. It fits because a code that is not three digits then matches no
;; rule at all -- where a "/status/:code" template would match "/status/teapot"
;; and leave the controller with nothing good to answer.
(setf (ningle:route *app* "/status/([0-9]{3})" :regexp t)
      (lambda (params)
        (let ((code (parse-integer (first (cdr (assoc :captures params))))))
          (set-status code)
          (respond-text code))))

(defmethod ningle:not-found ((app ningle:app))
  (set-status 404)
  (respond-json
   (rontolisp:plist-hash-table
    (list :error "not found"
          :path (lack.request:request-path-info ningle:*request*)))))

(clack:clackup *app* :server :rontolisp :port 8080 :use-thread nil)


---

# FILE: references/examples/net/httpbin-tiny-routes.lisp

;; The tiny-routes flavour of httpbin-clack.lisp: the application is COMPOSED
;; out of routes and middleware. `pipe` threads the route table through
;; wrap-request-body (the raw body, read) and wrap-query-parameters (the query
;; string, parsed), and the JSON group through wrap-response-content-type -- so
;; the echo handlers neither drain a stream nor set a header. A route answers or
;; returns nil to DECLINE, which is how a wrong method reaches the catch-all and
;; how /status/:code refuses a bad code. `tiny` is the library's own nickname,
;; so nothing has to be imported to reach any of it.
;;
;; "tiny-routes/lite" is the ppcre-free opt-in system; the full "tiny-routes"
;; runs this file unchanged and costs a regex engine.
;;
;; Run (the first run downloads clack/lack/tiny-routes into ~/.rontolisp/quicklisp):
;;   rontolisp examples/net/httpbin-tiny-routes.lisp
;;   rontolisp examples/net/httpbin-tiny-routes.lisp -o HttpbinTinyRoutes.class && \
;;     java -cp rontolisp-exec.jar:. HttpbinTinyRoutes
;;   rontolisp examples/net/httpbin-tiny-routes.lisp -o httpbin-tiny-routes.wasm --component && \
;;     wasmtime serve -W gc=y -W exceptions=y -S cli=y -S tcp=y -S inherit-network=y \
;;       httpbin-tiny-routes.wasm
;; Preview 1 has no incoming TCP: the program compiles, clackup fails at run
;; time. Under --component the host owns the socket, so :port is ignored.
;;
;;   curl 'http://127.0.0.1:8080/get?a=1&b=two'
;;   curl -X POST -d '{"name":"rontolisp"}' http://127.0.0.1:8080/post
;;   curl http://127.0.0.1:8080/status/418

(ql:quickload '("clack" "tiny-routes/lite"))

;;; --- the handlers ------------------------------------------------------------

(defun body-json (body)
  (if (and (stringp body) (> (length body) 0)
           (or (eql (char body 0) #\{) (eql (char body 0) #\[)))
      (handler-case (rontolisp:json-parse body) (error () 'null))
      'null))

(defun json (object) (format nil "~a~%" (rontolisp:json-stringify object)))

;; The echo document. Everything it reports is already on the request, so this
;; only shapes JSON: plist-hash-table turns tiny-routes' query plist and this
;; plist into the string-keyed hash tables json-stringify renders as objects.
(defun echo (req with-body)
  (let ((info
         (rontolisp:plist-hash-table
          (list :args (rontolisp:plist-hash-table
                       (tiny:request-get req :query-parameters))
                :headers (tiny:request-headers req)
                :method (symbol-name (tiny:request-method req))
                :path (tiny:path-info req)))))
    (when with-body
      (let ((body (tiny:request-body req "")))
        (setf (gethash "data" info) body)
        (setf (gethash "json" info) (body-json body))))
    (tiny:ok (json info))))

;; Nothing claimed the request. The table is not a second dispatch: it is what
;; tells a known path that DECLINED on its method (405, naming the one that
;; works) from a path no route has (404).
(defparameter *endpoints*
  '(("/get" . :GET) ("/post" . :POST) ("/put" . :PUT) ("/patch" . :PATCH)
    ("/delete" . :DELETE)))

(defun no-route (req)
  (let* ((path (tiny:path-info req))
         (allowed (cdr (assoc path *endpoints* :test #'string=))))
    (if allowed
        (tiny:method-not-allowed
         (json
          (rontolisp:plist-hash-table
           (list :error "method not allowed" :allowed (symbol-name allowed)))))
        (tiny:not-found
         (json
          (rontolisp:plist-hash-table (list :error "not found" :path path)))))))

;;; --- the routes --------------------------------------------------------------

;; Each route answers the one method it names and declines every other, so the
;; catch-all at the bottom is reached by both a wrong method and an unknown path.
(tiny:define-routes *json-routes*
  (tiny:define-get "/get" (req) (echo req nil))
  (tiny:define-post "/post" (req) (echo req t))
  (tiny:define-put "/put" (req) (echo req t))
  ;; tiny-routes has no define-patch; matching the method is all the other
  ;; macros add over define-any, and that matcher is exported.
  (tiny:wrap-request-matches-method
   (tiny:define-any "/patch" (req) (echo req t)) :patch)
  (tiny:define-delete "/delete" (req) (echo req t))
  (tiny:define-any "*" (req) (no-route req)))

;; The one endpoint that does not answer JSON, so it is its own group with its
;; own content type. A :code that is not a number declines.
(defparameter *status-route*
  (tiny:pipe (tiny:define-get "/status/:code" (req)
               (let ((code
                      (parse-integer (tiny:path-parameter req :code)
                                     :junk-allowed t)))
                 (when code
                   (tiny:make-response :status code
                                       :body (format nil "~a~%" code)))))
             (tiny:wrap-response-content-type "text/plain; charset=utf-8")))

(tiny:define-routes *routes*
  *status-route*
  (tiny:pipe *json-routes*
             (tiny:wrap-response-content-type "application/json")))

(defparameter *app*
  (tiny:pipe *routes* (tiny:wrap-request-body) (tiny:wrap-query-parameters)))

(clack:clackup *app* :server :rontolisp :port 8080 :use-thread nil)


---

# FILE: references/examples/net/httpbin.lisp

;; A miniature httpbin (https://httpbin.ik.am) built on rontolisp:http-handler --
;; the advanced companion of http-handler.lisp. Five echo endpoints respond
;; with a JSON document describing the request, built with
;; rontolisp:json-stringify (and rontolisp:json-parse for the request body):
;;
;;   GET    /get      -> {"args": {...}, "headers": {...}, "method": "GET",  "path": "/get"}
;;   POST   /post     -> the same plus {"data": "", "json": }
;;   PUT    /put      -> ditto
;;   PATCH  /patch    -> ditto
;;   DELETE /delete   -> ditto
;;
;; A wrong method answers 405, an unknown path 404. Query strings are parsed
;; into "args" with rontolisp:query-params (keys and values url-decoded);
;; "json" is filled only when the body starts
;; with '{' or '[' (malformed JSON then signals an error -- rontolisp has no
;; condition handling to fall back to null like the real httpbin).
;; "headers" echoes the request header hash table the Clack environment
;; carries (names lowercased, repeated headers joined with ", ").
;;
;; Run (interpreter, blocking server on :8080):
;;   java -jar $JAR examples/net/httpbin.lisp
;; Run (JVM class; needs the rontolisp jar on the classpath):
;;   java -jar $JAR examples/net/httpbin.lisp -o Httpbin.class && java -cp $JAR:. Httpbin
;; Run (WASI component under wasmtime serve):
;;   java -jar $JAR examples/net/httpbin.lisp -o httpbin.wasm --component && \
;;     wasmtime serve -W gc=y -W exceptions=y httpbin.wasm
;; Talk to it with:
;;   curl 'http://127.0.0.1:8080/get?a=1&b=two'
;;   curl -X POST -d '{"name":"rontolisp"}' http://127.0.0.1:8080/post

;; --- request helpers ------------------------------------------------------

;; Parse the body as JSON when it looks like a JSON object or array.
(defun body-json (body)
  (if (and (stringp body) (> (length body) 0)
           (or (eql (char body 0) #\{) (eql (char body 0) #\[)))
      (rontolisp:json-parse body)
      'null))

;; --- responses ------------------------------------------------------------

(defun json-response (status obj)
  (list status '(:content-type "application/json")
        (list (format nil "~a~%" (rontolisp:json-stringify obj)))))

;; The common echo fields, as a JSON object. rontolisp:plist-hash-table (a
;; subset of alexandria:plist-hash-table) turns a keyword plist into a
;; string-keyed hash table, which json-stringify serializes as an object --
;; keyword keys are down-cased, so :method becomes "method". "args" is itself
;; an object: query-params gives an alist, so rontolisp:alist-hash-table (a
;; subset of alexandria:alist-hash-table) turns it into a hash table -- an
;; empty (or missing) query still serializes as {}. "headers" needs no
;; conversion: the env :headers is already a string-keyed hash table.
(defun request-info (env)
  (rontolisp:plist-hash-table
   (list :args (rontolisp:alist-hash-table
                (rontolisp:query-params (getf env :query-string)))
         :headers (getf env :headers)
         :method (symbol-name (getf env :request-method))
         :path (getf env :path-info))))

(defun echo (env) (json-response 200 (request-info env)))

(defun echo-with-body (env)
  (let ((info (request-info env)))
    (setf (gethash "data" info) (getf env :body))
    (setf (gethash "json" info) (body-json (getf env :body)))
    (json-response 200 info)))

;; Echo the request (with the body fields when with-body is non-nil) only
;; when the request used the expected method; otherwise 405. :request-method
;; is an interned keyword, so the comparison is eq.
(defun echo-when (env expected with-body)
  (cond ((not (eq (getf env :request-method) expected))
         (json-response 405
                        (rontolisp:plist-hash-table
                         (list :error "method not allowed"
                               :allowed (symbol-name expected)))))
        (with-body (echo-with-body env))
        (t (echo env))))

;; --- routing --------------------------------------------------------------

;; The env plist's :path-info carries the (percent-decoded) path only (the
;; query string arrives separately as :query-string), so the comparisons are
;; exact.
(defun route (env)
  (let ((path (getf env :path-info)))
    (cond ((string= path "/get") (echo-when env :GET nil))
          ((string= path "/post") (echo-when env :POST t))
          ((string= path "/put") (echo-when env :PUT t))
          ((string= path "/patch") (echo-when env :PATCH t))
          ((string= path "/delete") (echo-when env :DELETE t))
          (t (json-response 404
                            (rontolisp:plist-hash-table
                             (list :error "not found" :path path)))))))

;; The env :raw-body is an asynchronous stream on every backend; drain it once
;; here and hand the helpers an env whose :body is the whole string (getf
;; finds the prepended pair first).
(rontolisp:async-defun handle (env)
  (let ((body (rontolisp:await (rontolisp:read-all (getf env :raw-body)))))
    (route (append (list :body body) env))))

;; On the interpreter / JVM this blocks and serves on port 8080; under
;; --component the port argument is ignored (the host provides the socket).
(rontolisp:http-handler 'handle 8080)


---

# FILE: references/examples/net/https-hello.lisp

;; The TLS version of http-hello.lisp: a minimal HTTPS (HTTP/1.1 over TLS)
;; server on port 8443. Same behavior -- answers every request with a small
;; HTML page showing the request line and a running request counter, one
;; connection per request (Connection: close). Serves forever -- stop with
;; Ctrl-C.
;;
;; rontolisp:tls-listen needs a PKCS12 keystore holding the server key and
;; certificate. Generate a self-signed one for localhost with the JDK keytool
;; (or export one from openssl with `openssl pkcs12 -export`):
;;
;;   keytool -genkeypair -alias rontolisp-tls -keyalg EC -dname CN=localhost \
;;     -validity 365 -ext SAN=ip:127.0.0.1,dns:localhost \
;;     -storetype PKCS12 -keystore tls-server.p12 \
;;     -storepass changeit -keypass changeit
;;
;; The listener handle works with the portable usocket:socket-accept (usocket
;; is a shim over the built-in rontolisp:tcp-* functions), and each accepted
;; socket completes its TLS handshake on the first read -- everything after
;; tls-listen is identical to the plain-TCP http-hello.lisp. usocket has no TLS
;; listener of its own, so the listen call stays rontolisp:tls-listen.
;;
;; TLS is interpreter/JVM only (a compile error on the WASM backend).
;;
;; Run (interpreter):        java -jar $JAR examples/net/https-hello.lisp
;; Run (JVM):                java -jar $JAR examples/net/https-hello.lisp -o HttpsHello.class && java HttpsHello
;; Talk to it with:          curl -k https://127.0.0.1:8443/
;;                           (-k because the certificate is self-signed; or trust it
;;                           explicitly with --cacert after exporting the certificate)

;; Appends the carriage return of an HTTP CRLF line ending (write-line then
;; appends the newline).
(defun crlf (s) (concatenate 'string s (format nil "~a" (code-char 13))))

;; Consumes the request headers up to the blank line that ends them.
(defun drain-headers (sock)
  (do ((line (read-line sock) (read-line sock)))
      ((or (null line) (string= line "")))))

;; Unlike tcp-listen on the WASM backend, tls-listen never returns nil: a
;; missing keystore, a wrong password or a busy port signals an error instead
;; (interpreter and JVM both signal), so there is no nil check here.
(let ((listener (rontolisp:tls-listen "tls-server.p12" "changeit" 8443)))
  (write-line "https server listening on https://127.0.0.1:8443/")
  (do ((n 1 (+ n 1)))
      (nil)
    ;; with-server-socket closes the accepted socket on every exit.
    (usocket:with-server-socket (sock (usocket:socket-accept listener))
      (let* ((stream (usocket:socket-stream sock)) (request (read-line stream)))
        (if request
            (let ((body
                   (format nil
                           "
hello from rontolisp over TLS
request ~a: ~a
"
                           n request)))
              (drain-headers stream)
              (write-line (crlf "HTTP/1.1 200 OK") stream)
              (write-line (crlf "Content-Type: text/html") stream)
              ;; + 1: write-line terminates the body with a newline
              (write-line
               (crlf (format nil "Content-Length: ~a" (+ (length body) 1)))
               stream)
              (write-line (crlf "Connection: close") stream)
              (write-line (crlf "") stream)
              (write-line body stream)
              (write-line (format nil "served request ~a: ~a" n request))))))))


---

# FILE: references/examples/net/kv-server-tls.lisp

;; The TLS version of kv-server.lisp: the same miniature Redis-compatible
;; in-memory key-value server, but serving TLS on port 6380 (like a real
;; Redis with --tls-port). The RESP2 protocol handling is identical -- only
;; the listen call differs: rontolisp:tls-listen wraps the listener in TLS,
;; the portable usocket:socket-accept accepts connections (usocket has no TLS
;; listener of its own, so the listen call stays rontolisp:tls-listen), and
;; each accepted socket completes its handshake on the first read.
;;
;; See kv-server.lisp for the supported commands and protocol notes.
;;
;; rontolisp:tls-listen needs a PKCS12 keystore holding the server key and
;; certificate. Generate a self-signed one for localhost with the JDK keytool
;; (or export one from openssl with `openssl pkcs12 -export`):
;;
;;   keytool -genkeypair -alias rontolisp-tls -keyalg EC -dname CN=localhost \
;;     -validity 365 -ext SAN=ip:127.0.0.1,dns:localhost \
;;     -storetype PKCS12 -keystore tls-server.p12 \
;;     -storepass changeit -keypass changeit
;;
;; TLS is interpreter/JVM only (a compile error on the WASM backend).
;;
;; Run (interpreter):        java -jar $JAR examples/net/kv-server-tls.lisp
;; Run (JVM):                java -jar $JAR examples/net/kv-server-tls.lisp -o KvServerTls.class && java KvServerTls
;; Talk to it with:          redis-cli --tls --insecure -p 6380 set greeting hello
;;                           redis-cli --tls --insecure -p 6380 get greeting
;;                           (--insecure because the certificate is self-signed; or
;;                           trust it explicitly with --cacert after exporting it)

;; --- small string helpers ---------------------------------------------------

;; Appends the carriage return of a RESP CRLF line ending (write-line then
;; appends the newline).
(defun crlf (s) (concatenate 'string s (format nil "~a" (code-char 13))))

;; "SET key value" -> ("SET" "key" "value")
(defun split-words (s)
  (cond ((string= s "") nil)
        (t (let ((p (position #\space s)))
             (if p
                 (cons (subseq s 0 p) (split-words (subseq s (+ p 1))))
                 (list s))))))

;; ("hello" "world") -> "hello world"
(defun join-words (ws)
  (cond ((null ws) "")
        ((null (cdr ws)) (car ws))
        (t (concatenate 'string (car ws) " " (join-words (cdr ws))))))

;; t when s is a non-empty run of decimal digits (with an optional leading -).
(defun integer-string-p (s)
  (let* ((n (length s)) (start (if (and (> n 0) (char= (char s 0) #\-)) 1 0)))
    (and (> n start)
         (do ((i start (+ i 1)))
             ((or (>= i n) (not (digit-char-p (char s i)))) (>= i n))))))

;; --- RESP replies -----------------------------------------------------------

(defun reply-simple (s sock)
  (write-line (crlf (concatenate 'string "+" s)) sock))

(defun reply-error (s sock)
  (write-line (crlf (concatenate 'string "-ERR " s)) sock))

(defun reply-int (n sock) (write-line (crlf (format nil ":~a" n)) sock))

(defun reply-bulk (s sock)
  (if s
      (progn
        (write-line (crlf (format nil "$~a" (length s))) sock)
        (write-line (crlf s) sock))
      (write-line (crlf "$-1") sock)))

(defun reply-array-header (n sock)
  (write-line (crlf (format nil "*~a" n)) sock))

;; --- request framing --------------------------------------------------------

;; Reads one RESP bulk-string element: the "$" header line, then the
;; payload line (the payload must not contain a newline).
(defun read-bulk (sock)
  (let ((header (read-line sock))) (if header (read-line sock) nil)))

(defun read-resp-array (count sock acc)
  (if (<= count 0)
      (reverse acc)
      (let ((arg (read-bulk sock)))
        (if arg (read-resp-array (- count 1) sock (cons arg acc)) nil))))

;; Reads one command as a list of argument strings: a "*" line starts a
;; RESP2 array (what redis-cli sends); anything else is an inline command
;; (what telnet/nc users type). nil at connection close.
(defun read-command (sock)
  (let ((line (read-line sock)))
    (cond ((null line) nil)
          ((string= line "") (read-command sock))
          ((char= (char line 0) #\*)
           (let ((count (subseq line 1)))
             (if (integer-string-p count)
                 (read-resp-array (parse-integer count) sock nil)
                 (list "!bad-frame"))))
          (t (split-words line)))))

;; --- commands ---------------------------------------------------------------

;; Handles one command; returns nil after QUIT (closing the session).
(defun handle-command (args store sock)
  (let ((cmd (string-upcase (car args))) (key (cadr args)))
    (cond
     ((string= cmd "PING")
      (if key (reply-bulk key sock) (reply-simple "PONG" sock))
      t)
     ((string= cmd "SET")
      (if (and key (cddr args))
          (progn
            (setf (gethash key store) (join-words (cddr args)))
            (reply-simple "OK" sock))
          (reply-error "wrong number of arguments for 'set' command" sock))
      t)
     ((string= cmd "GET")
      (if key
          (reply-bulk (gethash key store) sock)
          (reply-error "wrong number of arguments for 'get' command" sock))
      t)
     ((string= cmd "DEL")
      (let ((removed 0))
        (dolist (k (cdr args))
          (when (gethash k store)
            (remhash k store)
            (incf removed)))
        (reply-int removed sock))
      t)
     ((string= cmd "EXISTS")
      (reply-int (if (and key (gethash key store)) 1 0) sock)
      t)
     ((string= cmd "INCR")
      (let ((current (if key (or (gethash key store) "0") "0")))
        (cond ((null key)
               (reply-error "wrong number of arguments for 'incr' command"
                            sock))
              ((integer-string-p current)
               (let ((n (+ (parse-integer current) 1)))
                 (setf (gethash key store) (format nil "~a" n))
                 (reply-int n sock)))
              (t (reply-error "value is not an integer or out of range" sock))))
      t)
     ((string= cmd "KEYS")
      (let ((pattern (or key "*")) (keys nil))
        (maphash (lambda (k v)
                   (if (or (string= pattern "*") (string= pattern k))
                       (push k keys))) store)
        (reply-array-header (length keys) sock)
        (dolist (k keys) (reply-bulk k sock)))
      t)
     ((string= cmd "DBSIZE")
      (reply-int (hash-table-count store) sock)
      t)
     ((string= cmd "COMMAND")
      ;; redis-cli asks COMMAND DOCS on connect; an empty array satisfies it.
      (reply-array-header 0 sock)
      t)
     ((string= cmd "QUIT")
      (reply-simple "OK" sock)
      nil)
     (t
      (reply-error (format nil "unknown command '~a'" (car args)) sock)
      t))))

;; --- server loop ------------------------------------------------------------

;; Unlike tcp-listen on the WASM backend, tls-listen never returns nil: a
;; missing keystore, a wrong password or a busy port signals an error instead
;; (interpreter and JVM both signal), so there is no nil check here.
(let ((store (make-hash-table))
      (listener (rontolisp:tls-listen "tls-server.p12" "changeit" 6380)))
  (write-line
   "mini-redis (TLS) listening on 127.0.0.1:6380 (try: redis-cli --tls --insecure -p 6380 ping)")
  (do ((n 1 (+ n 1)))
      (nil)
    ;; with-server-socket closes the accepted socket on every exit.
    (usocket:with-server-socket (sock (usocket:socket-accept listener))
      (let ((stream (usocket:socket-stream sock)))
        (do ((args (read-command stream) (read-command stream)))
            ((or (null args) (not (handle-command args store stream)))))))))


---

# FILE: references/examples/net/kv-server.lisp

;; A miniature Redis-compatible in-memory key-value server on port 6379.
;;
;; It speaks enough of RESP2 (the Redis serialization protocol) that the real
;; `redis-cli` connects and works, and -- like real Redis -- it also accepts
;; "inline commands" (a plain space-separated line), so `telnet 127.0.0.1 6379`
;; or `nc 127.0.0.1 6379` work too. Both framings arrive as CRLF-terminated
;; lines, which read-line reads as plain lines (one trailing carriage return is
;; stripped on every backend).
;;
;; Supported commands (case-insensitive):
;;   PING [msg]         -> +PONG or the message echoed back
;;   SET    -> +OK        (inline mode joins the rest of the line)
;;   GET           -> the value as a bulk string, or nil ($-1)
;;   DEL  [...]    -> :n removed
;;   EXISTS        -> :0 / :1
;;   INCR          -> :n (the value must be an integer; missing counts as 0)
;;   KEYS      -> array of keys ("*" lists all; anything else matches exactly)
;;   DBSIZE             -> :n
;;   QUIT               -> +OK, closes the connection (the server keeps running)
;; The store is a hash table with string keys that survives across connections.
;; Values are treated as ASCII (bulk-string lengths count characters, and a
;; value must not contain a newline). Stop the server with Ctrl-C.
;;
;; Networking goes through the portable usocket API (socket-listen /
;; socket-accept / socket-stream and the with-server-socket macro), which is a
;; shim over the built-in rontolisp:tcp-* functions, so the same source runs on
;; the interpreter, the JVM, and a WASM component.
;;
;; Run (interpreter):        java -jar $JAR examples/net/kv-server.lisp
;; Run (JVM):                java -jar $JAR examples/net/kv-server.lisp -o KvServer.class && java KvServer
;; Run (WASM component):     java -jar $JAR examples/net/kv-server.lisp -o kv-server.wasm --component && \
;;                           wasmtime run -W gc=y -W exceptions=y \
;;                             -S tcp=y -S inherit-network=y kv-server.wasm
;; Talk to it with:          redis-cli -p 6379 set greeting hello
;;                           redis-cli -p 6379 get greeting
;;                           telnet 127.0.0.1 6379   (then type: GET greeting)

;; --- small string helpers ---------------------------------------------------

;; Appends the carriage return of a RESP CRLF line ending (write-line then
;; appends the newline).
(defun crlf (s) (concatenate 'string s (format nil "~a" (code-char 13))))

;; "SET key value" -> ("SET" "key" "value")
(defun split-words (s)
  (cond ((string= s "") nil)
        (t (let ((p (position #\space s)))
             (if p
                 (cons (subseq s 0 p) (split-words (subseq s (+ p 1))))
                 (list s))))))

;; ("hello" "world") -> "hello world"
(defun join-words (ws)
  (cond ((null ws) "")
        ((null (cdr ws)) (car ws))
        (t (concatenate 'string (car ws) " " (join-words (cdr ws))))))

;; t when s is a non-empty run of decimal digits (with an optional leading -).
(defun integer-string-p (s)
  (let* ((n (length s)) (start (if (and (> n 0) (char= (char s 0) #\-)) 1 0)))
    (and (> n start)
         (do ((i start (+ i 1)))
             ((or (>= i n) (not (digit-char-p (char s i)))) (>= i n))))))

;; --- RESP replies -----------------------------------------------------------

(defun reply-simple (s sock)
  (write-line (crlf (concatenate 'string "+" s)) sock))

(defun reply-error (s sock)
  (write-line (crlf (concatenate 'string "-ERR " s)) sock))

(defun reply-int (n sock) (write-line (crlf (format nil ":~a" n)) sock))

(defun reply-bulk (s sock)
  (if s
      (progn
        (write-line (crlf (format nil "$~a" (length s))) sock)
        (write-line (crlf s) sock))
      (write-line (crlf "$-1") sock)))

(defun reply-array-header (n sock)
  (write-line (crlf (format nil "*~a" n)) sock))

;; --- request framing --------------------------------------------------------

;; Reads one RESP bulk-string element: the "$" header line, then the
;; payload line (the payload must not contain a newline).
(defun read-bulk (sock)
  (let ((header (read-line sock))) (if header (read-line sock) nil)))

(defun read-resp-array (count sock acc)
  (if (<= count 0)
      (reverse acc)
      (let ((arg (read-bulk sock)))
        (if arg (read-resp-array (- count 1) sock (cons arg acc)) nil))))

;; Reads one command as a list of argument strings: a "*" line starts a
;; RESP2 array (what redis-cli sends); anything else is an inline command
;; (what telnet/nc users type). nil at connection close.
(defun read-command (sock)
  (let ((line (read-line sock)))
    (cond ((null line) nil)
          ((string= line "") (read-command sock))
          ((char= (char line 0) #\*)
           (let ((count (subseq line 1)))
             (if (integer-string-p count)
                 (read-resp-array (parse-integer count) sock nil)
                 (list "!bad-frame"))))
          (t (split-words line)))))

;; --- commands ---------------------------------------------------------------

;; Handles one command; returns nil after QUIT (closing the session).
(defun handle-command (args store sock)
  (let ((cmd (string-upcase (car args))) (key (cadr args)))
    (cond
     ((string= cmd "PING")
      (if key (reply-bulk key sock) (reply-simple "PONG" sock))
      t)
     ((string= cmd "SET")
      (if (and key (cddr args))
          (progn
            (setf (gethash key store) (join-words (cddr args)))
            (reply-simple "OK" sock))
          (reply-error "wrong number of arguments for 'set' command" sock))
      t)
     ((string= cmd "GET")
      (if key
          (reply-bulk (gethash key store) sock)
          (reply-error "wrong number of arguments for 'get' command" sock))
      t)
     ((string= cmd "DEL")
      (let ((removed 0))
        (dolist (k (cdr args))
          (when (gethash k store)
            (remhash k store)
            (incf removed)))
        (reply-int removed sock))
      t)
     ((string= cmd "EXISTS")
      (reply-int (if (and key (gethash key store)) 1 0) sock)
      t)
     ((string= cmd "INCR")
      (let ((current (if key (or (gethash key store) "0") "0")))
        (cond ((null key)
               (reply-error "wrong number of arguments for 'incr' command"
                            sock))
              ((integer-string-p current)
               (let ((n (+ (parse-integer current) 1)))
                 (setf (gethash key store) (format nil "~a" n))
                 (reply-int n sock)))
              (t (reply-error "value is not an integer or out of range" sock))))
      t)
     ((string= cmd "KEYS")
      (let ((pattern (or key "*")) (keys nil))
        (maphash (lambda (k v)
                   (if (or (string= pattern "*") (string= pattern k))
                       (push k keys))) store)
        (reply-array-header (length keys) sock)
        (dolist (k keys) (reply-bulk k sock)))
      t)
     ((string= cmd "DBSIZE")
      (reply-int (hash-table-count store) sock)
      t)
     ((string= cmd "COMMAND")
      ;; redis-cli asks COMMAND DOCS on connect; an empty array satisfies it.
      (reply-array-header 0 sock)
      t)
     ((string= cmd "QUIT")
      (reply-simple "OK" sock)
      nil)
     (t
      (reply-error (format nil "unknown command '~a'" (car args)) sock)
      t))))

;; --- server loop ------------------------------------------------------------

(let ((store (make-hash-table)))
  (handler-case
    ;; usocket:socket-listen takes host first, then port (the reverse of
    ;; rontolisp:tcp-listen); on failure it signals usocket:socket-error
    ;; rather than returning nil.
    (let ((listener (usocket:socket-listen "127.0.0.1" 6379 :reuse-address t)))
      (write-line
       "mini-redis listening on 127.0.0.1:6379 (try: redis-cli -p 6379 ping)")
      (do ((n 1 (+ n 1)))
          (nil)
        ;; with-server-socket closes the accepted socket on every exit.
        (usocket:with-server-socket (sock (usocket:socket-accept listener))
          (let ((stream (usocket:socket-stream sock)))
            (do ((args (read-command stream) (read-command stream)))
                ((or (null args) (not (handle-command args store stream)))))))))
    (usocket:socket-error (e)
      (declare (ignore e))
      (write-line
       "socket-listen failed (is port 6379 already in use? a real redis, perhaps)"))))


---

# FILE: references/examples/net/linalg-api.lisp

;; A linear-algebra web service on rontolisp:http-handler -- the numerical
;; companion of httpbin.lisp. Two POST endpoints turn the linalg package into
;; a JSON API (rontolisp:json-parse in, rontolisp:json-stringify out):
;;
;;   POST /solve  {"a": [[2,1],[1,3]], "b": [5,10]}
;;     -> {"x": [1, 3], "det": 5}                      solves a.x = b
;;   POST /fit    {"degree": 1, "points": [[0,1],[1,2],[2,5],[3,5]]}
;;     -> {"coefficients": [1, 1.5], "fitted": [1, 2.5, 4, 5.5],
;;         "residuals": [0, -0.5, 1, -0.5],
;;         "squared-error": 1.5}                       least-squares polyfit
;;   GET  /       -> a JSON usage document
;;
;; /fit solves the normal equations (A^T A) c = A^T y over the Vandermonde
;; matrix of the xs, exactly like examples/ml/linear-regression.lisp -- but here
;; the samples arrive over HTTP. Integer inputs are solved exactly (ratios),
;; so the same request gives the same answer on every backend; ratios reach
;; the JSON as floats (json-stringify), e.g. 3/2 -> 1.5. Only the float
;; *rendering* of a ratio that is not binary-exact can differ on WASM
;; (33/10 prints as 3.3 on the interpreter/JVM but 3.299999 there).
;;
;; Invalid input (non-object body, a non-square or singular matrix, too few
;; points) answers 400 with {"error": ...}; a wrong method 405, an unknown
;; path 404. Because the service keeps no state between requests, it behaves
;; identically on all three backends -- including under wasmtime serve, where
;; each request runs in a fresh component instance.
;;
;; Run (interpreter, blocking server on :8080):
;;   java -jar $JAR examples/net/linalg-api.lisp
;; Run (JVM class; needs the rontolisp jar on the classpath):
;;   java -jar $JAR examples/net/linalg-api.lisp -o LinalgApi.class && java -cp $JAR:. LinalgApi
;; Run (WASI component under wasmtime serve):
;;   java -jar $JAR examples/net/linalg-api.lisp -o linalg-api.wasm --component && \
;;     wasmtime serve -W gc=y -W exceptions=y linalg-api.wasm
;; Talk to it with:
;;   curl -X POST -d '{"a": [[2,1],[1,3]], "b": [5,10]}' http://127.0.0.1:8080/solve
;;   curl -X POST -d '{"degree": 1, "points": [[0,1],[1,2],[2,5],[3,5]]}' http://127.0.0.1:8080/fit

;; --- JSON request/response helpers ---------------------------------------

(defun json-response (status obj)
  (list status '(:content-type "application/json")
        (list (format nil "~a~%" (rontolisp:json-stringify obj)))))

;; rontolisp:plist-hash-table (a subset of alexandria:plist-hash-table) turns a
;; keyword plist into a string-keyed hash table, which json-stringify serializes
;; as a JSON object; the keyword keys are down-cased (:det becomes "det").
(defun bad-request (message)
  (json-response 400 (rontolisp:plist-hash-table (list :error message))))

(defun method-not-allowed ()
  (json-response 405
                 (rontolisp:plist-hash-table
                  (list :error "method not allowed" :allowed "POST"))))

;; Parse the body as a JSON object into a hash table (string keys), or nil
;; when the body is not a JSON object (the cheap guard answers 400 without
;; wrapping the parse in handler-case; a malformed OBJECT body still signals).
(defun body-object (body)
  (if (and (stringp body) (> (length body) 0) (eql (char body 0) #\{))
      (rontolisp:json-parse body)
      nil))

;; JSON arrays parse to vectors; the validators and linalg below are
;; list-oriented, so deep-convert any vector (a string is a vector too, and is
;; left alone) to a list before feeding the numeric code.
(defun json-array->list (v)
  (if (and (vectorp v) (not (stringp v)))
      (let ((out nil))
        (do ((i (- (length v) 1) (- i 1)))
            ((< i 0) out)
          (setq out (cons (json-array->list (aref v i)) out))))
      v))

;; --- input validation ------------------------------------------------------

;; True when row is a list of exactly n numbers (n > 0).
(defun number-row-p (row n)
  (and (listp row) row (= (length row) n) (every (lambda (v) (numberp v)) row)))

;; True when rows is a non-empty list of equal-length number rows.
(defun matrix-spec-p (rows)
  (and (listp rows) rows (listp (first rows)) (first rows)
       (every (lambda (row) (number-row-p row (length (first rows)))) rows)))

;; --- POST /solve : solve a.x = b -------------------------------------------

(defun handle-solve (env)
  (let* ((spec (body-object (getf env :body)))
         (a (and spec (json-array->list (gethash "a" spec))))
         (b (and spec (json-array->list (gethash "b" spec)))))
    (cond ((null spec) (bad-request "the body must be a JSON object"))
     ((not (matrix-spec-p a))
      (bad-request "a must be a non-empty array of equal-length number rows"))
     ((not (= (length a) (length (first a)))) (bad-request "a must be square"))
     ((not (number-row-p b (length a)))
      (bad-request "b must be a number array as long as a"))
     (t (let* ((m (linalg:from-list a)) (det (linalg:det m)))
          (if (= det 0)
              (bad-request "a is singular")
              (json-response 200
                             (rontolisp:plist-hash-table
                              (list :x (linalg:to-list
                                        (linalg:solve m (linalg:from-list b)))
                                    :det det)))))))))

;; --- POST /fit : least-squares polynomial fitting ---------------------------

;; One row per sample x: (1 x x^2 ... x^degree).
(defun vandermonde (xs degree)
  (let* ((n (length xs)) (m (make-array (list n (+ degree 1)))))
    (do ((row 0 (+ row 1)) (rest xs (cdr rest)))
        ((>= row n) m)
      (do ((col 0 (+ col 1)))
          ((> col degree))
        (setf (aref m row col) (expt (car rest) col))))))

(defun handle-fit (env)
  (let* ((spec (body-object (getf env :body)))
         (degree (and spec (gethash "degree" spec)))
         (points (and spec (json-array->list (gethash "points" spec)))))
    (cond ((null spec) (bad-request "the body must be a JSON object"))
          ((not (and (integerp degree) (>= degree 0)))
           (bad-request "degree must be a non-negative integer"))
          ((not
            (and (listp points) points
                 (every (lambda (p) (number-row-p p 2)) points)))
           (bad-request "points must be a non-empty array of [x, y] pairs"))
          ((< (length points) (+ degree 1))
           (bad-request "need at least degree + 1 points"))
          (t (let* ((xs (mapcar (lambda (p) (first p)) points))
                    (ys (mapcar (lambda (p) (nth 1 p)) points))
                    (a (vandermonde xs degree))
                    (at (linalg:transpose a))
                    (ata (linalg:matmul at a)))
               (if (= (linalg:det ata) 0)
                   (bad-request
                    "points do not determine the polynomial (duplicate xs?)")
                   (let* ((coeffs
                           (linalg:solve ata
                                         (linalg:dot at (linalg:from-list ys))))
                          (fitted (linalg:dot a coeffs))
                          (residuals (linalg:sub (linalg:from-list ys) fitted)))
                     (json-response 200
                                    (rontolisp:plist-hash-table
                                     (list :coefficients (linalg:to-list coeffs)
                                           :fitted (linalg:to-list fitted)
                                           :residuals (linalg:to-list residuals)
                                           :squared-error
                                           (linalg:dot residuals
                                                       residuals)))))))))))

;; --- routing ----------------------------------------------------------------

(defun usage ()
  (json-response 200
   (rontolisp:plist-hash-table
    (list :service "linalg-api"
          :endpoints (list (rontolisp:plist-hash-table
                            (list :method "POST"
                             :path "/solve"
                             :body "{\"a\": [[2,1],[1,3]], \"b\": [5,10]}"))
                           (rontolisp:plist-hash-table
                            (list :method "POST"
                                  :path "/fit"
                                  :body
                                  "{\"degree\": 1, \"points\": [[0,1],[1,2],[2,5],[3,5]]}")))))))

;; The env plist's :path-info carries the (percent-decoded) path only (any
;; query string arrives separately as :query-string), so the comparisons are
;; exact; :request-method is an interned keyword, so the comparison is eq.
(defun route (env)
  (let ((path (getf env :path-info)) (method (getf env :request-method)))
    (cond ((string= path "/solve")
           (if (eq method :POST) (handle-solve env) (method-not-allowed)))
          ((string= path "/fit")
           (if (eq method :POST) (handle-fit env) (method-not-allowed)))
          ((string= path "/") (usage))
          (t (json-response 404
                            (rontolisp:plist-hash-table
                             (list :error "not found" :path path)))))))

;; The env :raw-body is an asynchronous stream on every backend; drain it once
;; here and hand the helpers an env whose :body is the whole string (getf
;; finds the prepended pair first).
(rontolisp:async-defun handle (env)
  (let ((body (rontolisp:await (rontolisp:read-all (getf env :raw-body)))))
    (route (append (list :body body) env))))

;; On the interpreter / JVM this blocks and serves on port 8080; under
;; --component the port argument is ignored (the host provides the socket).
(rontolisp:http-handler 'handle 8080)


---

# FILE: references/examples/net/magic-8-ball.lisp

;; The Magic 8 Ball -- a rontolisp reproduction of the classic Spin tutorial
;; (https://spinframework.dev/ "Building a Magic 8 Ball JSON API"), on
;; rontolisp:http-handler. Ask it a yes/no question and it answers with one
;; of the twenty canonical Magic 8 Ball replies, drawn at random, as JSON:
;;
;;   GET  /?question=Will+it+work    -> {"question": "Will it work", "answer": "..."}
;;   POST /  with body               -> the body is the question -- either raw
;;                                      text, or JSON {"question": "..."}
;;   (also served on /magic-8, the tutorial's path; other paths 404,
;;    a missing question 400)
;;
;; Run (interpreter, blocking server on :8080):
;;   java -jar $JAR examples/net/magic-8-ball.lisp
;; Run (JVM class; needs the rontolisp jar on the classpath):
;;   java -jar $JAR examples/net/magic-8-ball.lisp -o Magic8Ball.class && java -cp $JAR:. Magic8Ball
;; Run (WASI component under wasmtime serve):
;;   java -jar $JAR examples/net/magic-8-ball.lisp -o magic-8-ball.wasm --component && \
;;     wasmtime serve -W gc=y -W exceptions=y magic-8-ball.wasm
;; Talk to it with:
;;   curl 'http://127.0.0.1:8080/?question=Will+rontolisp+run+everywhere'
;;   curl -X POST -d '{"question": "Should I deploy on Friday?"}' http://127.0.0.1:8080/magic-8

;; The twenty canonical answers: ten affirmative, five non-committal,
;; five negative.
(defvar *answers*
  '("It is certain." "It is decidedly so." "Without a doubt." "Yes definitely."
    "You may rely on it." "As I see it, yes." "Most likely." "Outlook good."
    "Yes." "Signs point to yes." "Reply hazy, try again." "Ask again later."
    "Better not tell you now." "Cannot predict now."
    "Concentrate and ask again." "Don't count on it." "My reply is no."
    "My sources say no." "Outlook not so good." "Very doubtful."))

(defun json-response (status obj)
  (list status '(:content-type "application/json")
        (list (format nil "~a~%" (rontolisp:json-stringify obj)))))

;; --- pulling the question out of the request --------------------------------

;; ?question=... first (rontolisp:query-param url-decodes it, so + and %XX
;; become the spoken question); otherwise a JSON body's "question" field;
;; otherwise a non-empty raw body is the question itself.
(defun question-of (env)
  (let ((q (rontolisp:query-param (getf env :query-string) "question"))
        (body (getf env :body)))
    (cond ((and q (> (length q) 0)) q)
          ((and (stringp body) (> (length body) 0) (eql (char body 0) #\{))
           (gethash "question" (rontolisp:json-parse body)))
          ((and (stringp body) (> (length body) 0)) body)
          (t nil))))

;; --- consulting the ball -----------------------------------------------------

;; A fresh random draw per shake, like the real thing -- asking the same
;; question twice may answer differently.
(defun consult () (nth (random (length *answers*)) *answers*))

;; The env plist's :path-info carries the (percent-decoded) path only (the
;; query string arrives separately as :query-string), so the comparisons are
;; exact.
(defun route (env)
  (let ((path (getf env :path-info)))
    (if (or (string= path "/") (string= path "/magic-8"))
        (let ((question (question-of env)))
          (if question
              (json-response 200
                             (rontolisp:plist-hash-table
                              (list :question question :answer (consult))))
              (json-response 400
                             (rontolisp:plist-hash-table
                              (list :error "ask the ball a question"
                               :usage
                               "GET /?question=... or POST a question body")))))
        (json-response 404
         (rontolisp:plist-hash-table (list :error "not found" :path path))))))

;; The env :raw-body is an asynchronous stream on every backend; drain it once
;; here and hand the helpers an env whose :body is the whole string (getf
;; finds the prepended pair first).
(rontolisp:async-defun handle (env)
  (let ((body (rontolisp:await (rontolisp:read-all (getf env :raw-body)))))
    (route (append (list :body body) env))))

;; On the interpreter / JVM this blocks and serves on port 8080; under
;; --component the port argument is ignored (the host provides the socket).
(rontolisp:http-handler 'handle 8080)


---

# FILE: references/examples/wasmcloud/README.md

# wasmCloud template ports

rontolisp ports of the [wasmCloud Rust templates](https://github.com/wasmCloud/wasmCloud/tree/main/templates),
built on `rontolisp:http-handler` (see the
[Serving HTTP guide](https://making.github.io/rontolisp/guides/http-handler.html)).
Each port keeps the routes and response bodies of the original template.

| Template | Port | interpreter | JVM | wasmtime serve | wasmCloud (`wash dev`) |
|---|---|---|---|---|---|
| http-hello-world | [`http-hello-world/app.lisp`](http-hello-world/app.lisp) | yes | yes | yes | yes |
| http-handler | [`http-handler/app.lisp`](http-handler/app.lisp) | yes | yes | yes | yes |
| http-client | [`http-client/app.lisp`](http-client/app.lisp) | yes | yes | yes | yes (allowlist the upstream host) |
| http-kv-handler | [`http-kv-handler/app.lisp`](http-kv-handler/app.lisp) | yes (in-memory) | yes (in-memory) | not yet ported to `wasi:keyvalue` | not yet ported to `wasi:keyvalue` |
| service-tcp | [`service-tcp/`](https://github.com/making/rontolisp/blob/develop/examples/wasmcloud/service-tcp) | yes | yes | yes (http-api half; needs `-S cli=y -S tcp=y -S inherit-network=y`) | yes (both halves; service-leet runs as a v2 service) |
| http-api-with-distributed-workloads | not ported | - | - | - | - |

The `wasi:keyvalue` gap is not a missing capability: a served component can
import the interface, and
[`examples/wit/keyvalue/page-hits-server.lisp`](../wit/keyvalue/page-hits-server.lisp)
is a page-hit counter that does, keeping its counts on wasmCloud across
requests. This port has simply not been rewritten against it yet.

`service-tcp` runs on wasmCloud with both halves in one `wash dev`: wash
2.5.x provides `wasi:sockets` 0.3, and `service-leet.lisp` compiled with
`--component` exports `wasi:cli/run@0.3.0` -- exactly the shape of the
wasmCloud v2 service model -- so the directory's `.wash/config.yaml`
registers it as `dev.service_file` and the http-api component reaches it
over the workload's in-process virtual loopback. That virtual loopback is
also the one catch: inside a wasmCloud component, 127.0.0.1 never means the
machine's loopback, so a leet service running as a host process is
unreachable from there (non-loopback addresses connect over the real
network as usual). Under `wasmtime serve` the http-api half instead talks
to a real-loopback service-leet and needs `-S cli=y -S tcp=y
-S inherit-network=y` on top of the usual flags (see its header).

## Running

Every `app.lisp` carries its exact run commands in its header comment; the
pattern is the same for all of them:

```bash
# interpreter (blocking server on :8080)
rontolisp examples/wasmcloud/http-hello-world/app.lisp

# JVM class (running it needs the rontolisp jar on the classpath)
rontolisp examples/wasmcloud/http-hello-world/app.lisp -o App.class
java -cp rontolisp-0.1.0-SNAPSHOT-exec.jar:. App

# WASI HTTP component under wasmtime serve (wasmtime 47+ for concurrent throughput)
rontolisp examples/wasmcloud/http-hello-world/app.lisp -o app.wasm --component
wasmtime serve -W gc=y -W exceptions=y app.wasm
```

Each component directory keeps a `.wash/config.yaml` whose `build.command`
compiles the example with the `rontolisp` binary (expected on the `PATH`;
`service-tcp` chains both halves and registers the service via
`dev.service_file`) and whose `dev.wasm_proposals` lists the proposals the
component needs.
`http-client` additionally sets `workload.allowedHosts` -- wash denies all
outgoing HTTP unless the upstream host is allowlisted.

## wasmCloud status (checked 2026-07-16): WORKS

Since the callback-async cutover the serve component uses only **base
`component-model-async`**: the handle export is a callback async lift, and every
stream/future body operation is the asynchronous (non-blocking) built-in variant
with a blocking `waitable-set.wait` park -- no synchronous built-ins, no stackful
lift, so none of the gated wasmtime features. **Released `wash` (2.5.2) hosts the
components** with `dev.wasm_proposals: [gc, exception-handling,
component-model-async]` (each template's `.wash/config.yaml` sets them): `wash dev`
in `http-handler/` serves and answers
`curl http://127.0.0.1:8000/` with "Hello from wasmCloud!".

(Historical note: the pre-cutover 0.3 components were rejected at parse time with
"synchronous stream.write requires the component model more async builtins
feature" -- wash has no switch for that wasmtime feature. The cutover removed the
dependency instead.)

## service-tcp

The original template demonstrates the wasmCloud v2 service model: a
long-running TCP service plus a stateless HTTP component in one host. The
port runs the same two programs either as two host processes
(interpreter/JVM) or inside one wasmCloud host:

```bash
# two host processes (interpreter; JVM per the file headers)
rontolisp examples/wasmcloud/service-tcp/service-leet.lisp &   # TCP :7777
rontolisp examples/wasmcloud/service-tcp/http-api.lisp &       # HTTP :8080
curl -X POST -d '{"payload":"Hello World"}' http://127.0.0.1:8080/task
# H3110 W0r1d

# one wasmCloud host (builds both halves, runs service-leet as a service)
cd examples/wasmcloud/service-tcp && wash dev                  # HTTP :8000
curl -X POST -d '{"payload":"Hello World"}' http://127.0.0.1:8000/task
# H3110 W0r1d
```


---

# FILE: references/examples/wasmcloud/http-client/app.lisp

;; A rontolisp port of wasmCloud's http-client template
;; (templates/http-client in the wasmCloud repo): a served handler that makes
;; an outgoing HTTP request, i.e. rontolisp:fetch inside rontolisp:http-handler.
;; Every request is answered by proxying https://httpbin.ik.am/get -- the
;; upstream status and body are forwarded as-is, and a failed upstream
;; request maps to 502.
;;
;; Run (interpreter, blocking server on :8080):
;;   rontolisp examples/wasmcloud/http-client/app.lisp
;; Run (JVM class; running it needs the rontolisp jar on the classpath):
;;   rontolisp examples/wasmcloud/http-client/app.lisp -o App.class && \
;;     java -cp rontolisp-0.1.0-SNAPSHOT-exec.jar:. App
;; Run (WASI component under wasmtime serve; the wasi:http/client import that
;; carries the outbound fetch is host-provided by default):
;;   rontolisp examples/wasmcloud/http-client/app.lisp -o app.wasm --component && \
;;     wasmtime serve -W gc=y -W exceptions=y app.wasm
;; wasmCloud hosts it too: `wash dev` in this directory -- see ../README.md.
;; Talk to it with:
;;   curl http://127.0.0.1:8080/

;; The upstream to proxy, as in the original template.
(defun upstream-url () "https://httpbin.ik.am/get")

;; A failed fetch surfaces as a nil/non-integer :status, mapped to 502 --
;; anything else (including upstream 4xx/5xx) is forwarded unchanged.
(rontolisp:async-defun handle (env)
  ;; awaiting needs an async-defun; the fetch response :body is an
  ;; asynchronous stream on every backend, drained with read-all.
  (let* ((res (rontolisp:await (rontolisp:fetch (upstream-url))))
         (status (getf res :status))
         (body (rontolisp:await (rontolisp:read-all (getf res :body)))))
    (if (integerp status)
        (list status '(:content-type "application/json") (list body))
        (list 502 '(:content-type "text/plain")
              (list (format nil "upstream request failed~%"))))))

;; On the interpreter / JVM this blocks and serves on port 8080; under
;; --component the port argument is ignored (the host provides the socket).
(rontolisp:http-handler 'handle 8080)


---

# FILE: references/examples/wasmcloud/http-handler/app.lisp

;; A rontolisp port of wasmCloud's http-handler template
;; (templates/http-handler in the wasmCloud repo): an HTTP handler with
;; routing -- the axum Router of the original becomes a plain cond over the
;; request path.
;;
;;   GET  /                      -> "Hello from wasmCloud!"
;;   GET  /api/greet?name= -> "Hello, !" ("world" when omitted)
;;   POST /api/echo              -> echoes the JSON body {"message": "..."}
;;   wrong method                -> 405 "Method Not Allowed"
;;   unknown path                -> 404 "Not found"
;;
;; Run (interpreter, blocking server on :8080):
;;   rontolisp examples/wasmcloud/http-handler/app.lisp
;; Run (JVM class; running it needs the rontolisp jar on the classpath):
;;   rontolisp examples/wasmcloud/http-handler/app.lisp -o App.class && \
;;     java -cp rontolisp-0.1.0-SNAPSHOT-exec.jar:. App
;; Run (WASI component under wasmtime serve):
;;   rontolisp examples/wasmcloud/http-handler/app.lisp -o app.wasm --component && \
;;     wasmtime serve -W gc=y -W exceptions=y app.wasm
;; wasmCloud hosts it too: `wash dev` in this directory -- see ../README.md.
;; Talk to it with:
;;   curl http://127.0.0.1:8080/
;;   curl 'http://127.0.0.1:8080/api/greet?name=rontolisp'
;;   curl -X POST -d '{"message":"hi"}' http://127.0.0.1:8080/api/echo

;; --- request helpers --------------------------------------------------------

;; t when the body looks like a JSON object (json-parse signals on garbage;
;; the cheap guard answers 400 without wrapping the parse in handler-case).
(defun json-object-p (body)
  (and (stringp body) (> (length body) 0) (eql (char body 0) #\{)))

;; --- responses ---------------------------------------------------------------

(defun text-response (status body)
  (list status '(:content-type "text/plain") (list body)))

(defun json-response (status obj)
  (list status '(:content-type "application/json")
        (list (format nil "~a~%" (rontolisp:json-stringify obj)))))

;; --- handlers ----------------------------------------------------------------

(defun hello (env) (text-response 200 (format nil "Hello from wasmCloud!~%")))

;; The raw query string arrives as :query-string; rontolisp:query-param
;; url-decodes the value.
(defun greet (env)
  (let ((name (rontolisp:query-param (getf env :query-string) "name")))
    (text-response 200 (format nil "Hello, ~a!~%" (if name name "world")))))

(rontolisp:async-defun echo (env)
  ;; The env :raw-body is an asynchronous stream on every backend; drain it
  ;; first. The router stays a plain defun: the served dispatch awaits the
  ;; handler's result, and await flattens the future echo returns.
  (let ((body (rontolisp:await (rontolisp:read-all (getf env :raw-body)))))
    (if (json-object-p body)
        (let ((message (gethash "message" (rontolisp:json-parse body))))
          (if (stringp message)
              (json-response 200
               (rontolisp:plist-hash-table (list :message message)))
              (json-response 400
               (rontolisp:plist-hash-table
                (list :error
                      "expected a JSON object with a string message field")))))
        (json-response 400
         (rontolisp:plist-hash-table
          (list
           :error "expected a JSON object with a string message field"))))))

(defun not-found (env) (text-response 404 (format nil "Not found~%")))

(defun method-not-allowed (env)
  (text-response 405 (format nil "Method Not Allowed~%")))

;; --- routing -----------------------------------------------------------------

;; Dispatch on (path, method), answering 405 when the path exists but the
;; method does not match -- the same behavior as the axum Router. The env
;; plist's :path-info carries the path only, so the comparisons are exact;
;; :request-method is an interned keyword, so the comparisons are eq.
(defun handle (env)
  (let ((path (getf env :path-info)) (method (getf env :request-method)))
    (cond ((string= path "/")
           (if (eq method :GET) (hello env) (method-not-allowed env)))
          ((string= path "/api/greet")
           (if (eq method :GET) (greet env) (method-not-allowed env)))
          ((string= path "/api/echo")
           (if (eq method :POST) (echo env) (method-not-allowed env)))
          (t (not-found env)))))

;; On the interpreter / JVM this blocks and serves on port 8080; under
;; --component the port argument is ignored (the host provides the socket).
(rontolisp:http-handler 'handle 8080)


---

# FILE: references/examples/wasmcloud/http-hello-world/app.lisp

;; A rontolisp port of wasmCloud's http-hello-world template
;; (templates/http-hello-world in the wasmCloud repo): the minimal HTTP
;; handler component.
;;
;;   GET /    -> "Hello from wasmCloud!"
;;   others   -> 404 "Not found"
;;
;; Run (interpreter, blocking server on :8080):
;;   rontolisp examples/wasmcloud/http-hello-world/app.lisp
;; Run (JVM class; running it needs the rontolisp jar on the classpath):
;;   rontolisp examples/wasmcloud/http-hello-world/app.lisp -o App.class && \
;;     java -cp rontolisp-0.1.0-SNAPSHOT-exec.jar:. App
;; Run (WASI component under wasmtime serve):
;;   rontolisp examples/wasmcloud/http-hello-world/app.lisp -o app.wasm --component && \
;;     wasmtime serve -W gc=y -W exceptions=y app.wasm
;; wasmCloud hosts it too: `wash dev` in this directory -- see ../README.md.
;; Talk to it with:
;;   curl http://127.0.0.1:8080/

(defun text-response (status body)
  (list status '(:content-type "text/plain") (list body)))

(defun home (env) (text-response 200 (format nil "Hello from wasmCloud!~%")))

(defun not-found (env) (text-response 404 (format nil "Not found~%")))

;; The env plist's :path-info carries the (percent-decoded) path only (any
;; query string arrives separately as :query-string), so the comparison is
;; exact.
(defun handle (env)
  (if (string= (getf env :path-info) "/") (home env) (not-found env)))

;; On the interpreter / JVM this blocks and serves on port 8080; under
;; --component the port argument is ignored (the host provides the socket).
(rontolisp:http-handler 'handle 8080)


---

# FILE: references/examples/wasmcloud/http-kv-handler/app.lisp

;; A rontolisp port of wasmCloud's http-kv-handler template
;; (templates/http-kv-handler in the wasmCloud repo): an HTTP handler backed
;; by a key-value store. The original stores through wasi:keyvalue with a
;; pluggable backend; this port has not been rewritten against the real
;; wasi:keyvalue interface yet (examples/wit/keyvalue shows how a component
;; imports it), so it implements the template's default "in_memory" backend as
;; a global hash table.
;;
;;   POST /            {"key":"...","value":"..."}  -> stores the pair
;;   GET  /?key=                               -> the stored value, or 404
;;   other methods                                  -> 405 "Method Not Allowed"
;;
;; Interpreter / JVM only: both keep the process (and so the hash table)
;; alive across requests. It compiles under --component too, but WASI HTTP
;; hosts either instantiate the component per request (wasmtime serve) or
;; reset the heap between requests (jco, wasmCloud), so the store would be
;; empty on every request -- a real WASM port needs wasi:keyvalue.
;;
;; Run (interpreter, blocking server on :8080):
;;   rontolisp examples/wasmcloud/http-kv-handler/app.lisp
;; Run (JVM class; running it needs the rontolisp jar on the classpath):
;;   rontolisp examples/wasmcloud/http-kv-handler/app.lisp -o App.class && \
;;     java -cp rontolisp-0.1.0-SNAPSHOT-exec.jar:. App
;; Talk to it with:
;;   curl -X POST -d '{"key":"greeting","value":"hello"}' http://127.0.0.1:8080/
;;   curl 'http://127.0.0.1:8080/?key=greeting'

;; The in-memory store; string keys work as hash keys on every backend.
(defvar *store* (make-hash-table))

;; --- request helpers --------------------------------------------------------

;; t when the body looks like a JSON object (json-parse signals on garbage;
;; the cheap guard answers 400 without wrapping the parse in handler-case).
(defun json-object-p (body)
  (and (stringp body) (> (length body) 0) (eql (char body 0) #\{)))

(defun text-response (status body)
  (list status '(:content-type "text/plain") (list body)))

;; --- handlers ----------------------------------------------------------------

;; POST / with {"key":"...","value":"..."} stores the pair.
(defun handle-post (env)
  (let ((body (getf env :body)))
    (if (json-object-p body)
        (let* ((payload (rontolisp:json-parse body))
               (key (gethash "key" payload))
               (value (gethash "value" payload)))
          (if (and (stringp key) (stringp value))
              (progn
                (setf (gethash key *store*) value)
                (text-response 200
                 (format nil "[in_memory] Stored key '~a'~%" key)))
              (text-response 400
               (format nil
                "Invalid JSON (expected key and value string fields)~%"))))
        (text-response 400
         (format nil
                 "Invalid JSON (expected key and value string fields)~%")))))

;; GET /?key= answers the stored value, or 404 when the key is unknown.
;; The raw query string arrives as :query-string; rontolisp:query-param
;; url-decodes the value.
(defun handle-get (env)
  (let ((key (rontolisp:query-param (getf env :query-string) "key")))
    (if key
        (let ((value (gethash key *store*)))
          (if value
              (text-response 200 (format nil "[in_memory] ~a~%" value))
              (text-response 404
               (format nil "[in_memory] Key '~a' not found~%" key))))
        (text-response 400
         (format nil "Missing required query parameter: key~%")))))

;; :request-method is an interned keyword, so the comparisons are eq.
(defun route (env)
  (let ((method (getf env :request-method)))
    (cond ((eq method :POST) (handle-post env))
          ((eq method :GET) (handle-get env))
          (t (text-response 405 (format nil "Method Not Allowed~%"))))))

;; The env :raw-body is an asynchronous stream on every backend; drain it once
;; here and hand the helpers an env whose :body is the whole string (getf
;; finds the prepended pair first).
(rontolisp:async-defun handle (env)
  (let ((body (rontolisp:await (rontolisp:read-all (getf env :raw-body)))))
    (route (append (list :body body) env))))

;; Blocks and serves on port 8080.
(rontolisp:http-handler 'handle 8080)


---

# FILE: references/examples/wasmcloud/service-tcp/http-api.lisp

;; A rontolisp port of the component half of wasmCloud's service-tcp template
;; (templates/service-tcp/http-api in the wasmCloud repo): an HTTP API that
;; forwards text to the leet TCP service (service-leet.lisp, listening on
;; 127.0.0.1:7777) and returns the transformed result.
;;
;;   GET  /      -> a short usage text (the original serves an HTML UI)
;;   POST /task  {"payload":"..."}  -> the payload in leet speak
;;   unknown path -> 404 "Not found"
;;
;; Works everywhere, wasmCloud included. Under wasmtime serve note the extra
;; -S cli=y, without which the serve linker reports the tcp-socket resource
;; as missing. Under `wash dev` 127.0.0.1 names the workload's in-process
;; virtual loopback, not the machine's -- so the leet service must run
;; INSIDE wasmCloud too (service-leet.lisp deployed as a v2 service; this
;; directory's .wash/config.yaml builds and registers both halves), while a
;; leet service running as a host process is unreachable from the component.
;; Non-loopback addresses connect over the real network.
;;
;; Run (start service-leet.lisp first; then, interpreter):
;;   rontolisp examples/wasmcloud/service-tcp/http-api.lisp
;; Run (JVM class; running it needs the rontolisp jar on the classpath):
;;   rontolisp examples/wasmcloud/service-tcp/http-api.lisp -o HttpApi.class && \
;;     java -cp rontolisp-0.1.0-SNAPSHOT-exec.jar:. HttpApi
;; Run (WASI component under wasmtime serve):
;;   rontolisp examples/wasmcloud/service-tcp/http-api.lisp -o http-api.wasm --component && \
;;     wasmtime serve -W gc=y -W exceptions=y -S cli=y -S tcp=y -S inherit-network=y http-api.wasm
;; Run (wasmCloud, both halves in one host; serves on :8000):
;;   cd examples/wasmcloud/service-tcp && wash dev
;; Talk to it with:
;;   curl -X POST -d '{"payload":"Hello World"}' http://127.0.0.1:8080/task
;;   -> H3110 W0r1d

;; t when the body looks like a JSON object (json-parse signals on garbage;
;; the cheap guard answers 400 without wrapping the parse in handler-case).
(defun json-object-p (body)
  (and (stringp body) (> (length body) 0) (eql (char body 0) #\{)))

(defun text-response (status body)
  (list status '(:content-type "text/plain") (list body)))

;; One round trip to the leet service: send the payload as a line, read the
;; transformed line back. A refused connection signals an error on the
;; interpreter/JVM but yields a nil sock on the WASM backend, so the nil
;; guard answers 502 there instead of trapping on (read-line nil); a nil
;; reply (service closed early) maps to 502 as well.
(defun leet-request (payload)
  (let ((sock (rontolisp:tcp-connect "127.0.0.1" 7777)))
    (if sock
        (progn
          (write-line payload sock)
          (let ((reply (read-line sock)))
            (close sock)
            reply))
        nil)))

(defun handle-task (env)
  (let ((body (getf env :body)))
    (if (json-object-p body)
        (let ((payload (gethash "payload" (rontolisp:json-parse body))))
          (if (stringp payload)
              (let ((reply (leet-request payload)))
                (if reply
                    (text-response 200 (format nil "~a~%" reply))
                    (text-response 502
                                   (format nil "leet service unavailable~%"))))
              (text-response 400
               (format nil
                "expected a JSON object with a string payload field~%"))))
        (text-response 400
         (format nil "expected a JSON object with a string payload field~%")))))

(defun home (env)
  (text-response 200
   (format nil
    "POST /task with {\"payload\":\"...\"} to get it back in leet speak~%")))

;; The env plist's :path-info carries the (percent-decoded) path only (any
;; query string arrives separately as :query-string), so the comparisons are
;; exact; :request-method is an interned keyword, so the comparison is eq.
(defun route (env)
  (let ((path (getf env :path-info)))
    (cond ((string= path "/") (home env))
          ((string= path "/task")
           (if (eq (getf env :request-method) :POST)
               (handle-task env)
               (text-response 405 (format nil "Method Not Allowed~%"))))
          (t (text-response 404 (format nil "Not found~%"))))))

;; The env :raw-body is an asynchronous stream on every backend; drain it once
;; here and hand the helpers an env whose :body is the whole string (getf
;; finds the prepended pair first).
(rontolisp:async-defun handle (env)
  (let ((body (rontolisp:await (rontolisp:read-all (getf env :raw-body)))))
    (route (append (list :body body) env))))

;; Blocks and serves on port 8080.
(rontolisp:http-handler 'handle 8080)


---

# FILE: references/examples/wasmcloud/service-tcp/service-leet.lisp

;; A rontolisp port of the service half of wasmCloud's service-tcp template
;; (templates/service-tcp/service-leet in the wasmCloud repo): a long-running
;; TCP server that accepts connections on 127.0.0.1:7777, reads lines and
;; replies with the leet-speak transformation of each line.
;;
;; Runs on the interpreter, the JVM, as a WASI component under wasmtime run,
;; and on wasmCloud as a v2 SERVICE: compiled with --component the program
;; exports wasi:cli/run@0.3.0, which is exactly the shape wash's service
;; model hosts (this directory's .wash/config.yaml registers the build as
;; dev.service_file, so one `wash dev` runs both halves). On wasmCloud the
;; listener binds the workload's in-process virtual loopback, reachable only
;; by components in the same workload -- and the listen host must be the
;; explicit "127.0.0.1" below: wash's bind policy rejects the 0.0.0.0
;; default before its loopback rewrite.
;;
;; Connections are served one at a time (accept returns to the loop when the
;; client closes), which is enough for the per-request open/close pattern of
;; http-api.lisp. Stop it with Ctrl-C.
;;
;; Run (interpreter):
;;   rontolisp examples/wasmcloud/service-tcp/service-leet.lisp
;; Run (JVM):
;;   rontolisp examples/wasmcloud/service-tcp/service-leet.lisp -o ServiceLeet.class && \
;;     java ServiceLeet
;; Run (WASI component under wasmtime run):
;;   rontolisp examples/wasmcloud/service-tcp/service-leet.lisp -o service-leet.wasm --component && \
;;     wasmtime run -W gc=y -W exceptions=y -S tcp=y -S inherit-network=y service-leet.wasm
;; Run (wasmCloud, both halves in one host):
;;   cd examples/wasmcloud/service-tcp && wash dev
;; Talk to it with:
;;   nc 127.0.0.1 7777      (type a line, read it back in leet speak)

;; The same mapping as the original: match on the downcased character, but a
;; character that maps to itself keeps its original case ("Hello World" ->
;; "H3110 W0r1d").
(defun leet-char (c)
  (let ((lc (char-downcase c)))
    (cond ((char= lc #\a) #\4)
          ((char= lc #\e) #\3)
          ((char= lc #\i) #\1)
          ((char= lc #\o) #\0)
          ((char= lc #\s) #\5)
          ((char= lc #\t) #\7)
          ((char= lc #\l) #\1)
          (t c))))

(defun to-leet-speak (s) (map 'string #'leet-char s))

;; The explicit loopback host matters on wasmCloud (wash rejects a 0.0.0.0
;; bind from a service); everywhere else it just narrows the listener.
(let ((listener (rontolisp:tcp-listen 7777 "127.0.0.1")))
  (if listener
      (progn
        (write-line "leet service listening on 127.0.0.1:7777")
        (do ((n 1 (+ n 1)))
            (nil)
          (let ((sock (rontolisp:tcp-accept listener)))
            (write-line (format nil "client ~a connected" n))
            (do ((line (read-line sock) (read-line sock)))
                ((null line)
                 (close sock)
                 (write-line "client disconnected"))
              (write-line (to-leet-speak line) sock)))))
      (write-line "tcp-listen failed (is port 7777 already in use?)")))


---

# FILE: references/examples/wit/keyvalue/README.md

# wit/keyvalue -- one WIT interface, three stores behind it

[`wit/world/`](https://github.com/making/rontolisp/blob/develop/examples/wit/world) implements a WIT world: the functions a program
*exports*. This is the other half -- the functions a program **calls**.
[`page-hits.lisp`](page-hits.lisp) is a page-view counter written against
[`wasi:keyvalue/store`](wit/keyvalue.wit) that never says where the pairs live:

```lisp
(rontolisp:wit-import "wit/keyvalue.wit"
                      :interface "wasi:keyvalue/store@0.2.0-draft"
                      :package kv)

(defun record-hit (bucket page)
  (let ((seen (kv:bucket-get bucket page)))
    (kv:bucket-set bucket page
                   (princ-to-string (+ 1 (if seen (parse-integer seen) 0))))))
```

`kv:bucket-get` and `kv:bucket-set` are ordinary Lisp functions the compiler
wrote from the `.wit`. What they *reach* is decided elsewhere, by whoever binds a
**provider**:

| File | What it is |
| --- | --- |
| [`wit/keyvalue.wit`](wit/keyvalue.wit) | The interface: the real [wasi:keyvalue](https://github.com/WebAssembly/wasi-keyvalue) `store`, vendored verbatim |
| [`page-hits.lisp`](page-hits.lisp) | The program. It knows the WIT and nothing else |
| [`memory-store.lisp`](memory-store.lisp) | An implementation: a portable Lisp hash-table store, ~50 lines, ending in one `rontolisp:wit-provide` |
| [`java-store.lisp`](java-store.lisp) | The **same** interface over a real `java.util.LinkedHashMap`. Bound after the first, so it replaces it |
| [`page-hits-server.lisp`](page-hits-server.lisp) | The same counter as an HTTP server ([§6](#6-serve-it)) |

Run it on the interpreter and the Lisp store answers; compile to the JVM and the
Java store answers; compile to a **WASI component** and wasmtime's own
`wasi:keyvalue` implementation answers -- a host that has never heard of this
program. **The output is identical all three ways**, and that identity is the
point.

Commands below say `rontolisp`, the native binary; with the executable JAR it is
`java -jar ../../target/rontolisp-0.1.0-SNAPSHOT-exec.jar`. Run them from this
directory.

## 1. The interface

[`wit/keyvalue.wit`](wit/keyvalue.wit) is upstream's `store`, name for name --
not a subset, which is what lets the component below talk to a real host with no
adapter in between. It declares a `variant error`, a `record key-response`,
`open: func(identifier: string) -> result` and a
`resource bucket` with `get`/`set`/`delete`/`exists`/`list-keys`.

`:package kv` puts the bindings in a package of their own. The names are
mechanical:

| WIT | Lisp |
| --- | --- |
| `open: func(identifier: string)` | `(kv:open identifier)` |
| `bucket.get`, a **resource method** | `(kv:bucket-get b key)` -- the handle comes first |
| a resource **constructor** | `bucket-new` |
| a resource **static** func `f` | `bucket-f` |

A method is prefixed with its resource so the Lisp-2 function namespace stays
unambiguous when two resources declare the same method; the receiver, implicit
in WIT, becomes the leading argument. Parameter names come across verbatim.

The types are rontolisp's settled WIT mapping:

| WIT | Lisp value |
| --- | --- |
| `result` | the bucket handle -- and the **error arm signals** `rontolisp:wit-error` |
| `option>` | the value string, or `nil` when the key is absent |
| `list` | a string (bytes, one per character) |
| `option` | a number, or `nil` |
| `record key-response` | a keyword plist: `(:keys ("/index" ...) :cursor nil)` |
| a `resource` handle | an opaque integer -- pass it back, never interpret it |

There is no `unwrap` step and no error-code plumbing: the ok arm *is* the return
value, and a failure is a condition.

```lisp
(handler-case (kv:open "not-a-store-anyone-has")
  (rontolisp:wit-error (e)
    (format t "bad store: ~a~%" (rontolisp:wit-error-payload e))))
```

## 2. The store is user code -- rontolisp ships none

rontolisp knows how to **bind** a provider to a WIT interface. It does not know
what `wasi:keyvalue` is and ships no store for it, nor for any other interface:
a new host interface should cost a `.wit` file, not a change to the language.

A provider is an **ordinary Lisp callable**. It takes the bound function's Lisp
member name -- `"open"`, `"bucket-get"`, ... -- and then that function's
arguments, a resource method's handle included. That is the whole contract:

```lisp
;;; memory-store.lisp
(defun memory-store (member &rest args)
  (cond ((string= member "open")       (kv-open (nth 0 args)))
        ((string= member "bucket-get") (gethash (nth 1 args) (kv-bucket (nth 0 args))))
        ...))

(rontolisp:wit-provide "wasi:keyvalue/store@0.2.0-draft" #'memory-store)
```

Nothing is wrapped on the way out: the ok arm of a `result` **is** the return
value, and the error arm is the provider signaling `rontolisp:wit-error` with
the WIT variant as its payload. `page-hits.lisp` pulls the file in with
`(require :kv-memory "memory-store.lisp")`.

## 3. Run it on the interpreter

```console
$ rontolisp page-hits.lisp

hits per page:
  /docs = 1
  /index = 3
  /pricing = 2

/docs exists?      yes
/docs exists now?  no
keys:              ("/index" "/pricing")
/nope:             NIL
bad store:         NO-SUCH-STORE
seeded:            ("/a" "/b")
```

A `wasi:keyvalue` program normally needs a host before it can run at all. Here
it is a script.

## 4. On the JVM: swap the store, not the program

`java-store.lisp` is the same dispatch function over a real
`java.util.LinkedHashMap`, reached through
[`java:` interop](https://github.com/making/rontolisp/blob/develop/examples/wit/doc/en/guides/java-interop.md):

```lisp
(defun java-store (member &rest args)
  (cond ((string= member "bucket-get")
         (java:call (java-bucket (nth 0 args)) "get" (nth 1 args)))
        ...))

(rontolisp:wit-provide "wasi:keyvalue/store@0.2.0-draft" #'java-store)
```

`wit-provide` **replaces** whatever was bound, so requiring this file after
`memory-store.lisp` is the whole swap. `java:` interop is a JVM-and-interpreter
thing, which is why `page-hits.lisp` guards that one line with
`#+rontolisp-jvm` -- the only line in the program about a backend rather than
about page hits.

```console
$ rontolisp page-hits.lisp -o PageHits.class && java -cp . PageHits
;; [java store] open "" -> handle 500
;; [java store] set /index = 1
...
```

The Java store traces every call it serves. Strip those lines and the two runs
are the same report, character for character.

## 5. As a component -- against a store nobody here wrote

`--component` makes the program a WASI component that **imports
`wasi:keyvalue/store`**. There is no provider in it at all: the calls go out
through the canonical ABI, and whatever the host plugs in answers.

```console
$ rontolisp page-hits.lisp -o page-hits.wasm --component
$ wasmtime run -W gc=y -W exceptions=y -S keyvalue=y page-hits.wasm
```

Character for character the interpreter's report -- from a store written in
Rust, inside the runtime, by people who have never seen this program. The `.wit`
is the only thing the two ends share. Nothing in `page-hits.lisp` mentions a
component and neither store file is consulted: on the WASM backends a
`wit-provide` is *inert*, so one source compiles everywhere.

What crossed that boundary is the whole type mapping at once: a `resource`
handle, a `string`, a `list`, an `option`, a `bool`, a `record` holding a
`list` and an `option`, and a `result` whose **error arm arrived as
a condition** and was caught by `handler-case`.

`--emit-wit` writes the component's real type next to it, with the imported
interface pruned to the functions the program actually calls. That is also how
you compose: a component importing `wasi:keyvalue/store` plugs into any
component exporting it, in any language, with
[`wac`](https://github.com/bytecodealliance/wac) -- the host need not be a
runtime built-in.

Each WIT function lowers into an **ordinary `defun`**, not a special call form,
so `#'kv:bucket-set` is a function value that `funcall`, `apply` and `mapcar`
take. That holds on the component too, where those calls are canonical-ABI
lowerings.

## 6. Serve it

[`page-hits-server.lisp`](page-hits-server.lisp) is the same counter behind
`rontolisp:http-handler`. The two halves need each other: a served component's
globals are **not** state, because a `wasi:http` host instantiates it afresh for
every request, so a hash-table counter reads back empty every time. Through a
store it works -- the store is the one thing that outlives the instance.

```lisp
(defun handle (env)
  (let* ((page (getf env :path-info))
         (bucket (kv:open *store*))
         (hits (record-hit bucket page)))
    (list 200 '(:content-type "text/plain")
          (list (format nil "~a -> ~a hit~:[s~;~]~%~%hits per page:~%~a"
                        page hits (= hits 1) (report bucket))))))

(rontolisp:http-handler 'handle 8080)
```

On the interpreter and the JVM the process outlives the requests, so the Lisp
store keeps the counts. As a component it exports `wasi:http/incoming-handler`
and imports `wasi:keyvalue/store`, and whose store answers is the host's
business:

```console
$ rontolisp page-hits-server.lisp -o server.wasm --component
$ wasmtime serve -W gc=y -W exceptions=y -S keyvalue=y server.wasm
```

wasmtime's key-value host is an **in-memory store it rebuilds per instance**, so
under `wasmtime serve` the tally starts over every request. The calls really do
cross into it -- seed one with `-S keyvalue-in-memory-data=/index=41` and the
first request answers 42 -- but the counts do not accumulate, and nothing in the
component can change that.

A host that links an **out-of-process** provider keeps them. wasmCloud does, and
[`.wash/config.yaml`](https://github.com/making/rontolisp/blob/develop/examples/wit/keyvalue/.wash/config.yaml) is the whole configuration: `wash dev`
compiles this directory, deploys the component and links it, and the counts
accumulate on `:8000`. That the component cannot tell the two hosts apart is the
point of the boundary.

## Limitations

Two WASM backends this example does not reach, both worth knowing before writing
a `.wit` of your own:

1. **Preview 1 (`-o out.wasm`) carries the flat set only.** A Preview 1 import
   is a bare core-WASM function with no component type to describe a richer
   shape, so only the integer scalars up to 32 bits, the float scalars, `bool`,
   `string`, `list` and resource handles cross. Every function of
   `wasi:keyvalue` returns a `result`, so this WIT is a compile error there,
   naming the line that cannot cross.

   `wit-import` *does* work on Preview 1 for an interface written within that
   set -- a WebGL binding, say, which is all handles and scalars. It lowers into
   one `rontolisp:wasm-import` per WIT function, byte-identically to the
   hand-written import block, and `--optimize` still shakes out what the program
   never calls. (A WIT label becomes a `createShader`-style field by default --
   the JavaScript convention, and what `jco` produces; `:field-style :kebab`
   keeps the label verbatim.)
2. **`--no-gc` rejects `wit-import` outright**: its contract is a plain MVP
   module that imports nothing at all.

On the interpreter and the JVM there is no type restriction: the boundary is an
ordinary Lisp call, so every representation in the mapping crosses (records as
plists, variants as tagged lists, enums as keywords). Only `stream` and `future`
are refused, having no rontolisp value on any backend yet.

Full reference:
[`rontolisp:wit-import`](https://github.com/making/rontolisp/blob/develop/examples/wit/doc/en/reference/functions/rontolisp-wit-import.md),
[`rontolisp:wit-provide`](https://github.com/making/rontolisp/blob/develop/examples/wit/doc/en/reference/functions/rontolisp-wit-provide.md).


---

# FILE: references/examples/wit/keyvalue/java-store.lisp

;;;; java-store.lisp -- the SAME wasi:keyvalue/store interface, backed by a real
;;;; Java map instead of a Lisp hash table.
;;;;
;;;; This is the point of the whole exercise: page-hits.lisp does not change, and
;;;; does not know this file exists. It calls (kv:bucket-set b "/index" "3"); what
;;;; that lands in is decided here, by one rontolisp:wit-provide.
;;;;
;;;; java: interop is reflection, so this file runs on the JVM (and the JVM-hosted
;;;; interpreter) only -- which is exactly why page-hits.lisp requires it under
;;;; #+rontolisp-jvm. A real deployment would swap java.util.LinkedHashMap for a
;;;; Redis client or a JDBC connection; nothing else in this file would change,
;;;; and nothing at all in the program would.
;;;;
;;;; Use it with:  (require :kv-java "java-store.lisp")

(provide :kv-java)

;;; The DATA lives under the store's identifier; a handle is only a reference TO a
;;; store, so dropping one does not take the store with it (memory-store.lisp makes
;;; the same distinction, and for the same reason).
(defvar *java-stores* (java:new "java.util.HashMap"))

(defvar *java-handles* (java:new "java.util.HashMap"))

(defvar *java-next-handle* 500)

;;; The identifiers this store recognizes -- the same rule the real host follows:
;;; the default store under the empty identifier, no-such-store for anything else.
(defvar *java-identifiers* '(""))

(defun java-bucket (handle)
  ;; An unknown handle -- never opened, or already dropped.
  (let ((identifier (java:call *java-handles* "get" handle)))
    (if (null identifier)
        (error 'rontolisp:wit-error
               :payload :no-such-store
               :message "java store: not an open bucket handle")
        (java:call *java-stores* "get" identifier))))

(defun java-open (identifier)
  (if (not (member identifier *java-identifiers* :test #'string=))
      (error 'rontolisp:wit-error
       :payload :no-such-store
       :message (concatenate 'string "java store: no such store " identifier))
      ;; A fresh handle per open, onto the one store the identifier names.
      (let ((handle *java-next-handle*))
        (setq *java-next-handle* (+ handle 1))
        (if (null (java:call *java-stores* "get" identifier))
            (java:call *java-stores* "put" identifier
                       (java:new "java.util.LinkedHashMap")))
        (java:call *java-handles* "put" handle identifier)
        (format t ";; [java store] open ~s -> handle ~a~%" identifier handle)
        handle)))

(defun java-store (member &rest args)
  (cond ((string= member "open") (java-open (nth 0 args)))
        ((string= member "bucket-get")
         (java:call (java-bucket (nth 0 args)) "get" (nth 1 args)))
        ((string= member "bucket-set")
         (format t ";; [java store] set ~a = ~a~%" (nth 1 args) (nth 2 args))
         (java:call (java-bucket (nth 0 args)) "put" (nth 1 args) (nth 2 args))
         nil)
        ((string= member "bucket-delete")
         (format t ";; [java store] delete ~a~%" (nth 1 args))
         (java:call (java-bucket (nth 0 args)) "remove" (nth 1 args))
         nil)
        ((string= member "bucket-exists")
         (if (java:call (java-bucket (nth 0 args)) "containsKey" (nth 1 args))
             t
             nil))
        ((string= member "bucket-list-keys")
         ;; java.util.LinkedHashMap keeps insertion order, so the keys come back
         ;; in the order they were first written. The cursor (arg 1) stays nil:
         ;; this store hands back every key at once, so there is no next page --
         ;; and the key-response record crosses as a keyword plist.
         (list :keys (java:call (java:call (java-bucket (nth 0 args)) "keySet")
                                "toArray")
               :cursor nil))
        ((string= member "bucket-drop")
         ;; Releasing the reference, not the store: the LinkedHashMap stays, and the
         ;; next open sees every key still in it.
         (format t ";; [java store] drop handle ~a~%" (nth 0 args))
         (java:call *java-handles* "remove" (nth 0 args))
         nil)
        (t (error 'rontolisp:wit-error
                  :payload :other
                  :message (concatenate 'string "java store: no such member "
                                        member)))))

;;; Bound AFTER the memory store, so this one wins: rontolisp:wit-provide replaces
;;; whatever was bound for the interface before it.
(rontolisp:wit-provide "wasi:keyvalue/store@0.2.0-draft" #'java-store)


---

# FILE: references/examples/wit/keyvalue/memory-store.lisp

;;;; memory-store.lisp -- an in-memory implementation of wasi:keyvalue/store.
;;;;
;;;; rontolisp knows how to BIND a provider to a WIT interface. It does not know
;;;; what wasi:keyvalue is, and it ships no store. A store is ordinary user code,
;;;; and this file is one -- portable Lisp, nothing backend-specific, ~50 lines.
;;;;
;;;; A provider is an ordinary Lisp callable taking the bound function's Lisp
;;;; member name -- a STRING: "open", "bucket-get", ... -- and then that
;;;; function's arguments, a resource method's handle included. That is the whole
;;;; contract, so a store is just a function.
;;;;
;;;; It implements the SAME wit/keyvalue.wit that the component's host (wasmtime's
;;;; own wasi:keyvalue) implements -- which is why page-hits.lisp prints the same
;;;; thing against either one, down to the no-such-store error arm.
;;;;
;;;; Use it with:  (require :kv-memory "memory-store.lisp")

(provide :kv-memory)

;;; THE DATA lives under the store's IDENTIFIER, not under a handle: a handle is a
;;; REFERENCE to a store, never the store itself, so dropping one does not take the
;;; store with it. That distinction is the whole point of the two tables below, and
;;; it is what `bucket-drop` is here to make visible.
;;;
;;; (One measured difference from wasmtime's own `-S keyvalue=y` provider, which is
;;; an in-memory convenience rather than a real store: it hands each `open` an
;;; INDEPENDENT snapshot, so a write through one bucket is invisible to a later open
;;; there. A store that means it -- this one, wasmCloud's, a redis -- shares.)
(defvar *kv-stores* (make-hash-table :test #'equal))

;;; A WIT resource is an opaque integer handle. Nothing may interpret it but the
;;; provider that handed it out, so any counter will do; this one maps the handles
;;; still outstanding to the store each one refers to. `bucket-drop` deletes an
;;; entry HERE and nothing else.
(defvar *kv-handles* (make-hash-table :test #'eql))

(defvar *kv-next-handle* 1)

;;; The identifiers this store recognizes. Like every wasi:keyvalue host, it
;;; answers the default store under the empty identifier and raises no-such-store
;;; for anything else -- the WIT says so, so a store that means it must too.
(defvar *kv-identifiers* '(""))

(defun kv-bucket (handle)
  ;; An unknown handle -- never opened, or already dropped -- is the error arm of
  ;; every one of the resource's methods, and the settled WIT mapping says an error
  ;; arm SIGNALS a condition, on every backend. So that is what a provider does, and
  ;; the payload is the WIT variant.
  (let ((identifier (gethash handle *kv-handles*)))
    (if (null identifier)
        (error 'rontolisp:wit-error
               :payload :no-such-store
               :message "memory store: not an open bucket handle")
        (gethash identifier *kv-stores*))))

(defun kv-open (identifier)
  (if (not (member identifier *kv-identifiers* :test #'string=))
      (error 'rontolisp:wit-error
       :payload :no-such-store
       :message (concatenate 'string "memory store: no such store " identifier))
      ;; Every open hands out a FRESH handle -- what a real host does, each one an
      ;; owned resource the guest gives back on its own -- onto the one store the
      ;; identifier names, created on first sight.
      (let ((handle *kv-next-handle*))
        (setq *kv-next-handle* (+ handle 1))
        (if (null (gethash identifier *kv-stores*))
            (setf (gethash identifier *kv-stores*)
                  (make-hash-table :test #'equal)))
        (setf (gethash handle *kv-handles*) identifier)
        handle)))

(defun memory-store (member &rest args)
  ;; The ok arm of a WIT result IS the return value, so nothing is wrapped:
  ;; `get` answers option>, which is the value string or nil; the three
  ;; result<_, error> writers answer nil; and `list-keys` answers the record
  ;; key-response, which is a keyword plist.
  (cond ((string= member "open") (kv-open (nth 0 args)))
        ((string= member "bucket-get")
         (gethash (nth 1 args) (kv-bucket (nth 0 args))))
        ((string= member "bucket-set")
         (setf (gethash (nth 1 args) (kv-bucket (nth 0 args))) (nth 2 args))
         nil)
        ((string= member "bucket-delete")
         (remhash (nth 1 args) (kv-bucket (nth 0 args)))
         nil)
        ((string= member "bucket-exists")
         (if (gethash (nth 1 args) (kv-bucket (nth 0 args))) t nil))
        ((string= member "bucket-list-keys")
         ;; The cursor (arg 1, an option) is nil here and stays nil: this
         ;; store hands back every key at once, so there is never a next page.
         (let ((keys nil))
           (maphash (lambda (key value)
                      value
                      (push key keys)) (kv-bucket (nth 0 args)))
           (list :keys (nreverse keys) :cursor nil)))
        ((string= member "bucket-drop")
         ;; A resource is released by its interface's `drop`, which WIT declares no
         ;; function for -- rontolisp spells it `-drop` and dispatches it
         ;; here like any other member. Dropping a handle RELEASES THE REFERENCE, it
         ;; does not delete the store: the data is keyed by identifier, and the next
         ;; open sees it all. A provider with nothing to release just answers nil.
         (remhash (nth 0 args) *kv-handles*)
         nil)
        (t (error 'rontolisp:wit-error
                  :payload :other
                  :message (concatenate 'string "memory store: no such member "
                                        member)))))

;;; This is the whole binding: the interface id from the .wit, and a function.
(rontolisp:wit-provide "wasi:keyvalue/store@0.2.0-draft" #'memory-store)


---

# FILE: references/examples/wit/keyvalue/page-hits-server.lisp

;;;; page-hits-server -- the page-hit counter of page-hits.lisp, but SERVED.
;;;;
;;;; page-hits.lisp counts page views in a store and exits. This one is the same
;;;; counter behind an HTTP server: every request records a hit for its own path
;;;; and answers the running tally. Two halves that belong together --
;;;; `rontolisp:http-handler` (the server) and `rontolisp:wit-import` (the store)
;;;; -- and the point of putting them together is that the state lives OUTSIDE the
;;;; program, in whatever implements the WIT.
;;;;
;;;; That is not a nicety for a served component: a wasi:http host instantiates it
;;;; AFRESH FOR EVERY REQUEST, so a global hash table reads back empty every time.
;;;; A page-hit counter simply cannot be written that way. Through the store it
;;;; can, and the very same source still runs on the interpreter and the JVM.
;;;;
;;;;   rontolisp page-hits-server.lisp             the interpreter -- a blocking
;;;;                                               server on :8080, memory-store.lisp
;;;;                                               behind the WIT
;;;;   rontolisp page-hits-server.lisp \           the JVM -- same, but java-store.lisp
;;;;     -o Server.class                           REPLACES the store with a real
;;;;                                               java.util.LinkedHashMap
;;;;   rontolisp page-hits-server.lisp \           a WASI component that EXPORTS
;;;;     -o server.wasm --component                wasi:http/incoming-handler and
;;;;     + wash dev                                IMPORTS wasi:keyvalue/store: on
;;;;                                               wasmCloud the host serves the
;;;;                                               requests and a real key-value
;;;;                                               provider holds the counts, so they
;;;;                                               survive the instance
;;;;
;;;; The same component runs under `wasmtime serve -S keyvalue=y`, where the calls
;;;; reach wasmtime's own store -- but that store is wasmtime's in-memory provider,
;;;; which it rebuilds per instance (so per REQUEST here): the tally starts over
;;;; every time. Whether a store outlives the instance is the host's business, not
;;;; the component's, and that is exactly what the WIT boundary makes swappable.
;;;;
;;;; See README.md ("Serve it") for the commands.

(rontolisp:wit-import "wit/keyvalue.wit"
                      :interface "wasi:keyvalue/store@0.2.0-draft"
                      :package kv)

;;; The store behind the interface, on the backends that need one -- the same two
;;; files page-hits.lisp binds, and the same one line each. On the WASM backends
;;; the host IS the provider, so a wit-provide is inert there and these stores
;;; simply go unused.
(require :kv-memory "memory-store.lisp")

#+rontolisp-jvm (require :kv-java "java-store.lisp")

;;; The default store, which every wasi:keyvalue host recognizes under the empty
;;; identifier.
(defvar *store* "")

;;; Read the counter for PAGE, add one, write it back -- the record-hit of
;;; page-hits.lisp, answering the new count. A value is a list, which crosses
;;; as a string; an absent key is nil, so the first hit starts from 0.
(defun record-hit (bucket page)
  (let ((hits
         (+ 1
            (let ((seen (kv:bucket-get bucket page)))
              (if seen (parse-integer seen) 0)))))
    (kv:bucket-set bucket page (princ-to-string hits))
    hits))

(defun sorted-keys (bucket)
  (sort (getf (kv:bucket-list-keys bucket nil) :keys) #'string<))

;;; The tally, one line per path, as the response body.
(defun report (bucket)
  (let ((body ""))
    (dolist (key (sorted-keys bucket))
      (setq body
            (concatenate 'string body
             (format nil "~a = ~a~%" key (kv:bucket-get bucket key)))))
    body))

;;; The handler: one request = one hit. Nothing in it knows where the counts live,
;;; and nothing in it is about a backend -- it is an ordinary rontolisp:http-handler
;;; whose state happens to be somebody else's.
(defun handle (env)
  (let* ((page (getf env :path-info))
         (bucket (kv:open *store*))
         (hits (record-hit bucket page)))
    (list 200 '(:content-type "text/plain")
          (list
           (format nil "~a -> ~a hit~:[s~;~]~%~%hits per page:~%~a" page hits
                   (= hits 1) (report bucket))))))

;;; On the interpreter / JVM this blocks and serves on port 8080; under --component
;;; the port is ignored (the host provides the socket).
(rontolisp:http-handler 'handle 8080)


---

# FILE: references/examples/wit/keyvalue/page-hits.lisp

;;;; page-hits -- one WIT interface, a different store behind it per backend.
;;;;
;;;; A page-view counter written against the real wasi:keyvalue/store: open the
;;;; store, read a counter, write it back, ask what keys are in there. Nothing in
;;;; the program below says WHERE those key-value pairs live -- that is the point.
;;;;
;;;; `rontolisp:wit-import` reads wit/keyvalue.wit and binds the interface's
;;;; functions as ordinary Lisp functions in the package kv. What each one calls
;;;; is decided separately, and differently, per backend:
;;;;
;;;;   rontolisp page-hits.lisp                    the interpreter -- memory-store.lisp
;;;;                                               is the provider, a portable Lisp
;;;;                                               hash-table store
;;;;   rontolisp page-hits.lisp -o PageHits.class  the JVM -- java-store.lisp is
;;;;                                               required too, and REPLACES it with
;;;;                                               a real java.util.LinkedHashMap
;;;;                                               (the ";; [java store]" lines below
;;;;                                               are the proof that the calls land
;;;;                                               there)
;;;;   rontolisp page-hits.lisp -o kv.wasm --component
;;;;     + wasmtime run -S keyvalue=y ...          a WASI component -- and here the
;;;;                                               provider is the HOST: wasmtime's own
;;;;                                               wasi:keyvalue implementation, which
;;;;                                               has never heard of this program
;;;;
;;;; rontolisp itself ships NO store: it knows how to bind a provider to a WIT
;;;; interface, and nothing about what wasi:keyvalue is. Both Lisp stores below are
;;;; ordinary user code. That is the whole shape of the feature -- develop against
;;;; a fake, deploy against the real thing, and never touch the program in between.
;;;;
;;;; See README.md for the commands. (Preview 1 WASM is the one backend this does
;;;; not reach: a core import carries flat values only, and every function of this
;;;; interface returns a `result`.)

(rontolisp:wit-import "wit/keyvalue.wit"
                      :interface "wasi:keyvalue/store@0.2.0-draft"
                      :package kv)

;;; The names that just appeared, and where they come from:
;;;
;;;   kv:open              open: func(identifier: string) -> result
;;;   kv:bucket-get        bucket.get, with the handle as the first argument
;;;   kv:bucket-set        bucket.set
;;;   kv:bucket-delete     bucket.delete
;;;   kv:bucket-exists     bucket.exists
;;;   kv:bucket-list-keys  bucket.list-keys
;;;
;;; A `resource bucket`'s method `get` binds as `bucket-get` taking the handle
;;; first, so `b.get(key)` in WIT is `(kv:bucket-get b key)` here. The handle
;;; itself is opaque -- an integer you pass back, never one you interpret.
;;;
;;; The types come across as rontolisp's settled WIT mapping:
;;;
;;;   result        the bucket handle -- and the ERROR arm signals
;;;                                rontolisp:wit-error, which handler-case catches
;;;   option>             the value string, or nil when the key is absent
;;;   list                     a string (bytes, one per character)
;;;   option                  a number, or nil -- so a cursor-less call passes nil
;;;   record key-response          a keyword plist: (:keys ("/index" ...) :cursor nil)

;;; The store. Ordinary Lisp -- see memory-store.lisp, which ends in one
;;; (rontolisp:wit-provide "wasi:keyvalue/store@0.2.0-draft" #'memory-store).
;;; On the WASM backends the host IS the provider, so a wit-provide is inert there
;;; and this file's store simply goes unused.
(require :kv-memory "memory-store.lisp")

;;; ...and it is swappable. On the JVM, bind a store backed by a real Java map
;;; instead -- the same one line that would bind a Redis client. This is the only
;;; backend-specific line in the whole example, and the program never sees it.
#+rontolisp-jvm (require :kv-java "java-store.lisp")

;;; ---------------------------------------------------------------------------
;;; The program. Every line below is store-agnostic: it knows the WIT, and
;;; nothing else.
;;; ---------------------------------------------------------------------------

(defvar *requests* '("/index" "/pricing" "/index" "/docs" "/index" "/pricing"))

;;; The default store, which every wasi:keyvalue host recognizes under the empty
;;; identifier. An identifier a host does NOT recognize is the `no-such-store`
;;; error arm -- which is what the handler-case near the bottom shows.
(defvar *store* "")

;;; Read the counter for PAGE, add one, write it back. `bucket.get` answers an
;;; option, which is the value or nil -- no unwrapping ceremony, nil IS "absent".
;;; A value is a list, which crosses as a string.
(defun record-hit (bucket page)
  (let ((seen (kv:bucket-get bucket page)))
    (kv:bucket-set bucket page
                   (princ-to-string (+ 1 (if seen (parse-integer seen) 0))))))

;;; `bucket.list-keys` pages, so it takes an option cursor (nil = the first
;;; page) and answers a `key-response` record -- which crosses as a keyword plist,
;;; so the keys are (getf response :keys) and :cursor is nil when there are no
;;; more pages. These stores hand back everything at once.
(defun sorted-keys (bucket)
  (sort (getf (kv:bucket-list-keys bucket nil) :keys) #'string<))

;;; `open` answers a result: the ok arm IS the value, so the
;;; handle comes straight back. (The error arm would signal -- see below.)
(let ((bucket (kv:open *store*)))
  (dolist (page *requests*) (record-hit bucket page))

  (format t "~%hits per page:~%")
  (dolist (key (sorted-keys bucket))
    (format t "  ~a = ~a~%" key (kv:bucket-get bucket key)))

  (format t "~%/docs exists?      ~a~%"
          (if (kv:bucket-exists bucket "/docs") "yes" "no"))
  (kv:bucket-delete bucket "/docs")
  (format t "/docs exists now?  ~a~%"
          (if (kv:bucket-exists bucket "/docs") "yes" "no"))
  (format t "keys:              ~s~%" (sorted-keys bucket))
  (format t "/nope:             ~s~%" (kv:bucket-get bucket "/nope")))

;;; The error arm of a WIT result signals rontolisp:wit-error -- so a store's
;;; failures are caught with handler-case like any other condition, and the WIT
;;; variant that failed is the payload. No host recognizes this store identifier:
;;; the same no-such-store comes back from three completely different providers.
(handler-case (kv:open "not-a-store-anyone-has")
  (rontolisp:wit-error (e)
    (format t "bad store:         ~a~%" (rontolisp:wit-error-payload e))))

;;; Each binding is an ordinary defun, so it is an ordinary function VALUE too:
;;; #'kv:bucket-set is a first-class function, funcall takes it, mapcar maps it.
;;; Nothing about the WIT boundary leaks into the call sites.
(let ((bucket (kv:open *store*)) (set-key #'kv:bucket-set))
  (mapcar (lambda (key) (funcall set-key bucket key "seeded")) '("/a" "/b"))
  (format t "seeded:            ~s~%"
          (remove-if-not
           (lambda (key) (string= "seeded" (kv:bucket-get bucket key)))
           (sorted-keys bucket))))


---

# FILE: references/examples/wit/keyvalue/wit/keyvalue.wit

package wasi:keyvalue@0.2.0-draft;

/// The `store` interface of
/// [wasi:keyvalue](https://github.com/WebAssembly/wasi-keyvalue), vendored here
/// verbatim -- NOT a simplification, and that is the point: this is the interface
/// a real host implements, so the component this example compiles to talks to
/// `wasmtime -S keyvalue=y` with no adapter and no rewriting, and the very same
/// file is what the two Lisp-side stores beside it implement.
///
/// The version really is `0.2.0-draft`: that is what wasi-keyvalue publishes and
/// what wasmtime provides today.
interface store {
  /// The set of errors which may be raised by functions in this package.
  variant error {
    /// The host does not recognize the store identifier requested.
    no-such-store,
    /// The requesting component does not have access to the specified store
    /// (which may or may not exist).
    access-denied,
    /// Some implementation-specific error has occurred (e.g. I/O).
    other(string),
  }

  /// A response to a `list-keys` operation.
  record key-response {
    /// The list of keys returned by the query.
    keys: list,
    /// The continuation token to use to fetch the next page of keys. If this is
    /// `null`, then there are no more keys to fetch.
    cursor: option,
  }

  /// Get the bucket with the specified identifier.
  ///
  /// `error::no-such-store` will be raised if the `identifier` is not recognized.
  open: func(identifier: string) -> result;

  /// A bucket is a collection of key-value pairs. Each key-value pair is stored
  /// as an entry in the bucket, and the bucket itself acts as a collection of all
  /// these entries.
  resource bucket {
    /// Get the value associated with the specified `key`.
    get: func(key: string) -> result>, error>;

    /// Set the value associated with the key in the store.
    set: func(key: string, value: list) -> result<_, error>;

    /// Delete the key-value pair associated with the key in the store.
    delete: func(key: string) -> result<_, error>;

    /// Check if the key exists in the store.
    exists: func(key: string) -> result;

    /// Get all the keys in the store with an optional cursor (for use in
    /// pagination).
    list-keys: func(cursor: option) -> result;
  }
}


---

# FILE: references/examples/wit/lisp-calls-rust/README.md

# wit/lisp-calls-rust -- a Lisp program calling a Rust component

[`wit/keyvalue/`](https://github.com/making/rontolisp/blob/develop/examples/wit/keyvalue) calls a WIT interface that a **host** implements.
This directory calls one that **another guest language** implements: a Lisp
command imports `example:textkit/casing`, a Rust component exports it, and
[`wac`](https://github.com/bytecodealliance/wac) composes the two into one
runnable component. The `.wit` is the only thing the two languages share.

```
   app.lisp                        rust-shouter
   (Lisp command)                  (Rust reactor)
   --------------                  --------------
   tk:shout("hello world")  ─────▶ export casing.shout
                                   "hello world".to_uppercase() + "!"
   "HELLO WORLD!" ◀────────────────
```

For the reverse direction see
[`wit/rust-calls-lisp/`](https://github.com/making/rontolisp/blob/develop/examples/wit/rust-calls-lisp). The app also runs **standalone** on
the interpreter and JVM, where a small Lisp `rontolisp:wit-provide` answers the
same interface (§2) — develop against the Lisp fallback, deploy against Rust,
and never touch the program in between.

## Prerequisites

- a rontolisp binary (or the executable JAR -- set `RONTOLISP` accordingly)
- a Rust toolchain with the `wasm32-unknown-unknown` target
  (`rustup target add wasm32-unknown-unknown`)
- [`cargo-component`](https://github.com/bytecodealliance/cargo-component)
  (`cargo install cargo-component`) to build the Rust component
- [`wac`](https://github.com/bytecodealliance/wac) (`cargo install wac-cli`) and
  `wasmtime` 46+ (plus `wasm-tools` if you want to inspect the components)

[`build.sh`](build.sh) runs it all end to end:

```bash
RONTOLISP=../../../target/rontolisp ./build.sh
# hello world  ->  HELLO WORLD!
# component model  ->  COMPONENT MODEL!
# rust and lisp  ->  RUST AND LISP!
```

## 1. The interface

[`wit/textkit.wit`](wit/textkit.wit):

```wit
package example:textkit;

interface casing {
  shout: func(text: string) -> string;
}
```

## 2. The Lisp app -- imports it, and provides a fallback

[`app.lisp`](app.lisp) binds the interface's functions with `rontolisp:wit-import`
(package `tk`), then calls `tk:shout`:

```lisp
(rontolisp:wit-import "wit/textkit.wit"
                      :interface "example:textkit/casing"
                      :package tk)

(dolist (phrase '("hello world" "component model" "rust and lisp"))
  (format t "~a  ->  ~a~%" phrase (tk:shout phrase)))
```

On the interpreter and JVM there is no Rust component to call, so a `wit-import`
needs a **provider** -- an ordinary Lisp callable, bound with
`rontolisp:wit-provide`, that takes the member name (`"shout"`) and the call's
arguments. A two-line lambda makes the file run **standalone**:

```lisp
(rontolisp:wit-provide "example:textkit/casing"
                       #'(lambda (member &rest args)
                           (cond ((string= member "shout")
                                  (concatenate 'string (string-upcase (first args)) "!"))
                                 (t (error "casing: unknown member ~a" member)))))
```

So the same file runs three ways and prints the same thing each time:

```bash
rontolisp app.lisp                                       # interpreter: the Lisp provider answers
rontolisp app.lisp -o App.class && java -cp . App        # JVM: the same provider
rontolisp app.lisp -o app.wasm --component --optimize    # imports the interface instead
```

On WASM a `wit-provide` is **inert**, so the composed Rust component answers
there: on the interpreter and JVM this is Lisp calling Lisp, composed it is Lisp
calling Rust.

## 3. The Rust component -- exports it

The crate was scaffolded with [`cargo-component`](https://github.com/bytecodealliance/cargo-component)
(`cargo install cargo-component`):

```bash
cargo component new --lib rust-shouter --name shouter --namespace example
```

That writes `rust-shouter/{Cargo.toml, src/lib.rs}` and a starter `wit/world.wit`.
Two edits turn the skeleton into this example: delete the crate-local `wit/` and
point [`Cargo.toml`](rust-shouter/Cargo.toml) at the **shared** WIT instead, so
the two sides cannot drift --

```toml
[package.metadata.component.target]
path = "../wit"
world = "emphasizer"
```

-- then fill in the one export in [`src/lib.rs`](rust-shouter/src/lib.rs) (the
`mod bindings` is generated by cargo-component from that WIT):

```rust
#[allow(warnings)]
mod bindings;
use bindings::exports::example::textkit::casing::Guest;

struct Component;
impl Guest for Component {
    fn shout(text: String) -> String {
        format!("{}!", text.to_uppercase())
    }
}
bindings::export!(Component with_types_in bindings);
```

It imports no WASI, so a `wasm32-unknown-unknown` build **is** the component --
cargo-component runs the componentization, no adapter and no separate
`wasm-tools component new`:

```bash
cd rust-shouter && cargo component build --release --target wasm32-unknown-unknown && cd ..
cp rust-shouter/target/wasm32-unknown-unknown/release/shouter.wasm shouter.wasm
```

## 4. Compose and run

`wac plug` fills the app's import with the Rust component's matching export,
pairing them by their WIT type:

```bash
wac plug app.wasm --plug shouter.wasm -o textkit.wasm
wasmtime run -W gc=y textkit.wasm
```
```console
hello world  ->  HELLO WORLD!
component model  ->  COMPONENT MODEL!
rust and lisp  ->  RUST AND LISP!
```

## Notes

Only `string` crosses here, in both directions — the flat, no-`result` subset of
rontolisp's WIT mapping, which is why no `-W exceptions=y` is needed. Full
mapping:
[`rontolisp:wit-import`](https://github.com/making/rontolisp/blob/develop/examples/wit/doc/en/reference/functions/rontolisp-wit-import.md).


---

# FILE: references/examples/wit/lisp-calls-rust/app.lisp

;;;; app -- a Lisp command that calls the `casing` interface.
;;;;
;;;; It IMPORTS the `casing` interface (wit/textkit.wit) and calls `shout` on a
;;;; few phrases. `tk:shout` is an ordinary Lisp function the compiler wrote from
;;;; the WIT; WHAT answers it is decided per backend, and nothing below changes:
;;;;
;;;;   rontolisp app.lisp                          the interpreter -- the Lisp
;;;;                                               provider below answers
;;;;   rontolisp app.lisp -o App.class && java App the JVM -- same provider
;;;;   rontolisp app.lisp -o app.wasm --component --optimize
;;;;     + wac plug + wasmtime                     a WASI component -- the composed
;;;;                                               Rust component answers instead
;;;;
;;;; So on the interpreter and JVM this is Lisp calling Lisp; composed with the
;;;; Rust component it is Lisp calling Rust. The output is identical either way.

(rontolisp:wit-import "wit/textkit.wit"
                      :interface "example:textkit/casing"
                      :package tk)

;;; A wit-import needs a provider on the interpreter and JVM (there is no Rust
;;; component to call there). This lambda reimplements `shout` in two lines: it
;;; takes the bound function's member name ("shout") and that function's
;;; arguments. On every WASM backend a rontolisp:wit-provide is INERT (the
;;; composed component is the provider), so the very same file compiles and runs
;;; there too.
(rontolisp:wit-provide "example:textkit/casing"
                       #'(lambda (member &rest args)
                           (cond
                            ((string= member "shout")
                             (concatenate 'string (string-upcase (first args))
                                          "!"))
                            (t (error "casing: unknown member ~a" member)))))

(defvar *phrases* '("hello world" "component model" "rust and lisp"))

(dolist (phrase *phrases*) (format t "~a  ->  ~a~%" phrase (tk:shout phrase)))


---

# FILE: references/examples/wit/lisp-calls-rust/build.sh

#!/usr/bin/env bash
#
# Lisp calls Rust: build both components, compose them with wac, run the result.
#
# Prerequisites on PATH: a rontolisp binary (or set RONTOLISP), a Rust toolchain
# with the wasm32-unknown-unknown target, wasm-tools, wac (cargo install wac-cli)
# and wasmtime 46+.
#
#   RONTOLISP=../../../target/rontolisp ./build.sh
#
set -euo pipefail
cd "$(dirname "$0")"

RONTOLISP="${RONTOLISP:-rontolisp}"

echo "== 1. the Lisp command (imports example:textkit/casing) =="
"$RONTOLISP" app.lisp -o app.wasm --component --optimize

echo "== 2. the Rust component (cargo-component builds a component directly) =="
# The crate was scaffolded with:
#   cargo component new --lib rust-shouter --name shouter --namespace example
# targeting the shared ../wit (see rust-shouter/Cargo.toml). No adapter needed --
# a wasm32-unknown-unknown reactor imports no WASI.
( cd rust-shouter && cargo component build --release --target wasm32-unknown-unknown )
cp rust-shouter/target/wasm32-unknown-unknown/release/shouter.wasm shouter.wasm

echo "== 3. compose: plug the Rust component into the Lisp app =="
wac plug app.wasm --plug shouter.wasm -o textkit.wasm

echo "== 4. run =="
wasmtime run -W gc=y textkit.wasm


---

# FILE: references/examples/wit/lisp-calls-rust/rust-shouter/Cargo.toml

[package]
name = "shouter"
version = "0.1.0"
edition = "2024"
publish = false

[dependencies]
wit-bindgen-rt = { version = "0.44.0", features = ["bitflags"] }

[lib]
crate-type = ["cdylib"]

[package.metadata.component]
package = "example:shouter"

# Target the shared world instead of a crate-local wit/: `../wit` is the same
# textkit.wit the Lisp app imports, so the two sides cannot drift.
[package.metadata.component.target]
path = "../wit"
world = "emphasizer"

[package.metadata.component.dependencies]

[profile.release]
opt-level = "s"
strip = true


---

# FILE: references/examples/wit/lisp-calls-rust/rust-shouter/src/lib.rs

//! The Rust emphasizer. It EXPORTS the `casing` interface -- `shout` uppercases
//! the text and adds an exclamation mark. The Lisp app calls it; nothing here
//! knows the caller is Lisp, only the WIT does.
//!
//! `mod bindings` is generated by cargo-component from ../wit (see Cargo.toml's
//! [package.metadata.component.target]); `cargo component build` regenerates it.

#[allow(warnings)]
mod bindings;

use bindings::exports::example::textkit::casing::Guest;

struct Component;

impl Guest for Component {
    fn shout(text: String) -> String {
        format!("{}!", text.to_uppercase())
    }
}

bindings::export!(Component with_types_in bindings);


---

# FILE: references/examples/wit/lisp-calls-rust/wit/textkit.wit

package example:textkit;

/// A text emphasizer, implemented in Rust. `shout` uppercases the text and adds
/// an exclamation mark.
interface casing {
  shout: func(text: string) -> string;
}

/// The Rust component's world: it EXPORTS the `casing` interface, and imports
/// nothing -- it needs nothing from anyone. This is the whole "Lisp calls Rust"
/// service.
///
/// There is no world for the Lisp app: `rontolisp:wit-import` binds an INTERFACE
/// (`:interface "example:textkit/casing"`) directly, so the app never names a
/// world. Only the exporting side (Rust here) needs one.
world emphasizer {
  export casing;
}


---

# FILE: references/examples/wit/pipeline/README.md

# wit/pipeline -- three components in one `wac compose`

[`wit/lisp-calls-rust/`](https://github.com/making/rontolisp/blob/develop/examples/wit/lisp-calls-rust) and
[`wit/rust-calls-lisp/`](https://github.com/making/rontolisp/blob/develop/examples/wit/rust-calls-lisp) each wire **one** import to **one**
export, which `wac plug` does in a single command. This example is the case
`plug` cannot do: a **chain** of three components across two languages, where one
component is a plug for another. That is a [`wac compose`](https://github.com/bytecodealliance/wac)
job -- a small script that writes out every edge by hand.

```
   app.lisp                rust-shouter                 stats.lisp
   (Lisp command)          (Rust reactor)               (Lisp reactor)
   ------------            --------------               --------------
   sh:emphasize  ───────▶  export shout.emphasize
   "hello world"           uppercases the text
                           import vowel-count  ───────▶  export vowel-count
                                              ◀───────   counts the vowels: 3
                           appends "!" x 3
   "HELLO WORLD!!!" ◀────  returns the string
```

`emphasize` is Rust's job (uppercasing); `vowel-count` is Lisp's job (counting).
One call to `sh:emphasize` crosses the language boundary twice -- Lisp → Rust →
Lisp -- and the app makes just one Lisp function call.

## Prerequisites

- a rontolisp binary (or the executable JAR -- set `RONTOLISP` accordingly)
- a Rust toolchain with the `wasm32-unknown-unknown` target
  (`rustup target add wasm32-unknown-unknown`)
- [`cargo-component`](https://github.com/bytecodealliance/cargo-component)
  (`cargo install cargo-component`) to build the Rust component
- [`wac`](https://github.com/bytecodealliance/wac) (`cargo install wac-cli`) and
  `wasmtime` 46+ (plus `wasm-tools` if you want to inspect the components)

[`build.sh`](build.sh) runs it all end to end:

```bash
RONTOLISP=../../../target/rontolisp ./build.sh
# hello world  ->  HELLO WORLD!!!
# component model  ->  COMPONENT MODEL!!!!!
# rust and lisp  ->  RUST AND LISP!!!
```

## 1. The three components

Same shapes as the two `plug` examples, combined:

| Component | Role | in [`wit/pipeline.wit`](wit/pipeline.wit) |
|---|---|---|
| [`app.lisp`](app.lisp) | Lisp command | `wit-import`s interface `example:pipeline/shout` |
| [`rust-shouter/`](https://github.com/making/rontolisp/blob/develop/examples/wit/pipeline/rust-shouter) | Rust reactor | world `shouter`: exports `shout`, imports `vowel-count` |
| [`stats.lisp`](stats.lisp) | Lisp reactor | world `statistician`: exports `vowel-count` |

The Rust crate was scaffolded with [`cargo-component`](https://github.com/bytecodealliance/cargo-component)
(`cargo install cargo-component`), then pointed at the shared WIT --
`cargo component new --lib rust-shouter --name shouter --namespace example`, its
crate-local `wit/` deleted, and `[package.metadata.component.target]` in
[`Cargo.toml`](rust-shouter/Cargo.toml) set to `path = "../wit"`,
`world = "shouter"`. See [`wit/lisp-calls-rust/`](https://github.com/making/rontolisp/blob/develop/examples/wit/lisp-calls-rust) for that
step spelled out. Its [`src/lib.rs`](rust-shouter/src/lib.rs) exports `emphasize`
and calls the imported `bindings::vowel_count`.

```bash
rontolisp app.lisp   -o app.wasm   --component --optimize
rontolisp stats.lisp -o stats.wasm --component --optimize
( cd rust-shouter && cargo component build --release --target wasm32-unknown-unknown && cd .. )
cp rust-shouter/target/wasm32-unknown-unknown/release/shouter.wasm shouter.wasm
```

## 2. The composition -- one script, every edge

`wac plug` fills a single socket's imports from plug components; it will not wire
one plug into another. The chain here (stats → shouter → app) needs exactly that,
so it is written out in [`composition.wac`](composition.wac):

```wac
package example:composed;

let stats   = new example:stats { ... };
let shouter = new example:shouter { "vowel-count": stats["vowel-count"], ... };
let app     = new example:app { "example:pipeline/shout": shouter["example:pipeline/shout"], ... };

export app...;
```

- `new example:stats { ... }` instantiates a component; `--dep example:stats=stats.wasm`
  (below) binds the name to the file.
- An instantiation argument is named after the component's **import**: the Rust
  shouter imports the plain function `vowel-count`, the Lisp app imports the
  interface `example:pipeline/shout`. Its value comes from another instance's
  matching **export**, `stats["vowel-count"]` / `shouter["example:pipeline/shout"]`.
- `...` lets every unmentioned import -- the WASI interfaces the Lisp components
  need -- fall through as an import of the composed component, which wasmtime
  satisfies at run time.
- `export app...;` re-exports the app's own exports (its `wasi:cli/run` entry
  point) so `wasmtime run` has something to start.

```bash
wac compose composition.wac \
  --dep example:app=app.wasm \
  --dep example:shouter=shouter.wasm \
  --dep example:stats=stats.wasm \
  -o pipeline.wasm
```

`pipeline.wasm` imports only WASI now -- every `example:pipeline` edge is wired
inside it.

## 3. Run it

```bash
wasmtime run -W gc=y pipeline.wasm
```
```console
hello world  ->  HELLO WORLD!!!
component model  ->  COMPONENT MODEL!!!!!
rust and lisp  ->  RUST AND LISP!!!
```

`component model` has five vowels, so it gets five `!` -- the count came from the
Lisp component, the uppercasing from Rust, and the orchestration from the Lisp
app, all through the one interface.

## plug vs. compose

| | [`wac plug`](https://github.com/making/rontolisp/blob/develop/examples/wit/lisp-calls-rust) | `wac compose` (here) |
|---|---|---|
| input | CLI flags only | a `.wac` script |
| wiring | automatic, by WIT type | written out per edge |
| topology | one socket ← plugs | any graph (chains, fan-out, …) |
| plug into a plug | no | yes |

For the two-component examples, `plug` is the shortest path; a chain like this one
is where `compose` earns its place. Only `string` and an `s32` cross the boundary
here, the flat no-`result` subset, so no `-W exceptions=y` is needed.


---

# FILE: references/examples/wit/pipeline/app.lisp

;;;; app -- the Lisp command that drives the pipeline.
;;;;
;;;; It IMPORTS the `shout` interface (wit/pipeline.wit) and calls `emphasize` on
;;;; a few phrases. `sh:emphasize` is an ordinary Lisp function the compiler wrote
;;;; from the WIT; behind it -- once the three components are composed -- is the
;;;; Rust shouter, which itself calls back into a Lisp component. One line of Lisp
;;;; here, two languages and three components underneath.
;;;;
;;;;   rontolisp app.lisp -o app.wasm --component --optimize
;;;;
;;;; produces a command component that imports `example:pipeline/shout`. It is not
;;;; run on its own -- `wac compose` wires all three components together (see
;;;; README.md / composition.wac / build.sh).

(rontolisp:wit-import "wit/pipeline.wit"
                      :interface "example:pipeline/shout"
                      :package sh)

(defvar *phrases* '("hello world" "component model" "rust and lisp"))

(dolist (phrase *phrases*)
  (format t "~a  ->  ~a~%" phrase (sh:emphasize phrase)))


---

# FILE: references/examples/wit/pipeline/build.sh

#!/usr/bin/env bash
#
# A three-component, two-language pipeline composed with ONE `wac compose`.
#
# Prerequisites on PATH: a rontolisp binary (or set RONTOLISP), a Rust toolchain
# with the wasm32-unknown-unknown target, wasm-tools, wac (cargo install wac-cli)
# and wasmtime 46+.
#
#   RONTOLISP=../../../target/rontolisp ./build.sh
#
set -euo pipefail
cd "$(dirname "$0")"

RONTOLISP="${RONTOLISP:-rontolisp}"

echo "== 1. the two Lisp components =="
"$RONTOLISP" app.lisp   -o app.wasm   --component --optimize
"$RONTOLISP" stats.lisp -o stats.wasm --component --optimize

echo "== 2. the Rust component (cargo-component builds a component directly) =="
# The crate was scaffolded with:
#   cargo component new --lib rust-shouter --name shouter --namespace example
# targeting the shared ../wit (see rust-shouter/Cargo.toml). No adapter needed --
# a wasm32-unknown-unknown reactor imports no WASI.
( cd rust-shouter && cargo component build --release --target wasm32-unknown-unknown )
cp rust-shouter/target/wasm32-unknown-unknown/release/shouter.wasm shouter.wasm

echo "== 3. compose all three with one wac compose =="
# stats -> shouter -> app, wired by composition.wac. `wac plug` cannot do this in
# one step (it fills a single socket; it will not wire a plug into another plug).
wac compose composition.wac \
  --dep example:app=app.wasm \
  --dep example:shouter=shouter.wasm \
  --dep example:stats=stats.wasm \
  -o pipeline.wasm

echo "== 4. run =="
wasmtime run -W gc=y pipeline.wasm


---

# FILE: references/examples/wit/pipeline/composition.wac

// composition.wac -- wire all three components together in one shot.
//
// `wac plug` fills one socket's imports from plug components; it cannot wire a
// plug into another plug. This chain needs exactly that (stats -> shouter ->
// app), so it is a `wac compose` job: every edge is written out by hand.
//
//   wac compose composition.wac \
//     --dep example:app=app.wasm \
//     --dep example:shouter=shouter.wasm \
//     --dep example:stats=stats.wasm \
//     -o pipeline.wasm
//
// Each `--dep name=file.wasm` binds a package name used below to a built
// component. The `...` lets every unmentioned import (the WASI interfaces the
// Lisp components need) fall through as an import of the composed component,
// which wasmtime satisfies at run time.

package example:composed;

// The Lisp vowel counter. Nothing to wire in -- it only needs WASI (via `...`).
let stats = new example:stats { ... };

// The Rust shouter. Its `vowel-count` import is satisfied by the counter above.
let shouter = new example:shouter {
  "vowel-count": stats["vowel-count"],
  ...
};

// The Lisp app. Its `example:pipeline/shout` import is satisfied by the shouter.
let app = new example:app {
  "example:pipeline/shout": shouter["example:pipeline/shout"],
  ...
};

// Re-export the app's own exports (its wasi:cli/run entry point) as the
// composition's exports, so `wasmtime run` has something to start.
export app...;


---

# FILE: references/examples/wit/pipeline/rust-shouter/Cargo.toml

[package]
name = "shouter"
version = "0.1.0"
edition = "2024"
publish = false

[dependencies]
wit-bindgen-rt = { version = "0.44.0", features = ["bitflags"] }

[lib]
crate-type = ["cdylib"]

[package.metadata.component]
package = "example:shouter"

# Target the shared world instead of a crate-local wit/: `../wit` is the same
# pipeline.wit the Lisp components use, so no side can drift.
[package.metadata.component.target]
path = "../wit"
world = "shouter"

[package.metadata.component.dependencies]

[profile.release]
opt-level = "s"
strip = true


---

# FILE: references/examples/wit/pipeline/rust-shouter/src/lib.rs

//! The Rust shouter -- one component on BOTH sides of the WIT boundary.
//!
//! It EXPORTS the `shout` interface (the Lisp app calls `emphasize`) and IMPORTS
//! the `vowel-count` function (a Lisp component supplies it). So a single call to
//! `emphasize` is Lisp -> Rust -> Lisp: Rust uppercases the text and asks Lisp to
//! count the vowels, to decide how many `!` to append. `mod bindings` is
//! generated by cargo-component from ../wit (see Cargo.toml).

#[allow(warnings)]
mod bindings;

use bindings::exports::example::pipeline::shout::Guest;
use bindings::vowel_count;

struct Component;

impl Guest for Component {
    fn emphasize(text: String) -> String {
        // Rust -> Lisp: the imported `vowel-count` is an ordinary function call.
        let vowels = vowel_count(&text);
        let mut out = text.to_uppercase();
        for _ in 0..vowels.max(0) {
            out.push('!');
        }
        out
    }
}

bindings::export!(Component with_types_in bindings);


---

# FILE: references/examples/wit/pipeline/stats.lisp

;;;; stats -- the Lisp statistician.
;;;;
;;;; One plain function, `vowel-count`, exported through the `statistician` world
;;;; of wit/pipeline.wit. The Rust shouter IMPORTS this function and calls it to
;;;; decide how many `!` to append -- so this is the "Rust calls Lisp" edge of the
;;;; pipeline.
;;;;
;;;;   rontolisp stats.lisp -o stats.wasm --component --optimize
;;;;
;;;; produces a reactor component that exports `vowel-count`. It is not run on its
;;;; own -- `wac compose` plugs it into the Rust shouter (see README.md).

;;; A vowel is one of a e i o u, in either case. Spelled out with char= so it
;;; lowers to the smallest subset, with no dependence on sequence built-ins.
(defun vowelp (c)
  (let ((d (char-downcase c)))
    (or (char= d #\a) (char= d #\e) (char= d #\i) (char= d #\o) (char= d #\u))))

;;; Count the vowels in TEXT.
;;; WIT: vowel-count: func(text: string) -> s32
(defun vowel-count (text)
  (let ((n 0))
    (dotimes (i (length text)) (when (vowelp (char text i)) (incf n)))
    n))

(rontolisp:wit-export "wit/pipeline.wit" :world statistician)


---

# FILE: references/examples/wit/pipeline/wit/pipeline.wit

package example:pipeline;

/// The text emphasizer. Implemented by the Rust component and called by the Lisp
/// app. `emphasize` uppercases the text and appends one `!` per vowel -- and it
/// learns the vowel count by calling back into a Lisp component, so a single call
/// crosses the language boundary twice.
interface shout {
  emphasize: func(text: string) -> string;
}

/// The Rust shouter's world. It EXPORTS the `shout` interface (what the Lisp app
/// imports), and IMPORTS a plain `vowel-count` function (what a Lisp component
/// supplies). So the Rust component is on both sides of the boundary.
world shouter {
  export shout;
  import vowel-count: func(text: string) -> s32;
}

/// The Lisp statistician's world. It EXPORTS `vowel-count` as a plain function --
/// exactly the shape the Rust world above imports. (The Lisp app needs no world:
/// `rontolisp:wit-import` binds the `shout` interface directly.)
world statistician {
  export vowel-count: func(text: string) -> s32;
}


---

# FILE: references/examples/wit/rust-calls-lisp/README.md

# wit/rust-calls-lisp -- a Rust program calling a Lisp component

The mirror image of [`wit/lisp-calls-rust/`](https://github.com/making/rontolisp/blob/develop/examples/wit/lisp-calls-rust). There a Lisp
program called Rust; here a **Rust** component calls a **Lisp** one. The Lisp
component exports a plain `vowel-count` function; the Rust component imports it,
and [`wac`](https://github.com/bytecodealliance/wac) composes the two into one
runnable component.

```
   describe("hello world")         rust-describer                counter.lisp
   (host --invoke)                 (Rust reactor)                (Lisp reactor)
                                   --------------                --------------
                            ─────▶ export describe
                                   import vowel-count  ────────▶ export vowel-count
                                                      ◀────────  counts: 3
                                   format!("... {} vowels")
   "\"hello world\" has 3 vowels" ◀───────────────────────────
```

`describe` is Rust's job (building the sentence); `vowel-count` is Lisp's job. The
host calls `describe`; nothing in the Rust code knows the counter is written in
Lisp -- only the WIT does.

## Prerequisites

- a rontolisp binary (or the executable JAR -- set `RONTOLISP` accordingly)
- a Rust toolchain with the `wasm32-unknown-unknown` target
  (`rustup target add wasm32-unknown-unknown`)
- [`cargo-component`](https://github.com/bytecodealliance/cargo-component)
  (`cargo install cargo-component`) to build the Rust component
- [`wac`](https://github.com/bytecodealliance/wac) (`cargo install wac-cli`) and
  `wasmtime` 46+ (plus `wasm-tools` if you want to inspect the components)

[`build.sh`](build.sh) runs it all end to end:

```bash
RONTOLISP=../../../target/rontolisp ./build.sh
# "\"hello world\" has 3 vowels"
```

## 1. The contract

[`wit/vowels.wit`](wit/vowels.wit) declares the interface Rust exports and the
plain function it imports:

```wit
package example:vowels;

interface describe {
  describe: func(text: string) -> string;
}

world describer {                                  // the Rust component
  export describe;
  import vowel-count: func(text: string) -> s32;
}

world counter {                                    // the Lisp component
  export vowel-count: func(text: string) -> s32;
}
```

`vowel-count` is a **plain function**, not an interface, because
`rontolisp:wit-export` implements plain function exports -- so the Rust world
imports it as a bare function, matching what Lisp exports.

## 2. The Lisp component -- exports the function

[`counter.lisp`](counter.lisp) implements the `counter` world:

```lisp
(defun vowel-count (text)
  (let ((n 0))
    (dotimes (i (length text))
      (when (vowelp (char text i)) (incf n)))
    n))

(rontolisp:wit-export "wit/vowels.wit" :world counter)
```

```bash
rontolisp counter.lisp -o counter.wasm --component --optimize
```

A reactor component that exports `vowel-count: func(text: string) -> s32`.

## 3. The Rust component -- imports it

The crate was scaffolded with [`cargo-component`](https://github.com/bytecodealliance/cargo-component)
(`cargo install cargo-component`):

```bash
cargo component new --lib rust-describer --name describer --namespace example
```

Delete the crate-local `wit/` it writes and point [`Cargo.toml`](rust-describer/Cargo.toml)
at the **shared** WIT instead:

```toml
[package.metadata.component.target]
path = "../wit"
world = "describer"
```

Then fill in [`src/lib.rs`](rust-describer/src/lib.rs), calling the imported
`vowel_count` like any other function (`mod bindings` is generated by
cargo-component from that WIT):

```rust
#[allow(warnings)]
mod bindings;
use bindings::exports::example::vowels::describe::Guest;
use bindings::vowel_count;

struct Component;
impl Guest for Component {
    fn describe(text: String) -> String {
        let n = vowel_count(&text);                // Rust -> Lisp
        let plural = if n == 1 { "" } else { "s" };
        format!("\"{text}\" has {n} vowel{plural}")
    }
}
bindings::export!(Component with_types_in bindings);
```

It imports no WASI, so a `wasm32-unknown-unknown` build **is** the component --
cargo-component runs the componentization, no adapter:

```bash
cd rust-describer && cargo component build --release --target wasm32-unknown-unknown && cd ..
cp rust-describer/target/wasm32-unknown-unknown/release/describer.wasm describer.wasm
```

## 4. Compose and run

`wac plug` fills the Rust describer's `vowel-count` import with the Lisp
component's matching export:

```bash
wac plug describer.wasm --plug counter.wasm -o vowels.wasm
```

`vowels.wasm` exports `describe` and no longer imports `vowel-count` -- it is
wired inside. The host calls the export by name with a
[WAVE](https://component-model.bytecodealliance.org/) argument:

```bash
wasmtime run -W gc=y --invoke 'describe("hello world")' vowels.wasm
```
```console
"\"hello world\" has 3 vowels"
```

The result is a `string`, so `wasmtime` prints it quoted (the inner quotes are
the ones `describe` put around the phrase). `describe("a")` answers
`"\"a\" has 1 vowel"` — the count that picked the singular came from the Lisp
component.

## Notes

Only `string` and an `s32` cross the boundary -- the flat, no-`result` subset, so
no `-W exceptions=y` is needed. The Rust component is a reactor invoked by the
host; making Rust a printing command instead would pull in WASI on the Rust side,
which is a separate concern from the cross-language call this example is about.
The full type mapping is documented in
[`rontolisp:wit-export`](https://github.com/making/rontolisp/blob/develop/examples/wit/doc/en/reference/functions/rontolisp-wit-export.md).


---

# FILE: references/examples/wit/rust-calls-lisp/build.sh

#!/usr/bin/env bash
#
# Rust calls Lisp: build both components, compose them with wac, run the result.
#
# Prerequisites on PATH: a rontolisp binary (or set RONTOLISP), a Rust toolchain
# with the wasm32-unknown-unknown target, wasm-tools, wac (cargo install wac-cli)
# and wasmtime 46+.
#
#   RONTOLISP=../../../target/rontolisp ./build.sh
#
set -euo pipefail
cd "$(dirname "$0")"

RONTOLISP="${RONTOLISP:-rontolisp}"

echo "== 1. the Lisp component (exports vowel-count; reactor) =="
"$RONTOLISP" counter.lisp -o counter.wasm --component --optimize

echo "== 2. the Rust component (cargo-component builds a component directly) =="
# The crate was scaffolded with:
#   cargo component new --lib rust-describer --name describer --namespace example
# targeting the shared ../wit (see rust-describer/Cargo.toml). No adapter needed --
# a wasm32-unknown-unknown reactor imports no WASI.
( cd rust-describer && cargo component build --release --target wasm32-unknown-unknown )
cp rust-describer/target/wasm32-unknown-unknown/release/describer.wasm describer.wasm

echo "== 3. compose: plug the Lisp counter into the Rust describer =="
wac plug describer.wasm --plug counter.wasm -o vowels.wasm

echo "== 4. run (the host invokes the Rust export, which calls back into Lisp) =="
wasmtime run -W gc=y --invoke 'describe("hello world")' vowels.wasm


---

# FILE: references/examples/wit/rust-calls-lisp/counter.lisp

;;;; counter -- the Lisp component a Rust program calls.
;;;;
;;;; It exports one plain function, `vowel-count`, through the `counter` world of
;;;; wit/vowels.wit. `rontolisp:wit-export` implements PLAIN function exports (not
;;;; interfaces), so the world declares `export vowel-count: func(...)` at the top
;;;; level, and the compiler checks the defun below against it on every build.
;;;;
;;;; Built as a reactor component (no _start, no printing of its own):
;;;;
;;;;   rontolisp counter.lisp -o counter.wasm --component --optimize
;;;;
;;;; `wac` plugs it straight into the Rust describer. It is not run on its own --
;;;; see README.md / build.sh.

;;; A vowel is one of a e i o u, in either case. Spelled out with char= so it
;;; lowers to the smallest subset, with no dependence on sequence built-ins.
(defun vowelp (c)
  (let ((d (char-downcase c)))
    (or (char= d #\a) (char= d #\e) (char= d #\i) (char= d #\o) (char= d #\u))))

;;; Count the vowels in TEXT.
;;; WIT: vowel-count: func(text: string) -> s32
(defun vowel-count (text)
  (let ((n 0))
    (dotimes (i (length text)) (when (vowelp (char text i)) (incf n)))
    n))

(rontolisp:wit-export "wit/vowels.wit" :world counter)


---

# FILE: references/examples/wit/rust-calls-lisp/rust-describer/Cargo.toml

[package]
name = "describer"
version = "0.1.0"
edition = "2024"
publish = false

[dependencies]
wit-bindgen-rt = { version = "0.44.0", features = ["bitflags"] }

[lib]
crate-type = ["cdylib"]

[package.metadata.component]
package = "example:describer"

# Target the shared world instead of a crate-local wit/: `../wit` is the same
# vowels.wit the Lisp counter implements, so the two sides cannot drift.
[package.metadata.component.target]
path = "../wit"
world = "describer"

[package.metadata.component.dependencies]

[profile.release]
opt-level = "s"
strip = true


---

# FILE: references/examples/wit/rust-calls-lisp/rust-describer/src/lib.rs

//! The Rust describer. It EXPORTS `describe` and, to do its job, IMPORTS
//! `vowel-count` from a Lisp component -- so calling `describe` is Rust -> Lisp.
//!
//! Rust does the sentence building; Lisp does the counting. The host (or another
//! component) calls `describe`; nothing here knows the counter is written in
//! Lisp -- only the WIT does. `mod bindings` is generated by cargo-component from
//! ../wit (see Cargo.toml); `cargo component build` regenerates it.

#[allow(warnings)]
mod bindings;

use bindings::exports::example::vowels::describe::Guest;
use bindings::vowel_count;

struct Component;

impl Guest for Component {
    fn describe(text: String) -> String {
        // Rust -> Lisp: the imported `vowel-count` is an ordinary function call.
        let n = vowel_count(&text);
        let plural = if n == 1 { "" } else { "s" };
        format!("\"{text}\" has {n} vowel{plural}")
    }
}

bindings::export!(Component with_types_in bindings);


---

# FILE: references/examples/wit/rust-calls-lisp/wit/vowels.wit

package example:vowels;

/// A one-line description of a piece of text. Implemented in Rust, but the vowel
/// count in it comes from a Lisp component.
interface describe {
  describe: func(text: string) -> string;
}

/// The Rust describer's world. It EXPORTS `describe` (the function a host, or
/// another component, calls), and to do its job it IMPORTS `vowel-count` -- a
/// plain function a Lisp component supplies. This is the "Rust calls Lisp" edge.
world describer {
  export describe;
  import vowel-count: func(text: string) -> s32;
}

/// The Lisp counter's world. It EXPORTS `vowel-count` as a plain function --
/// exactly the shape the Rust world above imports. `rontolisp:wit-export`
/// implements plain function exports, so `vowel-count` is a bare world function.
world counter {
  export vowel-count: func(text: string) -> s32;
}


---

# FILE: references/examples/wit/world/README.md

# wit/world -- someone handed me a `.wit`, now what

A [WIT](https://component-model.bytecodealliance.org/design/wit.html) world is a
language-independent contract: the functions a component must export, their
types, and which of them are async. The realistic starting point is therefore
not a Lisp file but a `.wit` someone hands you. This directory is that workflow,
end to end:

| Step | Command | File |
|---|---|---|
| the contract you were handed | -- | [`wit/analyzer.wit`](wit/analyzer.wit) |
| generate a skeleton from it | `rontolisp --scaffold-wit wit/analyzer.wit -o analyzer.lisp` | [`analyzer.lisp`](analyzer.lisp) |
| fill in the bodies | your editor | the same file |
| build and call it | `rontolisp analyzer.lisp --component -o analyzer.wasm` | `analyzer.wasm` |
| check the two never drift | the compiler, on every build | -- |
| print the component's own world back out | `--emit-wit` | `analyzer.wit` |

Commands say `rontolisp`, the native binary; with the executable JAR it is
`java -jar ../../target/rontolisp-0.1.0-SNAPSHOT-exec.jar`.

## 1. The world you were handed

```wit
package example:analyzer;

world analyzer {
  /// Count the words in the text. ...
  export word-count: func(text: string) -> s32;
  export longest-word: func(text: string) -> string;
  export is-palindrome: func(text: string) -> bool;
  /// Declared async because it performs I/O: a synchronous export may not block.
  export print-report: async func(text: string);
}
```

Four exports over three boundary types plus one resultless `async func`. Nothing
here is rontolisp-specific, and rontolisp implements the world under its real
name.

`print-report` is the interesting one. A component's exports are lifted
**synchronously** by default and a synchronous task may not block, so `print`,
`read` or `rontolisp:fetch` inside one traps at run time with *"cannot block a
synchronous task"*. Being async is a property of the **contract**, not of the
implementation -- which is why the world states it and an implementer never has
to guess. See
[Component-model function exports](https://github.com/making/rontolisp/blob/develop/examples/wit/doc/en/guides/wasm-component.md#component-model-function-exports-wasm-export).

## 2. Scaffold the implementation

```bash
rontolisp --scaffold-wit wit/analyzer.wit -o analyzer.lisp
```

writes a runnable program implementing the world -- one stub per export:

```lisp
;;; Count the words in the text. A word is a run of characters separated by
;;; spaces; leading, trailing and repeated spaces do not count.
;;; WIT: word-count: func(text: string) -> s32
(defun word-count (text)
  (error "word-count is not implemented yet"))

;;; ... one per export ...

(rontolisp:wit-export "wit/analyzer.wit" :world analyzer)
```

Read what survived the trip. Every parameter is named the way the **WIT** names
it (`text`, not `p0`) -- those labels are part of the component's type, so the
host sees them. Every `///` doc comment came across as a `;;;` comment, and under
it a `;;; WIT:` line restates the signature the body must satisfy, types and
`async` included, because the Lisp itself would otherwise say nothing about the
contract. The `wit-export` directive comes **last**: on the interpreter it is an
ordinary form evaluated in order, so it can only check what is defined above it.

What is *not* there is a `rontolisp:wasm-export` with `:params`/`:returns`. The
types stay in the WIT; the directive says only *I implement that world*. There
is no second place for the signature to be written, so no second place for it to
drift.

Two things worth knowing before you touch the file:

- **It already compiles.** The stubs signal at *run* time, so the world can be
  filled in one export at a time and every intermediate state is a real
  component.
- Drop `-o` to print to stdout, and pass `--world NAME` when the `.wit` declares
  more than one.

Extra `defun`s are free: the world constrains only the functions it names.

## 3. Fill in the bodies

Replace each stub, leave the comments alone:

```lisp
;;; WIT: word-count: func(text: string) -> s32
(defun word-count (text)
  (length (split-words text)))
```

[`analyzer.lisp`](analyzer.lisp) is exactly the scaffold with its four bodies
written and two helpers added; the header, comments, `defun` headers and
directive are all still the generator's.

## 4. Build the component and call it

The component's exports are typed, so `wasmtime` calls them by name with
[WAVE](https://component-model.bytecodealliance.org/) values -- no linear-memory
pointer arithmetic, no `__ronto_alloc`:

```console
$ rontolisp analyzer.lisp --component -o analyzer.wasm
$ W="wasmtime run -W gc=y"

$ $W --invoke 'word-count("the quick brown fox jumps over the lazy dog")' analyzer.wasm
9
$ $W --invoke 'longest-word("the quick brown fox jumps over the lazy dog")' analyzer.wasm
"quick"
$ $W --invoke 'is-palindrome("A man, a plan, a canal: Panama")' analyzer.wasm
true
$ $W --invoke 'print-report("A man, a plan, a canal: Panama")' analyzer.wasm
text:         A man, a plan, a canal: Panama
words:        7
longest word: canal:
palindrome:   yes
()
```

Each result is printed in the WIT type the world declared, and `print-report`'s
absent result as `()` after the four lines it printed. Those lines are the async
lift working: the same `format` inside a *synchronous* export would have
trapped.

(`-W gc=y` because the full language compiles to a wasm-GC component.
[`count-vowels/`](https://github.com/making/rontolisp/blob/develop/examples/count-vowels) stays inside the
[non-GC subset](https://github.com/making/rontolisp/blob/develop/examples/wit/doc/en/guides/wasm-nogc.md#eligible-subset) and needs no
engine flags at all.)

## 5. Drift is a compile error, and it names the WIT line

The contract is checked on **every** build, not just when an export is called:

```console
$ rontolisp analyzer.lisp --component -o analyzer.wasm
wit/analyzer.wit:16: export 'is-palindrome' has no matching (defun is-palindrome ...) in the program

wit/analyzer.wit:12: export 'longest-word' declares 1 parameter(s), but (defun longest-word ...) takes 2
```

Name, arity, parameter types, result type, async-ness: all of it, all named
against the WIT line. The check runs on **every** backend, including the ones
that export nothing, so a plain `rontolisp analyzer.lisp` catches a drifted
world in a second without producing a component — which makes it a usable CI
check.

## 6. `--emit-wit`: the component's own world, printed back out

```bash
rontolisp analyzer.lisp --component -o analyzer.wasm --emit-wit   # also writes analyzer.wit
```

The four exports come back **exactly** as handed in -- names, the parameter name
`text`, types, and `print-report` still an `async func`. That is the round trip
worth having. The rest of the file is not the input, and differs in exactly two
ways:

1. **It is normalized to `package root:component; world root`.** A component's
   *type* has no package or world name of its own — that is what every tool
   prints. `example:analyzer` names the *contract*, not the artifact, and lives
   in `wit/analyzer.wit` only.
2. **It lists the WASI imports the build actually links**, which the
   hand-written world never stated: `wasi:cli/stdout` (what `print-report`'s
   `format` compiles to), `wasi:filesystem`, `wasi:clocks`, `wasi:random`, plus
   the `wasi:cli/run` export every rontolisp component carries. The author wrote
   a contract about *what the component does*; the emitted world additionally
   describes *what it needs from its host*, which is a fact about the build.

One thing cannot survive at all: the `///` doc comments. A component's type does
not store them — which is also why `--scaffold-wit` reads the `.wit` *text*
rather than introspecting a `.wasm`.

So `--emit-wit` is a consistency check of the export **surface** and a way to
hand a host the imports it must satisfy. It is not a copy of the input, and the
input remains the source of truth.

## What the boundary carries today

| WIT type | Lisp value |
|---|---|
| `s32` | an integer |
| `s64` | an integer (a `u64` value of 2^63 or more traps at the boundary) |
| `f64` | a float |
| `bool` | `t` / `nil` |
| `string` | a string |
| no result | the function's value is discarded |

Every other WIT type (`record`, `list`, `option`, `result`, resources, ...) is a
clear compile error at the export boundary today, naming the representation it
is settled to have once marshalling lands. A world's `import` items are ignored
— a component's WASI imports come from the adapter surface it is built on. Full
reference:
[`rontolisp:wit-export`](https://github.com/making/rontolisp/blob/develop/examples/wit/doc/en/reference/functions/rontolisp-wit-export.md),
[Implementing a WIT World](https://github.com/making/rontolisp/blob/develop/examples/wit/doc/en/guides/wit-contracts.md#implementing-a-wit-world-wit-export).


---

# FILE: references/examples/wit/world/analyzer.lisp

;;;; Implementation of the WIT world 'analyzer' (wit/analyzer.wit).
;;;;
;;;; The world is the contract: the compiler checks every defun below against
;;;; it, so a renamed export, a changed arity or a changed type is a compile
;;;; error rather than a runtime surprise. Fill in the bodies; each one signals
;;;; until you do.
;;;;
;;;; This header, every ;;; doc comment and ";;; WIT:" signature line below, the
;;;; four defun headers and the directive at the bottom were all produced from
;;;; the world alone by
;;;;
;;;;   rontolisp --scaffold-wit wit/analyzer.wit -o analyzer.lisp
;;;;
;;;; The hand-written parts are the helpers and the four bodies, which replaced
;;;; the generated (error "... is not implemented yet") stubs. README.md walks
;;;; through the whole workflow.

;;; ---------------------------------------------------------------------------
;;; Helpers -- ordinary Lisp. The world constrains only the functions it names;
;;; the exports may call anything else in the file.
;;; ---------------------------------------------------------------------------

;;; The words of TEXT: the runs of characters between spaces. Leading, trailing
;;; and repeated spaces produce no empty words.
(defun split-words (text)
  (let ((words '()) (current ""))
    (dotimes (i (length text))
      (let ((c (char text i)))
        (if (char= c #\Space)
            (progn
              (when (> (length current) 0) (setq words (cons current words)))
              (setq current ""))
            (setq current (concatenate 'string current (string c))))))
    (when (> (length current) 0) (setq words (cons current words)))
    (reverse words)))

;;; The letters and digits of TEXT, folded to lower case: what a palindrome test
;;; actually compares.
(defun letters-and-digits (text)
  (let ((out ""))
    (dotimes (i (length text))
      (let ((c (char text i)))
        (when (or (alpha-char-p c) (digit-char-p c))
          (setq out (concatenate 'string out (string (char-downcase c)))))))
    out))

;;; ---------------------------------------------------------------------------
;;; The four exports the world declares.
;;; ---------------------------------------------------------------------------

;;; Count the words in the text. A word is a run of characters separated by
;;; spaces; leading, trailing and repeated spaces do not count.
;;; WIT: word-count: func(text: string) -> s32
(defun word-count (text) (length (split-words text)))

;;; Return the longest word in the text, or the empty string when it has none.
;;; WIT: longest-word: func(text: string) -> string
(defun longest-word (text)
  (let ((best ""))
    (dolist (word (split-words text))
      (when (> (length word) (length best)) (setq best word)))
    best))

;;; Report whether the text reads the same backwards, ignoring letter case and
;;; every character that is not a letter or a digit.
;;; WIT: is-palindrome: func(text: string) -> bool
(defun is-palindrome (text)
  (let ((s (letters-and-digits text))) (string= s (reverse s))))

;;; Print a human-readable report about the text to standard output. It is
;;; declared async because it performs I/O: a synchronous export may not block,
;;; so this is the one property the world has to state rather than let an
;;; implementer guess.
;;; WIT: print-report: async func(text: string)
(defun print-report (text)
  (format t "text:         ~a~%" text)
  (format t "words:        ~a~%" (word-count text))
  (format t "longest word: ~a~%" (longest-word text))
  (format t "palindrome:   ~a~%" (if (is-palindrome text) "yes" "no")))

(rontolisp:wit-export "wit/analyzer.wit" :world analyzer)


---

# FILE: references/examples/wit/world/wit/analyzer.wit

package example:analyzer;

/// A small text-analysis component. This file is the contract: it names the
/// functions a component must export, their parameter and result types, and
/// which of them are async. Any language with a WIT toolchain can implement it.
world analyzer {
  /// Count the words in the text. A word is a run of characters separated by
  /// spaces; leading, trailing and repeated spaces do not count.
  export word-count: func(text: string) -> s32;

  /// Return the longest word in the text, or the empty string when it has none.
  export longest-word: func(text: string) -> string;

  /// Report whether the text reads the same backwards, ignoring letter case and
  /// every character that is not a letter or a digit.
  export is-palindrome: func(text: string) -> bool;

  /// Print a human-readable report about the text to standard output. It is
  /// declared async because it performs I/O: a synchronous export may not block,
  /// so this is the one property the world has to state rather than let an
  /// implementer guess.
  export print-report: async func(text: string);
}


---

# FILE: references/examples.md

# Examples

Whole working programs from the repository, mirrored here as they are --
minus the build outputs, so a compiled `.wasm` or a `.bin` of weights is a
link to the repository rather than a file. Read the one closest in shape to
what you are about to write: it shows the imports, the entry point and the
build command that a reference page cannot.

Paths are relative to this file.

## asdf

- `examples/asdf/` -- Loading real ASDF libraries -- [README.md](examples/asdf/README.md), [alexandria-demo.lisp](examples/asdf/alexandria-demo.lisp), [assoc-utils-demo.lisp](examples/asdf/assoc-utils-demo.lisp), [chipz-demo.lisp](examples/asdf/chipz-demo.lisp), [cl-base64-demo.lisp](examples/asdf/cl-base64-demo.lisp), [cl-ppcre-demo.lisp](examples/asdf/cl-ppcre-demo.lisp), [cl-utilities-demo.lisp](examples/asdf/cl-utilities-demo.lisp), [cl-who-demo.lisp](examples/asdf/cl-who-demo.lisp), [clack-hello.lisp](examples/asdf/clack-hello.lisp), [ironclad-demo.lisp](examples/asdf/ironclad-demo.lisp) (+9 more in the directory)

## browser

- `examples/browser/hiragana/` -- 手書きひらがな認識 (rontolisp で CNN を学習 → WASM → ブラウザ canvas) -- [README.md](examples/browser/hiragana/README.md), [dataset.lisp](examples/browser/hiragana/dataset.lisp), [infer.lisp](examples/browser/hiragana/infer.lisp), [net.lisp](examples/browser/hiragana/net.lisp), [prototypes.lisp](examples/browser/hiragana/prototypes.lisp), [recognize.lisp](examples/browser/hiragana/recognize.lisp), [train.lisp](examples/browser/hiragana/train.lisp), [gen.sh](examples/browser/hiragana/gen.sh), [regen-glyphs.sh](examples/browser/hiragana/regen-glyphs.sh), [glyphs.js](examples/browser/hiragana/glyphs.js) (+2 more in the directory)
- `examples/browser/hiragana/glyphgen/` -- [GlyphGen.java](examples/browser/hiragana/glyphgen/GlyphGen.java)
- `examples/browser/hiragana/samples/` -- [a.txt](examples/browser/hiragana/samples/a.txt), [chi.txt](examples/browser/hiragana/samples/chi.txt), [e.txt](examples/browser/hiragana/samples/e.txt), [fu.txt](examples/browser/hiragana/samples/fu.txt), [ha.txt](examples/browser/hiragana/samples/ha.txt), [he.txt](examples/browser/hiragana/samples/he.txt), [hi.txt](examples/browser/hiragana/samples/hi.txt), [ho.txt](examples/browser/hiragana/samples/ho.txt), [i.txt](examples/browser/hiragana/samples/i.txt), [ka.txt](examples/browser/hiragana/samples/ka.txt) (+36 more in the directory)
- `examples/browser/hiragana/tools/k49/` -- 実データ (Kuzushiji-49) の取得と前処理 -- [README.md](examples/browser/hiragana/tools/k49/README.md), [prepare-k49.py](examples/browser/hiragana/tools/k49/prepare-k49.py)
- `examples/browser/minesweeper/` -- Minesweeper (one Lisp rulebook, two front-ends) -- [README.md](examples/browser/minesweeper/README.md), [minesweeper-core-test.lisp](examples/browser/minesweeper/minesweeper-core-test.lisp), [minesweeper-core.lisp](examples/browser/minesweeper/minesweeper-core.lisp), [minesweeper-swing.lisp](examples/browser/minesweeper/minesweeper-swing.lisp), [minesweeper-wasm.lisp](examples/browser/minesweeper/minesweeper-wasm.lisp), [build.sh](examples/browser/minesweeper/build.sh), [minesweeper.html](examples/browser/minesweeper/minesweeper.html)
- `examples/browser/rainbow/` -- [rainbow.lisp](examples/browser/rainbow/rainbow.lisp), [rainbow.html](examples/browser/rainbow/rainbow.html)
- `examples/browser/wasm-browser/` -- Running rontolisp WASM in the browser (plain HTML + JavaScript) -- [README.md](examples/browser/wasm-browser/README.md), [dice.lisp](examples/browser/wasm-browser/dice.lisp), [greet.lisp](examples/browser/wasm-browser/greet.lisp), [hello.lisp](examples/browser/wasm-browser/hello.lisp), [build.sh](examples/browser/wasm-browser/build.sh), [index.html](examples/browser/wasm-browser/index.html), [wasi-shim.js](examples/browser/wasm-browser/wasi-shim.js)
- `examples/browser/webgl-battlefront/` -- webgl-battlefront — a one-arena snow-battle skirmish, in Lisp -- [README.md](examples/browser/webgl-battlefront/README.md), [battlefront.lisp](examples/browser/webgl-battlefront/battlefront.lisp), [build.sh](examples/browser/webgl-battlefront/build.sh), [index.html](examples/browser/webgl-battlefront/index.html)
- `examples/browser/webgl-common/` -- webgl-common — the shared `gl` package for the WebGL demos -- [README.md](examples/browser/webgl-common/README.md), [gl.lisp](examples/browser/webgl-common/gl.lisp), [gl-imports.js](examples/browser/webgl-common/gl-imports.js), [gl.wit](examples/browser/webgl-common/gl.wit)
- `examples/browser/webgl-cube/` -- cube.lisp — hello 3D: a rotating cube, matrices and all, driven from Lisp -- [README.md](examples/browser/webgl-cube/README.md), [cube.lisp](examples/browser/webgl-cube/cube.lisp), [build.sh](examples/browser/webgl-cube/build.sh), [index.html](examples/browser/webgl-cube/index.html)
- `examples/browser/webgl-galaxy/` -- galaxy.lisp — a spiral galaxy: the WebGL pipeline driven from Lisp -- [README.md](examples/browser/webgl-galaxy/README.md), [galaxy.lisp](examples/browser/webgl-galaxy/galaxy.lisp), [build.sh](examples/browser/webgl-galaxy/build.sh), [index.html](examples/browser/webgl-galaxy/index.html)
- `examples/browser/webgl-heat3d/` -- heat3d.lisp — a rank-3 array diffusing heat, drawn by WebGL -- [README.md](examples/browser/webgl-heat3d/README.md), [heat3d.lisp](examples/browser/webgl-heat3d/heat3d.lisp), [build.sh](examples/browser/webgl-heat3d/build.sh), [index.html](examples/browser/webgl-heat3d/index.html)
- `examples/browser/webgl-platformer/` -- webgl-platformer — a one-stage 3D platformer, in Lisp -- [README.md](examples/browser/webgl-platformer/README.md), [platformer.lisp](examples/browser/webgl-platformer/platformer.lisp), [build.sh](examples/browser/webgl-platformer/build.sh), [index.html](examples/browser/webgl-platformer/index.html)
- `examples/browser/webgl-robot-arm/` -- robot-arm.lisp — 3D inverse kinematics with minimum-jerk motion, in Lisp -- [README.md](examples/browser/webgl-robot-arm/README.md), [ik-analytic.lisp](examples/browser/webgl-robot-arm/ik-analytic.lisp), [ik-fabrik.lisp](examples/browser/webgl-robot-arm/ik-fabrik.lisp), [ik-jacobian.lisp](examples/browser/webgl-robot-arm/ik-jacobian.lisp), [robot-arm.lisp](examples/browser/webgl-robot-arm/robot-arm.lisp), [build.sh](examples/browser/webgl-robot-arm/build.sh), [index.html](examples/browser/webgl-robot-arm/index.html)
- `examples/browser/webgl-triangle/` -- triangle.lisp — the WebGL hello world, driven from Lisp -- [README.md](examples/browser/webgl-triangle/README.md), [triangle.lisp](examples/browser/webgl-triangle/triangle.lisp), [build.sh](examples/browser/webgl-triangle/build.sh), [index.html](examples/browser/webgl-triangle/index.html)
- `examples/browser/wit-component/` -- A WebAssembly component in the browser -- [README.md](examples/browser/wit-component/README.md), [fractal.lisp](examples/browser/wit-component/fractal.lisp), [build.sh](examples/browser/wit-component/build.sh), [index.html](examples/browser/wit-component/index.html)
- `examples/browser/wit-component/wit/` -- [fractal.wit](examples/browser/wit-component/wit/fractal.wit)

## cloudflare-workers

- `examples/cloudflare-workers/` -- rontolisp on Cloudflare Workers -- [README.md](examples/cloudflare-workers/README.md)
- `examples/cloudflare-workers/btc-ticker/` -- btc-ticker — the Worker with nothing in it -- [README.md](examples/cloudflare-workers/btc-ticker/README.md), [worker.lisp](examples/cloudflare-workers/btc-ticker/worker.lisp), [build.sh](examples/cloudflare-workers/btc-ticker/build.sh), [package.json](examples/cloudflare-workers/btc-ticker/package.json), [wrangler.jsonc](examples/cloudflare-workers/btc-ticker/wrangler.jsonc)
- `examples/cloudflare-workers/btc-ticker/src/` -- [index.js](examples/cloudflare-workers/btc-ticker/src/index.js), [worker.js](examples/cloudflare-workers/btc-ticker/src/worker.js)
- `examples/cloudflare-workers/dog-fetcher/` -- dog-fetcher — a Worker that calls out -- [README.md](examples/cloudflare-workers/dog-fetcher/README.md), [worker.lisp](examples/cloudflare-workers/dog-fetcher/worker.lisp), [build.sh](examples/cloudflare-workers/dog-fetcher/build.sh), [package.json](examples/cloudflare-workers/dog-fetcher/package.json), [wrangler.jsonc](examples/cloudflare-workers/dog-fetcher/wrangler.jsonc)
- `examples/cloudflare-workers/dog-fetcher/src/` -- [index.js](examples/cloudflare-workers/dog-fetcher/src/index.js), [worker.js](examples/cloudflare-workers/dog-fetcher/src/worker.js)
- `examples/cloudflare-workers/dog-relay/` -- dog-relay — a Worker that relays, many at a time -- [README.md](examples/cloudflare-workers/dog-relay/README.md), [worker.lisp](examples/cloudflare-workers/dog-relay/worker.lisp), [build.sh](examples/cloudflare-workers/dog-relay/build.sh), [package.json](examples/cloudflare-workers/dog-relay/package.json), [wrangler.jsonc](examples/cloudflare-workers/dog-relay/wrangler.jsonc)
- `examples/cloudflare-workers/dog-relay/src/` -- [index.js](examples/cloudflare-workers/dog-relay/src/index.js), [worker.js](examples/cloudflare-workers/dog-relay/src/worker.js)
- `examples/cloudflare-workers/hello/` -- hello — the smallest rontolisp Worker -- [README.md](examples/cloudflare-workers/hello/README.md), [worker.lisp](examples/cloudflare-workers/hello/worker.lisp), [build.sh](examples/cloudflare-workers/hello/build.sh), [package.json](examples/cloudflare-workers/hello/package.json), [wrangler.jsonc](examples/cloudflare-workers/hello/wrangler.jsonc)
- `examples/cloudflare-workers/hello-clack/` -- hello-clack — a Clack application on Cloudflare Workers -- [README.md](examples/cloudflare-workers/hello-clack/README.md), [check.lisp](examples/cloudflare-workers/hello-clack/check.lisp), [worker.lisp](examples/cloudflare-workers/hello-clack/worker.lisp), [build.sh](examples/cloudflare-workers/hello-clack/build.sh), [package.json](examples/cloudflare-workers/hello-clack/package.json), [wrangler.jsonc](examples/cloudflare-workers/hello-clack/wrangler.jsonc)
- `examples/cloudflare-workers/hello-clack/src/` -- [index.js](examples/cloudflare-workers/hello-clack/src/index.js), [worker.js](examples/cloudflare-workers/hello-clack/src/worker.js)
- `examples/cloudflare-workers/hello-ningle/` -- hello-ningle — a Worker whose application is an object -- [README.md](examples/cloudflare-workers/hello-ningle/README.md), [check.lisp](examples/cloudflare-workers/hello-ningle/check.lisp), [worker.lisp](examples/cloudflare-workers/hello-ningle/worker.lisp), [build.sh](examples/cloudflare-workers/hello-ningle/build.sh), [package.json](examples/cloudflare-workers/hello-ningle/package.json), [wrangler.jsonc](examples/cloudflare-workers/hello-ningle/wrangler.jsonc)
- `examples/cloudflare-workers/hello-ningle/src/` -- [index.js](examples/cloudflare-workers/hello-ningle/src/index.js), [worker.js](examples/cloudflare-workers/hello-ningle/src/worker.js)
- `examples/cloudflare-workers/hello-tiny-routes/` -- hello-tiny-routes — a Worker composed out of routes -- [README.md](examples/cloudflare-workers/hello-tiny-routes/README.md), [check.lisp](examples/cloudflare-workers/hello-tiny-routes/check.lisp), [worker.lisp](examples/cloudflare-workers/hello-tiny-routes/worker.lisp), [build.sh](examples/cloudflare-workers/hello-tiny-routes/build.sh), [package.json](examples/cloudflare-workers/hello-tiny-routes/package.json), [wrangler.jsonc](examples/cloudflare-workers/hello-tiny-routes/wrangler.jsonc)
- `examples/cloudflare-workers/hello-tiny-routes/src/` -- [index.js](examples/cloudflare-workers/hello-tiny-routes/src/index.js), [worker.js](examples/cloudflare-workers/hello-tiny-routes/src/worker.js)
- `examples/cloudflare-workers/hello/src/` -- [index.js](examples/cloudflare-workers/hello/src/index.js)
- `examples/cloudflare-workers/httpbin/` -- httpbin — a mini httpbin on Cloudflare Workers, with no library -- [README.md](examples/cloudflare-workers/httpbin/README.md), [check.lisp](examples/cloudflare-workers/httpbin/check.lisp), [worker.lisp](examples/cloudflare-workers/httpbin/worker.lisp), [build.sh](examples/cloudflare-workers/httpbin/build.sh), [package.json](examples/cloudflare-workers/httpbin/package.json), [wrangler.jsonc](examples/cloudflare-workers/httpbin/wrangler.jsonc)
- `examples/cloudflare-workers/httpbin-clack/` -- httpbin-clack — the same endpoints as a plain Clack application -- [README.md](examples/cloudflare-workers/httpbin-clack/README.md), [check.lisp](examples/cloudflare-workers/httpbin-clack/check.lisp), [worker.lisp](examples/cloudflare-workers/httpbin-clack/worker.lisp), [build.sh](examples/cloudflare-workers/httpbin-clack/build.sh), [package.json](examples/cloudflare-workers/httpbin-clack/package.json), [wrangler.jsonc](examples/cloudflare-workers/httpbin-clack/wrangler.jsonc)
- `examples/cloudflare-workers/httpbin-clack-one-source/` -- httpbin-clack-one-source — one Clack file, every host, this one included -- [README.md](examples/cloudflare-workers/httpbin-clack-one-source/README.md), [build.sh](examples/cloudflare-workers/httpbin-clack-one-source/build.sh), [package.json](examples/cloudflare-workers/httpbin-clack-one-source/package.json), [wrangler.jsonc](examples/cloudflare-workers/httpbin-clack-one-source/wrangler.jsonc)
- `examples/cloudflare-workers/httpbin-clack-one-source/src/` -- [index.js](examples/cloudflare-workers/httpbin-clack-one-source/src/index.js), [worker.js](examples/cloudflare-workers/httpbin-clack-one-source/src/worker.js)
- `examples/cloudflare-workers/httpbin-clack/src/` -- [index.js](examples/cloudflare-workers/httpbin-clack/src/index.js), [worker.js](examples/cloudflare-workers/httpbin-clack/src/worker.js)
- `examples/cloudflare-workers/httpbin-component/` -- httpbin-component — the same handler, through the component model -- [README.md](examples/cloudflare-workers/httpbin-component/README.md), [build.sh](examples/cloudflare-workers/httpbin-component/build.sh), [package.json](examples/cloudflare-workers/httpbin-component/package.json), [wrangler.jsonc](examples/cloudflare-workers/httpbin-component/wrangler.jsonc)
- `examples/cloudflare-workers/httpbin-component/src/` -- [index.js](examples/cloudflare-workers/httpbin-component/src/index.js)
- `examples/cloudflare-workers/httpbin-ningle/` -- httpbin-ningle — the same endpoints on an application object -- [README.md](examples/cloudflare-workers/httpbin-ningle/README.md), [check.lisp](examples/cloudflare-workers/httpbin-ningle/check.lisp), [worker.lisp](examples/cloudflare-workers/httpbin-ningle/worker.lisp), [build.sh](examples/cloudflare-workers/httpbin-ningle/build.sh), [package.json](examples/cloudflare-workers/httpbin-ningle/package.json), [wrangler.jsonc](examples/cloudflare-workers/httpbin-ningle/wrangler.jsonc)
- `examples/cloudflare-workers/httpbin-ningle/src/` -- [index.js](examples/cloudflare-workers/httpbin-ningle/src/index.js), [worker.js](examples/cloudflare-workers/httpbin-ningle/src/worker.js)
- `examples/cloudflare-workers/httpbin-tiny-routes/` -- httpbin-tiny-routes — the same endpoints, composed -- [README.md](examples/cloudflare-workers/httpbin-tiny-routes/README.md), [check.lisp](examples/cloudflare-workers/httpbin-tiny-routes/check.lisp), [worker.lisp](examples/cloudflare-workers/httpbin-tiny-routes/worker.lisp), [build.sh](examples/cloudflare-workers/httpbin-tiny-routes/build.sh), [package.json](examples/cloudflare-workers/httpbin-tiny-routes/package.json), [wrangler.jsonc](examples/cloudflare-workers/httpbin-tiny-routes/wrangler.jsonc)
- `examples/cloudflare-workers/httpbin-tiny-routes/src/` -- [index.js](examples/cloudflare-workers/httpbin-tiny-routes/src/index.js), [worker.js](examples/cloudflare-workers/httpbin-tiny-routes/src/worker.js)
- `examples/cloudflare-workers/httpbin/src/` -- [index.js](examples/cloudflare-workers/httpbin/src/index.js)

## console

- `examples/console/` -- [calc.lisp](examples/console/calc.lisp), [contact-book.lisp](examples/console/contact-book.lisp), [error-handling.lisp](examples/console/error-handling.lisp), [hanoi.lisp](examples/console/hanoi.lisp), [l-system.lisp](examples/console/l-system.lisp), [life-core.lisp](examples/console/life-core.lisp), [life.lisp](examples/console/life.lisp), [line-numbers.lisp](examples/console/line-numbers.lisp), [mandelbrot-nogc.lisp](examples/console/mandelbrot-nogc.lisp), [mandelbrot.lisp](examples/console/mandelbrot.lisp) (+7 more in the directory)

## count-vowels

- `examples/count-vowels/` -- count-vowels -- sharing a string with a host through Wasm memory -- [README.md](examples/count-vowels/README.md), [count-vowels.lisp](examples/count-vowels/count-vowels.lisp), [count_vowels_component.wit](examples/count-vowels/count_vowels_component.wit), [pom.xml](examples/count-vowels/pom.xml)
- `examples/count-vowels/src/main/java/` -- [CountVowels.java](examples/count-vowels/src/main/java/CountVowels.java)

## db

- `examples/db/` -- db -- [README.md](examples/db/README.md), [bbs-api.lisp](examples/db/bbs-api.lisp), [database-url.lisp](examples/db/database-url.lisp), [postgres-crud.lisp](examples/db/postgres-crud.lisp), [postgres-hello.lisp](examples/db/postgres-hello.lisp), [postgres-web.lisp](examples/db/postgres-web.lisp), [postmodern-crud.lisp](examples/db/postmodern-crud.lisp), [postmodern-dao.lisp](examples/db/postmodern-dao.lisp)
- `examples/db/postgres-web/` -- [spin.toml](examples/db/postgres-web/spin.toml)

## deep-learning-from-scratch

- `examples/deep-learning-from-scratch/` -- Deep Learning from Scratch, in rontolisp -- [LICENSE.md](examples/deep-learning-from-scratch/LICENSE.md), [README.md](examples/deep-learning-from-scratch/README.md), [download-mnist.sh](examples/deep-learning-from-scratch/download-mnist.sh)
- `examples/deep-learning-from-scratch/ch02/` -- [and-gate.lisp](examples/deep-learning-from-scratch/ch02/and-gate.lisp), [nand-gate.lisp](examples/deep-learning-from-scratch/ch02/nand-gate.lisp), [or-gate.lisp](examples/deep-learning-from-scratch/ch02/or-gate.lisp), [xor-gate.lisp](examples/deep-learning-from-scratch/ch02/xor-gate.lisp)
- `examples/deep-learning-from-scratch/ch03/` -- [activation-functions.lisp](examples/deep-learning-from-scratch/ch03/activation-functions.lisp), [mnist-show.lisp](examples/deep-learning-from-scratch/ch03/mnist-show.lisp), [neuralnet-mnist-batch.lisp](examples/deep-learning-from-scratch/ch03/neuralnet-mnist-batch.lisp), [neuralnet-mnist.lisp](examples/deep-learning-from-scratch/ch03/neuralnet-mnist.lisp)
- `examples/deep-learning-from-scratch/ch04/` -- [gradient-1d.lisp](examples/deep-learning-from-scratch/ch04/gradient-1d.lisp), [gradient-2d.lisp](examples/deep-learning-from-scratch/ch04/gradient-2d.lisp), [gradient-method.lisp](examples/deep-learning-from-scratch/ch04/gradient-method.lisp), [gradient-simplenet.lisp](examples/deep-learning-from-scratch/ch04/gradient-simplenet.lisp), [train-neuralnet.lisp](examples/deep-learning-from-scratch/ch04/train-neuralnet.lisp), [two-layer-net.lisp](examples/deep-learning-from-scratch/ch04/two-layer-net.lisp)
- `examples/deep-learning-from-scratch/ch05/` -- [buy-apple-orange.lisp](examples/deep-learning-from-scratch/ch05/buy-apple-orange.lisp), [buy-apple.lisp](examples/deep-learning-from-scratch/ch05/buy-apple.lisp), [gradient-check.lisp](examples/deep-learning-from-scratch/ch05/gradient-check.lisp), [layer-naive.lisp](examples/deep-learning-from-scratch/ch05/layer-naive.lisp), [train-neuralnet.lisp](examples/deep-learning-from-scratch/ch05/train-neuralnet.lisp), [two-layer-net.lisp](examples/deep-learning-from-scratch/ch05/two-layer-net.lisp)
- `examples/deep-learning-from-scratch/ch06/` -- [batch-norm-gradient-check.lisp](examples/deep-learning-from-scratch/ch06/batch-norm-gradient-check.lisp), [batch-norm-test.lisp](examples/deep-learning-from-scratch/ch06/batch-norm-test.lisp), [hyperparameter-optimization.lisp](examples/deep-learning-from-scratch/ch06/hyperparameter-optimization.lisp), [optimizer-compare-mnist.lisp](examples/deep-learning-from-scratch/ch06/optimizer-compare-mnist.lisp), [optimizer-compare-naive.lisp](examples/deep-learning-from-scratch/ch06/optimizer-compare-naive.lisp), [overfit-dropout.lisp](examples/deep-learning-from-scratch/ch06/overfit-dropout.lisp), [overfit-weight-decay.lisp](examples/deep-learning-from-scratch/ch06/overfit-weight-decay.lisp), [weight-init-activation-histogram.lisp](examples/deep-learning-from-scratch/ch06/weight-init-activation-histogram.lisp), [weight-init-compare.lisp](examples/deep-learning-from-scratch/ch06/weight-init-compare.lisp)
- `examples/deep-learning-from-scratch/ch07/` -- [gradient-check.lisp](examples/deep-learning-from-scratch/ch07/gradient-check.lisp), [simple-convnet.lisp](examples/deep-learning-from-scratch/ch07/simple-convnet.lisp), [train-convnet.lisp](examples/deep-learning-from-scratch/ch07/train-convnet.lisp), [visualize-filter.lisp](examples/deep-learning-from-scratch/ch07/visualize-filter.lisp)
- `examples/deep-learning-from-scratch/ch08/` -- [deep-convnet.lisp](examples/deep-learning-from-scratch/ch08/deep-convnet.lisp), [half-float-network.lisp](examples/deep-learning-from-scratch/ch08/half-float-network.lisp), [misclassified-mnist.lisp](examples/deep-learning-from-scratch/ch08/misclassified-mnist.lisp), [train-deepnet.lisp](examples/deep-learning-from-scratch/ch08/train-deepnet.lisp)
- `examples/deep-learning-from-scratch/common/` -- [functions.lisp](examples/deep-learning-from-scratch/common/functions.lisp), [gradient.lisp](examples/deep-learning-from-scratch/common/gradient.lisp), [layers.lisp](examples/deep-learning-from-scratch/common/layers.lisp), [multi-layer-net-extend.lisp](examples/deep-learning-from-scratch/common/multi-layer-net-extend.lisp), [multi-layer-net.lisp](examples/deep-learning-from-scratch/common/multi-layer-net.lisp), [optimizer.lisp](examples/deep-learning-from-scratch/common/optimizer.lisp), [trainer.lisp](examples/deep-learning-from-scratch/common/trainer.lisp), [util.lisp](examples/deep-learning-from-scratch/common/util.lisp)
- `examples/deep-learning-from-scratch/dataset/` -- [mnist.lisp](examples/deep-learning-from-scratch/dataset/mnist.lisp), [rlw1.lisp](examples/deep-learning-from-scratch/dataset/rlw1.lisp)
- `examples/deep-learning-from-scratch/tools/` -- [export-sample-weight.py](examples/deep-learning-from-scratch/tools/export-sample-weight.py)

## jvm

- `examples/jvm/` -- [java-interop.lisp](examples/jvm/java-interop.lisp), [life-gui.lisp](examples/jvm/life-gui.lisp), [swing.lisp](examples/jvm/swing.lisp)

## llama2

- `examples/llama2/` -- llama2.c in rontolisp -- [README.md](examples/llama2/README.md), [llama2.lisp](examples/llama2/llama2.lisp), [download-stories15M.sh](examples/llama2/download-stories15M.sh)

## llm-from-scratch

- `examples/llm-from-scratch/` -- LLM from Scratch — the Transformer and GPT chapters, in rontolisp -- [README.md](examples/llm-from-scratch/README.md)
- `examples/llm-from-scratch/chapter02/` -- [section2.lisp](examples/llm-from-scratch/chapter02/section2.lisp), [section3.lisp](examples/llm-from-scratch/chapter02/section3.lisp), [section4.lisp](examples/llm-from-scratch/chapter02/section4.lisp), [section5.lisp](examples/llm-from-scratch/chapter02/section5.lisp)
- `examples/llm-from-scratch/chapter03/` -- [section2.lisp](examples/llm-from-scratch/chapter03/section2.lisp), [train-gpt-soseki.lisp](examples/llm-from-scratch/chapter03/train-gpt-soseki.lisp)
- `examples/llm-from-scratch/gpt/` -- [dataset.lisp](examples/llm-from-scratch/gpt/dataset.lisp), [model.lisp](examples/llm-from-scratch/gpt/model.lisp), [shapes.lisp](examples/llm-from-scratch/gpt/shapes.lisp), [tokenizer.lisp](examples/llm-from-scratch/gpt/tokenizer.lisp), [trainer.lisp](examples/llm-from-scratch/gpt/trainer.lisp)
- `examples/llm-from-scratch/transformer/` -- [attention.lisp](examples/llm-from-scratch/transformer/attention.lisp), [shapes.lisp](examples/llm-from-scratch/transformer/shapes.lisp), [transformer.lisp](examples/llm-from-scratch/transformer/transformer.lisp), [utils.lisp](examples/llm-from-scratch/transformer/utils.lisp)

## ml

- `examples/ml/` -- [deep-digits.lisp](examples/ml/deep-digits.lisp), [heat3d.lisp](examples/ml/heat3d.lisp), [linear-regression.lisp](examples/ml/linear-regression.lisp), [maze-rl.lisp](examples/ml/maze-rl.lisp), [mlp.lisp](examples/ml/mlp.lisp), [nn-vec.lisp](examples/ml/nn-vec.lisp), [nn.lisp](examples/ml/nn.lisp), [numerical-calculus.lisp](examples/ml/numerical-calculus.lisp), [simd-dot-jvm.lisp](examples/ml/simd-dot-jvm.lisp), [simd-dot.lisp](examples/ml/simd-dot.lisp) (+3 more in the directory)

## net

- `examples/net/` -- [dog-fetcher.lisp](examples/net/dog-fetcher.lisp), [echo-client.lisp](examples/net/echo-client.lisp), [echo-server.lisp](examples/net/echo-server.lisp), [http-handler-cl-who.lisp](examples/net/http-handler-cl-who.lisp), [http-handler.lisp](examples/net/http-handler.lisp), [http-hello.lisp](examples/net/http-hello.lisp), [httpbin-clack.lisp](examples/net/httpbin-clack.lisp), [httpbin-clos.lisp](examples/net/httpbin-clos.lisp), [httpbin-jzon.lisp](examples/net/httpbin-jzon.lisp), [httpbin-ningle.lisp](examples/net/httpbin-ningle.lisp) (+7 more in the directory)
- `examples/net/http-handler/` -- [spin.toml](examples/net/http-handler/spin.toml)
- `examples/net/httpbin-clos/` -- [spin.toml](examples/net/httpbin-clos/spin.toml)

## wasmcloud

- `examples/wasmcloud/` -- wasmCloud template ports -- [README.md](examples/wasmcloud/README.md)
- `examples/wasmcloud/http-client/` -- [app.lisp](examples/wasmcloud/http-client/app.lisp)
- `examples/wasmcloud/http-handler/` -- [app.lisp](examples/wasmcloud/http-handler/app.lisp)
- `examples/wasmcloud/http-hello-world/` -- [app.lisp](examples/wasmcloud/http-hello-world/app.lisp)
- `examples/wasmcloud/http-kv-handler/` -- [app.lisp](examples/wasmcloud/http-kv-handler/app.lisp)
- `examples/wasmcloud/service-tcp/` -- [http-api.lisp](examples/wasmcloud/service-tcp/http-api.lisp), [service-leet.lisp](examples/wasmcloud/service-tcp/service-leet.lisp)

## wit

- `examples/wit/keyvalue/` -- wit/keyvalue -- one WIT interface, three stores behind it -- [README.md](examples/wit/keyvalue/README.md), [java-store.lisp](examples/wit/keyvalue/java-store.lisp), [memory-store.lisp](examples/wit/keyvalue/memory-store.lisp), [page-hits-server.lisp](examples/wit/keyvalue/page-hits-server.lisp), [page-hits.lisp](examples/wit/keyvalue/page-hits.lisp)
- `examples/wit/keyvalue/wit/` -- [keyvalue.wit](examples/wit/keyvalue/wit/keyvalue.wit)
- `examples/wit/lisp-calls-rust/` -- wit/lisp-calls-rust -- a Lisp program calling a Rust component -- [README.md](examples/wit/lisp-calls-rust/README.md), [app.lisp](examples/wit/lisp-calls-rust/app.lisp), [build.sh](examples/wit/lisp-calls-rust/build.sh)
- `examples/wit/lisp-calls-rust/rust-shouter/` -- [Cargo.toml](examples/wit/lisp-calls-rust/rust-shouter/Cargo.toml)
- `examples/wit/lisp-calls-rust/rust-shouter/src/` -- [lib.rs](examples/wit/lisp-calls-rust/rust-shouter/src/lib.rs)
- `examples/wit/lisp-calls-rust/wit/` -- [textkit.wit](examples/wit/lisp-calls-rust/wit/textkit.wit)
- `examples/wit/pipeline/` -- wit/pipeline -- three components in one `wac compose` -- [README.md](examples/wit/pipeline/README.md), [app.lisp](examples/wit/pipeline/app.lisp), [stats.lisp](examples/wit/pipeline/stats.lisp), [build.sh](examples/wit/pipeline/build.sh), [composition.wac](examples/wit/pipeline/composition.wac)
- `examples/wit/pipeline/rust-shouter/` -- [Cargo.toml](examples/wit/pipeline/rust-shouter/Cargo.toml)
- `examples/wit/pipeline/rust-shouter/src/` -- [lib.rs](examples/wit/pipeline/rust-shouter/src/lib.rs)
- `examples/wit/pipeline/wit/` -- [pipeline.wit](examples/wit/pipeline/wit/pipeline.wit)
- `examples/wit/rust-calls-lisp/` -- wit/rust-calls-lisp -- a Rust program calling a Lisp component -- [README.md](examples/wit/rust-calls-lisp/README.md), [counter.lisp](examples/wit/rust-calls-lisp/counter.lisp), [build.sh](examples/wit/rust-calls-lisp/build.sh)
- `examples/wit/rust-calls-lisp/rust-describer/` -- [Cargo.toml](examples/wit/rust-calls-lisp/rust-describer/Cargo.toml)
- `examples/wit/rust-calls-lisp/rust-describer/src/` -- [lib.rs](examples/wit/rust-calls-lisp/rust-describer/src/lib.rs)
- `examples/wit/rust-calls-lisp/wit/` -- [vowels.wit](examples/wit/rust-calls-lisp/wit/vowels.wit)
- `examples/wit/world/` -- wit/world -- someone handed me a `.wit`, now what -- [README.md](examples/wit/world/README.md), [analyzer.lisp](examples/wit/world/analyzer.lisp)
- `examples/wit/world/wit/` -- [analyzer.wit](examples/wit/world/wit/analyzer.wit)


---

# FILE: references/contents.md

# Documentation index

Every page bundled with this skill, in the order the documentation site
presents it. Paths are relative to this file.

## Getting Started

- [Introduction](index.md)
- [Build & Install](getting-started/build.md)
- [REPL](getting-started/repl.md)
- [File Interpretation](getting-started/file-interpretation.md)
- [Formatting Source Code](getting-started/format.md)
- [Agent Skill](getting-started/agent-skill.md)

## Compiling

- [Compile to JVM Bytecode](compiling/jvm.md)
- [Compile to WASM](compiling/wasm.md)
- [Dynamic (Late Binding)](compiling/dynamic.md)
- [Self-Hosted REPL](compiling/self-hosted-repl.md)

## Language Reference

- [Data Types](reference/data-types.md)
- [Special Forms](reference/special-forms.md)
- [Macros](reference/macros.md)
- [Functions](reference/functions.md)
- [Packages](reference/packages.md)
- [The uiop Package](reference/uiop.md)
- [Function Namespace](reference/function-namespace.md)

## Guides

- [Vectors & Matrices (linalg)](guides/linear-algebra.md)
- [Vector Kernels & SIMD (vec, linalg)](guides/simd-acceleration.md)
- [Neural Networks (torch)](guides/neural-networks.md)
- [Java Interop](guides/java-interop.md)
- [Asynchronous Programming (async / await)](guides/async.md)
- [HTTP Requests (fetch)](guides/http-fetch.md)
- [Serving HTTP (http-handler)](guides/http-handler.md)
- [TCP Sockets](guides/tcp-sockets.md)
- [Gray Streams](guides/gray-streams.md)
- [Systems (asdf)](guides/asdf-systems.md)
- [Clack Web Applications](guides/clack.md)
- [O/R Mapping (mito, sxql)](guides/mito.md)
- [Testing (rove)](guides/testing.md)
- [Reader Case (Upcasing)](guides/reader-case.md)
- [Math Function Backends](guides/math-backends.md)
- [The Clock and Randomness](guides/clock-and-random.md)
- [WASM Host Boundary (wasm-export / wasm-import)](guides/wasm-host-boundary.md)
- [WIT Contracts (wit-export / wit-import)](guides/wit-contracts.md)
- [wasm-GC Core Module (Default Output)](guides/wasm-gc-module.md)
- [WASI 0.3 Component (--component)](guides/wasm-component.md)
- [WASM Non-GC Output (--no-gc)](guides/wasm-nogc.md)
- [Running WASM in a Browser](guides/wasm-browser.md)
- [Unsupported CL Features](guides/missing-features.md)
- [Compiled eval Limitations](guides/eval-limitations.md)
- [Compiled read/load Limitations](guides/read-load-limitations.md)

## Per-operator pages

- Functions: 668 pages under `reference/functions/` -- listed in [operators.md](operators.md)
- Macros: 105 pages under `reference/macros/` -- listed in [operators.md](operators.md)
- Special forms: 29 pages under `reference/special-forms/` -- listed in [operators.md](operators.md)


---

# FILE: references/operators.md

# Operator index

Every operator rontolisp ships. **A name that is not listed here does not
exist** -- rontolisp is a subset, so a Common Lisp operator missing from this
page is missing from the language, not from the page.

Each entry links to its detail page, which carries the signature, a runnable
example and that operator's own deviations from Common Lisp. Paths are
relative to this file.

## Functions (668)

`reference/functions/.md`

### Arithmetic

[`+`](reference/functions/plus.md), [`-`](reference/functions/minus.md), [`*`](reference/functions/mul.md), [`/`](reference/functions/div.md), [`mod`](reference/functions/mod.md), [`rem`](reference/functions/rem.md), [`1+`](reference/functions/1plus.md), [`1-`](reference/functions/1minus.md), [`abs`](reference/functions/abs.md), [`min`](reference/functions/min.md), [`max`](reference/functions/max.md), [`gcd`](reference/functions/gcd.md), [`lcm`](reference/functions/lcm.md), [`signum`](reference/functions/signum.md)

### Comparison & Number Predicates

[`=`](reference/functions/numeq.md), [`/=`](reference/functions/ne.md), [`<`](reference/functions/lt.md), [`>`](reference/functions/gt.md), [`<=`](reference/functions/le.md), [`>=`](reference/functions/ge.md), [`zerop`](reference/functions/zerop.md), [`plusp`](reference/functions/plusp.md), [`minusp`](reference/functions/minusp.md), [`evenp`](reference/functions/evenp.md), [`oddp`](reference/functions/oddp.md)

### Number Conversion & Rounding

[`float`](reference/functions/float.md), [`truncate`](reference/functions/truncate.md), [`floor`](reference/functions/floor.md), [`ceiling`](reference/functions/ceiling.md), [`round`](reference/functions/round.md), [`numerator`](reference/functions/numerator.md), [`denominator`](reference/functions/denominator.md)

### Math Functions

[`sqrt`](reference/functions/sqrt.md), [`isqrt`](reference/functions/isqrt.md), [`expt`](reference/functions/expt.md), [`exp`](reference/functions/exp.md), [`log`](reference/functions/log.md), [`sin cos tan`](reference/functions/sin-cos-tan.md), [`asin acos atan`](reference/functions/asin-acos-atan.md), [`sinh cosh tanh`](reference/functions/sinh-cosh-tanh.md), [`random`](reference/functions/random.md), [`make-random-state`](reference/functions/make-random-state.md), [`scale-float`](reference/functions/scale-float.md), [`decode-float`](reference/functions/decode-float.md)

### Bitwise Operations

[`logand`](reference/functions/logand.md), [`logior`](reference/functions/logior.md), [`logxor`](reference/functions/logxor.md), [`lognot`](reference/functions/lognot.md), [`logandc1`](reference/functions/logandc1.md), [`logandc2`](reference/functions/logandc2.md), [`logorc1`](reference/functions/logorc1.md), [`logorc2`](reference/functions/logorc2.md), [`ash`](reference/functions/ash.md), [`integer-length`](reference/functions/integer-length.md), [`logbitp`](reference/functions/logbitp.md), [`logtest`](reference/functions/logtest.md), [`byte`](reference/functions/byte.md), [`byte-size`](reference/functions/byte-size.md), [`byte-position`](reference/functions/byte-position.md), [`ldb`](reference/functions/ldb.md), [`dpb`](reference/functions/dpb.md), [`mask-field`](reference/functions/mask-field.md)

### Equality & Type Predicates

[`eq`](reference/functions/eq.md), [`eql`](reference/functions/eql.md), [`equal`](reference/functions/equal.md), [`equalp`](reference/functions/equalp.md), [`null`](reference/functions/null.md), [`not`](reference/functions/not.md), [`atom`](reference/functions/atom.md), [`numberp`](reference/functions/numberp.md), [`integerp`](reference/functions/integerp.md), [`floatp`](reference/functions/floatp.md), [`rationalp`](reference/functions/rationalp.md), [`symbolp`](reference/functions/symbolp.md), [`stringp`](reference/functions/stringp.md), [`simple-string-p`](reference/functions/simple-string-p.md), [`listp`](reference/functions/listp.md), [`consp`](reference/functions/consp.md), [`keywordp`](reference/functions/keywordp.md), [`characterp`](reference/functions/characterp.md), [`functionp`](reference/functions/functionp.md), [`vectorp`](reference/functions/vectorp.md), [`arrayp`](reference/functions/arrayp.md), [`constantp`](reference/functions/constantp.md), [`get-setf-expansion`](reference/functions/get-setf-expansion.md), [`streamp`](reference/functions/streamp.md), [`subtypep`](reference/functions/subtypep.md), [`class-of`](reference/functions/class-of.md), [`type-of`](reference/functions/type-of.md), [`pathnamep`](reference/functions/pathnamep.md)

### Output

[`print`](reference/functions/print.md), [`prin1`](reference/functions/prin1.md), [`princ`](reference/functions/princ.md), [`terpri`](reference/functions/terpri.md), [`fresh-line`](reference/functions/fresh-line.md), [`princ-to-string`](reference/functions/princ-to-string.md), [`prin1-to-string`](reference/functions/prin1-to-string.md), [`write-to-string`](reference/functions/write-to-string.md), [`write`](reference/functions/write.md), [`pprint pprint-newline pprint-indent pprint-tab`](reference/functions/pprint.md), [`copy-pprint-dispatch set-pprint-dispatch pprint-dispatch`](reference/functions/pprint-dispatch.md)

### Strings

[`concatenate`](reference/functions/concatenate.md), [`string`](reference/functions/string.md), [`make-string`](reference/functions/make-string.md), [`make-sequence`](reference/functions/make-sequence.md), [`replace`](reference/functions/replace.md), [`fill`](reference/functions/fill.md), [`string-upcase`](reference/functions/string-upcase.md), [`string-downcase`](reference/functions/string-downcase.md), [`string-capitalize`](reference/functions/string-capitalize.md), [`nstring-upcase nstring-downcase nstring-capitalize`](reference/functions/nstring-case.md), [`subseq`](reference/functions/subseq.md), [`string=`](reference/functions/string-eq.md), [`string< string> string<= string>= string/= string-lessp string-greaterp string-not-greaterp string-not-lessp string-not-equal`](reference/functions/string-compare.md), [`string-equal`](reference/functions/string-equal.md), [`string-trim`](reference/functions/string-trim.md), [`string-left-trim`](reference/functions/string-left-trim.md), [`string-right-trim`](reference/functions/string-right-trim.md)

### Characters

[`char schar`](reference/functions/char.md), [`char-code`](reference/functions/char-code.md), [`code-char`](reference/functions/code-char.md), [`char= char< char<= char> char>= char/= char-equal`](reference/functions/char-compare.md), [`char-lessp char-greaterp char-not-lessp char-not-greaterp char-not-equal`](reference/functions/char-compare-ci.md), [`char-upcase char-downcase`](reference/functions/char-case.md), [`alpha-char-p`](reference/functions/alpha-char-p.md), [`alphanumericp`](reference/functions/alphanumericp.md), [`graphic-char-p standard-char-p`](reference/functions/graphic-char-p.md), [`make-load-form-saving-slots`](reference/functions/make-load-form-saving-slots.md), [`sxhash`](reference/functions/sxhash.md), [`sbit`](reference/functions/sbit.md), [`bit`](reference/functions/bit.md), [`both-case-p`](reference/functions/both-case-p.md), [`special-operator-p`](reference/functions/special-operator-p.md), [`macro-function`](reference/functions/macro-function.md), [`compiled-function-p`](reference/functions/compiled-function-p.md), [`function-lambda-expression`](reference/functions/function-lambda-expression.md), [`list-all-packages`](reference/functions/list-all-packages.md), [`find-class`](reference/functions/find-class.md), [`allocate-instance`](reference/functions/allocate-instance.md), [`class-name`](reference/functions/class-name.md), [`get`](reference/functions/get.md), [`symbol-plist`](reference/functions/symbol-plist.md), [`remprop`](reference/functions/remprop.md), [`lower-case-p upper-case-p`](reference/functions/case-char-p.md), [`digit-char-p`](reference/functions/digit-char-p.md), [`digit-char`](reference/functions/digit-char.md), [`char-name`](reference/functions/char-name.md)

### Cons & List Construction

[`cons`](reference/functions/cons.md), [`car`](reference/functions/car.md), [`cdr`](reference/functions/cdr.md), [`caar cddddr`](reference/functions/car-cdr-compositions.md), [`first`](reference/functions/first.md), [`rest`](reference/functions/rest.md), [`second third fourth`](reference/functions/second-third-fourth.md), [`nth`](reference/functions/nth.md), [`nthcdr`](reference/functions/nthcdr.md), [`list`](reference/functions/list.md), [`list*`](reference/functions/list*.md), [`acons`](reference/functions/acons.md), [`make-list`](reference/functions/make-list.md), [`copy-list`](reference/functions/copy-list.md), [`copy-tree`](reference/functions/copy-tree.md)

### List Operations

[`length`](reference/functions/length.md), [`reverse`](reference/functions/reverse.md), [`nreverse`](reference/functions/nreverse.md), [`last`](reference/functions/last.md), [`butlast`](reference/functions/butlast.md), [`elt`](reference/functions/elt.md), [`endp`](reference/functions/endp.md), [`revappend`](reference/functions/revappend.md), [`nreconc`](reference/functions/nreconc.md), [`nconc`](reference/functions/nconc.md), [`sort`](reference/functions/sort.md), [`stable-sort`](reference/functions/stable-sort.md), [`merge`](reference/functions/merge.md), [`copy-seq`](reference/functions/copy-seq.md)

### List Search

[`member`](reference/functions/member.md), [`member-if`](reference/functions/member-if.md), [`find`](reference/functions/find.md), [`find-if`](reference/functions/find-if.md), [`find-if-not`](reference/functions/find-if-not.md), [`position`](reference/functions/position.md), [`position-if`](reference/functions/position-if.md), [`position-if-not`](reference/functions/position-if-not.md), [`count`](reference/functions/count.md), [`count-if`](reference/functions/count-if.md), [`count-if-not`](reference/functions/count-if-not.md), [`assoc`](reference/functions/assoc.md), [`assoc-if`](reference/functions/assoc-if.md), [`rassoc`](reference/functions/rassoc.md), [`rassoc-if`](reference/functions/rassoc-if.md), [`pairlis`](reference/functions/pairlis.md), [`copy-alist`](reference/functions/copy-alist.md), [`getf`](reference/functions/getf.md)

### List Filtering & Modification

[`remove`](reference/functions/remove.md), [`remove-if`](reference/functions/remove-if.md), [`remove-if-not`](reference/functions/remove-if-not.md), [`remove-duplicates`](reference/functions/remove-duplicates.md), [`delete-duplicates`](reference/functions/delete-duplicates.md), [`delete`](reference/functions/delete.md), [`delete-if`](reference/functions/delete-if.md), [`delete-if-not`](reference/functions/delete-if-not.md), [`subst`](reference/functions/subst.md), [`search`](reference/functions/search.md), [`mismatch`](reference/functions/mismatch.md), [`tree-equal`](reference/functions/tree-equal.md), [`substitute`](reference/functions/substitute.md), [`nsubstitute`](reference/functions/nsubstitute.md), [`substitute-if`](reference/functions/substitute-if.md), [`substitute-if-not`](reference/functions/substitute-if-not.md), [`nsubstitute-if`](reference/functions/nsubstitute-if.md), [`nsubstitute-if-not`](reference/functions/nsubstitute-if-not.md)

### Sets

[`union`](reference/functions/union.md), [`intersection`](reference/functions/intersection.md), [`set-difference`](reference/functions/set-difference.md), [`set-exclusive-or`](reference/functions/set-exclusive-or.md), [`adjoin`](reference/functions/adjoin.md)

### Destructive Cell Operations

[`rplaca`](reference/functions/rplaca.md), [`rplacd`](reference/functions/rplacd.md)

### Higher-Order Functions

[`funcall`](reference/functions/funcall.md), [`apply`](reference/functions/apply.md), [`values`](reference/functions/values.md), [`values-list`](reference/functions/values-list.md), [`mapcar`](reference/functions/mapcar.md), [`map`](reference/functions/map.md), [`map-into`](reference/functions/map-into.md), [`mapc`](reference/functions/mapc.md), [`mapcan`](reference/functions/mapcan.md), [`maplist`](reference/functions/maplist.md), [`mapcon`](reference/functions/mapcon.md), [`mapl`](reference/functions/mapl.md), [`reduce`](reference/functions/reduce.md), [`every`](reference/functions/every.md), [`some`](reference/functions/some.md), [`notany`](reference/functions/notany.md), [`notevery`](reference/functions/notevery.md), [`identity`](reference/functions/identity.md), [`constantly`](reference/functions/constantly.md), [`symbol-function`](reference/functions/symbol-function.md)

### Hash Tables

[`make-hash-table`](reference/functions/make-hash-table.md), [`gethash`](reference/functions/gethash.md), [`remhash`](reference/functions/remhash.md), [`clrhash`](reference/functions/clrhash.md), [`hash-table-count`](reference/functions/hash-table-count.md), [`hash-table-test`](reference/functions/hash-table-test.md), [`hash-table-size`](reference/functions/hash-table-size.md), [`hash-table-rehash-size`](reference/functions/hash-table-rehash-size.md), [`hash-table-rehash-threshold`](reference/functions/hash-table-rehash-threshold.md), [`hash-table-p`](reference/functions/hash-table-p.md), [`maphash`](reference/functions/maphash.md)

### Arrays

[`make-array`](reference/functions/make-array.md), [`aref`](reference/functions/aref.md), [`row-major-aref`](reference/functions/row-major-aref.md), [`array-row-major-index`](reference/functions/array-row-major-index.md), [`vector`](reference/functions/vector.md), [`svref`](reference/functions/svref.md), [`array-dimensions`](reference/functions/array-dimensions.md), [`array-dimension`](reference/functions/array-dimension.md), [`array-rank`](reference/functions/array-rank.md), [`array-total-size`](reference/functions/array-total-size.md), [`coerce`](reference/functions/coerce.md), [`fill-pointer`](reference/functions/fill-pointer.md), [`array-has-fill-pointer-p`](reference/functions/array-has-fill-pointer-p.md), [`adjustable-array-p`](reference/functions/adjustable-array-p.md), [`array-element-type`](reference/functions/array-element-type.md), [`vector-push`](reference/functions/vector-push.md), [`vector-pop`](reference/functions/vector-pop.md), [`vector-push-extend`](reference/functions/vector-push-extend.md), [`adjust-array`](reference/functions/adjust-array.md), [`array-displacement`](reference/functions/array-displacement.md)

### Reader & Evaluation

[`read`](reference/functions/read.md), [`read-line`](reference/functions/read-line.md), [`y-or-n-p`](reference/functions/y-or-n-p.md), [`read-char`](reference/functions/read-char.md), [`peek-char`](reference/functions/peek-char.md), [`read-char-no-hang`](reference/functions/read-char-no-hang.md), [`unread-char`](reference/functions/unread-char.md), [`read-from-string`](reference/functions/read-from-string.md), [`parse-integer`](reference/functions/parse-integer.md), [`copy-readtable`](reference/functions/copy-readtable.md), [`set-dispatch-macro-character`](reference/functions/set-dispatch-macro-character.md), [`readtable-case`](reference/functions/readtable-case.md), [`eval`](reference/functions/eval.md), [`compile`](reference/functions/compile.md), [`load`](reference/functions/load.md), [`require`](reference/functions/require.md), [`provide`](reference/functions/provide.md)

### Symbols & Macros

[`gensym`](reference/functions/gensym.md), [`make-symbol`](reference/functions/make-symbol.md), [`copy-symbol`](reference/functions/copy-symbol.md), [`intern`](reference/functions/intern.md), [`find-symbol`](reference/functions/find-symbol.md), [`find-package`](reference/functions/find-package.md), [`symbol-name`](reference/functions/symbol-name.md), [`symbol-package`](reference/functions/symbol-package.md), [`package-name`](reference/functions/package-name.md), [`package-use-list`](reference/functions/package-use-list.md), [`package-used-by-list`](reference/functions/package-used-by-list.md), [`package-shadowing-symbols`](reference/functions/package-shadowing-symbols.md), [`symbol-value`](reference/functions/symbol-value.md), [`boundp`](reference/functions/boundp.md), [`fboundp`](reference/functions/fboundp.md), [`fmakunbound`](reference/functions/fmakunbound.md), [`macroexpand-1`](reference/functions/macroexpand-1.md), [`macroexpand`](reference/functions/macroexpand.md), [`fdefinition`](reference/functions/fdefinition.md), [`use-package`](reference/functions/use-package.md), [`export`](reference/functions/export.md), [`unexport`](reference/functions/unexport.md), [`import`](reference/functions/import.md)

### Streams & Files

[`open`](reference/functions/open.md), [`close`](reference/functions/close.md), [`probe-file`](reference/functions/probe-file.md), [`directory`](reference/functions/directory.md), [`truename`](reference/functions/truename.md), [`pathname`](reference/functions/pathname.md), [`parse-namestring`](reference/functions/parse-namestring.md), [`merge-pathnames`](reference/functions/merge-pathnames.md), [`make-pathname`](reference/functions/make-pathname.md), [`namestring`](reference/functions/namestring.md), [`pathname-directory`](reference/functions/pathname-directory.md), [`pathname-name`](reference/functions/pathname-name.md), [`pathname-type`](reference/functions/pathname-type.md), [`pathname-host`](reference/functions/pathname-host.md), [`pathname-device`](reference/functions/pathname-device.md), [`pathname-version`](reference/functions/pathname-version.md), [`wild-pathname-p`](reference/functions/wild-pathname-p.md), [`enough-namestring`](reference/functions/enough-namestring.md), [`file-namestring directory-namestring host-namestring`](reference/functions/namestring-components.md), [`translate-pathname`](reference/functions/translate-pathname.md), [`translate-logical-pathname`](reference/functions/translate-logical-pathname.md), [`logical-pathname`](reference/functions/logical-pathname.md), [`open-stream-p`](reference/functions/open-stream-p.md), [`force-output`](reference/functions/force-output.md), [`finish-output`](reference/functions/finish-output.md), [`clear-output`](reference/functions/clear-output.md), [`listen`](reference/functions/listen.md), [`write-line`](reference/functions/write-line.md), [`write-string`](reference/functions/write-string.md), [`read-byte`](reference/functions/read-byte.md), [`write-byte`](reference/functions/write-byte.md), [`read-sequence`](reference/functions/read-sequence.md), [`write-sequence`](reference/functions/write-sequence.md), [`file-position`](reference/functions/file-position.md), [`file-length`](reference/functions/file-length.md), [`file-write-date`](reference/functions/file-write-date.md), [`ensure-directories-exist`](reference/functions/ensure-directories-exist.md), [`delete-file`](reference/functions/delete-file.md), [`rename-file`](reference/functions/rename-file.md), [`make-string-output-stream`](reference/functions/make-string-output-stream.md), [`make-string-input-stream`](reference/functions/make-string-input-stream.md), [`get-output-stream-string`](reference/functions/get-output-stream-string.md), [`make-synonym-stream`](reference/functions/make-synonym-stream.md), [`synonym-stream-symbol`](reference/functions/synonym-stream-symbol.md), [`make-broadcast-stream`](reference/functions/make-broadcast-stream.md), [`input-stream-p`](reference/functions/input-stream-p.md), [`output-stream-p`](reference/functions/output-stream-p.md), [`stream-element-type`](reference/functions/stream-element-type.md)

### System & Time

[`get-universal-time`](reference/functions/get-universal-time.md), [`encode-universal-time`](reference/functions/encode-universal-time.md), [`decode-universal-time`](reference/functions/decode-universal-time.md), [`get-internal-real-time`](reference/functions/get-internal-real-time.md), [`get-internal-run-time`](reference/functions/get-internal-run-time.md), [`sleep`](reference/functions/sleep.md), [`lisp-implementation-type lisp-implementation-version software-type software-version machine-type machine-version machine-instance short-site-name long-site-name`](reference/functions/environment-enquiry.md), [`user-homedir-pathname`](reference/functions/user-homedir-pathname.md), [`invoke-debugger`](reference/functions/invoke-debugger.md), [`compile-file compile-file-pathname remove-method`](reference/functions/compile-file.md)

### rontolisp Package

[`rontolisp:version`](reference/functions/rontolisp-version.md), [`rontolisp:random-bytes`](reference/functions/rontolisp-random-bytes.md), [`rontolisp:make-mutex`](reference/functions/rontolisp-make-mutex.md), [`rontolisp:mutex-acquire`](reference/functions/rontolisp-mutex-acquire.md), [`rontolisp:mutex-release`](reference/functions/rontolisp-mutex-release.md), [`rontolisp:make-thread`](reference/functions/rontolisp-make-thread.md), [`rontolisp:join-thread`](reference/functions/rontolisp-join-thread.md), [`rontolisp:threadp`](reference/functions/rontolisp-threadp.md), [`rontolisp:thread-alive-p`](reference/functions/rontolisp-thread-alive-p.md), [`rontolisp:destroy-thread`](reference/functions/rontolisp-destroy-thread.md), [`rontolisp:current-thread`](reference/functions/rontolisp-current-thread.md), [`rontolisp:list-functions`](reference/functions/rontolisp-list-functions.md), [`rontolisp:list-macros`](reference/functions/rontolisp-list-macros.md), [`rontolisp:list-special-forms`](reference/functions/rontolisp-list-special-forms.md), [`rontolisp:fetch`](reference/functions/rontolisp-fetch.md), [`rontolisp:futurep`](reference/functions/rontolisp-futurep.md), [`rontolisp:streamp`](reference/functions/rontolisp-streamp.md), [`rontolisp:make-stream`](reference/functions/rontolisp-make-stream.md), [`rontolisp:stream-read`](reference/functions/rontolisp-stream-read.md), [`rontolisp:stream-write`](reference/functions/rontolisp-stream-write.md), [`rontolisp:stream-close`](reference/functions/rontolisp-stream-close.md), [`rontolisp:read-all`](reference/functions/rontolisp-read-all.md), [`rontolisp:wait-for`](reference/functions/rontolisp-wait-for.md), [`rontolisp:then`](reference/functions/rontolisp-then.md), [`rontolisp:then*`](reference/functions/rontolisp-then-star.md), [`rontolisp:catch`](reference/functions/rontolisp-catch.md), [`rontolisp:finally`](reference/functions/rontolisp-finally.md), [`rontolisp:http-handler`](reference/functions/rontolisp-http-handler.md), [`rontolisp:json-parse`](reference/functions/rontolisp-json-parse.md), [`rontolisp:json-stringify`](reference/functions/rontolisp-json-stringify.md), [`rontolisp:plist-hash-table`](reference/functions/rontolisp-plist-hash-table.md), [`rontolisp:hash-table-plist`](reference/functions/rontolisp-hash-table-plist.md), [`rontolisp:alist-hash-table`](reference/functions/rontolisp-alist-hash-table.md), [`rontolisp:hash-table-alist`](reference/functions/rontolisp-hash-table-alist.md), [`rontolisp:alist-plist`](reference/functions/rontolisp-alist-plist.md), [`rontolisp:plist-alist`](reference/functions/rontolisp-plist-alist.md), [`rontolisp:url-decode`](reference/functions/rontolisp-url-decode.md), [`rontolisp:url-encode`](reference/functions/rontolisp-url-encode.md), [`rontolisp:query-params`](reference/functions/rontolisp-query-params.md), [`rontolisp:query-param`](reference/functions/rontolisp-query-param.md), [`rontolisp:url-path`](reference/functions/rontolisp-url-path.md), [`rontolisp:url-query`](reference/functions/rontolisp-url-query.md), [`rontolisp:tcp-connect`](reference/functions/rontolisp-tcp-connect.md), [`rontolisp:tcp-listen`](reference/functions/rontolisp-tcp-listen.md), [`rontolisp:tcp-accept`](reference/functions/rontolisp-tcp-accept.md), [`rontolisp:tcp-local-port`](reference/functions/rontolisp-tcp-local-port.md), [`rontolisp:tcp-local-address rontolisp:tcp-peer-address rontolisp:tcp-peer-port`](reference/functions/rontolisp-tcp-addresses.md), [`rontolisp:tcp-set-timeout`](reference/functions/rontolisp-tcp-set-timeout.md), [`rontolisp:tls-connect`](reference/functions/rontolisp-tls-connect.md), [`rontolisp:tls-listen`](reference/functions/rontolisp-tls-listen.md), [`rontolisp:tls-listen-pem`](reference/functions/rontolisp-tls-listen-pem.md), [`rontolisp:tls-upgrade`](reference/functions/rontolisp-tls-upgrade.md), [`rontolisp:wasm-export`](reference/functions/rontolisp-wasm-export.md), [`rontolisp:wasm-import`](reference/functions/rontolisp-wasm-import.md), [`rontolisp:wit-export`](reference/functions/rontolisp-wit-export.md), [`rontolisp:wit-import`](reference/functions/rontolisp-wit-import.md), [`rontolisp:wit-provide`](reference/functions/rontolisp-wit-provide.md)

### linalg Package

[`linalg:zeros`](reference/functions/linalg-zeros.md), [`linalg:ones`](reference/functions/linalg-ones.md), [`linalg:full`](reference/functions/linalg-full.md), [`linalg:zeros-like`](reference/functions/linalg-zeros-like.md), [`linalg:eye`](reference/functions/linalg-eye.md), [`linalg:arange`](reference/functions/linalg-arange.md), [`linalg:linspace`](reference/functions/linalg-linspace.md), [`linalg:from-list`](reference/functions/linalg-from-list.md), [`linalg:to-list`](reference/functions/linalg-to-list.md), [`linalg:shape`](reference/functions/linalg-shape.md), [`linalg:ndim`](reference/functions/linalg-ndim.md), [`linalg:size`](reference/functions/linalg-size.md), [`linalg:reshape`](reference/functions/linalg-reshape.md), [`linalg:flatten`](reference/functions/linalg-flatten.md), [`linalg:transpose`](reference/functions/linalg-transpose.md), [`linalg:pad`](reference/functions/linalg-pad.md), [`linalg:expand-dims`](reference/functions/linalg-expand-dims.md), [`linalg:squeeze`](reference/functions/linalg-squeeze.md), [`linalg:concatenate`](reference/functions/linalg-concatenate.md), [`linalg:stack`](reference/functions/linalg-stack.md), [`linalg:slice`](reference/functions/linalg-slice.md), [`linalg:triu`](reference/functions/linalg-triu.md), [`linalg:tril`](reference/functions/linalg-tril.md), [`linalg:add`](reference/functions/linalg-add.md), [`linalg:sub`](reference/functions/linalg-sub.md), [`linalg:mul`](reference/functions/linalg-mul.md), [`linalg:div`](reference/functions/linalg-div.md), [`linalg:+`](reference/functions/linalg-plus.md), [`linalg:-`](reference/functions/linalg-minus.md), [`linalg:*`](reference/functions/linalg-star.md), [`linalg:/`](reference/functions/linalg-slash.md), [`linalg:emap`](reference/functions/linalg-emap.md), [`linalg:exp`](reference/functions/linalg-exp.md), [`linalg:log`](reference/functions/linalg-log.md), [`linalg:tanh`](reference/functions/linalg-tanh.md), [`linalg:sin`](reference/functions/linalg-sin.md), [`linalg:cos`](reference/functions/linalg-cos.md), [`linalg:tan`](reference/functions/linalg-tan.md), [`linalg:asin`](reference/functions/linalg-asin.md), [`linalg:acos`](reference/functions/linalg-acos.md), [`linalg:atan`](reference/functions/linalg-atan.md), [`linalg:sinh`](reference/functions/linalg-sinh.md), [`linalg:cosh`](reference/functions/linalg-cosh.md), [`linalg:sqrt`](reference/functions/linalg-sqrt.md), [`linalg:abs`](reference/functions/linalg-abs.md), [`linalg:square`](reference/functions/linalg-square.md), [`linalg:negative`](reference/functions/linalg-negative.md), [`linalg:sign`](reference/functions/linalg-sign.md), [`linalg:reciprocal`](reference/functions/linalg-reciprocal.md), [`linalg:power`](reference/functions/linalg-power.md), [`linalg:maximum`](reference/functions/linalg-maximum.md), [`linalg:minimum`](reference/functions/linalg-minimum.md), [`linalg:clip`](reference/functions/linalg-clip.md), [`linalg:relu`](reference/functions/linalg-relu.md), [`linalg:erf`](reference/functions/linalg-erf.md), [`linalg:softmax`](reference/functions/linalg-softmax.md), [`linalg:log-softmax`](reference/functions/linalg-log-softmax.md), [`linalg:dot`](reference/functions/linalg-dot.md), [`linalg:matmul`](reference/functions/linalg-matmul.md), [`linalg:outer`](reference/functions/linalg-outer.md), [`linalg:sum`](reference/functions/linalg-sum.md), [`linalg:mean`](reference/functions/linalg-mean.md), [`linalg:var`](reference/functions/linalg-var.md), [`linalg:std`](reference/functions/linalg-std.md), [`linalg:amax`](reference/functions/linalg-amax.md), [`linalg:amin`](reference/functions/linalg-amin.md), [`linalg:argmax`](reference/functions/linalg-argmax.md), [`linalg:argmin`](reference/functions/linalg-argmin.md), [`linalg:norm`](reference/functions/linalg-norm.md), [`linalg:trace`](reference/functions/linalg-trace.md), [`linalg:diff`](reference/functions/linalg-diff.md), [`linalg:gradient`](reference/functions/linalg-gradient.md), [`linalg:det`](reference/functions/linalg-det.md), [`linalg:inv`](reference/functions/linalg-inv.md), [`linalg:solve`](reference/functions/linalg-solve.md), [`linalg:array-equal`](reference/functions/linalg-array-equal.md), [`linalg:equal`](reference/functions/linalg-equal.md), [`linalg:greater`](reference/functions/linalg-greater.md), [`linalg:greater-equal`](reference/functions/linalg-greater-equal.md), [`linalg:less`](reference/functions/linalg-less.md), [`linalg:less-equal`](reference/functions/linalg-less-equal.md), [`linalg:where`](reference/functions/linalg-where.md), [`linalg:take-rows`](reference/functions/linalg-take-rows.md), [`linalg:row`](reference/functions/linalg-row.md), [`linalg:gather`](reference/functions/linalg-gather.md), [`linalg:one-hot`](reference/functions/linalg-one-hot.md), [`linalg:seed`](reference/functions/linalg-seed.md), [`linalg:rand`](reference/functions/linalg-rand.md), [`linalg:randn`](reference/functions/linalg-randn.md), [`linalg:uniform`](reference/functions/linalg-uniform.md), [`linalg:choice`](reference/functions/linalg-choice.md), [`linalg:permutation`](reference/functions/linalg-permutation.md)

### torch Package

[`torch:tensor`](reference/functions/torch-tensor.md), [`torch:tensorp`](reference/functions/torch-tensorp.md), [`torch:data`](reference/functions/torch-data.md), [`torch:grad`](reference/functions/torch-grad.md), [`torch:shape`](reference/functions/torch-shape.md), [`torch:item`](reference/functions/torch-item.md), [`torch:detach`](reference/functions/torch-detach.md), [`torch:zero-grad`](reference/functions/torch-zero-grad.md), [`torch:requires-grad-p`](reference/functions/torch-requires-grad-p.md), [`torch:backward`](reference/functions/torch-backward.md), [`torch:add`](reference/functions/torch-add.md), [`torch:sub`](reference/functions/torch-sub.md), [`torch:mul`](reference/functions/torch-mul.md), [`torch:div`](reference/functions/torch-div.md), [`torch:neg`](reference/functions/torch-neg.md), [`torch:power`](reference/functions/torch-power.md), [`torch:exp`](reference/functions/torch-exp.md), [`torch:log`](reference/functions/torch-log.md), [`torch:sqrt`](reference/functions/torch-sqrt.md), [`torch:tanh`](reference/functions/torch-tanh.md), [`torch:relu`](reference/functions/torch-relu.md), [`torch:erf`](reference/functions/torch-erf.md), [`torch:gelu`](reference/functions/torch-gelu.md), [`torch:matmul`](reference/functions/torch-matmul.md), [`torch:sum`](reference/functions/torch-sum.md), [`torch:mean`](reference/functions/torch-mean.md), [`torch:var`](reference/functions/torch-var.md), [`torch:std`](reference/functions/torch-std.md), [`torch:amax`](reference/functions/torch-amax.md), [`torch:argmax`](reference/functions/torch-argmax.md), [`torch:topk`](reference/functions/torch-topk.md), [`torch:multinomial`](reference/functions/torch-multinomial.md), [`torch:softmax`](reference/functions/torch-softmax.md), [`torch:log-softmax`](reference/functions/torch-log-softmax.md), [`torch:masked-fill`](reference/functions/torch-masked-fill.md), [`torch:gather`](reference/functions/torch-gather.md), [`torch:index-select`](reference/functions/torch-index-select.md), [`torch:reshape`](reference/functions/torch-reshape.md), [`torch:view`](reference/functions/torch-view.md), [`torch:transpose`](reference/functions/torch-transpose.md), [`torch:unsqueeze`](reference/functions/torch-unsqueeze.md), [`torch:squeeze`](reference/functions/torch-squeeze.md), [`torch:cat`](reference/functions/torch-cat.md), [`torch:stack`](reference/functions/torch-stack.md), [`torch:slice`](reference/functions/torch-slice.md), [`torch:set-data`](reference/functions/torch-set-data.md), [`torch:module`](reference/functions/torch-module.md), [`torch:modulep`](reference/functions/torch-modulep.md), [`torch:module-kind`](reference/functions/torch-module-kind.md), [`torch:field`](reference/functions/torch-field.md), [`torch:fields`](reference/functions/torch-fields.md), [`torch:set-field`](reference/functions/torch-set-field.md), [`torch:forward`](reference/functions/torch-forward.md), [`torch:parameter`](reference/functions/torch-parameter.md), [`torch:parameters`](reference/functions/torch-parameters.md), [`torch:train`](reference/functions/torch-train.md), [`torch:eval`](reference/functions/torch-eval.md), [`torch:training-p`](reference/functions/torch-training-p.md), [`torch:linear`](reference/functions/torch-linear.md), [`torch:embedding`](reference/functions/torch-embedding.md), [`torch:sequential`](reference/functions/torch-sequential.md), [`torch:layer-norm`](reference/functions/torch-layer-norm.md), [`torch:dropout`](reference/functions/torch-dropout.md), [`torch:mse-loss`](reference/functions/torch-mse-loss.md), [`torch:cross-entropy-loss`](reference/functions/torch-cross-entropy-loss.md), [`torch:optimizer`](reference/functions/torch-optimizer.md), [`torch:optimizerp`](reference/functions/torch-optimizerp.md), [`torch:optimizer-kind`](reference/functions/torch-optimizer-kind.md), [`torch:optimizer-params`](reference/functions/torch-optimizer-params.md), [`torch:step`](reference/functions/torch-step.md), [`torch:step-count`](reference/functions/torch-step-count.md), [`torch:sgd`](reference/functions/torch-sgd.md), [`torch:adam`](reference/functions/torch-adam.md), [`torch:adamw`](reference/functions/torch-adamw.md), [`torch:clip-grad-norm`](reference/functions/torch-clip-grad-norm.md), [`torch:pad-sequence`](reference/functions/torch-pad-sequence.md), [`torch:shuffled-batches`](reference/functions/torch-shuffled-batches.md), [`torch:padding-mask`](reference/functions/torch-padding-mask.md), [`torch:subsequent-mask`](reference/functions/torch-subsequent-mask.md)

### java Package (JVM only)

[`java:new`](reference/functions/java-new.md), [`java:call`](reference/functions/java-call.md), [`java:static`](reference/functions/java-static.md), [`java:field`](reference/functions/java-field.md), [`java:proxy`](reference/functions/java-proxy.md)

### asdf Package (System Definitions)

[`asdf:defsystem`](reference/functions/asdf-defsystem.md), [`asdf:load-system`](reference/functions/asdf-load-system.md), [`asdf:test-system`](reference/functions/asdf-test-system.md), [`asdf:find-system`](reference/functions/asdf-find-system.md), [`asdf:registered-systems`](reference/functions/asdf-registered-systems.md), [`asdf:system-relative-pathname`](reference/functions/asdf-system-relative-pathname.md), [`asdf:component-pathname`](reference/functions/asdf-component-pathname.md), [`asdf:component-name`](reference/functions/asdf-component-name.md), [`asdf:component-version`](reference/functions/asdf-component-version.md), [`asdf:component-children`](reference/functions/asdf-component-children.md), [`asdf:component-sideway-dependencies`](reference/functions/asdf-component-sideway-dependencies.md), [`asdf:component-parent`](reference/functions/asdf-component-parent.md), [`asdf:component-system`](reference/functions/asdf-component-system.md)

### uiop Package (ASDF portability layer)

[`uiop:getenv`](reference/functions/uiop-getenv.md), [`uiop:file-exists-p`](reference/functions/uiop-file-exists-p.md), [`uiop:directory-exists-p`](reference/functions/uiop-directory-exists-p.md), [`uiop:directory-files`](reference/functions/uiop-directory-files.md), [`uiop:subdirectories`](reference/functions/uiop-subdirectories.md), [`uiop:collect-sub*directories`](reference/functions/uiop-collect-sub-directories.md), [`uiop:read-file-string`](reference/functions/uiop-read-file-string.md), [`uiop:merge-pathnames*`](reference/functions/uiop-merge-pathnames-star.md), [`uiop:subpathname`](reference/functions/uiop-subpathname.md), [`uiop:subpathp`](reference/functions/uiop-subpathp.md), [`uiop:parse-unix-namestring`](reference/functions/uiop-parse-unix-namestring.md), [`uiop:unix-namestring`](reference/functions/uiop-unix-namestring.md), [`uiop:ensure-pathname`](reference/functions/uiop-ensure-pathname.md), [`uiop:enough-pathname`](reference/functions/uiop-enough-pathname.md), [`uiop:pathname-directory-pathname`](reference/functions/uiop-pathname-directory-pathname.md), [`uiop:pathname-parent-directory-pathname`](reference/functions/uiop-pathname-parent-directory-pathname.md), [`uiop:split-name-type`](reference/functions/uiop-split-name-type.md), [`uiop:absolute-pathname-p`](reference/functions/uiop-absolute-pathname-p.md), [`uiop:relative-pathname-p`](reference/functions/uiop-relative-pathname-p.md), [`uiop:directory-pathname-p`](reference/functions/uiop-directory-pathname-p.md), [`uiop:file-pathname-p`](reference/functions/uiop-file-pathname-p.md), [`uiop:native-namestring`](reference/functions/uiop-native-namestring.md), [`uiop:add-package-local-nickname`](reference/functions/uiop-add-package-local-nickname.md), [`uiop:emptyp`](reference/functions/uiop-emptyp.md), [`uiop:first-char`](reference/functions/uiop-first-char.md), [`uiop:last-char`](reference/functions/uiop-last-char.md), [`uiop:split-string`](reference/functions/uiop-split-string.md), [`uiop:symbol-call`](reference/functions/uiop-symbol-call.md), [`uiop/image:print-condition-backtrace`](reference/functions/uiop-print-condition-backtrace.md)

### ql / ql-dist Packages (Quicklisp)

[`ql:quickload`](reference/functions/ql-quickload.md), [`ql-dist:install-dist`](reference/functions/ql-dist-install-dist.md), [`ql:update-dist`](reference/functions/ql-update-dist.md)

### usocket Package (usocket-compatible sockets)

[`usocket:socket-connect`](reference/functions/usocket-socket-connect.md), [`usocket:socket-listen`](reference/functions/usocket-socket-listen.md), [`usocket:socket-accept`](reference/functions/usocket-socket-accept.md), [`usocket:socket-stream usocket:socket-close`](reference/functions/usocket-socket-stream.md), [`usocket:get-local-port usocket:get-local-address usocket:get-local-name usocket:get-peer-address usocket:get-peer-port usocket:get-peer-name`](reference/functions/usocket-accessors.md), [`usocket:host-to-hostname usocket:get-host-by-name`](reference/functions/usocket-host-names.md)

### Conditions

[`simple-condition-format-control`](reference/functions/simple-condition-format-control.md), [`type-error-datum`](reference/functions/type-error-datum.md), [`type-error-expected-type`](reference/functions/type-error-expected-type.md), [`cell-error-name`](reference/functions/cell-error-name.md), [`unbound-slot-instance`](reference/functions/unbound-slot-instance.md), [`print-object`](reference/functions/print-object.md), [`simple-condition-format-arguments`](reference/functions/simple-condition-format-arguments.md), [`find-restart`](reference/functions/find-restart.md), [`invoke-restart`](reference/functions/invoke-restart.md), [`compute-restarts`](reference/functions/compute-restarts.md), [`restart-name`](reference/functions/restart-name.md), [`muffle-warning`](reference/functions/muffle-warning.md), [`abort`](reference/functions/abort.md), [`continue`](reference/functions/continue.md), [`use-value`](reference/functions/use-value.md), [`store-value`](reference/functions/store-value.md)

## Macros (105)

`reference/macros/.md`

### Macros

[`cond`](reference/macros/cond.md), [`case`](reference/macros/case.md), [`ecase`](reference/macros/ecase.md), [`ccase`](reference/macros/ccase.md), [`and`](reference/macros/and.md), [`or`](reference/macros/or.md), [`when`](reference/macros/when.md), [`unless`](reference/macros/unless.md), [`dotimes`](reference/macros/dotimes.md), [`do`](reference/macros/do.md), [`do*`](reference/macros/do-star.md), [`loop`](reference/macros/loop.md), [`prog1`](reference/macros/prog1.md), [`multiple-value-prog1`](reference/macros/multiple-value-prog1.md), [`prog2`](reference/macros/prog2.md), [`time`](reference/macros/time.md), [`psetq`](reference/macros/psetq.md), [`psetf`](reference/macros/psetf.md), [`typecase`](reference/macros/typecase.md), [`etypecase`](reference/macros/etypecase.md), [`ctypecase`](reference/macros/ctypecase.md), [`error`](reference/macros/error.md), [`cerror`](reference/macros/cerror.md), [`warn`](reference/macros/warn.md), [`block`](reference/macros/block.md), [`return-from`](reference/macros/return-from.md), [`setf`](reference/macros/setf.md), [`push`](reference/macros/push.md), [`pop`](reference/macros/pop.md), [`pushnew`](reference/macros/pushnew.md), [`remf`](reference/macros/remf.md), [`let*`](reference/macros/let-star.md), [`dolist`](reference/macros/dolist.md), [`incf`](reference/macros/incf.md), [`decf`](reference/macros/decf.md), [`format`](reference/macros/format.md), [`with-open-file`](reference/macros/with-open-file.md), [`with-open-stream`](reference/macros/with-open-stream.md), [`with-output-to-string`](reference/macros/with-output-to-string.md), [`with-input-from-string`](reference/macros/with-input-from-string.md), [`with-standard-io-syntax`](reference/macros/with-standard-io-syntax.md), [`check-type`](reference/macros/check-type.md), [`assert`](reference/macros/assert.md), [`declare`](reference/macros/declare.md), [`declaim`](reference/macros/declaim.md), [`proclaim`](reference/macros/proclaim.md), [`the`](reference/macros/the.md), [`eval-when`](reference/macros/eval-when.md), [`locally`](reference/macros/locally.md), [`write-char`](reference/macros/write-char.md), [`flet`](reference/macros/flet.md), [`labels`](reference/macros/labels.md), [`macrolet`](reference/macros/macrolet.md), [`symbol-macrolet`](reference/macros/symbol-macrolet.md), [`multiple-value-bind`](reference/macros/multiple-value-bind.md), [`multiple-value-list`](reference/macros/multiple-value-list.md), [`multiple-value-call`](reference/macros/multiple-value-call.md), [`nth-value`](reference/macros/nth-value.md), [`multiple-value-setq`](reference/macros/multiple-value-setq.md), [`rotatef`](reference/macros/rotatef.md), [`destructuring-bind`](reference/macros/destructuring-bind.md), [`complement`](reference/macros/complement.md), [`complex`](reference/macros/complex.md), [`deftype`](reference/macros/deftype.md), [`handler-case`](reference/macros/handler-case.md), [`ignore-errors`](reference/macros/ignore-errors.md), [`handler-bind`](reference/macros/handler-bind.md), [`signal`](reference/macros/signal.md), [`with-slots`](reference/macros/with-slots.md), [`define-condition`](reference/macros/define-condition.md), [`define-compiler-macro`](reference/macros/define-compiler-macro.md), [`define-modify-macro`](reference/macros/define-modify-macro.md), [`define-setf-expander`](reference/macros/define-setf-expander.md), [`defsetf`](reference/macros/defsetf.md), [`restart-case`](reference/macros/restart-case.md), [`restart-bind`](reference/macros/restart-bind.md), [`with-simple-restart`](reference/macros/with-simple-restart.md), [`make-condition`](reference/macros/make-condition.md), [`make-instance`](reference/macros/make-instance.md), [`slot-value`](reference/macros/slot-value.md), [`with-accessors`](reference/macros/with-accessors.md), [`change-class`](reference/macros/change-class.md), [`documentation`](reference/macros/documentation.md), [`rontolisp:with-arena`](reference/macros/rontolisp-with-arena.md), [`rontolisp:with-mutex`](reference/macros/rontolisp-with-mutex.md), [`usocket:with-client-socket usocket:with-connected-socket usocket:with-server-socket usocket:with-socket-listener`](reference/macros/usocket-with-macros.md), [`torch:no-grad`](reference/macros/torch-no-grad.md), [`uiop:if-let`](reference/macros/uiop-if-let.md), [`uiop:when-let`](reference/macros/uiop-when-let.md), [`uiop:when-let*`](reference/macros/uiop-when-let-star.md), [`uiop:with-deprecation`](reference/macros/uiop-with-deprecation.md), [`prog`](reference/macros/prog.md), [`prog*`](reference/macros/prog-star.md), [`shiftf`](reference/macros/shiftf.md), [`load-time-value`](reference/macros/load-time-value.md), [`typep`](reference/macros/typep.md), [`slot-boundp`](reference/macros/slot-boundp.md), [`slot-makunbound`](reference/macros/slot-makunbound.md), [`slot-exists-p`](reference/macros/slot-exists-p.md), [`print-unreadable-object`](reference/macros/print-unreadable-object.md), [`pprint-logical-block`](reference/macros/pprint-logical-block.md), [`with-package-iterator`](reference/macros/with-package-iterator.md), [`do-external-symbols`](reference/macros/do-external-symbols.md), [`do-symbols`](reference/macros/do-symbols.md), [`with-compilation-unit`](reference/macros/with-compilation-unit.md)

## Special forms (29)

`reference/special-forms/.md`

### Special Forms

[`quote`](reference/special-forms/quote.md), [`if`](reference/special-forms/if.md), [`let`](reference/special-forms/let.md), [`progv`](reference/special-forms/progv.md), [`lambda`](reference/special-forms/lambda.md), [`progn`](reference/special-forms/progn.md), [`setq`](reference/special-forms/setq.md), [`while`](reference/special-forms/while.md), [`return`](reference/special-forms/return.md), [`unwind-protect`](reference/special-forms/unwind-protect.md), [`defun`](reference/special-forms/defun.md), [`defmacro`](reference/special-forms/defmacro.md), [`defstruct`](reference/special-forms/defstruct.md), [`defclass`](reference/special-forms/defclass.md), [`defgeneric`](reference/special-forms/defgeneric.md), [`defmethod`](reference/special-forms/defmethod.md), [`defvar`](reference/special-forms/defvar.md), [`defparameter`](reference/special-forms/defparameter.md), [`defconstant`](reference/special-forms/defconstant.md), [`function`](reference/special-forms/function.md), [`defpackage`](reference/special-forms/defpackage.md), [`rontolisp:async`](reference/special-forms/rontolisp-async.md), [`rontolisp:async-defun`](reference/special-forms/rontolisp-async-defun.md), [`rontolisp:async-lambda`](reference/special-forms/rontolisp-async-lambda.md), [`rontolisp:await`](reference/special-forms/rontolisp-await.md), [`tagbody`](reference/special-forms/tagbody.md), [`go`](reference/special-forms/go.md), [`catch`](reference/special-forms/catch.md), [`throw`](reference/special-forms/throw.md)