mélangej · ABCL on Jooby and Netty
Which Lisp this is
Full ANSI Common Lisp, plus twelve symbols that only exist on the JVM.
Those twelve are in five files out of thirteen, and here they are by
name.
"We use Lisp" is not a specification. A reader deciding whether their code will run here needs three answers: which standard, how much of it, and what is in the language that is not in the standard. This chapter answers all three with numbers taken from the tree this book is part of.
The standard: ANSI X3.226-1994
mélangej runs on ABCL 1.9.2 — Armed Bear Common Lisp — which implements ANSI Common Lisp, X3.226-1994, the only Common Lisp standard there has ever been. There is no CL2, no edition, no revision. When this book says Common Lisp it means that document.
ABCL says so about itself:
CL-USER> (lisp-implementation-type)
"Armed Bear Common Lisp"
CL-USER> (lisp-implementation-version)
"1.9.2"
CL-USER> (and (member :ansi-cl *features*) t)
T
CL-USER> (sort (mapcar #'string *features*) #'string<)
("AARCH64" "ABCL" "ANSI-CL" "ARMEDBEAR" "CDR6" "COMMON-LISP" "DARWIN"
"JAVA-17" "JVM-17.0.19" "MOP" "NIO" "PACKAGE-LOCAL-NICKNAMES" "UNIX")
"Conforming" needs a caveat, and it is not ABCL's
No Common Lisp implementation passes the whole standard. ansi-test, the community conformance suite, finds failures in every implementation anyone runs it against — SBCL, CCL, CLISP, Allegro, LispWorks, ABCL. The standard is large, parts of it are read differently by different implementers, and a handful of its corners have never been implemented anywhere because nobody has needed them. "ANSI Common Lisp" in practice means a large shared core that every implementation gets right and a thin margin where they disagree.
ABCL is better than its reputation on this, and better than most JVM-hosted languages manage against their own reference. It carries :ansi-cl, it runs ansi-test in its own CI, and it does not quietly omit the hard parts: the condition system, CLOS, the MOP, bignums, ratios, logical pathnames, compile and the full reader are all there and all work. Where it differs from SBCL it is usually because the JVM made the choice — the character and float sections below — rather than because something is missing.
What that means for this book: :ansi-cl is a claim the implementation makes, not a certificate, and this project has not run ansi-test. Nothing here rests on the corners. Everything below was evaluated in the ABCL that comes with this distribution, and the standard machinery a web framework actually leans on is present and correct. If your code lives in the shared core — and application code almost always does — the implementation you are on stops mattering, which is the only form of the conformance claim worth anything.
(expt 2 200) => 16069380442589902755419620923411626025222029937827...
; a bignum, all 61 digits of it
(/ 1 3) => 1/3 ; ratios, not floats
(type-of 1l0) => DOUBLE-FLOAT ; see "where ABCL is its own thing"
(compile nil '(lambda (x) x)) ; a real compiler, to JVM bytecode
(handler-case (error 'simple-error ...) (error () :ok)) => :OK
(translate-logical-pathname "SP:A;B.TXT") => #P"/tmp/a/b.txt"
(find-package :mop) => #<PACKAGE "MOP">
Conditions, restarts, CLOS, the MOP, macros, the reader, bignums, ratios, logical pathnames and a compiler. Not a subset of the language.
How much of it we use
Less than you might expect, and that is a choice rather than a limit. The thirteen files that make up mélangej contain, in total:
13 defpackage
9 defmacro
0 defgeneric 0 defmethod
3 define-condition
9 handler-case 0 handler-bind
0 defclass 0 defstruct
0 set-macro-character / set-dispatch-macro-character
0 eval-when
0 declaim 0 (declare (optimize ...))
0 deftype 0 define-compiler-macro 0 defsetf
0 MOP, Gray streams, package-local nicknames
Three of those zeroes are worth a sentence each.
No defclass, no defstruct, no generic functions. The framework's data is plists and lists of lists. CLOS appears one layer up, in spicemine, and it appears as an open dispatch table rather than an object model: 25 generic functions and 119 methods, every method but two :around/:before wrappers dispatching on (eql :keyword):
(defgeneric scm-serve (engine repo-dir ctx-plist))
(defmethod scm-serve ((engine (eql :fossil)) repo-dir ctx) ...)
(defmethod scm-serve ((engine (eql :git)) repo-dir ctx) ...)
(defmethod scm-serve ((engine (eql :lore)) repo-dir ctx) ...)
A new engine is a file of defmethods and a keyword; nothing registers it and no base class constrains it. Domain data stays plists and lists of lists, because it comes out of a database as rows and goes into HTML as text, and an object in the middle would be a layer that only renames things. (spicemine has four defstructs, each a small record inside one file; the framework has none.)
No reader macros, ours or anybody's. There used to be one: jss's #"method", which is ABCL's and not ours, at 113 call sites. It is gone - not for taste but for nine seconds. (require :abcl-contrib) plus (require :jss) cost 8.2 of a 9.4-second cold start, measured, and jss was the only thing in the tree that needed either. #"foo" is (apply #'jcall "foo" object args) in jss's own source, so every site became (java:jcall "foo" object args) and the two requires went with it. A restart is now about two seconds. A codebase you can read is one where ( means what you think, and this one now also starts before you have finished reading the log line.
No declaim, no optimize declarations. The performance work in chapter 11 took the HTML emitter from 11.1 to 5.8 CPU-milliseconds per request, and none of it came from a declaration. It came from not calling format in a loop.
The JVM surface
This is the part that is not Common Lisp. mélangej calls Java in 176 places, and every one of them uses a symbol from the java: package that is part of ABCL - no contrib, no reader macro, nothing to require:
java:jcall instance call 125 sites
java:jstatic static call 15
java:jnew constructor 11
java:make-immediate-object deprecated, still needed 6
java:jnew-array 5
java:jarray-ref array element 5
java:jarray-length 3
java:jmethod 2
java:jnew-runtime-class Jooby reads bytecode 1
java:jobject-lisp-value 1
java:jfield 1
java:get-default-classloader 1
Twelve symbols. That is the whole of it: there is no other implementation-specific call anywhere in the thirteen files. spicemine, on top, uses more - 586 sites and 24 symbols across the whole tree, threads: for its locks and workers among them, and ext:quit in the command-line tools to set an exit status. Call it the JVM surface: it has a name because a thing with a name can be counted, and counting it is how you know whether it is growing.
It has already shrunk once. threads:synchronized-on used to appear five times, serializing the one shared JDBC connection. The connection pool in 0.1.0 replaced that with HikariCP and a dynamic binding, and the symbol left the list — which is the useful thing about counting: a jar that does its job properly takes interop out of the Lisp.
Where it lives:
file lines interop sites
spice-http 560 78 ← Jooby and Netty, on purpose
spice-db 376 53 ← JDBC, HikariCP
spice-json 107 23
spice-auth 74 15 ← javax.crypto
spice-migrate 449 7
---------------------------------
spice-md 453 0
spice-html 187 0
spice-tmpl 255 0
spice-ds 198 0
spice-i18n 3,100 0
spice-skin 588 0
spice-banner 503 0
spice-openapi 354 0
---------------------------------
total 7,204 176 2.4% of lines
Eight of the thirteen files contain no Java at all, and they are the ones that hold the markup, the words, the colours, the Markdown, the templates and the Datastar wire format — 5,638 lines, 78% of the framework. (spice-i18n is long because it carries spicemine's catalog as well as the mechanism; MELANGEJ.md says why it is on this side.)
When this chapter was first measured, at mélangej 0.2.0, the count ran over every file then in the tree, framework and spicemine together, and came to 2.9% of lines. Over the framework alone it is 2.4%.
That is the design claim of chapter 1 stated as a measurement rather than an intention. The interop is not spread thin; it is in spice-http and four adapters, and the rest of the framework is Common Lisp that happens to be running on a JVM.
It is portable in the way that implies
If the Java is confined, the rest should run anywhere. It does. Measured at mélangej 0.2.0: loaded into SBCL 2.x, with nothing supplied but a stub defpackage for spice/http — the package, no behaviour behind it — eight files loaded and ran unchanged, and they are the same eight that are the framework's Java-free half today:
OK spice-openapi OK spice-ds
OK spice-md OK spice-banner
OK spice-html OK spice-i18n
OK spice-tmpl OK spice-skin
(spice/html:html (list :p "it <works>")) => "<p>it <works></p>"
(spice/md:md->html "# h") => "<h2>h</h2>"
(spice/ds:patch-elements "<div id=\"a\">x</div>")
=> "event: datastar-patch-elements / data: mode outer / …"
spice-ds is the newest of those and the one that says the boundary is holding: a whole feature arrived in 0.1.0 and stayed on the portable side of it (chapter 6). spice-web, counted with them then and spicemine's since, needed a second stub package, so it was left out rather than counted generously.
This is not a portability promise — mélangej targets ABCL and nothing else, and the HTTP layer cannot follow. It is evidence about where the boundary sits. The template compiler, the Markdown renderer, the HTML emitter and the message catalog are ordinary Common Lisp, and if this project were abandoned tomorrow those four files would still be useful on any implementation.
Where ABCL is its own thing
ANSI leaves choices to the implementation, and ABCL makes the JVM's. Three of them you will meet.
Characters are UTF-16 code units.
CL-USER> char-code-limit
65536
A character in ABCL is a Java char, which is 16 bits. Everything in the Basic Multilingual Plane is one character. Everything above it — emoji, the CJK extension blocks, musical notation, many historic scripts — is two characters, a surrogate pair, exactly as Java sees it. This is conforming: the standard sets no floor on char-code-limit. It is also the deviation most likely to bite a web application, because web applications are full of text people typed.
What follows from it, concretely:
(length "a?���b")is 4, not 3.- Reversing such a string swaps the surrogates and produces garbage.
- Truncating to n characters can cut a pair in half and emit a lone surrogate, which is not valid UTF-8 and which some encoders will refuse and others will silently replace.
mélangej does not hit this today: it has no truncation helper and never reverses user text, and both the HTML emitter and the template compiler copy characters through a run at a time without indexing into the middle of one. If you add "first 80 characters of the description", that is the function to write carefully — test it with an emoji, because substring is where this always shows up.
Fixnums are 31 bits.
CL-USER> most-positive-fixnum
2147483647
Past that you get bignums, which are correct and slower. Row ids, byte counts and timestamps-in-milliseconds all live comfortably below it; nanosecond timestamps do not. Conforming — ANSI requires only that most-positive-fixnum be at least 32767.
long-float is double-float. All four float types collapse onto the JVM's double. Conforming, and the usual arrangement.
Two smaller ones: Gray streams exist but are in abcl-contrib, so they need (require :gray-streams) before the package appears; and features carries :package-local-nicknames, which this project does not use.
The one that is not about the language
ABCL interprets code that arrives through load of a source file. It is not a conformance question — the standard permits an implementation to interpret — but it is the single fact about ABCL most likely to make you wrong about a measurement. The same page renders in 23.3 ms interpreted and 7.5 ms compiled: a factor of 3.1, for nothing but whether compile-file ran.
The distribution carries .abcl fasls and boot.lisp falls back to source so a working tree still runs without a build step. Chapter 11 measures it, and core/bench/README.md has the numbers in full.
So: which Lisp?
ANSI Common Lisp (X3.226-1994), all of it, as implemented by ABCL 1.9.2, plus twelve symbols for reaching the JVM. No subset, no dialect, nothing of our own invention in the language itself.
If you write Common Lisp, you can read every line of this. If you want to take a piece of it somewhere else, take one of the eight files that have no Java in them. And if you need something ABCL does not have, the answer is usually a jar rather than a language extension — which is the whole reason this runs on a JVM.