Templating

There are three ways to get HTML out of mélangej, and you can use all three in one application:

  s-expressions    spice-html, markup as Lisp data
                   the default; reach for it when the page has logic in it

  .html templates  spice-tmpl, Lisp inside an .html file
                   when somebody else edits the markup, or the page
                   arrived as HTML

  static files     Jooby's asset handler, no rendering at all
                   CSS, images, fonts, a built front-end bundle

The first is the house style and the rest of this chapter is mostly about it. The other two exist because "a page is a function" is a good default and a bad rule.

1. s-expressions, the default

HTML is written as s-expressions, and it escapes by default:

(html (:div :class "card"
        (:h3 (:a :href url name))
        (:p :class "hint" note)))

name and note can contain anything a person can type. They come out as text, every time, without anyone remembering to ask.

What you get

Escaping is the default, not a step. There is exactly one way to emit markup that has not been escaped — (:raw x) — and it is two characters long, so grep :raw is a complete list of where unescaped content enters a page. In spicemine that list is 101 entries, all of them ours: the icon set, the banner, glossary tooltips, the language switcher, a CSRF field, the Markdown renderer. None of them carries a visitor's text.

It is Lisp, so it composes. A list of nodes splices, which means a loop is a child:

(html (list :table
            (list :tr (list :th "repo") (list :th "engine"))
            (loop for (name engine vis) in rows
                  collect (list :tr
                                (list :td (list :a :href (repo-url name) name))
                                (list :td (list :span :class "chip" engine))
                                (list :td :class (when (string= vis "private") "warn")
                                      vis)))))

NIL disappears, so an optional attribute is a value instead of two branches of a string:

(list :option :value slug :selected (equal personal "1") slug)

T writes a bare attribute (disabled, not disabled="t"). A void element given children is an error, because the alternative is dropping them silently.

A page is a function. You can call it from a REPL and print the result; you can test it with string=; you can pass half of it to another function. There is no template context to assemble and no renderer to start.

Repetition becomes visible. This was the surprise. Inside a quoted string with ~a in it, duplication does not look like duplication. In s-expressions it looks exactly like what it is, and six functions fell out of spicemine's UI that nobody had thought to write:

(field label name &key type autocomplete hint placeholder)
(revoke-form action id label)
(credential-card title meta action id label &key chip)
(section-head label term)
(step-line n text)
(space-chips user slug role)

field means an autocomplete hint and a capitalisation rule cannot drift between two pages. revoke-form means the CSRF field cannot be the thing a hurried edit forgets. Neither is tidying; each removes a class of bug.

It is 187 lines with no dependencies, and 23 tests. The tests that look paranoid are each a bug that has reached production somewhere: a display name that opens a tag, an attribute value that closes its own quote, an ampersand escaped twice until a page fills with &.

What it does not give you

No template inheritance, blocks or slots. Composition is function calls, which covers the same ground, but it does not look like Django and nobody will arrive already knowing it.

No compile-time optimisation. The emitter walks a tree at runtime. A macro version that compiled constant structure into a single string is possible and is not written, because rendering was never the bottleneck (chapter 11).

No HTML parsing or sanitising of someone else's markup. That is a different job and a different file: spice-md renders Markdown by escaping every character first and only then recognising markup, so a <script> in a README is already <script> before the renderer looks at it.

The markup lives in Lisp files. Which is the one that mattered enough to build the next section.

2. Templates in .html files

The gap in the s-expression approach is not expressiveness, it is custody. The markup sits inside Lisp source, so handing a page to somebody who does not write Lisp means handing them a .lisp file, and starting from markup that already exists — a designer's file, a static-site export, something pasted out of a browser — means translating it before it renders at all.

spice-tmpl is the other shape: an .html file with Lisp in it.

<?params (title rows) ?>
<h1><?= title ?></h1>
<table>
  <?lisp (dolist (r rows) ?>
    <tr><td><?= (first r) ?></td><td><?= (second r) ?></td></tr>
  <?lisp ) ?>
</table>

(spice/tmpl:render "views/repos.html" "Repositories" rows)

Five forms, and four of them are punctuation:

• <?params (a b) ?> — the template's lambda list. Ordinary, so &optional and &key work.
• <?= expr ?> — evaluate, escape, emit.
• <?raw expr ?> — evaluate, emit unescaped. The one door, same as (:raw), so grep -c '<?raw' views/ is the same one-minute audit.
• <?lisp forms ?> — run; emit nothing.
• <?tr :key ?> — a message-catalog lookup (chapter 7), escaped.

It is not a template language

That is the whole design. There is no {% if %}, no {% for %}, no filter syntax and no inheritance keyword, because the language inside the tags is Common Lisp and a template language is mostly a worse Lisp you cannot unit-test.

This works because of how the file is compiled. The template is turned into the source text of one lambda — literal runs become write-string, <?lisp ?> chunks are spliced in verbatim — and that text is then read and compiled. So a dolist opened in one tag and closed in another genuinely brackets the text between them: after splicing it is one ordinary form. Nesting, let, handler-case, multiple-value-bind all work, and none of them is a feature anybody implemented.

Three consequences worth having:

It compiles to JVM bytecode, once. A template is not re-parsed per request and not interpreted per request; the first render compiles it and the cache keys on the file's write date, so editing markup takes effect on the next request and nothing else does. Chapter 11's 3× interpreted-versus- compiled gap applies to templates too, and this falls on the right side of it — measured there against both other paths.

A broken template is a Lisp error with a form in it. The condition is a template-error carrying the generated source, so a missing paren shows you the code it tried to compile rather than a parser's opinion about line 12.

Escaping is still the default, which matters more here than anywhere else: this is precisely the file you hand to someone who is thinking about layout and not about a display name with a <script> in it.

What it costs

A template is code. There is no sandbox — a template can call anything Lisp can call. So templates belong in the repository under the same review as everything else, and must never be user-submitted or fetched at runtime from anywhere a visitor can write. If that is your requirement, this is the wrong tool and a restricted evaluator is the right one.

It is slower than s-expressions, which are slower than concatenation. Chapter 11 measures all three on the same page, so the choice is made on a number rather than a preference.

It is still Lisp in the file. A designer can move a <div>, restyle it, and reorder the page. They cannot avoid seeing <?= ?>. That is a smaller ask than editing Lisp source and it is not nothing.

3. Static files

Sometimes the answer is not to render anything. Jooby's asset handler is in the core jar — no new dependency — so this is two lines:

(serve-static app "/static/*" "/var/lib/spicemine/static" :max-age 31536000)
(serve-classpath-static app "/assets/*" "assets")   ; packaged inside the jar

It is worth using rather than writing, because it does the parts that are tedious and quietly wrong in hand-rolled file handlers: content type from the extension, ETag, Last-Modified, conditional requests answered 304 instead of re-sending, Range requests, and directory traversal refused — GET /static/../../etc/passwd resolves against the root and 404s, which is the bug the hand-rolled version comes with. All of that is verified in the test suite rather than assumed.

:max-age is seconds and defaults to 0. Long max-ages are correct for fingerprinted filenames and wrong for everything else.

A built front-end bundle is this plus nothing: point serve-static at the build output, serve one HTML page from a route, and mélangej is a JSON API with a file server attached.

Why not Djula

Djula is a good template language and the obvious choice on ningle. It is not the choice here, for three reasons that are about this project rather than about Djula.

It would be the first Lisp dependency. The whole architecture is that there are none (chapter 1). The first one brings ASDF, Quicklisp and a transitive graph — and Djula's graph is written and tested mostly on SBCL. It probably works on ABCL. "Probably" is a thing you want to find out before a deploy, not during one.

The pages are not documents with holes in them. Every page here calls (tr :key) for its words, checks a right before rendering a control, and loops over database rows. A Django-style template pushes that logic either into the template, where it cannot be unit-tested, or into a context dictionary assembled only to be unpacked again.

The problem it solves is not the problem we had. Template languages earn their keep when non-programmers edit templates, or when markup must be reloaded without restarting. Neither applies: the markup is edited by the same people who write the handlers, and the words — the part that genuinely gets edited by others — were moved to a message catalog instead.

cl-who has the right shape but does not escape by default, which is the same hazard with different spelling. spinneret is closer still and pulls in serapeum and parenscript for pages that contain no JavaScript.

Where the line is

format builds values: a URL, a timestamp, a class name. html builds HTML. spicemine has 35 of the former left and zero of the latter outside the emitter, which is the state to keep it in.

Forms

There is no form library: no schema, no binding, no validator, no @Valid. A form is an HTML form, a POST, and a handler that reads strings. What exists instead is four small things, and they are small because each one closes a specific hole.

ctx-form reads Jooby's parsed map, never the body.

(defun ctx-form (ctx)
  (handler-case (java:jcall "formMap" ctx)
    (error () nil)))

That handler-case is not politeness. Jooby parses a form body itself, and by the time a handler runs the request stream is spent — read the bytes and you get an empty form and a POST that fails with nothing in the log. This is the single most expensive thing to rediscover about form handling on Jooby, which is why the function exists at all rather than pages calling formMap themselves.

form-value returns a Lisp string or NIL, and accepts an alist as well as the Java map, so a handler can be called from a test with '(("name" . "ada")) and no HTTP at all.

A field that is absent and a field that is empty both come back as something falsy-ish — NIL and "" — and they are not the same thing for a checkbox. An unchecked checkbox sends no field. (form-value form "personal") is NIL for unchecked and "1" for checked, so the test is spice/db:as-bool, not equal against a string. Comparing a flag to "1" is how a Postgres deployment read a permission as false that SQLite read as true (chapter 2's cousin, written up in DATABASE.md).

csrf-field on the way out, csrf-ok-p on the way in. Every POST in the UI carries a hidden field; every POST handler checks it with a constant-time comparison, and a missing session fails rather than passes. The emitter takes the token as an argument on the two anonymous forms (login and signup, where there is no session yet) rather than falling back to one, because an empty hidden field is the kind of bug that looks like it works: the form posts, the check fails, and the message blames the reader for a stale form.

fail-to sends the reader back where they were with a catalog key, not a sentence — the message has to exist in both languages or the build fails (chapter 7). It arrives back in the query string, which means it is attacker-supplied text on the way in, so it is escaped like everything else and rendered as words: never as a link, never as an instruction to act.

The rule that is not about forms

def-post (chapter 4) refuses an anonymous POST and a bad CSRF token before the body runs. The body then re-checks whatever right the form was rendered from. Not because the first check might be wrong, but because the form is not the control:

a hidden button is a hint and never a control

If a page did not render the delete button for you, nothing stops you posting to it anyway. Three handlers in spice-ui revoke credentials, and each one re-establishes the right to do so from the session and the row, not from the fact that a POST arrived.

JavaScript

The UI has one <script>. It is ten lines, it lives in spice-banner.lisp, and all it does is ask the browser what time it is, because a request carries no timezone:

(defparameter +sky-script+
  "<script>(function(){try{ ... })();</script>")

With scripting off, nothing runs and the header strip keeps the server's idea of the hour. That is the whole of the client-side story: every other interaction in spicemine is a link or a form.

What that buys, and what it costs

Nothing to build, so there is no bundler, no node_modules, no lockfile and no step between editing a page and reloading it. Nothing to serve, so there is no cache-busting scheme and no asset pipeline. Nothing to break when scripting is off, which for a service aimed at people who distrust being mined is not a small thing. And a page is a function of the request, which is why a second instance is a configuration change (chapter 14).

The cost is that every interaction is a round trip and a full page. For this UI — forms, lists, a timeline — that is the right trade. For a diff view with collapsible hunks, or a file tree, or anything with live search, it will stop being the right trade.

If you do want JavaScript

Nothing in mélangej is in the way. Three things to know:

• serve-static and serve-classpath-static (chapter 11 measures them) will serve a bundle. A file is 26× cheaper to serve than rendering the same bytes, so a built front end costs the server almost nothing.
• spice-tmpl exists partly for this: if a front-end toolchain owns index.html, an .html file with a few <?= ?> tags in it is a template, and nobody has to translate the markup into s-expressions.
• The escaping default does not help you in JavaScript. spice-html escapes for HTML text and attributes. Interpolating a value into an inline <script> is a different grammar and an escape that is correct for one is wrong for the other. Pass data as JSON in a data- attribute or a <script type="application/json"> block and read it from the DOM.

Chapter 6 is the part of this that exists today: Datastar fragments, where a route answers an interaction with two SSE events instead of a document, and spice-tmpl and the emitter render the fragment with the same functions that render it inside the page.