Skip to content

ADR 0005 — Rule format: Python builders in packs, expression templates for user overrides

  • Status: Accepted
  • Date: 2026-09-27
  • Decides: D3 in roadmap.md

Context

A rule turns a CST node into a Doc (doc-ir.md). Two audiences write rules:

  • pack authors, who write every rule of a language and need full control (comments, options, lookahead into the source);
  • users, who need to override one rule from their project configuration without forking the pack (extending.md). That configuration is committed to a repository and read by editors and CI, so it must not be able to run arbitrary code.

The T5 spike implemented the same JSON/JSONC formatter three ways, sharing the pipeline and layout helpers (174 lines) so that the variants differed only in how rules are written:

  1. Python builders: each rule is a function (node, ctx) -> Doc.
  2. TOML templates: each rule is a TOML string holding a restricted Python-syntax expression (names, literals, lists, calls of named functions, a if c else b), checked with ast when loaded and evaluated without eval.
  3. Purpose-built DSL: one rule per line (pair = if has_comments then source() else .key ": " .value), with a hand-written parser.

All three passed the same 14 golden fixtures with identical output. The spike is preserved in git history (commit 7dd35f26).

Builders Templates DSL
Rule definitions 11 lines 8 lines of TOML 4 lines
Engine none (plain dispatch) 84 lines 132 lines
Time, 153 KiB / 6,000 pairs 0.65 s 1.03 s 0.98 s
Override a single rule a Python function TOML text DSL text
Safe in repository config no yes yes
Syntax errors Python's, at import on load, with column on load, with line and column
explain provenance file and line (inspect) rule name only¹ file and line
Tooling (editors, types, lint) full expression inside a string none

¹ tomllib exposes no positions; T6 (tomlkit) can supply the line.

Observations:

  • Real rules need more than templates can say: container (comment attachment, trailing commas, object_wrap, align_values) was Python in every variant. Templates and the DSL are only as capable as the helpers written in Python beneath them.
  • The DSL's concatenation-by-adjacency makes a missing comma legal: join(hardline children) parses and fails only when the rule runs, with a Python arity message. Fixing that means a larger grammar, which is a language users must learn for one feature.
  • Templates reuse Python expression syntax, so users need to learn only the builder names, and the checker is 30 lines on top of ast.

Decision

  1. Language packs write rules as Python functions. A rule is Rule[C] = Callable[[CstNode, C], Doc] where C satisfies rainbow_fmt.rules.RuleContext (doc(node), text(node)); a pack maps node types to rules. Nodes without a rule are printed verbatim (rainbow_fmt.rules.source_text).
  2. Users override single rules with expression templates in [[rule]] tables of their configuration:
[[rule]]
language = "json"
select   = "pair"                                  # node type
template = "[field('key'), ' : ', field('value')]"

Templates are checked when the configuration is loaded (rainbow_fmt.rules.templates). They can use the Doc builders, children, has_comments, field(name), source(), and the helpers a pack exports (JSON: container(open, close)). Every [[rule]] table is validated, including those for other languages. 3. The purpose-built DSL is rejected.

select is a node type for now; richer selectors (parent context, field names) can extend it without changing the template language.

Consequences

  • Pack authors get Python's tooling: types, debugger, tests, profiler. Builders were the fastest variant, which matters under ADR 0002.
  • Configuration cannot run code: templates have no attribute access, subscripts, operators, comprehensions, lambdas or keyword arguments, and can call only the names they are given.
  • A pack's template helpers are part of its public API and need documentation and stability like options.
  • explain can report module:line for pack rules (via inspect) and rule[N] plus file and line for overrides once T6 records positions.
  • Rules that bypass other rules are not overridable through them: JSON's align_values builds rows from keys and values directly, so a pair override does not apply to aligned objects.
  • The rule dispatch is recursive: JSON nested deeper than roughly 300–500 levels raises RecursionError (TASKS.md, follow-ups). tree-sitter itself handles 5,000 levels.
  • A pack that needs a hook that is not an expression (the hook key sketched in extending.md) must be installed as code, not configured; [[rule]] rejects unknown keys.

Alternatives considered

  • Templates for packs too (rules.toml as sketched in extending.md). Rejected: the interesting logic stays in Python helpers anyway, so packs would be split across two languages, with slower evaluation and no tooling for the TOML half.
  • Python overrides in user config (a dotted path to a function). Rejected for configuration: it runs repository code in every editor and CI job that formats the project. It remains possible for installed packs.
  • Purpose-built DSL. Rejected: a new syntax to learn, the most code to maintain, and a grammar whose error cases (see the missing comma) need more design than the benefit warrants.