Implementation status
90-roadmap.md is the phase-level plan. This document records what is implemented. Where the two disagree, this document describes the current state.
The command reference is 60-cli.md; task-oriented how-tos are in 63-guides.md.
Approach: connect a minimal build end to end first
The roadmap orders Phase 1 (core) to completion before Phase 2 (generation), but the implementation deviated from that order exactly once: parser → evaluation → target graph → action graph → ninja generation → actual C compilation were connected first, each in minimal form.
Two reasons:
- e2e verification becomes available from the start. “Compile real C, run it, and check the output” exists as a test, and every later change builds on that premise
- The constraints that cannot be retrofitted (docs/20-architecture.md section 2) get validated early. Lossless CSTs and span retention can only be validated once things connect all the way down to action generation
The incremental query engine and the persistent store were then plugged in
afterward. The insertion point is confined to
dowel_model::session::Session; both are in place (below).
Crate layout
| Crate | Responsibility |
|---|---|
dowel-support |
spans, source maps, diagnostics, structured logging, JSON output, SHA-256 |
dowel-syntax |
lexing, lossless CST, error-tolerant parser |
dowel-query |
memoization, dependency tracking, early cutoff, durability layers, cancellation |
dowel-store |
the on-disk store (append-only value log, fixed-length index, single writer) and the per-user probe-fact database |
dowel-eval |
typed values with provenance, expression evaluation, schema and merge semantics, configuration specialization, value serialization |
dowel-model |
package loading, targets, the dependency graph, interface merging, why |
dowel-build |
glob expansion, the action graph, the backend layer (ninja / direct / make / graph), compile_commands.json, execution |
dowel-lsp |
the language server: JSON-RPC framing, diagnostics publishing, hover |
dowel-cli |
the dowel binary |
dowel-up |
the dowelup binary: acquiring, pinning, and switching dowel itself |
Real-world test material lives in tests/projects/ (realistic fixtures) and
examples/ (the documented examples).
Implemented
Syntax (dowel-syntax)
- Lexing in which every byte belongs to exactly one token; whitespace, newlines, and comments are retained
- A lossless CST: walking the tree and concatenating reproduces the input (checked continuously by tests)
- Error tolerance: syntax errors do not stop parsing; an
Errornode is left and parsing continues. Recovery always consumes at least one token, guaranteeing loop progress - Table headers, array tables, key-values, arrays, inline tables, function
calls,
match, postfixwhen, namespace references - Robustness tests: over every prefix of real manifests, single-character deletions, and delimiter insertions, no panics and losslessness holds
Incrementality (dowel-query)
- Memoization and dependency tracking; keys read during a query are pushed onto the running frame
- Early cutoff: if recomputation fingerprints the same as before, dependents are not invalidated
- Durability layers (
Low/Medium/High); stable layers skip dependency traversal entirely - Cancellation, checked at query boundaries and propagated via
Result. The release profile ispanic = "abort", so unwinding is not used - Re-setting an input to identical content does not advance the version (judged by content, not by mtime)
- Memos retain the computation procedure, so chains of derivations can be
revalidated without recomputation. As a side effect, procedures can read
values only through
Db, and purity is enforced by types
Session reads files through this engine (dowel_model::query).
Session::reload re-reads the disk, but files whose contents did not change
are not re-lexed. The degree of reuse is observable via
Session::query_stats.
Early cutoff cannot help on per-file queries in principle (values contain
spans, so any change to the text changes the value). Where it does apply is
the per-target derivations (interface and compile_env), whose
fingerprints come from span-free summaries
(ADR-0011). A comment-only edit never
reaches merging. The path that displays provenance spans (dowel why)
bypasses the memo and redoes the merge on the spot.
Persistence (dowel-store)
.dowel/cache/<format-version>/ holds lock / values / index.
- An append-only value log and a fixed-length record index. The index needs no parsing to scan, and value bodies are not read until needed
- Index replacement is a temporary file +
rename. A same-directoryrenameis atomic; readers see either the old or the new - The writer is restricted to one via
flock(std::fs::File::try_lock). A process that cannot take the lock reads only and writes nothing - On load, records pointing outside the value log — and any trailing fragment — are discarded. After truncation or a mid-write crash, opening never fails
- Input change detection compares
(mtime, size, inode, ctime)and takes a content fingerprint only when they differ dowel cache info/dowel cache gc
Probe facts (dowel-store::facts, dowel-build::probe)
What dowel asked a tool and what it answered, recorded in the user’s
cache ($XDG_CACHE_HOME/dowel/facts/<format-version>/) rather than the
project’s (ADR-0028, docs/20-architecture.md
section 9).
- Outside the project because a fact belongs to the tool: the same compiler
gives the same answer in every tree, and under
.dowel/cache/the question is re-asked once per tree — the top of the durability hierarchy living in the most volatile place - The key carries the tool’s identity (path, size, mtime) beside the
question, which is why there is no invalidation mechanism: replace the
tool and the key changes, so the stale fact is unreachable.
cache gccollects old format versions - Only questions that start a process are recorded:
-dumpmachineand--version. ScanningPATHis a fewstatcalls, and recording it would cost more to keep honest than it saves. A recorded resolution is still checked for existence, since a tool can be removed withoutPATHchanging - “It did not answer” is recorded too —
clhas no-dumpmachine, and without recording the silence every run asks again - Its first reader replaced an unrecorded input: the host triple used to be
assembled from the OS and architecture dowel itself was compiled for, so a
machine whose compiler says
x86_64-pc-linux-gnuwould treat that spelling as a cross target and demand a runner.configurenow asks the host C compiler, and both spellings count as the host - Unwritable is not an error, the same judgment as the store: what is lost is the saving, not the answer
Session records the files it read into the store, and the next process
checks against that record to judge changes. The verdicts
(UnchangedByStat / UnchangedByContent / Changed) appear at
--log-level=trace.
Only evaluation results (Evaluated) are stored; per-target derivations stay
in the in-process memo. The reasons are in
ADR-0012. Storage is limited to files that
evaluated without emitting a single diagnostic.
Restoration is keyed on a matching content fingerprint. On a match, the file
is neither lexed, parsed, nor evaluated. A per-run summary appears at
--log-level=debug:
store: wrote 1 values, restored 3, skipped 0 with diagnostics
FileId is the hash of the normalized path, so the same file has the same
identifier across processes (ADR-0009).
Restoring stored values requires no renumbering pass, and a restored document
carries its own FileId, so key collisions are detectable on the value side.
Serialization lives in dowel_eval::codec: length-prefixed bytes covering
every type in Document. A failed restore returns None, and the store
treats unreadable values as absent. Whether the format version mismatches,
the file was truncated, or something external rewrote it — the result never
changes, only the speed.
Evaluation (dowel-eval)
Value = { type, data, provenance }. Provenance is a constituent of the value and traceable to the rootPathis a distinct type fromStr; the language provides no string concatenation for building pathsCfg<T>: resolution ofmatchand postfixwhenis deferred to specialization, so switching--releaseor--targetdoes not re-run manifest evaluationglobis not expanded during evaluation either: scanning at evaluation time would mix in the current file system — an unrecorded input- Merge rules belong to types:
union/append/error_on_conflict/must_equal/replace/max - Features are additive, and exclusivity is declared
(ADR-0021).
[features] exclusivetakes sets of features that must not be active together; two or more of a group active for a package isconflicting-features, naming them and where each came from (defaultis the one that gets forgotten, so it is called out with--no-default-features). Nothing is inferred — dowel cannot see that two sources define the same symbol. Without it, choosing an implementation with stackedwhens compiles both: abinfails with the linker’smultiple definition, and alibsucceeds, keeping whichever archive member the linker reached first (issue #82) pkg.name/pkg.versionare package constants readable in value position (ADR-0020). They are the only namespace that can appear as a value, and the only one refused as amatchscrutinee — a package’s own version is not an axis a build varies along. They resolve at specialization, not at evaluation: evaluation results are stored keyed by file content, and adowel.builddoes not change when itsdowel.toml’s version does, so substituting earlier would keep a stale version in the store (issue #80)- A
definesvalue’s type decides its-Dform: aStrbecomes a C string literal, anIntorBoola bare token. A version arriving as a bare0.4.0could not be passed to%s, which would leavepkg.versionunusable for the case it exists for - An
abilabel may name a boundary instead of a language:cmatches every label and never replaces one (ADR-0019). Without it, a C library and a C++ consumer each stating its own language honestly produce different labels and the build is refused, and the way out is for the consumer to copy the library’s label — at which point the label stops describing an ABI (issue #78). The exemption belongs to the ABI label vocabulary, not tomust_equal, which still means equality everywhere else - Exhaustiveness checking of
match: closed-domaincfgkeys require full enumeration, and open-domaincfg.targetrequires_ - The strictness of
dowel.tomlis imposed by validation, not by a separate grammar. Its top-level tables are a closed set: an unknown one isunknown-table, and a name fromdowel.build’s vocabulary is told where it belongs rather than skipped (issue #74).[policy]remains accepted as documented-reserved
Model (dowel-model)
- Loading multiple packages by following
path,git, andversiondependencies. An entry names exactly one source: none isincomplete-dependency, two or more isconflicting-dependency-source. The second half is what the rule was missing — one of the declarations would never be read, and nothing said which one won, so apathleft behind while switching togitkept building for whoever still had the tree (issue #79) - git dependencies are pinned to a full 40-digit commit sha (anything else
is
unpinned-dependency) and fetched once into.dowel/deps/<name>-<rev12>/by delegating to thegitcommand; the checkout is placed atomically with a completion marker, and later runs never touch the network. Because the rev pins the content exactly, no lock file is involved - Archive dependencies name a
urland asha256(ADR-0029), which is how most C is actually distributed. The hash is required for the same reasonrevis: a URL is a name, and the bytes behind a name change. Fetching and unpacking are delegated (curlorwget, thentar) but verification is not — the tool that computes SHA-256 differs per system, and a pin that can only be checked where a particular tool exists is a weaker promise than a pin, so it is implemented in-tree (dowel_support::sha256, pinned by the published test vectors). The archive is verified before unpacking, since unpacking lets the archive decide where bytes land. Layout follows the git checkout:.dowel/deps/<name>-<hash12>/with a completion marker written last. One top-level directory is stripped by looking rather than by declaring astrip_components versiondependencies resolve through the system pkg-config (ADR-0015): the constraint is a minimum (--atleast-version), and--cflags/--libsbecome the public flags and link flags of a synthetic external node. Failure isunsatisfied-dependency. Each resolution is reconciled againstdowel.lock— appended when new, silent when matching,lockfile-drift(never a silent rewrite) when differing. Editor sessions skip resolution entirely: the LSP starts no external processes- Diagnostics for unknown properties and type mismatches against the schema (with suggestions)
- The configuration vocabulary gained
target.os/target.arch, derived from the target triple and finite, somatchon them is exhaustiveness-checked (ADR-0026, part of Q1). Before that the target could only be reached as a free-form triple, and the word that read like the OS —host.os— meant the build host, so the obvious spelling compiled and selected the wrong sources (issue #115).otheris in both domains because--targettakes any string, and a domain that cannot be closed brings back the_arm - The executable’s spelling follows
target.os:bin/<name>.exefor a Windows target, decided in one place so the runner,artifacts,inspect,dowel debug, thebuilt:line, and the freshness fingerprint read the same value (issue #112). While they did not, the build succeeded and every later stage was handed a path that did not exist — and, because “the output is missing” and “it has not been built yet” are the same state, the incremental build never converged: relinking every run, silently, with only the elapsed time to notice by - A target’s name is unique within its package across kinds:
[lib.foo]beside[bin.foo]isduplicate-target, naming both sites (issue #114). The name keys three separate things —target("..."), the<package>:<target>label, andobj/<package>/<target>/— so two of a name meant apublicblock reaching nobody, a graph whose steps could not be told apart, an ambiguity whose own diagnostic suggested a spelling that was still ambiguous, and, once both compiled the same source, two rules writing one object path (which surfaced in ninja’s words, not dowel’s). Qualifying all three by kind would buy only the coexistence oflibfoo.aandfoo - The
interface(T)/compile_env(T)split:privatedependencies affect one’s own compilation but do not propagate to dependents - Feature flags make dependency-graph edges appear and disappear. Features
are per package (ADR-0017): the active
set is carried as
<package>/<feature>,feature.<name>is answered qualified by the package whose manifest declared the value, and a[features]value spelleddep/featforwards into that dependency. A forward to an undeclared dependency isundeclared-dependency; one naming a feature the dependency does not declare isunknown-feature. Loading and resolution iterate to a fixpoint because a forwarded feature can activate an optional dependency - Optional dependencies (
optional = true) that are not enabled are not loaded — not just the edge but the node is absent. Feature selection is fixed beforeSessionloads - Cycle detection (iterative DFS, the path shown in a note)
dowel why— propagation-path display (text / json)
Build (dowel-build)
globexpansion (*/**/?), sorted lexicographically to be independent of traversal order- The action graph (compile / archive / link)
- C and C++. The compiler is chosen per source by extension (C++:
.cc.cp.cpp.cxx.c++.CPP.C;tc.cxxdefaults toc++, overridable with[toolchain] cxx). A link whose closure contains any C++ translation unit uses the C++ driver, so the C++ runtime is linked even when the binary itself is pure C. The C++ toolchain is only required — and only probed — when C++ sources are present - Per-language flags:
flagsapplies to every language,c_flags/cxx_flagsfollow it and reach only their own language - The language standard is typed:
c_std/cxx_stdtake a value from a closed, ordered vocabulary and merge with themaxrule, so the highest standard in the closure wins (ADR-0016). A value outside the vocabulary isunknown-standard, checked where it is written (matcharms andwhenbranches included). The generated-std=precedesc_flags/cxx_flags, so an explicit flag still overrides it — the escape hatch for GNU dialects, which are deliberately not in the vocabulary link_flagsacceptPathelements, which expand to absolute paths. That is how a linker script inside the package is named (["-T", file("ld/app.ld")]): the link runs in the build directory, so a relative string never reaches it, and the language has no string concatenation with which to build an absolute one (issue #70)[package] targetsdeclares the triples a package is for. Any other triple — the host included — is refused withunsupported-targetbefore building, so a bare-metal tree does not quietly produce an x86-64 “firmware” image when--targetis forgotten. Undeclared means any triple, which keeps a package that builds for the host while swapping tools when cross-compiling expressible; the declaration is deliberately separate from[toolchain.<triple>](issue #71)link_flagsride the link closure,privateincluded: a static archive cannot carry its own link requirements, so the flags a library declares — and the--libsof aversiondependency it keeps private — reach the final link the same way its archive does. Thepublic/privatesplit keeps controlling translation propagation only, so “do not leak the headers” and “stay linkable” hold at the same time (issue #56)[<kind>.<name>.artifacts]derives files from a target’s artifact — the step embedded work needs after linking (objcopy -O binary/-O ihex, a stripped copy). Each entry names a toolchain tool by name and its arguments; the input and output are appended positionally (ADR-0008), the output being the artifact with its extension replaced. The transforms are ordinary graph nodes: they are produced bydowel build, skipped when the input has not changed, performed by the tool the triple selects, and rebuilt when the declaration changes. The tool is probed only when a declaration uses it (issue #60). A derived file appears whenever its target’s artifact does — a library reached only as a dependency keeps producing it (issue #64); nothing consumes a derived file, so it has to be named as a default explicitly or the backends would produce different trees[<kind>.<name>.inspect]declares reporting tools (size/nm/objdump/readelf) run bydowel inspect. An inspection produces no file, so it is deliberately outside the build graph: nothing about it can be up to date. The tool is named, not spelled out, so a cross build reports with the triple’s tool; output is passed through unparsed and a nonzero exit fails the run, which is how a budget check is expressible without dowel knowing any tool’s output format (issue #60) Inspection tools that produce no file (size,nm,objdump) are not expressible yet- The archiver is part of the toolchain:
[toolchain] ar(defaultar, also per-triple in[toolchain.<triple>], configuration keytc.ar) names the tool that creates static libraries, so cross builds do not fall back to the host’sar(issue #50). It is probed — and required — only when the build produces an archive; because the name is part of the action’s command line, changing it rebuilds the archive - The tool set is table-driven (
dowel_eval::config::TOOLS): the[toolchain]keys, thetc.*configuration vocabulary, the defaults, declaration copying, and thetoolchain-mismatchcomparison all follow the one table, and amissing-toolchainprobe helper is shared. Keys outside the table areunknown-propertywith a suggestion — a misspelled tool would otherwise silently fall back to its default (issue #59). Adding a future utility (a disassembler,objcopy, …) is one table row plus the plan-stage site that uses it — only when a tool is required stays a per-use-site judgment (the C compiler always, C++ when C++ sources appear, the archiver when an archive is produced) - A toolchain also has an argument style:
gnuormsvc(ADR-0027). Declaring a tool’s name was not enough to use it — the arguments dowel assembles were spelled Unix-only, so namingclproduced a command line noclcan read, and-MD(a request for a dependency record) is a valid MSVC flag meaning “link the dynamic CRT” — a choice of ABI, and the very flag00-overview.mdcites under “no single ABI” (issue #113). The style is derived from the triple (*-msvc) and[toolchain] styleoverrides it; it also decides the tools’ defaults (ar→lib) and addslink, which is empty under GNU (the driver links) andlink.exeunder MSVC. Only what dowel assembles is spelled per style — a user’sflagspass through untranslated, because a table of flag equivalences would be knowing the compiler. Header dependencies differ in mechanism: MSVC prints/showIncludeslines instead of writing a record, so whoever runs the compiler folds them into the same.d, and everything that reads the record stays style-agnostic. The cost is that under MSVC the record is not shared across backends (ninja keeps its own), so switching backends costs one recompile. Whether an MSVC build then succeeds is unverifiable here — there is no Windows CI — so the checks put a fakeclon the path and read the assembled command compile_commands.json(argumentsarray form)- The output stage is a backend layer over one neutral build graph
(ADR-0018). Four backends:
ninja(default),direct(in-process, sequential, mtime-based freshness reading depfiles),make(generates aMakefile), andgraph(writesbuild-graph.jsonand builds nothing). Each receives aBuildGraphand nothing else — the same value the document serializes — so a fact missing from the format is a broken build rather than a documentation defect. Adding a backend is one row inNAMESand one trait implementation build-graph.json(14-build-graph.md) is the interchange format for a backend outside this repository: versioned, parseable back into an equal graph, and the same documentdowel graph --kind=action --format=jsonprints. There is one JSON description of an action graph, not two- The record of “which command produced this output” belongs to the layer,
not to a backend, so it stays consistent across switching between them. It
is merged into the previous record rather than replacing it: an output
the current invocation did not plan is still the product of the command
last recorded for it, so a narrow call (
dowel test,dowel build <name>) does not make the next full build redo untouched work (issue #69). Header dependency records (.dfiles) stay on disk and are shared between the backends — ninja is not allowed to fold them into.ninja_deps(deps = gcc), because a record private to one backend makes the next conclude “up to date” with no dependency information at all, silently keeping stale artifacts (issue #41). As a backstop, the direct backend treats an output whose declared depfile is missing as stale instead of fresh makehas limits ninja does not: it cannot name a path containing whitespace,:,#,$,%,;,=,\,*,?,[, or]. The backend refuses such a build, naming the path, instead of writing a makefile that quietly builds something else- Per-configuration build directories. The identifier is folded to one path
component (anything outside
[A-Za-z0-9_.+-]becomes--), so a feature name containing/cannot split a configuration across two levels (issue #68) [toolchain.<triple>]indowel.toml— toolchain selection follows--target, the same shape as[runner.<triple>](issue #42). A target triple with no declared toolchain is refused before building withmissing-toolchain, next tomissing-runnerin spirit: building host artifacts under a foreign triple’s name would report the configuration mistake one stage later (as a runner’sInvalid ELF image, or not at all). Full toolchain descriptions (sysroots, probing) remain Phase 5- Transfer for
[runner.<triple>](transfer/remote_dir/host): when the target machine cannot see the build machine’s file system, artifacts are carried over before launch. Paths are not written in the manifest; the implementation appends them (ADR-0008) [runner.<triple>]— an execution wrapper per target triple.dowel test --target=<triple>launches through the wrapper transparently. If the triple differs from the host and no runner is declared, launch is refused with a diagnostic beforehand — afterward it would surface asExec format error, reporting a configuration mistake as a test failuredowel debug <target>[/<case>]— builds the target and starts the declared debugger on its artifact, package root as the working directory (ADR-0024).debugis a toolchain tool (defaultgdb), so a cross build names its own the way it names its compiler, and it is probed only here. The positional argument reaches a case as well, resolved asdowel testresolves it, carrying itsargs/env/cwd(and a harness’srunplus the discovered name) into the session — a debugger is wanted for passing cases too, and routing every path through the failure record meant failing on purpose to create one (issue #110). Debugging another triple’s artifact needs a stub, and the runner declares it —debug_argshosts the program,debug_connectsays where to attach. dowel does not parse the runner’s flags, so neither is derivable from the other; a cross target declaring neither is refused (missing-debug-stub) rather than pointing a host gdb at a foreign binary, and one declaring only half is told which half (issue #109).debug_argsgoes before the runner’sargs, sinceargsmay end with the flag that takes the artifact (-kernel) and anything inserted after it is eaten as that flag’s operand (issue #107).--dapwrites the launch configuration to stdout and starts nothing. Nosubstitute-pathis emitted: nothing is remapped yet, so there is nothing to compensate for (docs/30-devexp.md 2.1)dowel test --debug-failed— reopens the failing case under the debugger with its declaredargs,env, andcwd(docs/30-devexp.md 2.3). The join it was described as: the test job becomes the debug launch, through the sameprepareasdowel debug. Needs the selection to come to exactly one case (a debugger attaches to one process; several failures are listed with a note to name one), narrows like--faileddoes, and composes with--dap, which then carries the case’s arguments and environment. The verdict record is not updated — a debugger session is interactive, not a judgmentdowel bench— buildsbenchtargets and measures whole-process wall-clock time, min/median over--iterationsruns (default 10; ADR-0025). No framework is imposed and none is read — there is no C convention for measurement output.[bench.<name>.cases]reuses the test-case shape minusshould_fail(a benchmark is measured, not judged; the property is refused). Always sequential — parallel measurements are each other’s noise. Speed has no verdict: a failure is a run that could not be completed, and a failed measurement reports no numbers at all. JSON (bench-result) carries times as integer microsecondsdowel test— launches tests and judges pass/fail by exit status. There is no test harness; the C convention (“exit status 0 means success”) applies. The working directory is the package root unless a case declarescwd. Only failing tests’ output is shown[test.<name>.cases]registers several tests from one binary (ADR-0022):argsdistinguish them, and each carries its ownenv,timeout,should_fail,labels, andcwd(issue #95 — the default, the package root, is a promise now, not an observation). A case’s label is<package>:<target>/<case>, and selection (--label,--failed) and reporting operate on cases. A target with no cases is one test, unchanged. Nothing is imposed on the binary — dowel never asks it what cases it contains, so which C test framework a project uses stays the project’s decision- A case may itself be conditional —
match/whenapply to the case, not only to the values inside it (issue #92). The strong use is a case that must not exist for some target at all; expressing that by splitting the[test.<name>]instead would add translation units, which is what cases exist to avoid. Every arm is validated, since the condition is not resolved until specialization - The declaration is checked: a case name that breaks the
<package>:<target>/<case>label grammar isinvalid-name(issue #97), a non-positivetimeoutisinvalid-value— it would silently mean “wait forever” (issue #96) — an emptycasesblock isempty-blockrather than one bare run (issue #99), a type error underlines the key that is wrong rather than the whole case (issue #101), and writing a case as its own table says what the right shape is (issue #98) [test.<name>.harness]is the other shape (ADR-0023):listarguments make the binary print its case names, one per line, andrunarguments precede the name when running one. dowel knows no test framework — only those two argument lists — so a framework whose listing differs needs a wrapper in the project that chose it. The listing runs at test time through the same runner as the tests; failing, timing out, or listing nothing is a failure of that target, never a silent zero. A listed name is held to the same label grammar as one written in the manifest, and a name that breaks it is reported like any other listing failure: the contents of the line stay uninterpreted, but the grammar of an acceptable name is one whichever entrance it came through — and the entrance the user cannot edit is the likelier source of a broken one (issue #108).casesandharnesstogether isconflicting-declarationtimeoutpollstry_wait; the standard library has no wait with a deadline and the core takes no dependencies. The kill reaches the test process only, so a test that spawns grandchildren leaks them--fail-faststops at the first failure. The default keeps going (the full picture matters); when cut short, the summary reports how many were not run--failedreruns only what failed last time. Verdicts persist in the build directory; verdicts of targets not run are kept- Selection works on cases: a positional argument names a target or a
case (
app:unit/parse, the spelling the output prints),--labelpicks by declared label, and--failedreruns what failed. A selection that matches nothing fails — the report goes to stderr where a CI log buries it, so the exit status has to carry it (issues #89 / #91 / #93). A tree with no tests, and--failedwhen nothing failed, stay successes: neither contradicts what was asked --no-runbuilds and then lists the cases that would run, with their labels,should_fail, andtimeout— the only way to see what exists without running it, and what makes the label vocabulary discoverable (issue #94). It launches nothing, so a cross target needs no runner--test-jobs=<n>runs tests in parallel. The default is sequential: C tests may use shared resources (the same working directory, fixed ports, output files), and a parallel default produces order-dependent failures. Display is always in request order. A case with its owncwdno longer shares the first of those- A case killed by a signal fails,
should_failor not: what that declares is a nonzero exit, and a crash is not one (issue #88).should_failis written where broken input is fed in, which is also where a crash is most likely — treating the two alike turns the defect most worth catching green --no-run/--nocapture, and--message-format=jsonfor onetest-resultper line. The target and the case are separate fields, so grouping by target does not mean splitting a string, and the three ways to end without an exit status each have their own field:timed_out,signal,launch_error(issue #100)
Language server (dowel-lsp)
dowel lsp speaks LSP on stdin/stdout. The editor is the starting party,
which distinguishes it from the resident daemon rejected by
ADR-0002.
- Full-document sync:
publishDiagnosticsin response todidOpen/didChange/didSave/didClose - Beyond parsing and evaluation, diagnostics include the type checking
decidable from the single open file (
unknown-property/type-mismatch/unknown-kind, …), produced by the same implementation as the CLI. A check enforces that every code in the case table either reaches the editor or has a reason indowel_lsp::UNSUPPORTED textDocument/hover: property types and merge rules, each level of a table header, builtin function signatures, configuration key domains, and the tables that are not property blocks —cases,harness,artifacts,inspect,[runner.<triple>]. The source is the same tabledowel schema dumpreads, and the words naming those tables now live there too: keeping them in the type checker alone is what letcasesbe known to the checker while the dump and the editor said nothing (issue #90)- Diagnostic ranges are 0-based lines and UTF-16 columns; notes and fix-suggestion text are folded into the body
dowel.tomlis recognized by name and held to strict TOML validation (ADR-0003)- JSON-RPC framing and body reading are in-house (ADR-0007); an unreadable body is discarded and the next one read, so one bad message does not drop the connection
Documents inside a package are diagnosed to the same depth as check
(ADR-0010): a workspace model is built per
change — the open buffers overlay the disk (the buffer is the source of
truth), and the model is loaded from every open manifest’s directory, so a
document edited as someone’s dependency gets its diagnostics (e.g. its half
of a merge conflict) from the dependent’s model — and then the plan stage
runs over it, producing glob-expansion, path-resolution, and
toolchain-existence diagnostics (empty-glob / unresolved-path /
invalid-source / no-sources / missing-toolchain) from real file-system
scans. Everything is read-only: the editor session never touches the
network (git checkouts are reused, not fetched), never reads or writes the
store, starts no external processes, and is created and dropped per
change — it is not a daemon. What remains excluded — fetching, --target
triggered checks, and system-package resolution — is listed with reasons
in dowel_lsp::UNSUPPORTED.
VS Code extension (editors/vscode)
Starts dowel lsp and relays diagnostics and hover to the editor, with
syntax highlighting for dowel.build (a TextMate grammar). Zero runtime
dependencies; framing and JSON-RPC correlation are in-house (the design is in
editors/vscode/README.md). Development happens inside the container via
editors/vscode/dev.sh, and the checks include an integration test that
talks to the real dowel lsp. It is not yet published to the marketplace.
Acquisition (dowel-up)
dowelup acquires dowel itself and pins a version per project
(ADR-0013; usage in
61-acquisition.md).
- Resolves specifiers (
stable/nightly/nightly-<date>/X.Y.Z/branch:/tag:/ sha) to a commit sha, builds a mirror checkout withcargo build --release, and places it under$DOWELUP_HOME/versions/<sha>/. History and network operations are delegated togit, building tocargo - Selection via
.dowel-version(pin) and the default. Launched under the namedowelit acts as a shim and execs the selected version; a leading+<specifier>picks directly among installed versions. Selection never touches the network - Pins contain only resolved shas. A hand-written name is not resolved; the
error points to
dowelup pin stablecannot resolve until a release tag appears upstream. Prebuilt binary distribution is not started (Q10)
Migration (dowel-build)
dowel migrate verify <compile_commands.json>compares a reference compile database against the plan, source by source, after normalization (-DX≡-D X≡-DX=1;-Iresolved against each entry’sdirectory; compiler name,-c/-o, and the-MDfamily ignored; configuration-level flags — optimization, debug info,NDEBUG— dropped from both sides, since dowel’scfg.optand the reference’s build type supply them independently (issue #54); remaining flags as a multiset). Differing ported sources fail the run; unported sources are reported without failing — porting is incremental (docs/40-migration.md 4)- Both reference forms are read (
argumentsarray and shell-quotedcommandstring). Output is text or--format=json dowel migrate import <cmake-build-dir>draftsdowel.toml/dowel.buildinto the CMake source directory from a File APIcodemodel-v2reply, refusing to overwrite. The draft is marked UNVERIFIED in a header comment (the favored shape of Q6; a machine gate on unverified targets remains open) and points atmigrate verify. Everything lands inprivateblocks — the public/private intent is unknowable from the File API — and sources are listed explicitly, not globbed, so the draft stays faithful to the extracted projection. Configuration-level flags from the CMake build type (-O/-g/-DNDEBUG) are not copied: dowel’s--configsupplies them, and copying them unconditionally would make a draft imported from Release produce optimizedNDEBUGdebug builds (issue #54)
Scaffolding (dowel-cli)
dowel new <path>generates a workingbinpackage, or a library with a passing test target under--lib. The skeletons matchexamples/hello, and e2e builds and runs every generated form, so they cannot silently rotdowel add <path>creates a library package in a subdirectory and appends the[[dependencies]]entry todowel.toml(an append preserves the existing text; array-table position carries no meaning in strict TOML). Wiringdeps = [dep("...")]into a target stays explicit — the command prints the line to adddowel add --git <url> [--rev <rev>]declares a git dependency. The manifest only receives a full 40-digit sha: a name (or omitted rev = HEAD) is resolved once viagit ls-remote, the same judgment asdowelup pin
Diagnostics and logging
- Severity, stable codes, multiple labels, notes, mechanically applicable fix suggestions
- Human rendering (rustc format) and
--message-format=json(one diagnostic per line) - Unknown names get edit-distance suggestions (properties, functions,
configuration keys, feature names,
matcharms, CLI options and commands) - The domain of feature names is defined by
[features]indowel.toml, checked both for references fromdowel.buildand for--features. The judgment is only possible after reading the manifest, so it lives in neither evaluation nor argument parsing - Per-stage timing, dependency-graph edges, and action command lines go to the log
checkruns through the planning stage (ADR-0010): glob expansion, path resolution, and toolchain existence cannot be judged during evaluation, andcheckemits the same diagnosticsbuildwould
What --log-level=trace shows (the material for tracing “why did this
argument end up like this” when debugging):
| Source | Contents |
|---|---|
session |
files read and their sizes, tables and key values evaluated, properties assigned to targets |
input |
per-input verdicts against the previous run |
query |
input changes and version advancement, files parsed/evaluated, memo validation vs recomputation |
graph |
edge resolution, topological order |
interface |
per-property counts of arriving values and merge results (both interface and compile_env) |
specialize |
which arm match chose, which elements when dropped |
glob |
files scanned with match/no-match, directories pruned, match counts |
plan |
resolved sources, includes, defines, flags; the full command line of every action |
exec |
why something was judged fresh; why something re-ran (which input was newer) |
test |
the list of tests to launch (before launching), their working directories and commands |
runner |
the declared wrappers and the command chosen for the configuration |
Verification
One entry point; local runs and CI execute the same thing. What each layer answers, and where a new test belongs, is in 51-testing.md.
make verify # run every stage, leaving results in .work/verify/
A mid-run failure does not stop the run; it proceeds to the end and fails
afterward. Results land in summary.md (for humans and the GitHub summary),
results.json (machine-readable), and logs/<stage>.log. CI
(.github/workflows/verify.yml) stores these as artifacts and prints the
summary into the job summary. Details in
50-development.md section 3.1.
Current breakdown (620 tests):
| Stage | Contents | Count |
|---|---|---|
fmt / clippy |
formatting check and lints (-D warnings) |
— |
unit-* |
per-crate unit tests | 330 |
syntax-robustness |
no panics and losslessness on broken input | 5 |
model-integration |
manifest loading through interface merging | 10 |
model-incremental |
counting what a reload did not recompute | 10 |
e2e |
compile real C and C++, run it, check the output | 206 |
scenario |
operation sequences over time (edit and rebuild, configuration switches, cross-process change detection and restore) | 24 |
fixture |
real-shaped projects (tests/projects/) end to end |
11 |
diagnostics |
diagnostics reaching the CLI (66 cases), applying fix suggestions, location presence, check scope, coverage tracking |
12 |
example |
build the real examples/hello and run its tests |
3 |
up |
dowelup resolution, acquisition, and switching against an upstream fixture |
3 |
docs |
link resolution, index consistency, and reference completeness | 6 |
startup |
startup-time measurement (informational; machine noise does not fail the run) | — |
The scenario / fixture / diagnostics layers were added later. Their
first runs surfaced the following four defects, none of which could appear in
the pre-existing layers:
| Defect | Why the existing layers could not catch it |
|---|---|
merging deduplicated by relative path only, dropping another package’s include/ once dependencies exceeded two levels |
the synthetic project has only one dependency level |
| the direct backend omitted the command line from freshness, missing flag changes | a single run never sees the second execution |
a directory in sources surfaced as the linker’s input file unused |
invalid-source had never been reached |
a nonexistent source surfaced as ninja’s no known rule |
unresolved-path had never been reached |
Measurements
The startup budget is under 10ms with nothing to do
(docs/20-architecture.md 5.4). Release build, a 2-package / 2-target
configuration, min/median of 20 runs. make measure produces these on their
own.
| Run | Min | Median |
|---|---|---|
dowel --version |
1.6ms | 1.7ms |
dowel check |
2.3ms | 2.5ms |
dowel graph --format=json |
2.0ms | 2.2ms |
Binary 1.2MB; 4 dynamic links (libc and friends). Currently inside budget.
The previous figures (--version 1.2/1.4ms, check 1.5/1.7ms, graph
1.4/1.6ms) were taken on a different machine and are not directly
comparable. The same-machine change in check is recorded in
ADR-0010.
The effect of storing evaluation results
check min/median measured on one machine before and after
ADR-0012, separated into runs without manifest
changes (where restores happen) and with changes (where stores happen).
| Subject | unchanged, before | unchanged, after | changed, before | changed, after |
|---|---|---|---|---|
examples/hello |
2.35/2.70ms | 2.24/2.44ms | 2.36/2.53ms | 5.21/7.22ms |
tests/projects/layered |
3.48/3.71ms | 3.28/3.60ms | 3.70/4.22ms | 7.48/9.35ms |
At the current fixture sizes, restoring saves 0.1–0.2ms: manifests are a few hundred bytes, and lexing + parsing + evaluation together are small next to the fixed startup cost.
Runs that store gain 3–5ms, all of it the two sync_data calls in
Writer::commit (measuring without sync makes the increase vanish). The cost
falls only on runs that changed a manifest — the runs that proceed to a
build anyway. Unchanged runs have nothing to write and skip the sync.
The savings scale with size; the sync cost does not. The break-even size cannot be measured with the current fixtures; the scale fixture (51-testing.md, “Future”) is needed.
Not implemented (deliberately deferred)
| Item | Standing |
|---|---|
| mmap-ing the index (currently read whole) | Phase 1; reading whole suffices up to thousands of records |
making loading and name resolution queries (Declared / Deps as derivations) |
Phase 1; today Session assembles them and passes them as inputs |
| the probe-fact DB | Phase 2 |
the template / toolchain kinds |
Phase 4 |
Meson introspect import |
Phase 3 backlog; CMake File API import and migrate verify are implemented |
language-server diagnostics that need fetching, --target, or external processes |
the editor session is read-only and host-targeted by design; the remaining exclusions are listed with reasons in dowel_lsp::UNSUPPORTED |
| cleaning up artifacts left on target machines; skipping redundant transfers | Phase 4; transfers run every time |
| a native registry / tarball dependency source | Phase 5; version deps delegate to pkg-config (ADR-0015) and dowel.lock records their resolutions — a dowel-run registry, if ever wanted, is a separate future decision |
prebuilt acquisition for dowelup |
Q10; today source builds only |
| automatic ABI label computation | Phase 6; today only must_equal verification of a hand-written abi. Nothing verifies that a surface declaring abi = "c" really is extern "C" — the claim is narrower and more checkable than a language label, and is what an IDL or a header scan would confirm (ADR-0019) |
| automatic composition of the ABI label from its components | Q2; c_std / cxx_std are now typed values the label can read (ADR-0016), but which components make up the label, and at what granularity, is still open |
Divergences from the design documents
Points where the implementation departed from the documents, made explicit. Whether to amend the documents is decided separately.
| Where | Document | Implementation | Reason |
|---|---|---|---|
| the consequence in ADR-0003 | “there will be two parsers” | one parser; dowel.toml strictness is imposed by validation |
the ADR’s rationale (third-party tools read it without a custom parser) is equally satisfied by validation, and a single tree keeps provenance and diagnostics paths simpler |
| types | defines : Map<Ident, Val> |
Val implemented as a type |
the document’s notation was adopted as-is |
abi |
ABI labels are computed | currently a hand-written string | computation is Phase 6; only the must_equal path is wired up |
| 30-devexp.md section 1 | args = ["-L", sysroot()] |
args : List<Str>; sysroot() cannot be written |
sysroot-based paths are Phase 4 (unimplemented-path-base); strings work first, widening to List<Val> when bases land |
| 50-development.md section 3 | CI runs in a --network none container built from dotfiles |
GitHub Actions runners (staying so for now) | the path for evaluating the dotfiles flake from this repository’s CI is not set up, and there is no present need. The checks are defined solely in scripts/verify.sh, so a migration later swaps only the workflow’s internals |