Skip to main content
A conformance contract asserts an adapter behaves; a fixture asserts the contract still contains those assertions; a ledger asserts every one of those assertions still ran.

Quick Start

1

Run the contracts against your adapter

Each port ships a runnable contract. Register your adapter and both the fake and the shipping adapter are held to the same cases.
2

Prove the contract can still fail

The fixture spawns the real contract against a deliberately broken adapter, one break at a time. It takes a single mode string.
The break modes plus the none control:
3

Check the floor and the control

contracts.test.ts counts passing cases from a real none run and asserts a floor per contract: secrets ≥ 14, storage ≥ 15, time ≥ 8, shell ≥ 35. The none control must pass green, so a fixture that failed for an unrelated reason cannot masquerade as proof.
4

Wire the assertion ledger

Each contract counts every assertion it makes and checks the exact total in its last case, so a deleted assertion turns the run red by name.
Declare assert with an explicit Ledger["assert"] type rather than destructuring — assert carries asserts signatures and TS2775 refuses those through a binding pattern. Register the count case last so every case above has already run, and use EXPECTED_ASSERTIONS + (realClock ? REAL_CLOCK_ASSERTIONS : 0) where a branch adds assertions.
This runs automatically inside contracts.test.ts under npm test. There is no flag to enable and no opt-in.

Why Three Layers

The contract asserts the adapter behaves; the fixture asserts the contract still catches a break; the ledger asserts every assertion still ran. A contract with no broken-implementation test is documentation with a test() around it. Deleting assert.ok(fired >= 2) from the time contract — the one assertion that catches setInterval becoming setTimeout, which stops every polling loop in the app after a single tick — left the suite at 1035 pass, 0 fail. Fourteen assertions across all four contracts could be deleted for free.

The hollowed case

A break-mode table catches a case that has been deleted. It does not catch a case that still runs and asserts nothing. Seven assertions across the conformance suites could be gutted to assert nothing and the build stayed green — a deletion guard never fires when the test() block is still there. One break mode per hollowable case is what closes that, because a break mode reddens the case by name. The order of defence is behaviour → contract → break mode per hollowable case: the contract asserts the adapter behaves, the fixture asserts the contract still catches a break, and the break mode ties each named case to a defect that must redden it.

The in-case hole the ledger closes

The break-mode fixture protects exactly one assertion per case — the first one to trip. Sweeping the four contracts one assertion at a time, 62 of 73 could be deleted with a fully green run: thirteen break modes were protecting only eleven assertions. The ledger closes that hole with a counting proxy over node:assert/strict and an exact count checked in the contract’s last case. Exact rather than a floor: a floor lets an assertion be swapped for a weaker one at the same arity, which is the same hollowing wearing a different hat. The ledger does not claim an assertion still asserts something useful — only that it is still there and still ran. contract-fixture.ts remains the thing that proves an assertion has teeth. The two are complementary; neither subsumes the other.

The Shell Reads What It Forwarded

The shell contract now asserts on the value the adapter handed the OS, not just whether the promise settled. ShellHarness carries forwarded(): readonly string[] — the URLs openExternal actually handed the OS, in order. A shell can validate one string and forward another, and doesNotReject cannot see the difference, so the harness surfaces the forwarded value and the contract reads it. forwarded() is required, not optional, so a shell cannot opt out of being checked.
A padded-URL case then reads it directly: a URL that passes the allowlist must reach the OS trimmed ("https://ok.example", never the padded input), and a padded javascript: must still be refused before anything is forwarded.

Host presence is part of conformance

An adapter that detects its host — the Tauri bridge’s isPresent() in adapters/src/tauri/bridge.ts — must apply a both, not either discipline: it reads present only when every function the host contract requires is present, and absent when any one is missing. A half-present global read as present makes listen call a missing transformCallback, which throws, is caught, and turns every subscription into a silent no-op — so any future adapter with a “detect the host” step owes the same rule. Pinned by "a HALF-present Tauri global is treated as absent" and its pair "a FULLY present Tauri global is treated as present -- the pair". See Native Shell → Presence detection.

Keyboard height

Two cases in the shared contract pin the Tauri shell’s viewport fallback and the single-writer handover to the native event; a third pin covers the Android insets global taking over from the viewport reading. The tests live at the tail of adapters/src/conformance/contracts.test.ts and drive createTauriShell({ view }) with a createFakeWindow().
See Shell & Adapters → The three-source keyboard model.

The Break Modes

Each mode breaks an adapter in one named way, and the matching contract case must go red by name. Two of the modes are shell modes — the shell contract now spawns its own break modes, on top of the case-count floor that still complements them. A none control runs unbroken, so a fixture that failed for an unrelated reason — a syntax error, a missing import — cannot masquerade as proof.

The secrets contract now has durability and presence branches

The SecretsPort contract grew two optional branches, supplied only by an adapter that can back them — a durable native store (reopen) and one that can count reads (readsOf). The web adapter, a module-scoped Map, claims neither, so it is not asked to. The four durability cases (adapters/src/conformance/secrets-contract.ts):
  1. a stored secret survives a relaunch
  2. presence survives a relaunch, not just the value — the settings row reads presence, so a store that kept the value but rebuilt its index would show “Not set” beside a stored key.
  3. a deletion survives a relaunch too — a store that persisted writes but forgot deletes would resurrect a credential the user removed.
  4. a FRESH store has none of another store's secrets — the control that stops a single process-wide Map from passing “survives a relaunch” while proving nothing.
The one presence case:
  • has() answers without reading the value — asserted on both a hit and a miss, so an adapter that short-circuits a hit but reads on a miss is still caught.
Housekeeping: EXPECTED_ASSERTIONS stays 16, with DURABLE_ASSERTIONS = 6 added when reopen is supplied and PRESENCE_ASSERTIONS = 4 added when readsOf is — 16 + (durable ? 6 : 0) + (presence ? 4 : 0), checked in the contract’s last case. The secrets case floor rises 9 → 14 and the break-mode table guard 15 → 18.

The storage contract now has atomicity and relaunch cases

The two new storage break modes have teeth because the contract itself grew two new dimensions (adapters/src/conformance/storage-contract.ts):
  • Atomicity. Concurrent writers against a live reader; every observation must be a whole payload or absent, never a prefix. This is what a durable adapter must pass to be drop-in for the web adapter. A control assertion requires the reader to have observed something — a reader that saw nothing would report no tearing while proving nothing.
  • Relaunch. Three cases — a value, the chat list, and a deletion each survive a second port over the same backing store, supplied by an optional reopen. A store that persisted writes but forgot deletes would resurrect a conversation the user deliberately removed, which is worse than losing one. This is what a durable adapter must pass to be durable. The in-memory fake does not claim durability, so it is not asked to reopen.
The ledger count follows time-contract.ts’s conditional-branch precedent: EXPECTED_ASSERTIONS = 17, plus DURABLE_ASSERTIONS = 3 when the adapter supplies reopen17 + 3 when durable, up from 15. A contract cannot quietly shrink: the last case asserts the exact total and reddens by name if an assertion is deleted.
The three relaunch cases are why the crash-recovery.test.ts relaunch proof one layer up means anything — they decide, at the adapter level, that “Your conversations are saved” is not a lie. See Chat Recovery → What a relaunch preserves.

The Tauri storage adapter

createTauriStorage is run against the same StoragePort conformance contract as the web adapter, through a strict stand-in host that throws on an unknown command or a missing argument. That proves the adapter — command names, argument names, reply handling, namespacing — and not that the filesystem is atomic. The filesystem is store.rs’s own tests, where there is a real disk to interrupt. A native host stores chats in the native store, not localStorage — pinned by a native host stores chats in the NATIVE store, not localStorage, and by the paired tauri storage: a written value reads back exactly with listIds returns only its own namespace.

The Shrink Floor

A case with no break mode could still be deleted, so contracts.test.ts counts passing cases from a real run of the fixture — not a regex over source text — and asserts a floor. Raise these numbers when you add cases. A drop means a contract lost coverage, and that is exactly the event worth a red build. Re-derive each per-contract floor from a real none run rather than adding by hand — a new break mode targets an existing case in some contracts and a fresh case in others. A separate guard pins the break-mode table itself: ADAPTER_BREAKS.length >= 18, raised as new modes land (>= 15 before the secrets durability and presence branches, >= 13 before the two storage durability modes). This counts break-mode rows, not passing cases per contract, so it moves independently of the per-contract floors above. The shell contract now carries its own spawned break modes (shell_scheme_case_sensitive, shell_negative_insets) as well as this case-count floor, so the Break Modes table above covers every contract, shell included.
Node 22 emits TAP when stdout is a pipe; Node 24 emits the spec reporter. The fixture is spawned with --test-reporter=tap so a test grepping for not ok behaves the same on both.

The Assertion Ledger

Where the shrink floor pins the number of cases, the ledger pins the number of assertions, so an assertion deleted inside a surviving case still turns the run red. Each describeXContract takes its own ledger() and uses counting.assert in place of the bare node:assert/strict. The contract’s last registered case reads made() and checks it against EXPECTED_ASSERTIONS. Delete one assertion → the count is short and the case fails by name. Add one → the constant must be updated deliberately, which is where you notice the new assertion probably also wants a break mode in contract-fixture.ts. Two contracts branch on a capability. The time contract’s realClock branch adds two assertions (REAL_CLOCK_ASSERTIONS = 2); the storage contract’s durable branch adds three (DURABLE_ASSERTIONS = 3), each checking EXPECTED_ASSERTIONS + (capable ? EXTRA : 0).

The ledger defends itself

The ledger’s own rawAssert.equal(made(), expected) is now the single point of failure, so contracts.test.ts proves it fails when it should.
  • Four tests, one per ledgered contract (secrets, storage, time, shell), read the contract file, remove its first single-line assertion, spawn runAdapterFixture("none"), and require it to go red on the ledger — matched by /made every assertion it is supposed to make/, not incidentally — then restore the file byte-for-byte in a finally.
  • A paired test asserts the untouched none run is green, so a runner that failed regardless could not masquerade as proof.
Two alternatives were tried and rejected. A temp tree breaks the contracts’ relative ../../../core imports — a “failure” would prove only that the file never loaded. A dynamic import() of a probe module needs top-level await, which the boundary scanner’s esbuild (iife) pass refuses — and loosening a gate to make a test of a gate work is the wrong direction.

Why The Fixture Builds Broken Adapters Inline

adapters may not import testing — enforced by tools/depgraph.mjs — so the fixture builds its broken adapters inline, the same way engines/src/contract-fixture.ts does.
The parallel engines fixture carries its own break modes for the AgentEnginePort contract — no_start, start_only, tool_failure_as_ok, empty_is_fine, decide_always_true, and abort_ignored. start_only opens a run and then stops with no terminal event; it is caught by “a run emits start first and exactly one terminal event last”. abort_ignored never consults the abort signal, caught by “aborting the signal stops the stream”. See Mobile Engines → Conformance.

Adding A New Case Or Break Mode

1

Add the contract case with a stable name

The name is matched by regex, so keep it stable once other code depends on it.
2

Add the break mode in contract-fixture.ts

Guard the defect behind a mode === "..." branch.
3

Add the pair to the ADAPTER_BREAKS table

contracts.test.ts maps each mode to the case name it must redden.
4

Bump the shrink floor for that contract by one

A new case raises the floor by one, so a later deletion is caught.
5

Update EXPECTED_ASSERTIONS by the number of assert.* calls you added

Do it deliberately. The count is where you notice the new assertion probably also needs a break mode in contract-fixture.ts — the ledger only proves the assertion still runs; the break mode is what proves it has teeth.

Testing utilities — the fake HTTP transport

The testing/ package ships the fakes engine and adapter tests drive against, and two of createFakeHttp’s behaviours are load-bearing enough that engine tests silently depend on them.
streamOf never emits an empty chunk. A zero-length chunk would let a bug in the SSE reader’s pending-CR handling pass unnoticed, so the fake refuses to produce one. Pinned in the new fake-http.test.ts.

Best Practices

An inline broken adapter that re-implements the assertion proves an assertion of that shape would catch the defect — not that the contract still contains it. Spawn the real contract instead.
Without the unbroken control, a fixture that failed everything — a syntax error, a runner that cannot start — would satisfy every break mode while proving nothing.
A regex over test( is satisfied by a case that asserts nothing. Counting passing cases from a real none run pins behaviour, not shape.
Spawn the fixture with --test-reporter=tap so a test grepping for not ok behaves identically on Node 22 and Node 24.
A floor lets an assertion be swapped for a weaker one at the same arity — the same hollowing wearing a different hat. Update EXPECTED_ASSERTIONS deliberately when you add or remove an assertion, and while you’re there, decide whether the new one also needs a break mode in contract-fixture.ts.
Behaviour that one adapter alone owes — the Tauri shell deduping identical safe-area payloads, for instance — needs its own test in that adapter’s suite, not the shared contract: the fake and web shells republish, so a shared “does not republish” assertion fails two of three. And a shared harness whose emitInsets takes a full SafeAreaInsets cannot express a partial payload, so the guard for partial payloads is unreachable from the shared harness and its test lives in the Tauri suite. Write it in the shared contract first, and move it the moment the harness cannot carry the input.

Storage & Secrets

The two ports the secrets and storage break modes pin.

Time & Pacing

The port the two time break modes pin.

Shell & Adapters

The shell contract that sits in the same file.

Mobile Engines

The parallel fixture pattern one directory over.