The Brainy API Contract — version 2
In this section
Brainy ships as two engines. Open Brainy (@soulcraftlabs/brainy, MIT) is the reference implementation. Brainy (@soulcraft/brainy) is the native engine. Until now they were coupled as a pair: a native release was pinned to one reference-engine version, and the fleet gated on that tuple. A pair is not a contract — it says these two builds were tested together, never this is what a Brainy is.
This page is the contract. It states what every conforming implementation must do, in terms no implementation owns. The version number below is what the fleet gates on.
contract version: 2Read it from code instead of from this page:
import engine from '@soulcraft/brainy'
engine.contractVersion() // 1 — the engine plugin's door
import { Brainy } from '@soulcraft/brainy'
new Brainy(config).contractVersion() // 1 — the API layer's doorTSand from packaging:
// @soulcraft/brainy's package.json
{ "brainyContract": 1 }JSONCThe machine-readable form of everything below is docs/api-contract.json, emitted from src/contract/contract.ts by scripts/contract/emit-manifest.mjs. That TypeScript module — not this page, and not either engine's .d.ts — is the source of truth; this page is its prose, and the JSON is its wire form.
§0 What "conforming" means
An implementation conforms to contract 2 when:
Every REQUIRED door in §2–§10 exists with the stated shape.
Every stated semantic holds — the ordering laws, the merge law, the cascade law, the visibility law, the refusal law.
Every error class in §11 is exported under its stated name and thrown under its stated condition.
The operator law (§12), the field-addressing law (§13) and the health law (§14) hold exactly, including their documented divergences.
It answers
contractVersion() === 2.
An OPTIONAL door may be absent. Optional is not "second class" — it is the honest marker for a door the convergence program has not yet committed to owning in both engines. Promoting an optional door to required is a MAJOR change (§1).
What conformance does NOT promise
Not identical ids.
relate()mints a relation id per call. Two engines seeded with identical inputs agree on(from, to, type, subtype), never on the relation'sid. Compare edges by that tuple.Not identical tie-breaks in vector top-k. Two rows at exactly equal distance may be ordered either way. What IS promised: the result COUNT is the same, and every returned id is a member of the tied set — never a different row.
Not identical performance. The contract is about answers.
§1 The compatibility rule
Change | Version move |
|---|---|
A new OPTIONAL door | minor — additive |
An operator moving from refused to served | minor — additive |
A new error class | minor — additive |
A door removed | major |
A door's answers narrowed | major |
An ordering or merge law changed | major |
An OPTIONAL door promoted to REQUIRED | major |
An operator moving from served to refused | major |
contractVersion() answers the MAJOR number — that is the number a fleet service gates on, because a minor move can never break it. Additive changes within a major are visible in the emitted manifest's door list, which src/contract/contract.test.ts diffs against a fresh emit; a contract change that forgets to re-emit reds the gate.
Contract 2 is additive for CALLERS. The move from 1 to 2 promoted three temporal doors (asOf, diff, history) from optional to required. That is a major move because it raises the bar on an IMPLEMENTATION — an engine that omitted them no longer conforms — but it takes nothing away from a caller: every door that answered under contract 1 answers the same way under contract 2, and code written against 1 needs no change. The frozen reference engine (Open Brainy 10.4.13) declares contract 1 and stays there under decision open-brainy-freeze; it remains the format's exit door, read at the contract it shipped, and check:contract reports that divergence as information rather than failure.
§2 Lifecycle
Door | Shape | Req |
|---|---|---|
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
init() is idempotent and polymorphic. Calling it twice is a no-op, and every door that needs an open brain reaches it lazily through the same override chain — so a subclass that gates on init() gates the lazy path too.
close() releases every hold on the storage root: the writer lock, open file descriptors, and mmap regions. A conforming implementation can be reopened on the same root by a fresh process immediately after close() resolves.
flush() is the durability boundary. Once it resolves, every accepted write is visible to a fresh reader of the same root. Writes are durable at their own cadence too; flush() is how a caller demands the boundary now.
What an acknowledged write guarantees under log authority. On a store whose authority is its canonical log (_system/log-authority.json, the fleet default), an acknowledged write is durable in that canonical log: a single-op write at its commit window (the group commit; flush() closes the window now), a transact() at return. The derived indexes are projections of that log, not part of the acknowledgement. The metadata index's own shard logs are made durable at the flush cadence, never per write, so an add or an update pays no index barrier of its own. If a power cut loses an index tail the last flush did not cover, the next open finds the index's stamp behind the committed head and replays the canonical changes above it behind the doors (job metadata-fact-replay; reads wait for it in process and refuse MetadataIndexNotReadyError, retryable, on the wire after up to 2,000 ms). Every acknowledged row is served again once it completes. The index's stamp never claims a generation that a flush or that replay has not made durable. A plain process kill loses nothing either way, because the operating system keeps the written pages. The id mapper, whose ids every index shares, keeps a barrier per new entity.
Host integration — serving brainy behind your own wire
After an open, some projections are still being made ready behind the doors: the metadata index checks its files' integrity hashes (metadata-index-verify) and the time index builds (time-index-build). In process, a read that needs one of them WAITS for it with the event loop free, then answers. A server with its own wire usually should not hold a connection that long, so the package root exports one door for it: refuseNotReadyWaits(fn). Inside the marked scope, a read that would wait on a projection still being made ready refuses retryably instead: the metadata index waits up to 2,000 ms (the wait ends the moment the check does) and then throws MetadataIndexNotReadyError with retryable: true and an own retryAfterMs estimated from the bytes still owed; a date question refuses TimeIndexNotReadyError with retryable: true at once. Answer either with 503 and Retry-After; a retry answers once the projection is ready. Writes are never refused for this: run write routes OUTSIDE the scope, so a write's own reads (the rule check every write runs first) wait with it. Outside the scope every read waits in process. labelIndexWaits(label, fn) names a caller's own index waits in the operator narration. brainy host dispatches every read route through this same door. refuseIndexWaits(fn), the name 12.13.2 through 12.13.5 exported for this scope, is the same door and stays exported, so a host written against 12.13.x keeps working; inside it a date question now refuses retryably too.
§3 Write
Door | Shape | Req |
|---|---|---|
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
|
| optional |
|
| optional |
The store's own write rules. declareSubtypeRule and declareFieldValues persist a per-type subtype rule and a field vocabulary at _system/write-rules.json, and every process that writes the store — the engine in process, brainy host, brainy serve's native doors — enforces them at add, addMany and update, refusing ValidationRefusedError. requireSubtype() and trackField()'s values register the same kinds of rule in one process's memory and bind only that process; a write must satisfy both. An unreadable record refuses the open (and, natively, StoreUnreadableError) rather than reading as "no rules".
The id law. A caller-supplied id is honored exactly. An omitted one is minted by the engine.
The merge law. update() MERGES: keys present in the supplied metadata replace their stored values, keys absent from it survive untouched, and _rev advances by exactly one. There is no replace-mode update.
The cascade law. remove(id) deletes the entity AND every relation incident to it. After it resolves, related({ node: id }) answers [] — not a dangling edge, not a relation pointing at a missing endpoint. Removing an id that does not exist is a no-op, never an error.
The projection law. When add() resolves, every derived projection the engine serves reflects the write — with ONE exception: the semantic (vector) projection when the vector is embedded asynchronously. waitForIndexed() (§10) is the door for that case; the alternative — a read that silently answers from a projection that has not caught up — is banned.
§4 Read
Door | Shape | Req |
|---|---|---|
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
|
| optional |
|
| optional |
The absence law. get() answers null for a row that does not exist — it does not throw, and it never answers a partially-populated entity. batchGet() OMITS missing ids from its map rather than mapping them to null.
The field-projection law. get, batchGet and find all accept fields?: readonly string[] — return only these fields on each row instead of the whole record. Field names follow §13's one addressing law: a bare name is the caller's own metadata, system.* is one of the ten engine scalars. A requested field the row does not carry is simply ABSENT — never an error, and never present with an undefined value. id (and, on find, score) rides every row regardless of fields. Absent fields ⇒ the full record, exactly as without the option. On find, the projection reaches every top-level convenience field the row exposes (type, subtype, visibility, metadata, data, confidence, weight, _rev) as well as the nested entity — a caller reading row.metadata or row.data sees the same trim a caller reading row.entity.metadata does, never the full record by a side door. §5 covers related's own fields.
The vector law. A row's stored vector rides a result ONLY when the caller asks for it with includeVectors: true — on get, batchGet and every shape of find alike (query, vector, filter, graph, and the filter-less find({ limit })). Without it, entity.vector is the empty array. The filter-less find used to be the one exception, carrying every row's vector because it hydrated from the store's walk rather than by key; it now hydrates the same way every other find does, and the rule has no exception.
The three-state law for find() (this is the load-bearing one):
A query that cannot be answered THROWS. If you get
[], the query resolved and honestly matched nothing. If you get results, they are ordered by what you asked for. There is no third state.
An unresolvable field name, an unserviceable operator, or an option the engine cannot honor is a typed refusal (§11, §12, §13) — never an empty array and never a silent fallback to insertion order. Accepted-and-ignored options are banned under the same law.
The ordering law.
Entities MISSING the
orderByfield (or holdingnull) sort last in both directions — never dropped from the result.Ties break by id ascending, in both directions. Ordering is fully deterministic.
With no
orderByand novector, result order is unspecified. Compare result SETS across implementations, not sequences.
The visibility law. Every entity carries one of three tiers:
Tier | Default | Reached by |
|---|---|---|
| visible | always |
| hidden |
|
| hidden |
|
system is the engine's own tier. A consumer cannot create a system-tier row through AddParams.visibility; the VFS root (00000000-0000-0000-0000-000000000000) is the well-known member every conforming brain holds.
counts.entities() and counts.relationships() are O(1) ledger reads, not scans, and the by-dimension doors read the same ledger. A count that would require a store walk to be exact is a health question (§14), not a count.
findDuplicates(options?) is a composed read, not an index of its own, and a BOUNDED SCAN: a near-duplicate scan is paged by rows and by time; on a 77k-row brain a candidate costs about 19 ms, so the defaults answer in about 2 s. One call examines at most options.scan candidate rows (integer 1 to 50,000, default 200) and stops once options.budgetMs has elapsed (integer 100 to 60,000, default 2,000), whichever comes first, overshooting the budget by at most one candidate's cost; anything outside those ranges is BadQueryParameterError. It walks the store's canonical id order after options.cursor (absent = the start). Each unprocessed candidate is checked against similar()'s top 20 and grouped into { entity, duplicates: [{ entity, similarity }] } above options.threshold (default 0.85); options.type narrows candidates and matches; options.limit (default 100) caps the groups in one page, and a page that reaches it ends early. The answer is { duplicates, generation, scanned, complete, cursor? }: scanned counts the candidate rows this call examined, complete: true means the walk reached the end of the brain (no cursor), otherwise (either bound cut the page) cursor is the opaque token for the next page. The cursor names the last row examined, so it survives writes between pages (a row added behind the position is not seen by that walk); a cursor this door did not issue is BadQueryParameterError. A pair matched on an earlier page can be reported again on a later one, anchored on its later row, because the set of already-matched ids lives for one call. The engine's own hidden rows (for example the VFS root) are never examined and never counted in scanned. A page is bounded by scan and budgetMs, not by the store, so it answers in the same wall on a small brain and a large one; a small brain answers complete: true on the first page. Served brainy host (13.0.0), one shape in process and on the wire; brainy serve has no native duplicate-detection implementation yet (docs/serve.md).
§5 Graph
Door | Shape | Req |
|---|---|---|
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
|
| optional |
|
| optional |
Traversal directionality. related({ node }) is UNDIRECTED — every edge touching the node, in either direction. related({ from }) and related({ to }) are directed. type and subtype narrow the edge set. A node with no edges answers [].
Relation identity. relate() mints the relation id; a caller cannot supply one. Identity is the full four-part tuple (from, to, type, subtype) — an ABSENT subtype is its own identity value, never confused with any particular non-empty subtype string. A call naming a subtype matches only an edge carrying that exact subtype; a call with no subtype matches only an edge with none. relate() (and relateMany(), and transact()'s relate op) is IDEMPOTENT on this identity: a call whose identity already exists returns the EXISTING relation's id — never an error, never a second edge. Cross-implementation comparison is by the same tuple (see §0).
Reads and removals that name no subtype address EVERY edge of the named type. related({ from, to, type }) with no subtype returns every edge of that type between the pair, whatever subtype (or none) each carries; adding subtype narrows to the one identity. The same rule governs a removal door shaped (from, to, type, subtype?): omitting subtype removes every edge of type; naming one removes only that edge. unrelate(id) is unaffected — it already addresses one relation by its own id, never by identity.
updateRelation() (and transact()'s updateRelation op) MOVES an edge's identity; it never merges or duplicates. Changing type and/or subtype moves the row onto the new identity in place — same id, same metadata, same history, now answering to (from, to, newType, newSubtype). If another relation already holds that target identity — staged earlier in the same transact() batch, or already committed — the move is refused with RelationIdentityConflictError (§11) and NEITHER edge is touched: never a silent merge into the existing edge, never a second row beside it. A plain weight / confidence / data / metadata update, changing neither type nor subtype, never consults this check.
related({ fields }) projects each relation to its own body — see §4's field-projection law. The identity survives any projection: id, from, to, type, and subtype when the edge carries one (the four-part identity above) ride every row regardless of fields. fields then names what else rides: a bare name reaches the edge's own metadata, and system.* reaches one of the edge's own scalars (weight, confidence, visibility, service, createdAt, updatedAt, data, evidence). This projects the edge's own body only — related() still hydrates no entity record for from / to; projecting the endpoints is a separate, undecided capability.
neighbors(entityId, options?) answers the adjacent entity ids only — bare string[], never a hydrated relation or entity. options.direction ("outgoing", "incoming" or "both"; anything else is refused BadQueryParameterError) narrows to outgoing/incoming edges or reads both (the default); options.verbType (one VerbType or several) narrows the edge types counted; options.depth walks more than one hop, re-applying the SAME verb-type filter at every hop and bounding the whole walk by options.limit rather than expanding without bound before a final slice. Served by brainy host (13.0.0); brainy serve's own native core has no graph-neighbour traversal implementation yet and refuses NotYetNativeError by name until it does — see docs/serve.md.
§6 Transact
Door | Shape | Req |
|---|---|---|
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
The operation set. Six declarative operations: add, update, remove, relate, unrelate, updateRelation. Each mirrors its single-operation door's semantics exactly — validation (the store's field-index policy included), subtype enforcement, per-entity ifRev CAS, relation-identity dedupe/move — applied inside the batch's one atomic commit rather than one commit apiece.
The batch law. transact() applies its operations in order and atomically: either every operation commits or none does. Later operations in one batch OBSERVE earlier ones — [{op:'add', id, …}, {op:'update', id, …}] commits one row carrying the merged result, and the returned Db's receipt names both operation ids.
The generation law. generation() is monotonic and advances on every committed write. now() returns an immutable Db value pinned to the current generation; a Db never changes underneath its holder.
The time-travel law. asOf, diff and history are REQUIRED doors. A read through an asOf view answers as of its generation on EVERY leg — the metadata filter sees the metadata committed then, the graph hop sees the edges that existed then, and the vector leg ranks by the embeddings carried then. A row born after the pinned generation is absent; a row removed after it is present. An implementation that cannot answer a requested generation REFUSES (GenerationCompactedError, or a typed retention refusal naming the oldest answerable generation) — it never answers from current state without saying so.
transactionLog(options?) reads the reified commit log — one entry per committed generation (generation, timestamp, and meta when the commit was a transact() carrying one), newest first. Under Model-B every write is its own generation, so single-operation writes appear here too, without meta (a transact()-only concept). from/to (a generation number or a Date) bound the window INCLUSIVE on both ends — a log window names the commits it spans, in deliberate contrast to since()'s exclusive lower bound — resolved through the SAME resolveAsOfGeneration helper asOf, diff and history share; a resolved from after to throws RangeError. limit is applied LAST, after windowing. Not one caller's own rows — every generation any caller committed — so this door is reached by the SERVICE key only, never a brain-principal key (unlike history, which is scoped to one entity). Served by brainy host (13.0.0); brainy serve has no native implementation yet — see docs/serve.md.
The dial's doors: generationAt, historyBounds, historySamples. A date-to-generation question is a fact about the commit log, so the engine answers it, not the caller. generationAt(at) (at epoch milliseconds or a Date) answers the newest committed generation whose commit stamp is at or before at, with that stamp; an at after the newest stamp answers the newest (exclusive is asOf's option, not this door's), and an at before the first stamped commit refuses NoGenerationAtError - never generation 0, never a neighbour. historyBounds() answers the oldest and newest stamped generations and refuses NoGenerationAtError on a history with none; its oldest is the oldest REACHABLE stamped generation (at or above the compaction horizon, the rule asOf applies), never a compacted one asOf could not serve. historySamples(n) (n an integer from 1 to 10,000; anything else is RangeError in process, BadQueryParameterError over the wire) spreads n instants evenly by TIME between the bounds and answers, for each, the newest generation at or before it - de-duplicated, ascending by generation, never an interpolation, so fewer than n pairs come back when several instants resolve to one generation. All three read an in-memory (generation, stamp) index, O(log n) per call, built at every open by the named background job time-index-build - never on the open's foreground - and extended by every commit. While it builds, these doors, and every Date argument of asOf, diff, history and transactionLog, wait in process and refuse TimeIndexNotReadyError with retryable: true over the wire (whenServing({ families: ['time-index'] }) is the explicit wait); nothing scans the log in its place. In process, a date question WAITS on the index (it arms the build and awaits it, narrated like every other wait; the build is bounded by the store's own size, MEASURED at 344 ms per million generations), so no in-process caller needs a retry loop; it refuses only when waiting cannot help. Over the wire it does not hold a connection: it answers the retryable refusal (503 with Retry-After) while the build runs. A generation the log carries no stamp for (a crash between the commit and its log line, a fold-replayed window) is filled from its own delta behind the doors and, failing that, is left out - never guessed, so generationAt can answer generation N while N+1 committed without a stamp. Commit stamps are raw wall-clock reads: a backwards clock step is PRESERVED, never clamped, and the doors answer exactly what the old scan answered - the newest generation whose stamp is at or before the instant. The doors are horizon-aware: generationAt(at) whose answer falls below the compaction horizon refuses GenerationCompactedError (naming the horizon), the same refusal asOf gives there, never a clamp up to the oldest reachable generation; historySamples spreads between the reachable bounds. asOf(Date) before the first commit still pins generation 0; only the new doors refuse. Served by brainy host; brainy serve has no native implementation yet - see docs/serve.md.
The Db value itself. now(), transact() and asOf() all return a Db<T> — a readonly view pinned at one generation. It serves the FULL find() surface at that pin, not a reduced one: metadata filtering, graph traversal, vector and semantic search, cursor pagination, AND find({ aggregate }) all answer as of the pinned generation, exactly as they would against the live brain at "now". A Db pin holds compactHistory() back from reclaiming that generation's history until the pin is released — call release() on the Db when you are done with it. A FinalizationRegistry backstop releases a leaked pin at garbage-collection time, but explicit release is what makes compactHistory()'s timing deterministic rather than dependent on the collector.
These three were optional through contract 1. They are required from contract 2 because the product engine now serves the whole of asOf natively, including the unfiltered semantic leg, and a database whose history is optional is not one callers can build on. The frozen reference engine (Open Brainy 10.4.13) implements the PREVIOUS contract (1), where they were optional; it remains the format's exit door and is not expected to move — read it as the reader of last resort, not as a conforming implementation of contract 2.
CAS. A write carrying an expected _rev that no longer matches the stored revision throws RevisionConflictError — it never overwrites. A transaction committed against a generation that has since advanced incompatibly throws GenerationConflictError. A pinned generation that has been compacted away throws GenerationCompactedError. All three are refusals, never silent last-write-wins.
§7 Aggregate
Door | Shape | Req |
|---|---|---|
|
| required |
|
| required |
|
| required |
|
| optional |
An aggregate is incrementally maintained, not computed at query time: a write that changes a member row updates the aggregate's state on the write path. queryAggregate() reads that state.
The where inside an AggregateSource obeys the SAME operator law as find() (§12) — identical vocabulary, identical verdicts, identical refusals. An operator that refuses in find() refuses here too.
§8 Embed
Door | Shape | Req |
|---|---|---|
|
| required |
|
| required |
|
| optional |
embed() produces exactly the vector add() would store for the same data. embedBatch() returns one vector per input, in input order.
The contract does NOT fix the embedding model or the vector dimensionality — those are per-brain configuration, and two brains with different embedders are both conforming. What the contract fixes is that a brain's embedder is consistent with itself: the vector embed(x) returns is the vector a subsequent add({ data: x }) stores.
§9 VFS
Door | Shape | Req |
|---|---|---|
|
| required |
The virtual filesystem is stored in the same brain as system-tier entities, rooted at the well-known id 00000000-0000-0000-0000-000000000000. Its rows are therefore reachable through find({ includeSystem: true }) on any conforming implementation — which is exactly why the root id is part of the contract rather than an implementation detail.
Working-state files
vfs.writeFile(path, data, { retention: 'latest-only' }) — and appendFile, which shares the same option — is a second write CLASS for a path, alongside the default 'history' class every path has had until now. It exists for a row written often and never needed historically: a co-editing session's live CRDT snapshot, a presence heartbeat, anything whose only interesting state is the newest one.
A 'latest-only' write:
Replaces the content and releases the prior blob at once. After N overwrites, exactly one blob is retained for the path — verified through the blob store's own reference accounting, never the filesystem.
Keeps no before-image and no history generation record for the row.
readFile(path, { asOf })andhistory(path)refuse by name (WorkingStateHasNoHistoryError, §11) rather than fabricate a temporal answer they cannot honestly give; the store-wide temporal reads (diff,changedBetween) simply never mention the row, for the same reason — there is nothing to report, not something withheld.Never embeds — not even the lightweight descriptor a binary file's overwrite gets under the default class.
Emits no change-feed event for the write. The generation itself still advances (ordering and durability are exactly as strong as any other write); only the retention is skipped.
Carries
workingState: truein the row's metadata, so a reader can tell.
The class is sticky per path, and the first write decides it. Once a path is 'latest-only', a later write that does not repeat retention: 'latest-only' — including an ordinary writeFile(path, data) call with no options, which is what every other path has always accepted — refuses by name rather than silently starting (or silently continuing to skip) retention. retention: 'history' is the explicit, one-time door back to the default class.
The same primitive is available one level down, off the VFS surface, as retention?: 'history' | 'latest-only' on AddParams/UpdateParams: it controls history retention and the change-feed event for that one write, with none of writeFile's per-path stickiness (a bare add()/update() call has no path to be sticky about).
Listing a folder with each entry's own metadata
vfs.readdir(path, { withFileTypes: true, withMetadata: true }) answers each entry as a VFSDirent that carries, beside name, path and type, its own entityId and its full metadata — the bag get(entityId).metadata returns, exactly — in the SAME call, so a host lists a folder with one door call and no second search. The entries are the whole entities the listing already reads in its single batched read, so the option adds no read and no find(). It applies to every entry of a recursive: true listing too. withMetadata requires withFileTypes: true; passed without it the door refuses with a VFSError (EINVAL) rather than ignoring the option. Without withMetadata the answer is byte-identical to what it always was: no metadata key on any entry. Over brainy host's vfs door (op: "readdir") the option travels in options unchanged and each dirent crosses the wire with both fields.
writeFile(path, data, { metadata }) applies the caller's metadata to the FILE entity exactly as mkdir's metadata does for a folder — the keys merge into the entity's own bag (over the previous bag on an overwrite), so a file's dirent and get(entityId).metadata carry them. The engine's own fields (path, name, size, mime type, storage, the vfs markers, permissions, timestamps, content-derived fields, system.* and attributes) always win: a caller key naming one is refused with a VFSError (EINVAL) rather than overridden. Extended attributes stay on setxattr, under metadata.attributes.
Moving a directory, and reading a path's retired history
vfs.rename()/vfs.move() on a DIRECTORY commit every descendant's path update, plus the two Contains edge changes on the directory's own row, as ONE atomic batch — a directory move of any size lands as a single generation, never one write per descendant. A file move is unaffected: it was already a small, constant number of writes.
readFile(path, { asOf }) and history(path) are both resolved as of the generation named, not resolved against the live tree and then materialized backward:
readFile(path, { asOf })pins the target generation first, then resolvespathagainst THAT generation's ownContainsedges andpathfields. A path that has since been renamed away or unlinked still answers, as long asasOfnames a generation at or before the change and that generation is still within the retention window.history(path)resolvespathagainst the LIVE tree first. When that fails (the path was renamed away or unlinked since), it searches the retained generation records for the most recently retired row that carried this exact canonical path, and answers that row's version history up to the generation that moved the path away. ENOENT is thrown only when no retained generation ever heldpathat all — a path whose only versions have fallen out of the retention window is reported as no versions retained, never fabricated, and a caller naming a compactedasOfgeneration onreadFilegets the generation store's ownGenerationCompactedError(§11), never a bare ENOENT.
The batch door: history(paths: string[]) → Map<string, FileVersion[]> answers every named path's history in one call. A path this brain never retained any version of maps to an empty array rather than throwing, so one absent path in a batch never fails the others; a real refusal (WorkingStateHasNoHistoryError) still fails the whole call. Reachable in-process on brain.vfs.history(paths), and over brain host's wire as the vfs door's op: "history" (with path for one file or paths for a batch) — brainy-serve's native engine does not answer this operation yet and refuses it by its own UnknownVfsOperationError, matching every other VFS write door's current NotYetNativeError posture there.
What history() costs, and the window that bounds it
history() does not walk versions — it walks GENERATIONS. To learn which generations wrote one row, the generation store reads one per-generation delta (_generations/<N>/tx.json) per generation in the searched range, because that is where "who changed at this generation" is recorded. So the unbounded form's cost is a term in the STORE'S AGE, not in the file's version count. MEASURED in the field (anima, PLATFORM-RENAME-TRAIN-FLEET r461, 2026-09-22, a brain at generation ≈693,000): one vfs.history(path) did not answer in 110 s while readdir of the same folder answered in 277 ms, and a batch over 41 paths sat in flight for an hour.
history(path, options) and history(paths, options) take the window that bounds it:
Option | Bounds | Cost |
|---|---|---|
| The WALK. A number is a generation and is EXCLUSIVE — the same boundary | One delta read per generation IN THE WINDOW, plus one record read to seed the dedup rule (below). A |
| The answer's upper end (INCLUSIVE), clamped down to the current generation. The newest in-window version still reports its real content — the before-image of the first write above the bound, not the row's live state. | Does not shorten the walk; adds the distance from the bound to that next write. |
| The ANSWER only. The newest | None — the walk is unchanged, because the newest versions are at its END. |
The window's boundary is honest in both directions: versions outside it are not answered, and nothing is fabricated to fill it. A windowed answer is exactly the sub-range of the unbounded answer that falls inside the window — including the consecutive-identical-content rule that collapses a rename, a setMetadata() patch or a deferred-embed landing into no new version. That rule compares against the last version actually answered, so a window that simply restarted it would report the version before its own start a second time at exactly the boundary a caller pages on; instead the window SEEDS the rule from the before-image its first touching generation recorded, which is the row's content at the window's start. That is the one extra record read a windowed call makes. An invalid bound (a negative or non-integer generation or limit, an unparseable Date) throws RangeError rather than silently answering a different window.
The retired-path search takes the same since. That search is PATH-keyed, not id-keyed — it walks generations newest-first asking each touched noun's own before-image whether it carried this exact path — so no per-id index can answer it, and since is the only thing that bounds it. Without one it descends to the retention horizon, exactly as before.
rename() and move() never consult generation records at all: they resolve the live tree and commit one transaction, so none of this cost applies to them.
MEASURED SHAPE, from the engine's own counters rather than a clock (Brainy.historyCounters(): touchingWalkReads for the per-id walk, retiredWalkReads for the retired-path walk), pinned in src/engine/lib/vfs/vfsHistoryWindowCost.test.ts: on a 3,000-generation store the unbounded call reads one delta per retained generation while the windowed call reads at most its own window, and the two answers agree — the windowed list is exactly the suffix of the unbounded one that falls inside the window.
The touch index — what makes an unbounded history() cheap
The window bounds the walk when a caller can name one. The PER-ENTITY TOUCH INDEX removes it when they cannot.
Generations are grouped into fixed BLOCKS of 4,096. A block every one of whose generations is committed is COMPLETE, and a complete block gets an inverse index written behind the doors: for each id, the generations inside that block that touched it. Two files per block, under _generations/_touch/:
File | What it is |
|---|---|
| the block's full inverse index — one entry per id the block touched, generations stored as offsets from the block's first |
| a bloom filter over every id in the block, sized for a 1% false-positive rate. Small, and it is what makes the index cheap to USE: the blooms load once per process and stay resident, so a call reads the FULL index only of the blocks whose bloom says the id may be there |
A per-id walk then costs, per call: the blocks its window covers (a bloom probe each, in memory), the full index of the one or two that may hold the id, and a delta scan of the OPEN TAIL block only — the block still being written, which is never indexed and whose 4,096 generations are exactly what the store's delta cache is sized to hold.
The index is DERIVED, and that is the whole safety of it. A block whose index is missing, torn, of a format this build does not know, or simply not written yet is treated as ABSENT: its generations are scanned exactly as they were before the index existed, and the answer is identical. Never a refusal, never a different answer — only a dearer one. An older engine ignores _generations/_touch/ entirely, which is what makes a rollback free.
Existing brains are indexed by a BACKFILL behind the doors, newest block first (the recent past is what callers ask about), on the same background job runtime the history-byte census uses and under the same installment contract: it yields to any waiting foreground door and stands down at close. It is resumable with no cursor file, because the block indexes already on disk ARE its watermark — a pass skips them. A reader never writes one: a read-only store refuses the pass by name and scans. Nothing is written on the commit path: a block that has just completed is indexed by a later pass, so no commit pays for an index write.
WHEN IT RUNS. Never during an open, and never into a busy store. The pass is SCHEDULED rather than armed: it waits until no open is in flight anywhere in the process, the population has served no foreground door, and this store has committed no generation, for thirty seconds. If an open starts while it is running it stands down at the next BLOCK BOUNDARY — finishing the block in hand rather than discarding it — and the scheduler re-arms it once the open is over and the store is quiet again. The scheduler keeps watching while any block remains owed, because a block completes every 4,096 generations and one that completes mid-pass is not in the set that pass is walking.
This is not caution for its own sake. MEASURED at anima on Wed 2026-09-23, with the pass armed from inside init(): projection-init:graph took 180,196 ms instead of its usual ~4 s, index-init-gate held for 180,723 ms, and the door served empty writes and health 000 for twenty-eight minutes at 9 % CPU — I/O-bound throughout. The engine's existing arbitration could not see it: foregroundBus counts doors SERVED, and an open in its index-init-gate serves none.
HOW IT PACES. At most touchIndex.readsPerSecond generation deltas per second (default 2,000 — roughly a block every two seconds). When a window's budget is spent the pass SLEEPS out the rest of the window before its next read; a yield would return as soon as the event loop is free, which on an I/O-bound box is immediately, and that is the shape that starved the open. It holds one block's deltas at a time, and every read goes through the store's own delta cache.
THE HOLD. BRAINY_TOUCH_INDEX_BACKFILL=hold (or touchIndex: { backfill: 'hold' }) stops the pass arming at all. Per-row history then answers from the delta scan for every unindexed block — exactly what it did before the index existed: never wrong, only dearer. The environment variable WINS over the config option, in both directions, because it is the lever an operator reaches for when something is already wrong on a running fleet. The health row says held: true, so a held store is never mistaken for one with nothing left to do.
brain.maintenanceDebt().history.touchIndex reports { blocks, indexed, owed, backfilling, held, blockSize, blocksWrittenThisSession }, and brain.touchIndexStatus() answers the same row directly. owed is the number to act on — blocks that are complete, unindexed and outside every history gap, computed by the same definition the pass itself walks. Never derive it from blocks - indexed: a block inside a history gap is complete and unindexed and is never owed, and "minus one for the open tail" is an off-by-one the field already got wrong (blocks 195, indexed 193 — the block actually owed was 193, not the tail). owed > 0 with backfilling: false and held: false is a pass that is not running and should be. A pass that ends having written no block while blocks were owed is a NAMED FAILURE in the job's lastError, never state: done.
brain.maintenanceDebt().history.timeIndex reports the time index's own status: the date-to-generation list behind generationAt, historyBounds and historySamples, built once behind the doors. Its fields are state, entries, bytes, inversions, gapFillOwed, attempts, lastError, linesRead, malformedLines, outOfOrderLines, boundaryRows, holesFound, holesFilled, holesUnfilled and buildMs. state is one of four. unbuilt: not started (it starts when a date question arrives or the store has been quiet for 3 s). building: the build is running. ready: it answers; gapFillOwed: true means it is still filling stamps its log lacked (holesFound, holesFilled, holesUnfilled), which never makes an answer wrong, only leaves those generations out until filled. failed: the build failed attempts times (lastError names the last); reopen the brain. While the state is unbuilt or building, the three date doors refuse retryably over the wire (retryable: true) and wait in process; when it is failed they refuse without retryable. The row reads memory, never storage, and is absent on a store with no generation history. brain.health() carries it as the time-index check: fail when failed, warn while building or while a gap fill is owed, pass otherwise.
brain.maintenanceDebt().metadata.debt.indexVerify reports the metadata index's deferred integrity check: after an open the engine hashes each index file's body on worker threads, behind the doors, and serves no file before its hash matches. Its fields are state (idle, owed, running, done or stood-down), filesVerified, bytesHashed, filesTorn, filesRemaining, bytesRemaining, workerMs, calls and waitingDoors; it is absent on a read-only attach. While it runs, a read that needs the index waits in process (the process stays free) and, over the wire, waits up to 2 seconds and then refuses MetadataIndexNotReadyError with retryable: true and a Retry-After; a write is held, never refused. brain.health() carries it as the index-verify check: pass while it runs and once it is done, fail when any file's body failed its hash (repairIndex({ rebuild: ['metadata'] }) rewrites it). While it runs, the health checks that need the index (seeded-records, guardrails-legacy-visibility) answer warn ("verifying") at once instead of waiting. The engine's mechanism is docs/open-law.md §2d.
Compaction removes a block's index with its generations: a block every one of whose generations has been reclaimed loses both files at the same pass that reclaims them. A block STRADDLING the horizon keeps its index and stays honest, because an index-named generation is filtered through the committed set before it is answered — so a reclaimed generation is never reported.
MEASURED SHAPE, from the engine's own counters (historyCounters().touchIndexBlockReads and touchingWalkReads), pinned in src/engine/lib/db/generationTouchIndex.test.ts over a store of several real blocks plus an open tail: the indexed walk scans at most the open tail block's generations and reads at most one block index per complete block in the window, while the same walk unindexed reads every generation in it; a second identical call reads no block index at all; an id the store never touched costs no block read (the blooms answer); and the index's answer equals the scan's for every window tested — including windows whose edges land exactly on block boundaries, an id spread across several blocks, an id inside one, and an id in none.
§10 Health and repair
Door | Shape | Req |
|---|---|---|
|
| required |
|
| required |
|
| required |
|
| required |
|
| required |
|
| optional |
|
| optional |
|
| optional |
|
| optional |
|
| required |
|
| required |
|
| optional |
|
| optional |
|
| optional |
|
| optional |
The semantics are ADR-008's, restated here as contract rather than as one engine's design (docs/ADR-008-index-health-and-repair.md):
Health is accounting, not sampling (D1). Every projection maintains an exact coverage ledger on its write path and reports health as a subtraction. A sampled probe may be a diagnostic; it may never be a verdict.
Verdicts are graded and named (D2).
health().checksnames every condition;isReady()-style readiness is a VIEW of the same report. There are no unnamed latches.Rebuild is online and generational (D3). New structures are built beside the serving ones and swapped atomically. Reads never go dark for a rebuild.
Heal is incremental, and never read-triggered (D4). The common cure is re-posting the named missing items — O(missing), not O(store). A read path may refuse loudly; it may never start a store walk.
repairIndex()is the one explicit operator door.A throwing probe is
heal: 'none'(D5), with the error indetail. Flakiness can never buy a rebuild.
The serving law (the D2/D4 amendment of 2026-08-26, co-signed by both engines): heal grades one question only — could an answer be WRONG? — never how expensive is the fix? Serving is withheld ONLY by these named, rebuild-graded invariants:
index-initialized · durable-state-present · manifest-residency ·
replay-clean · strand-latchstrand-latch withholds serving only for the lost-mapper, torn-manifest, replay-unclean and archive-orphan classes. Everything else — a saturated vector repair, a coverage shortfall, a short generation during an online rebuild — is repair-graded: healthy: false, the row red with the number, and serving: true. An implementation that withholds serving for any other reason is non-conforming; that exact defect took two fleet services down in one day before the amendment.
requireProviders(keys) THROWS when any named key fell back to the engine default. It is the structural gate the native package's loud-refusal law is built on — diagnostics() reports, requireProviders() enforces.
Relation-identity duplicates (12.9.1) are counted, never scanned for on a read path. repairIndex() (the one explicit operator door, per D4) walks every relation grouped by (from, to, type, subtype) and counts groups holding more than one edge — data that predates the identity fix above, or a concurrent-write race the fix's read-then-write duplicate check cannot close. health() reports that pass's own numbers as the relation-identity- duplicates check (warn when a pass found any, pass with the count when a pass found none, and a named "no pass yet" pass before the first repairIndex() run) — it never re-walks itself. This is accounting, not repair: a duplicate group is never dropped or merged, and there is no repair door for it yet.
repairIndex(options?)'s scope and its stop (13.1.2). { only: ['provider'] } runs only the provider phases — the explicit rebuilds rebuild names and the report-driven pass over the other families — and none of the store walks; any other only value refuses with TypeError. A repair stops at its brain's close: before every phase, every page of every walk it drives, and every write its rebuilds make (opening a shadow generation, each field-index base, the swap, the commit stamp), it asks whether the brain is closing or closed, or whether the instance lost the writer lease it began with; when it has, the repair rejects with RepairStoppedError and writes nothing further — the unfinished generation it leaves is reclaimed by the next open. Every repair ends in one line: repairIndex complete in N ms — F/F families checked, H heal(s), or repairIndex ABORTED at phase P after N ms — why.
scanFacts(options?) opens a sequential scan over the generation FACT LOG — the append-only after-image record of every committed generation — the streaming substrate for index heals and incremental replays: one sequential read in commit order replaces a per-entity directory walk. null when this store hosts no fact log at all (predates dual-write, or the storage adapter lacks the binary append primitives fact logging needs); non-null otherwise, even over an empty window. options.fromGeneration/toGeneration bound the scan inclusive on both ends; options.kinds (an array of "noun" and/or "verb"; an unknown kind is refused BadQueryParameterError, never dropped) narrows each yielded fact's own ops to 'noun'/'verb'; options.batchSize caps facts per yielded batch (default 256). A detected gap or damaged segment ABORTS the scan loudly, never a silent skip. Service-key only — this door can walk the WHOLE store's after-image history, a bigger disclosure than any single find()/get() call. Served by brainy host (13.0.0); brainy serve has no native implementation yet — see docs/serve.md.
retractOrphanedIndexIds(options?) and orphanedIndexIdsStatus() are the orphan-id retraction ceremony: find, and on the caller's word retract, metadata-index id entries whose row canonical no longer holds (each candidate is re-checked in canonical under the commit lock before it is retracted, so a row written while the run is going is never retracted). It is a dry run unless apply: true: without it the ceremony counts census.retractable, writes only its own receipt and retracts nothing. In process the start door resolves with the finished OrphanIdRetractionReceipt; orphanedIndexIdsStatus() reads the stored receipt (null when the ceremony never ran) and writes nothing, adding running — true while the engine's own body is running, whether a start call began it or this brain's open resumed an unfinished run by itself. Over brainy host the pair is the ONE start-and-status pattern every operator ceremony follows: POST /v1/brains/{brain}/ceremonies/orphan-ids starts the run and answers a CeremonyRun at once, and GET on the same path reads it — { ceremony, brain, runId, state, startedAt, finishedAt, receipt, resumedByEngine, refusal } with state one of running, done, failed, interrupted, none, and receipt the ceremony's own receipt, unchanged. The host answers from the run it holds first; with none held it answers from the engine's status: running (resumedByEngine: true, runId: null) when the engine is working — an open resumed the run — done for a finished receipt, interrupted for an unfinished receipt nothing is working on, and none without a receipt. It never says running for a run nobody is working on. BrainyClient starts and then polls, so await client.retractOrphanedIndexIds({ apply }) resolves with the same receipt the in-process call does. A second start while one runs, or while an unfinished one is owed, is refused CeremonyInProgressError (HTTP 409, carrying the running run); a read-only brain is refused CeremonyReadOnlyError; a brain without the native runner or id table is refused CeremonyUnavailableError. Service-key only — the tier repairIndex sits in. Both doors are optional: an engine that has no id table to sweep owes nothing. brainy serve refuses both by name — see docs/serve.md.
persist(path) and persistStatus() are the backup door: snapshot the store's latest generation into a folder of its own, cut at the instant the call holds the commit lock, never refused for a store that moved on (docs/snapshot-safety.md). In process the argument is a path, the call resolves { generation }, and persistStatus() reads which phase an in-flight snapshot is in — flush, lock-wait, phase-1, linking, sealing — with its start instant and the generation it is cut at (null until the commit lock is held) — and, while linking, the pass's live linked, byteCopied and dirs counts — or null when none is in flight; it reads memory, writes nothing and keeps no record across a restart. In process, brain.persistAbort(reason) stops an in-flight persist at its next directory: both leases are released, the seal is never written, the target is left unsealed (and refused by Brainy.load()), and the persist rejects PersistAbortedError; close() does the same for any persist in flight. Over brainy host the disk is the one place the two deployments differ: the body carries { name } — one snapshot name, never a path — and the host places the snapshot at <root>/<brain>/<name> inside the folder its operator set with --snapshots-dir. It follows the operator ceremonies' one pattern: POST /v1/brains/{brain}/ceremonies/persist starts the run and answers a CeremonyRun at once, GET on the same path reads it, and the finished run's receipt is { generation, name } — no answer names a path — plus rewrittenDuringCopy / retiredDuringCopy (store-relative paths) only when the live store rewrote or removed a derived raw object (a cache or stamp the engine rebuilds) while the snapshot copied. A run this host no longer holds reads none (never interrupted: the backup keeps no durable receipt), and BrainyClient.persist(name) throws CeremonyRunLostError for it rather than return none. The name is one legal path segment by the same rule brain names follow — 1 to 255 bytes, no / or \, not . or .., not a dotfile — refused BadQueryParameterError before any filesystem call, as is a body that carries a path. A host started without the flag refuses SnapshotRootNotConfiguredError; a root that is missing, is not a directory, is equal to, inside or containing the brains directory, or a folder that resolves outside it refuses SnapshotRootUnusableError; a name that already holds files refuses SnapshotTargetExistsError (the host never overwrites and never deletes). A second start on a brain while one runs is refused CeremonyInProgressError. Service-key only. Both doors are optional; brainy serve refuses both by name — see docs/serve.md.
§11 Error classes
Every name below is an EXPORTED class of a conforming implementation's public surface — a caller cannot instanceof-narrow a refusal it cannot import. BrainyError is the base every engine refusal extends, and it carries a retryable: boolean (isRetryable(error) narrows any Error) — whether waiting and calling again can turn this SAME refusal into an answer, with nothing else required. Most refusals are not: a caller-supplied bad argument or a structural mismatch does not become valid by waiting. The three NotReady classes below are the case where the answer genuinely depends on which refusal this is, not on the class: each construction site says which, and brainy host's wire envelope (docs/serve.md) carries the same boolean for every transport, plus an HTTP Retry-After header when it reads true.
Class | Thrown when |
|---|---|
| base class for every engine refusal |
| a door requiring an existing entity gets an id that does not exist (not |
| a door requiring an existing relation gets an id that does not exist |
|
|
| a field name resolves to neither user metadata nor one of the ten |
| a field address is not addressable at all (an unknown |
| a projected read ( |
| a |
| a CAS write loses the race. Over |
| a transaction is committed against an incompatibly-advanced generation |
| a pinned generation has been compacted away |
| a speculative overlay could not be reconciled with the committed log |
| canonical storage and the committed log disagree irreconcilably |
| durability was requested for writes a pending flush could not make durable |
| a maintenance walk cannot enumerate canonical rows |
|
|
|
|
|
|
| the vector projection cannot be trusted to answer — |
| the metadata projection cannot be trusted to answer — same |
| the graph projection cannot be trusted to answer — same |
| the time index behind |
|
|
| a required derived artifact is absent and cannot be rebuilt in place |
| a write would damage an engine-protected artifact |
| a door is called while an on-disk migration holds the brain |
|
|
| a canonical record on disk is partially written |
| the licensed product's own layer refuses to open unlicensed, invalid, or past its coverage window — Open Brainy never throws it (no license concept), but exports the class so a caller can |
| a memtable shard's persistence layer proves unreliable (its directory is gone, or its append log I/O-faulted) and the native engine poisons that one brain for writes rather than corrupting the recovery contract or aborting the process for every brain it serves — Open Brainy never throws it (no native memtable, nothing that poisons this way), but exports the class for the same |
|
|
| a |
| a minted |
| a request ASSERTS the user on the call — |
| any VFS door refuses — one class for every filesystem refusal, carrying the POSIX |
|
|
| a caller-supplied parameter fails the engine's own validation before a write reaches storage — a required |
| a mutating Accords door ( |
| a record-profile write (an Accords filing, decision or round; a guardrail rule) would create a row whose id is already on the board — nothing is ever overwritten; carries |
| a discovered embed service (the conventional fleet/customer name, or |
| a store's canonical flip pointer names an absolute |
| any governed write door ( |
|
|
| a door that can only be answered by a background job is called while the operator's maintenance hold is holding that job ( |
|
|
|
|
| an operator ceremony ( |
| an operator ceremony is asked of a read-only brain — it writes its own receipt (and, on apply, the metadata index), which a reader may not do. Carries |
| an operator ceremony needs engine pieces this brain lacks: no background job runner is open on it, or its metadata index is not the native provider (no id table to sweep). Never a degraded fallback. Carries |
| the orphan-id retraction ceremony met an id-table number it cannot hold exactly — the table's next int, or a stored receipt's high-water or cursor, outside 0 .. 2^53 - 1. Refused before anything is marked, written or retracted, never rounded. The table is dense from 0, so this means its header is wrong: run the id-mapper consistency sweep first. Carries |
| the backup door ( |
| the snapshots root the host was configured with cannot be written into safely: it does not exist (the host never creates it), is not a directory, is equal to, inside or containing the brains directory (the host refuses to start on that spelling too), or the snapshot's folder resolves outside the root. The message never carries the root's absolute path. |
| a snapshot is asked for a target that already holds files — |
|
|
|
|
| a persist was stopped while it linked files — |
| a persist refused because something its link pass listed was gone when the pass came to it — a directory its parent's read named, or a file, removed before it was linked. The snapshot lease defers every reaper's deletes, so a vanishing means a deleter ignored it; a snapshot missing what it listed is never served as complete. The pass runs to its end so the refusal names every vanished path; the leases are released and the seal is never written, so the target is an unsealed partial snapshot |
§12 The operator law
Full accounting: docs/filter-operator-conformance.md. The contract's binding statement is the jointly frozen set and its three verdicts — served, served beyond the baseline, refused by name. There is no fourth verdict, and a quiet wrong answer is never one of them.
Served (both engines answer identically) — 18 tokens:
eq · equals · ne · notEquals · in · oneOf · gt · greaterThan ·
gte · greaterThanOrEqual · lt · lessThan · lte · lessThanOrEqual ·
between · contains · exists · missingCombinators (clause-level, served by both) — 3 tokens:
allOf · anyOf · notServed beyond the baseline — the native engine answers these three posting-set expressions exactly (hasAll = intersection, noneOf = universe minus union, excludes = negation of contains); the reference engine's index path answers all three with [] today. The native engine is the correct side, and this is a RULED divergence, pinned on both sides:
hasAll · noneOf · excludesRefused by name — the native engine THROWS, naming the operator and the field; the reference engine's find() resolves with []:
startsWith · endsWith · length · matchesThe refusal is a proof, not a shortfall: the metadata index stores normalized posting keys (lowercased, trimmed, one posting per array element, canonical numbers), which provably erases the distinctions those four operators read. matches carries a second, independent reason — JavaScript RegExp and Rust's regex are different dialects. The durable cure is to make the predicate's answer an indexed value at write time and filter on it with a served operator.
The vocabulary is closed. An operator token outside these 25 value operators is not "unsupported" — it is a typo, and a conforming implementation refuses it by name rather than matching nothing.
Known divergence — negation with case or padding. The native engine's metadata index normalizes a posted string (lowercase + trim); the reference engine's does not. Positive operators are blind to this (the index answers a superset, find() narrows it exactly), but a NEGATION (ne / noneOf / excludes) on a string field whose only difference from the operand is letter case or surrounding whitespace drops rows the reference engine keeps. Ruled, documented, pinned on both sides.
§13 The field-addressing law
Full text: docs/field-addressing.md. One rule, no exceptions:
A bare field name always means user metadata. Engine scalars are reached explicitly as
system.<field>.
Origin — never name — decides the storage key. An engine scalar can only land on a system.* key; a user metadata field can only land on a bare key. The two never merge, which is what makes the rule enforceable at the storage layer rather than by resolution priority. A user field named level, createdAt or type is ordinary user data and always works.
The ten entity scalars (system.<name>):
id · type · subtype · createdAt · updatedAt ·
confidence · weight · visibility · service · createdByThe relation scalars (system.<name>):
verb · sourceId · targetId · subtype · createdAt · updatedAt ·
confidence · weight · visibility · service · createdByInvisible plumbing — never resolvable, never indexable, never orderable, not bare and not through system.:
vector · connections · level · data · _revReserved namespace. A user metadata field whose name would flatten into system.* (a field literally named system.foo, or an object field named system) is REFUSED at write time. A scalar field named exactly system is ordinary user data.
The refusal names both candidates. system.<anything else> does not exist and refuses. An unresolvable bare name refuses with a message naming both the missing user field and the system.* correction.
export()'s fields option is a consumer of this law, not a second one. export(selector, { fields: ['title', 'system.weight'] }) projects each entity down to the named fields — through the same door get/batchGet/ find's own fields option uses — plus the reserved identity fields every row always carries (id/type/subtype/visibility/createdAt). An unresolvable field is simply absent from the row (the same permissive read semantics GetOptions.fields documents), never a refusal — a projection asks "these, if you have them." data is never projectable: it is on the invisible-plumbing list above, which is the entire byte saving.
fields cannot be combined with includeVectors — on ANY of get/ batchGet/find/export() — nor, on export() alone, with includeContent. Both throw ProjectionConflictError (§11) by name. The vector case is a priced trade-off, not an oversight: a vector lives only in canonical storage, which a projected read exists specifically to avoid touching for a row the column store can already answer in full, so serving one alongside a projection would cost a second canonical crossing for exactly the rows the projection is fastest for. get/batchGet/find used to resolve this by letting fields silently win (a caller who named both got the projection and a quietly-empty vector) — the accepted-and-ignored shape the field-addressing law forbids; this refusal replaces that silent precedence everywhere the projection door is reached. fields also requires the live, current-generation view on export(): a historical asOf() pin or a speculative with() overlay throws ExportFieldsProjectionUnavailableError (§11) instead.
export()'s limit option is a ceiling, never a page size. Omitted (default): unbounded, exactly today's behavior. Set, it is checked against the selector's resolved CANDIDATE count — before any predicate re-check can only narrow it further — before a single row is read; a selector that would exceed it throws ExportRowLimitExceededError (§11), naming the candidate count and the ceiling, rather than returning a silently-truncated document that looks complete.
§17 Profile doors — the accord, memory and session profiles
A store that DECLARES a record profile gains doors for it. declareProfile() persists the shape with the store (_system/record-profiles.json), and the engine then serves that shape's own namespace: brain.accords.* for the accord profile, thirteen doors over the coordination record — threads, rounds, decisions, actions and mandates.
These are contract terms with one structural difference from every door in §2–§10, which is why they are a separate family in the manifest (profileDoors, profileErrors) rather than a flag on the existing one:
They are not members of
Brainy.prototype. A profile door hangs off a namespace accessor and exists only for a store that declared the profile.They are the PRODUCT engine's. The frozen reference engine (Open Brainy 10.4.13) declares contract 1 and will never grow one. Declaring them among the engine doors would assert something untrue about a shipped surface, and the drift pin that proves every door exists on the reference prototype would have to be weakened to let them through — which is exactly the pin worth keeping.
They are OPTIONAL. That is the honest requirement today: they land natively in the write execution's own release (13.2.0, Fri 2026-10-23), and a build that declared them REQUIRED before it served them would be claiming conformance it does not have. A new optional door is a MINOR move under §1's table, so publishing these shapes does not move the version at all; promoting them to REQUIRED is the MAJOR move, and it is taken when the native doors ship, not when their shapes are written down.
The doors
Door | Kind | Enforces | Refuses with |
|---|---|---|---|
| write | R1 R2 R3 R4 R7 R9 R11 R12 R17 S1 S6 |
|
| write | R1 R9 R12 R17 S1 S6 |
|
| write | R4 R6 R9 R11 R12 R17 S1 S2 S6 |
|
| write | R9 R12 S1 S2 S3 S6 S8 |
|
| write | R12 S1 S4 |
|
| write | R9 R10 R12 R13 R14 S7 |
|
| write | R5 R7 R9 R10 R12 R15 R18 S2 |
|
| write | R7 R8 R10 R12 S2 |
|
| read | R8 R16 S7 |
|
| read | R8 S7 |
|
| read | — | — |
| read | S5 |
|
| read | S5 |
|
A RESOLVED thread freezes rounds. Posting to one is refused by name, carrying the thread's id. The one deliberate move out is a round that carries reopen: true and a reason: it flips the thread back to OPEN, records who reopened it and why in the timeline, and lands the round — all in the same transaction, so a thread's history and its status can never disagree. A reopen with no reason is refused, and anything about a NEW matter files a new record.
The rule ids are the accord protocol's own, written up in docs/accords-profile.md §R. The manifest carries each profile door's FIELD TABLE — every argument as a named field, with its type, whether it is required and a one-line description — as the source of truth; the door's printed signature is GENERATED from that table, never hand-typed beside it. This table is the readable form of the same data, and src/contract/contract.test.ts proves the two agree, including a round-trip pin per door: table → generated schema → a fixture built from the table validates against that schema and comes back unchanged.
One vocabulary, spoken once
The accord profile's doors speak one vocabulary: actor · id · text · body · title · kind · reason · resolution · via · generation · related · schedule · query · where · limit · scope — the same sixteen words a consumer meets at every door, rather than a different word per door for the same concept. Five fields changed name to reach it (door-names 12.6); the older spellings below are read as an alias of the field beside them, through 12.x:
Older spelling | Current spelling | Where |
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
An alias is never a second, disagreeing field: at most one of a pair need be supplied, and the field table above lists which spelling a generated client's schema currently marks required — the older one, where a caller already built against it, per door.
The refusals
Eighteen classes, one per rule that can refuse — RecommendationError is the one name two rules share (S3 and S8 are the same law about the same field, so a caller writes one handler for both). Every one carries fields beyond its message, because a refusal a caller cannot act on is a crash with extra steps: RoundsMovedError carries the rounds you missed, ConsentIncompleteError carries who has not cleared, IllegalTransitionError carries the transitions that were legal, AskUnaddressedError carries who you named beside who the parties are, and NotMyResolveError carries the owners whose stamp is actually owed. The full list, with what each carries and the status it maps to, is profileErrors in the manifest and §R of the profile page.
The memory profile
A second profile, two doors. brain.memory.turn() records what one exchange was about — the running gist of a conversation, so a later session can find it without replaying it. It is not remember(): a memory is something the brain learned, and a turn is the record of an exchange, addressed by the pair it happened at.
Door | Kind | Enforces | Refuses with |
|---|---|---|---|
| write | M1 M2 M3 M4 M5 |
|
| write | M6 M7 M8 |
|
Its five laws, written up in src/memory/rules.ts:
M1 — the pair addresses the turn. A turn's id is
turn:{conversationId}:{turn}, DERIVED by the engine and never supplied, and the position must be an integer at or above zero. A door handed no record answersStateRequiredErrornaming the record to read, rather than leaving a caller to compose the id — an id law re-derived by a caller is an id law with two implementations.M2 — an over-long gist is refused, never truncated. Above 512 tokens the call refuses with what was measured and what the budget is. A gist is a claim about what a turn was about; a cut claim reads as a whole one.
M3 — a turn happened once. An identical re-send is absorbed as a no-op success, because a network that dropped the ack is indistinguishable from a call that never landed. A DIFFERENT gist for the same pair refuses with
TurnAlreadyRecordedError, carrying the gist that is recorded.M4 — the speaker vocabulary is closed:
person,agent,tool,system. A recall that filters by who spoke cannot filter on a vocabulary that grows per caller.
Its row is an event with subtype conversation-turn, and asked / done / open are FREE TEXT, each optional and omitted when there is nothing to say — a turn that asked nothing has no answer to the question rather than an empty one. The field names and their closed sets are fixed by the standalone-parity ruling, so a client written against the brainy host reaches this door with the names it already uses.
M5 — the counter is the caller's, and it must exist. Token counts are tokenizer-specific, so a door that counted for itself would count in a vocabulary the caller's model does not use. A call with no
measuredTokensrefuses withTokenCounterAbsentErrorrather than being measured wrong.
brain.memory.abstract() is the write side of the abstraction tier that context() and collapseTree() already read. It writes a rollup and absorbs the rows it stands for — the abstraction row, the processing.abstractedIntoMemoryId on every absorbed row, and the abstractedInto edges — in ONE transaction pinned to the generation the decision was made at. The words are the caller's: no language model runs inside the engine, so the door lands prose a service produced rather than composing it.
Its three laws:
M6 — an abstraction absorbs at least one row.
absorbs: []refuses withAbsorbsEmptyError. A rollup with no members is indistinguishable, to every walk that reads it, from a rollup whose members were lost — accepting one puts a permanent false negative in the store.M7 — every id must resolve, and the refusal names all that do not.
AbsorbedRowMissingErrorcarries every missing id rather than the first, so a caller absorbing forty episodes fixes its list in one round trip. Nothing is written.M8 — all three facts land together or none do. A store that committed between the decision and the write refuses with
AbstractionMovedErrorcarrying the pinned generation, rather than retrying: one of the rows being absorbed may have been retired in between, and only the caller can decide about that. Without the pin a kill mid-sequence leaves an episode absorbed in the field and not in the graph — which reads as data rather than as damage, and nothing refuses or heals.
Like the accord doors both are OPTIONAL, and for the same reason. Unlike them they execute in the engine's own process today; over a wire they are refused by name, because brainy serve attaches every store read-only and holds no transaction to commit a write in.
The session profile
A third profile, four doors, one for each hook the terminal plugin fires against across a session's life: start (SessionStart), capture (PostToolUse), save (Stop and PreCompact), wrap (SessionEnd). Ruled S1–S5 on the record (action ANIMA-OWNER-LAYER-BEHIND-THE-DOOR) — labelled SN1–SN5 in this manifest's own rule-id namespace only, because S1/S3/S5 already name accord rules above; the ruling's own numbering and every law are otherwise unchanged.
Door | Kind | Enforces | Refuses with |
|---|---|---|---|
| relay | SN1 SN2 SN3 SN4 SN5 |
|
| relay | SN1 SN2 SN3 SN4 SN5 |
|
| relay | SN1 SN2 SN3 SN4 SN5 |
|
| relay | SN1 SN2 SN3 SN4 SN5 |
|
A third door KIND, relay — neither read nor write: a session door's body is composed by the key's OWNER LAYER, never by this store, so it writes no entity and needs no transaction to commit one in. write and relay enforce the identical ceremony (a non-empty rule table, a non-empty refusal list) and both want write scope from a minted token — a read-scoped token is refused BRAINY_KEY_SCOPE for either kind, on every server this engine ships. What differs is disposition: a write door may bind NotYetNative or InProcessOnly (there is no transaction here to commit one in) but never Served; a relay door binds ONLY Served — it answers { served: true, … } (the hosted door only) or { served: false, reason } (every self-hosted build) from the engine's own rules, with nothing to commit either way. Unlike the accord and memory profiles, no build of this engine composes a session yet — session composition is the HOSTED door's own work; the shape a relay door answers is the contract term here, not the behavior behind it. Its laws, written up in src/session/rules.ts:
SN1 — answer by name. Every session door answers
served: trueorserved: false, never a silent no-op and never a 404 for a known door. Ahookoutside the five the harness fires refusesSessionHookUnknownError.SN2 — door-side redaction. What a hook sends is redacted by the door before any row is stored, on top of (never instead of) the client's own fixed denylist — the door never asks the plugin for semantics.
SN3 — the key decides. What a door composes, captures or saves follows the credential it sees — owner, grant scope, self-hosted host — never anything the request claims about itself.
SN4 — thin relay. A hook prints only what the door returns, verbatim; the plugin carries no logic of its own.
SN5 — the harness is never wedged. A door failure or timeout is one named stderr line and exit 0 at the hook; the door itself bounds the request at 262,144 bytes (256 KiB) serialized and refuses over-size with
SessionPayloadTooLargeError, carrying what was measured and the limit, rather than reading an unbounded body.
Like the other two profiles, OPTIONAL, and for the reason above: promising a required door this build cannot compose would be a false claim.
§14 What the contract does not cover
Deliberately outside contract 2, so it cannot be mistaken for a promise:
On-disk layout. Canonical storage layout is
docs/canonical-layout-spec.md's subject, versioned separately. Contract 1 is an API contract; two conforming implementations may hold different bytes. The canonical seam (below) is the bridge until Stage 2 moves canonical storage into the native core.Neural import, MCP, integrations, OData. Shipped by the reference engine and re-exported by the native package, but not contract-mandated at version 1.
Performance.
docs/performance-budget.mdcarries the CI-enforced latency budget for the native engine. It is a product commitment, not a contract term.
§15 The canonical seam
Until Stage 2 lands the segment-log canonical inside the native core, the native package reaches canonical storage through the reference engine's FileSystemStorage. That dependency is ONE named module — src/api/canonicalSeam.ts — which enumerates the exact adapter methods the engine calls and asserts every one is present at open, refusing by name if the reference engine drops one.
The seam is named so that it can be RETIRED, not so that it can grow. Its surface is frozen at contract 2; adding a method to it is a deliberate, reviewed change, and Stage 2 deletes the module.
§16 How this page stays true
src/contract/contract.tsis the source of truth; this page is its prose.scripts/contract/emit-manifest.mjsemitsdocs/api-contract.jsonfrom it.src/contract/contract.test.tsfails red when the committed manifest and a fresh emit disagree, AND when any door named here is absent from the shipped reference-engine surface, AND when the field-addressing vocabulary drifts from its shared catalog, AND when a §17 profile door names a rule or a refusal the profile's own rule table (src/accords/rules.ts) does not declare — in either direction.scripts/contract/check-operator-vocabulary.mjs(part ofcheck:contract, §8 below) fails red when the operator vocabulary drifts from the engine's ownVALUE_OPERATORSsource — this proof runs at gate time, against a swappable source path, rather than inside a unit test that must read the vendored TypeScript.src/conformance/oracle.test.tsruns one workload against three subjects — the reference engine, the native provider stack, and the one-install package constructed the public way — and asserts the same answers.scripts/check-contract.mjs(npm run check:contract) verifies both engines answer contract 2 before a release.
Finding: four alias spellings in the operator page are not in the shipped vocabulary
docs/filter-operator-conformance.md's table lists is, isNot, greaterEqual and lessEqual as served aliases. Open Brainy 10.4.3's shipped VALUE_OPERATORS set declares 25 tokens and none of those four is among them — the engine throws BrainyError('INVALID_QUERY') on an unrecognized operator, so those four spellings refuse rather than serve. This contract publishes the SHIPPED vocabulary (read from the engine at gate time by scripts/contract/check-operator-vocabulary.mjs, npm run check:contract §8), not the four extra spellings. Filed for the operator page to correct; recorded here so the difference is never discovered as a surprise.