The rerank stage

const rows = await brain.find({
  query: 'what did we decide about the embedding spaces',
  limit: 20,
  rerank: { topK: 32, budgetMs: 2000 }
})TS

Default off. Present, the top K rows of the candidate set are re-scored by a cross-encoder that reads the query and each row together, and returned re-ordered — each scored row carrying rerankScore beside the score it already had.

Why a second model at all

The vector leg scores a query and a row separately: two forward passes that never see each other, compared by cosine distance. That separation is what makes it fast enough to walk a corpus, and it is also why the top of its list is only approximately the top of the list. A row can sit close to the query in embedding space and answer a different question.

A cross-encoder reads the pair as one sequence — [CLS] query [SEP] row [SEP] — and emits a single relevance logit. It cannot be pre-computed and it cannot be indexed, because the score does not exist until the query arrives. So it is only ever run over a short list the cheap leg already produced. That is exactly this stage: find() builds the candidate set, and its head is re-scored.

Where it runs, and why that is the point

On the engine's threads. The forward pass is a napi AsyncTask on libuv's threadpool, so find(), get() and every write keep being answered while the batch computes.

This matters because the alternative was measured. A consuming service ran this same model on its own JS thread and paid 0.7–1.4 s per recall for it, in a process that also served every other door. Moving the model into the engine buys nothing if the engine then runs it on the caller's thread anyway — so that is pinned, not asserted: src/rerank/rerankEventLoop.e2e.test.ts runs the identical batch through the blocking door and through the async one while sampling event-loop lag, and requires the async arm's p95 lag to be a small fraction of the blocking arm's. The blocking arm is the control: the suite fails if the probe does not catch it stalling the thread, because a probe that cannot detect a known stall proves nothing about the door that matters.

The model

cross-encoder/ms-marco-MiniLM-L12-v2 — Apache-2.0, a BertForSequenceClassification with one label and an identity output activation, 12 layers at hidden 384.

It ships inside the package: 133,469,020 bytes of safetensors, its tokenizer and its config, under dist/assets/models/ms-marco-MiniLM-L12-v2 with the Apache-2.0 text and a PROVENANCE.md beside them. Nothing is downloaded at runtime and no scoring leaves your machine. The weights are memory-mapped by candle, so a box running many brainy processes holds one copy of the file's pages, not one per process.

It sits under assets/models/ at the package root rather than beside the embedder's weights in engine/assets/models/, and that is deliberate: src/engine/ is the vendored reference engine, carrying its own provenance manifest and per-file digests, and its asset directory is checked for integrity by npm run vendor:verify — which exits non-zero on a model-asset problem there. A model the reference engine never had does not belong inside a record of what the reference engine shipped. Same build step, same files[] law, its own directory.

The L6 sibling was measured against it on the same candidate pages before this choice was made — see What it costs.

What comes back

Every scored row carries rerankScore: the model's raw logit, higher meaning more relevant. It sits beside score, which stays the hybrid rank the candidate set was built on. The logits are unbounded and are comparable only within one call — never across queries, and never across model variants.

An absent rerankScore on a reranked page is meaningful: that row was not scored, not scored zero. Three rows can be unscored — one past topK, one with no embeddable text, and one on a page shorter than topK.

The call's own stamp comes back two ways:

import { rerankStampOf } from '@soulcraft/brainy'

const stamp = rerankStampOf(rows)
// { model: { id: 'cross-encoder/ms-marco-MiniLM-L12-v2', variant: 'L12' },
//   k: 20, ms: 512, budgetMs: 2000, unscorable: 0 }TS

and, when you are already using the per-call engine budget, at report.rerank:

const { results, report } = await brain.findWithReport({ query, limit: 20, rerank })
report.rerank              // the same stamp
report.phases.rerank       // { ms, calls, scored, unscorable }
report.phases.rerankLoad   // the one-time model load, if this call paid itTS

The stamp on the array is the one to read under concurrent traffic: findWithReport() arms a process-wide budget and refuses to run two at once.

Ordering

Rows are ordered by rerankScore, descending, with one rule on top: scores within 1e-3 of each other are a tie, and a tie keeps the candidate set's own order. Cross-encoder logits are computed in f32 and their last bits depend on how the batch was blocked; without a tie band, that noise would silently reshuffle rows the model considers equivalent and a page's order would not be reproducible.

Rows inside the window that carry no embeddable text are not scored. They keep their relative order after the scored rows and are counted in the stamp's unscorable, so a page the model only partly ordered says so.

Rows past topK are untouched, in their original order.

Batching

The window is scored in sub-batches, ordered by text length. Every pair in one forward pass is padded to the longest pair in it, and a padded position costs the same matmuls as a real one — so one long row among short ones would make the whole batch cost what the long row costs. Ordering by length puts similar lengths together; a pair's score depends only on that pair, so which batch it travelled in cannot change it.

Longest-first, deliberately: the first batch is then the most expensive, so the rate the budget projects from is the worst case rather than the best. A conservative projection refuses early; an optimistic one discovers the overrun after paying for it.

What a row is scored on

The same text the row was embedded from. Deriving it any other way would produce an order that is nearly right — the worst outcome, because nothing would fail.

The refusals

Never a silent pass-through. The failure this stage replaces was exactly that: a JS reranker whose loader "resolves to null rather than throwing" logged rerank=0 ms and served the un-reranked order for weeks, and nothing could tell.

RerankBudgetExceededError

The stage would spend more than budgetMs. It refuses with k, elapsedMs, budgetMs and how many rows it had scored — a partial re-order is a page in an order nobody can describe, so none is returned.

The budget is checked between sub-batches, projected from this call's own measured rate, so a budget that will be blown is refused as early as the measurement allows rather than after the whole batch is paid for. The wall after the last batch is checked too, for a single batch that overran on its own.

budgetMs is the scoring wall. The one-time model load is not scoring and is not charged to it — a first query would otherwise refuse for a cost no later query pays. It is reported separately (report.phases.rerankLoad) and can be paid up front:

await brain.warmRerankStage()
// { model: { id, variant }, loadMs, assetsDir, weightsSource: 'mmapped-file' }TS

A serving process under a latency law should call it at start-up.

RerankUnavailableError

Names one of four causes, because their cures differ:

The last two are the re-embed ceremony's law applied here: a space you cannot name is a space you must not score across. The stage is paired with the MiniLM family at 384 components, and it says so in the refusal — both the space the store is in and the one it expects.

Caller errors, refused before any work

rerank.topK must be a positive integer and rerank.budgetMs a positive number of milliseconds; neither has a default, because the two numbers are what make the stage's promise checkable. And rerank requires a text query: a cross-encoder scores pairs, so find({ vector, rerank }) has nothing to read a candidate against and refuses rather than silently skipping the stage.

What it costs

MEASURED on bxl9000 (32 cores, CPU only) on 2026-09-04, solo under the box's gate lock, at commit 4bc4ce6. Every number below is a measurement; nothing here is projected.

The rerank leg, by K and by row length

src/rerank/rerankLegBench (scripts/rerank-leg-bench.mjs), 5 runs per cell, one warm pass per shape discarded. The wall is the whole leg — tokenize, pad, forward pass, read the logits.

The stage does not meet a 100 ms budget at K=32 on CPU. At the shortest row length it is ~8x over; at the row lengths a real brain holds it is tens of times over. That is stated here rather than discovered in production, and it is exactly why budgetMs is a refusal and not a hint: a caller who sets 100 gets a typed RerankBudgetExceededError naming the numbers, never a slow page pretending to be a fast one.

Set budgetMs from the table for your own row length, or lower topK. Row length dominates K: 16 rows of 120 words cost more than 64 rows of 30 words.

Why it costs that, measured

The forward pass barely scales with cores. At K=32, 30-word rows:

1.35x from 32 cores. The wall is essentially one core's fp32 matmul throughput, so the levers that would actually move it are quantized weights, a smaller checkpoint, or a device that is not a CPU — not more cores. None of those is in this release.

Which checkpoint ships, and why

scripts/rerank-fixture-quality.mjs against a consumer's own graded recall fixture (4 queries, 15 candidates each, scored on the fixture's snippets — a lower bound on full-text quality, and identical inputs for every arm):

The stage earns its milliseconds: NDCG@10 rises from 0.4468 to 0.6528 (+46%) over the order the vector leg produced on the same rows, and the top-3 more than doubles its overlap with what the consumer's own pipeline judged relevant.

L12 ships. L6 is exactly twice as fast and 8.5% worse on NDCG@10 — not equal, and since neither checkpoint reaches a 100 ms budget, halving a wall that is over by 8x does not buy the target, it only costs quality. Point --alt-assets at any checkpoint to re-run the comparison.

Model load and memory

The load is not charged to budgetMs; brain.warmRerankStage() pays it up front. The weights are mapped, so a second process on the same box shares the file's pages rather than adding another 127 MB.

Default off is byte-identical

With no rerank option: no provider is looked up, no model is loaded, no timer is armed and no property is attached. find() returns exactly what it returned before this stage existed. The stamp is attached under a Symbol.for key and is non-enumerable, so even on a reranked page it is invisible to JSON.stringify, Object.keys and every for…in a caller already runs — the wire format of a result array does not change.