The Cost of a Save

There was a page of links. Not a note—a list of 154 unique URLs, mostly repositories. Then Save.

Three minutes and thirty-four seconds later the page existed. Nothing had crashed and nothing was frozen. The app had decided that writing a page meant reading the internet first, one link after another, in the request. That is the kind of slowness that does not look like a bug, because every line in the log says status=200. It looks like thoroughness. It was two unrelated things welded together.

The Network in the Request

SaveWikiPage wrote the page, diffed its URL set, and then called the ordinary URL intake for every new link: fetch, readability extraction, HTML to Markdown, favicon, language detection, snapshot, index row. Exactly right for a link. Exactly wrong inside a save—especially because the page was written last. For three and a half minutes there was no page on disk, only 154 sources pointing at a page that did not exist. Ninety of those seconds were fetches. The rest was worse.

The Queue Behind It

The pump decides which step runs next: it walks every source, finds the first incomplete step, and queues it—it has to look at all of them, because the pipeline’s order is a statement about the whole vault. What it also did was rescan the vault once per queued source, loading a full row with derived aggregates and a step record from disk. Quadratic, and synchronous: the pump held its lock for the whole pass. The log was exact—155 passes after the save, 781 milliseconds each, 120 seconds, one pass per link. The same mistake made the next step crawl: one outline every 78 seconds, for 120 documents two and a half hours of appearing to do nothing.

The fix is a boundary, not parallelism: a save ends at the vault. The page is written, the links are diffed, the call returns. A new link is claimed immediately—a row and a record at the URL’s stable address, capture pending, later steps skipped because there is nothing to read yet—and the fetch in the background fills that row instead of filing a second source beside it. A link is a source before it has bytes. A link that never answers keeps its row, red, with the reason recorded: claimed, not read.

With the save fixed, the queue became visible—and it was not honest. A step marked itself executing after publishing its status, so the only surface that answers “what is the app doing?” answered “nothing”. The reason a job waits was a constant naming whichever of four holders the gate actually had. And the claim watchdog released claims that were executing, because its rule was “older than a run budget” instead of “nothing behind it”—160 sources freed at once, each free to be derived twice. Underneath sat one more lie with a large effect: a source’s .meta sidecar was written through a temp file with a fixed name, so two writers shared it—eighteen unparseable records in a vault of two hundred, each taking the source’s identity with it.

The Shelf Above the Claim

Everything else in the system is derived; only a page’s tags were typed by a human. Saving one now queues a small bounded model call that writes three to six coarse tags—a tag is the shelf a page is filed under, one level above a claim’s topic. The derived tags live apart from the written ones, as auto_tags: a run that overwrote human tags would be a machine editing notes, and one that re-added a deleted tag would be worse. Both appear in the cloud and filter the same; the derived ones are marked, and deleting one sticks. The tags then got what topics already had—a curated vocabulary with two namespaces and an alias table. A rename rewrites one line of frontmatter or one claim label; neither re-derives a document.

What It Cost

The page became the test that forced every layer at once: 154 sources, 14,000 claims, 423 distill runs, 25 million tokens, about ninety minutes. The lesson is not that saves must be fast. It is that a pipeline whose steps are strictly ordered over the whole vault must be asked what that ordering costs per decision. A global rule enforced by a per-source scan invites quadratic behaviour, and it hid for weeks behind a status=200.

Three and a half minutes to save a page is not a performance problem but a category error with a stopwatch: a save that reads the internet, a scheduler that rescans the vault per source, a queue that reports waiting while it executes, a watchdog that releases running work, and a sidecar two writers shared. None of them were slow; all of them were lying about what they were doing, and the app wrote the lie down, pass by pass, until someone read it. The chapter’s repair is a boundary—a save ends at the vault—and its general lesson is that global ordering enforced by per-source scans invites quadratic behaviour.