Shaping the Scope

With the non-goals written, the v1 boundary was mostly bookkeeping. Two decisions were not, though, and both have held up under load.

A Source Is Exactly One File

Intake accepts single .md, .pdf, .txt files, or one URL. A directory is rejected outright. It felt pedantic until I tried to reason about the alternative: a folder is a mutable, nestable container, and “immutable raw material” and “mutable container” cannot be the same thing. A batch is expressed by dropping several files—each becomes its own content-addressed source.

The name of a source is the SHA-256 of its bytes, sharded as sources/<ab>/<cd>/<full-hash>.<ext>. Two consequences I use daily: identical bytes collapse to one file, and “correcting” a source means adding a new file, never editing an old one. An edit would silently invalidate every claim that cited it.

The Vault Versus the Index

Everything lives in exactly one of two places, and the rule that separates them is one sentence: if losing it would lose knowledge, it lives in the vault; if it can be recomputed, it lives in the index.

• The vault (~/thought-keeper) holds sources/, atoms/, wiki/ as plain Markdown. It is the artifact—readable, greppable, worth versioning, portable between machines.
• The index (~/.local/share/thought-keeper) holds the SQLite database, the per-source tree sidecars, and later the favicons. All derived, all rebuildable, all machine-specific.

The index must therefore never live inside the artifact: no binary blobs next to the notes, no cache files in git. A backup is the vault. The database is not worth backing up.

Three Layers, Three Questions

The stack of layers got its final shape here:

• sources/ — immutable, content-addressed. What did the source say.
• atoms/ — one claim per file, in branch folders, with {hash, node, ocr} provenance. What do we now believe. The source of truth, and the Zettelkasten.
• wiki/ — compiled pages in topic folders, with atoms: provenance. How do we explain it. Derived, never hand-edited.

This choice also settled how retrieval would work: the three layers are the three retrieval surfaces, plus lexical search and the link graph.

The Application Model

The last scoping decision was about the shape of the thing: a desktop application, not a service.

• One binary, launched by the user; it brings up the HTTP service on loopback, the file watcher, and the agent.
• The PWA is the UI, served by the same process, same origin—no CORS, no token, no auth, because nothing is exposed.
• The CLI is a client of the running instance. Content verbs go over HTTP, so the CLI never touches the database.
• A single instance owns the vault via a lock file. A second launch cannot start a second engine; it hands over to the running one.
• The running instance writes its address to a runtime file so the CLI can find it without guessing a port.

Three decisions carry the rest of the design: one file per source, the vault as truth with the index as cache, and the UI in the same process as the service. Each looks like a constraint and behaves like a multiplier—re-ingest becomes tractable, reindex becomes a guarantee, and an entire class of authentication and CORS problems disappears. The chapter’s argument is that boundaries are not limits on what can be built; they are what makes the next feature cheap.