Ask how an assistant should search a person's documents and the answer we kept being handed was the same one: embed everything, search by similarity, return the top matches. It is a fine answer to one question. The trouble is that a single corpus of personal documents gets asked three different questions, and they demand different — mutually incompatible — instruments. This entry is the three acts, across three months, by which we learned that; the third act was forced by a failure.
Find what I typed. The first index, built in May, is lexical: keyword matching with deliberate fuzziness and prefix search. Because the person hunting for a file is typing a half-remembered name, probably with a typo in it, and wants the machine to meet them halfway — budgt should find the budget, Mer should find the Meridian folder. Forgiveness is the feature.
Find what I mean. The second, in August, is semantic: document bodies chunked and embedded, then rank-fused with the keyword scores into one hybrid list — so a note reading "bring it forward to January" answers the question "when do we start the renewal conversation?", which shares not one word with it. And it was retuned against a labelled evaluation set rather than by taste, because (as an earlier entry in this series found the hard way) similarity thresholds calibrated on toy corpora do not survive contact with real ones.
Two indexes, two questions, both answered well. Which made the third failure more instructive, because the tooling that produced it was good.
The question neither index could answer
In late August an agent was asked which services a project was built on. It read a folder listing, saw a document titled "Inbox setup with" a particular named service, and asserted the service was in use. It had never opened the file. The claim surfaced only because the user challenged it, and the agent's own admission named the failure better than we could: "I saw the filename, not the file."
The reflex is to call that a behaviour problem — the model should have checked. The diagnosis that held up is structural, and it indicts the toolkit, not the temperament. "Do we use X?" is an absence-proof question: its honest answers are "yes, here" and "no, nowhere", and the second one requires exhaustiveness. The ranked retriever — our second index — cannot produce it, for three reasons that are properties of its design rather than bugs in it.
The instrument, and the decisions inside it
So the third question got its own instrument: exact, exhaustive matching — every readable text file in scope, read in full, no index in the loop — returning matches with line numbers and exact totals. The interesting engineering is not the scan; anyone can write a scan. It is five decisions about what an absence proof is allowed to be, each of which we would defend in any system that makes claims from a corpus.
The candidate universe is one function, shared with the index. An absence proof is only as good as its definition of "every file", and that definition — which paths count as readable text, which are excluded, what the agent's read scope permits — is computed by a single function used by both the scan and the search index. Two hand-maintained copies of which files count would drift, and drifted universes turn "nowhere" into a confident wrong answer. The proof and the everyday search must agree about what the world contains, structurally, not by diligence.
A partial scan forbids the claim. Completeness is a computed fact — zero read errors, files scanned equal to files in scope — and anything less returns a result that opens by declaring itself partial and forbidding any absence claim built on it. A false zero is this instrument's one unacceptable failure, the equivalent of a clean bill of health from a test that didn't run; the design treats "we could not check everything" as a first-class answer, never rounded to "not found". Partial results are not even cached.
Literal by default. The pattern is treated as a literal string, metacharacters escaped, with regex as the explicit opt-in — because an "exact" mode that is quietly a regex lies in precisely the identifier-shaped cases the tool exists for. A dot that means "any character" turns fal.ai into a pattern that matches things nobody searched for; exactness that is approximately exact is the old fuzziness wearing a lab coat.
Counts stay exact when the display is capped. "47 occurrences, showing the first 200 lines" is a useful answer; a count silently truncated at the display cap is a wrong answer to "how many" — one of the questions the tool is for. What is sampled and what is counted are different things, and only the sample gets capped.
And the decision that could have gone the other way: the evidence rides along, always. The obvious wiring is a fallback — instruct the model to verify with the exact scan before making a checkable claim. We rejected it, on the argument that decided the whole feature: noticing that you are making a checkable claim is the very judgment that just failed. There is also no null to fall back from — the dangerous case is not an empty search but twelve plausible hits. So exact evidence is folded into every ranked search automatically: alongside the fuzzy results, a literal whole-corpus count for each claim-shaped term in the query. The model does not choose to verify; verification is in its hand before it speaks.
The counterweights, because the pattern invites overreach
Two boundaries keep this from being a bigger claim than it is.
The same lesson runs the other way in the fact graph. Its entity search is deliberately not embeddings — proper-noun lookup is a spelling problem, not a meaning problem, as an earlier entry argued — which is this post's thesis from the opposite bank: the modern instrument is not wrong, it is specific, and the discipline is matching instrument to question rather than standardising on whichever is newest.
And the load-bearing non-goal: this proves things about the document corpus. The question that prompted it — which services a project actually depends on — is answered by manifests, lockfiles and CI configuration, which live in a repository the assistant does not hold. The feature converts asserted without evidence into verified, or declared unverified; it does not put a missing corpus within reach, and saying otherwise would be committing the original sin with better tooling. Whether that corpus should come within reach is a separate decision, still open.
What transfers
One corpus, three questions: find what I typed wants forgiveness, find what I mean wants embeddings, prove it is not there wants exhaustiveness — and the third cannot be built from the first two, because ranking is structurally incapable of expressing absence. If your system makes claims from retrieval, audit it for the absence case: what does a user see when the answer genuinely is "nowhere", and can they distinguish it from "not in the top ten"? Derive your proof's universe from the same function your index uses. Treat partial coverage as an answer, never as noise. And when the failure you are guarding against is a lapse of judgment, do not wire the guard behind that judgment — put the evidence in the model's hand on every question, because the moment it would have chosen to check is exactly the moment it didn't.
Postscript, August 2026. The behavioural half became a standing integration scenario — an agent probed with exactly the trap that started this, expected to check content before claiming use and to decline to assert from a filename — so the fix is guarded by the machinery an earlier entry described rather than by memory. "A miss proves nothing" is now closed for the document corpus: a zero from the exact scan, on a complete read, proves precisely what it says. The fact graph's version of the same promise remains open, as does the repository question. And the phrase "I saw the filename, not the file" has entered the team vocabulary as the name for a whole class of failure — evidence about a container presented as evidence about its contents — which we now hear ourselves catching in places that have nothing to do with search.