Every conversation with a language model has a hard ceiling — the context window, a fixed budget of tokens that every conversation eventually fills — and unlike most resource limits, this one lands on the worst possible victim: a conversation in progress. The thread you have been working in all afternoon is where the state lives — what was decided, which files were touched, the half-finished task — and the naive handling, which we shipped, is an eviction notice: this conversation has grown too long; start a new chat. Architecture leaking straight into UX, telling the user to abandon the one thing the product exists to maintain.
The naive alternatives are each wrong in their own way. Keep everything and you hit the wall. Truncate the oldest messages and a memory-first assistant develops visible amnesia about this morning — the single worst look available to us. Summarise silently and aggressively and you trade the wall for quiet context loss the user never agreed to. This entry is the phase where we built the real machinery, and it begins with a diagnosis that reshaped the whole design.
Two different walls wearing one error
The overflow error is one string, and it turns out to describe two structurally different events that demand different fixes — and our error copy had been assuming the wrong one.
Mode A: the single-turn blast. One turn's tool loop accumulates enough output to overflow within the turn. The bug that forced the decision was exactly this: a single research request fired four-plus web page fetches, each up to ~50,000 tokens of markdown, all kept in context, with the loop re-sending the whole accumulating feed on every iteration and no input ceiling anywhere. The conversation was young — and the error told the user it had "grown too long". No amount of compacting prior turns can save Mode A, because the blast is inside one turn.
Mode B: honest accumulation. A genuinely long working thread crosses the window over many turns. This is the case the error copy assumed was always true, and it is the rarer one — nearly every historical overflow we investigated traced to bulky tool payloads, not conversation.
The transferable point before any mechanism: diagnose which wall you hit before you build the fix, because the fixes do not overlap, and a team that conflates them will ship a compactor and keep dying on research turns.
Mode A: spill and handle, never truncate
The tempting fix for the blast is a cap — truncate big tool results at the source. It is lossy in exactly the wrong way: a cap tight enough to be safe is too tight to be useful, and the content it throws away is often the content the model needed. What shipped instead is context offloading: an oversized tool result — a fetched page, a large file read, a big data query — is spilled in full to a short-lived GCS buffer, and what enters the model's context is a bounded preview plus an opaque handle. A companion tool lets the model drill back in on demand — a keyword or section slice of the stored body — so nothing is lost; it is simply not all resident. The same backstop wraps every tool result at the dispatcher, so no single payload can dominate the window regardless of which tool produced it.
One design argument from that week is worth the whole entry to the right reader: the spill store is an offload buffer, not a cache, and keeping those concepts separate dissolved a tension that looked fundamental. Blobs are keyed by an opaque per-fetch handle, never by URL, and a fresh fetch of the same URL always re-hits the web — the store never answers a new request with an old page. With that rule in place, the blob's TTL stops being a correctness knob (staleness is impossible by construction) and becomes a pure housekeeping knob, so it can be generous — fourteen days — without a single staleness argument. And a handle referenced after its blob has expired self-heals: the original URL is recovered from the transcript, re-fetched fresh, and the handle revived, silently. Cache-versus-buffer is a distinction we had been blurring; naming it converted three hard open questions into one lifecycle rule.
Mode B: fold the feed, never the transcript
For honest accumulation, the mechanism is compaction — and one invariant outranks everything else in its design: the user's transcript is never mutated. Compaction operates on the model's feed exclusively. The full conversation still scrolls in the app, still reloads from the store intact, forever. What changes is what the model is shown: when the active window crosses a threshold (110,000 tokens by default), the oldest complete turns are folded into a rolling recap — written by a cheap model, seeded from the conversation's running summary — while the most recent stretch (40,000 tokens) stays verbatim, along with the opening framing of what the conversation is even about.
The mechanics have two properties that earned their tests. The fold boundary is tool-pair safe: the API rejects any request containing a tool call without its matching result, so folding snaps to turn boundaries, never mid-pair — an invariant, not a preference, because violating it does not degrade the request, it kills it. And the recap is incremental and fault-tolerant: already-folded turns are never re-summarised (the recap persists and extends), and any failure anywhere in compaction falls back to the full feed — a broken fold must never produce a broken request or lose a byte of anything.
Who decides is split the way the register's whole design philosophy splits it: the machine's housekeeping is silent (auto-compaction happens without ceremony, because narrating your own memory management is noise), while the user's action is visible — typing /compact folds harder, down to a lean tail, and answers with a plain line saying how many earlier messages were folded. And the recap can afford to be brief for a reason specific to this product: anything durable in those folded turns has already been extracted into long-term memory by the post-turn machinery, and is retrievable on demand. The conversation is transient; the memory is durable; compaction leans on that instead of trying to be an archive.
The wall nobody is present to hit
The same week supplied the reminder that ceilings are hit hardest where nobody is watching. A real ninety-minute session died at the window's edge — most of what filled it being tool and retrieval machinery riding in history, which got its own fix in an earlier entry — and the postmortem question was not "how did this happen" but "what happens when this happens at 3am to a scheduled run?" An interactive user can be offered a way out. An autonomous stage cannot click anything.
So the unattended path got its own context guard: long autonomous runs and multi-stage jobs now budget the window per stage, rather than discovering the ceiling as a crash. And the design for overflow recovery inverts from compaction to handoff — and the distinction is worth naming, because they look similar and are not. Compaction thins the model's feed within a session. A handoff ends the session gracefully: the whole working state is distilled into a durable brief — objective, key facts, current state, open threads — written to the user's vault as a real document, and a fresh session (fresh window, fresh cache) continues from the brief, with the original session preserved and linked so any buried detail remains one read away. Lossless on demand, rather than lossy in advance. For unattended work the design has it run automatically, with a cap on consecutive auto-handoffs — because a job that compacts itself forever is a runaway loop wearing a coping mechanism, and after the cap it surfaces to a human instead of quietly eating the window again.
What transfers
Separate the two overflows before designing anything — in-turn blasts and cross-turn accumulation share an error string and nothing else. Spill oversized results to a handle-keyed buffer and keep the distinction between a buffer and a cache sacred; it converts staleness from a correctness problem into a lifecycle rule. Reduce the model's feed and never the user's record, fold only at boundaries the API can survive, and make every reduction fault-tolerant toward the full feed. Keep the machine's housekeeping silent and the user's actions visible. And design the unattended path first when you design for ceilings — the interactive user was always going to be fine; the run with nobody watching is the one the wall was built for.
Postscript, August 2026. The machinery held, and the deferred pieces stayed honestly deferred: the task-aware digest per large fetch remains unbuilt, extractive drill-in having proved sufficient so far; the handoff itself (brief, fresh session, unattended cap) remains a design ahead of its build; and the gentle proactive nudge arrived at the end of the month in a simplified form, offering a fresh chat rather than a brief — the ceiling has stopped being something users meet, which was the point. The buffer-not-cache distinction has since earned its keep in a second place, and the handoff brief's shape — objective, facts, state, open threads — turned out to be the same shape a long-running job wants for resuming work generally, which is a convergence we noticed rather than planned: what a session needs to survive its own death is what any successor needs to pick up its life.