An assistant that holds someone's working life is not a stateless request handler wearing a friendly tone. The process answering a message holds a live credential to that person's mail, their calendar, their files — and it can act: send the email, move the meeting, file the document. That one property decides the architecture question most web software never has to ask seriously: what, physically, separates one customer from another?
The cheap answer is the standard one — a shared process and an identifier threaded through every query — and for most software it is right. The failure mode is what changes here. In an ordinary application, a missing filter shows someone the wrong data: bad, visible, recoverable. In an assistant that acts, a missing filter is an action taken in someone else's name, with their credentials, in their mailbox. We did not want that class of bug to be possible at the application layer, so we took the expensive road: every person gets their own process, in their own Kubernetes namespace, running under its own GCP identity that can reach their data and nothing else. The wall is enforced by infrastructure that has no notion of our application logic, which means no bug in our application logic can breach it.
That decision came first. Everything in this entry — an autoscaler we deleted on day one, a provisioning pipeline, a pre-pulled image, and a memory that travels as an archive — is downstream of it, because a process per person creates three problems on the spot: how a person gets one, how it sleeps (most of them must — nobody talks to their assistant twenty-three hours a day), and where its state lives while it does.
Day one: what an autoscaler cannot see
The first answer to the sleeping problem lasted eleven minutes. Sleeping pods need waking, and there is well-built, widely-used tooling for scale-to-zero on HTTP traffic. We configured it — zero to one pod, ten minutes idle before sleep — and wrote a comment in the manifest noting that a streaming connection would hold the idle timer open for the length of a conversation. That comment is where the wrong assumption was hiding. The tooling scales on pending requests, which measures load when a request means arrive-work-return-close. An agent reply is not that shape: the connection opens at send and stays open while the answer is composed — minutes, sometimes; across a conversation, the whole conversation. Pending-requests pins at one and stays there, so the signal saturates precisely when it needs to discriminate: from outside, a person thinking mid-conversation and a person who left an hour ago look identical. This is documented behaviour, not a discovery — the tool says it is built for short request/response cycles. We brought it the wrong shape of traffic.
The fix was not a better autoscaler; it was noticing what already existed. Every request already passes through a component that authenticates who is asking and routes them to their own space — that part is mandatory, it is the load-bearing half of the isolation. And it already knows the three facts an autoscaler infers: who this is, that a request just arrived, and when the last one came. So the router took the job. It brings a pod up on the request it is already authenticating, and a background sweep puts pods back to sleep after ten quiet minutes, measured from last real activity rather than from the state of a connection. The dead design had made the mistake general-purpose tooling almost has to make: treating tenants as alike enough to be handled as a crowd. Ours hold state that must survive between conversations, wake for one person and sleep when that person stops, and are emphatically not interchangeable.
Making the wake bearable
The cost of the boundary is concentrated in one moment: someone's first message of the day lands on a pod that does not exist yet, and they wait while it is created — at exactly the moment a new user is deciding whether this thing is any good. This week's work was mostly about that moment, and none of it is one big idea; it is the accumulation of small ones, which is what latency work always turns out to be.
The container image is now pre-pulled onto every node in the cluster, so waking a pod never includes downloading the software — the largest single component of the wait, removed by paying for it in advance, once per node instead of once per wake. The pod reports readiness through a startup probe staged for what boot actually does, rather than a generic health check that lies in both directions. Messages sent while the pod is still waking are queued and delivered when it is ready, so an early message is never dropped and never errors — the person types, and the system's job is to make the typing valid. And the wait itself was given honest UX: a staged sequence naming what is actually happening — your environment is starting, your memory is loading — because thirty seconds of specific progress reads as a system working and thirty seconds of spinner reads as a system broken.
A memory that travels as an archive
A process that sleeps poses the state question sharply: the whole point of this product is that the assistant remembers, and the pod holding those memories is destroyed after ten quiet minutes. The memory store is ChromaDB — a SQLite database plus vector-index files, a storage engine that seeks and locks and rewrites pages in place. Running that directly against object storage is a non-starter, and not marginally: GCS speaks whole-object semantics, and SQLite's random-access pattern against it is catastrophically slow. This is a known incompatibility we designed around rather than discovered.
So the state lives on the pod's local disk while it runs — fast, ordinary, local — and round-trips as an archive: downloaded and extracted at boot, packed and uploaded when the pod is told to shut down, with a periodic sync as a safety net between. The trade is honest. Boot pays for the restore, which is part of the cold start above; the sync window means a crash (as opposed to an orderly sleep) can lose the last few minutes of memory writes; and the upload path carries the sharpest risk in the design — a pod that booted from a bad restore must never be allowed to pack up its bad state and overwrite the good backup. That risk is named on the whiteboard and not yet written into the code.
The bet
We bought a boundary and we pay for it in latency at the worst possible moment, plus an archive round-trip, plus a provisioning pipeline most products never need. It is a bet that people handing an assistant their inbox and calendar will care more about where the wall is than about the first seconds of the first reply — made, at the time, without much to copy: the excellent tooling for running untrusted code in a disposable box has the isolation and throws away the memory; the durable-workflow tooling keeps state and has no opinion about who owns it. What this product needs is a third thing — a process that is warm, tied to one person, wakes when they speak, sleeps when they stop, and remembers between times. For now it is assembled out of a gateway, a Kubernetes namespace, a GCP service account and a sweep. That is not elegant. It works.
Postscript, August 2026. The shape held; three edges moved, each in the direction the body predicted. The memory archive grew its protections within days — timestamped backups with thirty-day retention first, and later the rule the design had only implied became enforced code: a restore that fails integrity checks marks the boot bad, and a pod that booted bad refuses to upload, so a corrupt local state can never clobber the last good archive. Provisioning moved off the request path entirely once a scalability review ranked a launch-day signup burst as the most realistic failure: signup now enqueues a job and returns immediately, with a bounded-concurrency worker, retry with backoff, and a claim mechanism that survives a gateway restart — the ten-second synchronous provision had been both a wall and a timeout waiting to happen. And the per-user identity hit a wall we did not see from the whiteboard: GCP caps service accounts at one hundred per project, and we were forty-six in. The redesign — spike-validated, migration staged — keeps the property that matters (GCS itself refuses cross-tenant access) while binding IAM permissions to each pod's Workload Identity principal on per-tenant managed folders instead of minting a service account per user. The isolation model was never the mistake; the implementation unit of it had a ceiling, and the fix swaps the unit while the wall stands. Around us, the tooling landscape has grown exactly where we expected: the ephemeral, disposable agent sandbox is now well served. The persistent, identity-bound, half-asleep session that has to remember you next week — we are still assembling that one by hand.