The system prompt of a capable agent is written by many hands over many months, and nobody takes anything out. Each feature adds its rules, each connector adds its usage guidance, each tool adds a schema, and all of it is sent on every request by every caller, whether or not this conversation will touch any of it. Ours grew from 47,659 characters to 247,614 in five and a half months, with no month in which it fell. Before this work, a chat request carried about 130,000 tokens before the person had typed a word, and a routine running at seven in the morning carried instructions about suggestion chips and clarifying questions it had nobody to show them to.
The bill is the visible part. The quieter cost falls on the agent itself: the field's measurements say instruction following falls as instructions are added [1, 2, 3], and a prompt that sits near the context window leaves the conversation as the only thing left to trim. When our Creator Studio caller crossed its budget line by 54 tokens, the fix that landed was to shorten two tool descriptions, which cleared the line by five tokens, and no behavioural test was re-run. That edit is the shape of the problem. A growing prompt turns every new feature into a small, unmeasured tax on every old one, and the tax is paid in attention as well as money. Taking the prompt apart by how often each part changes meant answering five questions: why the growth costs every caller, which prompt cache misses were multiplying the cost, which layers to cut it into and who had the idea first, which two parts of the design are ours, and what moving a rule does to whether it is followed, which was the finding we had not planned for.
Why a growing system prompt costs every caller
The fixed overhead is everything a request carries before the person's own words. Measured on 2026-10-29 with the budget gate that fails our build when any caller crosses its line, it looked like this.
| Part of every request | Tokens | Who pays it |
|---|---|---|
| Platform block, shared by every agent | 61,346 | every caller, every request |
| of which the capabilities section | about 39,000 | every caller |
| Agent block: identity, style, personal instructions | 6,144 | every caller |
| Dynamic block: session facts, connected services, the hint gate | 17,903 | every caller, uncached |
| Tool schemas, chat | 44,628 | chat |
| Tool schemas, a Creator Studio session | 64,594 | Studio sessions |
| Tool schemas, routine runs and dashboard refreshes | 23,023 to 61,176 | unattended callers |
Chat came to 130,033 tokens, a Studio session to 149,999 against a line of 150,000, a dashboard refresh to 146,581. The gate measures a floor, an account with nothing connected; a real account runs higher. One hosted chat of our own carried a fixed prefix of 182,560 tokens plus a dynamic block of about 38,000, and a three-message test conversation on it cost $1.31 in model charges.
A parser run over the shared prompt template at sampled commits, interpolations excluded, gives the growth.
The series is monotonic, and the reason is structural: nothing was ever paid for by the feature that added it. A rule for Creator Studio lands on routine runs that never touch Studio. A user with no connectors pays for the calling rules of a telephony connector they have never linked. A paragraph that helps one surface costs every other surface a little, and no single paragraph is ever the one to cut.
The cost of carrying all of it showed up as a ceiling before it showed up on an invoice. In September, on one of our own accounts with nothing connected, the fixed overhead of an unattended run measured 163,473 tokens against a line of 150,000 that exists to leave room for the run's own work. Past that line a run has no room to do anything. The budget gate dates from that measurement. An earlier entry covers the first round of this work, measuring the prompt with the meter that bills it and deferring the long tail of tools behind a search tool [O1]. What remained was the prose, and the cache.
Why more instructions make an agent worse at following each one
The size of the prompt is not only a cost. Across twenty models from seven providers, the best reached 68% accuracy at a density of 500 instructions, with a bias towards the earlier ones [1]. Across ten models, performance degraded consistently as instructions were added, closely enough that a logistic regression on the count alone predicts it [2]. Part of the decline is the tension and conflict between instructions, which grows as more are added [3]. Length on its own has the same shape: eighteen models grew increasingly unreliable as input length grew, a single distractor lowered performance, and on a long-conversation benchmark, focused prompts of about 300 tokens beat full prompts of about 113,000 on every model [4]. The field calls that last effect context rot, and an agent whose fixed overhead is 130,000 tokens starts every conversation well inside it.
Two of our own observations sit on top of those. At a fixed overhead near the window, the guard that keeps a request inside the model's limit can only trim the conversation down to its minimum, so the agent loses the user's own history first, and before this work, on one of our own hosted chats at that overhead, the guard was already dropping tool results from the model's view early in the conversation. The conversation-side ceiling and what we do about it is its own entry [O2]; the point here is that the fixed part had eaten the room. And a budget that is measured line by line shows what each line buys: about 550 tokens on every request paid for a planning checklist whose behaviour did not occur in six of six baseline runs. Removing guidance about tools a caller cannot call takes away instructions it can only conflict with, which by the measurements above should be a quality change in the right direction. We measured whether it was.
The prompt cache misses that multiplied the cost
Prompt caching is what makes a large fixed prefix affordable at all. A provider that offers it stores the prefix of a request, and later requests that begin with the same bytes read it at a tenth of the input price, having paid 1.25 to 2 times that price to write it [5, 6]. The match is exact: "Cache reuse requires the entire rendered prefix to match" [6], in the order tools, then system, then messages [5]. One production team has called the KV-cache hit rate "the single most important metric for a production-stage AI agent" [7]. Ours was missing for two reasons, and only one of them is written down anywhere we looked.
Minute changed
Read155,741Written26,265–Same minute
Read183,259Written454Median written, turn 3 and later
Before the fix29,625After the fix1,7530100k200kThe clock. The dynamic block carried the current time to the minute, and it rendered before the messages. The block was deliberately left uncached, as the earlier entry describes [O1], and that reasoning was incomplete: an uncached block is still part of the prefix of everything after it, and the conversation history, cached by a breakpoint at its tail, came after it. When the minute changed between two turns, the block and the whole history behind it were written again. On production's request layout, turns where the minute had changed read 155,741 tokens and wrote 26,265 and 27,518; a turn in the same minute read 183,259 and wrote 454. This is the documented anti-pattern. One provider's caching guide works through exactly this layout, static system context followed by "a per-request block containing a timestamp", and gives the result: "No cache hit. You pay for a fresh cache write on every request and never get a read" [5]. The Manus team's write-up puts it as "A common mistake is including a timestamp—especially one precise to the second—at the beginning of the system prompt" [7], and OpenAI's guide says to put such values "at the end rather than the beginning, or move them into later conversation messages" [6]. We had read the advice as being about the system prompt and broken it one block down.
Per-turn blocks on the live message. The request also attached context to the live user message, such as what the client can display, the file open beside the chat and the screen in view, and did not store it. The history breakpoint sat on that message, so every cached entry ended in bytes that would never be sent again, and no later turn could read any entry, whatever the clock said. A direct probe made it plain: in a five-turn conversation, by turn 4 the cache read was stuck at 1,825 tokens while the write had grown to 4,107; the same conversation with no per-turn block read 4,841 and wrote 1,016. Our integration harness sends no such blocks, so it could not see this. It was found by reading the code that assembles a request, and we did not find the case in the provider documentation we read: the documented examples are the timestamp, and a breakpoint on content that changes every request.
The fix moved one thing the model reads and one thing it does not. The time now rides the per-turn tag on the live message, so the system blocks are stable for a session. The history breakpoint now goes on the last stored block before the live turn on a turn's first model call, with later calls in the tool loop keeping it on the live message so the loop's own results are read from cache. The probe after the change read 3,868 and wrote 1,023, the shape of the control. In the hosted harness, over 25 runs a side on the same commit, the median cache write on turn 3 and later fell from 29,625 tokens to 1,753, and the agent's cost across the slice from $6.67 to $4.97. In production's usage records, read a day after the deploy and with the after side on one account, so a direction rather than a result, the median write on a later chat turn fell from 75,862 to 11,832, and the share of later turns writing over 50,000 tokens from 47 of 69 to 2 of 12.
Two smaller rewrites followed from the same rule, that a number which moves with use does not belong in a cached block. The dynamic block's one-line map of the memory store carried a count of stored memories, so filing a memory mid-conversation changed the prompt and re-wrote the history behind it; the line now carries names only and holds them for the conversation. And a conversation's first model call had no breakpoint after the agent block, so a new chat paid for its dynamic block three times; with a breakpoint on it for that call alone, a new chat's first turn carries 1,300 to 15,000 uncached input tokens where it carried 38,000 to 40,000. Because this fault class had passed every gate we owned, it has a gate of its own now: a short hosted run that fails when any later turn's first call writes more than 30,000 tokens, or when the hash of any system block changes between turns.
Four layers, ordered by how often they change, and whose idea each was
The rule that holds the design together is that content enters the prompt at the earliest layer at which it is stable, and later layers only ever append. Nothing that changes mid-session may sit in front of anything that does not.
| Layer | Holds | Changes | Cache behaviour |
|---|---|---|---|
| Core | guardrails, writing rules, the tool and memory contracts, a one-line catalogue of every capability and connector | on deploy | shared by every user of a model |
| Session pack | the agent's identity and instructions, usage guidance for the connectors this account has linked, the variant for this kind of caller | once per session | per user, read for the whole session |
| Guides | playbooks for dashboards, records, meeting transcripts, local folders and discovery hints, loaded when the conversation needs them | on demand | part of the cached history from then on |
| Turn tail | the time and the per-turn facts | every turn | after the last breakpoint, never stored |
Core
On deploy
Session pack
Once per session
Guides
On demand
Turn tail
Every turn
after the last breakpoint, never stored
Cache breakpoint
Most of this is settled practice. Ordering a prompt by volatility, stable first and dynamic last, is the advice of every provider caching guide we read, and of the Manus team [5, 6, 7]. Tool search, deferring tools behind a search tool with the discovered schemas appended after the cached prefix, is now shipped by two providers in the same shape: a typical multi-server setup can carry about 55,000 tokens of tool definitions before any work is done and search typically cuts that by over 85 percent, with selection accuracy degrading past 30 to 50 visible tools [8]; discovered tools are "injected at the end of the context window" [9]. Loading a skill's name and description always and its body only on relevance is progressive disclosure as the Agent Skills design describes it, and the amount of context a skill can carry is "effectively unbounded" [10]. Composing a prompt from sections, each gated by "a condition predicate over a runtime context dictionary", with a cached stable block and an uncached dynamic one, is published in the OpenDev coding agent's write-up, which reports an "approximately 88% reduction in input token cost for the cached portion" [11]. The field's name for the whole discipline is context engineering, curating "the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome" [12]. We built on all of it rather than beside it.
Three parts are ours, and each is modest. First, a connector's usage guidance renders only for an account that has linked that connector, while the core keeps one line per connector saying what it is for and how to connect it, so the agent can still recommend a connector it cannot yet use. The skills design does the same for skills; we found no precedent for doing it to connectors. Second, the pack for an unattended caller drops the prose for the tools the caller has already lost. Each dropped section names the tools it describes, and a test fails if any of those tools is on a run's tool list, using the same filter that builds that list, so prose and tools cannot drift apart silently. Third, a before-and-after protocol for every one of these moves: two arms on one commit, three hosted runs a side, medians per scenario, rubric items and cache writes per turn. The Instruction Bleed study surveyed four open-source prompt-composed agent systems and found that none "currently implements any form of regression test" for how its modules compose [13]; the protocol is the part of this work most likely to transfer.
One published lesson points the other way, and the difference is the layer. The Manus team keeps the tool set fixed within a task and masks token logits instead of removing tools, because tool definitions sit near the front of the context and removing one both invalidates the cache and orphans earlier references to it [7]. Our connector gating is a decision made once per session, from the account's link status, about prose; it is stable for the session, and the tool list already varied by what a user had linked, so no cross-user cache sharing existed to lose.
Connector guidance only for what is linked, and a pack for each kind of caller
The per-connector sections, how to page a CRM's search, a calling connector's rules, what a social feed returns, left the core and now render at the top of the session block, under a heading that says they bind, for the connectors this account has linked and granted to this agent. Routine runs, dashboard refreshes and workflow stages have no person present, and their tool list already dropped the interactive panels; their pack now drops the sections about suggestion chips, clarifying questions, adding the app to a home screen, dictated messages and the hint gate as well, about 13,000 tokens that had been telling a run to offer panels to nobody. The detail for dashboards, records, meeting transcripts and local folders moved into guides the model loads on a one-line trigger, 5,612, 3,639, 2,027 and 1,205 tokens respectively, with the hint themes, about 2,900 more, following them. A loaded guide is paid once per conversation and read from cache after that.
By the gate, the platform block fell from 61,345 tokens to 41,439 for a chat and 33,669 for a run. Chat's fixed overhead went from 130,033 to 110,403, and a routine run's from between 108,000 and 130,000 to between 79,070 and 100,731. Three structural dedupes took it further. The catalogue of 62 Kits in the dynamic block had been carrying each Kit's full pitch, 9,055 tokens, and now carries one line each, about 2,800, with the pitch behind a lookup. Several Creator Studio tools each carried the same description of one parameter, about 4,000 tokens between them, and now one carries it and the rest point at it. And the tool that emits suggestion chips and hint cards had a description that restated the prompt's rules for it, had drifted into contradicting them, and sent the list of hint themes a second time; it is now a pointer to the one section that holds the rules. Chat's fixed overhead stands at 101,857 tokens at the floor and a routine run's at about 73,000.
That is short of the estimate the design was built on, a core of 25,000 to 35,000 and a chat under 100,000. Chat's tool schemas alone are 44,628 tokens, the 101 tools that stay resident in every chat request, about 41,000 tokens with their descriptions, were chosen by hand, and the section of rules the model uses at the end of most replies cannot become a guide without adding a tool call to nearly every turn.
The same words in a different place: testing a prompt change for quality
The first version of the connector pack, measured in the harness before anything shipped, moved the CRM guidance into the session block under the heading that lists connected services. The agent stopped confirming before writing to the CRM: 2 of 2 runs had confirmed before the move, and 0 of 3 confirmed after it. Moved to the top of the block under a heading that said it binds, 1 of 3. With one sentence back in the core, that a write to a connected system of record needs a yes first even when the user asked for it directly, 5 of 5. The calling connector's rule about offering a destination went the same way, 3 of 3, then 1 of 3, then 2 of 3. The words reached the model unchanged throughout; what changed was that they now sat about 245,000 characters into the prompt, under a heading that read as facts. The same thing happened inside a guide: the dashboard rules were appended at line 390 of a 414-line skill body, were loaded and followed less, and a rubric item passed 0 of 3; moved to the top of the body, 2 of 3, and the scenario went from 82 to 100.
Both were caught before they shipped, and both were caught only because scenarios existed that would fail if the rule stopped being followed. The field's measurements predict the shape without naming it: a bias towards earlier instructions [1]; in the OpenDev write-up, reminders in the user role "consistently produced stronger compliance" than the same reminders in the system role [11]; and in Instruction Bleed, adding one irrelevant entry to a shared rules file shifted a deployed agent's scores (d = 0.63) where adding volume (d = −0.11) or changing form (d = −0.29) did not, in what the authors call "a sub-threshold regime invisible to standard QA" [13]. Position is part of the instruction.
The protocol delivered two findings we did not go looking for. The suggestion-chip tool's description, the one that had restated and then contradicted the prompt's rules, was collapsed to a pointer and freed about 2,500 tokens a request; in the same change, the card that answers a direct question about what the agent can do fired in 3 of 3 runs where it had fired in 1 of 3. A structural change, measured, with an effect. Our one deliberate change of wording, a rewrite of the rule about what a hint card may name, scored 77, 69 and 100 against 77, 73 and 77 on three runs a side, which our thresholds do not call a change, and was reverted the same day. On these rules the model's behaviour appears to be its own salience and not our phrasing, and a change of prose with no measured improvement does not ship. Every trim that stayed is structural.
The protocol's limits are as much a finding as its results. The harness puts a simulated user, the agent and an LLM-as-a-judge in a room and scores the transcript [O3]. It covers flows, not clauses: about 360 scenarios, one model tier per sweep, no memory store, and no dashboard refreshes or workflow stages at all. The spread between two runs of one scenario on one commit reached 38 points, so we call a change only at ten or more points with non-overlapping ranges, and several of this week's apparent regressions turned out to be new scenarios making an old miss countable. "No measured regression" means none in the covered flows at three runs a side. That bar is right for a structural change, which fails in a way a scenario can see. It is not enough for a prose trim, which is why we stopped making them.
What transfers to any agent whose prompt is assembled from parts
For anyone running an agent whose system prompt is written by several hands and sent to several kinds of caller:
- Sort the prompt by rate of change and put a cache breakpoint at each boundary. An uncached volatile block in front of the history still uncaches the history, because the cache looks backward for entries that prior requests wrote, not for content that happens to be stable [5].
- A number that moves with use never belongs in a cached block: not the time, not a count of anything. Per-turn facts go after the last breakpoint, on the live message, in the user's own role, and are not stored.
- Derive what prose a caller gets and what tools it gets from one source, and put a test on their agreement, so a run that loses a tool loses its paragraph in the same step.
- A rule about one parameter belongs beside the parameter; a playbook that spans a family belongs in a guide the model loads; a rule that binds always belongs in the core, near the top.
- Position is part of the instruction. Measure a move before shipping it, with scenarios that fail when the rule stops being followed, and treat a trim made to fit a budget as a quality change rather than a free edit.
- Gate the cache shape, not only the prompt's size: a later turn's first call should write a bounded number of tokens, and the system blocks' hashes should not change between turns. The fault class passes every other gate.
- A tool the prompt names must stay resident. When, in our own use, a deferred tool was once called blind, by a name the prompt had steered the model to, the model guessed its parameters and the call was rejected three times.
What is still open
About half of cold new chats still write 61,000 to 65,000 tokens on their second turn, a few minutes in, on two accounts, for a reason not yet named; every usage record now carries a hash of each system block so that the block which moves can be. The 101 resident tools were chosen by hand; the pod records every tool call, and the next step is to read the call frequency per tool over recent weeks and defer the ones that are rarely called and never named by the prompt, keeping the proactive panels resident. The gate measures a floor of about 102,000 tokens on an account with nothing connected, while a real account measures over 200,000 by the model API's own token count, so the number the gate reports is not yet the number that costs money. And we have not found the second cache miss, the unstored per-turn block that put every history entry out of reach, written down anywhere else; if it is, we would like to read it.
Sources
- [1]D. Jaroslawicz, B. Whiting, P. Shah, K. Maamari. How Many Instructions Can LLMs Follow at Once? 2025. Even the best frontier models reach 68% accuracy at a density of 500 instructions, with a bias towards earlier instructions. arXiv:2507.11538
- [2]K. Harada, Y. Yamazaki, M. Taniguchi, E. Marrese-Taylor, T. Kojima, Y. Iwasawa, Y. Matsuo. When Instructions Multiply: Measuring and Estimating LLM Capabilities of Multiple Instructions Following. EMNLP, 2025. Performance consistently degrades as the number of instructions increases; a logistic regression on the count predicts it. arXiv:2509.21051
- [3]B. Elder, E. Duesterwald, V. Muthusamy. Boosting Instruction Following at Scale. 2025. The decline tracks the tension and conflict that arises as instructions are added; a conflict-scoring tool. arXiv:2510.14842
- [4]K. Hong, A. Troynikov, J. Huber. Context Rot: How Increasing Input Tokens Impacts LLM Performance. Chroma Technical Report, 2025. Eighteen models grow increasingly unreliable as input length grows; focused prompts of about 300 tokens beat full prompts of about 113k; a single distractor lowers performance. trychroma.com/research/context-rot
- [5]Anthropic. Prompt caching. Prefix order tools, system, messages; up to four breakpoints; cache reads look backward for entries prior requests wrote; the worked timestamp mistake; write and read price multipliers. platform.claude.com/docs/en/build-with-claude/prompt-caching
- [6]OpenAI. Prompt caching. Reuse requires the entire rendered prefix to match; stable instructions first, timestamps at the end or in later messages; write and read multipliers. developers.openai.com/api/docs/guides/prompt-caching
- [7]Y. Ji. Context Engineering for AI Agents: Lessons from Building Manus. Manus, 2025. KV-cache hit rate as the production metric; the timestamp mistake; mask, don't remove. manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus
- [8]Anthropic. Tool search tool. About 55k tokens of definitions in a typical multi-server setup, cut by over 85 percent; selection accuracy degrades past 30 to 50 tools; deferred tools stay out of the cached prefix. platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool
- [9]OpenAI. Tool search. Deferred tools are injected at the end of the context window when discovered; eager loading for a small set of functions needed by most tasks. developers.openai.com/api/docs/guides/tools-tool-search
- [10]Anthropic. Equipping agents for the real world with Agent Skills. 2025. Progressive disclosure as the core design principle; a skill's name and description are pre-loaded and its body loaded on relevance. anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills
- [11]N. D. Q. Bui. Building Effective AI Coding Agents for the Terminal: Scaffolding, Harness, Context Engineering, and Lessons Learned. 2026. Prompt sections gated by a condition predicate over a runtime context; a cached stable block and an uncached dynamic block; user-role reminders and compliance. arXiv:2603.05344
- [12]Anthropic. Effective context engineering for AI agents. 2025. Context engineering defined; context as a finite resource with diminishing marginal returns. anthropic.com/engineering/effective-context-engineering-for-ai-agents
- [13]C.-Y. Lin, Y. Liu. Instruction Bleed: Cross-Module Interference in Prompt-Composed Agentic Systems. 2026. Content added to a non-focal module shifts a deployed agent's scores where volume and form do not; none of four surveyed open-source systems tests composition. arXiv:2606.26356
- [O1]The Prompt Is a Budget. 1Presence build log, July 2026.
- [O2]Running Out of Room. 1Presence build log, July 2026.
- [O3]Judging a System That Never Repeats Itself. 1Presence build log, June 2026.