A customer's agent runs long tasks with lots of chat history and tool output, and quality drops over time. How do you manage context?
Answer it out loud first, then compare. That's what makes it stick.
I treat context as a budget. First I measure where quality starts to fall, using evals at different context sizes. Then I cut what does not earn its place: clear old tool results, summarise or compact earlier turns, and keep durable facts in external memory the agent can query. Stable instructions stay at the top and cached. Long documents go before the question. And I give the agent a way to retrieve detail on demand rather than carrying it everywhere.
- Measure degradation with evals at increasing context sizes
- Trim: clear stale tool results and thinking
- Compact or summarise older turns
- Externalise memory and retrieve on demand
- Layout: stable prefix cached, documents first, question last
Long-running agents fail in a predictable way: tool outputs and history pile up, signal-to-noise drops, cost and latency climb, and recall of early instructions weakens. Anthropic's docs name this context rot and treat curating context as the main skill.
Step 1: measure, do not guess. Build an eval with tasks that need information from early in the run. Score it at, say, 25k, 100k and 300k tokens of accumulated history. The size where pass rate drops is your effective window, which is often well below the advertised one.
Step 2: the management toolbox (all in Anthropic's docs):
- Tool result clearing: drop old, large tool outputs once the agent has used them. Cheap and usually the biggest win.
- Thinking block clearing: on models that keep previous thinking blocks by default, clear what is no longer needed.
- Server-side compaction: the API summarises earlier turns so the conversation continues past the window. Beta, for Claude 4.6 and later.
- External memory: write decisions, constraints and state to a file or store, and reload at session start. Docs describe a memory tool and multi-session patterns for this.
- Defer tool definitions with tool search if the agent has dozens of tools.
Step 3: layout. Keep the system prompt and tool block as a stable, cached prefix. Put long reference documents near the top and the live question at the end; Anthropic reports up to a 30 percent quality gain for queries at the end on complex multi-document inputs.
Step 4: guardrails. For long documents, have the model extract word-for-word quotes before answering. Add a periodic re-statement of the goal and constraints so they are not buried 200k tokens back.
Trade-off to name: compaction is lossy. If a detail must survive (a customer ID, a compliance rule), pin it in the system prompt or memory rather than trusting a summary.
Numbers: report effective-window size, tokens per task before and after, and pass rate before and after.
- What would you never let compaction summarise away?
- How is clearing tool results different from compaction?
- How do you test that an early instruction is still being followed at 300k tokens?
- Anthropic Forward Deployed Engineer interview guide (prep, unknown) Says the hiring manager round may ask how you handle long-context tasks, memory and chat history without degrading output.
- Context windows - Claude docs (docs, 2026-11) Context rot, server-side compaction, tool result clearing and thinking block clearing.
- Prompting best practices - Claude docs (docs, 2026-11) Put long data at the top and queries at the end, with up to 30 percent improvement reported.