Explain the context window to a non-technical customer exec, and tell me what actually counts against it.
Answer it out loud first, then compare. That's what makes it stick.
The context window is the model's working memory for one request: everything it can look at while answering, including its own answer. It is not training data and it does not persist between calls. System prompt, conversation history, tool definitions, tool results, documents, images and the output all count, and thinking tokens count too. Current models go up to one million tokens, but bigger is not automatically better, because accuracy can degrade as it fills.
- Working memory per request, not long-term memory
- List what counts: system, history, tools, results, files, output, thinking
- Size today: up to 1M input, 128k output on current models
- More context can mean worse recall (context rot)
- Curate and measure, do not just fill
The analogy for the exec: the model has no memory between calls. Each request is a desk you lay out from scratch, and the context window is the size of the desk. Whatever is not on the desk, the model cannot see.
What counts (per Anthropic's context window docs): the system prompt, every message including tool results, images and documents, your tool definitions, and the output Claude generates, including thinking. If you use prompt caching, cached tokens still occupy the window; caching changes the price, not the footprint. If input alone exceeds the window the API returns a 400 'prompt is too long' error.
Current sizes. The latest Claude models (Opus 5.5, Sonnet 5.5, Haiku 5.5, Fable 5.2) list a 1M-token window and up to 128k output tokens. A request can carry up to 600 images or PDF pages. Some older models stay at 200k.
The part seniors add: more is not better. The docs call out context rot: as token count grows, accuracy and recall degrade, so curating what goes in matters as much as capacity. A practical test is to run your eval set at 20k, 100k and 400k tokens of context and plot quality, not trust the advertised limit.
Customer-facing implications:
- Cost and latency scale with the filled window. Sending 600k tokens on every turn of a chat is expensive and slow even if it fits.
- Long histories need management. Options are server-side compaction, clearing old tool results, or summarising and storing facts in external memory.
- Put long documents first and the question last. Anthropic's guidance says queries at the end can improve quality by up to 30 percent in tests on complex multi-document inputs.
Say this to close: the window is a budget to be spent deliberately. For each deployment I decide what must be on the desk (instructions, the relevant records) and what should be fetched on demand (retrieval, tools).
- Does prompt caching make a long context free?
- What happens when a conversation grows past the window?
- How would you test whether a customer's long inputs are hurting accuracy?
- Context windows - Claude docs (docs, 2026-11) What counts toward the window, 1M size, context rot, and the 400 overflow error.
- Anthropic Forward Deployed Engineer interview guide (prep, unknown) States the technical screen focuses on reliability, context-window management and consistent production output.