What does temperature do, and how would you set sampling parameters for a customer's extraction pipeline?
Answer it out loud first, then compare. That's what makes it stick.
Temperature reshapes the model's probability distribution over the next token: low is more deterministic, high is more varied. Top-p and top-k limit which tokens are even candidates. For extraction I want consistency, so historically I would use low temperature. But newer Claude models restrict or ignore these knobs, so I check the model's current docs first and get reliability from structured output, clear instructions, examples and validation instead.
- Temperature scales randomness; top-p and top-k trim candidates
- Low for extraction and classification, higher for ideation
- Check current model support, as newer models limit these knobs
- Get determinism from schemas, examples and validation
- Measure variance by running the same input N times
Mechanics. At each step the model scores every possible next token. Temperature divides those scores before they become probabilities: near 0 the top token nearly always wins, higher values flatten the distribution. Top-p keeps the smallest set of tokens whose probability adds up to p. Top-k keeps the k most likely. Lower values give more repeatable output, higher values more varied wording.
Classic settings. Extraction, classification, routing: temperature at or near 0. Brainstorming, marketing variants: higher. Never tune temperature and top-p together without a reason; change one.
The 2026 wrinkle. Secondary migration guides I found report that Opus 4.7 dropped support for non-default temperature, top_p and top_k, and that later models treat them similarly (some sources say a 400 error, an OpenRouter note says ignored). I could not confirm the exact current behaviour from Anthropic's pages, so in an interview and with a customer I say: check the migration guide for the exact model you deploy, and remove sampling parameters rather than retune them if the docs say so. Also note that temperature 0 never guaranteed bit-identical output even where it was supported.
So where does consistency come from now?
- Structured outputs with a JSON schema so the shape cannot vary.
- Clear instructions plus 3-5 diverse examples (Anthropic's recommendation).
- Enumerated values instead of free text for categories.
- Validation and retry on schema or business-rule failure.
- Prompt and model pinning: Claude model IDs are pinned snapshots, so behaviour does not shift under you.
Measure it. Run each of 100 golden inputs 5 times and report the share of inputs with identical normalised output. Anything under about 98 percent for a field that drives automation needs a fix: tighter schema, better examples, or a validator.
Customer takeaway: consistency is a system property you test, not a single knob you set.
- Is temperature 0 actually deterministic? Why not?
- How would you measure output variance for a customer?
- When would you want higher temperature in production?
- Claude 4.7 migration notes - OpenRouter docs (docs, unknown) Reports that sampling parameters are not honoured on Opus 4.7; confirm against Anthropic's per-model migration guide.
- Structured outputs - Claude docs (docs, 2026-11) Schema-constrained output as the way to get consistent response shape.
- Prompting best practices - Claude docs (docs, 2026-11) Recommends 3-5 relevant, diverse examples for consistency.