What to Expect From Every New LLM Version: A Field Guide for the Next Upgrade

TL;DR

• A new version is a new coworker, not a faster old one. Capability moves, but so do behavior, defaults, API surface, cost structure, safety rules, and the harness around the model.
• The surprise is usually attribution, not capability. When Opus 5.5 started “planning by default” for one engineer in October 2026, three documented causes could each produce it — an effort setting, a trained model change, or a harness prompt — and vendor docs alone cannot separate them.
• Both Anthropic and OpenAI document auditing your instruction files and re-testing on representative tasks. Their effort guidance diverges: Anthropic says re-run your effort sweep rather than carrying a setting over; OpenAI says preserve your current effective effort and compare. Removing obsolete workarounds is Anthropic’s advice specifically.
• Cost per token is the wrong unit. On Opus 5.5, input/output prices fell 20% but cache reads fell 60%, and Anthropic reports cost per task down 40% at default settings. Find which token type dominates your workload, then measure tokens per task.
• The seven-category checklist below is editorial inference from the September–October 2026 releases of Claude Opus 5.5, GPT-6 Astra, and Gemini 4 Argon. The audit/re-test core of the ritual in the next section is vendor-documented; the effort guidance diverges. Keep those confidence levels separate.

What You Will Learn Here

• A worked case: three documented causes for one surprise behavior, and what evidence separates them
• The seven things worth checking on every new LLM version, with the confidence level of each
• The upgrade ritual Anthropic and OpenAI document, in their own words — including where they disagree
• A 90-minute version-upgrade protocol you can run before swapping a model ID in production
• A three-arm experiment that attributes unexpected behavior to model, effort, or harness
• How to think about cost when per-token prices fall unevenly across token types

The Upgrade That Wasn’t Just a Model

In October 2026, an engineer switched to Claude Opus 5.5 at high effort and noticed something they had not seen before: the model generated a plan by default. The obvious explanation — “the new training made it plan” — turns out to be only one of three documented candidates:

1. Effort. Anthropic’s effort documentation says higher effort levels might “Explain the plan before taking action,” while lower levels “Proceed directly to action without preamble.” But that section is generic to every effort-supporting model, and Opus 5 also ran at high when effort was omitted. Effort explains a plan at high; it cannot explain why the plan appeared now, against an equal-effort Opus 5 baseline.
2. Model change. The Opus 5.5 what’s-new page reports “more thinking per turn at a given effort level” than Opus 5 — a trained change. But note the display problem: thinking blocks are returned with an empty thinking field at the default display: "omitted" setting, and text between tool calls now arrives inside thinking blocks too. So more thinking only becomes a visible plan if your tool displays thinking summaries. That bridge is an inference, and it depends on a harness setting.
3. Harness. Agent tools inject their own instructions. In the session that produced this article, the hosting harness injected a “write a complete plan first” instruction before the model did anything — a plan-first behavior manufactured entirely outside the model. Whether removing it would remove the plan is, again, an inference: the counterfactual was never run.

A search for corroboration came back thinner than expected. Hacker News phrase searches (“plans before”, “asks before acting”) returned nothing — though a bare “opus 5.5” search returns thousands of hits, so that proves phrase absence, not topic absence. GitHub’s anthropics/claude-code issues contain an adjacent report (issue #103624, October 20, 2026): a user whose Claude Projects orchestrator “asks instead of acting. It added a ‘go ahead?’ step before every action… In a plain terminal session none of this happens.” The model is not named, so it does not tie Opus 5.5 to unprompted planning — but the same user, two setups, behavior in only one of them is exactly what a harness cause looks like. A second issue (#105782, October 28, 2026) names Opus 5.5 but reports the opposite-adjacent behavior: deferring related work to future sessions.

The honest status of the original observation: plausible trained change, untested. The test that would settle it is in the worked example below. First, the general lesson.

Seven Things That Change on Every Version

Editorial inference (medium confidence): this taxonomy is extrapolated from three September–October 2026 releases (Opus 5.5, September 22; GPT-6 Astra, September 3 per this site’s earlier coverage; Gemini 4 Argon, October 13). Treat it as a checklist of things to verify, not a law.

Two cross-vendor data points support the pattern. OpenAI’s GPT-6 guide warns that Astra is “more sensitive to instructions contained in skills and other files, such as AGENTS.md” — your accumulated prompt files bite harder on the new model. Google’s Gemini 4 Argon post (October 13, 2026) describes a model trained to pause for human confirmation before irreversible steps, plus an automated monitor that reads the agent’s reasoning in real time. Initiative, caution, and oversight are all being re-tuned every generation — in different directions per vendor.

The Vendor-Documented Upgrade Ritual

This part is mostly not inference — with one correction to the usual telling. Each source below is a migration note for a single release, so “do this on every version” is the pattern I infer from them; what each vendor actually said is quoted.

Shared by both vendors (documented): audit instruction files, and re-test on representative tasks.

• Audit your prompt artifacts. Claude Code 2.1.283 shipped /doctor prompt-audit to audit “your CLAUDE.md files, skills, agents and commands for prompting patterns written for older models.” OpenAI’s GPT-6 guide: “We strongly recommend auditing skills and other files accessible to your model for instructions that could influence its behavior.”
• Re-test before trusting. OpenAI’s migration quickstart pairs its effort advice with “compare results on representative tasks”; Anthropic’s what’s-new tells you to set effort explicitly and re-run your sweep rather than carrying results over.

Where they diverge: effort.

• Anthropic on Opus 5.5: “Re-run your effort sweep rather than carrying a setting over.” A setting tuned on the old model is a stale assumption on the new one. (For how much the sweep matters, see Do we need deep thinking?)
• OpenAI on GPT-6: “Preserve your current effective reasoning effort where supported,” then compare on representative tasks. Carry the setting over as your starting point — the opposite default.

Anthropic-only: delete obsolete workarounds. “prompt-side vision workarounds built for earlier models might no longer be needed.” Every generation absorbs some of your prompt hacks into the model — and the leftover hacks become behavior bugs.

If you do nothing else on upgrade day, audit your instruction files and re-test on your own tasks. Then handle effort your vendor’s way — or better, measure both ways, which is what the protocol below does.

Implementation: A 90-Minute Version-Upgrade Protocol

Run this before a new model version touches production traffic. It assumes you have a small golden task set; if you don’t, building one is step zero.

The recording discipline in step 2 is the part teams skip. Note the form of any surprise, not just its existence. That is what makes the next section possible.

Worked Example: Attributing “Plans by Default”

When a new version does something unexpected, resist naming a cause until you have run the attribution test. A before/now comparison inside one tool cannot confirm a model change — the harness may have updated in the same window (see item 6 above). It can only rule causes out or keep them alive. The raw-API arms are what isolate the model.

flowchart TD
    A[Unexpected behavior on new version] --> B{What form does it take?}
    B -->|Inside a displayed thinking summary| C[Fits: model change via display bridge]
    B -->|Prose preamble in a text block| D[Fits: effort effect]
    B -->|Todo list, plan doc, approval card| E[Fits: harness prompt]
    C --> F[Raw API, empty system prompt<br/>baseline model vs new model, equal effort, 3+ runs]
    D --> F
    E --> G[Harness arm: same tool, both efforts, 3+ runs]
    F --> H{New model plans more<br/>at equal effort?}
    H -->|Yes| I[Model change - trained behavior]
    H -->|Only at high effort| J[Effort effect]
    G --> K{Plans in harness<br/>but not raw API?}
    K -->|Yes| L[Harness cause]
    K -->|No| M[Causes combine - quantify each arm]

Run it as a small grid and count rates, because documented effects are probabilistic (“might explain the plan”). One run settles nothing; three per cell is a floor, ten is better when the split is close:

# Pseudocode — run_once() is your own API/agent call with those settings.
# Attribution sweep: count how often a plan preamble appears per cell.
# Cells: (model, effort, system) — keep prompt and tools identical.
cells = [
    ("baseline-model", "medium", ""), ("baseline-model", "high", ""),
    ("new-model",      "medium", ""), ("new-model",      "high", ""),
]
for model, effort, system in cells:
    plans = sum(run_once(model, effort, system, GOLDEN_TASK).starts_with_plan()
                for _ in range(3))
    print(f"{model} @ {effort}: plan in {plans}/3 runs")
# More plans on new-model at equal effort  -> model change
# Plans only at high, both models          -> effort effect
# Plans in your harness but never here     -> harness cause

In the Opus 5.5 case that opened this article, that test has not been run — which is why the conclusion stays “plausible but untested” rather than becoming a claim about training.

Cost: Per Token Is Not Per Task

Opus 5.5’s pricing (September 2026, vendor-reported) is the cleanest recent example of why per-token headlines mislead:

Anthropic reports cost per task down 40% “at default settings” on typical workloads. That figure mixes three effects: cheaper tokens, fewer tokens per task (tester reports range from Factory’s 20–25% fewer output tokens — 5.5 at medium matching Opus 5 at high — to Kiro’s half the tokens on a command-line benchmark), and a lower default effort — 5.5 defaults to medium where Opus 5 defaulted to high. Crediting the 40% to any single factor is an error; for agentic work, cache reads are the majority of cost, so the −60% line may matter more than the headline −20%.

This is not an Anthropic-only pattern: OpenAI’s GPT-6 guide reports that Astra’s “estimated API cost per task was lower than earlier models despite its higher per-token pricing” — per-task cost falling while the rate card rises, the mirror image of the Opus case.

The transferable rule: price cuts are not uniform across token types. Find which type dominates your workload, then measure tokens per task before and after the switch. In another release the same arithmetic can run against you — more thinking per turn, a new tokenizer, or larger images can outweigh a cheaper rate card. That direction is an inference; the measurement habit is not.

What Not to Expect

• Not monotonic improvement. Point releases can be mostly pricing or safety. Families stagger: Opus 5.5 shipped September 22, 2026 with Sonnet and Haiku “in the coming weeks,” so “the new version” is a rolling window, not an event.
• Not vendor-neutral numbers. Each lab’s launch table is run in its own harness. Anthropic’s own post concedes benchmark margins have become “a less reliable guide to real-world differences” at this level.
• Not community confirmation on demand. Adjacent behavior reports (an orchestrator that asks before acting; a model that defers work) are not reports of your behavior. Adjacent is a hypothesis generator, nothing more.
• Not a settled cause without an isolating test. If your evidence is “same tool, new model, new behavior,” you have a correlation with at least three documented candidate causes.

Sources

• What’s new in Claude Opus 5.5 — accessed October 29, 2026 — thinking always on, default effort medium, more thinking per turn, thinking blocks tied to model and conversation, forced tool use removed.
• Effort — accessed October 29, 2026 — “Effort with tool use”: higher effort might “Explain the plan before taking action”; lower levels “Proceed directly to action without preamble.”
• Prompting Claude Opus 5.5 — accessed October 29, 2026 — scoped “tends to get to work quickly” guidance and checklist recommendations.
• Introducing Claude Opus 5.5 — September 22, 2026 — pricing by token type, 40% cost-per-task claim at default settings, tester reports of fewer tokens per task.
• Claude Code changelog — accessed October 29, 2026 — 2.1.280 (Opus 5.5 “now the default Opus model”), 2.1.283 (/doctor prompt-audit), 2.1.284 (interactive terminal and VS Code sessions start in auto mode when no permission mode is configured, on every plan and provider).
• GitHub issue #103624 — October 20, 2026 — orchestrator self-added a “go ahead?” step, absent in plain terminal (model not named).
• GitHub issue #105782 — October 28, 2026 — Opus 5.5 deferring related work to future sessions.
• Using GPT-6 — accessed October 29, 2026 — initiative/asking behavior, async tool calling, mid-turn steering, skills-file audit recommendation; migration quickstart: “Preserve your current effective reasoning effort where supported” and compare on representative tasks; estimated API cost per task lower than earlier models despite higher per-token pricing.
• Gemini 4 Argon — October 13, 2026 — pause-before-irreversible-steps training, real-time misalignment monitor.