How Long Should an LLM Response Be? Deriving a Length Budget From Human Reading Capacity

Every engineer I know has the same complaint about 2026 frontier models: the answers are too long. You ask a scoped question and get an essay. You review an agent’s report and realize somewhere around screen three that you stopped reading and started pattern-matching.

The usual response is a style preference: “be concise.” That fails in practice because it has no number attached. Concise compared to what? A reviewer cannot enforce a vibe, and neither can a system prompt.

So put a number on it. Human reading speed and human review capacity have both been measured, repeatedly, across large samples. Adults read English non-fiction at 238 words per minute, so thirty seconds of a reader’s attention buys about 120 words, or 160 output tokens. Five minutes buys 1,200. Decide how much of someone’s time the answer is worth, and the ceiling falls out of the arithmetic.

Below is that derivation: a budget per response class, the four conditions that justify going over it, and what it takes to enforce in production. The uncomfortable part is what the review evidence shows about where readers quietly stop looking.

Too Long For Whom, Doing What?

“Is this response too long?” is unanswerable as stated. The answerable version is: too long for whose attention, doing what task, with how many seconds available? That turns an argument about taste into arithmetic. The ceiling falls out of two measured quantities: how fast people read, and how long they spend before they stop.

I could not find published per-model measurements of average output length for current frontier models, including Claude Opus 5. The complaint that they are too verbose is widespread and matches my own experience, but I treat it here as practitioner observation rather than established fact. That absence is itself part of the problem — we measure model quality obsessively and model verbosity almost not at all.

What Reading Actually Costs

The foundational number comes from Marc Brysbaert’s 2019 meta-analysis in the Journal of Memory and Language, covering 190 studies and 18,573 participants. The mean for silent reading of English non-fiction is 238 words per minute, with most people falling between 175 and 300. That is about 20% lower than the 300 wpm figure commonly cited in popular writing, which Brysbaert attributes to small samples and inconsistent methods.

That gives the core conversion. At 238 wpm, one minute of reader attention buys roughly 238 words. OpenAI’s tokenizer documentation puts English at approximately three-quarters of a word per token, so one minute of attention is roughly 315 output tokens.

But reading speed is the optimistic bound, because it assumes the reader reads. They do not. Nielsen Norman Group’s 2008 analysis of page-visit data found that users have time for at most 28% of the words on an average page. That ceiling assumes they spend the entire visit reading, which is why the realistic figure is nearer 20%. Their model prices attention directly: about 25 seconds of fixed time, plus 4.4 seconds for every additional 100 words. Treat this as a directional bound, since it measures web-page behavior rather than chat and predates the current reading context by nearly two decades.

The most useful number in that study is this one: readers get through half the content only on pages of 111 words or fewer. Past that, the proportion consumed falls away quickly.

So the accounting is two-layered: reading speed tells you what fits in the time, and scanning behavior tells you what fraction of it lands.

The Length Budget, Derived

Pick the time a reader will realistically give the response, multiply by 238 words per minute, then divide by 0.75 to get output tokens.

The 30-second routine answer lands at roughly 120 words, close to Nielsen Norman Group’s 111-word threshold for content readers actually finish. That is a useful sanity check rather than independent corroboration — I chose the 30-second budget — but it does suggest the short tier is set somewhere sensible.

The 12-minute tier is not arbitrary either. My own readability tooling in this repo flags anything above roughly 2,500 words or 12 minutes of reading as carrying length fatigue.

Measure this article against that gate and the answer depends on what you count. Body prose alone runs just under 3,000 words, about 12 minutes at 238 wpm. Add the sources and related links and it is roughly 3,600. Add the tables and code and it is about 4,000. All three exceed the tier, but by anything from 60 words to well over a thousand — which is exactly the kind of detail a length budget has to pin down before it means anything. I fix that rule in code later in this article.

Why Reviewers Give Up Before They Complain

The reading numbers explain fatigue. The review literature explains something worse: long output does not merely tire reviewers, it measurably hides defects.

The sharpest evidence comes from Fregnan and colleagues at ESEC/FSE 2022. They analyzed review comments across 219,476 pull requests from 138 Java projects and found that files shown earlier receive more comments than files shown later, even after controlling for confounds. They then ran a controlled experiment with 106 developers, seeding two unrelated defects into different files and switching their positions. One defect type showed no positional effect. For the other, a corner-case defect, participants had 64% lower odds of identifying it when its file was displayed last rather than first.

I read that as a statement about response length, and the transfer is an inference worth flagging. Fregnan measured file position in code review, not paragraph position in generated prose, and I have found no study testing that jump directly. But the mechanism is the same: finite attention, spent in order. When your agent buries a critical caveat in paragraph fourteen, assume it has functionally hidden it.

The surrounding numbers reinforce the point:

• At Google, the median change is 24 lines across roughly nine million reviewed changes. The median review has one reviewer and is approved in under four hours. Small units are what make review work.
• At Microsoft, an analysis of 570 review comments found only about 14% actually identified defects, despite defect-finding being the top stated motivation. Review attention is scarcer than review activity suggests.
• Google’s Tricorder work set explicit alert-fatigue thresholds. Analyzers whose findings are marked not-useful at or above 10% of the time go on probation, and 25% is grounds for disabling them. A quarter noise is where humans stop believing a channel.

None of that describes readers who lack discipline. It describes a finite resource being spent.

The cost shows up at the organizational level too. A BetterUp Labs and Stanford Social Media Lab survey of 1,150 US desk workers, published in Harvard Business Review in September 2025, found 40% had received AI-generated “workslop” in the previous month. Each incident cost an average of just under two hours. That is self-reported data, so read it as perception rather than measurement. Set it beside the METR randomized controlled trial, which found experienced developers 19% slower with early-2025 AI tools while believing they had been 20% faster. METR did not measure verbosity, but both results point at the same blind spot: the people closest to AI-generated work underestimate what it costs.

Verbose output does not fail loudly. It fails by being approved.

The Four Conditions That Justify Expanding

Cutting length by default is only half a policy. The other half is knowing when a longer answer is correct.

The usual advice is to expand when the task is “high stakes,” which is too vague to act on because every engineer believes their task is high stakes. The operational version is a comparison: expand when the cost of a missed detail exceeds the cost of the reading time. That test decomposes into four triggers, and each one promotes the response exactly one tier in the table above.

The triggers themselves are not surprising, and you will have seen versions of them before. What matters is the third column, because a budget is only operational if it says what the extra tier is allowed to spend its words on.

If a longer response is not spending its extra words on that specific content, it is not expanding — it is padding. Hedging, restating the question, summarizing what was just said, and listing caveats that do not change the action all fail the test.

flowchart TD
    Start["Response ready to send"] --> Class{"Which response class?"}
    Class -->|"Status / routine"| Short["Budget: 20-120 words"]
    Class -->|"Explanation"| Mid["Budget: ~475 words"]
    Class -->|"Decision / teaching"| Long["Budget: 1,200-2,850 words"]

    Short --> Test{"Does any expansion<br/>trigger apply?"}
    Mid --> Test
    Long --> Test

    Test -->|"Irreversible action"| Expand["Expand: promote one tier"]
    Test -->|"Reader reproduces reasoning"| Expand
    Test -->|"Artifact is evidence"| Expand
    Test -->|"Reader lacks the schema"| Expand
    Test -->|"None apply"| Enforce["Hold the budget"]

    Expand --> Layer["Answer first,<br/>detail below the fold"]
    Enforce --> Ship["Ship"]
    Layer --> Ship

Note what the expansion path does: it still puts the answer first.

Enforcing the Budget: What Works, and What Only Looks Like It Works

Teams reach for three mechanisms. They are not equally effective.

Asking in the prompt is unreliable. Yuan and colleagues built length-instructed versions of AlpacaEval 2 and MT-Bench and measured violation rates directly. GPT-4 (0409) exceeded the stated limit on 49.3% of AlpacaEval-LI responses and 44.2% on MT-Bench-LI. Smaller instruction-tuned models did better — Llama3-8B-Instruct violated 7.0% and 20.0% respectively — but a coin-flip compliance rate from a frontier model means prompt text alone is not a control. This is not a solved 2024 problem. In a 2026 ACL study, GPT-5 scores 95.9% on IFEval, but on word-count constraints specifically its consistency drops from 96.0% to 88.0% once the same request is rephrased ten ways. Weaker models collapse: Qwen3-32B falls from 83.1% to 40.0%. The same work found the failure brittle in an unsettling way: changing a constraint from “at most 600” to “at most 610” flips many previously passing cases. That is not the behavior of a system that understands the limit.

max_tokens is a safety limit, not a length control. It does not make the model write a shorter answer; it stops generation mid-flight. Anthropic returns stop_reason: "max_tokens", with documented guidance to raise the cap or continue the response; OpenAI signals the same with finish_reason: "length". Both return HTTP 200, so a truncated answer reaches your user looking exactly like a complete one unless you check the field. For structured output that is worse than useless: you get invalid JSON.

Verbosity controls are the real lever, within limits. OpenAI’s GPT-5 family exposes verbosity (low, medium, high) nested under text in the Responses API, alongside reasoning.effort. Their documentation is explicit that verbosity “influences detail level but not hard limits” — it shifts the distribution, it does not impose a ceiling.

So the working pattern is layered. Set the verbosity control, state the budget in the system prompt, and cap max_tokens generously as a runaway guard. Then verify what comes back.

type ResponseClass = "status" | "routine" | "explanation" | "decision" | "teaching";

// Derived from 238 wpm (Brysbaert 2019) at ~0.75 words per token.
const WORD_BUDGET: Record<ResponseClass, number> = {
  status: 20,
  routine: 120,
  explanation: 475,
  decision: 1200,
  teaching: 2850,
};

const FENCE = new RegExp("`{3}[\\s\\S]*?`{3}", "g");

// Prose only: code blocks and tables carry their own budget.
const countWords = (text: string) =>
  text
    .replace(FENCE, " ")
    .split("\n")
    .filter((line) => !line.trim().startsWith("|"))
    .join(" ")
    .trim()
    .split(/\s+/)
    .filter(Boolean).length;

export function checkBudget(text: string, cls: ResponseClass) {
  const words = countWords(text);
  const ceiling = WORD_BUDGET[cls];

  return {
    words,
    ceiling,
    tokenCeiling: Math.round(ceiling / 0.75),
    overBy: Math.max(0, words - ceiling),
    violated: words > ceiling,
    readingMinutes: +(words / 238).toFixed(1),
  };
}

Write it down, because a budget you cannot compute is one nobody enforces.

Measuring Whether the Budget Holds

Treat length like any other regression: measure a rate.

Violation rate is the primary metric, and the length-instructed benchmark work gives you both the method and a reference point. Run your production prompt distribution, classify each response, compare word count against its budget, and report the percentage over. For scale, GPT-4 sat near 49% on that benchmark in 2024, and even GPT-5 loses eight points of word-count consistency under rephrasing in 2026. Pick your own target; I use 10% as an arbitrary but stable line.

Two secondary checks keep the metric honest, because optimizing length alone is easy to game:

• Coverage. Have a judge model verify that the required facts survived compression. Length without coverage is just truncation, and this is where over-compression actually hurts.
• Answer position. Check that the conclusion appears in the opening. The Fregnan result implies that content early in the response is what actually gets read, so burying the answer is a defect even inside a compliant word count.

Track violation rate and coverage together. Either one alone points somewhere bad.

The Gap in My Own Toolchain

Writing this exposed an inconsistency in my own setup.

This repo runs a readability audit over every article I publish. It computes a Read Difficulty Index, Flesch-Kincaid grade, Gunning Fog, SMOG, average sentence length, the share of sentences over 25 words, and reading time at 200 words per minute. The thresholds are real: Read Difficulty under 55, grade level 10 to 13, sentences averaging 15 to 22 words, and a length-fatigue flag above roughly 2,500 words.

So prose I write by hand passes a quantitative length gate. Prose a model generates for me passes nothing at all.

That asymmetry is everywhere. We have spent two years building evals for accuracy, safety, and tool use, and almost none for whether output is the right size for the person who reads it. The measurement problem is not hard — the code above is a lookup table and a word count. We have simply never treated it as a quality dimension.

One caveat, since I rely on those formulas: Flesch-Kincaid and its relatives were built for graded school texts, not technical documentation. They reward short sentences and short words, so a response can improve its score while getting worse. Use them as a drift alarm, not a target.

Closing the Gap: A Skill That Counts and an Agent That Judges

So I built the missing gate. It is two files, and the split between them is the entire design.

The script counts. The agent decides what the count means. Never let a model measure its own output. The evidence above is the reason: GPT-5 loses eight points of word-count consistency under rephrasing, and Qwen3-32B falls by half. A model that cannot reliably obey a length constraint cannot reliably audit one either. So the number comes from 150 lines of dependency-free Python, and the model only gets to reason about it.

The skill does the measuring

.claude/skills/response-budget/scripts/budget.py takes a file or stdin and a response class:

python3 .claude/skills/response-budget/scripts/budget.py --class routine reply.md

Response budget - class: routine
==============================================
  Verdict        : OVER BUDGET
  Words          : 148 (ceiling 120)
  Over by        : 28
  Reading time   : ~0.6 min at 238 wpm (budgeted 30s)
  Token ceiling  : ~160
  Opening para   : 8 words
  WARNING        : opens with preamble, not the answer
  ACTION         : cut 28 words, or justify the overage with an expansion trigger

It counts prose only, excluding code blocks and table rows, because they are not sentences and folding them in makes the number meaningless. It reports the opening paragraph separately, since that is the part guaranteed to be read. It flags responses that open with filler instead of the answer. And it exits 1 when over budget, which is all a CI job needs.

The agent does the judging

.claude/agents/response-budget-auditor.md wraps that number in the decision the script cannot make. It classifies the response before measuring it, so the class cannot be chosen after the fact to make a violation disappear. It runs the script, confirms the conclusion sits in the opening, then tests any overage against the four expansion triggers and promotes the response at most one tier.

Its most useful rule is a refusal: reject “the topic is complex” and “the user might want more detail” as justifications. Both are unfalsifiable, and both are how budgets quietly die.

Wiring it into a workflow

• Claude Code picks up .claude/agents/ and .claude/skills/ automatically. Ask for the response-budget auditor by name on any generated artifact, or invoke the skill directly when you only want the number.
• Cursor has no equivalent auto-discovery, so the same role lives as a prompt template in .cursor/subagents/response-budget-auditor.md, pasted into a general-purpose subagent. The script is shared, so both clients measure identically.
• CI needs no agent at all. The exit code is the gate: budget.py --class decision generated-report.md fails the job when a generated artifact outgrows its reader.

The honest placement is as a review step, not a generation step. Length is a property of finished output, and gating at the end is what keeps the budget from becoming another instruction the model half-follows.

What it says about this article

Running it on this article, classed as teaching, returns over budget by about 800 words. That is a real result and I am leaving it visible, because two things fall out of it.

The first is that fixing the counting rule in code settles the ambiguity from earlier. The script counts sources and related links as prose and excludes code and tables. That is now the rule, not a judgment I re-make each time, and it is why the script’s number is higher than the body-prose figure I quoted above.

The second is that an overage of more than a quarter has to clear my own bar. I think it does, on the fourth trigger: a reader who does not already have this method cannot expand a compressed version of it, and the extra words go to the definitions and worked examples that trigger demands. That is also the most abusable of the four, so it is worth saying what would change my mind. If those words were going to hedging, restatement, or a second example that taught nothing new, the trigger would not apply and I would owe you a cut.

The Practical Version

If you take one thing: pick the reader’s time budget first, and let the length fall out of it. Thirty seconds is 120 words. Five minutes is 1,200. The arithmetic is not the hard part.

The hard part is accepting what the review evidence says. The back half of a long response is demonstrably under-reviewed, so a caveat you put there is hidden, not communicated. When a model answers in 2,000 words and 200 would do, it has not been thorough — it moved the cost of its uncertainty onto you, and you will pay it by approving something you did not read. Pick the time budget before the model picks it for you.

Related Reading

• Why Concise AI Responses Work Better — the cognitive-load case for concision and progressive disclosure, which this article puts numbers behind.
• Is AI Generating a World of Code Slop? — the same review-capacity argument applied to generated code volume.
• How to Review AI-Generated Responses — the rubric for judging what you do read.

Sources

• Brysbaert, M., How many words do we read per minute? A review and meta-analysis of reading rate, Journal of Memory and Language 109, article 104047, December 2019 - 190 studies and 18,573 participants; 238 wpm silent reading of English non-fiction, typical range 175-300 wpm.
• Nielsen, J., How Little Do Users Read?, Nielsen Norman Group, May 6, 2008 - at most 28% of words read per average visit, 20% more realistic; readers finish half the content only at 111 words or fewer.
• Fregnan, E., Braz, L., D’Ambros, M., Çalıklı, G., and Bacchelli, A., First Come First Served: The Impact of File Position on Code Review, ESEC/FSE 2022 - 219,476 pull requests plus a 106-participant controlled experiment; 64% lower odds of detecting a corner-case defect when its file is shown last. The second seeded defect type showed no positional effect.
• Sadowski, C., Söderberg, E., Church, L., Sipko, M., and Bacchelli, A., Modern Code Review: A Case Study at Google, ICSE-SEIP 2018 - approximately nine million reviewed changes; median change size 24 lines, median one reviewer, median time to approval under four hours.
• Bacchelli, A., and Bird, C., Expectations, Outcomes, and Challenges of Modern Code Review, ICSE 2013 - analysis of 570 review comments; roughly 14% identified defects despite defect-finding being the top stated motivation.
• Sadowski, C., van Gogh, J., Jaspan, C., Söderberg, E., and Winter, C., Tricorder: Building a Program Analysis Ecosystem, ICSE 2015, pages 598-608 - alert-fatigue governance thresholds; a not-useful rate at or above 10% places an analyzer on probation, and above 25% it may be disabled.
• Yuan, W., Kulikov, I., Yu, P., Cho, K., Sukhbaatar, S., Weston, J., and Xu, J., Following Length Constraints in Instructions, arXiv preprint, June 2024 - GPT-4 (0409) violated explicit length limits on 49.3% of AlpacaEval-LI and 44.2% of MT-Bench-LI responses; Llama3-8B-Instruct violated 7.0% and 20.0%.
• Dong, J., Zhang, Y., Liu, Y., Zhong, Z., Wei, T., Zhang, C., and Qiu, H., Revisiting the Reliability of Language Models in Instruction-Following, ACL 2026, pages 7784-7812 - a pilot over 10,944 instantiated prompts (Qwen3-8B) showing that a small change to a requested word count, such as “at most 600” to “at most 610”, flips many previously passing cases. On the number words constraint the drop from single-prompt accuracy to reliable@10 across ten rephrasings is 96.0 to 88.0 for GPT-5 and 83.1 to 40.0 for Qwen3-32B. Note this measures consistency across rephrasings, not a raw violation rate.
• OpenAI, Controlling the length of OpenAI model responses - the GPT-5 verbosity control accepts low, medium, and high, and “influences detail level but not hard limits”; reasoning.effort is documented separately.
• OpenAI, Understanding and counting tokens - official English rules of thumb: one token is approximately four characters, approximately three-quarters of a word, and 100 tokens is approximately 75 words.
• Anthropic, Handling stop reasons - stop_reason: "max_tokens" indicates a truncated response, with the documented remedy being to raise max_tokens or continue the response.
• METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, July 10, 2025 - randomized controlled trial with 16 developers across 246 tasks; 19% slowdown against a forecast 24% speedup and a post-hoc belief of 20% speedup. METR now banners this page as out of date and points to a February 2026 follow-up. Treat the 19% as a result about early-2025 tooling, not current tooling.
• Niederhoffer, K., Kellerman, G. R., Lee, A., Liebscher, A., Rapuano, K., and Hancock, J. T., AI-Generated “Workslop” Is Destroying Productivity, Harvard Business Review, September 22, 2025 - BetterUp Labs and Stanford Social Media Lab survey of 1,150 US desk workers; 40% received workslop in the prior month. Self-reported survey data, useful as perception rather than measurement. The HBR body text is paywalled. The sample size, the 40%, and the time cost are confirmed on the open BetterUp research page, which states it as “2 hrs”. The more precise 1 hour 56 minutes figure circulates only in secondary coverage, so it is not used here.