How Fast Is gpt-oss-120b? Speed, Quality, and Routing Tradeoffs in 2026

TL;DR

• gpt-oss-120b is a published OpenAI open-weight model. OpenAI released it on August 5, 2025 under Apache 2.0. It is a MoE model with 117B total parameters, 5.1B active parameters per token, 128K context, native MXFP4 quantization, and an advertised single 80GB GPU deployment target.
• Fast depends on the benchmark layer. BenchLM’s Artificial Analysis rollup, last updated June 19, 2026, lists gpt-oss-120b at ~262 tok/s with a composite quality score of ~34. Artificial Analysis provider-level pages show much higher numbers for optimized hosted providers, especially Cerebras, because they measure a specific model/provider pairing.
• OpenRouter trends make the provider point visible. OpenRouter’s gpt-oss-120b performance page shows throughput, latency, uptime, and provider trends over time; its docs also expose throughput-high-to-low sorting and the :nitro shortcut for routing to faster providers.
• The engineering win is controllability, not universal model quality. gpt-oss-120b is attractive when you need open weights, local or private deployment, provider choice, and fast enough decode. For high-risk agentic work, treat it as a candidate to evaluate, not a default replacement for every closed model.
• There are credible alternatives if you want faster LLMs that are still smart enough. As of July 3, 2026, the practical shortlist is not only gpt-oss-120b: Gemini 3.5 Flash, GLM-5.2 (max), Qwen3.7 Max, Grok 4.20/4.3, GPT-5.4 mini-class models, Command A+, Nemotron, Mercury 2, and provider routes through Cerebras, Groq, SambaNova, Fireworks, Baseten, Nebius, Google Vertex, Azure, Cloudflare, Together AI, and DeepInfra are all worth testing depending on the workload.

What You Will Learn Here

This article is written for engineers deciding whether gpt-oss-120b belongs in their stack and how to read the speed claims around it.

• What gpt-oss-120b actually is (architecture, license, context, release date)
• Why BenchLM, Artificial Analysis provider pages, and OpenRouter trends can show different throughput numbers
• How gpt-oss-120b differs from gpt-oss-20b
• Where provider routing changes latency, throughput, uptime, and cost
• Which other published models and providers are worth testing for fast-enough, smart-enough workflows
• The cost stack behind faster inference
• A practical evaluation frame that treats gpt-oss-120b as one candidate, not a universal default

Start Here: What Is gpt-oss-120b?

OpenAI’s published model name is gpt-oss-120b. It is part of the gpt-oss open-weight family alongside gpt-oss-20b.

OpenAI positions gpt-oss-120b as a strong open checkpoint you can run, customize, and route through several providers. That is different from saying it is the best model for every production agent workload.

Engineer asks: "Should I run gpt-oss-120b as my default LLM?"

Only after you test it on your own evals.

Better default split:

    Open weights, private deployment, tunable serving  -> gpt-oss-120b
    Small local/dev path                              -> gpt-oss-20b
    Managed throughput priority                       -> gpt-oss-120b via fastest provider route
    Hard agent loops                                  -> compare against your strongest managed candidates
    Raw tok/s above all else                          -> check current Artificial Analysis/OpenRouter rankings

Sibling model: gpt-oss-20b

OpenAI also ships gpt-oss-20b with 21B total parameters, 3.6B active parameters per token, and an advertised 16GB memory footprint. BenchLM lists it at ~313 tok/s with quality score ~16. Use it for local/dev routing sandboxes and small structured tasks; use gpt-oss-120b when you want more open-weight reasoning headroom.

Why the “fastest and smartest” story breaks down

• Model-level speed: BenchLM’s Artificial Analysis rollup compares models across a common table. In that view, gpt-oss-120b sits in the ultra-fast tier, but it is not the top row.
• Provider-level speed: Artificial Analysis provider pages compare the same model served by different providers. In that view, provider choice can change throughput by multiples.
• Router-level speed: OpenRouter can route the same model to different backends. Its performance page and :nitro routing mode make that tradeoff explicit.
• Capability: A composite benchmark score is not your task accuracy. gpt-oss-120b may be excellent for structured, private, or high-volume work and still lose on a hard coding-agent eval.

gpt-oss-120b is strongest when the requirements include open weights, provider choice, private deployment, or controlled serving economics. It should earn harder agent workloads through eval results, not branding.

Capability: Useful, Not Universal

There is still no single “IQ number” for LLMs. For gpt-oss-120b, the source-backed capability summary is:

• OpenAI’s August 2025 launch post says gpt-oss-120b achieves near-parity with o4-mini on core reasoning benchmarks and supports tool use, configurable reasoning effort, structured outputs, and full chain-of-thought access.
• The same launch post says gpt-oss-20b can run on edge devices with 16GB of memory and has similar results to o3-mini on common benchmarks.
• BenchLM’s Artificial Analysis rollup gives gpt-oss-120b a quality score of ~34 and gpt-oss-20b a score of ~16. Treat those as leaderboard signals, not product requirements.
• For high-risk coding agents, long-context workflows, or tasks where a wrong answer is expensive, run a task-specific bakeoff against your managed-model shortlist.

Long-context engineering agents

For 1M-context agent platforms, this repo already covered the May 2026 landscape in Alternatives to Claude Opus 4.7 1M for Engineers in 2026. The short version: context window, tool behavior, pricing, and IDE integration matter alongside raw speed.

gpt-oss-120b has a 128K context limit in OpenAI’s published launch materials. That can be plenty for many routing and extraction workloads, but it is not a substitute for a 1M-context architecture when the product truly depends on very large working sets.

Fastest in July 2026: Where gpt-oss-120b Ranks

The safest way to read speed claims is to keep three views separate.

1. Model-level rollup: BenchLM / Artificial Analysis

BenchLM aggregates Artificial Analysis speed data and was last updated June 19, 2026. It reports median tokens/s and a first-answer latency metric. For reasoning models, the latency metric can include thinking time, so it is not always comparable to raw time-to-first-token streaming.

This view answers: how fast is the model in a normalized public model leaderboard?

2. Provider-level benchmark: Artificial Analysis

Artificial Analysis also publishes provider-level pages for gpt-oss-120b. Those pages compare providers serving the same model, including Cerebras, Groq, SambaNova, Fireworks, Cloudflare, Azure, DeepInfra, Baseten, Together AI, and others.

The important pattern is not one frozen number. It is that provider choice can dominate the throughput result. On July 3, 2026, Artificial Analysis showed Cerebras leading gpt-oss-120b provider throughput by a large margin, with the next providers clustered hundreds of tokens per second lower. The exact ranking and numbers can move as providers update serving stacks.

This view answers: which provider is fastest for this exact model right now?

3. Router-level trend: OpenRouter

OpenRouter’s gpt-oss-120b page explains that the same model can be hosted by different companies, and that routing mode affects the backend you hit. Its performance page tracks throughput, latency, end-to-end latency, uptime, and provider distribution over time.

In the OpenRouter performance view checked on July 3, 2026, the gpt-oss-120b provider trend showed:

• Cerebras around ~698 tok/s average throughput in the one-week provider trend
• Groq around ~383 tok/s average throughput in the same trend view
• very low average latency for the fastest providers, with uptime and error-rate differences that also matter in production

OpenRouter also exposes this as an integration pattern:

• GET /api/v1/models?sort=throughput-high-to-low sorts model listings by recent median throughput.
• Appending :nitro to a model slug prioritizes providers by throughput.
• Setting provider.sort to "throughput" does the same thing explicitly.

This view answers: how should my app route for speed under current provider conditions?

Engineer asks: "Where does gpt-oss-120b fit?"

    Model-level benchmark
        -> gpt-oss-120b is fast, but not the fastest row in the rollup

    Provider-level benchmark
        -> Cerebras has recently led gpt-oss-120b throughput on Artificial Analysis

    Router-level trend
        -> OpenRouter :nitro / provider.sort can change which backend serves the model

    Local or private deployment
        -> gpt-oss-120b if 80GB-class serving is acceptable

    Smaller local/dev path
        -> gpt-oss-20b

    Hard agentic workflow
        -> benchmark against your managed-model shortlist before routing by default

Alternatives: Faster and Smart Enough

If the real question is “what else should I try besides gpt-oss-120b?”, start with a threshold instead of a brand. For a user-facing app, a useful first pass is:

Candidate model
    |
    +--> Is it fast enough for the interaction?       (tok/s, TTFT, end-to-end latency)
    +--> Is it smart enough for the task?             (eval score + your own task set)
    +--> Is the provider reliable enough?             (uptime, fallback behavior, data policy)
    +--> Is the price acceptable at expected volume?  (input, output, cache, retries)

Model alternatives to test

Using BenchLM’s Artificial Analysis-derived table from June 19, 2026, plus Artificial Analysis pages checked on July 3, 2026, these are the most useful published alternatives to include in a bakeoff:

My practical read: if the phrase is “fastest LLM”, Mercury 2 belongs in the conversation. If the phrase is “fast and smart enough”, Gemini 3.5 Flash, Qwen3.7 Max, GLM-5.2 (max), and the faster Grok rows are stronger first tests. If the phrase is “open or controllable serving”, GLM-5.2 (max), gpt-oss-120b, gpt-oss-20b, and Nemotron-class models are more relevant than raw leaderboard winners. BenchLM’s July 3, 2026 GLM/Qwen comparison frames them as a split decision: GLM-5.2 is stronger for agentic tasks, while Qwen3.7 Max has the edge for coding and knowledge tasks.

Provider alternatives to test

For the same model, provider choice can matter as much as model choice. On July 3, 2026, Artificial Analysis provider pages for gpt-oss-120b showed Cerebras leading throughput, while OpenRouter exposed live provider trends and routing controls.

The best evaluation set is usually small:

Fast + high score
    -> Gemini 3.5 Flash, Qwen3.7 Max, GLM-5.2 (max), Grok 4.20/4.3

Open or controllable
    -> GLM-5.2 (max), gpt-oss-120b, gpt-oss-20b, Nemotron 3 Super 100B

Raw speed baseline
    -> Mercury 2, Command A+, provider-optimized gpt-oss-120b

Provider bakeoff
    -> Cerebras, Groq, SambaNova, Fireworks, Baseten, Nebius, plus your required cloud/provider

Why OpenAI Ships Two gpt-oss Sizes

OpenAI’s August 2025 open-model post frames gpt-oss as a way to give developers open weights that complement hosted models. The two sizes make different serving tradeoffs:

                    +------------------+
                    | gpt-oss family   |
                    | open weights     |
                    +--------+---------+
                             |
            +----------------+----------------+
            |                                 |
            v                                 v
 +---------------------+           +---------------------+
 | gpt-oss-20b         |           | gpt-oss-120b        |
 | 21B total           |           | 117B total          |
 | 3.6B active/token   |           | 5.1B active/token   |
 | 16GB memory target  |           | 80GB memory target  |
 +----------+----------+           +----------+----------+
            |                                 |
            v                                 v
 small local/dev tasks              stronger open reasoning route

The serving economics come from the combination of MoE sparsity, quantization, and routing control. You are not moving every parameter for every token, and you can choose whether to run the model yourself or route through providers that specialize in throughput.

Why Provider Routing Matters

For gpt-oss-120b, “which model?” is only half the decision. The other half is “which provider, with which routing policy?”

OpenRouter’s docs describe several relevant controls:

• provider.order lets you try providers in a fixed order.
• provider.sort: "throughput" prioritizes the fastest provider.
• :nitro is the shorthand for throughput-optimized routing.
• preferred_min_throughput lets you express a soft throughput preference.
• allow_fallbacks controls whether OpenRouter can fall through to another provider.

Those controls are not just operational details. They change user experience. A provider with better throughput may have different first-token latency, price, uptime, data policy, or tool-call behavior. A production router should log all of those fields, not just model name.

Where gpt-oss-120b Fits

Use gpt-oss-120b when the product constraint points toward open weights or controllable serving:

• private or self-hosted inference
• high-volume classification, extraction, transformation, and routing
• workloads where 128K context is enough
• teams that want to fine-tune or inspect an open checkpoint
• speed-sensitive managed inference where provider routing can target a fast backend

Be more cautious when the product constraint points toward maximum task accuracy:

• autonomous multi-step coding agents
• long-context analysis above 128K
• tasks that need multimodal inputs not covered by your gpt-oss serving path
• workflows where provider fallback changes behavior in unacceptable ways
• regulated use cases where self-hosting shifts safety, logging, and audit work onto your team

Where OpenAI is not always #1 on speed

Public speed charts are full of specialists. In the BenchLM rollup, Mercury 2 leads raw model-level output speed. Gemini 3.5 Flash has a much higher quality score than gpt-oss-120b while still landing above it on tokens/sec. On OpenRouter and Artificial Analysis provider views, specialized infrastructure such as Cerebras can dominate a specific model/provider pairing.

That does not make gpt-oss-120b weak. It means the correct sentence is more precise:

The Cost of Fast Inference

“Faster” can mean three different budgets. Engineering orgs routinely confuse them.

Budget 1: Serving an open model yourself

OpenAI’s deployment claim gives the starting point: gpt-oss-120b can run within 80GB of memory thanks to MXFP4 quantization, while gpt-oss-20b targets 16GB. That does not make self-hosting free. You still pay for GPUs, orchestration, monitoring, batching, incident response, security reviews, and eval maintenance.

The self-host path makes sense when utilization, data control, customization, or latency justify owning the serving stack.

Budget 2: Making an existing model faster at serve time

This is where most user-visible “speed” comes from in 2026:

Recent systems research underscores the hidden piece: host-side orchestration overhead. The TaxBreak paper (arXiv:2603.12465, 2026) shows MoE models can dispatch 8-11x more kernels per output token than dense models, making CPU orchestration and launch latency first-order bottlenecks, not just GPU FLOPs.

Another 2026 arXiv paper on MoE inference (“qs inequality,” arXiv:2603.08960) argues that training-time FLOP efficiency is a poor proxy for inference economics, especially at long context, because all experts must stay resident in memory even when only a few activate per token.

gpt-oss-120b is the practical example: 117B total capacity, 5.1B active, MXFP4 on one 80GB GPU class of memory. The serving path is narrower than a dense model with the same total parameter count, but the provider stack still matters.

Practical takeaway: For application teams, the realistic speed lever is usually not training a new model. It is choosing the right model size, provider, batching policy, cache policy, and fallback behavior.

Budget 3: Paying a provider or router for speed

If you do not own the serving stack, you buy speed through routing and provider choice:

• Model size (gpt-oss-20b vs gpt-oss-120b)
• Provider routing (priority processing, dedicated capacity)
• Architecture choices (shorter context, fewer tool roundtrips, smaller output budgets)
• Hybrid routing (fast open model first, stronger managed model on escalation)

That third budget is where most production LLM stacks should live.

If you want a deeper comparison framework for IDE-native coding models specifically, start with Claude Sonnet 4.6 1M vs Composer 2.5 Fast. If you need long-context vendor selection, use Opus 4.7 1M alternatives.

Limitations and What This Article Does Not Prove

Keep these boundaries visible when you wire gpt-oss-120b into production:

• 262 tok/s is a model-level rollup, not a guarantee. BenchLM medians reflect its Artificial Analysis-derived table. Your self-hosted vLLM config, batch size, GPU, prompt shape, and reasoning settings matter.
• Provider-level speed is not model-level speed. Artificial Analysis provider pages and OpenRouter trends can show much higher or lower throughput because they measure a specific backend.
• Quality score ~34 is a leaderboard composite, not your task accuracy. Run your own eval before replacing a managed API default.
• Open weights shift work onto your team. Safety policy, logging, abuse handling, prompt templates, evals, and incident response become part of your implementation.
• Router shortcuts are not a substitute for observability. :nitro and throughput sorting are useful, but you still need logs for provider, latency, fallback, error rate, and cost.

What I’d Watch Next

As of July 3, 2026, these are the signals that could reshuffle the leaderboard:

1. Artificial Analysis provider rankings - whether Cerebras keeps the top gpt-oss-120b throughput position and how close GPU providers get.
2. OpenRouter fast-model trends - whether gpt-oss-120b remains among the fastest practical router choices once uptime, latency, and provider distribution are included.
3. Fine-tuned gpt-oss-120b checkpoints - whether domain post-training closes quality gaps enough for your workload.
4. Longer open contexts - if the gpt-oss family moves beyond 128K, revisit long-context routing assumptions.
5. Serving-stack improvements - vLLM, TensorRT-LLM, speculative decoding, and provider-specific hardware can move the cost/speed line without changing the base model.

Sources

Primary and high-confidence secondary sources used for this article:

• OpenAI - Introducing gpt-oss (August 5, 2025)
• OpenAI - gpt-oss-120b API model docs
• OpenAI - gpt-oss GitHub repository
• Artificial Analysis - LLM leaderboard
• Artificial Analysis - GLM-5.2 (max) intelligence, performance, and price analysis
• Artificial Analysis - Qwen3.7 Max intelligence, performance, and price analysis
• Artificial Analysis - gpt-oss-120b provider performance and price analysis
• BenchLM - LLM speed data sourced from Artificial Analysis (updated June 19, 2026)
• BenchLM - GLM-5.2 vs Qwen3.7 Max comparison (updated July 3, 2026)
• OpenRouter - gpt-oss-120b API pricing and benchmarks
• OpenRouter - gpt-oss-120b performance trends
• OpenRouter - GLM 5.2 API pricing and benchmarks
• OpenRouter - Qwen3.7 Max API pricing and benchmarks
• OpenRouter docs - Models API and throughput sorting
• OpenRouter docs - Nitro high-speed routing variant
• OpenRouter blog - How model routing works
• arXiv:2603.12465 - TaxBreak: Hidden Costs of LLM Inference Through Overhead Decomposition (2026)
• arXiv:2603.08960 - The qs Inequality: MoE Double Penalty at Inference (2026)

Related articles in this devlog:

• Claude Sonnet 4.6 1M vs Composer 2.5 Fast
• Alternatives to Claude Opus 4.7 1M for Engineers in 2026
• 1M vs 200K LLM Context Windows