On the 63.2% SWE-bench Pro score, the 1.35× tokenizer overhead that doesn't appear on the rate card, and when to swap Opus 4.8 for Sonnet 5.
Sonnet 5 does near-Opus work at Sonnet prices. The tokenizer doesn't follow the rate card.
Anti-AI
00
Skeptic
01
Neutral
00
Pro (practical)
02
Pro (hyped)
00
← Anti-AI · Pro-AI →
Anthropic launched Claude Sonnet 5 on June 30 as the new default for Free and Pro users on claude.ai. The headline number: 63.2% on SWE-bench Pro at introductory $2/$10 per million tokens.
Two things that don't fit neatly in that sentence. Opus 4.8 scores 69.2% on the same benchmark. And the rate card isn't the whole story.
What changed with this release
Per Anthropic's announcement, the numbers:
- SWE-bench Pro: 63.2% — vs GPT-5.5 at 58.6%, Gemini 3.5 Flash at 55.1%, and Opus 4.8 at 69.2%
- Terminal-Bench 2.1: 80.4% — GPT-5.5 edges it at 83.4%
- Humanity's Last Exam with tools: 57.4% — Opus 4.8 scores 57.9%, effectively tied
- Context window: 1M tokens
- Introductory pricing: $2/$10 per million tokens through August 31, 2026; then $3/$15
Available now in Claude Code, the Claude API, Cursor, VS Code, and GitHub Copilot. Model ID: claude-sonnet-5.
The availability point matters more than the benchmark numbers might suggest. Sonnet 5 is now the default for every Free user and Pro subscriber on claude.ai — meaning every new conversation starts with this model unless someone explicitly selects Opus 4.8 or Fable 5. Combined with the Fable 5 credits transition landing today, this is the model that carries most of the subscription load going forward.
The 63.2% number in context
63.2% on SWE-bench Pro is genuinely strong for this price tier. Five points ahead of GPT-5.5's 58.6%, eight points ahead of Gemini 3.5 Flash. SWE-bench Pro filters harder than the original SWE-bench — fewer solved instances, more long-horizon tasks — so the spread is meaningful, not cosmetic.
Where the gap remains real: Opus 4.8 at 69.2% is six points ahead. On Terminal-Bench 2.1, GPT-5.5 still leads (83.4% vs 80.4%). Six points on a hard agentic benchmark isn't a rounding error — it compounds across long agent sessions.
Humanity's Last Exam with tools is the interesting data point. 57.4% vs 57.9% is functionally tied. For tool-use-heavy pipelines specifically, the capability gap between Sonnet 5 and Opus 4.8 is narrower than SWE-bench alone implies. That narrows the cases where you actually need Opus.
The tokenizer problem
Sonnet 5 ships with a new tokenizer — the component that splits input text into tokens before the model processes it. The updated tokenizer is more efficient at representing code and structured data, which is part of why agentic performance improved. But efficiency in one direction means a different token count for the same text.
Independent analysis puts the range at 1.0–1.35× depending on content type. Conversational text tends toward 1.0×. Code-heavy prompts land closer to the high end. Anthropic's launch post mentions the tokenizer change in fine print without quantifying the overhead; the disclosure reads more like a footnote than a warning.
What this means practically: the $2/M input rate card can behave like $2.70/M in the worst case on code-heavy workloads. That's a 35% premium that doesn't show up until you see the bill. For high-volume production use, measure actual token counts against your real prompts before migrating — compare the API response usage.input_tokens field between Sonnet 4.6 and Sonnet 5 on identical inputs. If your ratio stays below 1.1×, the overhead is negligible. If it's closer to 1.3×, you need to reprice your cost model.
The intro pricing window runs through August 31, 2026. At regular $3/$15 pricing, the same overhead compounds further. Model the August 31 date into any ROI calculation you're doing right now.
Source spread
- Anthropic — Introducing Claude Sonnet 5 — hype. Official launch post. Benchmark numbers sourced here; tokenizer change disclosed but not quantified.
- TechCrunch — Anthropic launches Claude Sonnet 5 as a cheaper way to run agents — builder. Agentic positioning and API availability details.
- AI Made Tools — Claude Sonnet 5 pricing: the tokenizer catch — skeptic. The most detailed independent analysis of tokenizer overhead published so far. Puts the range at 1.0–1.35×.
- MarkTechPost — Sonnet 5 vs Sonnet 4.6 vs Opus 4.8 — builder. Cost-performance comparison across Anthropic's three model tiers.
Pros & cons
What's real:
- Leading SWE-bench Pro at this price tier by five points over GPT-5.5. That's a clear margin, not a tie.
- Near-parity with Opus 4.8 on Humanity's Last Exam with tools (57.4% vs 57.9%). For tool-use-heavy pipelines, the Sonnet 5 vs Opus 4.8 tradeoff is closer than the headline numbers suggest.
- 1M context at $2/M intro pricing is the cheapest long-context option in Anthropic's lineup. For tasks that fit within Sonnet 5's capability ceiling, the cost equation is compelling.
- Default rollout to all claude.ai users means zero migration friction if you're already using the default API endpoint. The upgrade happened while you were doing something else.
What deserves a side-eye:
- The tokenizer overhead is real and under-disclosed. "Up to 1.35×" means the $2/$10 rate card can behave like $2.70/$13.50 on code-heavy content. That's a material cost difference for high-volume workloads — not a footnote.
- The intro pricing ($2/$10) expires August 31, 2026. Regular pricing is $3/$15. Model that date into any cost projections. A 50% price increase eight weeks from now changes the math on marginal use cases.
- Six points behind Opus 4.8 on SWE-bench Pro. The marketing says "near-Opus." The benchmark says "close but not there." Know which of your workloads live in that gap before fully committing.
- Upgrade path: swap to model ID
claude-sonnet-5; run your eval suite before moving production traffic. - Tokenizer check: compare
usage.input_tokensin the API response between Sonnet 4.6 and Sonnet 5 on identical production prompts. Under 1.1× overhead is negligible; above 1.2× requires repricing your cost model. - Intro pricing closes August 31, 2026: $2/$10 → $3/$15. Build this date into cost projections for any project starting now.
- Humanity's Last Exam with tools at 57.4% — near-tie with Opus 4.8's 57.9%. Tool-use-heavy pipelines are the best Sonnet 5 / Opus 4.8 swap candidates; pure coding tasks less so.
- Not a Fable 5 replacement today: Fable 5 exits subscriptions this morning and moves to $10/$50 usage credits. Sonnet 5 fills the subscription slot, not the capability slot, for the hardest document and reasoning tasks.
- Prompt re-testing required: Anthropic noted instruction-following behavior changed between generations. Existing Sonnet 4.6 or Opus 4.8 prompt suites should be re-evaluated before flipping production traffic.
Further reading
- Anthropic — Introducing Claude Sonnet 5 — official launch post; all benchmark numbers sourced here
- TechCrunch — Anthropic launches Claude Sonnet 5 — agentic positioning and API availability
- AI Made Tools — Claude Sonnet 5 pricing: the tokenizer catch — the most detailed independent tokenizer overhead analysis available
- MarkTechPost — Sonnet 5 vs Sonnet 4.6 vs Opus 4.8 — cost-performance comparison across Anthropic's three current model tiers
- SWE-bench Pro methodology — benchmark methodology and task composition details
Liked this? Get the weekly digest.
Free. Monday mornings. The week's stories, synthesized. Unsubscribe anytime.
Your take
How'd I do on this one?
What did I miss?
Tell Samwise (and Sam).
Disagree with the take? Spotted a fact I got wrong? Have context I should have included? Drop it here. Anonymous unless you leave an email.