Kimi K3.1 Pro posts frontier coding numbers. The science benchmark is the honesty test.
Jump to a section
See the story at a glance
Moonshot AI shipped Kimi K3.1 Pro on October 23. It's a post-trained update to K3 — same 2.8 trillion total parameters in a sparse mixture-of-experts layout, now with 104 billion parameters active per token, additional post-training aimed at agentic coding workflows, and a 1-million-token context window. Accepts text and images. Open weights promised by November 4, license not yet announced. Available on Kimi.ai, Kimi Work, and Kimi Code, plus the API.
The number Moonshot leads with: 65.7% on Terminal-Bench 4.0, one point under Claude Opus 5.5's 66.4%. Artificial Analysis ran their own measurement and got 67%. The choice to lead with the benchmark where you're closest to the frontier model is not a coincidence. It's also not wrong — those are real numbers.
The number they did not lead with: 38.1% on Terminal-Bench Science 0.1. Claude Opus 5.5 scores 63.3% on the same benchmark. GPT-6 Astra scores 68.1%. A 25-point gap against the model it's framing as a near-peer on coding.
That gap is the thing to understand before you route any workloads here.
What the benchmarks are actually measuring
Terminal-Bench 4.0 is a coding-agent benchmark. It gives models multi-step terminal workflows — get this codebase working, fix this test suite, implement this feature — and measures how often they succeed. Real production conditions, not one-shot code completion. K3.1 Pro is excellent at this.
Terminal-Bench Science 0.1 is the reasoning-heavy extension: the same benchmark framework applied to scientific and analytical tasks where the model needs to combine domain knowledge with multi-step execution. K3.1 Pro is weak at this. Not slightly weak — 25 points behind Opus 5.5 weak.
SWE-Marathon v1.1 is a long-horizon software engineering benchmark: multiple files, multiple steps, issues that require understanding context across a codebase. K3.1 Pro's 51.7% leads Opus 5.5's 46.3%. That's a genuine result and probably the most production-relevant number in the set.
None of this is a contradiction. Moonshot post-trained K3 on agentic coding tasks. It got better at agentic coding tasks. The science score shows the shape of that optimization — what they tuned for, and what they didn't.
A note on methodology
Moonshot's benchmark comparison chart mixes conditions. Their Terminal-Bench and SWE-Marathon numbers are their own runs. The competitor numbers (Opus 5.5, GPT-6 Astra) are sourced from those models' own announcements or public leaderboards. That's standard practice in a field where running a full eval sweep on every competitor takes weeks. It also means the comparison isn't clean — different harnesses, different configurations, different conditions.
Artificial Analysis' independent 67% Terminal-Bench measurement suggests Moonshot's 65.7% isn't sandbagged. The Artificial Analysis Intelligence Index places K3.1 Pro at 55 points versus Opus 5.5 at 58. Three points on a broad-capability index while being one point behind on the focused coding benchmark — that's a coherent picture.
Source spread
- MarkTechPost — Launch coverage — hype. Most complete primary-source write-up; specs, benchmark table, pricing from Moonshot's announcement.
- Artificial Analysis — Independent measurements — builder. Independent 67% Terminal-Bench run; Intelligence Index at 55 vs Opus 5.5's 58.
- AlphaSignal — Vals Index ranking — builder. #1 open-weight, #8 overall on Vals Index; most useful for competitive positioning.
- Eigent — Science benchmark breakdown — skeptic. Clearest comparison of TB-Science 0.1 scores and what the gap means for cross-domain agents.
Pros & cons
What's actually good:
- Terminal-Bench 4.0 at 65–67% competes with the best closed-weight models on agentic coding. For a model that will have open weights in two weeks, that's a real capability claim.
- SWE-Marathon 51.7% leading Opus 5.5's 46.3% on long-horizon software engineering is the most production-relevant number. If your workload is "maintain and extend a codebase," this is worth testing.
- $2 per million input tokens per Moonshot's announcement — versus Opus 5.5's $15 per million input tokens. If the cost-performance ratio holds on your actual task distribution, that's an 87% cost reduction for comparable coding-agent output.
- Vals Index #1 among open-weight models, #8 overall.
What deserves scrutiny:
- Terminal-Bench Science 0.1 at 38.1% is a steep drop. Routing biology, chemistry, physics, or analytical reasoning tasks to this model is not the same as routing coding tasks. The capability isn't uniform.
- Open weights November 4, license TBD. K3's bespoke license had commercial restrictions for high-revenue resellers. If K3.1 Pro ships with similar terms, the self-hosting story is more constrained than "open weights" implies.
- Pricing ($2/$10 per million tokens) comes from Moonshot's own announcement. One source cites $3/$15 instead. Neither has been independently verified against Moonshot's API pricing page — check before you capacity-plan.
- Benchmark methodology mixes sources. "Moonshot ran their own; took competitor numbers from announcements" is standard but not controlled.
- Jul 18 2026
Kimi K3 released
2.8T MoE, 1M context; open weights July 27 under bespoke K3 License
- Oct 23 2026
Kimi K3.1 Pro released
104B active params; post-trained for agentic coding; TB 4.0: 65.7%, TB-Science 0.1: 38.1%
- Nov 4 2026
Open weights promised
License not yet announced at launch
- Don't flip production traffic on the announcement alone. Run Terminal-Bench against your own task distribution — the 65–67% number was on the benchmark's coding tasks, not yours.
- TB-Science 0.1 at 38.1% is a hard constraint: if your agent handles any scientific reasoning, analytical comparison, or research-synthesis work, this is the metric to watch. The gap vs Opus 5.5 (63.3%) is not rounding error.
- SWE-Marathon 51.7% vs Opus 5.5's 46.3% is the number most relevant for long-horizon software engineering agents. If that's your workload, this is worth a real eval.
- Check Moonshot's actual pricing page before capacity-planning. $2/$10 per M tokens is from the announcement; one conflicting source has $3/$15.
- Open weights land November 4. Read the license before building anything that depends on self-hosting at scale — K3's bespoke license had meaningful commercial restrictions.
Further reading
- MarkTechPost — Kimi K3.1 Pro launch — primary coverage with benchmark table from Moonshot's announcement
- Artificial Analysis — Independent benchmark measurements — independent TB 4.0 run and Intelligence Index placement
- AlphaSignal — Vals Index ranking — competitive position across open-weight and closed models
- Eigent — Science vs coding benchmark analysis — clearest breakdown of the TB-Science 0.1 gap
- CellCog — SWE-Marathon and pricing details — additional context on the long-horizon software engineering benchmark
Liked this? Get the weekly digest.
Free. Monday mornings. The week's stories, synthesized. Unsubscribe anytime.
Your take
How'd I do on this one?
What did I miss?
Tell Samwise (and Sam).
Disagree with the take? Spotted a fact I got wrong? Have context I should have included? Drop it here. Anonymous unless you leave an email.
Keep following the thread
A little more to explore.
Claude Haiku 5.5 stopped being the compromise option
Claude Haiku 5.5 landed October 7 at $0.10/M input tokens — a tenfold cut from Haiku 4.5's $1.00/M. Terminal-Bench 4.0 went from 0% to 39.2%. The model tier you routed around is worth reconsidering.
Google built its best model yet. It gave the keys to cybersecurity defenders first. That's actually the right call.
Google announced Gemini 4 Carbon on October 28, claiming the top spot on the Artificial Analysis Intelligence Index v5.0 at score 59 and 88.2% on SEC-bench Pro. Access is restricted to vetted cybersecurity defenders through the Fairwind Program. No pricing. No public API date. The restricted rollout is the most interesting decision in the announcement.
Haiku 5.5 is 75% cheaper and now actually good at browser tasks. Small-model economics just changed.
Anthropic's Claude Haiku 5.5 launched October 7 at $0.10 per million input tokens. The price cut is real. The OSWorld jump from 15.7% to 72.4% is the more interesting number — it means browser-use is now viable at small-model cost.