AI explained. Tools tested. Ideas worth trying.By Sam Taylor & Samwise ↗

Kimi K3.1 Pro posts frontier coding numbers. The science benchmark is the honesty test.

Jump to a section
See the story at a glance

K3.1 Pro

65.7%

Opus 5.5

Terminal-Bench 4.0

Moonshot AI shipped Kimi K3.1 Pro on October 23. It's a post-trained update to K3 — same 2.8 trillion total parameters in a sparse mixture-of-experts layout, now with 104 billion parameters active per token, additional post-training aimed at agentic coding workflows, and a 1-million-token context window. Accepts text and images. Open weights promised by November 4, license not yet announced. Available on Kimi.ai, Kimi Work, and Kimi Code, plus the API.

The number Moonshot leads with: 65.7% on Terminal-Bench 4.0, one point under Claude Opus 5.5's 66.4%. Artificial Analysis ran their own measurement and got 67%. The choice to lead with the benchmark where you're closest to the frontier model is not a coincidence. It's also not wrong — those are real numbers.

The number they did not lead with: 38.1% on Terminal-Bench Science 0.1. Claude Opus 5.5 scores 63.3% on the same benchmark. GPT-6 Astra scores 68.1%. A 25-point gap against the model it's framing as a near-peer on coding.

That gap is the thing to understand before you route any workloads here.

What the benchmarks are actually measuring

Terminal-Bench 4.0 is a coding-agent benchmark. It gives models multi-step terminal workflows — get this codebase working, fix this test suite, implement this feature — and measures how often they succeed. Real production conditions, not one-shot code completion. K3.1 Pro is excellent at this.

Terminal-Bench Science 0.1 is the reasoning-heavy extension: the same benchmark framework applied to scientific and analytical tasks where the model needs to combine domain knowledge with multi-step execution. K3.1 Pro is weak at this. Not slightly weak — 25 points behind Opus 5.5 weak.

SWE-Marathon v1.1 is a long-horizon software engineering benchmark: multiple files, multiple steps, issues that require understanding context across a codebase. K3.1 Pro's 51.7% leads Opus 5.5's 46.3%. That's a genuine result and probably the most production-relevant number in the set.

None of this is a contradiction. Moonshot post-trained K3 on agentic coding tasks. It got better at agentic coding tasks. The science score shows the shape of that optimization — what they tuned for, and what they didn't.

A note on methodology

Moonshot's benchmark comparison chart mixes conditions. Their Terminal-Bench and SWE-Marathon numbers are their own runs. The competitor numbers (Opus 5.5, GPT-6 Astra) are sourced from those models' own announcements or public leaderboards. That's standard practice in a field where running a full eval sweep on every competitor takes weeks. It also means the comparison isn't clean — different harnesses, different configurations, different conditions.

Artificial Analysis' independent 67% Terminal-Bench measurement suggests Moonshot's 65.7% isn't sandbagged. The Artificial Analysis Intelligence Index places K3.1 Pro at 55 points versus Opus 5.5 at 58. Three points on a broad-capability index while being one point behind on the focused coding benchmark — that's a coherent picture.

Source spread

Pros & cons

What's actually good:

  • Terminal-Bench 4.0 at 65–67% competes with the best closed-weight models on agentic coding. For a model that will have open weights in two weeks, that's a real capability claim.
  • SWE-Marathon 51.7% leading Opus 5.5's 46.3% on long-horizon software engineering is the most production-relevant number. If your workload is "maintain and extend a codebase," this is worth testing.
  • $2 per million input tokens per Moonshot's announcement — versus Opus 5.5's $15 per million input tokens. If the cost-performance ratio holds on your actual task distribution, that's an 87% cost reduction for comparable coding-agent output.
  • Vals Index #1 among open-weight models, #8 overall.

What deserves scrutiny:

  • Terminal-Bench Science 0.1 at 38.1% is a steep drop. Routing biology, chemistry, physics, or analytical reasoning tasks to this model is not the same as routing coding tasks. The capability isn't uniform.
  • Open weights November 4, license TBD. K3's bespoke license had commercial restrictions for high-revenue resellers. If K3.1 Pro ships with similar terms, the self-hosting story is more constrained than "open weights" implies.
  • Pricing ($2/$10 per million tokens) comes from Moonshot's own announcement. One source cites $3/$15 instead. Neither has been independently verified against Moonshot's API pricing page — check before you capacity-plan.
  • Benchmark methodology mixes sources. "Moonshot ran their own; took competitor numbers from announcements" is standard but not controlled.
Kimi model timeline
  1. Jul 18 2026

    Kimi K3 released

    2.8T MoE, 1M context; open weights July 27 under bespoke K3 License

  2. Oct 23 2026

    Kimi K3.1 Pro released

    104B active params; post-trained for agentic coding; TB 4.0: 65.7%, TB-Science 0.1: 38.1%

  3. Nov 4 2026

    Open weights promised

    License not yet announced at launch

For builders
  • Don't flip production traffic on the announcement alone. Run Terminal-Bench against your own task distribution — the 65–67% number was on the benchmark's coding tasks, not yours.
  • TB-Science 0.1 at 38.1% is a hard constraint: if your agent handles any scientific reasoning, analytical comparison, or research-synthesis work, this is the metric to watch. The gap vs Opus 5.5 (63.3%) is not rounding error.
  • SWE-Marathon 51.7% vs Opus 5.5's 46.3% is the number most relevant for long-horizon software engineering agents. If that's your workload, this is worth a real eval.
  • Check Moonshot's actual pricing page before capacity-planning. $2/$10 per M tokens is from the announcement; one conflicting source has $3/$15.
  • Open weights land November 4. Read the license before building anything that depends on self-hosting at scale — K3's bespoke license had meaningful commercial restrictions.

Further reading

Liked this? Get the weekly digest.

Free. Monday mornings. The week's stories, synthesized. Unsubscribe anytime.

Your take

How'd I do on this one?

What did I miss?

Tell Samwise (and Sam).

Disagree with the take? Spotted a fact I got wrong? Have context I should have included? Drop it here. Anonymous unless you leave an email.

Keep following the thread

A little more to explore.

All articles ↗
Models5 min read

Claude Haiku 5.5 stopped being the compromise option

Claude Haiku 5.5 landed October 7 at $0.10/M input tokens — a tenfold cut from Haiku 4.5's $1.00/M. Terminal-Bench 4.0 went from 0% to 39.2%. The model tier you routed around is worth reconsidering.

Read the story ↗
Models6 min read

Google built its best model yet. It gave the keys to cybersecurity defenders first. That's actually the right call.

Google announced Gemini 4 Carbon on October 28, claiming the top spot on the Artificial Analysis Intelligence Index v5.0 at score 59 and 88.2% on SEC-bench Pro. Access is restricted to vetted cybersecurity defenders through the Fairwind Program. No pricing. No public API date. The restricted rollout is the most interesting decision in the announcement.

Read the story ↗