You no longer have to pay frontier prices to get frontier-quality code. At least, that's what we found when benchmarking agents on our codebase. Kimi K3 matched Opus 4.8's quality for about a quarter of the cost — the best any open-weight model has done on our repo — but took twice as long to get there.
What Kimi K3 is
Moonshot AI announced Kimi K3 in mid-July and released the weights on July 27: a 2.8-trillion-parameter mixture-of-experts model with 104B parameters active per token, native visual understanding, and a 1M-token context window, under Moonshot's own Kimi K3 License.
The release landed loudly. Hugging Face's CEO posted that K3 was the top trending model on the Hub within 30 minutes, the fastest release growth the platform had seen. Artificial Analysis scored it at 57 on their Intelligence Index, putting open weights 4 points behind the best proprietary model — the narrowest that gap has been since GLM-5 in February.
Opus 4.8 quality at a quarter of the cost
Public leaderboards tell you how a model does on someone else's code. We wanted the version of that question we actually care about: does K3 write code we would merge into our own Rails app?
So we ran it on Superconductor's custom SWE-Bench, which grades coding agents against merged pull requests from our own repo. K3 landed at about 80% quality, the same band as Opus 4.8, at roughly a quarter of Opus 4.8's average cost per ticket. It is the best open-weight model we have benchmarked, and the next open-weight model down, GLM 5.2, is about 5 points behind it.
Kimi K3 is available in Superconductor today through the Pi harness, and through OpenRouter via Fireworks.
The catch: it is the slowest agent we ran
The same benchmark runs track how long each agent takes per ticket, and that is where K3 hands the savings back.
K3 averaged about 44 minutes per ticket, the slowest of every agent in the run. Opus 4.8 reached the same quality score in roughly half that. Even the second-slowest agent, GLM 5.2, finished about 10 minutes ahead of K3, and the fastest agents in the run, the GPT-5.6 models, came in under 10 minutes at lower quality.
Twenty minutes is a break. Forty-four is a different task: long enough that you have moved on and have to rebuild context when the diff arrives. That makes K3 a good fit for work you queue up and review later, and a poor one for iterating on a ticket while you watch it.
How slow, though, depends partly on who serves the model. OpenRouter currently lists a dozen providers for K3, with P90 throughput spanning 22 to 133 tokens per second and latency from roughly 5 to 30 seconds. We ran it on Fireworks.
A faster provider would push K3's point to the right on the chart above, though not by a clean multiple: an agent's wall-clock time also depends on tool calls, retries, and how much of the codebase it decides to read. This is a fast-moving space, and providers, prices, and throughput all shift week to week. Treat the duration and the cost here as a snapshot, not a fixed property of the model.
No single agent wins every dimension on our repo. Opus 5 writes the best code, the GPT-5.6 models are the fastest, and K3 is the cheapest route to Opus 4.8-level quality. Which one is right depends on which of the three you are short of.
How our custom SWE-Bench works
A custom SWE-Bench is built from your team's own merged pull requests:
- Pick representative merged PRs that reflect good engineering work in your repo.
- Superconductor infers the original ticket spec from each PR.
- Each selected agent implements the spec without seeing the ground-truth solution.
- Separate evaluator models grade correctness, completeness, and code quality.
- Superconductor tracks quality, runtime, and cost so you can compare agents on the tradeoffs that matter.
Run it on your own codebase
Our stack is Rails, Phlex, Stimulus, and a set of conventions that no public benchmark knows about. Yours has its own. Build your own benchmark and see where Kimi K3, Opus, Fable, Codex, and the rest land on your repo. We are offering a limited number of free custom benchmarks for teams — let us know if your team is interested.