Four points. That is what is left of a lead that was supposed to be permanent.
On SWE-bench Verified — the benchmark that hands a model 500 real GitHub issues from production Python repositories and asks it to do what a junior developer does — the best open-weight model, Moonshot AI's Kimi K2.5, scored 76.8% in February 2026. The best proprietary model, Claude Opus 4.5, scored 80.9%. Two years earlier, the same comparison wasn't close: the open field sat more than twenty points behind the frontier — close enough to demo, too weak to deploy.
Nobody serious is calling the race finished — the gap hasn't closed, it has narrowed to a margin. But a four-point margin forces a harder question than a chasm did: what, exactly, is the premium buying?
The test that looks like a job
Most AI benchmarks don't measure engineering. HumanEval tests single-function Python completion, which is closer to a coding quiz than a job. LiveCodeBench measures competitive-programming puzzles. Both are useful instruments. Neither asks whether a model can survive contact with a real codebase.
SWE-bench Verified does. Its 500 problems are drawn from real issues in popular open-source repositories, each human-validated to confirm the task is well-posed and the tests genuinely verify the fix. The model has to read a bug report written by a human, explore an unfamiliar codebase, edit the right files — potentially several of them — and pass the existing test suite without breaking anything else. No multiple choice. No toy functions.
Independent analysts describe it as the closest available proxy for agentic software engineering, which is the thing companies actually want AI to do. It is the working reality behind multi-agent systems moving into production. Companies don't buy benchmark points. They buy merged pull requests. A benchmark built from real issues, real repositories and real test suites measures the distance between a model and a paycheck.
Walk through what a single problem demands. The model gets a bug report in loose, human language, and a repository it has never seen. It has to find the responsible code, understand what it was supposed to do, change it — sometimes in more than one file — and prove the change by passing tests written to catch exactly this class of failure, without breaking tests that passed before it arrived. A model that scores well here hasn't memorized answers; it has done the junior engineers' loop, hundreds of times in a row.
For two years, proprietary models owned it outright, and the ownership looked structural — a consequence of training scale and data that open projects supposedly couldn't match. 2026 is the year that assumption cracked. The crack doesn't make open models the frontier. It makes them contenders, and contenders change how buyers negotiate and how much of the premium survives scrutiny.
Four labs, four different bets
The February leaderboard had a shape worth reading slowly: behind Opus 4.5's 80.9% sat Claude Opus 4.6 at 80.8% and GPT-5.2 at 80.0%. Then the open field — Kimi K2.5 at 76.8%, the highest open-source score recorded, followed by GLM-4.7 at 73.8%, DeepSeek V3.2 at 73.1% and Qwen3-Coder-Next at 70.6%.
Four different organizations above 70% — a broad-based advance in open training, not a one-lab fluke.
Depth cuts the other way too. The April numbers show the open field stratifying: a small group of large models in the low-to-mid 70s, then a steep drop to Gemma 4 31B at 42.1%. "Open source" is not one capability level; it's a field with a strong head and a long, weak tail. The four-points story belongs to the biggest open models. It does not transfer to the small ones.
Kimi K2.5 holds the open record and, in independent comparisons, leads convincingly across knowledge, math and code benchmarks. Its weakness isn't capability: analysts note its GitHub presence — around 8,200 stars — trails DeepSeek's community.
DeepSeek plays the economics card, and plays it hard. V3.2 scored 73.1% on SWE-bench Verified. The newer DeepSeek-V4-Flash claims 78.6% — a vendor self-report, with independent re-measurement slightly lower — and prices API access from $0.22 per million input tokens off-peak, roughly two orders of magnitude cheaper than frontier proprietary APIs. Its weights are MIT-licensed, and its 74,000-plus GitHub stars are the group's deepest open-source community.
Two orders of magnitude means the same token budget buys roughly a hundred times the usage. For a team, that isn't a discount; it's a different operating model. Workloads rationed at frontier prices — agents across entire repositories, retries on hard problems, coding assistants live all day — become cheap enough to run by default. And because the weights are downloadable under an MIT license, the API price is a ceiling, not a dependency.
Qwen3-Coder-480B, from Alibaba Cloud, is the enterprise-friendly pick: Apache-2.0 license, mixture-of-experts design with 480 billion total parameters and 35 billion active, context window 256K extrapolatable to 1M, and 69.6% on SWE-bench Verified. GLM-4.7 and its successor GLM-5.2, also MIT-licensed, lead somewhere else entirely: long-horizon agentic runs, the multi-hour, whole-repository tasks where persistence matters more than single-shot accuracy.
That last specialty is easy to underrate. A benchmark run is a single task with a clean start; real engineering work is messier — migrations touching whole repositories, debugging sessions running for hours, consequences surfacing many steps later. A model that leads on long-horizon agentic runs leads on the work that most resembles an engineer's week rather than an engineer's exam. Each leader is optimized for a different buyer's real constraint — price, permission, persistence, or peak score.
Where "overtake" holds up
So has open source overtaken proprietary? In specific, dated matchups, yes — and the matchups aren't trivial. Qwen3-Coder's 69.6% beats Claude 3.7 Sonnet's 63.2% and Gemini 2.5 Pro's 61.8% on SWE-bench Verified. An open model from 2026 beating a proprietary flagship from a year earlier is a real milestone: the open ecosystem runs roughly one generation behind the frontier, not three — and one generation behind with downloadable weights is a very different strategic position from three generations behind with nothing to run.
By April 2026 the ordering held: Claude Opus 4.6 at 83.7%, Qwen3-Coder-Next 70.6%, DeepSeek V3.2 70.2%, Gemma 4 31B 42.1%.
What "one generation behind" buys is the strategic heart of the story: three generations behind with nothing to run means total dependence — the vendor sets price, terms and roadmap. One generation behind with downloadable weights means a merely somewhat weaker alternative, and a credible alternative changes every negotiation, even for teams that never deploy it. Failure modes change too: a proprietary API can be repriced, deprecated or withdrawn, while weights on your own infrastructure can't be taken back — the permanence that moved open models from experiment to default in cost-constrained shops.
But the live frontier still leads, and on the hardest tests the lead is wide: SWE-bench Pro, a tougher and unsaturated benchmark, shows GPT-5.4 at 59.1% against the best open results in the low 50s, and on Humanity's Last Exam, the most demanding general benchmark, the frontier leads by large margins. The accurate summary: open-weight coding models have closed to within single digits of the best proprietary models on the standard engineering benchmark, they win on cost by orders of magnitude, and they trail meaningfully on the hardest evaluations. For production code generation, that's the rational default for cost-constrained teams and a genuine alternative for everyone else.
Run the economics once. A 73%-capable open model can cost $0.22 per million tokens. A 63%-capable proprietary alternative can run to roughly $80. The open model is better and cheaper at the same time — and when that happens, the economics of AI-assisted development invert.
Cost is only the first consequence. Deployment is the second. Open weights run on your own infrastructure, which matters for codebases that cannot leave the building, for teams answering to AI governance obligations, and for serving strategies built on decentralized inference networks rather than a single cloud vendor. Control over where the model runs is control over where the code goes.
The third consequence is compounding. Every open release becomes training signal and scaffolding for the next one. The open field improves on a schedule set by everyone participating, not by a single lab's roadmap — each release hands the next team a better starting point than the last one had. That's the mechanism that compressed a twenty-point gap into four.
Read the decimals with suspicion
One caution applies to every figure above. Benchmark scores are scaffold-dependent: the same model can score several points higher or lower depending on the agent harness, tooling and prompting around it. Independent testers advise reading cross-scaffold comparisons as accurate within a few points, not exact — and most published scores are vendor self-reports on the vendors' own scaffolds.
That doesn't make the numbers meaningless. It sets the right resolution. A four-point gap between Kimi K2.5 and Claude Opus 4.5 is real signal; a one-point mid-tier difference is noise. And the trajectory is signal too: the remaining gap looks like a matter of scale and time rather than a fundamental barrier, so the strategic question has already changed. Not "is open source good enough?" anymore. "What are we paying the proprietary premium for, exactly?"
The Artificial Analysis index is the tiebreaker worth trusting. A composite of ten evaluations is harder to game than any single benchmark, and an index that re-measures models itself — rather than republishing vendor claims — removes the scaffold advantage. When its ordering matches the vendors' own ordering, the sensible conclusion isn't that every decimal is accurate. It's that the shape of the field is real: proprietary models ahead, open models within single digits on the engineering benchmark, open models ahead on price by a wide margin.
When a vendor trumpets a record score, ask whose scaffold produced it. The ordering is the signal. The decimals are the noise.
Benchmark parity raises a harder question than licensing: where does the model actually run? That is the ground covered by open-weight frontier models and the local-inference, quantization and sovereignty trade-offs facing enterprise deployers — today's benchmark table is a snapshot, not a settlement.
Sources
- Dev.to / Shaam AI — "Best Open-Source LLM for Coding in 2026: Qwen3-Coder vs GLM-5.2 vs DeepSeek V4 Flash" (69.6% Qwen3-Coder, licensing, pricing): dev.to
- GitHub / jcaose — "Benching local LLMs on Apple Silicon" (SWE-bench Verified cross-scaffold scores, DeepSeek-V4-Flash 78.6%, Artificial Analysis cross-check): github.com
- GitHub / greaveselliott — "Open Model Landscape April 2026" (SWE-bench Verified methodology, April 2026 scores, Opus 4.6 83.7%): github.com
- Dev.to / Van Massey — "Kimi K3 vs DeepSeek V4 Pro vs Qwen 3.8" (benchmark table, July 2026 leaderboard data): dev.to
- Medium / Likhit Kumar — "Which Local LLM is Better? A Deep Dive into Open-Source AI Models in 2026" (Feb 2026 leaderboard: Kimi K2.5 76.8%, Opus 4.5 80.9%): medium.com
