The gap between open weights LLMs and closed source LLMs
Points and comments are a snapshot, not live.
Open-weight LLMs could catch closed-source performance by December 2026 on one benchmark, but other metrics show a stable ~5-month gap.
The author analyzes the Artificial Analysis Intelligence Index, showing the gap between open-weight and closed-source LLM performance narrowed from mid-2024. A line of best fit predicts open weights will match closed source on this index by December 3, 2026. However, repeating the analysis across 18 benchmarks reveals a nearly flat average gap of under 5 months over the entire period. Most improvement comes from coding benchmarks; gaps in other areas have grown moderately. The exercise highlights the difficulty of measuring LLM quality, as different benchmarks yield opposing forecasts.
What commenters are saying
The top comment criticizes the article for conflating open weights with open source, but replies argue the distinction is practically irrelevant for most users. Another commenter likens the shrinking gap to Zeno's paradox, noting open models may catch up via distillation from closed models. A third warns that open-weight releases depend on corporate philanthropy (e.g., DeepSeek) and could stop. Rebuttals note open weights cannot be taken away, citing incentives for NVIDIA and Chinese labs to keep releasing models, plus the ability to improve weights via post-training and self-distillation.