Kimi K3, and what we can still learn from the pelican benchmark
Points and comments are a snapshot, not live.
Kimi K3, a 2.8 trillion parameter model, is Moonshot AI's most capable and most expensive open-weights model yet.
Moonshot AI announced Kimi K3, a 2.8 trillion parameter model, calling it the first open 3T-class model. Self-reported benchmarks show it mostly beats Claude Opus 4.8 and GPT-5.5 but loses to Claude Fable 5 and GPT-5.6 Sol. Pricing is $3/M input and $15/M output tokens, making it the most expensive Chinese AI model. Simon Willison's pelican SVG test cost 25 cents, used 13,241 reasoning tokens, and revealed an 85-token hidden system prompt. The author notes the pelican benchmark is no longer a good model comparison metric but remains useful as a "hello world" test for cost, reasoning, and spatial awareness.
What commenters are saying
Commenters debated the value of the pelican benchmark. Some defended it as a useful taste test, while others called it a performative joke for attention. Several commenters argued the benchmark is likely in training data, undermining its novelty. Others pointed out that even if present, the quality of training examples is low, so improvement still indicates genuine capability. A few commenters shared alternative personal benchmarks, like video generation with Remotion, noting frontier models vary in creativity and taste.