Cerebras Deploys Trillion-Parameter AI Models

235 points · 75 comments on HN · read original →

Points and comments are a snapshot, not live.

Cerebras deploys 3-trillion-parameter models across 59 wafer-scale chips in a pipeline.

Cerebras explains how it runs trillion-parameter models like Kimi K3 (3T params, 69 linear-attention layers, 24 dense layers) on its WSE-3T wafers (44 GB on-chip SRAM each). Using 4-bit weight quantization and 8-bit KV cache, each layer requires ~16 GB. Dense layers need KV cache space, so one fits per wafer; linear layers don't, so two fit per wafer. This yields 59 wafers total. The chips are connected in a pipeline: activations flow between wafers via RDMA over Ethernet (~2 µs per hop). Multiple requests stream concurrently through pipeline stages, achieving 1000-2000+ tokens/second. For a 256k request, each dense-layer wafer supports ~219 concurrent requests.

The approach scales to larger models by adding more wafers, keeping weights stationary and moving only activations.

What commenters are saying

Commenters question reliability: a 59-wafer pipeline creates a failure domain where one bad wafer takes the whole system down, unlike GPU deployments that degrade gracefully. Power consumption is estimated at ~1.18 MW (20 kW per wafer), yielding ~4.1 J/token. Some debate whether GPUs could also use pipeline parallelism, but others note Cerebras's spatial architecture makes it natural. A few commenters argue that extreme token speeds (1000-2000 t/s) are overkill for individual developers and that human review remains the bottleneck for quality work. Others counter that fast inference benefits shared organizational use, like an "instantaneous compiler." Technical discussion covers why chips use square tilings (EDA tools don't support diagonal wires) and how data flows through the 2D PE mesh.