Performance per dollar is getting faster and cheaper
Points and comments are a snapshot, not live.
AMD MI355X offers 2x cheaper inference than NVIDIA B200 with 80% performance.
Wafer benchmarked GLM5.2 on AMD MI355X vs NVIDIA B200. At 2.4 RPS with 20k in/1k out and 60% cache hits, MI355X achieved 2626 tok/s/node at 80% of B200's performance but over 2x cheaper. They quantized GLM5.2 to MXFP4 using AMD Quark with minimal accuracy loss. Speculative decode required two bug fixes in sglang's ROCm support. Prefill throughput improved by switching from TP8 to TP4×DP2 and tuning MoE kernel selection. Wafer claims the CUDA moat is eroding as AMD's software support improves.
What commenters are saying
Commenters split between optimism and skepticism about AMD's viability. Several noted AMD's higher power draw (MI355X at 1400W vs B200 at 1200W) complicates performance-per-watt comparisons. Critics pointed to accuracy degradation in MXFP4 quantization despite Wafer calling it lossless. A commenter highlighted Wafer's own failed product launch as concerning. Supporters noted Meta and OpenAI use AMD, though delivery dates slipped. One commenter emphasized that outside the US, supply constraints make AMD attractive despite software friction. Cooling and power infrastructure limits were flagged as often-ignored costs.