Wafer serves GLM-5.2 on AMD MI355X at 2x lower cost than Blackwell
Original: Performance per dollar is getting faster and cheaper
Why This Matters
Demonstrates AMD GPUs can rival NVIDIA Blackwell inference performance at significantly lower cost, pressuring the AI inference market.
Wafer announced it achieved 2,626 tok/s/node aggregate throughput and 213 tok/s single stream for GLM-5.2 on AMD MI355X GPUs, in collaboration with Vercel AI Gateway and OpenRouter, at over 2x lower cost than NVIDIA Blackwell hardware.
Wafer, an AI inference startup, published a technical blog post on July 3, 2026, detailing how it served the GLM-5.2 model on AMD MI355X GPUs sourced from TensorWave at over 2x lower cost compared to NVIDIA B200/B300 (Blackwell) hardware. On a 20k input / 1k output, 60% cache hit rate workload, Wafer achieved 2,626 tok/s/node at 2.4 RPS with a TTFT knee of ≤5s — roughly 80% of B200 performance at less than half the price. Single-stream throughput reached 213 tok/s on 10k input / 1.5k output following Artificial Analysis standards.
For quantization, Wafer used AMD Quark to convert the base bf16 GLM-5.2 to MXFP4, which proved lossless compared to z-ai's official FP8 baseline across GSM8K, GPQA-Diamond, and tau2 benchmarks. The team selected sglang as the inference framework over vLLM (no working MXFP4 + GlmMoeDsa path) and ATOM (output degradation at long context). Additional engineering was required to enable speculative decoding on the ROCm sglang image, which does not support it out of the box. AMD MI355X GPUs are approximately 2.75x cheaper per unit than NVIDIA B300 on average, making them an attractive option as frontier model demand outpaces Blackwell GPU supply.