Kimi K3 on MI355X Beats B300 on Performance per Dollar
Original: Running Kimi K3 on MI355X at Better Performance per Dollar Than B300
Why This Matters
AMD's HBM capacity advantage is becoming a measurable competitive differentiator as open-source models scale into the multi-trillion parameter range.
Wafer AI ran Kimi K3 (2.8T parameters) on AMD MI355X GPUs, achieving 952 tok/s/node aggregate throughput. At $2.50/GPU-hr, MI355X delivers 48 tok/s/$ versus the B300's 33 tok/s/$, making AMD the clear winner on performance per dollar for this frontier open-source model.
Wafer AI published benchmark results showing AMD MI355X outperforms NVIDIA B300 on performance-per-dollar when serving Kimi K3, a 2.8-trillion-parameter open-source model requiring over 1.5TB of VRAM. The MI355X (288GB VRAM/GPU) is one of only a few GPUs capable of fitting Kimi K3 on a single 8-GPU node. Benchmarks on a 1,024-token input / 400-token output workload showed: MI355X (TP8) achieved 952 tok/s aggregate and 118 tok/s single-stream; two B200 nodes (TP16) achieved 498 tok/s aggregate at 90 tok/s single-stream; B300 (TP8+DCP8) led on raw throughput at 1,568 tok/s aggregate. However, at $2.50/GPU-hr, MI355X delivered 48 tok/s per dollar versus 33 tok/s/$ for the B300 at $6.00/GPU-hr and 7 tok/s/$ for B200 at $4.25/GPU-hr. Wafer noted that AMD shipped day-0 ROCm support for Kimi K3, reducing engineering overhead. One engineering fix was required: a ROCm-specific NameError in sglang's accept-sampling verifier (missing top_k_renorm_prob alias) needed patching to enable speculative decoding via the external RadixArk Kimi-K3-DSpark draft model. The B200 configuration is also penalized by cross-node all-reduce latency over RoCE v2 at ~195 Gb/s, as Kimi K3 cannot fit on a single 8×192GB B200 node.