DeepSeek V4 Flash runs on single AMD MI300X GPU
Original: DeepSeek V4 Flash on a Single AMD MI300X
Why This Matters
Demonstrates that large frontier MoE models can run on AMD hardware without quantization, expanding deployment options beyond NVIDIA GPUs.
Developer ryanzhou published a GitHub repo showing DeepSeek-V4-Flash-0731 running on a single AMD MI300X GPU in production, achieving 168.6 tok/s single-stream decode and 256K context length validated, without weight quantization or offloading.
Developer ryanzhou has released a public GitHub repository detailing how to run DeepSeek-AI's DeepSeek-V4-Flash-0731 model on a single AMD MI300X GPU in a production environment. The setup uses a Docker Compose stack with SHA-256-pinned file overlays, built on vLLM ROCm nightly (0.26.1rc1.dev229) and AITER 0.1.19. Performance benchmarks from the pinned stack show: single-stream decode at 168.6 tok/s (median), prefill throughput of approximately 7.9–8.5K tok/s, 8 concurrent streams at 542 tok/s aggregate (90.3 tok/s per stream median), and a 64-stream burst at 830 tok/s aggregate with no out-of-memory errors. The model's weights occupy 156.67 GiB of HBM with no additional quantization or offloading. Context length is validated to 256K tokens, though the architecture supports up to 1M. The official vLLM recipe targets NVIDIA and newer AMD hardware; this repository includes custom patches addressing FP8 format compatibility, MoE routing at high concurrency, causal speculative verification, CPU-KV synchronization, and untuned kernel shapes specific to the MI300X.