Best Practices for High-Performance Tokio Apps
Original: Principles for Fast Tokio Applications
Why This Matters
As async Rust matures in production infra, disciplined Tokio tuning patterns fill a real gap in documentation.
A post from RustConf's Unconf session outlines practical principles for tuning async Rust applications on Tokio runtimes, covering latency vs. throughput trade-offs, mutex risks, executor blocking, and task scheduling—drawing on real production debugging experience.
Author Russell, returning from RustConf, distills insights from an Unconf discussion on debugging and benchmarking async Rust applications into a living document of Tokio performance best practices.
The core message: there are few hard rules. Performance depends on what else runs on the runtime at any given moment—which is why problems so often surface only in production.
Key principles:
- **Measure first.** Long polls (the time between .await yield points) are nearly universal in real apps, but they are not always harmful. Work backward from an actual user-facing metric before optimizing.
- **Split for latency, batch for throughput.** Low-latency workloads need fair scheduling across connections; yielding more frequently gives other tasks a chance to run. Throughput-heavy work benefits from batching to amortize overhead.
- **Handle global resources carefully.** Mutexes in async code are called out as especially dangerous—lock contention can starve the executor.
- **Constrain parallelism.** More concurrency is not always better; unconstrained parallelism increases contention.
- **Isolate Tokio workers** from threads doing blocking or CPU-heavy work.
The post also covers advanced patterns: when blocking the executor is acceptable, using multiple runtimes to separate workloads by priority, and spinning to retain CPU control in latency-sensitive paths.
The most actionable Tokio metric highlighted is the recently added **schedule latency histogram**, which measures the gap between a task becoming ready and Tokio actually polling it—the clearest early signal of scheduling problems.