RTK cuts terminal output, but not necessarily your AI coding bill
Original: RTK reports token savings, but our cost benchmarks disagree
Why This Matters
Shows that token-count reduction tools don't reliably translate to lower costs at the agent level — the metric that matters.
Quesma ran 1,740 benchmark attempts costing over $1,500 to test RTK (Rust Token Killer), a tool with 79k GitHub stars that filters terminal output before AI agents read it. Results: Claude Code costs fell 5% total but rose 1% per task; DeepSeek costs rose 7–17% with RTK enabled.
RTK compresses shell output — stripping dates, owners, and verbose formatting — before an AI coding agent like Claude Code reads it. The project has 79k GitHub stars and a viral X post claimed up to 60% token savings with Claude Code. RTK's own README hedges: it cuts up to 90% of bash output, 'not the same as cutting your bill by 90%.' Quesma decided to actually measure the difference.
They ran Terminal-Bench 2.1 with two setups: Claude Code using Fable 5.0, and OpenCode with DeepSeek V4 Pro 0813. Each of 85–89 tasks ran five times with RTK and five without, totaling 1,740 attempts. For Claude/Fable, total spend fell 5% with RTK, but pass rate dropped 1%. On a per-task weighted basis — which controls for one expensive task dominating the numbers — costs were essentially flat (+1%). For DeepSeek, the picture was worse: total costs rose 7%, and the per-task average increased 17%, even among the 36 tasks where every single attempt passed.
Nearly all of Fable's headline savings traced back to one task ('winning-avg-corewars'), where RTK happened to cut turns in half. Strip that out and the rest of the benchmark showed under 1% savings. DeepSeek had the opposite outcome on that same task. The pattern suggests RTK's effect is highly task-dependent: it can help, do nothing, or actively hurt by triggering more turns or degraded output quality.