Why Software Should No Longer Be Slow: AI Lowers the Bar for Performance Work
Original: There's no reason for software to be slow anymore
Why This Matters
AI-assisted optimization removes longstanding skill and cost barriers in systems software development.
Author Dan Luu argues that LLMs have dramatically reduced the cost and skill required for performance optimization, making formerly specialized work accessible to anyone who can write a few sentences — enabling custom, workload-specific software at scale.
Dan Luu's essay argues that AI coding tools have fundamentally changed the economics of software performance optimization. Historically, tasks like writing JIT compilers or heavily tuned regex engines required rare expertise and significant engineering resources, limiting such work to large-scale or high-revenue projects. LLMs have lowered that barrier substantially.
As evidence, Luu references FRE, a regex engine built by running an agent loop for a month against the rebar benchmark suite. While FRE initially overfit to the benchmark, it demonstrated strong performance on longer searches in native AOT-compiled form. Luu extended this experiment by having an agent perform code surgery — integrating background native compilation into ripgrep — in just minutes of human time. Results showed 2x–4x throughput gains on longer queries.
Marc Brooker noted that 'dynamic custom software, fitted to a particular workload rather than a class of workloads' is a likely outcome, comparing the trend to FFTW and demoscene tricks. Michael Malis added that JIT compilers — once too complex to justify building for most projects — are now within reach, citing pgrust as an example of more ambitious database software enabled by AI assistance.
Luu's broader point: performance work that once required specialized teams can now be done by almost anyone, meaning optimizations previously too costly for small projects are now economically viable across the board.