4B model beats Postgres query plans by 81% via RL
Original: Training a 4B model to produce 81% faster query plans than Postgres
Why This Matters
Demonstrates that small open-weights models can meaningfully outperform hand-tuned database planners using execution feedback alone.
Researcher Rohan Bansal post-trained a 4B Qwen model using supervised fine-tuning and agentic reinforcement learning to generate Postgres query hints, achieving up to 81% faster execution and a 44.7% average latency reduction across 113 join-heavy benchmark queries.
Bansal's experiment targets one of database engineering's oldest headaches: query optimizers that routinely pick suboptimal join orders. Leis et al. flagged this in 2015 and confirmed it was still broken in 2025. Join ordering is NP-hard, so cost-based planners like Postgres rely on cardinality estimates that are often badly wrong.
The approach sidesteps the optimizer entirely. A Qwen 4B model generates pg_hint_plan hints—specifying join methods, join order, and scan types—which are injected into Postgres before execution. The model learns which hints produce fast plans by running them and measuring actual execution time.
Training has two phases. First, SFT using a teacher model's rollouts bootstraps the 4B model, which initially failed to produce valid plans for 99 of 113 test queries. Second, a custom GRPO-based RL loop runs four rollouts per query, measures each against Postgres's default plan, assigns scalar rewards, and nudges weights toward faster strategies via LoRA.
Noise control was non-trivial. Bansal built a multi-container Postgres measurement rig specifically designed to suppress Linux page cache interference—without it, reward signals would have been unreliable. Inference ran on vLLM; the trainer ran on a separate machine.
On the Join Order Benchmark, the model achieves up to 81% latency reduction on individual queries and a 44.7% geometric mean speedup across the full 113-query workload. The result holds on the Cardinality Estimation Benchmark as well.