AI agents tested across JS frameworks: results vary
Original: The AI cost of JavaScript frameworks
Why This Matters
First structured benchmark comparing AI agent output quality across major JS frameworks.
Developer Nolan Lawson benchmarked AI agents building a Pokémon app across major JS frameworks—Next.js, Nuxt, SvelteKit—measuring tokens spent, dev time, lines of code, and Lighthouse scores to cut through framework hype.
Tired of evidence-free claims about which JavaScript frameworks work best for AI agents, developer Nolan Lawson built a benchmark to find out. The test instructs Claude's Opus 5.5 (high effort mode) to build a Pokémon data browser app using the PokéAPI, once per framework/meta-framework—Next.js, Nuxt, SvelteKit, and others. Each agent ran in an isolated sandbox, used the official MCP or skills for its target framework, and had no knowledge it would be evaluated on Lighthouse performance and accessibility metrics. Results were scored on development time, tokens consumed, lines of code generated, and Lighthouse scores.
Lawson is explicit about limitations: the app is one holotype—content-heavy, low interactivity, closer to Wikipedia than Gmail. He also avoided mandating SSR vs CSR architecture, leaving those choices to the agent. CSS frameworks like Tailwind were excluded to keep focus on the JS framework differences.
The motivation is pointed. Lawson argues that "vibes-based" decision-making has long plagued frontend development, and the agentic era hasn't fixed that. Teams are rewriting frontends because a framework is "better for agents"—often without data. Lawson's post, still publishing its full findings, promises a conformance test suite to prevent agents from gaming the spec.