HN Post Triggers AI Bot Traffic Flood
Original: I posted to HN, and then my website was flooded with AI bot traffic
Why This Matters
AI crawler volume is now large enough to destabilize small sites post-viral-post.
After posting his feed reader sprinklz.io to Hacker News and earning ~100 upvotes, developer Sammy Straus found his server overwhelmed — not by human visitors, but by AI crawlers hammering endpoints repeatedly.
Sammy Straus posted his customizable feed reader, sprinklz.io, to Hacker News roughly a month ago. It got around 100 upvotes. Then his server slowed to a crawl — and kept slowing days after the post's traffic should have faded.
Investigating his NGINX logs, Straus found that the majority of requests weren't even loading the JavaScript needed to render his single-page app. Nearly half of all traffic traced back to a single service that described itself as monitoring websites for changes to summarize them into an AI-generated newsletter — polling the site many times per minute. Blocked.
The rest were a mix of familiar crawlers: ClaudeBot, OpenAI's bot, Google's indexer, and assorted vulnerability scanners probing for WordPress paths that don't exist on his stack. Straus added a robots.txt to block AI scrapers while preserving search indexer access, implemented rate limiting, and used Claude itself to simulate scanner requests and check for information leaks.
The incident pushed him to overhaul site infrastructure for better performance under load — a silver lining from being buried in bot traffic.