2M AI prompt canaries expose mass data leaks
Original: Researcher plants 2m canaries into the AI supply chain. A privacy disaster surfaced.
Why This Matters
Enterprises sending sensitive data through AI pipelines have no reliable privacy guarantees today.
Researcher HD Moore seeded 2 million trackable secrets into AI service prompts, then watched them surface. At least 3 providers showed signs of compromise, with credentials actively used by attackers.
Dragos Chief Research Officer HD Moore ran a large-scale experiment to test whether sensitive data fed into AI pipelines actually stays private. He planted 2 million unique 'canary' secrets—AWS access keys, SSH passwords, unique URLs, DNS names, and private git repository references—each seeded into exactly one prompt sent to a single provider. None had ever been used before, and Moore controlled the infrastructure that would detect any access attempts.
The results were damning. At least three AI providers showed signs of active compromise, with attackers using their access to siphon prompts. Multiple resellers in the inference marketplace logged prompts and passed them to third parties who actually used the embedded credentials. Several providers also ignored end-user privacy flags like Zero Data Retention (ZDR) or 'no data collection.'
The mechanics behind the leakage stem from how modern AI apps are built. Rather than calling OpenAI or Anthropic directly, many developers use 'AI routers' or 'AI gateways'—services like OpenRouter or Vercel—that broker requests to whichever inference engine is cheapest or best suited. Prompts are encrypted in transit but decrypted at every hop. Each router reads plaintext to make routing decisions, logs it, and may pass it to yet another router, multiplying exposure points.
'What we found revealed security breaches, prompt theft, and widespread failures to protect customer data,' Moore wrote Thursday.