Anthropic drafts bill to block AI models over mass casualty risk
Original: Anthropic pitches AI safety bill that may block a model if 50+ people could be killed
Why This Matters
Anthropic's proposal could set the template for the first binding federal AI safety law.
Anthropic is circulating a draft AI safety bill in Congress that would allow courts to block model releases if they pose foreseeable risk of killing 50+ people or causing $1B+ in property damage.
Anthropic has been shopping a draft AI safety bill to federal lawmakers, Politico reports. The legislation would permit a D.C. judge to halt an AI model's release under specific catastrophic risk conditions: a foreseeable, imminent threat of killing more than 50 people, causing over $1 billion in property damage, or enabling a biological weapon, cyberattack, or a model evading its developer's controls.
Risk assessments would be conducted by independent evaluators registered with the U.S. Commerce Department. Those evaluators could be paid by the AI labs themselves — as long as payment isn't tied to evaluation outcomes. How that independence would be enforced in practice isn't addressed. When evaluators flag an imminent risk, they'd have 72 hours to notify Commerce, which would then petition a judge. Critically, no federal agency could independently block a model's release — only the courts could.
The 50-person and $1 billion thresholds mirror California's recently enacted Senate Bill 53, which used identical figures to define catastrophic risk. Anthropic confirmed the bill to Politico, with a spokesperson noting it reflects the company's Advanced AI Framework, which it has been pitching to state and federal lawmakers since June.