Anthropic's draft federal AI safety bill would hand the government the power to block a frontier model from release — but only after a judge signs off, a procedural twist that has already forced the company to walk back the language that states would have no role at all in shutdown decisions.
The draft, reviewed by The Washington Post, proposes a federal framework in which independent evaluators registered with the Commerce Department assess the riskiest frontier AI systems. If a model presents a foreseeable and imminent risk capable of killing 50 or more people, or causing roughly $1 billion in damage, authorities could move to block its release. The mechanism that drew the most scrutiny is procedural rather than substantive: intervention would require judicial approval, a built-in hurdle designed to shield companies from arbitrary executive action, but one that also means a shutdown cannot be fast enough to keep pace with a rapidly escalating incident.
Critics in both directions have piled on. Industry groups argue the bill would effectively freeze the frontier, since the evidentiary standard — a specific, imminent, quantified threat — is almost impossible to meet before deployment, not after. Safety researchers, conversely, have complained that the judicial gate and the retroactive posture would leave states with no immediate tool when a model misbehaves, which is why Anthropic has signaled it will revise the preemption language to preserve a state-level role in shutdown decisions rather than ceding the entire question to a single federal statute.
The stakes extend well past the lab. JPMorgan's Jamie Dimon has called advanced-model risk "a real issue," and Wall Street is watching to see whether this bill becomes the template for how the U.S. governs the next generation of frontier systems. For Anthropic, which is preparing a mid-November IPO at a valuation that dwarfs most peers, the bill is also a marketing instrument: it signals to investors and regulators that the company is building its compliance architecture before the government forces it to. The draft is unpolished and contested, but it is the first time a major AI lab has sketched a federal statute this granular, and the details of where it lands will shape who gets to say no, and how quickly, to the next model.
Image source: i.ibb.co