Anthropic is pitching US lawmakers a draft AI safety bill that would allow the government to block an advanced AI model only under narrowly defined, catastrophic circumstances. According to reporting by Politico, via Reuters and TechCrunch, the proposal sets a threshold of a foreseeable, imminent risk of more than $1 billion in damage or the death or serious injury of more than 50 people.
The draft is not law. It is a proposal Anthropic has privately presented to lawmakers, and its details have been reported from a draft obtained by Politico. The Anthropic AI safety bill reflects the company’s effort to translate its existing safety framework into federal legislation—but it also raises questions about how much power regulators should have to intervene before a dangerous system is released. At issue is not merely whether the Anthropic AI safety bill recognises severe risks, but whether its legal trigger is broad enough to address them.
What the Anthropic AI Safety Bill Would Allow
Under the reported draft, authorities could seek to stop a model’s release or use only when it presents a foreseeable and imminent risk exceeding the specified human-harm or financial-damage threshold. The qualifying danger would have to arise from one of three categories: creating or releasing a biological weapon, a model evading its developer’s controls, or a cyberattack.
That means the proposal is not a general mechanism for blocking AI products over every safety concern, inaccurate output or harmful use. The Anthropic AI safety bill focuses on a limited set of severe threats, with a high bar for government action. In effect, the Anthropic AI safety bill would reserve model suspension for a narrow emergency category.
The draft reportedly assigns the intervention process to the US Commerce Department. It could ask a federal judge in the District of Columbia to suspend a model, while other federal agencies would be barred from independently ordering a halt. This would place the decision under a defined legal process rather than leaving each agency with its own authority to stop a model.
Independent Evaluators Would Monitor Frontier AI
The draft would also establish a system of independent evaluators registered with the Commerce Department. They would periodically assess AI companies and their frontier models, looking for risks that could meet the bill’s intervention criteria.
According to Reuters’ account of Politico’s reporting, companies could pay evaluators, provided payment was not tied to the outcome of an assessment. That condition is intended to separate compensation from whether a model passes or fails, but it leaves a practical question: what safeguards would ensure evaluators remain genuinely independent when the companies being reviewed fund their work?
The proposal reportedly gives evaluators 72 hours to notify Commerce after completing an assessment that identifies an imminent, specified catastrophic risk. The department would then need to seek judicial action rather than unilaterally suspend a model.
For Anthropic, the bill’s oversight structure is a key part of its safety proposal. For critics, the design raises questions about whether evaluator registration, reporting duties and court review would provide sufficient transparency and independence. The credibility of the Anthropic AI safety bill may therefore depend on how evaluator independence works in practice.
Why the 50-Person and $1 Billion Thresholds Matter
The thresholds are intended to distinguish catastrophic risks from lower-level problems. Yet they also invite debate about what happens when a model creates serious harm that falls short of the specified scale, or when risks are difficult to quantify before deployment.
The Anthropic AI safety bill’s supporters may argue that clear triggers make government intervention more predictable and prevent regulators from halting systems based on vague or speculative concerns. A defined threshold could also provide companies with a more consistent standard for assessing when extraordinary restrictions might apply.
Opponents may see the same threshold as too restrictive. A danger involving fewer than 50 people could still be grave, and some harms—such as privacy violations, discriminatory outcomes, manipulation or disruption to essential services—may not fit neatly into the three listed categories or a billion-dollar damage estimate.
The policy challenge is therefore not simply whether catastrophic risks should be regulated. It is how to combine clear emergency powers with broader mechanisms for addressing harms that are real but do not meet an exceptionally high intervention standard. This is one of the central questions raised by the Anthropic AI safety bill.
How the Proposal Fits Into the Wider AI Policy Debate
The proposal arrives amid a broader US debate over whether AI regulation should be set nationally or through a patchwork of state laws. Reports say the draft would pre-empt certain state requirements related to AI risk disclosure, third-party evaluation and incident reporting. That could make compliance more uniform for developers operating across the country, but it could also limit states’ ability to impose stricter safeguards.
The Anthropic AI safety bill has similarities to elements of California’s SB 53, which addresses frontier AI safety and catastrophic-risk reporting. The overlap suggests that lawmakers and technology companies are negotiating not only what risks deserve attention, but also which institutions should define and enforce the rules.
Anthropic has said the draft reflects its Advanced AI Framework, which it has been advocating with federal and state lawmakers. Still, the legislation remains a company-backed proposal, not a settled national consensus. Other policymakers, researchers and public-interest groups may favour different thresholds, oversight models or protections. The Anthropic AI safety bill will have to contend with those competing approaches if it moves into formal debate.
Questions About Industry-Shaped Regulation
When a major AI developer proposes rules for its own industry, the proposal deserves scrutiny on both its safety provisions and its institutional design. The Anthropic AI safety bill could create formal oversight, independent evaluations and a court-based process for catastrophic threats. At the same time, the company’s role in drafting the framework raises questions about whose interests the rules prioritise.
Key details will matter: how evaluators are selected and monitored, what information they can access, how findings are disclosed, and whether regulators can respond to risks outside the narrow emergency criteria. The draft’s reported restrictions on other agencies’ authority are also likely to be debated.
These questions do not establish that the proposal is ineffective. They show why a draft law should be assessed in detail, with input from technical experts, civil society, lawmakers and communities potentially affected by AI systems. That scrutiny will shape whether the Anthropic AI safety bill becomes a workable public safeguard or remains a framework chiefly designed by industry.
What Happens Next?
The proposal’s next stage depends on whether lawmakers choose to consider it, revise it or develop competing approaches. Because the draft has been privately pitched and is not enacted legislation, its provisions could change substantially during any formal legislative process.
For now, the Anthropic AI safety bill offers a view of how one leading AI company wants catastrophic risks to be regulated: narrowly defined triggers, registered evaluators, a central role for Commerce and judicial review before a model can be halted. The next debate will test whether the Anthropic AI safety bill’s proposed safeguards match the scale of the risks lawmakers want to manage.
The larger debate is whether those mechanisms would provide meaningful protection while preserving predictable rules for AI development. The answer will depend on the final legal language, the independence and authority of evaluators, and whether lawmakers address serious harms that fall outside the draft’s stated thresholds.
Final Thoughts
Anthropic’s proposal puts a difficult question at the centre of AI policy: how should governments intervene when an advanced model could cause catastrophic harm? Its draft would set a high bar—more than 50 potential deaths or serious injuries, or over $1 billion in damage, under specified imminent-risk conditions.
The Anthropic AI safety bill could bring more structure to frontier AI oversight, but its narrow triggers and industry-shaped design are likely to draw debate. It remains a proposal, and its eventual impact will depend on whether lawmakers adopt, amend or reject its approach.
Sources
More AI coverage: explore Progressive Robot's AI Models, Tools & Releases hub — hands-on reviews, setup guides and benchmarks in one place.
