Anthropic has pitched a draft AI safety bill to US lawmakers that could allow the government to seek restrictions on advanced artificial intelligence models when they pose an imminent risk of catastrophic harm. The proposal sets out specific thresholds: a foreseeable and imminent risk of more than 50 people being killed or seriously injured, or more than $1 billion in property damage, under certain circumstances.

The proposed Anthropic AI safety bill focuses on a limited set of threats involving biological weapons, cyberattacks and AI models evading the safety controls established by their developers. Rather than giving government agencies unrestricted authority to stop AI releases, the reported framework would create a formal evaluation and court-review process.

According to TechCrunch’s report, which followed reporting by Politico, the draft would establish a role for independent evaluators and give the US Department of Commerce responsibility for seeking judicial intervention when qualifying risks are identified.

The proposal has not been enacted into law, and its final wording, legislative prospects and practical effects remain uncertain. Nevertheless, it provides a significant example of how an AI developer is proposing to translate its safety framework into federal legislation.

1. The Bill Sets a High Threshold for Government Intervention

One of the most notable features of the Anthropic AI safety bill is its proposed threshold for intervention. The reported draft would apply when a frontier AI model creates a foreseeable, imminent risk of more than 50 deaths or serious injuries, or more than $1 billion in property damage.

These thresholds are not intended to cover every AI failure, inaccurate response or instance of harmful output. The proposal focuses on specified catastrophic scenarios rather than establishing a general government power to block models whenever they produce undesirable results.

The reported risk categories include AI contributing to the creation or release of a biological weapon, enabling a cyberattack, or evading the controls put in place by its developer. The connection between the model and the potential harm would need to satisfy the proposal’s requirements.

This approach would distinguish catastrophic risks from more routine problems associated with AI products. For example, a chatbot producing an incorrect answer would not automatically meet the stated threshold. The question would be whether the model presents the defined imminent risk through one of the covered scenarios.

Supporters of a narrow threshold could argue that it offers companies a clearer understanding of when government intervention might occur. Critics may question whether authorities could identify a catastrophic threat early enough, particularly when the consequences of a model’s behaviour are difficult to predict.

The Anthropic AI safety bill therefore raises an important policy question: how should regulators balance a high evidentiary threshold against the need to act before a potentially severe incident occurs?

2. Independent Evaluators Would Assess Frontier AI Models

A central element of the Anthropic AI safety bill is the proposed use of independent evaluators to examine AI companies and their most capable models. Under the reported draft, evaluators would register with the US Department of Commerce and periodically assess frontier AI systems.

Independent testing could help identify dangerous capabilities or failures that a developer’s internal assessment might overlook. Evaluators could examine how models behave under challenging conditions, whether safety controls remain effective and whether specific risks require additional safeguards.

The draft reportedly allows AI companies to pay the evaluators, provided payment is not contingent on the outcome of an assessment. This arrangement is intended to separate compensation from the findings, although it also raises questions about how independence would be protected in practice.

A credible evaluation system would need transparent standards, appropriate technical expertise and safeguards against conflicts of interest. It would also need to establish how evaluators access sensitive information without unnecessarily exposing proprietary technology, confidential data or security vulnerabilities.

The effectiveness of the proposed framework would depend on more than the existence of external reviewers. Their authority, access, reporting obligations and ability to reach independent conclusions would all matter.

If adopted, the Anthropic AI safety bill could make external assessment a more formal part of the oversight process for the most capable models. However, the proposal’s actual effectiveness would depend on the final legislation and how the evaluation system was implemented.

3. The Commerce Department Would Take the Lead

The proposed Anthropic AI safety bill would establish a specific route for responding to a model assessed as presenting an imminent catastrophic risk.

According to reporting on the draft, an independent evaluator who identifies a qualifying risk would have 72 hours to notify the Commerce Department after completing the assessment. The department could then ask a federal judge in Washington, DC, to halt the model’s release or use.

This arrangement would place judicial review between a regulatory risk assessment and a potential restriction. The Commerce Department would not simply receive a report and automatically block the system under the reported process; it would need to pursue a court order.

The draft also reportedly restricts other federal departments, agencies and officials from independently suspending a model’s development, deployment or internal use. This would concentrate the proposed intervention process within one federal channel.

Such a structure could create a more consistent process for addressing certain frontier AI risks. However, it could also generate debate about whether one department should have such a central role in decisions that might affect cybersecurity, biological safety and other areas of public protection.

Questions would also remain about how quickly the process could operate during a rapidly developing threat. The proposal’s practical value would depend on the quality of the evidence, the speed of regulatory review and the court’s ability to assess complex technical claims.

The Anthropic AI safety bill would therefore establish not just risk thresholds, but a specific institutional process for deciding when a model’s release or use should be restricted.

4. The Proposal Builds on Anthropic’s Existing Safety Framework

Anthropic has described the draft as an extension of proposals outlined in its Advanced AI Framework. A company spokesperson told Politico that Anthropic had been advocating these proposals with federal and state lawmakers since June 2026.

The distinction between a voluntary framework and legislation is important. A company can publish its own safety commitments and establish internal testing procedures, but statutory requirements could create formal obligations and a defined process for regulatory intervention.

The Anthropic AI safety bill would reportedly combine evaluations, disclosure requirements and the possibility of blocking particularly dangerous deployments. This approach attempts to create a mechanism for dealing with specific catastrophic risks without giving the government unlimited discretion over the release of AI systems.

The company has also supported independent evaluation initiatives. In October, TechCrunch reported that Anthropic had granted researchers from the nonprofit METR employee-level access to assess its models, safety practices and risk management. METR was expected to publish its own assessments, with Anthropic unable to veto publication subject to narrowly defined redactions.

That separate arrangement does not mean the proposed bill has been adopted or that its requirements are already in force. It does, however, illustrate the broader discussion around how independent scrutiny could be incorporated into AI development.

Whether voluntary arrangements are sufficient or statutory requirements are necessary remains a subject of debate. Legislation could establish consistent obligations, while poorly designed rules could create compliance burdens or concentrate authority in ways that deserve scrutiny.

5. Questions Remain About Independence and Enforcement

Although the proposal sets out a formal process, several implementation questions remain. One concerns the independence of evaluators who may be paid by the companies whose systems they assess. Separating payment from the result of an evaluation is one safeguard, but the selection process, disclosure rules and protection against conflicts of interest would also matter.

Another question concerns the proposed threshold for intervention. A requirement to demonstrate an imminent risk of catastrophic harm may help limit unnecessary restrictions, but risks involving emerging technology can be uncertain. Regulators and courts would need to determine how much evidence is sufficient when the potential consequences are severe.

The proposed concentration of authority in the Commerce Department could also become a point of disagreement. Supporters might favour a consistent federal process, while critics could question whether other agencies should retain independent powers when risks overlap with their own responsibilities.

The Anthropic AI safety bill would also need clear procedures for protecting confidential information, handling disputed assessments and reviewing restrictions as new evidence becomes available. Without these safeguards, the process could face questions about fairness, transparency and accountability.

Finally, the proposal remains a draft that has been pitched to lawmakers. Its details could change during negotiations, and there is no guarantee Congress will introduce or pass it in its current form.

These unresolved issues will help determine whether the proposal becomes a workable regulatory framework or remains one company’s contribution to the wider debate over AI governance.

What the Proposal Could Mean for AI Developers and Businesses

For AI developers, the proposed framework highlights the possibility that increasingly capable models could face more formal external assessments before release or during deployment. Companies developing frontier systems may need to demonstrate how they test for catastrophic risks, document safeguards and respond to evidence of dangerous capabilities.

For businesses using AI products, the immediate practical lesson is to distinguish a proposed policy from an existing legal obligation. The draft does not itself impose new requirements on ordinary users or automatically block commercially available models.

Organisations should nevertheless pay attention to the broader direction of AI governance. Stronger evaluation practices, clearer incident reporting and better documentation can help companies understand the risks associated with advanced AI systems, particularly when they are connected to sensitive data or operational tools.

The Anthropic AI safety bill also illustrates the tension between safety and innovation. Rules that are too weak may fail to address serious risks, while rules that are overly broad or unclear could make it harder to develop and deploy useful technologies.

A durable framework would need to define its scope carefully, establish credible evaluation standards and provide a transparent route for decisions. It would also need to account for changing capabilities as AI models and the tools connected to them become more sophisticated.

Final Thoughts

Anthropic’s proposed legislation would create a narrow route for restricting advanced AI models when independent assessments identify an imminent risk of catastrophic harm. Its reported thresholds of more than 50 deaths or serious injuries, or more than $1 billion in property damage, distinguish the proposal from broad restrictions aimed at routine AI failures.

The Anthropic AI safety bill would also establish independent evaluation requirements and give the US Commerce Department a central role in seeking court-ordered intervention. Its proposed safeguards raise important questions about evaluator independence, government authority and how quickly regulators can respond to emerging threats.

For now, the proposal remains a draft rather than enacted law. Its eventual significance will depend on whether lawmakers take it forward, how its provisions change and whether the resulting system can identify serious risks without unnecessarily restricting beneficial AI development.

The wider debate is no longer simply about whether advanced AI should be tested for safety. It is increasingly about who should conduct those tests, what evidence should trigger intervention and how public accountability can be maintained as the technology advances.

Sources: