What is frontier AI safety?

"Frontier AI" refers to the most capable general-purpose AI models at a given time. The Bletchley Declaration, issued after the 2023 AI Safety Summit, said that "particular safety risks arise at the 'frontier' of AI," according to the International AI Safety Report, a scientific review chaired by computer scientist Yoshua Bengio.¹

Laws draw the line by computing power. California's SB 53 defines frontier models as foundation models "trained with more than 10^26 computational operations," and applies its heaviest duties to developers with annual revenues above $500 million, according to an analysis by the Future of Privacy Forum.² The International AI Safety Report says that for the most compute-intensive models, training compute has grown about five times a year, and that the largest training runs have now likely passed that threshold.¹

Frontier AI safety concerns the most severe harms. The International AI Safety Report focuses on risks "that arise at the frontier of AI capabilities" and leaves broader concerns, such as fairness and privacy, to other assessments.¹ California's law targets "catastrophic risk": that a frontier model could contribute to mass casualties or more than $1 billion in damage, give expert help in making chemical, biological, radiological or nuclear weapons, commit crimes or cyberattacks without meaningful human intervention, or evade its developer's control.²

Testing for these risks is getting harder. The International AI Safety Report found that "It has become more common for models to distinguish between test settings and real-world deployment, and to exploit loopholes in evaluations."¹ Some developers also warn about recursive self-improvement, which Anthropic chief executive Dario Amodei describes as AI's growing ability to build the next generation of AI. "Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all," he wrote in September 2026.³ Anthropic reported that month that "Claude is not operating fully autonomously for any measured subset of AI R&D work."⁴

The International AI Safety Report describes most company safety measures as voluntary. The International AI Safety Report counts 12 companies that published or updated frontier AI safety frameworks in 2025, documents describing how they plan to manage risks as models grow more capable, and notes that most such measures are not required by law.¹

Outside testing is a central dispute. Amodei proposed in September 2026 that each frontier company give "ongoing, employee-like access to a team of embedded third-party evaluators,"³ and OpenAI chief executive Sam Altman said he agreed.⁵ Later that month, President Trump and leaders of Google, Anthropic, Meta, OpenAI, xAI and Nvidia were listed on a White House accord committing each company to internal controls, an independent outside auditor and board oversight. It says that "Over time, it may make sense to codify these steps into laws or regulations."⁶ On October 1, OpenAI said it had fired three safety researchers because "these individuals mishandled sensitive information outside established company procedures."⁷ The researchers dispute that account in a letter titled "OpenAI cannot make AI safe on its own."⁸

California's SB 53, signed September 29, 2025, requires large frontier developers to publish a framework for managing catastrophic risk, report critical safety incidents to the state Office of Emergency Services within 15 days, or 24 hours when there is imminent danger, and protect employees who report risks. Violations carry civil penalties of up to $1 million each.² In September 2026, Gov. Gavin Newsom ordered recommendations on further steps, including onsite independent verifiers, a frontier-model "kill switch" and loss-of-control incident reporting.⁹

No comprehensive federal law covers frontier AI. Bills introduced in 2026 range from oversight to prohibition. The FRONTIER Act from Reps. Jay Obernolte and Lori Trahan would require third-party safety assessments, which OpenAI has backed.¹⁰ Sens. Michael Bennet and Peter Welch proposed an AI Regulator Act that "would require pre-certification for frontier AI models" and allow a pause of up to six months on the public distribution of models with the potential for catastrophic risk.¹¹ Sen. Bernie Sanders and Rep. Greg Casar introduced a bill to ban artificial superintelligence and pause advanced AI development until a federal regulator is in place.¹² Roll Call reported in September 2026 that no bill seemed to have a critical mass of support.¹³

Federal policy also includes executive action. On October 9, the White House Super Intelligence Force said companies must disclose model incidents and remediate harm. Its statement cites no statute, regulation or order and does not identify enforcement mechanisms or penalties.¹⁷

Critics of new rules say the risks are overstated or already covered by existing law. President Trump called the issue a "hoax" and wrote that "The only control or 'guardrails' that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT," adding: "We already have tremendous CRIMINAL and REGULATORY power over these companies!"¹³ Sen. Bernie Moreno questioned Anthropic's warnings as it prepared to sell shares to the public, writing that he did "not understand how you can ask them to invest their hard-earned savings in Anthropic's continued development while warning those same Americans that your product could ultimately lead to their extinction."¹⁴ A proposed class action filed in September 2026 alleges that Anthropic, OpenAI, Google and SpaceXAI unlawfully coordinated to slow AI development.¹⁵

Amodei has argued for discussing AI risks alongside AI's benefits and continued development, while rejecting both fatalism and extreme measures without evidence.¹⁶ Others want firmer limits than any company has offered, including the pause Sanders proposed.¹²