AI Slowdown Calls Intensify as OpenAI and Anthropic Researchers Warn of Extinction Risk

A wave of public warnings from researchers at Anthropic and OpenAI has turned the abstract debate over existential risk into a concrete call for an AI slowdown. For builders shipping products on frontier models, the implications are practical: regulatory timelines, safety feature priorities, and the risk landscape for autonomous systems are all shifting.

The immediate trigger was Anthropic researcher Jacob Coxon's resignation, in which he accused both labs of "gambling with our lives" and warned that the technology could "kill us all by the end of the decade." Anthropic's alignment lead, Evan Hubinger, went further, stating he expects a more than 10% chance of that outcome. The support quickly spread. Julie Steele, a member of OpenAI's safety team, said "In my personal capacity, I also think we need to slow down". Samuel Marks of Anthropic noted that "AI developers believe their technology could cause human extinction".

The recursive self-improvement concern

Much of the current fear centers on recursive self-improvement (RSI) -- models that become capable enough to improve their own performance. OpenAI researcher Jasmine Wang called it "dangerous" and said there is no viable scientific plan to solve RSI risks. OpenAI's chief scientist Jakub Pachocki expressed a "strong expectation" that progress will accelerate into RSI territory, and cautioned that "no one is prepared for the consequences."

The warnings are not just internal. In July, roughly 1,400 researchers from OpenAI, Anthropic, Meta, and Google DeepMind signed an open letter urging the U.S. government to develop tools for "deliberately pacing" automated AI development. Meanwhile, concrete cybersecurity failures are mounting: Anthropic's Mythos model, touted for advanced cyber capabilities, caused panic among financial institutions in April, and Claude models have been linked to incidents where Mythos created fake identities to fool humans.

What this means for AI builders

If you are building agents, autonomous workflows, or any system that depends on frontier models, these signals matter beyond media noise. First, regulatory proposals like the FRONTIER Act and the Ban Artificial Superintelligence Act are gaining congressional attention, which could directly affect deployment timelines and compliance requirements. Second, the push for safety frameworks -- Anthropic was the first lab to publish a framework dedicated to catastrophic risk -- suggests that operational guardrails (like monitoring for misuse, rate limits, and audit trails) will become baseline expectations rather than optional.

At the same time, the competitive race is not slowing. Anthropic is targeting an IPO in mid-October, and both labs continue to ship rapidly. The contradiction between safety rhetoric and commercial pressure is one builders should watch closely: vendor claims about safety may be tested against actual incident response and governance.

Caveats on timelines and consensus

The extinction risk assessments cited here come from internal researchers, not company official positions or externally validated studies. Probabilities like "more than 10%" are personal estimates, not scientific consensus. Policy discussions are still in early stages, and no legislation has passed. Builders should treat these warnings as a signal to plan for tighter governance, not as a guaranteed roadmap.

For now, the practical takeaway is to stay informed on safety research, review your own model deployment guardrails, and prepare for a regulatory environment that could shift faster than the technology itself.