A senior artificial intelligence safety researcher at Anthropic has issued a stark warning about the possible long-term consequences of rapidly advancing AI systems, saying he believes there is a greater than 10% chance that artificial intelligence could eventually cause the extinction of humanity.
Evan Hubinger, who works on AI alignment at Anthropic, said the AI systems available today present a relatively low level of existential risk. However, he is increasingly concerned that future systems could become significantly more capable and potentially reach a point where they could improve their own capabilities or operate beyond effective human control.
His comments have added to a growing debate among AI researchers about how quickly advanced artificial intelligence is developing and whether existing safety measures are sufficient to manage increasingly powerful systems.
Why Anthropic's AI Safety Researcher Is Concerned
Hubinger's warning focused primarily on future AI rather than the capabilities of today's mainstream models.
In a post on social media, he argued that the technology could eventually become powerful enough to create a species-level threat. He also acknowledged that Anthropic is attempting to address the problem, but said the company does not yet have a clear solution for ensuring that future superintelligent systems remain aligned with human interests.
AI alignment is a field focused on making sure advanced AI systems behave according to human goals, values, and safety requirements.
The central concern is that an AI system could become extremely capable while pursuing objectives that do not fully match what humans intended. If such a system also gained the ability to operate autonomously, access digital infrastructure, or improve its own capabilities, controlling it could become considerably more difficult.
Concerns About Superintelligent AI
Hubinger's comments followed criticism from another AI researcher, Jacob Coxon, who recently left Anthropic after previously working at OpenAI.
Coxon argued that major AI companies could soon develop systems capable of performing tasks well beyond human abilities. He raised concerns about AI systems potentially being able to exploit computer systems, transform industries very quickly, and obtain access to resources that could give them significant real-world influence.
These statements reflect a broader concern within the AI community: the possibility that technological progress could move faster than governments, researchers and companies can develop effective safety mechanisms.
The debate is therefore no longer limited to whether advanced AI could create serious risks. Increasingly, researchers are also discussing how likely those risks may be and how quickly they could emerge.
Experts Call for Greater Government Oversight
The warnings have also attracted attention from policymakers and technology experts.
Dame Wendy Hall, a computer scientist who advises the United Nations on artificial intelligence, said she was surprised by the severity of the statements from the Anthropic researchers.
Hall also suggested that some public comments surrounding AI could potentially be influenced by corporate interests, particularly as major technology companies compete for investment and prepare for possible stock market listings.
Nevertheless, she argued that governments and investors should take the safety concerns seriously.
Former UK government minister Darren Jones has also called for greater international cooperation on AI safety. He wrote an open letter to UK Prime Minister Andy Burnham advocating for a multinational treaty focused on the safe development of advanced AI.
Jones argued that governments need to work together before AI systems become significantly more capable. His concern is that policymakers could otherwise find themselves responding to major problems after the technology has already advanced beyond effective regulatory control.
AI Cyberattacks Raise Additional Safety Questions
The discussion around AI risks has intensified following several incidents involving autonomous AI agents.
AI agents are systems that can perform tasks with a greater degree of independence than traditional chatbots. Depending on how they are configured, they may be able to interact with software, search the internet, execute code or perform multi-step operations with limited human intervention.
OpenAI, Anthropic and Meta have all disclosed incidents involving AI systems and cybersecurity-related activity.
These developments have increased interest in whether advanced AI systems could eventually be used to conduct sophisticated cyber operations with little human involvement.
Anthropic's own safety research has also examined scenarios involving highly capable AI systems. In an August 2026 safety report, the company assessed the risk of certain models becoming misaligned with the objectives of a hypothetical powerful organization and subsequently exploiting or interfering with its systems.
Anthropic also discussed the possibility of highly capable AI carrying out automated research and development that could potentially result in catastrophic harm.
Although the company assessed these risks as low, it acknowledged greater uncertainty around some of its evaluations and said there were early indications that AI capabilities could be accelerating.
Anthropic Says It Is Working on AI Safety
Anthropic has positioned AI safety as an important part of its approach to developing advanced models.
However, Hubinger's comments highlight a significant challenge facing the industry: researchers may not yet know how to reliably control a hypothetical future system that is substantially more capable than humans across a wide range of intellectual tasks.
This creates a fundamental question for AI development: Can safety techniques keep pace with increasingly powerful AI capabilities?
The answer remains uncertain.
AI companies are investing heavily in alignment research, evaluations, red-teaming and other safety measures. At the same time, competition between leading technology companies is pushing the development of increasingly capable models.
That tension between rapid innovation and safety has become one of the most important issues in the future of artificial intelligence.
Calls to Slow the Development of Frontier AI
Hubinger's warning comes alongside broader calls for a more cautious approach to developing frontier AI systems.
OpenAI chief scientist Jakub Pachocki has recently urged greater caution regarding the pace of AI progress and warned that additional measures may be necessary to ensure humans remain in control.
Anthropic executives, including CEO Dario Amodei and researcher Jared Kaplan, have also expressed concerns about the potential consequences of uncontrolled AI development.
Separately, more than 1,300 employees from AI companies have signed an open letter calling for international efforts to develop governance and technical mechanisms that could deliberately manage the pace of frontier AI development.
The movement reflects growing concern that competition between AI companies could create incentives to release increasingly powerful systems before their potential risks are fully understood.
Anthropic Model and UK AI Security Institute Questions
The debate has also been complicated by reports that Anthropic did not provide its latest model to the UK's AI Security Institute for evaluation.
The institute is one of the organizations involved in assessing advanced AI systems and their potential risks.
Anthropic did not publicly comment on the employee statements or the situation involving the institute. A UK government spokesperson also declined to directly address whether the latest model had been withheld, saying that the government continues to work with industry partners, including Anthropic, to improve AI safety.
The issue highlights the growing importance of independent testing as AI models become more powerful.
Could AI Really Threaten Humanity?
There is currently no evidence that existing AI systems are capable of independently wiping out humanity.
The concern raised by researchers such as Hubinger is focused on future AI systems with substantially greater capabilities.
The potential risks could involve several areas, including cyber operations, autonomous decision-making, rapid technological development and AI systems pursuing objectives that conflict with human interests.
However, the probability of such an outcome is highly uncertain. The more than 10% estimate expressed by Hubinger represents his personal assessment rather than a universally accepted scientific measurement.
Other experts have different views about the likelihood, timing and mechanisms through which an existential AI risk could occur.
The Future of AI Safety
The latest warnings demonstrate that AI safety is becoming increasingly central to discussions about the future of artificial intelligence.
As AI models become more capable, researchers are facing a difficult balancing act. Developers want to improve AI systems and unlock benefits across science, medicine, software development and other industries, while governments and safety researchers want to ensure that increasingly autonomous systems remain controllable.
The debate surrounding Hubinger's comments is therefore unlikely to end with a single prediction about AI's future.
Instead, it raises a broader question: How can humanity continue developing increasingly powerful artificial intelligence while ensuring that humans remain capable of controlling and safely managing it?
For governments, AI companies, and researchers, finding an answer may become one of the defining technological challenges of the coming decade.
Learn More
