Anthropic Researcher Quits, Warns AI Could Pose Existential Threat To Humanity

A researcher who recently resigned from artificial intelligence company Anthropic has warned that the rapid development of advanced AI systems could pose an unprecedented threat to humanity, including the possibility of human extinction within the next decade.

Jacob Coxon, who said he spent the past three years conducting pre-training research at both OpenAI and Anthropic, announced his resignation on Wednesday, accusing leading AI companies of pursuing self-improving superintelligence without adequate safeguards.

Coxon said the pace of AI development should not be underestimated, warning that increasingly advanced systems could eventually outperform humans in critical areas, compromise digital systems and gain access to significant resources.

He argued that the capabilities of AI were advancing rapidly across several domains and warned that the technology could become significantly more powerful if development continued without effective safety measures.

According to Coxon, some people involved in developing advanced AI privately recognise the possibility of catastrophic outcomes, despite adopting more measured language when discussing the risks publicly.

His warning was echoed by another Anthropic researcher, Evan Hubinger, who said he believed there was a greater than 10 per cent chance that AI could kill all humans within the next decade.

Hubinger, while saying he believed Anthropic was making serious efforts to address AI safety, said the company had yet to develop a reliable solution to the alignment problem associated with superintelligent AI.

The comments come amid growing international concern over the development of increasingly autonomous and capable AI systems and the ability of researchers, governments and technology companies to control their potential risks.

AI safety researchers have warned that future systems could present risks ranging from cyberattacks and misuse of autonomous agents to loss of human control over highly capable systems.

The debate has intensified as major technology companies continue investing heavily in advanced AI, while governments and researchers seek stronger safeguards, evaluation standards and regulatory frameworks.

Coxon’s resignation has therefore added to an increasingly prominent debate within the AI industry over whether the race to develop more powerful systems is moving faster than efforts to ensure they remain safe and aligned with human interests.