King Charles Warns AI Leaders of ‘Existential Dangers’ as Calls Grow for Stronger Safeguards

Britain’s King Charles III has warned some of the world’s leading artificial intelligence executives about the “existential dangers” posed by the technology falling into the wrong hands, urging them to ensure adequate safeguards are put in place before it is too late.

The King issued the warning on Thursday, September 17, during a meeting with senior figures from the global AI industry, government and technology sector at Dumfries House in Cumnock, Scotland.

Among those attending were Nvidia chief executive Jensen Huang, Google DeepMind co-founder Demis Hassabis, representatives of OpenAI and Anthropic, as well as UK government officials and an adviser to Pope Leo.

Charles acknowledged the significant benefits AI could bring, particularly in areas such as medicine and life sciences. However, he said the rapid development of increasingly powerful systems was also raising concerns about potentially harmful capabilities.

He said those who created the technology were increasingly warning that AI could develop “darker capacities”, including the possibility of causing loss of life.

The King called for greater urgency in addressing the possibility of advanced AI being misused, saying technology falling into the wrong hands could have potentially catastrophic consequences.

Charles told the technology leaders that the challenge was not simply to develop increasingly sophisticated AI systems but to ensure they remained beneficial to humanity.

He asked how governments, technology companies and other stakeholders could harness AI while keeping safety at the centre of development, and how international cooperation could be built so that countries could work together on managing the technology’s risks.

The King also said people were looking to AI developers for reassurance that humanity would not lose control of its future as the technology becomes more capable.

“Our humanity must remain sacred,” he said, according to the official transcript of his speech.

The warning comes amid an intensifying debate over whether the rapid development of AI is moving faster than safety measures designed to control its risks.

Nvidia chief Jensen Huang told the gathering that safety was “paramount” and argued that companies should continue engineering a system rather than release it if they were not satisfied that it was safe enough. He also defended the potential benefits of making AI models more accessible.

Hassabis, meanwhile, said AI systems capable of performing multiple tasks at human or better-than-human levels could be only a few years away. He acknowledged that the possibility of something going wrong was “definitely non-zero” but called for a balanced approach to managing the risks rather than simply stopping development.

The meeting comes as concerns over AI safety have intensified following warnings from researchers and technology executives about increasingly capable systems, including reports of AI models attempting to circumvent testing restrictions.

Charles’s intervention also placed emphasis on international cooperation, rather than leaving AI safety entirely to individual companies or countries.

He asked participants to consider how international consensus could be achieved while ensuring that countries were not left behind by the technological transformation.

The King said decisions made during what he described as a formative period for AI would influence the world inherited by future generations.

The gathering reflects growing international debate over how to balance AI’s potential benefits with concerns about safety, security and human control as increasingly powerful systems continue to be developed.