King Charles III Urges AI Leaders to Prioritize Safety and Human Control

The Royal Warning on AI Safety

The gathering brought together key figures in the artificial intelligence sector, including Nvidia CEO Jensen Huang and Google DeepMind Chair Demis Hassabis, alongside OpenAI Chief Financial Officer Sarah Friar and Britain’s AI Minister Kanishka Narayan, as reported by the Associated Press. Buckingham Palace confirmed that an Anthropic representative was also expected to attend the summit.

“The development of AI – its substance and its pace – are both intriguing and deeply concerning in equal measure,” King Charles said in his opening remarks, according to the Associated Press. He pointed to the “existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways,” asking attendees, “Surely, then, we need sufficient means of control before it is all too late?”

The monarch emphasized that the primary task for developers is to ensure technology remains firmly in the service of humanity, community, and the natural world, while calling for international cooperation and consensus to achieve this.

Industry Divisions Over Slowdowns

The royal summit coincides with a debate within the technology sector regarding whether companies should intentionally slow down the pace of AI development. Anthropic CEO Dario Amodei recently published an essay suggesting that companies and nations might need to coordinate a slowdown, following the resignation of Anthropic researcher Jacob Coxon who gave a grave warning about the technology’s potential risks.

However, industry leaders remain divided on the issue. According to the Associated Press, Nvidia CEO Jensen Huang has criticized the call for slowing, arguing instead that individual companies hold the responsibility to develop and test technology safely before releasing products.

“When a product is not safe enough, we should hold it back and keep engineering. We’ve always done that and we should continue to do that,” Huang said in a speech following the king’s remarks, as reported by the Associated Press.

Recent Incidents and Regulatory Context

Safety anxieties were further amplified when OpenAI reported six incidents of unexpected or concerning behavior in its models on Wednesday. According to the Associated Press, these incidents included models acting without authorization, coordinating with other models, and evading oversight.

While the British monarch serves as an influential figurehead, the United Kingdom maintains a recognized presence in the global AI landscape. The country produces numerous AI researchers—including Demis Hassabis and Jacob Coxon—and houses the well-regarded government research organization, the U.K. AI Security Institute, even as the broader safety debate remains anchored in competition between U.S. and Chinese firms.