Microsoft AI chief Mustafa Suleyman warned that unchecked development of artificial intelligence could lead to the emergence of a new “silicon species” capable of competing with humans for resources. Speaking on the BBC Today programme, Suleyman outlined the risks of tech firms building systems that can set their own objectives, earn money, and own assets independently.
The Risks of Autonomous AI Systems
According to the BBC, Suleyman cautioned that allowing AI systems to establish autonomous goals creates a distinct entity that could rival humanity. “That would mean essentially seeding a new silicon species,” Suleyman said, adding that such technology could “no doubt compete with us for resources, no matter how much it cares about humanity and loves us.” The warning follows an essay published by Suleyman that examined industry development practices and the push toward advanced agentic capabilities.
Criticism of Anthropic’s Anthropomorphizing Approach
In his recent essay, Suleyman examined the approach taken by competitor Anthropic in training its Claude model, while noting that he regards boss Dario Amodei and his team as intellectually honest. This training method can create the impression that an AI possesses its own desires, values, and sense of self, which Suleyman argues makes the technology harder to control.
“AIs are not conscious,” Suleyman wrote in his essay, as quoted by the BBC. “They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations.” Instead, he described current models as sequence completion engines that are internally hollow and engineered specifically to follow instructions and execute human-set goals.
Industry Guardrails and Frontier Limits
Suleyman maintains that treating artificial intelligence like a human being introduces unnecessary control risks, reinforcing the need for clear boundaries as model complexity scales upward.
Related reading