ControlAI Campaigns for Legal Bans on Superintelligence
A non-governmental organization is pushing lawmakers in four Western nations to legally ban the development of artificial superintelligence and recursive self-improvement. ControlAI warns that advanced AI systems capable of outperforming humans across every dimension present an uncontrollable risk, as tech companies engage in a race to automate AI research itself.
Germany Director Benjamin Balde Details Rapid AI Capabilities
Benjamin Balde, head of ControlAI in Germany, stated in an interview reported by heise.de and the KI-Update podcast that current AI models already outpace most humans in specific technical domains. Balde noted that AI programs write code better than a majority of human programmers and recently helped solve hundreds of open problems in theoretical computer science and mathematics. KI kann besser programmieren als der Großteil aller Menschen mit Programmierkenntnissen,
so Balde. He noted that some of these mathematical proofs, if accomplished by a human, could have led to a Fields Medal, the mathematical equivalent of the Nobel Prize. Balde also highlighted how AI increases speed in other areas, such as AlphaFold mass-accelerating the protein folding problem for researching new proteins and aiding massively in the recognition of radiological images.
Balde explained that developers are no longer aiming solely for Artificial General Intelligence (AGI) at human equivalence. Instead, the explicit target is superintelligence that surpasses human ability in all areas. He cautioned that once companies automate AI research, the step of AGI could be bypassed. Podcast host Isabel Grünewald noted that AI is already taking over up to 26 percent of AI research itself, though Balde mentioned this figure should be taken with a pinch of salt. When AI developers Dario Amodei and Sam Altman warned urgently about the dangers of their own technology a few weeks ago, the term superintelligence reappeared repeatedly. Altman stated a year ago that by the end of the decade, AI will be able to do things and make scientific discoveries that humans alone would not be able to achieve, with this timeframe shrinking further with every new, more powerful AI model.
Global Regulatory Proposals Target High-Risk Training Infrastructure
Unlike the European Union’s AI Act, which focuses on what happens when models come to market, ControlAI advocates for restrictions during the training phase. According to heise.de coverage, organization representatives have met with more than 400 parliamentarians across the United States, Canada, the United Kingdom, and Germany over a two-year period.
Balde argued that self-regulation among tech firms fails, drawing parallels to social media companies. Without binding state intervention, many companies would be ready to do more for safety if they knew this applied to all competitors.
Enforcing a Superintelligence Ban
ControlAI does not seek a blanket ban on generative AI. Balde explained the core risk as a loss of control, noting that if moral alignment is not mastered by that point, control over the systems risks derailing. Balde compared humans to a four-year-old child tasked with putting an adult—representing the AI—to bed, explaining, Ich glaube, ohne ein passendes System macht der Erwachsene, was er will, weil er stärker ist, mehr Wissen und mehr Möglichkeiten hat.
Frequently Asked Questions About Superintelligence Regulation
What specific definition does ControlAI use for superintelligence?
ControlAI defines superintelligence as an artificial intelligence system that outperforms humans in every dimension, rendering it more powerful than any individual, corporation, or nation, as reported by heise.de.
Which countries are currently considering legislative proposals backed by ControlAI?
ControlAI has engaged lawmakers and supported concrete legislative drafts in the United States, Canada, the United Kingdom, and Germany, according to interview details published on the KI-Update podcast.
Why does ControlAI argue that industry self-regulation cannot prevent risks?
Benjamin Balde stated that competitive market pressures mean many companies would be more willing to act on safety if they knew the same conditions applied to all competitors.
Worth a look