Randomness in Scientific Discovery: Recent Model Challenges Traditional Methods

The pursuit of a “virtual scientist”—an AI capable of conducting research from inception to publication—is driving innovation and prompting fundamental questions about the scientific method itself. Recent research suggests that, contrary to conventional wisdom, random exploration may be more effective at generating useful theories than carefully planned experiments.

Challenging Established Scientific Practices

A new paper published in Collective Intelligence applies the scientific method to itself, revealing that strategies commonly considered hallmarks of good experimental design can actually hinder the development of effective theories. The study, led by Marina Dubova, a Complexity Postdoctoral Fellow at the Santa Fe Institute (SFI), found that randomly chosen experiments often outperform those designed to confirm existing knowledge, disprove prevailing theories, or resolve conflicting viewpoints.

“These results contradict some common intuitions about the scientific method,” says Dubova. “The traditional ways we teach people to do experiments seem very premeditated: let’s confirm what we know, let’s try to falsify a dominant theory, let’s resolve a disagreement between two theories. But weirdly enough, we found that such carefully motivated experiments don’t seem to guide scientists toward useful theories as well as randomly chosen ones.”

Agent-Based Modeling and the Illusion of Progress

Dubova collaborated with Arseny Moskvichev, a former SFI Postdoctoral Fellow, and Kevin Zollman of Carnegie Mellon University to conduct the research. The team employed an agent-based model, a technique from complexity science, to simulate human scientists as individual “agents” within a computer program. These agents conducted experiments, formed theories based on the results, and shared their findings—mimicking the collaborative nature of real-world scientific inquiry.

The researchers created a statistical “ground truth” for the agents to explore, analogous to the characteristics of a hypothetical alien species that scientists might investigate. The model revealed that the most informative and predictive results emerged when agents randomly collected data, rather than pursuing theory-driven experiments.

Interestingly, agents focused on confirming or disproving existing theories often developed a false sense of progress. They were able to construct plausible explanations for the “ground truth,” but these explanations were frequently inaccurate. “The agents were able to develop an illusion of progress,” Dubova explains. “Using theory-motivated experimentation strategies, agents collected a narrower set of data, which made it less likely for them to encounter observations that challenged their theories.”

Implications for the Future of Science

Although Dubova cautions against abandoning carefully designed experiments altogether, she emphasizes the importance of scientists being aware of their own biases and assumptions. “There is a vicious cycle you can enter, where you collect data using what you think is a good strategy and grow confident in your success, but actually, you’re not learning much about the world.”

This research arrives amidst a surge of interest in artificial intelligence’s potential to revolutionize scientific discovery. The Santa Fe Institute is actively involved in projects exploring the foundations of AI and its application to complex systems, including the development of AI systems capable of independent scientific research. Artificial intelligence: Foundations to frontiers

Melanie Mitchell, a Professor at the Santa Fe Institute and expert in artificial intelligence and complex systems, has extensively studied the challenges of creating truly intelligent machines. Melanie Mitchell | Santa Fe Institute Her work highlights the gap between current AI capabilities and the commonsense reasoning and abstract thinking that characterize human intelligence.

Further Reading

• Reassessing the scientific method | Santa Fe Institute
• Artificial intelligence: Foundations to frontiers

Citation: Dubova, M., Moskvichev, A., & Zollman, K. (2026). Against theory-motivated experimentation: Can random experimental choice lead to better theories?. Collective Intelligence. DOI: 10.1177/26339137261421577