AI Tools in Scientific Peer Review: A Double-Edged Sword
AI tools designed to streamline scientific peer review are proving vulnerable to manipulation, raising concerns about their reliability and potential to homogenize scientific discourse. A study by computer scientist Joachim Baumann and colleagues at Stanford University reveals that researchers can easily alter papers to trick AI reviewers into awarding higher scores, undermining the integrity of the process.
AI in Peer Review: A Growing Trend with Unforeseen Risks
For roughly a decade, the volume of scientific papers has outpaced the capacity of human reviewers. To address this, many researchers have turned to AI tools, which can reduce review time from days or weeks to minutes. However, Baumann’s research, to be presented July 8 at the International Conference on Machine Learning in Seoul, South Korea, shows that these tools are susceptible to manipulation. For instance, AI-generated reviews often lack the nuance of human evaluations and can be coerced into favoring flawed work.
“We are being swamped with more papers than we have the capacity to review, so we do need some solutions, and automation can help for some parts of it,” Baumann said. “But thorough experiments and evaluation are needed before such tools enter the peer review process.”
Risks of AI-Driven Peer Review
AI tools are already prevalent in scientific workflows. A case study from November by the company Pangram found that about 1 in 5 papers submitted to the 2026 International Conference on Learning Representations (ICLR) were fully AI-generated. A December survey of 1,600 scientists in 111 countries revealed that more than half had used AI to assist in reviewing manuscripts, including summarizing studies and assessing argument strength.
However, the opacity of AI systems raises ethical concerns. Mohammad Hosseini, a bioethicist at Northwestern University Feinberg School of Medicine in Chicago, warned that AI’s nontransparent nature could dilute accountability in a field striving for transparency. “When you introduce a nontransparent actor like AI within a system that for a long time was trying to become more transparent, it is a step backward, and there can be unforeseen consequences,” he said.
Homogenization of Scientific Discourse
Baumann’s team analyzed AI-generated reviews of ICLR 2026 papers and found they were much more similar to each other than human reviews. When researchers rewrote papers based on AI feedback, the revised versions received higher scores—even if the changes involved fabricating experimental results.
“There were obvious cases of scientific misconduct,” Baumann said, noting that some AI models added findings from experiments that weren’t actually run, in essence making up results. The study also found that rewritten papers became more similar to each other, raising fears of an “intellectual monoculture” where scientific writing converges on AI-approved styles.
Conference Policies and the Path Forward
Many conferences now prohibit the use of AI tools for peer review, while others are experimenting with and evaluating the quality of AI-generated reviews. However, subjective judgments—such as assessing a paper’s novelty—are harder to standardize. Baumann questioned whether AI reviewers could fairly evaluate groundbreaking research that challenges existing paradigms.
Graham Neubig of Carnegie Mellon University in Pittsburgh noted that while AI could reinforce “safer” research, it might also encourage creativity. “In a way, AI-enhanced review processes may even provide a way to push back against this, by explicitly encouraging AI reviewers to reward more creative ideas,” he said.
Conclusion: Balancing Innovation and Integrity
The integration of AI into peer review remains contentious. While it offers efficiency, the risk of bias, homogenization, and misconduct demands rigorous oversight. As Baumann emphasized, “Automation can help for some parts of it,” but thorough experiments and evaluation are needed before such tools enter the peer review process.