Refine: AI Tool Revolutionizing Academic Paper Review
A new artificial intelligence tool, Refine, developed by Yann Calvó López and Ben Golub, is rapidly gaining traction within the academic community for its ability to provide remarkably insightful feedback on scholarly work. Economist John H. Cochrane, in a recent review, described the tool as “stunning,” stating the quality of its comments rivals the best feedback he’s received throughout his entire academic career.
How Refine Works and Initial Impressions
Refine is designed to refine academic articles, offering detailed critiques and suggestions for improvement. Cochrane tested the tool with a draft of his booklet on inflation, utilizing the free trial mode. He found the comments to be not only insightful but as well more concise and organized than traditional peer review feedback. Although acknowledging the tool isn’t perfect, Cochrane expressed astonishment at its analytical capabilities, questioning how such functionality arises from language prediction models.
Key Areas of Feedback from Refine
Refine’s analysis of Cochrane’s work focused on several key areas:
Operationalizing the “Fiscal News” Narrative
The AI highlighted a potential circularity in Cochrane’s argument regarding the start and end of inflation in 2021-2022, tied to perceptions of government debt. Refine suggested the need for more concrete, dated observables – such as legislative hurdles or election probabilities – to support the claim that “fiscal news” drove inflationary expectations and long-bond prices. Without these specific data points, the analysis risks appearing as a post-hoc interpretation.
Clarifying the Fiscal Regime Distinction
Refine pointed out a tension in the manuscript’s claim of “completeness” when compared to existing New Keynesian (NK) models. The AI suggested reframing the critique to focus on differing assumptions regarding fiscal behavior, arguing that the disagreement isn’t about the presence of a valuation equation, but rather how it determines prices (as in the FTPL model) versus future surpluses (in the NK model).
Resolving Ambiguity in the Transmission Mechanism
The tool identified a potential inconsistency in how the text describes the impact of interest rate hikes. Refine noted that rate hikes can lower inflation in the short-run by affecting bond values, but also potentially increase inflation through their impact on the present value of surpluses. The AI suggested clarifying the conditions under which each effect dominates.
Strengthening the Discrimination Against Monetarist Alternatives
Refine challenged Cochrane to address how the FTPL explanation outperforms even sophisticated Monetarist views that account for Interest on Reserves (IOR). The AI suggested that dismissing simpler Monetarist benchmarks weakens the argument.
Beyond Paper Review: Potential Applications
Cochrane also experimented with another AI tool, Claude, to automate graph creation from data series fetched from the Federal Reserve Economic Data (FRED) database. While Claude required some refinement in data selection, it successfully generated code and graphs with minimal intervention, significantly reducing the time required for data visualization.
The Future of Academic Writing and AI
Cochrane predicts a significant shift in academic workflows due to AI tools like Refine. He suggests that editors may soon routinely use AI to evaluate submissions and that authors will proactively seek AI feedback before submitting their work. However, he emphasizes that human evaluation will remain crucial, as AI-generated feedback still requires careful consideration and interpretation. He also raised concerns about the potential for AI to shape the consensus within academia, particularly if training datasets are biased or reflect settled viewpoints on controversial topics.
About the Developers
Yann Calvó López, CEO of Refine.ink, holds an MS in Computer Science from Harvard University. Ben Golub is a Professor of Economics at Northwestern University whose research focuses on social and economic networks. Both were also authors of a 2024 blog post discussing supply chain vulnerabilities.