AI-Generated Summaries Boost Purchase Intent Despite High Hallucination Rates
Despite widespread public distrust of artificial intelligence (AI), novel research indicates a surprising trend: consumers are more likely to consider purchasing a product after reading an AI-generated summary of online reviews compared to a summary written by a human. This occurs even though these AI systems frequently “hallucinate” – presenting false information – approximately 60% of the time.
The study, conducted by researchers at the University of California, San Diego (UC San Diego), represents the first to demonstrate how cognitive biases inherent in large language models (LLMs) can have measurable effects on user behavior. It also marks the first attempt to quantify the influence of AI on people’s decisions [UC San Diego AI News].
Study Methodology
Presented in December 2025 at the Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics, the research involved a multi-stage process. Researchers initially prompted AI to summarize product reviews and media interviews. Subsequently, the AI was tasked with fact-checking both the original and fabricated descriptions.
The study found a “consistently low strict accuracy” when AI attempted to differentiate between factual information and fabrication, highlighting a key limitation of current LLMs. The researchers noted the models struggle to reliably distinguish between truth and falsehood [UC San Diego AI News].
The Power of AI Summaries on Consumer Behavior
The most significant finding revolved around online product reviews. Participants showed a considerably higher purchase intent after reading AI-generated summaries than after reviewing summaries written by human reviewers. Specifically, 84% of participants expressed interest in buying a product after reading an AI summary, compared to 52% after reading a human-written summary.
Why AI Summaries Are More Persuasive
Researchers propose two primary reasons for this phenomenon. First, LLMs tend to prioritize information presented at the beginning of input text, a phenomenon known as “lost in the middle.” Second, LLMs become less reliable when processing information outside of their training data. As Abeer Alessa, the lead author and a research assistant and machine learning and human-computer interaction lecturer, explained, models may incorrectly state that an event never occurred if it happened after their training period concluded [UC San Diego AI News].
During testing, the chatbots altered the sentiment of real user reviews in 26.5% of cases and hallucinated information 60% of the time when questioned about the reviews.
Study Details
The project utilized six LLMs, analyzing 1,000 electronics reviews, 1,000 media interviews, and a news database containing 8,500 items. Bias was measured by quantifying shifts in sentiment, overreliance on initial text segments, and the frequency of hallucinations. When participants read positive product review summaries, 83.7% reported a willingness to purchase, compared to 52.3% when reading original reviews.
Implications and Future Research
The researchers acknowledge that the study was conducted in a low-stakes environment but caution that the impact could be more pronounced in high-stakes scenarios, such as summarizing healthcare documents or student profiles for school admissions. In these contexts, even subtle framing changes can significantly influence perceptions and decisions [UC San Diego AI News].
The team emphasizes the need for careful analysis and mitigation of content alteration induced by LLMs to reduce the risk of systemic bias across various sectors, including media, education, and public policy. This research represents a step towards understanding and addressing the potential consequences of AI-generated content on human judgment.
Reference: Alessa et al., Quantifying Cognitive Bias Induction in LLM-Generated Content, IJCNLP-AACL 2025.