Human Brain and AI Speech Recognition Decode Speech in Similar Ways, Study Finds

Researchers have discovered that the way the human brain processes spoken language shares striking similarities with how artificial intelligence (AI) systems, specifically recurrent neural networks (RNNs), decode speech. This finding, published in Nature Machine Intelligence, offers modern insights into the computational principles underlying language understanding and could pave the way for more advanced brain-computer interfaces and AI models.

Bridging the Gap Between Neuroscience and AI

For decades, computer scientists have been developing AI systems capable of processing human speech. Yet, the extent to which these models mirror the brain’s natural speech processing mechanisms remained unclear. A team from Columbia University, IBM Research, and the Feinstein Institutes for Medical Research sought to address this question by comparing the internal workings of AI speech recognition systems with brain activity recorded from human subjects.

How the Study Was Conducted

The study involved 15 patients undergoing monitoring for epilepsy, who had high-resolution electrodes implanted directly in their auditory cortex. While being monitored, these patients listened to 30 minutes of narrated stories. Researchers then compared the brain activity recordings with the internal states of RNNs processing the same audio. RNNs were chosen due to the fact that, unlike many common AI models, they process information sequentially, mirroring the step-by-step nature of human auditory processing.

Key Findings: A Hierarchical Alignment

The researchers found a remarkable correspondence between the brain’s hierarchical processing of speech and the layered structure of the RNNs. Both systems progressively decode speech, moving from basic acoustic features to phonetic, lexical (word-based), and ultimately semantic information. As information flows deeper into the AI’s layers, it maps topographically to the human cortical hierarchy, progressing from the primary auditory core to higher-order language regions.

“Both systems follow a nearly identical sequence, progressing from basic acoustic features to phonetic, then lexical (words), and finally semantic information,” explained Nima Mesgarani, senior author of the paper, in a TechXplore interview. “This suggests that this specific hierarchical transformation is likely a robust, efficient computational solution discovered by both biological evolution and task-optimized AI.”

Implications for Future Research and AI Development

These findings suggest that RNNs, in particular, capture fundamental aspects of how the brain processes speech. This opens up possibilities for using AI models as “transparent” versions of the brain to better understand its computational principles. Researchers also noted that the brain’s language processing is often lateralized, with the left hemisphere playing a dominant role. Future research will explore whether building “dual pathway” AI models that mimic this asymmetry could yield further insights.

The study also revealed that the brain-like computational strategy in RNNs only emerged when the models were trained on speech in a specific language, prompting further investigation into how the brain processes native versus foreign languages.

Looking Ahead

the goal is to leverage these AI models as explicit hypotheses for how the brain solves the sound-to-meaning problem. Researchers envision a future where we can simulate brain language processing to understand the unique aspects of biological intelligence and how these processes can be impaired. This research not only advances AI technology but also provides a powerful tool for understanding the complexities of the human brain.

Citation: Keshishian, M., et al. Parallel hierarchical encoding of linguistic representations in the human auditory cortex and recurrent automatic speech recognition systems. Nature Machine Intelligence (2026). DOI: 10.1038/s42256-026-01185-0

© 2026 Science X Network