Deep Learning Models to Study Sentence Comprehension in the Human Brain

TL;DR

This paper reviews how deep learning models, especially neural networks, can serve as tools to understand the neural mechanisms of sentence comprehension in the human brain, highlighting similarities and challenges.

Contribution

It provides a comprehensive review of studies comparing artificial neural networks with brain activity, emphasizing the alignment of word meaning representations and processing hierarchies.

Findings

Neural word representations align with context-dependent word vectors.

Processing hierarchies in neural networks broadly match brain patterns.

Inconsistencies across studies highlight challenges in modeling comprehension.

Abstract

Recent artificial neural networks that process natural language achieve unprecedented performance in tasks requiring sentence-level understanding. As such, they could be interesting models of the integration of linguistic information in the human brain. We review works that compare these artificial language models with human brain activity and we assess the extent to which this approach has improved our understanding of the neural processes involved in natural language comprehension. Two main results emerge. First, the neural representation of word meaning aligns with the context-dependent, dense word vectors used by the artificial neural networks. Second, the processing hierarchy that emerges within artificial neural networks broadly matches the brain, but is surprisingly inconsistent across studies. We discuss current challenges in establishing artificial neural networks as process models of natural language comprehension. We suggest exploiting the highly structured representational geometry of artificial neural networks when mapping representations to brain data.