When Language Models Lose Their Mind: The Consequences of Brain Misalignment

TL;DR

This paper investigates the importance of brain alignment in large language models by comparing well-aligned and misaligned models across numerous linguistic tasks, revealing that misalignment significantly hampers language understanding.

Contribution

It introduces brain-misaligned models to study the impact of brain alignment on linguistic performance, providing new insights into neural and linguistic relationship in LLMs.

Findings

Brain misalignment impairs downstream linguistic performance

Brain alignment is crucial for robust language understanding

Misaligned models perform worse across diverse linguistic tasks

Abstract

While brain-aligned large language models (LLMs) have garnered attention for their potential as cognitive models and for potential for enhanced safety and trustworthiness in AI, the role of this brain alignment for linguistic competence remains uncertain. In this work, we investigate the functional implications of brain alignment by introducing brain-misaligned models--LLMs intentionally trained to predict brain activity poorly while maintaining high language modeling performance. We evaluate these models on over 200 downstream tasks encompassing diverse linguistic domains, including semantics, syntax, discourse, reasoning, and morphology. By comparing brain-misaligned models with well-matched brain-aligned counterparts, we isolate the specific impact of brain alignment on language understanding. Our experiments reveal that brain misalignment substantially impairs downstream performance, highlighting the critical role of brain alignment in achieving robust linguistic competence. These findings underscore the importance of brain alignment in LLMs and offer novel insights into the relationship between neural representations and linguistic processing.