A Brain-like Synergistic Core in LLMs Drives Behaviour and Learning

TL;DR

This paper reveals that large language models develop brain-like synergistic cores that are crucial for their behavior and learning, highlighting the importance of synergy in intelligence.

Contribution

It demonstrates that LLMs spontaneously develop synergistic cores similar to biological brains, and that these cores are essential for performance and learning.

Findings

Synergistic cores emerge through learning in LLMs.

Ablating synergistic components impairs performance.

Reinforcement learning enhances synergy and performance.

Abstract

The independent evolution of intelligence in biological and artificial systems offers a unique opportunity to identify its fundamental computational principles. Here we show that large language models spontaneously develop synergistic cores -- components where information integration exceeds individual parts -- remarkably similar to those in the human brain. Using principles of information decomposition across multiple LLM model families and architectures, we find that areas in middle layers exhibit synergistic processing while early and late layers rely on redundancy, mirroring the informational organisation in biological brains. This organisation emerges through learning and is absent in randomly initialised networks. Crucially, ablating synergistic components causes disproportionate behavioural changes and performance loss, aligning with theoretical predictions about the fragility of synergy. Moreover, fine-tuning synergistic regions through reinforcement learning yields significantly greater performance gains than training redundant components, yet supervised fine-tuning shows no such advantage. This convergence suggests that synergistic information processing is a fundamental property of intelligence, providing targets for principled model design and testable predictions for biological intelligence.