On the Analogy between Human Brain and LLMs: Spotting Key Neurons in Grammar Perception

TL;DR

This paper demonstrates that Large Language Models like Llama 3 process grammatical categories using specific neurons, similar to how the human brain handles language, revealing a neural subspace dedicated to part-of-speech recognition.

Contribution

It identifies key neurons in LLMs associated with grammatical categories and shows their activation patterns can predict part-of-speech tags, paralleling neuroscientific findings.

Findings

Key neurons correlate with part-of-speech prediction

Activation patterns reliably classify parts of speech

Evidence of a grammatical subspace in LLMs

Abstract

Artificial Neural Networks, the building blocks of AI, were inspired by the human brain's network of neurons. Over the years, these networks have evolved to replicate the complex capabilities of the brain, allowing them to handle tasks such as image and language processing. In the realm of Large Language Models, there has been a keen interest in making the language learning process more akin to that of humans. While neuroscientific research has shown that different grammatical categories are processed by different neurons in the brain, we show that LLMs operate in a similar way. Utilizing Llama 3, we identify the most important neurons associated with the prediction of words belonging to different part-of-speech tags. Using the achieved knowledge, we train a classifier on a dataset, which shows that the activation patterns of these key neurons can reliably predict part-of-speech tags on fresh data. The results suggest the presence of a subspace in LLMs focused on capturing part-of-speech tag concepts, resembling patterns observed in lesion studies of the brain in neuroscience.