Discovering Decoupled Functional Modules in Large Language Models

TL;DR

This paper introduces a novel unsupervised framework for discovering and interpreting functional modules within large language models, enhancing understanding of their internal organization and semantic specialization.

Contribution

It proposes the ULCMOD framework with a new objective and iterative algorithm to disentangle neurons into meaningful modules, advancing interpretability of LLMs.

Findings

Discovered high-quality, disentangled modules capturing semantic information

Modules show semantic coherence and interpretability

Improved downstream task performance

Abstract

Understanding the internal functional organization of Large Language Models (LLMs) is crucial for improving their trustworthiness and performance. However, how LLMs organize different functions into modules remains highly unexplored. To bridge this gap, we formulate a functional module discovery problem and propose an Unsupervised LLM Cross-layer MOdule Discovery (ULCMOD) framework that simultaneously disentangles the large set of neurons in the entire LLM into modules while discovering the topics of input samples related to these modules. Our framework introduces a novel objective function and an efficient Iterative Decoupling (IterD) algorithm. Extensive experiments show that our method discovers high-quality, disentangled modules that capture more meaningful semantic information and achieve superior performance in various downstream tasks. Moreover, our qualitative analysis reveals that the discovered modules show semantic coherence, correspond to interpretable specializations, and a clear spatial and hierarchical organization within the LLM. Our work provides a novel tool for interpreting the functional modules of LLMs, filling a critical blank in LLM's interpretability research.