Deconstructing Pre-training: Knowledge Attribution Analysis in MoE and Dense Models

TL;DR

This paper introduces a neuron-level attribution metric to compare knowledge acquisition in MoE and dense models, revealing that MoE's sparsity leads to early, stable, and distributed knowledge storage during pre-training.

Contribution

It presents Gated-LPI, a novel neuron attribution method, and provides the first detailed comparison of knowledge dynamics in MoE versus dense architectures during training.

Findings

MoE neurons form a high-utility core capturing most positive updates

MoE models stabilize importance profiles early in training

Sparsity in MoE leads to distributed and robust knowledge storage

Abstract

Mixture-of-Experts (MoE) architectures decouple model capacity from per-token computation, enabling scaling beyond the computational limits imposed by dense scaling laws. Yet how MoE architectures shape knowledge acquisition during pre-training, and how this process differs from dense architectures, remains unknown. To address this issue, we introduce Gated-LPI (Log-Probability Increase), a neuron-level attribution metric that decomposes log-probability increase across neurons. We present a time-resolved comparison of knowledge acquisition dynamics in MoE and dense architectures, tracking checkpoints over 1.2M training steps (~ 5.0T tokens) and 600K training steps (~ 2.5T tokens), respectively. Our experiments uncover three patterns: (1) Low-entropy backbone. The top approximately 1% of MoE neurons capture over 45% of positive updates, forming a high-utility core, which is absent in the dense baseline. (2) Early consolidation. The MoE model locks into a stable importance profile within < 100K steps, whereas the dense model remains volatile throughout training. (3) Functional robustness. Masking the ten most important MoE attention heads reduces relational HIT@10 by < 10%, compared with > 50% for the dense model, showing that sparsity fosters distributed -- rather than brittle -- knowledge storage. These patterns collectively demonstrate that sparsity fosters an intrinsically stable and distributed computational backbone from early in training, helping bridge the gap between sparse architectures and training-time interpretability.