Unveiling Memorization-Generalization Coexistence: A Case Study on Arithmetic Tasks with Label Noise

TL;DR

This study explores how over-parameterized neural networks can memorize noisy labels while still generalizing well, revealing internal structures and proposing methods to extract generalizable knowledge.

Contribution

The paper uncovers the coexistence mechanisms of memorization and generalization in neural networks and introduces a frequency-based method to extract generalization structures.

Findings

Larger models tend to generalize better with proper configurations.

Noisy labels are memorized faster than clean data.

Frequency-based methods can extract generalization structures even with high label noise.

Abstract

Highly over-parameterized models can simultaneously memorize noisy labels and generalize well, yet how these behaviors coexist remains poorly understood. In this work, we investigate the underlying mechanisms of this coexistence using modular arithmetic tasks under heavy label noise. Through extensive experiments on two-layer neural networks, we find that larger models tend to generalize better under appropriate optimization and model configurations, while noisy labels are memorized faster than clean data. Over-parameterized models internally form a generalization structure, but its expression in the output is suppressed by the need to fit noisy labels. Remarkably, even with 80\% label noise, near-perfect test accuracy can be achieved by extracting this internal structure using frequency-based methods. We further propose a task-agnostic method to partition networks into generalization and memorization components. Although this subnetwork improves generalization, it is limited compared with frequency-based extraction, indicating that the generalization structure is distributed across neurons and motivating the development of new tools to retrieve generalizable knowledge from over-parameterized networks.