Survival of the Fittest Representation: A Case Study with Modular Addition

TL;DR

This paper investigates how neural networks select among multiple solutions during training by modeling their competitive dynamics using ecological principles, specifically in the context of modular addition tasks.

Contribution

It introduces a novel ecological analogy to neural network training, demonstrating how representations compete and survive based on their initial signal strength and complexity.

Findings

High initial signal frequencies are more likely to survive.

Increasing embedding dimension results in more surviving frequencies.

Representation dynamics can be modeled by Lotka-Volterra equations.

Abstract

When a neural network can learn multiple distinct algorithms to solve a task, how does it "choose" between them during training? To approach this question, we take inspiration from ecology: when multiple species coexist, they eventually reach an equilibrium where some survive while others die out. Analogously, we suggest that a neural network at initialization contains many solutions (representations and algorithms), which compete with each other under pressure from resource constraints, with the "fittest" ultimately prevailing. To investigate this Survival of the Fittest hypothesis, we conduct a case study on neural networks performing modular addition, and find that these networks' multiple circular representations at different Fourier frequencies undergo such competitive dynamics, with only a few circles surviving at the end. We find that the frequencies with high initial signals and gradients, the "fittest," are more likely to survive. By increasing the embedding dimension, we also observe more surviving frequencies. Inspired by the Lotka-Volterra equations describing the dynamics between species, we find that the dynamics of the circles can be nicely characterized by a set of linear differential equations. Our results with modular addition show that it is possible to decompose complicated representations into simpler components, along with their basic interactions, to offer insight on the training dynamics of representations.